Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-09-06
15:26:49 esberglu That line should allow tee to be run as root with any parameters
15:27:24 efried esberglu Sorry, I mean are we using RootwrapDaemonHelper or RootwrapProcessHelper?
15:30:25 openstackgerrit Matt Riedemann proposed openstack/nova master: Add release note for force live migration allocations https://review.openstack.org/501314
15:30:26 mriedem bauzas: here you go ^
15:30:59 bauzas mriedem: roger. just in a meeting now
15:31:21 openstackgerrit Matt Riedemann proposed openstack/nova master: Add release note for force live migration allocations https://review.openstack.org/501314
15:31:33 efried esberglu Can you try setting `use_rootwrap_daemon` to True in the compute conf and see if the problem magically disappears?
15:31:57 efried esberglu That's in the [DEFAULT] section
15:32:01 esberglu efried: Yeah that isn't set currently. I can give it a try
15:32:23 efried esberglu Though if non-daemon is busted, that seems like it oughtta be a bug to nova.
15:34:05 openstackgerrit Merged openstack/nova master: Trim the fat from InstanceInfo https://review.openstack.org/471146
15:34:31 sdague efried: yeh, though the real fix would be to get rid of it entirely for privsep
15:34:39 sdague be aware, that mikal's patch series is doing that
15:34:44 efried sdague "it" which?
15:34:58 efried The Process version?
15:35:13 sdague https://review.openstack.org/#/c/489438
15:35:20 sdague that's 5 patches in
15:35:44 sdague and it will probably take some time to land, but rootwrap for tee is removed at that point
15:36:21 sdague efried: the tee call at all
15:37:18 efried sdague Mm. And the resolution is to decorate my function with this privsep gizmo and then do a regular ol `with open(): write()` ?
15:37:27 sdague yep
15:37:41 sdague but his first patch has to land before that will work
15:38:36 efried esberglu ^^ FYI. Need to add that to the watchlist so we can react when it lands.
15:39:54 esberglu efried: Will do
15:40:27 efried esberglu It would seem we could transition as soon as https://review.openstack.org/#/c/459166/ is in.
15:40:51 efried But in the meantime, <shrug> let's see if using the rootwrap daemon helps.
15:44:06 stephenfin jaypipes: https://review.openstack.org/#/c/498817/ and friends have their gate +1 now, if you were idle enough :)
15:56:19 dansmith mriedem: yeah I had marked that as -W ahead of me writing the release note, but then I wiped it when I pushed a little fix
15:57:00 mriedem i don't know how the tests are even passing
15:57:12 dansmith mriedem: because you missed that there is a patch before this?
15:57:41 mriedem yes
16:00:14 esberglu efried: ^ 459166 just got W+1
16:00:59 esberglu efried: Oh wait nvm looking at the wrong thing
16:01:25 efried esberglu Ya know, we could patch that guy in and make our own patch that uses it...
16:05:33 openstackgerrit Merged openstack/nova master: Remove unneeded attributes from context https://review.openstack.org/498615
16:06:14 openstackgerrit Merged openstack/nova master: Pass config object to oslo_reports https://review.openstack.org/485575
16:07:16 openstackgerrit Merged openstack/nova master: Enhance support matrix document https://review.openstack.org/482020
16:08:17 openstackgerrit Merged openstack/nova master: trivial: Remove some single use function from utils https://review.openstack.org/491513
16:18:51 openstackgerrit Stephen Finucane proposed openstack/nova master: doc: Split flavors docs into admin and user guides https://review.openstack.org/501342
16:20:23 dtantsur dansmith: thanks, I'll check it tomorrow - day full of errands today :(
16:20:32 openstackgerrit Dan Smith proposed openstack/nova master: Add nova-manage db command for ironic flavor migrations https://review.openstack.org/501025
16:20:33 openstackgerrit Dan Smith proposed openstack/nova master: Split out the core of the ironic flavor migration https://review.openstack.org/501024
16:20:33 openstackgerrit Dan Smith proposed openstack/nova master: Add ComputeNodeList.get_by_hypervisor_type() https://review.openstack.org/501343
16:20:42 dansmith dtantsur: ack, np
16:21:20 dtantsur cdent: any specific schedule plans for API SIG PTG?
16:21:31 cdent dansmith: could you give me some early feedback on whether you think this is the right approach: https://review.openstack.org/#/c/501051/ (it’s related to post /allocations)
16:21:35 dtantsur cdent: I'm figuring out conflicts with https://etherpad.openstack.org/p/InteropDenver2017PTG
16:22:00 dims jaypipes : when is the hurricane expected?
16:22:03 dansmith cdent: yeah I know I need to do that, sorry
16:22:12 dansmith dims: weekend I think
16:22:59 dansmith dims: meanwhile portland is on fire and florida refuses to share the water: the
16:23:00 openstackgerrit Stephen Finucane proposed openstack/nova master: Provide hints when nova-manage db sync fails to sync cell0 https://review.openstack.org/486660
16:23:03 cdent dtantsur: edleafe and I talked about that yesterday and we’re going to try to cook a schedule, but as there aren’t specific topics (nobody signed up for guided reviews) that are really on the agenda, it is mostly going to be workshopping guideline improvements and whatever comes up
16:23:03 dansmith http://www.oregonlive.com/wildfires/index.ssf/2017/09/eagle_creek_fire_rages_in_colu.html
16:23:12 cdent no worries dansmith
16:23:14 dims dansmith : y i heard :)
16:23:42 dtantsur cdent: surprisingly to me, the guided review idea has not got too much traction with other ironic team members..
16:23:49 dims dansmith : how far is it away from you?
16:24:21 dansmith dims: not far enough, but not currently a risk for us
16:24:40 dansmith dims: but it's crazy smoky outside and ash is falling on everything
16:25:15 dansmith walking outside is like getting a face full of camp fire
16:25:27 dims dansmith : dear lord
16:25:38 dansmith meanwhile, florida is all like "oh water, what are we going to do with all this water? did we mention we have a lot of water on the way?"
16:25:54 cdent that’s just rude
16:25:59 dansmith ikr
16:26:14 mnaser ahem im trying to troubleshoot (what i think is a nova bug).. any idea why instances can be seen in the list, but viewing them returns a 404?
16:26:32 mnaser ex: nova list shows them, nova show <uuid> says no server matching :X
16:26:41 dansmith mnaser: instance mapping missing?
16:26:45 dims dansmith : jaypipes : hopefully you will both make it to ptg
16:26:57 dansmith dims: thanks :)
16:27:24 dansmith mnaser: sounds like cell mappings are in place, so it can iterate and list, but no instance mapping for when it needs to go from instance uuid -> cell
16:27:35 dansmith mnaser: look in api database, instance_mappings table, see if there is one for the instance you're trying
16:27:38 mnaser i wonder why the mapping didnt go in then (customer says they recently failed builds)
16:28:29 dansmith mnaser: idk, should get an instance mapping once you're scheduled (or map you to cell0 if you fail) so we'd need to know what happened exactly to figure it out
16:28:32 mnaser /var/log/nova/nova-conductor.log:2017-09-06 10:42:20.512 829 WARNING nova.scheduler.utils [req-dd7ba87f-8539-4642-b238-3d1e4e4a686c 8873fec3d88b45c09af91ac942917650 eb60ffb2745a40e9b76223e5481e7c5e - default default] [instance: 90ecb4cc-8ea9-4419-9a3d-f67c9436df82] Setting instance to ERROR state.: TooManyInstances: Quota exceeded for ram: Requested 32768, but already used 716800 of 716800 ram
16:28:49 mnaser i wonder if it fails there meaning it doesn't end up getting scheduled?
16:28:58 dansmith mnaser: yeah so that should map you to cell0
16:29:10 mnaser time to check if it did
16:29:36 dansmith mnaser: well, it seems like it's not mapped, but after confirming that, see if it's in the cell0 database
16:29:56 mnaser MariaDB [nova_cell0]> select * from instances where id='90ecb4cc-8ea9-4419-9a3d-f67c9436df82'; => empty set
16:30:24 dansmith mnaser: uuid
16:30:29 mnaser oh duh
16:30:43 mnaser empty
16:30:52 dansmith check the other?
16:30:54 mnaser select * from instance_mappings where instance_uuid='90ecb4cc-8ea9-4419-9a3d-f67c9436df82'; => cell_id is null
16:31:41 mnaser i can see it in my default clel db
16:31:57 mnaser (one that was created upgrading to ocata)
16:32:07 mnaser and the only one other than cell0
16:32:18 dansmith okay that's weird
16:32:25 dansmith mnaser: can you file a bug with those details?
16:32:39 mnaser dansmith will do and ill do a bit more research too
16:32:42 mnaser maybe it happens when you hit quotas
16:32:52 dansmith mnaser: also, you should be able to do map_instances on your non-cell0 cell and fix that mapping I think
16:33:14 dansmith although you might have to nuke the instance mapping row first, which is also a bug for maintainabiity
16:33:18 dansmith mnaser: thanks
16:36:03 openstackgerrit Merged openstack/nova master: Glance download: only fsync files https://review.openstack.org/451094
16:36:37 openstackgerrit Merged openstack/nova master: spelling mistake https://review.openstack.org/501215
16:36:52 mnaser dansmith found the bug im pretty sure
16:37:00 mnaser https://github.com/openstack/nova/blob/a6c429bd7916211aa04d96caa54443ad92cb99ff/nova/conductor/manager.py#L992-L1006
16:37:03 mnaser no bury in there

Earlier   Later