Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-12-02
17:39:13 efried Sundar: Not *any* failures.
17:39:18 Sundar If we're all ok with that, I'm good. But there are a few nuances to consider. First, the function to get the resolved ARQs is now defined in https://review.opendev.org/631245, while it needs to be used in the previous patch https://review.opendev.org/631244.
17:39:24 sean-k-mooney gibi: the use of multiple port bindings for live migration and the rx/tx queue size options were both intoduced in rocky so it should be backportable to all affected version if i do it right
17:40:00 efried Sundar: As dansmith suggested (oh, maybe it was in IRC, not in the patch) you should still fail and clean up the binding if you get e.g. a 404.
17:40:10 efried meaning the instance was deleted by the time you went to emit the notification.
17:40:55 sean-k-mooney gibi: well assuming the call order works out ill follow up with this later
17:41:25 efried So, to be clear, the 422 is a special case, with a prominent NOTE and a nice debug log stating that the binding completed before the instance landed on the host, so it's acceptable that the notification is dropped.
17:42:06 efried Sundar: refactoring method X from patch Z into patch Y because that's where it's now going to be used, yeah, that's SOP.
17:42:19 Sundar efried: There could be a bunch of different failure modes: cannot locate n-api service, located but it timed out?, etc. We need to distinguish that category from 'called n-api and it returned an error'. In the latter case, is there granularity of error codes for why it failed?
17:42:50 efried Sundar: Yes, the 422 is a special case; any other error is an error.
17:43:41 Sundar Need to ceck if 422 is used for any other error case.
17:43:44 Sundar *check
17:47:10 Sundar Re. the patch series, I think we need to squash https://review.opendev.org/631245 into the previous patch https://review.opendev.org/631244. Because once the get_resolved_arqs() moves to previous patch, the only things left behind are the virt driver changes. It would arguably be easier to understand those changes together with the changes to get
17:47:11 Sundar the arqs in compute/manager.py.
17:50:49 mriedem nice https://docs.openstack.org/nova/latest/admin/#overview - "nova-api Receives XML requests"
17:56:17 melwitt stephenfin: I noticed a new recent comment on this bug https://bugs.launchpad.net/nova/+bug/1289064 saying, maybe it can be closed bc numa aware lm was merged? seems so to me but wanted to check with you
17:56:17 openstack Launchpad bug 1289064 in OpenStack Compute (nova) "live migration of instance should claim resources on target compute node" [Medium,In progress] - Assigned to Artom Lifshitz (notartom)
17:56:35 Sundar Only one use of 422 error code in create: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/server_external_events.py . So, we are probably good.
17:56:58 efried yup (I checked that earlier)
17:57:06 artom melwitt, stephenfin, oh, well - not entirely - only NUMA LM does claims
17:57:19 artom Then again, only NUMA LM has non-placement resources, so maybe yes?
17:57:26 artom SRIOV is handled as well, albeit separately
17:58:26 melwitt er, or artom, sorry ^ (I pinged stephen bc his patches about rejecting lm requests that have numa topology were merged as Related-Bug)
17:59:28 melwitt hm, ok. I don't really understand what that means so I dunno either
17:59:29 sean-k-mooney melwitt: ya so stephens patch just reject live migration if the guest has a numa toplogy at the api
18:00:07 sean-k-mooney sriov live migration directly allocates and claims the devices during live migration
18:00:22 sean-k-mooney numa live migration uses a move claim to claim resouces
18:00:58 sean-k-mooney (cpus and mempages/hugepages)
18:02:03 artom sean-k-mooney, so, could we say that all non-placement resources are not accounted correctly during live migration?
18:02:15 sean-k-mooney no
18:02:33 sean-k-mooney i dont think this is an issue on master
18:02:47 artom Err, *are accounted
18:02:50 eandersson Looks good gibi
18:03:18 eandersson btw mriedem did you see the minor bug I mentioned over the weekend
18:03:47 sean-k-mooney artom: right non-placement rescoues (cpu,mempages,sriov devices) should be accounted for in train+ and we blocked numa/sriov migration before that retoactivly
18:04:06 sean-k-mooney artom: melwitt so we might just be able to close https://bugs.launchpad.net/nova/+bug/1289064
18:04:06 openstack Launchpad bug 1289064 in OpenStack Compute (nova) "live migration of instance should claim resources on target compute node" [Medium,In progress] - Assigned to Artom Lifshitz (notartom)
18:04:28 eandersson https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2075 <-- eventlet.Timeout (based on BaseException, not Exception) may be raised here causing an UnboundError.
18:05:32 mriedem eandersson: nope
18:06:03 melwitt sean-k-mooney: if there's not anything more to do there, I'd think it would be nice to close it
18:07:10 sean-k-mooney well there are still cases where live migration is not supported but on master i think we account for all resouces correctly in the cases where live migration is supported
18:07:17 mriedem eandersson: open a bug
18:07:49 artom sean-k-mooney, melwitt, so.... fix release 'Train'?
18:08:17 dansmith efried: Sundar: I was on a call during your discussion, but...we're all good now?
18:08:20 sean-k-mooney artom: ya i think so
18:09:06 melwitt yeah, I can't remember how/if there's a way to mark it as fixed in the past. but I don't think that matters too much
18:09:25 artom Done.
18:09:29 sean-k-mooney you can add the sepcific release series to the bug
18:09:41 sean-k-mooney i left a comment in the bug too
18:10:12 eandersson mriedem https://bugs.launchpad.net/nova/+bug/1854848
18:10:12 openstack Launchpad bug 1854848 in OpenStack Compute (nova) "build_and_run_instance can error out with an UnboundException" [Undecided,New]
18:10:28 Sundar efried: I think so, if you and all are ok with whatever I said above -- squishing the patches, etc. ignoring 422 alone seems fine. if n-api is unreachable or times out, that will still be an error that fails the binding.
18:10:30 sean-k-mooney artom: added train
18:10:39 eandersson It's a minor issue, but caused VMs to get stuck in BUILD... fyi was caused by selinux not being configured properly.
18:11:04 sean-k-mooney artom: melwitt there is a "Target to series" button you can use
18:11:17 sean-k-mooney you need to be in the nova bug team on LP to see/use it
18:12:09 melwitt yeah, I have that, but I don't think that will stop it from saying "released in <U version number>" whenever it's released next
18:12:58 melwitt it'll still count it as landing "today" first and then "backported" to train. I don't think it matters, just that's what the automation is gonna do when it comments later
18:13:56 sean-k-mooney ya proably
18:14:39 sean-k-mooney i dont think we have automation tieded to that out side of the completed specs thing but that uses blueprints not bugs
18:16:11 sean-k-mooney artom: if you are triaging nova bugs upstream you should join https://launchpad.net/~nova-bugs its an open team so you will be automatically approved
18:16:32 artom sean-k-mooney, can't say I'm triaging dilligently
18:17:25 sean-k-mooney well you know where to find it if you change your mind
18:17:52 melwitt sean-k-mooney: I mean the thing that will make a comment on the bug that says "this bug was fixed in version x.x.x.x" that will be technically wrong because it's gonna say a U version. again, I don't think it matters, just that's what it's going to say on the comment
18:18:12 sean-k-mooney 00.
18:18:14 sean-k-mooney 00.1
18:18:20 sean-k-mooney droped phone
18:18:42 sean-k-mooney am isnt that done by the release tooling
18:18:48 sean-k-mooney not launchpad
18:19:15 melwitt it is yeah
18:19:28 sean-k-mooney but ya i guess in this case its not going to be set since artoms change did not have closes bug
18:20:00 sean-k-mooney anyway it should be fine
18:20:17 melwitt I don't think it matters, I think it just does a sweep through things that are Fix Released and makes a comment with the fixed in version
18:27:47 efried dansmith: Yes, thanks.
18:30:16 dansmith efried: so there'll need to be a change to cyborg for the 422 thing presumably
18:30:24 efried yes
18:31:02 dansmith efried: Sundar: is there a testing patch somewhere in the series that wires this all up with a fake cyborg driver?
18:31:22 dansmith I assume that would be the one that needs to depends-on that cyborg change
18:31:26 efried the CI job is working last I checked
18:31:58 sean-k-mooney there is a patch to add a tempest job againd nova nad i think the job is now running on all cyborg patches
18:31:59 dansmith there are a lot of patches that have various confused histories
18:32:05 efried dansmith: https://review.opendev.org/#/c/670999/20
18:32:10 efried passing atm
18:32:11 dansmith ack just found that
18:32:34 sean-k-mooney same was going to link it
18:33:15 efried there was a time when I looked and it wasn't doing anything, but I thought the last time I checked it was actually running through the spawn flow with a real (pseudo) device
18:33:29 sean-k-mooney it is doing two things
18:33:31 efried still very basic though
18:33:35 efried green path only
18:33:51 sean-k-mooney its cratein a device profile and then a super minima test of booting a vm
18:33:55 sean-k-mooney althogh that one is not correct
18:34:09 sean-k-mooney its booting a vm but not waiting for ti to go active before deleting it
18:34:35 sean-k-mooney at least the last time i looked but everythin is in place to add proper lifecycle tests
18:34:37 dansmith seems like a lot of vm_state=error in the compute logs for that, I'll have to track it down closer to see if it's related
18:35:17 sean-k-mooney https://opendev.org/openstack/cyborg-tempest-plugin/src/branch/master/cyborg_tempest_plugin/tests/scenario/test_accelerator_basic_ops.py#L45-L60
18:35:46 dansmith does that even wait for it to become running?
18:36:03 sean-k-mooney no which i mentioned before
18:36:22 sean-k-mooney to the cyborg folks
18:36:32 sean-k-mooney create server does https://opendev.org/openstack/cyborg-tempest-plugin/src/branch/master/cyborg_tempest_plugin/tests/scenario/manager.py#L245-L257
18:36:33 dansmith heh okay

Earlier   Later