Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-08-21
18:04:29 openstackgerrit Dan Smith proposed openstack/nova stable/queens: Wait for network-vif-plugged before starting live migration https://review.openstack.org/591735
18:04:30 openstackgerrit Dan Smith proposed openstack/nova stable/queens: DNM: Debug patch to test live migration waiting https://review.openstack.org/591775
18:05:23 melwitt I guess I don't understand what the lifecycle event gives us if we already know success or failure
18:05:55 melwitt without listening for it
18:07:06 mriedem in rocky we started listening on 2 new events,
18:07:12 mriedem one is post-copy and one is migration completed
18:07:31 mriedem the idea is that as soon as we switch we activate the port bindings on the dest host for minimal downtime
18:07:50 mriedem the problem is we get the latter event even if live migration fails
18:08:01 mriedem and we're not doing any conditional logic in that one to see if the job failed or not
18:09:26 melwitt I think I understand that part, it's just when you said "we'll activate port bindings on success like we always did before the change" it makes me not understand what gain the lifecycle event was supposed to give, if we already know success or failure without it
18:11:55 mriedem because if we can activate the network on the dest at the point the guest is paused to complete the transfer, it makes the network downtime window shorter
18:14:16 melwitt and without the lifecycle event, we activate the network later on after the guest is paused
18:14:19 mriedem we might still need my other fix for rollback, i'm not sure; what sean is hitting isn't a failure after post-copy
18:14:32 mriedem we activate the network after the guest transfer is complete and resumed on the dest
18:16:49 melwitt ok, got it
18:30:58 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: Don't react to VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED events https://review.openstack.org/594508
18:33:11 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: Don't react to VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED events https://review.openstack.org/594508
19:36:26 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: check job status for VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED event https://review.openstack.org/594527
19:37:36 mriedem melwitt: dansmith: alright fyi ^ would need sean-k-mooney to test out the 2nd more complicated fix since he has the env that recreates the bug
19:37:43 mriedem and i need to run to an appt
19:39:36 mriedem also threw that stuff in the rc todo etherpad
19:43:27 melwitt ack
19:54:56 openstackgerrit Eric Fried proposed openstack/nova master: [placement] Add /reshaper handler for POST https://review.openstack.org/576927
19:54:57 openstackgerrit Eric Fried proposed openstack/nova master: reshaper: Look up provider if not in inventories https://review.openstack.org/585033
19:54:58 openstackgerrit Eric Fried proposed openstack/nova master: Make get_allocations_for_resource_provider raise https://review.openstack.org/584598
19:54:59 openstackgerrit Eric Fried proposed openstack/nova master: Report client: Real get_allocs_for_consumer https://review.openstack.org/584599
19:55:00 openstackgerrit Eric Fried proposed openstack/nova master: Report client: get_allocations_for_provider_tree https://review.openstack.org/584648
19:55:01 openstackgerrit Eric Fried proposed openstack/nova master: Report client: _reshape helper, placement min bump https://review.openstack.org/585034
19:55:02 openstackgerrit Eric Fried proposed openstack/nova master: Report client: update_from_provider_tree w/reshape https://review.openstack.org/585049
19:55:03 openstackgerrit Eric Fried proposed openstack/nova master: Compute: Handle reshaped provider trees https://review.openstack.org/576236
20:14:31 dansmith so I
20:14:48 dansmith am pretty sure my batching breaks tssurya's down cell in its current form
20:15:02 dansmith more specifically, it'll cause scatter gather to never notice failues
20:24:16 melwitt dansmith: it's specific to the batching? it looks like pre-batch the code still removes the "did not respond" and "raised exception" results
20:24:29 dansmith still removes?
20:25:21 melwitt dansmith: this part looks like it's removing "down cells" from results? https://review.openstack.org/#/c/592698/5/nova/compute/multi_cell_list.py@311
20:26:00 dansmith right but with the batching we never hit that because we don't start executing the queries until the heapq
20:27:10 melwitt oh, I see
20:27:45 melwitt hmm
20:29:21 dansmith I'll just have to get a little more into the middle of that process and we just won't get the standard handlers from scatter gather
21:13:09 mriedem huh, you don't see instance.save() messaging timeouts in the gate very often http://logs.openstack.org/98/591898/3/check/nova-next/2d5e60c/logs/screen-n-cpu.txt.gz#_Aug_21_17_10_17_732263
21:14:24 mriedem guessing this isn't good http://logs.openstack.org/98/591898/3/check/nova-next/2d5e60c/logs/screen-n-cpu.txt.gz#_Aug_21_17_10_17_426420
21:16:54 mriedem seen here too http://logs.openstack.org/85/567785/7/check/nova-tox-functional-py35/d5a8036/job-output.txt#_2018-08-17_09_06_20_241427
21:36:54 mriedem melwitt: what do you think is missing from placement for shared storage providers support? as far as i know, it's the nova stuff that's lacking as we identified ~2 weeks ago
21:37:05 mriedem for sure move operations are not ready for shared storage providers
21:37:18 mriedem the placement side of shared storage is pretty simple though, and has been done for a long time
21:40:28 openstackgerrit Dmitry Sutyagin proposed openstack/nova-specs master: Allow disabling KSM / mem-merge via extra spec https://review.openstack.org/593197
21:41:23 melwitt mriedem: it's not that I think anything is missing, I'm pragmatically thinking of the integration work and if there is something unforeseen we need to fix. I expect bugs to shake out when we integrate something for the first time. I think not having bugs shake out will be the rarer case
21:41:38 dansmith mriedem: I think we were saying the same about aggregates being done in placement before we added the placement filter stuff and realized we needed tweaks
21:41:56 dansmith and surely thought what was being done in placement for NRPs was going to be usable by nova until we thought about it
21:46:01 mriedem dansmith: the member_of thing with aggs right?
21:46:17 dansmith we needed member_of with and and or
21:46:33 dansmith and I meant prefilter above
21:46:37 dansmith or request filter
21:46:40 dansmith or whatever that guy called it
21:47:29 dansmith the granular request stuff is another similar example, where we try to use what we think is just clean resource requests for actual nova stuff and realize we need this giant complex syntax instead
21:47:54 dansmith all that could be developed in two separate rooms for sure, just like multiattach or multiple host bindings
21:48:05 dansmith and hot damn, in a few years we'll be golden
21:49:31 melwitt yeah. in case it wasn't clear, the thing I care about is delivering stuff that operators and users need, that we know they need, and I don't see how becoming two separate groups helps that
21:49:31 dansmith mriedem: by the way, I wonder if the object action rpc methods to conductor ought to be long_rpc_timeouts
21:49:56 dansmith mriedem: they really should never hang for a long time, but just piling up more because we time out, run the periodic again and generate more traffic is probably worse
21:50:18 dansmith mriedem: re: your save timeout
21:52:00 mriedem dansmith: it seems something weird happened with the servicegroup in that failure
21:52:23 dansmith yeah, not related to your actual thing, but just thinking of what that reminds me of, which is conductor is overwhelmed
21:58:56 mriedem dansmith: also, i was talking with efried about kevin's exclusive trait thing, and guess what https://review.openstack.org/#/c/593475/
21:58:59 mriedem it's already been proposed :)
22:00:21 dansmith well, having not read it and skimmed the -1, I'm assuming it's for the same reason I think it's a non-starter
22:03:41 mriedem encoding metadata in a trait name
22:05:56 dansmith and it's only one special prefix,
22:05:57 dansmith which means one class
22:06:46 mriedem CUSTOM_INTEL_FOR_SERIOUS_WORKLOADS
22:06:52 mriedem i can see it now
22:08:39 efried { 'name': 'CUSTOM_INTEL_FOR_SERIOUS_WORKLOADS,
22:08:39 efried POST /traits/CUSTOM_INTEL_FOR_SERIOUS_WORKLOADS
22:08:40 efried }
22:08:40 efried ...
22:08:40 efried 'allowed_project_ids': [...],
22:08:40 efried 'allowed_user_ids': [...],
22:08:40 efried 'required': true,
22:08:57 mriedem queue jay vomit
22:09:01 mriedem *cue
22:09:07 efried we could do the same thing with aggregates
22:09:27 mriedem dansmith: re granular, we could still do POST queries....
22:09:29 mriedem just saying
22:10:49 mriedem granular request group syntax is likely something that could benefit from some kind of flavor extra specs validate api
22:13:26 efried no argument there
23:05:25 openstackgerrit Eric Fried proposed openstack/nova master: [placement] Regex consts for placement schema https://review.openstack.org/591863
23:05:26 openstackgerrit Eric Fried proposed openstack/nova master: [placement] Add /reshaper handler for POST https://review.openstack.org/576927
23:05:27 openstackgerrit Eric Fried proposed openstack/nova master: reshaper: Look up provider if not in inventories https://review.openstack.org/585033
23:05:28 openstackgerrit Eric Fried proposed openstack/nova master: Make get_allocations_for_resource_provider raise https://review.openstack.org/584598
23:05:29 openstackgerrit Eric Fried proposed openstack/nova master: Report client: Real get_allocs_for_consumer https://review.openstack.org/584599
23:05:30 openstackgerrit Eric Fried proposed openstack/nova master: Report client: get_allocations_for_provider_tree https://review.openstack.org/584648
23:05:31 openstackgerrit Eric Fried proposed openstack/nova master: Report client: _reshape helper, placement min bump https://review.openstack.org/585034
23:05:32 openstackgerrit Eric Fried proposed openstack/nova master: Report client: update_from_provider_tree w/reshape https://review.openstack.org/585049
23:05:33 openstackgerrit Eric Fried proposed openstack/nova master: Compute: Handle reshaped provider trees https://review.openstack.org/576236
23:42:17 openstackgerrit Brin Zhang proposed openstack/nova-specs master: Resource retrieving: add change-before filter https://review.openstack.org/591976
23:48:21 openstackgerrit Dan Smith proposed openstack/nova master: Batch results per cell when doing cross-cell listing https://review.openstack.org/592698
23:48:22 openstackgerrit Dan Smith proposed openstack/nova master: List instances from all cells explicitly https://review.openstack.org/593717
23:48:23 openstackgerrit Dan Smith proposed openstack/nova master: Make instance_list perform per-cell batching https://review.openstack.org/593131
23:48:24 openstackgerrit Dan Smith proposed openstack/nova master: Record cell success/failure/timeout in CrossCellLister https://review.openstack.org/594265

Earlier   Later