| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-09 | |||
| 20:37:31 | mriedem | but not backportable | |
| 20:37:38 | Sundar | which any driver/plugin/module can invoke | |
| 20:37:43 | efried | Sundar: Offhand I don't see a problem with that. Is it ever going to be the case that you need to run one but not both (i.e. a driver but not its corresponding plugin, or vice versa) on a given system? | |
| 20:37:53 | sean-k-mooney | efried: Sundar provided there is a well defiend versioned interface its ok but Sundar i dont thin os-acc should be able to alter the hypervior context | |
| 20:38:18 | efried | Sundar: Let's move to #openstack-cyborg so we're not cross-talking with the others. | |
| 20:38:21 | sean-k-mooney | e.g. just like os-vif os-acc should not be able to modify the libvirt xml | |
| 20:38:27 | dansmith | mriedem: yeah, I was just looking through compute manager wondering why the fsck it was in there too | |
| 20:38:46 | dansmith | mriedem: does that not work for LB for some reason? | |
| 20:39:20 | mriedem | does what not work? | |
| 20:39:22 | openstackgerrit | Merged openstack/nova stable/queens: Reload oslo_context after calling monkey_patch() https://review.openstack.org/589249 | |
| 20:39:22 | mriedem | https://review.openstack.org/#/c/558001/ ? | |
| 20:39:26 | dansmith | yeah | |
| 20:39:29 | openstackgerrit | Merged openstack/nova stable/queens: Fix message for unexpected external event https://review.openstack.org/589505 | |
| 20:39:34 | mriedem | prometheanfire is failing in pike | |
| 20:39:36 | mriedem | https://review.openstack.org/#/c/558001/ is rocky | |
| 20:39:41 | openstackgerrit | Merged openstack/nova master: Trivial fix on migration doc https://review.openstack.org/589028 | |
| 20:39:45 | mriedem | b/c we backported sahid's patch | |
| 20:39:48 | openstackgerrit | Merged openstack/nova master: Add a prelude release note for the 18.0.0 Rocky GA https://review.openstack.org/589303 | |
| 20:39:49 | Sundar | sean-k-mooney: os-acc may provide device-specific XML snippets, for example, which libvirt driver would compose into a domain XML. | |
| 20:40:02 | Sundar | efried: Sure, joined #openstack-cyborg | |
| 20:40:03 | dansmith | mriedem: no, I realize that | |
| 20:40:24 | sean-k-mooney | Sundar: i really hope not or that code should live in the nova tree | |
| 20:40:33 | efried | sean-k-mooney: Can you join us in -cyborg? | |
| 20:40:40 | sean-k-mooney | efried: sure | |
| 20:40:41 | dansmith | mriedem: what I'm saying is, because the event gets triggered from pre-migration, the wait should really be up a level in compute manager, which you added in rocky | |
| 20:41:04 | dansmith | mriedem: and I'm asking if there's some reason why the wait in compute manager can't work with LB | |
| 20:41:32 | dansmith | mriedem: so we like just rip sahid's stuff out of everywhere and make sure that you're including events in the compute manager wait | |
| 20:42:18 | sean-k-mooney | efried: #openstack-cyborg? | |
| 20:42:27 | efried | yes | |
| 20:44:28 | mriedem | dansmith: it should work for LB as far as i know | |
| 20:44:41 | dansmith | seems like it | |
| 20:44:52 | mriedem | the only backend i know that won't work, | |
| 20:44:54 | mriedem | is ODL | |
| 20:45:10 | dansmith | in fact | |
| 20:45:10 | mriedem | because that doesn't send events on vif plug/unplug, only port host binding changes | |
| 20:45:13 | sean-k-mooney | mriedem: lb polls for new interfaces and can miss the addtion and removal of interfaces in some cases | |
| 20:45:25 | dansmith | mriedem: you're not only waiting for ovs interfaces there right? | |
| 20:45:29 | sean-k-mooney | so we can rely on lb to emit the event | |
| 20:45:31 | mriedem | dansmith: correct | |
| 20:45:46 | sean-k-mooney | or rathar the lb l2 agent | |
| 20:45:49 | dansmith | mriedem: so you should be eating them up there, and then his wait is definitely never going to get them right? | |
| 20:46:10 | dansmith | so in pike, I expect it races, | |
| 20:46:16 | dansmith | and in rocky it never ever works at all | |
| 20:47:03 | mriedem | yeah maybe | |
| 20:47:10 | mriedem | i could dig up my 2 node lb ci patch, | |
| 20:47:12 | dansmith | oh sweet baby jesus thank you for this day | |
| 20:47:17 | mriedem | and enable this waiter in nova on master, | |
| 20:47:40 | mriedem | and we'd have to probably turn the vif plugging timeout way down to actually see if we hit a timeout | |
| 20:47:53 | mriedem | otherwise i'd expect in the gate, live migration with a tiny cirros guest not doing anything transfers pretty fast | |
| 20:48:03 | dansmith | yeah | |
| 20:48:17 | mriedem | heh, and i was just going to start mowing and packing | |
| 20:48:37 | dansmith | melwitt: so, honestly, reverting sahid's thing for rocky needs to be high prio I think | |
| 20:48:47 | dansmith | melwitt: live migration with LB is completely broken I expect | |
| 20:48:55 | mriedem | i'll update that ci patch | |
| 20:49:02 | melwitt | ok, so is this a RC1 thing or a RC2 thing? | |
| 20:49:06 | dansmith | melwitt: and it probably needs to be reverted out of the older releases too | |
| 20:49:13 | dansmith | melwitt: your call but RCsomething, IMHO | |
| 20:49:36 | prometheanfire | back | |
| 20:49:41 | prometheanfire | but leaving soonish | |
| 20:49:42 | dansmith | I would think we could do something like what mriedem did in rocky for those releases | |
| 20:50:05 | melwitt | ok, definitely RC2. trying to do it by the end of today will be hard unless it gets approved, stat | |
| 20:50:20 | melwitt | *definitely RC2, at least | |
| 20:50:54 | dansmith | well, maybe not since he needed a signal from the remote machine that it was going to do the wait ... | |
| 20:51:02 | prometheanfire | mriedem: I'll try to get a bug reported if you think that's the next step | |
| 20:51:04 | dansmith | although the event should still trigger | |
| 20:51:07 | mriedem | prometheanfire: yes please | |
| 20:51:09 | dansmith | prometheanfire: yes | |
| 20:51:10 | mriedem | we'll track that for rc2 | |
| 20:51:27 | prometheanfire | rc2? | |
| 20:51:29 | mriedem | was going to try and see how long a guest transfer takes in the gate | |
| 20:51:37 | mriedem | prometheanfire: today is release candidate 1 day | |
| 20:51:43 | prometheanfire | I thought this didn't hit rocky | |
| 20:51:50 | prometheanfire | but I'll leave that to you | |
| 20:51:53 | mriedem | prometheanfire: new wrinkle | |
| 20:51:57 | prometheanfire | oh, nice | |
| 20:52:00 | dansmith | worse wrinkle | |
| 20:52:02 | mriedem | (1) probably race fail on pike | |
| 20:52:05 | prometheanfire | happy to help :P | |
| 20:52:06 | mriedem | (2) totes broken on master | |
| 20:52:13 | prometheanfire | even better | |
| 20:53:40 | dansmith | prometheanfire: hook me up with a bug number and I'll propose the revert | |
| 20:53:41 | dansmith | and I can comment on the bug with all the deets | |
| 20:53:50 | dansmith | since mriedem will be busy with the ci patch and packing for da nang | |
| 20:53:59 | mriedem | isn't da nang vietnam? | |
| 20:54:02 | dansmith | yes | |
| 20:54:18 | openstackgerrit | Matt Riedemann proposed openstack/nova master: placement: ignore policy scope check failures if not enforcing scope https://review.openstack.org/590445 | |
| 20:54:33 | prometheanfire | I'm hoping to get the user to report, but I will if he went home | |
| 20:55:16 | dansmith | the revert is complete conflict | |
| 20:55:19 | dansmith | wonderful. | |
| 20:56:33 | dansmith | the other benefit of doing this in the manager is that we don't need the silly artificial speed limit | |
| 20:56:48 | dansmith | although we probably need sahid and libvirt people to confirm that there's not something we're missing here | |
| 20:57:32 | dansmith | because I asked him specifically about doing this early on in his patch and he said it wasn't possible, but I believed that qemu/libvirt on the dest machine were responsible for the plugging at the time | |
| 20:57:34 | dansmith | and maybe he did too | |
| 21:00:46 | mriedem | meanwhile, our granite seller is being an ass and i have to get back to our plumber | |
| 21:01:37 | dansmith | #firstworldrichpersonproblems | |
| 21:06:47 | mriedem | ok so looking at a job, we register waiting for events starting here http://logs.openstack.org/98/587498/1/check/nova-live-migration/5ff805a/logs/screen-n-cpu.txt.gz#_Jul_31_17_18_19_562999 | |
| 21:06:52 | mriedem | Jul 31 17:18:19.562999 ubuntu-xenial-inap-mtl01-0001077052 nova-compute[2320]: DEBUG nova.compute.manager [None req-942f438c-3cbb-4ce7-8afb-ecd250c98f75 tempest-LiveAutoBlockMigrationV225Test-1515676049 tempest-LiveAutoBlockMigrationV225Test-1515676049] [instance: 7f68c430-f565-433a-8f87-27b9a00d29a0] Preparing to wait for external event network-vif-plugged-6b030652-5fe6-471a-b7ec-0b70e95159a4 {{(pid=2320) prepare_for_instanc | |
| 21:06:52 | mriedem | ent /opt/stack/new/nova/nova/compute/manager.py:328}} | |
| 21:07:51 | mriedem | pre_live_migration takes about 7 seconds | |
| 21:07:51 | mriedem | Jul 31 17:18:26.178239 ubuntu-xenial-inap-mtl01-0001077052 nova-compute[2320]: INFO nova.compute.manager [None req-994d9893-545f-47b9-b93e-1e21cb439db7 tempest-LiveMigrationTest-233418614 tempest-LiveMigrationTest-233418614] [instance: 7f68c430-f565-433a-8f87-27b9a00d29a0] Took 6.61 seconds for pre_live_migration on destination host ubuntu-xenial-inap-mtl01-0001077053. | |