| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-05 | |||
| 20:06:32 | mriedem | and the waiter code was waiting on the source | |
| 20:06:34 | mriedem | so a race and all that | |
| 20:06:53 | artom | I was thinking more narrowly in terms of the VIF type logic | |
| 20:07:08 | dansmith | artom: https://review.openstack.org/#/q/Ibf2b5eeafd962e93ae4ab6290015d58c33024132 | |
| 20:07:17 | artom | IOW, is "Currently we only use the mechanism of waiting for neutron events during live-migration for linux-bridge" false? | |
| 20:08:01 | dansmith | artom: it turned out that neutron was sending those events, this just excluded waiting for them because of other bugs.. papering over the real problem | |
| 20:08:13 | dansmith | it's been a while, but there was a LOT of analysis and discussion | |
| 20:08:30 | mriedem | artom: true that is false :) | |
| 20:08:35 | aspiers | mriedem: well one reason is that we (SUSE) will need to carry our own downstream backport to Rocky, so the more which lands in Stein the easier it will get for us ... but if that is in conflict with upstream goals then of course I would entirely understand | |
| 20:08:48 | mriedem | artom: https://docs.openstack.org/nova/latest/configuration/config.html#compute.live_migration_wait_for_vif_plug | |
| 20:08:59 | mriedem | ^ we wait for networking events during live migration based on config | |
| 20:09:01 | mriedem | since rocky | |
| 20:09:07 | mriedem | and part of that was backported | |
| 20:09:14 | mriedem | by mr smith | |
| 20:09:48 | mriedem | aspiers: heh, sure - i just don't think we want the sev capability showing up automatically in stein on the compute node resource provider when there is really nothing you can do with it | |
| 20:10:33 | dansmith | ah right, artom, the Note:: in that doc explains why the vif_type thing isn't usable and why you have to config your way out of this for certain backends | |
| 20:10:34 | dansmith | which sucsk | |
| 20:10:48 | aspiers | mriedem: yeah, totally understood ... we were really hoping to get all the patches ready in time for stein but it looks like we just weren't quite quick enough | |
| 20:11:33 | mriedem | aspiers: no problem, you got that capabilities patch landed which i'd abandoned essentially, which is pretty good for your first big contribution to nova | |
| 20:11:52 | artom | dansmith, I get all of it except for "without an accompanying port binding:host_id" | |
| 20:11:54 | mriedem | plus you had that acid trip of a venn diagram | |
| 20:12:06 | mriedem | artom: that's what i was saying earlier, | |
| 20:12:15 | dansmith | artom: ever get the feeling you stepped into a hornet's nest? | |
| 20:12:17 | mriedem | ODL will only send network-vif-plugged for port binding changes to the host | |
| 20:12:34 | mriedem | if the host doesn't change on the port binding, but we plug/unplug vifs or whatever, ODL doesn't send the event | |
| 20:12:36 | aspiers | mriedem: thanks :) feels good to be giving to upstream not just "taking" (i.e. asking for our own pet feature to be reviewed and merged) | |
| 20:12:52 | mriedem | artom: hence why it had to be reverted from that hard reboot change | |
| 20:13:00 | mriedem | this: https://review.openstack.org/#/c/553035/ | |
| 20:13:01 | artom | dansmith, yes | |
| 20:13:25 | aspiers | mriedem: haha, well my first version of that venn diagram was quite different, you can blame^Wthank efried for a lot of that second version | |
| 20:13:37 | mriedem | linuxbridge also had some problems with that in the neutron lb job because something about polling for changes and it would miss them and not send the event | |
| 20:13:42 | melwitt | I concur, thanks aspiers++ | |
| 20:13:48 | artom | mriedem, ah, so the note is just worded poorly to say "ODL will only send us vif-plugged if the host has changed" | |
| 20:13:51 | dansmith | mriedem: ah, yeah that's the thing I was expecting to find in there, forgot that had to be reverted too | |
| 20:13:53 | aspiers | thanks melwitt :) | |
| 20:14:04 | artom | ... which obviously it doesn't on hard reboot | |
| 20:14:05 | mriedem | artom: it is worded perfectly | |
| 20:14:38 | artom | Nuh uh, communication is only perfect if the thickest of idiots understand it. I didn't understand this. QED. | |
| 20:14:49 | mriedem | of course the port binding host *does* change during live migration, | |
| 20:15:03 | mriedem | but that happens after / during the guest transfer, not as a result of plug_vifs during pre_live_migration | |
| 20:15:06 | mriedem | which is what that config is used for | |
| 20:15:53 | mriedem | also, let me say, that i'm afraid sean-k-mooney might be dead or something because this entire conversation is the kind of thing he stays up until 3am for | |
| 20:16:01 | artom | Seriously. | |
| 20:16:23 | aspiers | LOL | |
| 20:16:25 | artom | RIP sean I guess. We'll do a minute of silence i Denver. | |
| 20:18:21 | artom | mriedem, wait, so why is that note even there? It's talking about not sending events when the host binding doesn't change. | |
| 20:18:30 | artom | That doesn't apply to live migration... | |
| 20:18:52 | jaypipes | mriedem: was I not supposed to +2 something? | |
| 20:20:47 | aspiers | jaypipes: hopefully +2 is OK, W+1 might not have been though :) | |
| 20:21:16 | artom | So, I think I should ignore the whole Neutron backend thing for now. It's out of scope, and I shouldn't introduce new bugs by ignoring it because it's currently ignored anyways. We can continue to wait for events in the same situations as currently, we just need to change the timing of the waits. | |
| 20:23:20 | melwitt | poor wording, that reminds me, I was supposed to update the consoleauth reno (or was it nova-status?) yet again to make more sense. I need to find IRC log from that convo with imacdonn | |
| 20:29:22 | imacdonn | melwitt: http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-02-20.log.html#t2019-02-20T18:46:52 | |
| 20:29:58 | edmondsw | mriedem yeah, I agree with you and efried. Should be fine, except that we do need to go update the OOT driver as well, so the heads up is definitely appreciated. | |
| 20:30:16 | melwitt | imacdonn: thanks, you're awesome | |
| 20:30:22 | imacdonn | :) | |
| 20:32:05 | mriedem | artom: ... | |
| 20:32:18 | mriedem | artom: it's saying where this config is used, might not make sense for all networking backends (ODL), | |
| 20:32:27 | mriedem | because the config is only for vif plugging, not port binding host changes, | |
| 20:32:34 | mriedem | ODL doesn't send events for the former, but does for the latter, | |
| 20:32:38 | mriedem | but the latter is not where that config is used | |
| 20:32:49 | mriedem | port binding happens during/after the guest transfer, | |
| 20:32:57 | mriedem | we're waiting for the event before starting the guest transfer | |
| 20:33:09 | mriedem | because if the dest plug fails or something, we don't want to start transferring the guest | |
| 20:33:57 | mriedem | dansmith: check my comments on takashin's fix here https://review.openstack.org/#/c/636271/ - should i just not care about a functional test for live migration or stack it up on top myself if i care? | |
| 20:34:06 | mriedem | i'm probably reaching | |
| 20:34:27 | mriedem | jaypipes: see my comment on https://review.openstack.org/#/c/638680/ | |
| 20:35:05 | dansmith | mriedem: I dunno, the real change is in that object loader, which seems pretty well covered by two operations, so .. seemed like enough to me | |
| 20:35:21 | artom | mriedem, ah, got it, I think. Sorry, am slow. | |
| 20:35:28 | dansmith | I pulled it down and poked at it too, both fail in the expected way... | |
| 20:35:33 | mriedem | dansmith: yeah ok, i just don't like being told it's not an issue for live migration :) | |
| 20:36:04 | dansmith | well, I don't think stacking one on top is a problem if it makes you feel better | |
| 20:36:33 | mriedem | it won't fill the hole inside | |
| 20:36:44 | dansmith | hah | |
| 20:36:59 | artom | mriedem, oh, ODL *doesn't* send events for binding host changes | |
| 20:37:27 | mriedem | WRONG | |
| 20:37:35 | artom | DAMMIT | |
| 20:37:41 | mriedem | it *only* sends events in that case | |
| 20:37:42 | dansmith | ARGH | |
| 20:37:55 | artom | That note is damn confusing, I'm not backing down from that >_< | |
| 20:38:14 | mriedem | is it time for a truth table in that config option note? | |
| 20:38:27 | mriedem | if x and y but not z = False?! | |
| 20:38:38 | artom | ... | |
| 20:39:17 | artom | OK, sorry, it's fine. | |
| 20:39:23 | artom | I misread mriedem's line here. | |
| 20:39:51 | artom | I mean, mostly fine, it could be worded better. | |
| 20:40:57 | dansmith | but, the note is for an operator, | |
| 20:41:04 | dansmith | and all they need to know is ODL is not okay | |
| 20:41:16 | dansmith | it's not meant to explain the inner workings of both services, like you want it to, right? | |
| 20:41:21 | artom | Yeah | |
| 20:41:50 | artom | I just think explicitly saying what "OK" (or "will not currently work") would have been better | |
| 20:42:33 | artom | I need to be off anyways | |
| 20:42:41 | openstackgerrit | Merged openstack/nova master: Fix an error when generating a host ID https://review.openstack.org/639936 | |
| 20:43:31 | mriedem | for those that *need* to know the inner details, the bug report is linked | |
| 20:44:12 | openstackgerrit | Adam Spiers proposed openstack/nova master: Add detection of SEV support from QEMU/AMD-SP/libvirt on AMD hosts https://review.openstack.org/633855 | |
| 20:46:32 | openstackgerrit | Adam Spiers proposed openstack/nova master: Add detection of SEV support from QEMU/AMD-SP/libvirt on AMD hosts https://review.openstack.org/633855 | |
| 20:55:24 | sean-k-mooney | mriedem: :) i was having dinner but i see ye were dicussing how odl breaks all the things when it comes to vif-plugged events | |
| 20:57:54 | sean-k-mooney | artom: by the way i straed on https://review.openstack.org/#/c/635083/ to recored which ml2 driver bound a port so we can use it in nova for thing and i intend to add another feature that report when each ml2 driver sends events. e.g. on portbinding vs when the backend wires it up | |
| 21:04:50 | mriedem | easy bug fix here https://review.openstack.org/#/c/638821/ | |
| 21:04:53 | sean-k-mooney | artom: what im hpoing we can do in train is take all the guess work out of when the event will be sent and have neutron tell us but i need to bring this up with the neutron folks and work with them to create a spec. | |
| 21:08:25 | sean-k-mooney | i know we made the DriverBlockDevice inherit from dict to make tanstioning to it simpler but at some point it would be better to bit the bullet and just make it a reall nova object/ovo | |