Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-10-12
16:15:36 mriedem sean-k-mooney: so going back to your question,
16:16:00 mriedem what happens today if you try live migrating an instance with a linuxbridge vif on the source host to a dest host which is using ovs?
16:16:04 mriedem does vif plugging explode?
16:16:23 sean-k-mooney no everything works perfectly with no error... in that direction
16:16:30 sean-k-mooney but you have no network connectivity
16:16:49 mriedem ok so it doesn't work
16:16:57 mriedem it doesn't blow up, but it doesn't work, right?
16:17:03 sean-k-mooney what libvirt did undder the hood was creat a linux bridge an plug the tap into it and neutron never knew about it
16:17:27 sean-k-mooney so the live migration succeeds
16:17:43 sean-k-mooney if you do a hard reboot everything gets fixed
16:17:53 sean-k-mooney but the bridge does not get cleaned up
16:18:24 sean-k-mooney going the other way os-vif explodes if your linux bridge node does not have ovs-vsctl available
16:19:05 sean-k-mooney if it does same thing. we create and ovs bridge called br-int and add the tap to it and the linux bridge agent never know about it so it never get wired up
16:19:57 mriedem huh, seems like something we should be explicitly checking for and failing on during pre live migration
16:20:38 mriedem LB and OVS aside, i mean you also have like macvtap and vhostuser and all of the other vif types
16:20:46 mriedem so who know what goes wrong when migrating between those right?
16:20:50 sean-k-mooney mriedem: you would think so but this actully used to work before hevana but it was a undocument "feature" so it got broken in icehouse
16:21:27 sean-k-mooney mriedem: provided the vif model is still virtio then it actully works correctly if we can update the xml
16:21:54 mriedem can't we just assert that if both nodes in the live migration are not new enough to do the port binding thing that we don't allow it?
16:22:03 mriedem and the vif type changes i mean
16:22:23 sean-k-mooney yes we can that was what i was going to suggest
16:22:35 mriedem if the vif type doesn't change, then maybe we can still support the port binding thing for mixed version computes?
16:23:08 mriedem although if the dest host is old we won't have a port binding to flip to active in post live migrate
16:23:14 sean-k-mooney if the vif_type and vif binding_details does not change we should not need to update the xml either so we could activate the binding before live migrate
16:23:21 mriedem so we can really only do the port binding if going from old to new yes?
16:24:01 mriedem to reset, we plan to create the dest host port binding during pre live migration yes?
16:24:18 sean-k-mooney there are basically 3 cases where we have old+new. i am going to document them in the spec
16:24:21 mriedem that may or may not happen if we're doing that on an old ode
16:24:22 mriedem *node
16:24:30 mriedem that would be helpful :)
16:24:55 mriedem btw, https://docs.openstack.org/nova/latest/reference/live-migration.html is uber helpful
16:25:24 sean-k-mooney yes but to reset if its an old source node it wont create the binding but we could do that in do_migrate on the destination.
16:25:39 mriedem pre_live_migration runs on the dest host and i think that's where the dest host port binding was going to be created, but i'm not sure
16:25:42 sean-k-mooney if you have a new source node and old destination there is no way to update the xml
16:26:01 mriedem ok so we create the dest host port binding from the source node?
16:26:48 mriedem live_migrate runs on the source node, calls pre_live_migration on the dest node, and then live_migrate on the source starts doing it's thing
16:26:54 sean-k-mooney mriedem: ill duble check i have been confusing myself with this most of the afternoon so kindo of mixed up what happens where
16:27:02 mriedem use https://docs.openstack.org/nova/latest/reference/live-migration.html
16:28:29 kashyap kaisers: When you're back; glad you noted the correction in the bug: "1) correction regarding my last comment (#6): block commit -> blockRebase"
16:28:37 kashyap kaisers: They're both completely opposite.
16:29:32 sean-k-mooney right ok ill work on the spec and use https://docs.openstack.org/nova/latest/reference/live-migration.html to keep it straight
16:31:39 sean-k-mooney mriedem: basically what i wanted to know was is it ok to change the sematic in the event that we need an xml update but cant do one and fail the migrate instead of migrating and haveing broken networkin that is then fixed with a hard reboot
16:32:21 mriedem sean-k-mooney: it seems to me we shouldn't be allowing that today anyway
16:32:25 mriedem so yes i'd be ok with that
16:33:27 sean-k-mooney ok ill capture that in the spec with the different combinations. thanks
16:35:07 openstackgerrit Eric Fried proposed openstack/nova-specs master: Remove barbican from scope of ksa adapter work https://review.openstack.org/511524
16:35:10 efried sdague FYI, crickets in -barbican
16:35:23 efried sdague mriedem ^ updated spec accordingly.
17:02:37 openstackgerrit Elod Illes proposed openstack/nova master: WIP: Transform scheduler.select_destinations notification https://review.openstack.org/508506
17:06:00 openstackgerrit Brianna Poulos proposed openstack/nova master: Add trusted_image_certificates to REST API https://review.openstack.org/486204
17:17:53 openstackgerrit Merged openstack/nova master: api-ref: add warning about force evacuate for ironic https://review.openstack.org/511477
17:42:21 openstackgerrit melanie witt proposed openstack/nova stable/pike: Fix target_cell usage for scatter_gather_cells https://review.openstack.org/511538
17:54:38 mriedem dansmith: on this migrations API paging spec https://review.openstack.org/#/c/506030/ i don't think we need to do the elaborate scatter/gather thing like you did for instance listing probably, with the built in merge sorting and all
17:55:07 dansmith mriedem: migrations are per instance right?
17:55:11 mriedem we could build in the scatter/gather on it's own if we wanted, but it seems we could just merge sort the results once we pull them out of the db, the user can't pass in sort keys and this is admin-only anyway so performance shouldn't be a huge concern
17:55:12 dansmith if so, there's no scattering
17:56:02 mriedem not sure i follow,
17:56:13 mriedem couldn't we use the scatter/gather thing on this today? https://github.com/openstack/nova/blob/master/nova/compute/api.py#L3995
17:56:29 mriedem it would just run the db query concurrently per cell and munge the results
17:56:29 dansmith mriedem: you list migrations for an instance, right?
17:56:37 mriedem not this api
17:56:46 mriedem this is all migrations regardless of instance
17:56:49 dansmith oh
17:57:14 dansmith don't we have one that is per instance?
17:57:17 mriedem we do
17:57:24 dansmith *awesome*
17:57:33 mriedem but paging that one would be more like paging simple tenant usage i think,
17:57:36 mriedem which was weird
17:57:46 dansmith anyway,
17:58:46 dansmith I would think just doing linear paging for that would be fine yeah
17:58:49 dansmith until someone complains
17:59:41 mriedem yup cool
18:00:01 mriedem only cburgess would have that many migration records to page anyway
18:00:07 cburgess Do what?
18:00:21 mriedem migrating like a mofo
18:00:24 cburgess yes
18:00:26 cburgess yes we do..
18:00:28 cburgess *sigh*
18:00:39 mriedem well then do we have the spec for you! https://review.openstack.org/#/c/506030/
18:10:02 _ix Hello friends. I can think of a way to do this, but I was hoping there might be a better way. I'm looking to determine whether some of our nodes that are believed to be running in an HA capacity are in fact running on the same hypervisors.
18:10:35 _ix Is there a way to determine from the guest VM which hypervisor host it's running on, at instantiation or otherwise?
18:12:03 dansmith _ix: you can do this from the nova api
18:13:04 _ix I'm familiar with that approach, but I was hoping to register a custom fact with puppet, and instead just used puppetdb to keep an eye on the status of these nodes.
18:14:35 _ix I would suspect that the guest never needs to know who its host is.
18:15:25 dansmith or shouldn't
18:15:41 dansmith I don't know that we expose the sanitized host id or not
18:15:44 dansmith in metadata I mean
18:15:50 dansmith I kinda expect not, but I'd have to look
18:16:22 mriedem i looked, didn't see it
18:34:57 mriedem we should be able to remove the safe_connect decorator now can't we? https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L56
18:35:04 mriedem that was for the newton->ocata transition
18:35:12 mriedem placement is definitely required at this point and a hard failure if it's not there
18:35:39 mriedem and in ocata nova-compute doesn't start either if placement isn't configured for the compute node
18:37:44 dansmith it's also confusing,
18:37:48 dansmith because 404 -> None
18:37:50 dansmith instead of explody
18:43:09 mriedem yeah that's what made me think of it, per gibi's comment in your change
18:43:24 dansmith yeah I just saw that
18:44:01 mriedem shall i patchify the removification?
18:44:56 dansmith if and as you wishitifcate

Earlier   Later