Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-10-11
09:53:47 sean-k-mooney ...
09:53:49 sean-k-mooney "connectivity": "l2",
09:53:51 sean-k-mooney "bound_drivers": {"0": "openvswtich"}
09:53:51 gibi thanks
09:53:53 sean-k-mooney }
09:54:00 sean-k-mooney gibi: we could use the bound_drivers to infer this
09:54:13 sean-k-mooney the spec orginaly had an events section that told use when it sent events
09:54:42 sean-k-mooney that was unfrotunetly removed because we could not agree on how to model that
09:55:12 sean-k-mooney so we would have to create a lookup table in nova and map the driver name to it
09:55:36 sean-k-mooney opendaylight was bind time, opendaylight_v2 is plug time
09:56:36 gibi ack. I might need to create a solution on Pike level downstream hence my continued digging in it
09:56:41 sean-k-mooney the conneectivy field is waht we need to use to know if we can safely allow unadress port too
09:56:45 gibi but if the infor is in train then I cannot use that in Pike
09:56:48 sean-k-mooney if its l2 it does not need an ip
09:57:43 sean-k-mooney gibi: i think a workaround config option to force_rebind_on_reboot is the way to go and backport that
09:57:55 gibi yeah I thinking about a config too
09:58:02 sean-k-mooney there are reasons to have that on master by the way
09:58:11 sean-k-mooney combidnind with unpluging on reboot
09:58:52 sean-k-mooney basically if you make reboot: power off-> unplug-> unbind->rebind->plug->spwan
09:59:21 sean-k-mooney then you can do things like change the firewall driver or maybe even ml2/backend
09:59:22 gibi why do you want to add bind to the picture?
09:59:28 gibi ahh I see
09:59:37 sean-k-mooney currently you have to live migrate
09:59:40 gibi to allow change thing that needs a binding
09:59:53 sean-k-mooney but we have some customer saying they want to do it inplace and are ok with stoping the vm
09:59:58 sean-k-mooney yes
10:00:55 sean-k-mooney basicaly some large customes have said to use we cant live migrate every vm we have but we can stop them cause we run everything in duplex mode and can go simplex for a period of time without issue
10:01:45 gibi OK so there is two parallel direction to pursue. 1) add a workaround flag for something like wait_for_plug_event_at_reboot and 2) force_rebind_on_reboot to allow in place update of binding related data
10:02:06 sean-k-mooney so they are fine with stop(tearing down all networking) and start(setting up all networking) to "migrate" form one firewall driver to another (iptables->ovs) or between mech drvers(ml2/ovs->,l2/ovn)
10:02:31 sean-k-mooney gibi: am yes i guess so
10:02:51 gibi the problem (most probably) on my side is that they do VM reboots in big batches due to upgrade and nova sends a lot of unplug - plug event to ODL which fails to finish setting up OVS before the VM starts requesting IP via DHCP
10:04:08 gibi so if they can set a WA flag in nova that nova should wait for the network-vif-plugged before letting the VM run at reboot would help to control the situation
10:04:28 sean-k-mooney yep
10:04:38 gibi and they need it on Pike :/
10:05:51 gibi I will go with a backportable WA flag
10:07:13 sean-k-mooney ya so on master i think we need to look at how and when we use events and for pike workaround flag is definetly the simplest thing as only the custoemr that is affect can opt in by settting it minimising the risk to others that dont need the change
10:09:44 sean-k-mooney gibi: i had planned to bring up ovn live migration at the ptg(which is affected by the vif-plugged-events) but i might just make tha ta general topic since i think the ovn issue will require ovn changes to fully resolve
10:19:47 jpic hi all, i don't understand what is supposed to keep the websocket between the spice html5 client and the spice server? browsers will kill inactive websockets after 60 secs of inactivity, so i suppose the server is supposed to send ping packets regularily
10:40:28 gibi sean-k-mooney: ack
10:47:40 gibi sean-k-mooney: one more thing, I don't see "bound_drivers" ever got implemented in neutron. Only neturon-lib has the field definition https://review.opendev.org/c/openstack/neutron-lib/+/645288/6/neutron_lib/api/definitions/portbindings.py#55 but neutron does not use that
10:47:50 gibi I only see the connectivity field being implemented
10:47:55 gibi in https://review.opendev.org/c/openstack/neutron/+/645645/
11:05:52 sean-k-mooney i see... am i think it was ment to be implemented at the ml2 plugin level above the drivers but you might be right
11:06:36 sean-k-mooney https://review.opendev.org/c/openstack/neutron-lib/+/645288/6/neutron_lib/api/definitions/portbindings.py added bound_drivers to the extension
11:06:42 sean-k-mooney but im not sure if they have used it
11:08:32 sean-k-mooney looks like its only in neutron-lib https://codesearch.opendev.org/?q=VIF_DETAILS_BOUND_DRIVERS&i=nope&literal=nope&files=&excludeFiles=&repos=
11:08:48 gibi yeah that matches with my grep results
11:09:13 sean-k-mooney so ya looks like we would need to actully implement that on master before we can use it
11:09:20 gibi yepp, it looks like
11:09:21 sean-k-mooney i feel less bad for not using it yet
11:10:19 gibi :)
12:15:55 sean-k-mooney has anyone else had devstack time out on pip resolver issues
12:22:33 sean-k-mooney Collecting oslo.policy>=3.8.1
12:22:35 sean-k-mooney Using cached oslo.policy-3.8.2-py3-none-any.whl (86 kB)
12:22:37 sean-k-mooney INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
12:40:30 gibi sean-k-mooney: I haven't seen that pip issue yet
12:47:25 opendevreview Balazs Gibizer proposed openstack/nova master: Add a WA flag waiting for vif-plugged event during reboot https://review.opendev.org/c/openstack/nova/+/813419
12:47:51 gibi sean-k-mooney: ^^ this is my first try on the hard reboot - vif plug issue
12:48:56 sean-k-mooney i think its cause by cinder's dep on oslo.vmware which is pulling in suds-jurko
12:49:06 sean-k-mooney which is an unmaintained soap lib fork
12:50:22 sean-k-mooney gibi: that is just the events right are you working on a second patch for rebinding
12:51:02 sean-k-mooney i guess you might not need that in your case
12:51:06 gibi sean-k-mooney: it is jut the event waiting
12:51:15 sean-k-mooney was pikes version of netwokring-odl new enough to send plug time events
12:51:29 gibi sean-k-mooney: yes, the plug time event patch was merged in pike
12:51:37 sean-k-mooney ah ok cool
12:52:04 gibi https://review.opendev.org/c/openstack/networking-odl/+/465463
12:52:06 gibi this is the patch
12:52:26 sean-k-mooney do you have a DNM patch that sets that to true or set it in the job so we can see it work
12:52:42 gibi sean-k-mooney: good point I will add that
12:52:52 gibi sean-k-mooney: ml2/ovs works for me with this locally
12:53:06 gibi sean-k-mooney: so we can enable this WA in our normal ovs jobs
12:53:11 sean-k-mooney cool, ya currently its disabled so we wont see if this breaks anything although it will break ovn actully
12:53:29 sean-k-mooney ya we could enable it in nova-next
12:53:30 gibi I think I will enable this flaf in nova-next
12:53:34 gibi exactly :D
12:53:58 sean-k-mooney yep also we might want to renable a multinode jobs with ml2/ovs at some point
12:54:27 sean-k-mooney im not sure what coverage we have/have lost with the ovn default swap
12:55:08 sean-k-mooney i think nova-next might be the only job that is still using ml2/ovs (other then linuxbrige job) in our gate today
12:55:42 sean-k-mooney i dont think we have a multi node job with a backend that send plugtime events currently
12:56:59 gibi sean-k-mooney: nova-next is multinode afaik
12:57:11 gibi parent: tempest-multinode-full-py3
12:57:11 gibi name: nova-next
12:57:16 gibi yepp it is multinode
12:58:00 opendevreview Balazs Gibizer proposed openstack/nova master: Add a WA flag waiting for vif-plugged event during reboot https://review.opendev.org/c/openstack/nova/+/813419
13:05:07 sean-k-mooney gibi: oh ok cool
13:20:09 gibi bauzas: fyi, I now remember that there will by TC + PTL discussion slot on the PTG. https://etherpad.opendev.org/p/tc-yoga-ptg L15 (Monday 15 UTC)
13:20:29 gibi bauzas: https://etherpad.opendev.org/p/tc-ptl-interaction
13:20:37 gibi I will join too
13:20:45 gibi just to listen in
13:20:53 gibi or fill in info from the past if needed
13:44:04 bauzas gibi: sorry was off
13:44:15 bauzas gibi: ack, I was wondering when the slot was there
14:02:07 opendevreview Balazs Gibizer proposed openstack/nova stable/pike: Add a WA flag waiting for vif-plugged event during reboot https://review.opendev.org/c/openstack/nova/+/813437
14:26:17 opendevreview norman shen proposed openstack/nova master: Add a timeout for waiting volume snapshot ready https://review.opendev.org/c/openstack/nova/+/813443
15:13:07 dansmith gibi: wow, pike huh? :)
15:13:53 gibi dansmith: last week I was pulled into a discussion upgrading Mitaka deployments to Victoria. Pike is easy compared to that
15:14:01 dansmith hah
15:14:47 gibi telcos are slooow
15:14:55 dansmith :)

Earlier   Later