Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-02-15
07:00:29 opendevreview Takashi Kajinami proposed openstack/nova stable/xena: Fix the wrong exception used to retry detach API calls https://review.opendev.org/c/openstack/nova/+/829049
08:27:23 gibi sean-k-mooney: hi! when you are up, regarding https://bugs.launchpad.net/neutron/+bug/1942329 I made some progress and added a proposal to the today's neutron meeting https://wiki.openstack.org/wiki/Network/Meetings#On_Demand_Agenda you might be interested
09:44:51 opendevreview alecorps proposed openstack/nova master: VMware: Support volumes backed by VStorageObject https://review.opendev.org/c/openstack/nova/+/808791
11:14:54 sean-k-mooney gibi: oh ok cool so that is something we can fix entirly on the neutron side
11:15:03 sean-k-mooney * entirly on the nova side
11:15:14 gibi sean-k-mooney: nope we need both nova and neutron change
11:15:18 sean-k-mooney we just need to update the mac addres in the neutron port when we migrate
11:15:36 sean-k-mooney well you are thinking of extending the port bindings?
11:15:45 sean-k-mooney to avoid just doing the mac update?
11:16:01 gibi 1) in single binding case we need to update the mac on a bound port that is rejected today by neutron
11:16:13 sean-k-mooney ah ok
11:16:21 sean-k-mooney i didnt know that would be rejected
11:16:21 gibi 2) in multi binding case we might want to keep the MAC per binding to be able to switch
11:16:38 sean-k-mooney yep
11:17:05 sean-k-mooney the thing is the mac is not part of the port binding
11:17:19 sean-k-mooney which i guess is why we need neutron changes
11:17:24 gibi but then we need neturon to use the MAC from the active binding when returning a port
11:17:45 gibi yes, and that is a neutron change too
11:17:46 sean-k-mooney right but the mac is an atribute or the port extention
11:17:52 gibi yes,
11:18:26 sean-k-mooney rather then port binding extentions so ya if we do that we obviously need to handel this slight differently
11:18:27 gibi so we need to be carefully overlay the mac from the active binding top of the port.mac_address field
11:18:44 gibi if there is a binding
11:19:01 sean-k-mooney what i was orginaily thinking is we would not update the mac until after we had activated teh dest binding
11:19:10 sean-k-mooney but if we cant update the port while its bound
11:19:13 sean-k-mooney then that is not an option
11:20:21 sean-k-mooney by the way technically when we update binding:host_id we are unbinding and rebinding the port so we really should be able to update the mac in that case
11:20:21 gibi we could relax the check in neutron to allow mac update if the same request updates the binding:host_id but it becomes messy when we have multiple binding or if nova needs to update the mac separately from the binding activation
11:20:33 sean-k-mooney ya
11:20:51 sean-k-mooney i can see why you would want to extend the port bidnign extention
11:21:00 sean-k-mooney that seams reasonable to me
11:21:13 sean-k-mooney and just have neutron do the mac update when we do the activation
11:21:21 gibi the API request that activates the binding cannot carry a mac update today so that path is a lot harder to take
11:21:37 sean-k-mooney in the cold migrate case we will still need to be able to update the mac when setting the host id however
11:21:37 gibi so I would put the mac to the binding instead
11:22:04 gibi sean-k-mooney: you are right we have two cases
11:22:29 sean-k-mooney we are not allow to write to the port binding:details but you can put the "migrate_mac" in the binding:profile
11:22:40 gibi sean-k-mooney: I'm wondering if we can add the mac to the binding in case of cold migrate and let neutron multiple binding logic see that as the single active binding
11:23:04 sean-k-mooney gibi: we could also finally start using multiple port binding for cold migration
11:23:12 sean-k-mooney we already do for cross cell
11:23:14 gibi that is a biiig change
11:23:26 gibi afaik
11:23:29 gibi but yes
11:23:32 gibi that would be nice
11:23:46 sean-k-mooney it is but its on out todo list for a long time. perhaps after the bug is fixed :)
11:24:02 sean-k-mooney unless we just want to do this as a feature next cycel
11:24:05 gibi the keyword is _after_ ;)
11:24:24 gibi this MAC update thing is something I would like to backport to victoria if possible
11:24:30 gibi as we need this downstream
11:24:54 sean-k-mooney so what i woudl propsoe is addign a migrat_mac to the binding profile like the other migrating too metadata we have
11:25:23 sean-k-mooney and then allow updating the mac to that migration_mac when we set the host id
11:25:37 sean-k-mooney and in the multiple case have that automaticaly update when we activate the port binding
11:25:57 gibi yeah that is my plan
11:26:52 gibi and you have a point, if we actually update the mac in the db then we can simplify the logic in neutron. initially I planned to just overlay the binding mac over the port mac, but that is complicated from implementation perspective
11:28:27 gibi and one extra conditional that we only need to do this whole thing to PFs (direct-physical) ports. In the rest of the case the mac flows from neutron to the device, but for PF the mac flows from the device to neturon
11:29:05 sean-k-mooney well if we want to do that we can basicaly have neutron do (if host_id == profile:migrating_to then old_mac=mac; mac=profile:migration_mac profile:old_mac=old_mac;
11:29:18 sean-k-mooney )
11:30:33 sean-k-mooney we would need to record the old mac for revert althoug we wont have cleared the souce pci device allcoation so we might not need old_mac we can just look that up if we revert on the source
11:30:33 gibi I think we don't even have to save the old mac. either it is in an inactive binding already or the port is being unbound
11:31:00 gibi when neutron unbinds a PF it generates a random mac for it
11:31:16 sean-k-mooney ya we dont for cold migration we have it and for live migration we eitehr activate teh dest binding and never rever or we revert before the mac update
11:31:43 sean-k-mooney gibi: no nueton does not generate random macs for pfs
11:31:43 gibi but if we revert then we revert to the source binding that will have the source mac
11:31:54 sean-k-mooney it generate a mac when you first create the port
11:32:01 sean-k-mooney but that is the same for any port
11:32:15 sean-k-mooney it does not generate random macs after that when you change the bining
11:32:55 gibi sean-k-mooney: a/neutron/plugins/ml2/plugin.py
11:33:01 gibi sean-k-mooney: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py#L438
11:33:16 sean-k-mooney that looks like a bug
11:33:24 gibi called from https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py#L1782
11:33:45 sean-k-mooney actully it looks like they tried to work around this once before by adding that
11:34:28 gibi today nova writes the port.mac_address for PF ports before the first binding. So if there won't be a reset in neutron then after unbind the port would keep the mac of a real device
11:34:46 sean-k-mooney https://bugs.launchpad.net/neutron/+bug/1830383
11:35:00 sean-k-mooney ya so that code was an incorrect fix for the current issue your trying to solve
11:35:16 sean-k-mooney gibi: that should be reverted
11:35:36 gibi but then what mac a PF port should have after it is unbound
11:35:38 gibi ?
11:35:48 gibi empty?
11:35:51 sean-k-mooney the same one it had when it was bound
11:35:59 sean-k-mooney nova is still telling the vm to use that mac
11:36:08 sean-k-mooney in the cold migration case
11:36:20 gibi hm
11:36:56 sean-k-mooney i can see why they did this but today since we are not updating the mac on the neutron port in the cold migration case this only help in the event we detach the port
11:36:57 gibi so if I boot a VM with a PF port, then that port gets the MAC of the PF today. Then I delete the VM. The port should forget the MAC of the PF
11:37:30 sean-k-mooney ya in that case the currnt reset hack works
11:37:40 gibi so we have to keep that reset for the unbinding case
11:37:48 sean-k-mooney we can yes
11:38:10 gibi for the rebinding there will be a new MAC from nova
11:38:49 sean-k-mooney however if you have 2 vm and 3 hosts and you move vm a form host a to host c and vm b form host b to host a today
11:39:01 sean-k-mooney vm b will get a dhcp responce for vm a
11:39:28 sean-k-mooney because in neutorn we are not updating the macs and vm a still has a port with the mac for port b
11:39:36 sean-k-mooney *for host a
11:39:49 sean-k-mooney assuming vm a and b are on the same netork
11:40:28 sean-k-mooney so the reset is not happeing today when we unbindi the port unle when we detach the port
11:40:59 gibi I have verified it. When I delete the VM neutron resets the MAC
11:41:02 gibi today
11:41:05 sean-k-mooney well i guess it depend on how you define "unbind"
11:41:09 sean-k-mooney yep
11:41:53 sean-k-mooney so if you update the host-id today the port will breifly go to vif_type=unbound
11:42:13 sean-k-mooney so i consider updating the host-id to be unbinding/rebinding the port

Earlier   Later