| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-02-15 | |||
| 11:16:13 | sean-k-mooney | ah ok | |
| 11:16:21 | sean-k-mooney | i didnt know that would be rejected | |
| 11:16:21 | gibi | 2) in multi binding case we might want to keep the MAC per binding to be able to switch | |
| 11:16:38 | sean-k-mooney | yep | |
| 11:17:05 | sean-k-mooney | the thing is the mac is not part of the port binding | |
| 11:17:19 | sean-k-mooney | which i guess is why we need neutron changes | |
| 11:17:24 | gibi | but then we need neturon to use the MAC from the active binding when returning a port | |
| 11:17:45 | gibi | yes, and that is a neutron change too | |
| 11:17:46 | sean-k-mooney | right but the mac is an atribute or the port extention | |
| 11:17:52 | gibi | yes, | |
| 11:18:26 | sean-k-mooney | rather then port binding extentions so ya if we do that we obviously need to handel this slight differently | |
| 11:18:27 | gibi | so we need to be carefully overlay the mac from the active binding top of the port.mac_address field | |
| 11:18:44 | gibi | if there is a binding | |
| 11:19:01 | sean-k-mooney | what i was orginaily thinking is we would not update the mac until after we had activated teh dest binding | |
| 11:19:10 | sean-k-mooney | but if we cant update the port while its bound | |
| 11:19:13 | sean-k-mooney | then that is not an option | |
| 11:20:21 | sean-k-mooney | by the way technically when we update binding:host_id we are unbinding and rebinding the port so we really should be able to update the mac in that case | |
| 11:20:21 | gibi | we could relax the check in neutron to allow mac update if the same request updates the binding:host_id but it becomes messy when we have multiple binding or if nova needs to update the mac separately from the binding activation | |
| 11:20:33 | sean-k-mooney | ya | |
| 11:20:51 | sean-k-mooney | i can see why you would want to extend the port bidnign extention | |
| 11:21:00 | sean-k-mooney | that seams reasonable to me | |
| 11:21:13 | sean-k-mooney | and just have neutron do the mac update when we do the activation | |
| 11:21:21 | gibi | the API request that activates the binding cannot carry a mac update today so that path is a lot harder to take | |
| 11:21:37 | sean-k-mooney | in the cold migrate case we will still need to be able to update the mac when setting the host id however | |
| 11:21:37 | gibi | so I would put the mac to the binding instead | |
| 11:22:04 | gibi | sean-k-mooney: you are right we have two cases | |
| 11:22:29 | sean-k-mooney | we are not allow to write to the port binding:details but you can put the "migrate_mac" in the binding:profile | |
| 11:22:40 | gibi | sean-k-mooney: I'm wondering if we can add the mac to the binding in case of cold migrate and let neutron multiple binding logic see that as the single active binding | |
| 11:23:04 | sean-k-mooney | gibi: we could also finally start using multiple port binding for cold migration | |
| 11:23:12 | sean-k-mooney | we already do for cross cell | |
| 11:23:14 | gibi | that is a biiig change | |
| 11:23:26 | gibi | afaik | |
| 11:23:29 | gibi | but yes | |
| 11:23:32 | gibi | that would be nice | |
| 11:23:46 | sean-k-mooney | it is but its on out todo list for a long time. perhaps after the bug is fixed :) | |
| 11:24:02 | sean-k-mooney | unless we just want to do this as a feature next cycel | |
| 11:24:05 | gibi | the keyword is _after_ ;) | |
| 11:24:24 | gibi | this MAC update thing is something I would like to backport to victoria if possible | |
| 11:24:30 | gibi | as we need this downstream | |
| 11:24:54 | sean-k-mooney | so what i woudl propsoe is addign a migrat_mac to the binding profile like the other migrating too metadata we have | |
| 11:25:23 | sean-k-mooney | and then allow updating the mac to that migration_mac when we set the host id | |
| 11:25:37 | sean-k-mooney | and in the multiple case have that automaticaly update when we activate the port binding | |
| 11:25:57 | gibi | yeah that is my plan | |
| 11:26:52 | gibi | and you have a point, if we actually update the mac in the db then we can simplify the logic in neutron. initially I planned to just overlay the binding mac over the port mac, but that is complicated from implementation perspective | |
| 11:28:27 | gibi | and one extra conditional that we only need to do this whole thing to PFs (direct-physical) ports. In the rest of the case the mac flows from neutron to the device, but for PF the mac flows from the device to neturon | |
| 11:29:05 | sean-k-mooney | well if we want to do that we can basicaly have neutron do (if host_id == profile:migrating_to then old_mac=mac; mac=profile:migration_mac profile:old_mac=old_mac; | |
| 11:29:18 | sean-k-mooney | ) | |
| 11:30:33 | sean-k-mooney | we would need to record the old mac for revert althoug we wont have cleared the souce pci device allcoation so we might not need old_mac we can just look that up if we revert on the source | |
| 11:30:33 | gibi | I think we don't even have to save the old mac. either it is in an inactive binding already or the port is being unbound | |
| 11:31:00 | gibi | when neutron unbinds a PF it generates a random mac for it | |
| 11:31:16 | sean-k-mooney | ya we dont for cold migration we have it and for live migration we eitehr activate teh dest binding and never rever or we revert before the mac update | |
| 11:31:43 | sean-k-mooney | gibi: no nueton does not generate random macs for pfs | |
| 11:31:43 | gibi | but if we revert then we revert to the source binding that will have the source mac | |
| 11:31:54 | sean-k-mooney | it generate a mac when you first create the port | |
| 11:32:01 | sean-k-mooney | but that is the same for any port | |
| 11:32:15 | sean-k-mooney | it does not generate random macs after that when you change the bining | |
| 11:32:55 | gibi | sean-k-mooney: a/neutron/plugins/ml2/plugin.py | |
| 11:33:01 | gibi | sean-k-mooney: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py#L438 | |
| 11:33:16 | sean-k-mooney | that looks like a bug | |
| 11:33:24 | gibi | called from https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py#L1782 | |
| 11:33:45 | sean-k-mooney | actully it looks like they tried to work around this once before by adding that | |
| 11:34:28 | gibi | today nova writes the port.mac_address for PF ports before the first binding. So if there won't be a reset in neutron then after unbind the port would keep the mac of a real device | |
| 11:34:46 | sean-k-mooney | https://bugs.launchpad.net/neutron/+bug/1830383 | |
| 11:35:00 | sean-k-mooney | ya so that code was an incorrect fix for the current issue your trying to solve | |
| 11:35:16 | sean-k-mooney | gibi: that should be reverted | |
| 11:35:36 | gibi | but then what mac a PF port should have after it is unbound | |
| 11:35:38 | gibi | ? | |
| 11:35:48 | gibi | empty? | |
| 11:35:51 | sean-k-mooney | the same one it had when it was bound | |
| 11:35:59 | sean-k-mooney | nova is still telling the vm to use that mac | |
| 11:36:08 | sean-k-mooney | in the cold migration case | |
| 11:36:20 | gibi | hm | |
| 11:36:56 | sean-k-mooney | i can see why they did this but today since we are not updating the mac on the neutron port in the cold migration case this only help in the event we detach the port | |
| 11:36:57 | gibi | so if I boot a VM with a PF port, then that port gets the MAC of the PF today. Then I delete the VM. The port should forget the MAC of the PF | |
| 11:37:30 | sean-k-mooney | ya in that case the currnt reset hack works | |
| 11:37:40 | gibi | so we have to keep that reset for the unbinding case | |
| 11:37:48 | sean-k-mooney | we can yes | |
| 11:38:10 | gibi | for the rebinding there will be a new MAC from nova | |
| 11:38:49 | sean-k-mooney | however if you have 2 vm and 3 hosts and you move vm a form host a to host c and vm b form host b to host a today | |
| 11:39:01 | sean-k-mooney | vm b will get a dhcp responce for vm a | |
| 11:39:28 | sean-k-mooney | because in neutorn we are not updating the macs and vm a still has a port with the mac for port b | |
| 11:39:36 | sean-k-mooney | *for host a | |
| 11:39:49 | sean-k-mooney | assuming vm a and b are on the same netork | |
| 11:40:28 | sean-k-mooney | so the reset is not happeing today when we unbindi the port unle when we detach the port | |
| 11:40:59 | gibi | I have verified it. When I delete the VM neutron resets the MAC | |
| 11:41:02 | gibi | today | |
| 11:41:05 | sean-k-mooney | well i guess it depend on how you define "unbind" | |
| 11:41:09 | sean-k-mooney | yep | |
| 11:41:53 | sean-k-mooney | so if you update the host-id today the port will breifly go to vif_type=unbound | |
| 11:42:13 | sean-k-mooney | so i consider updating the host-id to be unbinding/rebinding the port | |
| 11:42:14 | gibi | but yes if I migrate the VM (and not delet it) then I can create the mixed MAC case you described | |
| 11:42:48 | sean-k-mooney | updating the device_id/device_onwer to '' i do not consider to be unbinding the port | |
| 11:42:55 | sean-k-mooney | that is detaching the port | |
| 11:43:56 | gibi | ahh I see | |
| 11:44:31 | sean-k-mooney | gibi: sorry if im beign pedantic with the terminology there it just the device_id and device_owner filed are not part of the port_bindign or port_binding_extended api extentions | |
| 11:44:43 | gibi | true | |
| 11:44:54 | gibi | just the host_id | |
| 11:45:04 | gibi | but I think that is OK | |
| 11:45:14 | sean-k-mooney | https://github.com/openstack/neutron-lib/blob/master/neutron_lib/api/definitions/portbindings_extended.py#L75-L105 yes | |
| 11:45:44 | sean-k-mooney | so that is why updating the host-id in the cold migrat case set the vif_type to unbound and then to the correct vif | |