| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-02-15 | |||
| 13:56:55 | dmitriis | i.e. I could tell Nova that a VF is remote-managed but run ovs-vswitchd and ovn-controller locally | |
| 13:56:59 | sean-k-mooney | technically however i dont think we shoudl really support that | |
| 13:57:08 | sean-k-mooney | we certenly should not document it in any upstream docs | |
| 13:57:25 | sean-k-mooney | except perhaps a developer docs | |
| 13:57:26 | dmitriis | yes, I don't intend to to avoid a clash with the existing approach | |
| 13:58:24 | sean-k-mooney | well its not so much that it would be a clash its more that its semanticaly incorrect. im also not sure what would add teh representor vf to ovs in that case | |
| 13:58:34 | sean-k-mooney | since os-vif wont do that nor will nova | |
| 13:58:46 | dmitriis | sean-k-mooney: that would be ovn-controller with ovn-vif | |
| 13:58:47 | sean-k-mooney | ovn would have to do that | |
| 13:58:49 | sean-k-mooney | ya | |
| 13:58:58 | sean-k-mooney | so you would have to also deploy that | |
| 13:58:58 | dmitriis | yep, much like we do on BF2 | |
| 13:59:03 | dmitriis | yes | |
| 13:59:17 | sean-k-mooney | so for a dev setup it coudl be used to fake a BF2 | |
| 13:59:34 | dmitriis | yes, exactly. ConnectX devices also have a VPD | |
| 13:59:35 | sean-k-mooney | but we dont want operators doing that really becasue we should not assuem that will always work | |
| 14:00:08 | dmitriis | sean-k-mooney: ack, agreed | |
| 14:00:08 | sean-k-mooney | dmitriis: ya i think there are some broadcom cards that have it too not sure about intel | |
| 14:00:28 | sean-k-mooney | in princiapl vpd is all that is needed | |
| 14:00:57 | sean-k-mooney | well and switch deve prot representer netdevs i guess for ovs-vif | |
| 14:01:01 | sean-k-mooney | *ovn-vif | |
| 14:01:05 | dmitriis | yes | |
| 14:01:40 | sean-k-mooney | hopefully other manufactures will implement both and they can just be enabeld using the work you already did | |
| 14:02:05 | dmitriis | sean-k-mooney: for now the switchdev-capable NIC kernel driver. Plenty of drivers in DPDK support representors so we are looking into that too | |
| 14:02:26 | sean-k-mooney | the dpdk ones are userspace only however | |
| 14:02:32 | sean-k-mooney | we dont have support for them in openstack | |
| 14:02:50 | sean-k-mooney | so you would have to add supprot for that end to end in a similar way | |
| 14:03:17 | sean-k-mooney | although i assume they just use vhost-user on the qemu side | |
| 14:03:23 | dmitriis | sean-k-mooney: hmm, why would the hypervisor side VF care how a VF representor is handled at the remote side? | |
| 14:03:44 | dmitriis | i.e. it could be ovs + DPDK at the BF2 side while the hypervisor side VF would use anything it wants | |
| 14:03:59 | sean-k-mooney | because dpdk does not require VFs | |
| 14:04:26 | sean-k-mooney | it support userspace only represntors that use intels sio cabliteis too | |
| 14:04:46 | sean-k-mooney | so in generally you shoudl not assuem with dpdk that we woudl have a vf attach to the vm at all | |
| 14:05:04 | sean-k-mooney | it could just be a vhost-user prot that is then interally connect to a subfunction | |
| 14:05:44 | sean-k-mooney | useing ovs-dpdk on the BF2 side is different and likely can be hiddne since you are enabelign a VF based approch | |
| 14:05:57 | dmitriis | yeah | |
| 14:05:59 | dmitriis | https://doc.dpdk.org/guides/prog_guide/switch_representation.html#port-representors | |
| 14:06:00 | sean-k-mooney | dpdk woudl just be used for faster flow programing via dpdk flow instead fo tc_flower | |
| 14:06:12 | dmitriis | sean-k-mooney: yep, that's where I was going | |
| 14:06:31 | sean-k-mooney | ya so that is not the same as dpdk port-represntors feature | |
| 14:06:48 | dmitriis | right, I see | |
| 14:06:55 | dmitriis | so this would be to have a broader coverage at the DPU side (BF2 or not) | |
| 14:07:02 | sean-k-mooney | you can use vfs with it but it is more general | |
| 14:08:03 | sean-k-mooney | you should be able to use dpdk with what you have provided today | |
| 14:08:20 | opendevreview | Balazs Gibizer proposed openstack/nova master: Record SRIOV PF MAC in the binding profile https://review.opendev.org/c/openstack/nova/+/829248 | |
| 14:08:26 | sean-k-mooney | since we still have a VF on the host side | |
| 14:08:52 | sean-k-mooney | dmitriis: it would only change if we wanted to supprot subfunciton or ovs-dpdk runnign on the host with hardware offload | |
| 14:09:05 | dmitriis | sean-k-mooney: yes, probably with some minor modifications to os-vif but at the OpenStack side things would remain the same | |
| 14:09:36 | dmitriis | sean-k-mooney: yes, subfunctions need a little bit more thinking since they are more dynamic | |
| 14:09:52 | dmitriis | we currently just enable SR-IOV and expect VF representors to appear the other side | |
| 14:10:18 | sean-k-mooney | ya | |
| 14:10:42 | sean-k-mooney | so i proably would still prefer to track subfunciton in the pci tracker as a new type | |
| 14:10:47 | sean-k-mooney | like vdpa | |
| 14:11:05 | sean-k-mooney | even if they are not a vf and may not have a pci adress | |
| 14:11:21 | sean-k-mooney | at which point we might want to rename the module to be host_dev tracker or something | |
| 14:11:42 | sean-k-mooney | i kind of want it to eventualy track mdevs too | |
| 14:12:01 | sean-k-mooney | so it woudl be nice if we could unify it all in a common module | |
| 14:12:10 | sean-k-mooney | but that is not currently a priority for me | |
| 14:12:44 | sean-k-mooney | we alternitivly can use the generic resouces table to track these liek we do for persictent memroy namespaces | |
| 14:12:50 | sean-k-mooney | it was created to be generic | |
| 14:13:02 | sean-k-mooney | it might be a better fit for subfunciton and mdevs | |
| 14:13:19 | dmitriis | sean-k-mooney: on module renaming: yes, kind of going along the way VFIO is going (it has a common part and bus-specific parts) | |
| 14:13:28 | sean-k-mooney | but i have not really spent time looking, at least not in the last 2 years | |
| 14:15:26 | dmitriis | sean-k-mooney: one other direction which we haven't covered is ways for an operator to choose which device will be selected. E.g. in case of multiple DPUs per host or in the cloud we might want to supply product id-like info. | |
| 14:15:36 | dmitriis | currently InstancePCIRequest gets a generated spec | |
| 14:16:07 | sean-k-mooney | no | |
| 14:16:11 | sean-k-mooney | we do not | |
| 14:16:37 | sean-k-mooney | the operator gets to select which device are allowed | |
| 14:16:57 | sean-k-mooney | and can also associate them with a neutron phsyical network | |
| 14:17:18 | sean-k-mooney | but we explcitly do not want them to be able to select device via the pci aliase or the request by other means | |
| 14:17:41 | sean-k-mooney | what we could support is traits requests | |
| 14:17:58 | sean-k-mooney | or perhaps resouce classes | |
| 14:18:20 | sean-k-mooney | i have covered both of those in my pci in placment spec | |
| 14:18:44 | sean-k-mooney | https://review.opendev.org/c/openstack/nova-specs/+/791047 | |
| 14:18:51 | dmitriis | yeah, agreed. I didn't mean we want to specify bus-specific info in a request - just that there's no way to alter device selection | |
| 14:19:22 | dmitriis | ^ yes, that. I need to look at the spec again | |
| 14:19:24 | sean-k-mooney | it depend on what you mean by bus specific but in generally i dont think we shoudl | |
| 14:19:32 | sean-k-mooney | we can do it indrectly | |
| 14:19:38 | sean-k-mooney | vai resouce classes or traits | |
| 14:20:03 | sean-k-mooney | vendor id and product id are some what generic and somewhat specific | |
| 14:20:15 | sean-k-mooney | in that other buss tend to have a similar concept | |
| 14:20:29 | sean-k-mooney | like usb has a similar mapping | |
| 14:22:12 | dmitriis | sean-k-mooney: yeah, an indirect approach as with classes or traits would do it I think. | |
| 14:27:10 | sean-k-mooney | i was propsoing using CUSTOM_<Vendor_id>_<product_id> as the resouce class unless you set one in the pci whitelist | |
| 14:27:34 | sean-k-mooney | and extending the pci_alias to supprot a resouce class too | |
| 14:28:28 | sean-k-mooney | the neutron port could potentally specify it but in generally it woudl be nicer if it specified thigns like Support ipse via a HW_NET_IPSEC trait request instead | |
| 14:29:18 | sean-k-mooney | you would need a new neutron extenion however to tie that all together | |
| 14:39:07 | dmitriis | sean-k-mooney: yes, it would be preferable to use a Neutron extension the way I see it | |
| 14:40:20 | sean-k-mooney | there is a security consern with allow arbitry Resouce classes | |
| 14:40:36 | sean-k-mooney | but traits or vendor id and product id may be allowable | |
| 14:40:55 | sean-k-mooney | we perhasp could allow a RC but not allow you to specify the quantity | |
| 14:41:14 | sean-k-mooney | that is proably safe | |
| 14:53:31 | ralonsoh | sean-k-mooney, is it possible to run nova API without wsgi? | |
| 14:54:06 | sean-k-mooney | yes | |
| 14:54:11 | sean-k-mooney | well | |
| 14:54:20 | sean-k-mooney | without an external wsgi server yes | |
| 14:54:32 | sean-k-mooney | it still use wsgi but the python webserver | |
| 14:54:36 | sean-k-mooney | if you just run nova-api | |
| 14:54:38 | ralonsoh | to be able to run it in debug mode, with pycahrm | |
| 14:55:03 | sean-k-mooney | oh well that is harder you can try but eventlet tends to break things | |