| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-05-09 | |||
| 12:13:04 | gibi | *everything | |
| 12:13:16 | sean-k-mooney | gibi: im looking for the code now | |
| 12:13:31 | sean-k-mooney | but we had code to normalise the capablites and report them to placement that ralonsoh wrote in the past | |
| 12:13:33 | gibi | sean-k-mooney: yepp the today spec only supports additive traits | |
| 12:14:45 | sean-k-mooney | https://review.opendev.org/q/topic:bp%252Fenable-sriov-nic-features | |
| 12:15:09 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/466051 specficically | |
| 12:15:18 | sean-k-mooney | but you might be able to reuse some of the other work | |
| 12:16:19 | sean-k-mooney | gibi: the traits have already been added to os-traits https://github.com/openstack/os-traits/blob/master/os_traits/hw/nic/offload.py | |
| 12:17:18 | gibi | thanks, this helps | |
| 12:17:44 | gibi | so lets report capabilities as traits | |
| 12:17:59 | gibi | and then I will amend the spec to allow removing traits via device_list | |
| 12:18:08 | gibi | to disable capability | |
| 12:18:09 | sean-k-mooney | yep. right now this only makes sense for neutron nics really | |
| 12:18:20 | sean-k-mooney | since we dont really gather capablities for other devices | |
| 12:18:30 | gibi | ahh | |
| 12:18:36 | gibi | so no generic PCI caps | |
| 12:18:37 | sean-k-mooney | although remote_mannaged might be the excption | |
| 12:18:56 | sean-k-mooney | well for the remote managed deviecs we now have the vpd | |
| 12:18:57 | sean-k-mooney | capablity | |
| 12:19:17 | sean-k-mooney | that is not yet a trait so maybe we would want to report that | |
| 12:19:48 | sean-k-mooney | gibi: i think for the inital version we could keep it to jsut operator provided traits | |
| 12:19:54 | sean-k-mooney | if we want to keep it simple | |
| 12:20:33 | sean-k-mooney | then in the future we can auto discover device capablities and report them if that makes sense | |
| 12:20:44 | gibi | yeah that make sense, it is easy to add later | |
| 12:20:57 | gibi | so keeping traits just additive now as well | |
| 12:21:58 | gibi | OK | |
| 12:22:21 | gibi | next one | |
| 12:22:23 | gibi | https://review.opendev.org/c/openstack/nova-specs/+/791047/4/specs/zed/approved/pci-device-tracking-in-placement.rst#288 | |
| 12:22:33 | gibi | What to do if both ``resource_class`` and ``vendor_id`` and ``product_id`` are provided in the alias? | |
| 12:23:10 | sean-k-mooney | good question. i guess we have too options | |
| 12:23:15 | sean-k-mooney | first is consider that an error | |
| 12:23:33 | sean-k-mooney | second is use the resouce_class for placment queries | |
| 12:23:50 | sean-k-mooney | and the the rest for the pci/numa filter | |
| 12:23:53 | gibi | the only use case I can think of is that deployer use a generic RC but then later want to refine the alias via product id | |
| 12:24:20 | sean-k-mooney | my secret plan is to eventually remove the need for the alias | |
| 12:24:41 | sean-k-mooney | so over time it woudl be nice if we coudl move to jsut having the resouce class in the alias | |
| 12:24:42 | gibi | do you want to go with flavor extra_spec based resource? | |
| 12:25:00 | sean-k-mooney | yes and no | |
| 12:25:11 | sean-k-mooney | i current hate that we allow grouping in the extra_specs | |
| 12:25:36 | sean-k-mooney | so part of me wants to keep the alisa as we can insulate operators form that | |
| 12:26:14 | sean-k-mooney | but i do kind of like the idea of have just resouce:CUSTOM_<whatever>=1 | |
| 12:26:51 | gibi | OK, I get the goal that we want to move deployers to RC based alias in the future and if the deployer still want product id based filtering then the deployer can use traits for that | |
| 12:26:55 | sean-k-mooney | so i think having the RC take prescidence for the placment query and allowign vendor_id and product_id makes sense | |
| 12:27:40 | gibi | OK | |
| 12:28:06 | sean-k-mooney | so really what i would like is for operators to take devces with the custom resouce class and use the RC name as the "alias" | |
| 12:28:26 | gibi | so we keep the PCIFilter to keep filtering for vendor / product | |
| 12:28:33 | gibi | at least for now | |
| 12:28:46 | gibi | the RC name as alias make sense | |
| 12:28:57 | sean-k-mooney | ya for now althogh in princial i think you could trun it off if we did this right | |
| 12:29:42 | sean-k-mooney | my idea is | |
| 12:29:45 | gibi | yes if we let the product id filtering case go and no SRIOV in the deployment then we can turn off the PCIFilter | |
| 12:30:02 | gibi | neutron based SRIOV | |
| 12:30:16 | sean-k-mooney | device_list = some device -> RC gpu_gold | |
| 12:30:36 | sean-k-mooney | and then you could jsut ask for RC gpu_goal in the alias | |
| 12:31:01 | sean-k-mooney | right now the reason i dont wnat to go directly to resouce:gpu_gold=1 | |
| 12:31:19 | sean-k-mooney | is that intoduced problems wiht vgpu and generic mdev usage | |
| 12:31:31 | sean-k-mooney | it shoudl be resovlable | |
| 12:31:59 | sean-k-mooney | i.e. if we see that the resouce does not match any of the generic_mdev types listed in teh config we know tis a pci passthough request | |
| 12:32:18 | sean-k-mooney | but i tought that would complicate the spec more then need initally | |
| 12:33:11 | gibi | ahh yeah | |
| 12:33:22 | gibi | thanks for the background | |
| 12:33:35 | gibi | next | |
| 12:34:04 | gibi | I feel that both you and me want to keep the dependent device handling supported. But as stephenfin said, it is a lot of complexity | |
| 12:34:14 | gibi | so just double checking it that you still think this is needed | |
| 12:34:27 | gibi | as per https://review.opendev.org/c/openstack/nova-specs/+/791047/4/specs/zed/approved/pci-device-tracking-in-placement.rst#320 | |
| 12:34:40 | sean-k-mooney | honestly i would like to be able to remove it. but im concerned by the upgrade impact | |
| 12:35:11 | sean-k-mooney | i do know we have customer that want to dynically choose if they consime a device a a PF or VF when they boot the workload | |
| 12:35:23 | sean-k-mooney | but its not very reliable today | |
| 12:36:00 | sean-k-mooney | as in its easy for vms to consume 1 vf on all the devices | |
| 12:36:12 | gibi | I also think that there is many deployment out there that is was used without knowing it. I mean if somebody whitelisted a PF that had VFs then the VFs become scheduleable automatically | |
| 12:36:23 | sean-k-mooney | basically meaning you can not allocate PF even though your could have allocate the vfs differntly | |
| 12:37:03 | sean-k-mooney | right so i think what stephenfin had in mind was if you whitelist the PF and it has VFs we would only expose the VFs | |
| 12:37:22 | sean-k-mooney | where as today unless you use the product_id to filter | |
| 12:37:30 | sean-k-mooney | we expos both the VFs and PFs | |
| 12:37:55 | sean-k-mooney | if we maintian the current behavior we obvilly need ot dynamically adjst the reserved value | |
| 12:38:04 | gibi | yes, that is the complexity | |
| 12:38:07 | sean-k-mooney | to emulated the unclaimable state | |
| 12:38:08 | gibi | but it is solveable | |
| 12:38:31 | gibi | I think I will keep this open for bauzas or other reviews to chime in | |
| 12:38:59 | sean-k-mooney | sure | |
| 12:39:13 | sean-k-mooney | we decided to reduce flexiblity for cpu pinning | |
| 12:39:15 | sean-k-mooney | with isolate | |
| 12:39:25 | sean-k-mooney | and we know that not everyone was happy with that | |
| 12:40:01 | sean-k-mooney | we can elect to do the same here but we need to be deliberiate about it and comunicate it well if we want to force this change | |
| 12:40:09 | ralonsoh | sean-k-mooney, sorry, I was having lunch | |
| 12:40:13 | ralonsoh | what do you need? | |
| 12:40:22 | sean-k-mooney | if we can live with the complexity then we proably shoudl keep it | |
| 12:40:28 | sean-k-mooney | ralonsoh: i found it | |
| 12:40:36 | gibi | OK, I will plan with the complexity | |
| 12:40:37 | ralonsoh | ah perfect | |
| 12:40:39 | sean-k-mooney | ralonsoh: https://review.opendev.org/q/topic:bp%252Fenable-sriov-nic-features | |
| 12:40:47 | sean-k-mooney | ralonsoh: your code for tracking pci device in placment | |
| 12:41:12 | sean-k-mooney | ralonsoh: we are just discussing the spec to enable it. gibi will be taking on that feature | |
| 12:41:20 | ralonsoh | perfect | |
| 12:41:39 | gibi | ralonsoh: https://review.opendev.org/c/openstack/nova-specs/+/791047/4/specs/zed/approved/pci-device-tracking-in-placement.rst this is the spec if you are interested :) | |
| 12:41:47 | ralonsoh | sure | |
| 12:42:01 | gibi | sean-k-mooney: so I have one more open question | |
| 12:42:08 | sean-k-mooney | gibi: go for it | |
| 12:42:09 | gibi | upgrade | |
| 12:42:19 | gibi | obviously rolling upgrade is a pain | |