| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2026-09-11 | |||
| 08:54:15 | sean-k-mooney | we will likely reviit but not on the deployable | |
| 08:54:23 | sean-k-mooney | the attibutes shoudl have been on the device instead | |
| 08:54:50 | bogdando[m] | well, device is PF, and for smart nic, its more makes sense to attribute one or a group of its VFs (deployables) | |
| 08:54:55 | sean-k-mooney | bogdando[m]: but to be clear we will asy want to have a config driven approch as a primary approch | |
| 08:55:04 | sean-k-mooney | we may add some api driven suprpot later | |
| 08:55:16 | sean-k-mooney | but its really not a good ux at scale | |
| 08:55:33 | sean-k-mooney | device is not the pf | |
| 08:55:50 | sean-k-mooney | tdevice is the assiable thing it can be the pf or vf | |
| 08:56:04 | sean-k-mooney | a deployable is a pool of allcoatabel devices | |
| 08:56:41 | sean-k-mooney | currently resouce classes for palcement are tracked via the atibutes api | |
| 08:56:44 | sean-k-mooney | that shoudl be on the device | |
| 08:56:56 | sean-k-mooney | becuase each deployable corresponds to a placment resouce provider | |
| 08:57:01 | bogdando[m] | ack, it seems I missread "A device has a management interface, whose address is the control path identifier: for SR-IOV devices, this is usually the PCI Physical Function (PF)" - PF is indeed control managmenent interface of a device, not the device | |
| 08:57:13 | sean-k-mooney | and we want to be able to have 1 placement resouce provier with inventoreis of diffent resouce class | |
| 08:58:18 | bogdando[m] | yes, I like the idea that a deployable is a pool of VFs | |
| 08:58:20 | sean-k-mooney | to be clear i dont think that hte atibute api as orgianly specified is the right shap and the impelation that was done didnt follow the spec and is even worse | |
| 08:58:38 | bogdando[m] | but I am not sure about PFs VFs boundaries as you suggested | |
| 08:59:42 | bogdando[m] | grouping devices or PFs seems a bit another layer of abstraction than VFs affinity | |
| 09:00:05 | sean-k-mooney | placement is not deisign to have 100s of rps for a given host | |
| 09:00:27 | sean-k-mooney | it casue sever performance probelms and a singel nic can have 100s of VFs | |
| 09:00:47 | sean-k-mooney | we group VFs by pf in nova today | |
| 09:01:20 | bogdando[m] | I see, that makes sense to not break the contracts established like each deployable is a placement rp | |
| 09:01:43 | sean-k-mooney | there are 3 related thigns | |
| 09:01:54 | sean-k-mooney | devices deployables and attachmet handels | |
| 09:02:08 | sean-k-mooney | there is a 1:1 relathiship between devices adn attachment handels | |
| 09:02:24 | sean-k-mooney | and a many:1 relasthip between device and deployable | |
| 09:02:37 | sean-k-mooney | or attachmetn_handel and deployabel depending on how you look at it | |
| 09:03:11 | bogdando[m] | hm, my understanding was 1:many for devices and deployables and 1:many for deployables and ah | |
| 09:03:23 | sean-k-mooney | no | |
| 09:03:47 | bogdando[m] | it is based on https://specs.openstack.org/openstack/cyborg-specs/specs/train/implemented/cyborg-nova-placement.html#background | |
| 09:04:33 | bogdando[m] | "A Cyborg device has one or more components named deployables, each of which contains one or more accelerators" | |
| 09:05:00 | bogdando[m] | so accelerators to handles is 1:1, right | |
| 09:05:56 | bogdando[m] | everyting else seems to be 1:many :) | |
| 09:06:25 | sean-k-mooney | so in the db they are not corratled at all directly | |
| 09:06:39 | bogdando[m] | if there is some hidden knowledge based on placement desing, do we have a better place to read about it? | |
| 09:07:08 | sean-k-mooney | not really and the curen implation is both incodsitend and not scalable | |
| 09:07:23 | sean-k-mooney | you really need to look at the code and already know hwo placment was designe to be used | |
| 09:07:32 | bogdando[m] | I see, that comlpicates design review on shaky grounds | |
| 09:08:13 | sean-k-mooney | so the actuall relation ship is as follow | |
| 09:08:56 | sean-k-mooney | device <> contoplathID is 1:1 contolpath id <> attachment handel is 1:1 | |
| 09:09:18 | sean-k-mooney | attachment handel <> deployable is many to 1 | |
| 09:09:49 | sean-k-mooney | the attachment handel is actullth theing that maps contoplath ides to deployables | |
| 09:09:51 | bogdando[m] | would be nice to reflect that in dev docs | |
| 09:09:55 | sean-k-mooney | and devices are assocated transitivly | |
| 09:10:05 | sean-k-mooney | bogdando[m]: its on my todo list | |
| 09:10:21 | bogdando[m] | ack, thanks for explaining! | |
| 09:10:30 | sean-k-mooney | but you can see this by looking at the db schema today https://github.com/openstack/cyborg/blob/master/cyborg/db/sqlalchemy/models.py#L78-L210 | |
| 09:11:19 | sean-k-mooney | the problem iwth the atibutes api is it shoudl have been again the device or optionaly the device or deployable | |
| 09:11:25 | bogdando[m] | looking into code may confuse reader because old implementation paths not strictly following old design specs | |
| 09:11:39 | sean-k-mooney | this is consitent with the spec | |
| 09:11:52 | sean-k-mooney | but you need more context then is captured there | |
| 09:12:10 | sean-k-mooney | or rather ti coudl have done with better diagrams to make it very clear | |
| 09:12:47 | sean-k-mooney | https://specs.openstack.org/openstack/cyborg-specs/specs/train/implemented/cyborg-nova-placement.html#background does not show it clearly | |
| 09:12:52 | bogdando[m] | that comment about code was mostly for "to be clear i dont think that hte atibute api as orgianly specified is the right shap and the impelation that was done didnt follow the spec and is even worse" | |
| 09:13:21 | sean-k-mooney | ah well yes the implementation didnt follwo the spec | |
| 09:13:32 | sean-k-mooney | the spec was reasonabel the implelatin less so | |
| 09:14:56 | sean-k-mooney | i was effectivly planing to remvoe the top level /atibute api because that was neer approved and doesnt really work well | |
| 09:15:25 | sean-k-mooney | https://specs.openstack.org/openstack/cyborg-specs/specs/2023.2/implemented/attribute-api-support.html#rest-api-impact | |
| 09:15:37 | sean-k-mooney | the speci specificly said `/v2/deployable/{uuid}/attribute` | |
| 09:15:53 | sean-k-mooney | we shoudl have atibute son both deployabels and devices in my opion | |
| 09:16:07 | sean-k-mooney | but the impletion added /v2/attributes | |
| 09:16:21 | sean-k-mooney | https://docs.openstack.org/api-ref/accelerator/#list-attributes | |
| 09:16:49 | sean-k-mooney | the impltion also does not work properly and was never properly completed | |
| 09:17:47 | bogdando[m] | attach handles to deployables relation that you explained seems strictly following the original design link I provided above, and matches 1 to many / many to one - depending on which goes first. The difference seems to be more for controlpaths, devices and accelearators | |
| 09:17:54 | sean-k-mooney | my prefence is to add /v2/deployable/{uuid}/attribute and /v2/device/{uuid}/attribute and delete /attibute in a new microverison | |
| 09:19:06 | sean-k-mooney | not quite i think that is still following the sepc but again i was involved in the spec design and over all nova/placment design in person | |
| 09:19:14 | sean-k-mooney | so i have more context then may have been capatured | |
| 09:22:15 | bogdando[m] | back to the subject, it seems we might want to postpone the spec which suggests using phys_net in attach handles - until we have new attributes design implemented? | |
| 09:22:38 | bogdando[m] | to avoid double migration paths | |
| 09:24:44 | sean-k-mooney | no | |
| 09:24:58 | sean-k-mooney | the phsnet is already in the attachment handel | |
| 09:25:08 | bogdando[m] | ah | |
| 09:25:24 | sean-k-mooney | what that spec is doign is propsong we port the logic out of the intel only nic driver | |
| 09:25:31 | sean-k-mooney | into the generic pci driver | |
| 09:25:38 | sean-k-mooney | so it wokrs with any nic | |
| 09:26:28 | bogdando[m] | I see. Well my concern remains - phys net likely an attribute and needs another implementation | |
| 09:27:12 | sean-k-mooney | it may be at some point | |
| 09:27:18 | sean-k-mooney | but it shoudl be set via config | |
| 09:27:35 | sean-k-mooney | as the priamry way to configure it | |
| 09:27:56 | sean-k-mooney | nova never uses the atibute api and never will | |
| 09:28:13 | sean-k-mooney | it uses only the arq api and device profile apis | |
| 09:28:41 | sean-k-mooney | the attachment handel is the summeraisation of the information required to attach a device to a workload (vm or container) | |
| 09:29:16 | sean-k-mooney | bogdando[m]: you proably should read https://specs.openstack.org/openstack/nova-specs/specs/xena/implemented/sriov-smartnic-support.html | |
| 09:30:27 | bogdando[m] | also it looks like having phys net in ah of legacy intel driver is OKish, while having a fresh started geneeric pci driver design clean | |
| 09:30:32 | sean-k-mooney | when we implmented suprpot for cyborg managed neutron ports in nova in xena we added code in nova that checks teh phsynet | |
| 09:31:04 | sean-k-mooney | bogdando[m]: the attachmetn handle is the corect place for it | |
| 09:31:16 | sean-k-mooney | bogdando[m]: nova should never use the attibutes api ever | |
| 09:32:52 | bogdando[m] | that is a nice reference to read, thank you. I understood that there is another established contract for nova to cyborg and neutron interaction | |
| 09:33:53 | bogdando[m] | albeit do you think that adding network specific attributes, then for example gpu specific ones, and maybe later one more for nvmes or fpgas, still keeps that driver attach handle generic? | |
| 09:35:24 | bogdando[m] | am not trying to make a revolution here though :) | |
| 09:35:41 | sean-k-mooney | first of all yes, but secondly the only things that will be specifid via that interface are thigns that change how the device is assocated with the guest | |
| 09:36:27 | sean-k-mooney | bogdando[m]: currently your propsoal are misaligned with the driction i woudl liek to take the proejct. im open ot dicussion but i do not want to have nova need to know more about cybrog then it already does in general | |
| 09:36:44 | sean-k-mooney | nov ainteract with only 2 api, the device profile api to get the resouce request | |
| 09:36:57 | sean-k-mooney | and the the acclerator request api to bind allcoated devices | |
| 09:37:03 | bogdando[m] | I am ok with that just maybe renaming generic driver to something else then | |
| 09:37:12 | sean-k-mooney | the attchmeth handel is part of the repocne of the arq bind | |
| 09:37:20 | bogdando[m] | like universal driver | |
| 09:37:48 | sean-k-mooney | the term generic is chooen specifly to be vendor agnistic | |
| 09:37:56 | sean-k-mooney | we have a pci driver, and are addning an nvme dirver | |
| 09:38:08 | sean-k-mooney | they arem named after the bus tehy manage | |