Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-05-21
14:51:43 bauzas Uggla can give you some examples
14:51:43 sean-k-mooney ya so in the devspec you can advertise them https://specs.openstack.org/openstack/nova-specs/specs/zed/approved/pci-device-tracking-in-placement.html#pci-device-spec-configuration and in the aliss you can requet them but we dont have that in the doc today
14:51:45 dansmith gibi: sorry, I'm not farting around for no reason, I'm trying to get some other important stuff tested and this is in my way.. I promise I'm not ignoring your series for unimportant reasons
14:51:45 sean-k-mooney https://specs.openstack.org/openstack/nova-specs/specs/zed/approved/pci-device-tracking-in-placement.html#pci-alias-configuration
14:52:00 gibi ack
14:52:06 dansmith sean-k-mooney: yeah I think I understand the glue path now
14:52:08 bauzas gibi: I'd be more than happy to give a shot to your series
14:52:21 bauzas provided my meeting of meetings is done
14:53:00 sean-k-mooney so in https://docs.openstack.org/nova/latest/configuration/config.html#pci.alias
14:53:11 sean-k-mooney we do mention traits but only provide the eexample of a resouce class
14:53:39 dansmith sean-k-mooney: right
14:53:57 dansmith which is really just vendor/model (by default) so it doesn't look helpful to someone looking for it
14:57:25 bauzas we need one more example, I agree
14:57:34 bauzas without the product/vendor ID need
14:57:59 bauzas but IIRC, we still need to somehow define them, right Uggla ?
14:59:31 Uggla Give me a sec, need to reload the context.
15:00:26 bauzas long story short, can we skip the vendor/product ID settings in the pci alias ? I think we had to do something like providing a star (*)
15:01:26 dansmith bauzas: star does not help me here
15:01:36 dansmith but as I said, I will test this example shortly
15:01:45 bauzas dansmith: I guess I understood your case
15:02:00 bauzas you wanna request a group of arbitrary pci devices by traits
15:02:37 bauzas that's why I'm saying you shouldn't provide vendor and product IDs in the alias definition or you would restrict that list
15:03:00 gibi yeah so if you have two different devices with different product id then you can whitelist them via address (with full or partial match on the address) and connect them the the same resource class in the whitelist, you can add traits there as well. Then you can request that resource class in the alias
15:03:06 bauzas but IIRC, when we wanted to do that thing, we found some limitation that we fixed somehow I can't recall (maybe a star or something else)
15:03:38 Uggla Yes I think in the alias section we can skip product / vendor and only use ressource class or trait.
15:03:39 gibi the key is that you can use the same RC in multiple device_spec lines
15:05:49 Uggla Yep I updated the documentation for the alias section to highlight that product_id / vendor_id are optional.
15:07:14 dansmith I don't see that
15:07:34 dansmith ...that they are optional.. I do see the class example
15:08:02 Uggla dansmith, https://docs.openstack.org/nova/latest/configuration/config.html --> alias --> A JSON dictionary which describe a PCI device.
15:08:18 dansmith gibi: traits would be better for me that RC.. if they use the same RC, then I can't do inventory planning based on RC, and also, I might want to still request via RC for specific devices, or traits for generic "just give me a 256G nvme"
15:08:20 gibi https://github.com/openstack/nova/blob/221a3e89e8988bc664298106ee691a4e41ca71f9/nova/tests/functional/libvirt/test_pci_in_placement.py#L1759-L1772
15:08:57 dansmith Uggla: yeah I said I see the example, just not that vendor/product is optional (as it says for traits and rc)
15:09:04 gibi dansmith: if you don't define the RC then the RC will be based on the product_id so you cannot have a single alias requesting it as a single alias cannot request two different RC
15:09:27 dansmith gibi: ... but I can request just a trait no?
15:10:14 Uggla dansmith, --> Note that [...] indicates optional field.
15:10:16 gibi you can request a set of traits
15:10:36 gibi but you need to eithe provide an RC or a product/vendor id in the alias
15:10:49 gibi otherwise we cannot createa placement allocation candidate query
15:11:56 dansmith Uggla: fine, but very not as obvious as: "resource_class: The **optional** Placement resource class name"
15:12:21 dansmith gibi: okay if that's the case then that's definitely not very well explained in the docs here
15:12:33 dansmith lemme try with just the trait so I can see what happens
15:12:36 gibi lets improve the doc
15:13:18 gibi if you provides just traits in the alias and no RC or vendor/product id then you will get that the alias is invalid
15:13:27 gibi (I hope : )
15:15:07 dansmith I do not, on service startup at least
15:15:13 Uggla dansmith, I agree it could be better.
15:18:52 gibi dansmith: it is validated when the alias is used
15:19:12 gibi it was true for before PCI in Placement time
15:19:43 dansmith gibi: so, I might be missing something but it looks to me like I was able to create a server and it just silently ignored my pci request
15:20:30 dansmith https://paste.opendev.org/show/bkADIKJZwFGSAv3UcXCF/
15:21:38 gibi alias = {"traits": "CUSTOM_NVME256G", "device_type":"type-PCI", "name": "nvme256g"}
15:21:52 gibi this should have been rejected at server create. if not that is a bug
15:22:44 gibi it cannot work as we cannot generate a proper allocation candidate query as the resource class is mandatory there
15:22:46 dansmith you see it's running and you can confirm the embedded flavor attempted to request it right?
15:22:51 gibi yes I see
15:22:56 gibi so it is a bug
15:24:10 gibi dansmith: do you use the same nova-pci.conf for both compute and api?
15:24:39 dansmith both computes, api and scheduler yeah
15:24:45 dansmith that's why it's on mnt :)
15:25:08 gibi ack, I tend to make that mistake that I update the alias in the nova-compute conf and not in the nova-api conf
15:25:26 dansmith yep, tried to set myself up to avoid that this time
15:26:03 gibi (the whole above discussion shows that this is the first time we really started using the PCI in Placement feature so we are finding the real bugs now)
15:26:06 dansmith this is a just-rebuilt setup so let me test with the other aliases to make sure pci is working in general
15:28:16 dansmith hmm, no actually
15:28:55 dansmith I wonder if my pci filter is not getting set, hmm
15:29:33 dansmith ah, it's not
15:29:42 dansmith so that's probably why it worked before too, hrm
15:33:38 dansmith gibi: so, I had [scheduler] instead of [filter_scheduler]
15:33:40 Uggla dansmith, not sure but my pci_in_placement is in the filter_scheduler section.
15:33:51 Uggla :)
15:33:52 dansmith but, still works.. what all reads that? more than n-sch?
15:34:04 dansmith conductor maybe?
15:36:43 gibi if pci_in_placement is read by the api
15:36:49 gibi s/if//
15:37:04 gibi repot_in_placement is read by the compute
15:37:49 gibi (yes it is not optimal I know)
15:38:40 dansmith okay api is seeing the new value, still not stopping me
15:39:18 gibi I can try to reproduce the issue on my side...
15:39:41 bauzas what's the issue ?
15:40:13 dansmith but, even using a good alias isn't getting me the device or a failure, so something else must be broken. here
15:41:47 Uggla bauzas, my understanding is if you only specify an trait in the alias section, pci in placement should block that as it needs RC too. And it is not the case.
15:42:09 bauzas yeah, that could be the case
15:42:23 bauzas dansmith doesn't see anything in placement ?
15:42:33 bauzas about the PCI RPs
15:42:49 bauzas (just rebooted due to a wifi crash, <3 F42)
15:47:05 dansmith gibi: okay I had a typo in my flavor property so it was just being ignored. but, fixing that, here's what I see as a user:
15:47:06 dansmith https://paste.opendev.org/show/bI7lakvj0atefW6KPNkw/
15:47:26 dansmith the log does have something in it, but it's not quite obvious I suspect
15:47:50 dansmith I suspect no operator will know what "resources={CUSTOM_PCI_NONE_NONE=1}" means
15:49:36 gibi yepp we need to add an explicit validation
15:49:50 gibi that either RC or product/vendor needs to be provided in the alias
15:50:13 dansmith yeah
15:51:07 gibi I can file a bug and link it to the downstream PCI in Placement Jira as well so we will have time for it to fix
15:51:08 dansmith gibi: so ... why again can we not support just the trait? You can ask for all providers with that trait, no?
15:51:30 gibi you cannot have an allocation candidate without telling placement what resource you are allocating
15:51:51 dansmith because you're asking for inventory not a provider?
15:52:09 dansmith (and providers have the traits)
15:52:40 gibi I'm asking for things I can allocate, I cannot allocate a trait or a provider I can only allocate a piece of inventory
15:53:06 dansmith right, that's what I mean

Earlier   Later