Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-05-21
14:50:24 dansmith we need one here: https://docs.openstack.org/nova/latest/configuration/config.html#pci.alias
14:50:45 dansmith because that says you can use traits, but doesn't give me an example to see where that is the only thing used
14:50:53 gibi I can add that doc if that helps getting reviews on the eventlet series :)
14:50:57 bauzas dansmith: at least that's documented below in the list of accepted keys
14:50:57 dansmith I'll try this locally and then document if it works
14:51:04 bauzas cool
14:51:16 dansmith bauzas: I understand (now), but obviously not obvious to me :)
14:51:23 bauzas dansmith: we tested that with Uggla when he was working on vfio-pci
14:51:26 gibi (I assume it works there are functional test coverage on it)
14:51:26 bauzas should work
14:51:43 bauzas Uggla can give you some examples
14:51:43 sean-k-mooney ya so in the devspec you can advertise them https://specs.openstack.org/openstack/nova-specs/specs/zed/approved/pci-device-tracking-in-placement.html#pci-device-spec-configuration and in the aliss you can requet them but we dont have that in the doc today
14:51:45 dansmith gibi: sorry, I'm not farting around for no reason, I'm trying to get some other important stuff tested and this is in my way.. I promise I'm not ignoring your series for unimportant reasons
14:51:45 sean-k-mooney https://specs.openstack.org/openstack/nova-specs/specs/zed/approved/pci-device-tracking-in-placement.html#pci-alias-configuration
14:52:00 gibi ack
14:52:06 dansmith sean-k-mooney: yeah I think I understand the glue path now
14:52:08 bauzas gibi: I'd be more than happy to give a shot to your series
14:52:21 bauzas provided my meeting of meetings is done
14:53:00 sean-k-mooney so in https://docs.openstack.org/nova/latest/configuration/config.html#pci.alias
14:53:11 sean-k-mooney we do mention traits but only provide the eexample of a resouce class
14:53:39 dansmith sean-k-mooney: right
14:53:57 dansmith which is really just vendor/model (by default) so it doesn't look helpful to someone looking for it
14:57:25 bauzas we need one more example, I agree
14:57:34 bauzas without the product/vendor ID need
14:57:59 bauzas but IIRC, we still need to somehow define them, right Uggla ?
14:59:31 Uggla Give me a sec, need to reload the context.
15:00:26 bauzas long story short, can we skip the vendor/product ID settings in the pci alias ? I think we had to do something like providing a star (*)
15:01:26 dansmith bauzas: star does not help me here
15:01:36 dansmith but as I said, I will test this example shortly
15:01:45 bauzas dansmith: I guess I understood your case
15:02:00 bauzas you wanna request a group of arbitrary pci devices by traits
15:02:37 bauzas that's why I'm saying you shouldn't provide vendor and product IDs in the alias definition or you would restrict that list
15:03:00 gibi yeah so if you have two different devices with different product id then you can whitelist them via address (with full or partial match on the address) and connect them the the same resource class in the whitelist, you can add traits there as well. Then you can request that resource class in the alias
15:03:06 bauzas but IIRC, when we wanted to do that thing, we found some limitation that we fixed somehow I can't recall (maybe a star or something else)
15:03:38 Uggla Yes I think in the alias section we can skip product / vendor and only use ressource class or trait.
15:03:39 gibi the key is that you can use the same RC in multiple device_spec lines
15:05:49 Uggla Yep I updated the documentation for the alias section to highlight that product_id / vendor_id are optional.
15:07:14 dansmith I don't see that
15:07:34 dansmith ...that they are optional.. I do see the class example
15:08:02 Uggla dansmith, https://docs.openstack.org/nova/latest/configuration/config.html --> alias --> A JSON dictionary which describe a PCI device.
15:08:18 dansmith gibi: traits would be better for me that RC.. if they use the same RC, then I can't do inventory planning based on RC, and also, I might want to still request via RC for specific devices, or traits for generic "just give me a 256G nvme"
15:08:20 gibi https://github.com/openstack/nova/blob/221a3e89e8988bc664298106ee691a4e41ca71f9/nova/tests/functional/libvirt/test_pci_in_placement.py#L1759-L1772
15:08:57 dansmith Uggla: yeah I said I see the example, just not that vendor/product is optional (as it says for traits and rc)
15:09:04 gibi dansmith: if you don't define the RC then the RC will be based on the product_id so you cannot have a single alias requesting it as a single alias cannot request two different RC
15:09:27 dansmith gibi: ... but I can request just a trait no?
15:10:14 Uggla dansmith, --> Note that [...] indicates optional field.
15:10:16 gibi you can request a set of traits
15:10:36 gibi but you need to eithe provide an RC or a product/vendor id in the alias
15:10:49 gibi otherwise we cannot createa placement allocation candidate query
15:11:56 dansmith Uggla: fine, but very not as obvious as: "resource_class: The **optional** Placement resource class name"
15:12:21 dansmith gibi: okay if that's the case then that's definitely not very well explained in the docs here
15:12:33 dansmith lemme try with just the trait so I can see what happens
15:12:36 gibi lets improve the doc
15:13:18 gibi if you provides just traits in the alias and no RC or vendor/product id then you will get that the alias is invalid
15:13:27 gibi (I hope : )
15:15:07 dansmith I do not, on service startup at least
15:15:13 Uggla dansmith, I agree it could be better.
15:18:52 gibi dansmith: it is validated when the alias is used
15:19:12 gibi it was true for before PCI in Placement time
15:19:43 dansmith gibi: so, I might be missing something but it looks to me like I was able to create a server and it just silently ignored my pci request
15:20:30 dansmith https://paste.opendev.org/show/bkADIKJZwFGSAv3UcXCF/
15:21:38 gibi alias = {"traits": "CUSTOM_NVME256G", "device_type":"type-PCI", "name": "nvme256g"}
15:21:52 gibi this should have been rejected at server create. if not that is a bug
15:22:44 gibi it cannot work as we cannot generate a proper allocation candidate query as the resource class is mandatory there
15:22:46 dansmith you see it's running and you can confirm the embedded flavor attempted to request it right?
15:22:51 gibi yes I see
15:22:56 gibi so it is a bug
15:24:10 gibi dansmith: do you use the same nova-pci.conf for both compute and api?
15:24:39 dansmith both computes, api and scheduler yeah
15:24:45 dansmith that's why it's on mnt :)
15:25:08 gibi ack, I tend to make that mistake that I update the alias in the nova-compute conf and not in the nova-api conf
15:25:26 dansmith yep, tried to set myself up to avoid that this time
15:26:03 gibi (the whole above discussion shows that this is the first time we really started using the PCI in Placement feature so we are finding the real bugs now)
15:26:06 dansmith this is a just-rebuilt setup so let me test with the other aliases to make sure pci is working in general
15:28:16 dansmith hmm, no actually
15:28:55 dansmith I wonder if my pci filter is not getting set, hmm
15:29:33 dansmith ah, it's not
15:29:42 dansmith so that's probably why it worked before too, hrm
15:33:38 dansmith gibi: so, I had [scheduler] instead of [filter_scheduler]
15:33:40 Uggla dansmith, not sure but my pci_in_placement is in the filter_scheduler section.
15:33:51 Uggla :)
15:33:52 dansmith but, still works.. what all reads that? more than n-sch?
15:34:04 dansmith conductor maybe?
15:36:43 gibi if pci_in_placement is read by the api
15:36:49 gibi s/if//
15:37:04 gibi repot_in_placement is read by the compute
15:37:49 gibi (yes it is not optimal I know)
15:38:40 dansmith okay api is seeing the new value, still not stopping me
15:39:18 gibi I can try to reproduce the issue on my side...
15:39:41 bauzas what's the issue ?
15:40:13 dansmith but, even using a good alias isn't getting me the device or a failure, so something else must be broken. here
15:41:47 Uggla bauzas, my understanding is if you only specify an trait in the alias section, pci in placement should block that as it needs RC too. And it is not the case.
15:42:09 bauzas yeah, that could be the case
15:42:23 bauzas dansmith doesn't see anything in placement ?
15:42:33 bauzas about the PCI RPs
15:42:49 bauzas (just rebooted due to a wifi crash, <3 F42)
15:47:05 dansmith gibi: okay I had a typo in my flavor property so it was just being ignored. but, fixing that, here's what I see as a user:
15:47:06 dansmith https://paste.opendev.org/show/bI7lakvj0atefW6KPNkw/
15:47:26 dansmith the log does have something in it, but it's not quite obvious I suspect
15:47:50 dansmith I suspect no operator will know what "resources={CUSTOM_PCI_NONE_NONE=1}" means

Earlier   Later