| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-05-21 | |||
| 14:34:21 | gibi | https://bugs.launchpad.net/nova/+bug/2102038 | |
| 14:35:36 | gibi | it is definitely does not work with pci in placement. Without it it is accepted but based on Sean this never really fully worked there either | |
| 14:36:13 | bauzas | gibi: dansmith: in a current meeting but looking | |
| 14:36:14 | dansmith | hmm, okay | |
| 14:36:35 | dansmith | gibi: I thought you could also specify devices in alias by address, but maybe that's not right | |
| 14:36:35 | gibi | https://review.opendev.org/c/openstack/nova/+/944062 see the comments from sean-k-mooney | |
| 14:36:57 | gibi | dansmith: I don't have the full picture of this without PCI in Placement | |
| 14:37:07 | gibi | with PCI in Placement it is not trivial to support | |
| 14:37:11 | bauzas | are we talking of PCI aliases ? | |
| 14:37:17 | dansmith | okay I don't know if I agree with sean's comments about it never being intended to be supported | |
| 14:37:33 | sean-k-mooney | dansmith: no the alilas doe not have an adress | |
| 14:37:33 | gibi | as we cannot say OR between resouce classes in allocationa candidate query | |
| 14:37:48 | sean-k-mooney | dansmith:the devspec does but the alaise does not | |
| 14:37:59 | dansmith | I obviously wasn't closely involved with the early pci stuff, but I remember a conversation at a design summit about addresses specifically so you didn't have to commit all of one vendor/product | |
| 14:38:36 | sean-k-mooney | dansmith: that was rejected because the alsi is ment to eb the same on all hosts | |
| 14:38:49 | sean-k-mooney | and the adress woudl vary | |
| 14:39:08 | sean-k-mooney | you dont need to commit all of one vendor id and product id | |
| 14:39:14 | sean-k-mooney | you do the filtering in the dev spec | |
| 14:39:36 | sean-k-mooney | the alias is an abstraction like the resouce classs in placement | |
| 14:40:00 | bauzas | I think you can even star the devices | |
| 14:40:04 | bauzas | for the alias | |
| 14:40:38 | dansmith | okay digging back through old release config docs, I guess I'm thinking of the whitelist | |
| 14:41:05 | sean-k-mooney | ya the whitelists is now the devspec | |
| 14:41:10 | dansmith | it seems really unfortunate to not be able to construct a flavor that gets one of a set of multiple identical-enough devices | |
| 14:41:20 | bauzas | dansmith: it should | |
| 14:41:35 | sean-k-mooney | dansmith: you can | |
| 14:41:37 | bauzas | I don't get what we can't do | |
| 14:41:52 | bauzas | there are two different things | |
| 14:41:56 | bauzas | sec | |
| 14:42:17 | dansmith | oh, multiple aliases in the flavor with commas? | |
| 14:42:22 | sean-k-mooney | yes | |
| 14:42:23 | dansmith | and that will do one-of? | |
| 14:42:28 | dansmith | okay, gotcha | |
| 14:42:28 | sean-k-mooney | no | |
| 14:42:29 | bauzas | https://docs.openstack.org/nova/latest/configuration/config.html#pci.alias | |
| 14:42:47 | sean-k-mooney | in the flavor if you use commas that is asking for multipel devices | |
| 14:42:52 | sean-k-mooney | but not one of | |
| 14:42:55 | bauzas | in the alias, you can request PCI devices by vendor/product IDs | |
| 14:42:58 | dansmith | okay, so how do I do one of? | |
| 14:43:03 | sean-k-mooney | you cant | |
| 14:43:10 | sean-k-mooney | not with out breaking other things | |
| 14:43:18 | sean-k-mooney | well | |
| 14:43:30 | sean-k-mooney | ok so you can do one of via resouce classes an pci in placement | |
| 14:43:30 | bauzas | again, I don't see the problem | |
| 14:43:39 | dansmith | okay, so it _is_ a limitation that we can't do what I asserted above: a flavor that selects one of a set of identical-enough hardware | |
| 14:44:08 | sean-k-mooney | dansmith: identical enough for live migration means they device must be exactly the same | |
| 14:44:09 | bauzas | but we can say 'nvme256:1' in the flavor, right? | |
| 14:44:15 | sean-k-mooney | potentlaly down to the firmware level | |
| 14:44:55 | dansmith | sean-k-mooney: for things that care about live migration, but (a) I have two devices that are the same vendor/product but different firmwares *and* one supports crypto and the other does not, | |
| 14:45:10 | dansmith | so I'm sure qemu has to do more is-this-the-same-for-real checking if we expect that to work robustly | |
| 14:45:18 | bauzas | provided that alias would match different hardware based on the same alias combination (either the vendor/product id or a star, IIRC) | |
| 14:45:23 | sean-k-mooney | dansmith: right so you can requiest trait in the alias | |
| 14:45:36 | bauzas | but that requires PCI in placvement | |
| 14:45:48 | dansmith | bauzas: I only care about pci in placement :) | |
| 14:45:58 | dansmith | sean-k-mooney: I don't see that in the docs, can you link me? | |
| 14:46:03 | bauzas | ah, then provide device_spec with traits | |
| 14:46:13 | bauzas | and then request that trait thru the alias | |
| 14:46:17 | bauzas | https://docs.openstack.org/nova/latest/configuration/config.html#pci.device_spec | |
| 14:46:24 | bauzas | https://docs.openstack.org/nova/latest/configuration/config.html#pci.alias | |
| 14:46:40 | dansmith | ... I don't see in the docs how to request a trait with the alias | |
| 14:46:44 | sean-k-mooney | dansmith: https://github.com/openstack/nova/blob/master/nova/pci/request.py#L110-L115 | |
| 14:46:55 | dansmith | them's not docs :) | |
| 14:47:04 | sean-k-mooney | dansmith: it may or may not be but it was added with the pci in placement feature | |
| 14:47:05 | bauzas | dansmith: there is a traits dict key that you can add | |
| 14:47:08 | sean-k-mooney | so it shoudl be in the spec | |
| 14:47:14 | bauzas | traits | |
| 14:47:14 | bauzas | An optional comma separated list of Placement trait names requested to be present on the resource provider that fulfills this alias. Each trait can be a standard trait from os-traits lib or it can be an arbitrary string. If it is a non-standard trait then Nova will normalize the trait name by making it upper case, replacing any consecutive | |
| 14:47:14 | bauzas | character outside of [A-Z0-9_] with a single ‘_’, and prefixing the name with CUSTOM_ if not yet prefixed. The maximum allowed length of a trait name is 255 character including the prefix. Every trait in traits requested in the alias ensured to be in the list of traits provided in the traits field of the [pci]device_spec when scheduling the | |
| 14:47:14 | bauzas | request. This field can only be used only if [filter_scheduler]pci_in_placement is enabled. | |
| 14:47:17 | dansmith | https://docs.openstack.org/nova/latest/configuration/extra-specs.html#pci_passthrough:alias | |
| 14:47:32 | sean-k-mooney | that is the wrong doc | |
| 14:47:41 | bauzas | you don't specify the trait explicitely | |
| 14:47:49 | bauzas | I mean in the flavor | |
| 14:47:52 | dansmith | ohhh, you mean an alias in _config_ can be a trait instead of a vendor/product? | |
| 14:48:03 | bauzas | the flavor only requests an amount of that alias | |
| 14:48:18 | bauzas | and that alias is defined by having a traits key in it | |
| 14:48:28 | bauzas | dansmith: yes | |
| 14:48:52 | bauzas | and device_spec allows you to tag with traits some PCI devices automatically | |
| 14:48:54 | sean-k-mooney | dansmith: correct you can use placement resouce classes + traits instead of vendor and product id | |
| 14:49:01 | bauzas | just add traits to the device_spec dict in question | |
| 14:49:16 | dansmith | I see, I thought we were talking about the alias in the request | |
| 14:49:29 | sean-k-mooney | in the flavor no | |
| 14:49:31 | bauzas | that's transparent to the user or to the API admin | |
| 14:49:36 | dansmith | so the doc for alias needs a trait example I guess | |
| 14:49:40 | sean-k-mooney | this is all intentioally hiden form the flavor | |
| 14:49:50 | bauzas | dansmith: we have the conf docs | |
| 14:49:54 | sean-k-mooney | dansmith: ya https://docs.openstack.org/nova/latest/admin/pci-passthrough.html#pci-tracking-in-placement is where i was expectin git | |
| 14:50:06 | dansmith | sean-k-mooney: I understand, and that's how I expected it to work, I just didn't see the path to that alias requesting one of multiple things | |
| 14:50:07 | bauzas | but indeed the examples don't show< it | |
| 14:50:15 | sean-k-mooney | or maybe here https://docs.openstack.org/nova/latest/admin/pci-passthrough.html#configuring-pci-aliases-for-users | |
| 14:50:24 | dansmith | we need one here: https://docs.openstack.org/nova/latest/configuration/config.html#pci.alias | |
| 14:50:45 | dansmith | because that says you can use traits, but doesn't give me an example to see where that is the only thing used | |
| 14:50:53 | gibi | I can add that doc if that helps getting reviews on the eventlet series :) | |
| 14:50:57 | bauzas | dansmith: at least that's documented below in the list of accepted keys | |
| 14:50:57 | dansmith | I'll try this locally and then document if it works | |
| 14:51:04 | bauzas | cool | |
| 14:51:16 | dansmith | bauzas: I understand (now), but obviously not obvious to me :) | |
| 14:51:23 | bauzas | dansmith: we tested that with Uggla when he was working on vfio-pci | |
| 14:51:26 | gibi | (I assume it works there are functional test coverage on it) | |
| 14:51:26 | bauzas | should work | |