| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-08-25 | |||
| 14:23:51 | efried | https://review.openstack.org/#/c/496434/ | |
| 14:24:01 | claudiub | stephenfin: we are. | |
| 14:24:55 | claudiub | stephenfin: we're currently only reporting devices which have the domain:bus:slot.func format | |
| 14:25:05 | efried | ...with e.g. [pci]alias = {"name": "USB", "product_id": "8241", "vendor_id": "104c", "device_type": "type-PCI"} and [pci]passthrough_whitelist = {"product_id": "8241", "vendor_id": "104c"} | |
| 14:26:08 | efried | claudiub Those devices show up on the compute node? | |
| 14:26:21 | claudiub | stephenfin: but I've mainly whitelisted devices using product_id / vendor_id | |
| 14:26:42 | claudiub | efried: if they're are passthrough-able, yes. | |
| 14:26:56 | claudiub | efried: they have to be prepared for passthrough first | |
| 14:27:06 | efried | claudiub How? | |
| 14:27:38 | claudiub | efried: https://github.com/openstack/nova/blob/master/releasenotes/notes/hyper-v-pci-passthrough-babf104d6bc2baa6.yaml | |
| 14:28:41 | jaypipes | efried: reviewed. | |
| 14:29:02 | efried | Oh, thanks jaypipes :) | |
| 14:29:22 | claudiub | stephenfin: anyways. it would be nice to be also be able to whitelist devices using other ways than product_id / vendor_id, or domain:bus:slot.func. | |
| 14:29:59 | efried | jaypipes Nice. None of that was worthy of a -1? Really? | |
| 14:30:10 | stephenfin | claudiub: What kind of IDs would you expect, e.g. how else can devices be identified in Hyper-V? | |
| 14:30:45 | jaypipes | efried: :) | |
| 14:30:59 | jaypipes | stephenfin: by green card. | |
| 14:31:00 | efried | jaypipes I really didn't expect you to review it, but I would like to point out the spoof_pci_address method (https://review.openstack.org/#/c/496434/3/nova_powervm/virt/powervm/vm.py@884) | |
| 14:31:21 | claudiub | stephenfin: for example, I've had this issue when working on the sr-iov support. apparenty, different NIC vendors have different PCI device ID formats. for example, Intel NICs contain the vendor_id and product_id, which can be extracted and reported. But other NICs, like Mellanox or Chelsio, do not. | |
| 14:31:37 | claudiub | stephenfin: a simple device_id would do, IMO. | |
| 14:31:53 | jaypipes | efried: oh, trust me, I saw it :) | |
| 14:33:22 | efried | claudiub Right, so I would like my [pci]passthrough_whitelist entry to identify devices in whatever way my compute deems appropriate, and have the logic for doing that whitelist filtering live in my compute driver. | |
| 14:36:46 | claudiub | efried: well, the compute driver is already reporting "all" the PCI devices it sees, right? after which the PCI resource tracker filters those compute driver reported PCI devices according to the configured whitelist. is there anything that doesn't fit your usecase? | |
| 14:37:33 | efried | claudiub Yeah: the ability to specify individual devices in the whitelist (vs. just vendor/prod ID classes) | |
| 14:37:51 | efried | Because as soon as I start chucking addresses around, nova tries to get at them under /sys/bus/... | |
| 14:38:13 | stephenfin | efried, jaypipes: Would we still have per-compute-node filtering in the resource provider world? | |
| 14:38:32 | claudiub | efried: ah yes, i see. that would make sense, IMO. the pci resource tracker should be updated to allow other filters to be configured, imo. | |
| 14:39:19 | efried | stephenfin So the way I would think it would work in a RP world is... | |
| 14:39:46 | efried | Your RP (in this case the compute node) inventories the devices it has available, *already* filtered by whitelist. | |
| 14:40:25 | cdent | meaning filtering as a descendant of get_inventory? | |
| 14:40:36 | jaypipes | stephenfin: are you asking whether there'd be a need for a whitelist conf option when we manage PCI devices in placement? | |
| 14:40:48 | stephenfin | jaypipes: Yup | |
| 14:41:01 | efried | cdent Within get_inventory, yeah. | |
| 14:41:08 | cdent | cool, good | |
| 14:41:14 | jaypipes | stephenfin: yes, there would still be a need for a way to filter out host devices that should not be allocatable to a consumer. | |
| 14:42:00 | jaypipes | stephenfin: I just want the pci_passthrough_whitelist CONF option to do a single thing. I'd like to see a separate CONF option (or YAML file even) that contains device inventory information that cannot be auto-discovered. | |
| 14:42:05 | claudiub | jaypipes: isn't that already done at the api level? | |
| 14:42:14 | jaypipes | claudiub: hmm? | |
| 14:42:21 | jaypipes | claudiub: not sure I understand you | |
| 14:42:22 | claudiub | with pci device alias config opion | |
| 14:42:55 | claudiub | afaik, you request a pci device through an alias. if a pci device is not aliased, i don't think it's allocatable | |
| 14:43:11 | efried | Correct. passthrough_whitelist is intersected with alias to get the list of claimable devs. | |
| 14:43:12 | jaypipes | claudiub: well, that's yet another thing that the pci_passthrough_whitelist CONF option does :( alias things... | |
| 14:43:32 | efried | jaypipes Well, [pci]alias aliases things. | |
| 14:43:44 | efried | AFAICT that's in place to make it easier to specify them in the flavor | |
| 14:43:51 | efried | extra_specs pci_passthrough:alias=<alias>:<count> | |
| 14:44:10 | jaypipes | there's still alias stuff in the whitelist option. | |
| 14:44:17 | efried | ? | |
| 14:44:35 | efried | no lo veo | |
| 14:44:37 | claudiub | so, then, you can still request PCI devices through other means, like vendor_id / product_id, or other fields like this? i didn't see this documented. | |
| 14:44:58 | claudiub | not counting sr-iov. | |
| 14:45:19 | claudiub | that's requested whenever a port is vnic_type direct or something similar | |
| 14:45:34 | efried | oh, that. | |
| 14:45:54 | claudiub | that's requested automatically * | |
| 14:45:56 | efried | That's done based on the physical_network tag in the passthrough_whitelist, I believe. Aliases are not involved at all in that flow. | |
| 14:46:11 | claudiub | efried: yeah | |
| 14:46:12 | efried | which is... goofy. | |
| 14:46:38 | efried | Hey, it's Friday. | |
| 14:47:19 | fried_rice | Yeah, handling SR-IOV is a separate can of worms. | |
| 14:47:48 | fried_rice | And btw, I think SR-IOV is where a lot of the /sys/bus business comes into play. Or rather, why it's there in the first place. | |
| 14:47:55 | claudiub | oh yeah. which reminds me. i didn't find a proper way to "associate" different SR-IOV NICs to different "physical_networks" | |
| 14:48:28 | fried_rice | claudiub How so? | |
| 14:48:44 | fried_rice | oh, the thing where you can only actually have one physical network | |
| 14:49:20 | fried_rice | Yeah, there's a limitation in the binding metadata in neutron. We tried to "fix" it in I think ocata, but armax shot us down. | |
| 14:49:38 | armax | fried_rice: what did I do? | |
| 14:49:49 | fried_rice | armax :) Hold on, let me see if I can dig it up. | |
| 14:50:24 | fried_rice | armax I think it was this one: https://review.openstack.org/#/c/358125/ | |
| 14:50:25 | armax | I don’t typically shoot people down without proper justification :) | |
| 14:50:58 | fried_rice | https://bugs.launchpad.net/neutron/+bug/1615128 | |
| 14:50:59 | openstack | Launchpad bug 1615128 in neutron "Custom binding:profile values not coming through" [Undecided,Invalid] - Assigned to Eric Fried (efried) | |
| 14:51:04 | fried_rice | Oh, you provided justification | |
| 14:51:08 | fried_rice | And we backed off accordingly. | |
| 14:51:37 | jaypipes | fried_rice: apologies, I was wrong on the pci_passthrough + alias thing. :( | |
| 14:51:54 | armax | fried_rice: right, I remember this now…I suppose the next step from the -2 was to follow up with an RFE that never came | |
| 14:52:05 | armax | iirc | |
| 14:52:25 | fried_rice | armax That sounds right. | |
| 14:52:44 | fried_rice | armax Wasn't trying to bust your chops :) | |
| 14:52:53 | armax | fried_rice: so I only injured you, not shot you down mortally :) | |
| 14:53:28 | armax | fried_rice: not a problem, I make mistakes of judment all the time, I was just trying to figure out the context | |
| 14:53:35 | claudiub | fried_rice: hm, simply because the limitation of the passthrough_whitelist config option. I can't reliably get a vendor_id / product_id for the NICs I have, and I can't use the address field either. IMO, if device_id would be allowed, it would work. | |
| 14:53:56 | fried_rice | claudiub Well | |
| 14:53:57 | claudiub | hm, haven't tried devname though. | |
| 14:54:08 | fried_rice | claudiub I haven't actually found any reason you can't spoof the vendor/product ID. | |
| 14:54:20 | fried_rice | I don't *think* those are actually checked against anything real, ever. | |
| 14:54:54 | fried_rice | They're just correlated among the passthrough_whitelist (compute conf), the pci_passthrough_devices (compute-provided get_available_resource), and the alias list (api conf). | |
| 14:54:57 | claudiub | the PCI devices reported by the compute drivers have to have those vendor_id / product_ids | |
| 14:55:27 | fried_rice | Yes, have to have them, but I don't think anything cares that those values are really what the dev reports. | |
| 14:55:45 | claudiub | true. | |
| 14:56:09 | fried_rice | I'm... sort of counting on it, actually. Cause I have a vendor ID, but I'm not sure if the thing I'm using as a product ID is really a product ID. | |
| 14:56:24 | fried_rice | https://review.openstack.org/#/c/496434/3/nova_powervm/virt/powervm/host.py@111 | |
| 14:56:59 | claudiub | for sr-iov devices, i'm currently doing an md5, if i can't get the actual vendor_id, product_id, which feels wrong. :) | |
| 14:57:00 | fried_rice | Course, if you do that (or really, whatever you do), you have to document how your user needs to glean the right value to put in those spots in the whitelist/alias. | |
| 14:57:11 | fried_rice | claudiub Totally | |
| 14:57:16 | fried_rice | I feel your pain. | |
| 14:57:43 | fried_rice | claudiub I'm doing something similar with PCI addresses: https://review.openstack.org/#/c/496434/3/nova_powervm/virt/powervm/vm.py@884 | |
| 14:58:05 | claudiub | *ahem*, if only we could also whitelist via device_id, that would work for me. would it work for you as well? | |
| 14:58:25 | fried_rice | claudiub Where device_id can be an arbitrary string, yes, totally. | |
| 14:59:32 | fried_rice | claudiub In order to do wildcarding, though, you would need to outsource the validation/filtering to the compute driver. | |
| 14:59:46 | fried_rice | Cause he's the only guy who knows how to interpret that "arbitrary" format. | |
| 15:00:17 | fried_rice | Otherwise you would just get a straight string match, which means you have to enumerate every device (or continue to use classes via vendor/prod IDs) | |