Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-03-10
14:42:09 sean-k-mooney but its not strictly required
14:42:11 opendevreview Gaudenz Steinlin proposed openstack/nova master: Make SSH compression for remote copy configurable https://review.opendev.org/c/openstack/nova/+/943377
14:44:41 dansmith Okay well, I'll add some notes to the spec for this to cover the above details
15:06:03 opendevreview Sylvain Bauza proposed openstack/nova master: doc: mark the maximum microversion for 2025.1 Epoxy https://review.opendev.org/c/openstack/nova/+/943948
15:10:22 opendevreview Dan Smith proposed openstack/nova-specs master: Add one-time-use-devices spec https://review.opendev.org/c/openstack/nova-specs/+/943486
15:11:37 gibi dansmith: left couple of comments but I think the direction looks good
15:11:49 dansmith gibi: thanks!
15:12:34 gibi the only think I'm a bit hesitant about it is the long term effect of the decision that we go with placement to solve this while and therefore we are forced not to support OTU VFs.
15:13:28 dansmith gibi: tbh, I'm not really sure when VFs would need to use this, as I suspect any virt-knowing device would be sensitive to exposure between uses (perhaps not)
15:14:07 dansmith that said, I think that from the operator point of view, it doesn't necessarily matter that it's happening in placement (other than the steps to unreserve after cleaning I guess).. but the devspec wouldn't need to change if we started accounting for the one-time-use in, say, PCI tracker
15:14:20 gibi yeah that is a valid point. This is why I'm not against going with the placement route, just hesitant that our udnerstanding of SRIOV VFs are correct
15:14:53 gibi dansmith: yeah the unreserve is the problematic step if we go with the PCI tracker route
15:14:57 dansmith right
15:15:01 gibi nova has no API for that
15:15:16 dansmith an SRIOV NIC is safe for use without cleaning, because you can't update firmware I think
15:15:46 dansmith not sure about a VGPU and memory exposure I guess, but .. they have to know that the point of that is sharing, so I would expect it's okay
15:15:52 gibi I also think that, but I never checked what happens with GPU VFs or NVMe VFs
15:16:27 dansmith an SRIOV NVMe would be a problem I guess, but if you're de- and re-allocating them they should be safe (like new namespaces on a regular one)
15:17:03 gibi true, if the VF is mapped to a namespace then it is OK
15:17:06 dansmith yeah
15:17:26 gibi OK. I'm less hesitant now. We have good reasons to believe that this feature is not needed for VFs
15:17:46 gibi a sane VF implementor needs to handle the sharing issue
15:17:51 dansmith I think we just make sure it's clear in the doc that this is only for whole devices because they're not virt-aware, and we can also say "for more comprehensive situations you need cyborg anyway" :)
15:17:55 dansmith yeah
15:18:21 gibi OK
15:18:36 gibi and :) for cyborg as a fallback
15:19:42 dansmith I really don't want to blur the line of nova-vs-cyborg much, so I think keeping this limited to specific uses is helpful to avoid people trying to push nova to do more
15:19:49 opendevreview Sylvain Bauza proposed openstack/nova master: Update compute rpc alias for epoxy https://review.opendev.org/c/openstack/nova/+/943952
15:19:57 sean-k-mooney dansmith: gibi you cant update teh frimware vai the vf from an nic i have worked with
15:20:23 dansmith right
15:20:24 sean-k-mooney dansmith: you can flash ab bitrstream via a specific VF on an fpga that change the behvior of other VFs
15:20:35 sean-k-mooney but that a very different usecasue
15:20:46 sean-k-mooney to me that is not what you are creating this for
15:20:52 dansmith definitely not
15:20:59 sean-k-mooney if we add pci groups in the future
15:21:07 sean-k-mooney we could extend the concpet to the group
15:21:17 sean-k-mooney i.e. a one time group
15:21:52 dansmith tbh, the pci-in-placement approach seems to me to be a bit of a stretch of the model, since placement really assumes resources of a given class are fungible and does not track actual assignment
15:22:23 sean-k-mooney dansmith: well in general tehy are fungible
15:22:24 dansmith so anything specific to a given VF probably needs a new model and new API (which cyborg is well-positioned to handle, IMHO)
15:22:46 dansmith sean-k-mooney: VFs are, but not whole PCI devices
15:23:17 sean-k-mooney maybe you could argue it either way
15:23:21 sean-k-mooney i get where your coming form
15:23:48 dansmith the fact that we have a custom resource type for each device, even if its identical to another, with a total=1 is, I think, the stretch of the model
15:23:59 sean-k-mooney on the cyborg side do we watn to reach out to them during the ptg on any of these topics?
15:24:02 dansmith we're calling two devices that are identical but have different PCI addresses "different" :)
15:24:08 gibi yeah we made the assumption in PCI in Placement that the same RC inventory on an RP means fungible devices
15:24:20 sean-k-mooney dansmith: we are not
15:24:34 sean-k-mooney they have the saem resouce class if they have the same vendor_id and product id
15:24:37 gibi and we opted in to represent a PF as RC=1 while VFs as RC=N
15:25:11 sean-k-mooney the reason they are diffent RPs is because fo VFs
15:25:30 gibi two RPs with the same RC=1 inventory are non fungible due to the RP name encoding the PCI address
15:25:32 dansmith sean-k-mooney: okay, sorry, terminology.. same resource class, but separate providers..
15:25:40 dansmith right ^
15:26:12 sean-k-mooney gibi: yep and that because of other factors like numa or phsynet associativity
15:26:43 gibi yeah exactly numa and physnet granurality is also aligned with that
15:26:43 sean-k-mooney as it stands to day we could have an invetlory of PFs directly off the compute RP
15:27:04 sean-k-mooney the reason we dont was to allow numa and physnets to be modleed in the future
15:27:17 dansmith I guess it's not unlike if we had two storage providers on a host that you can't just arbitrarily sum together.. however, the inventory within those are still fungible, but it's not within a PCI provider, hence the only one inventory
15:27:21 sean-k-mooney we only need 1 RP per device in the VF case today
15:28:46 sean-k-mooney dansmith: we could in some cases coalese multipel PCI invetories into one provider
15:29:18 sean-k-mooney it just seamed harder to have it sometiems on the root provider and sometimes on its own
15:29:25 sean-k-mooney depending on what you set in the devspec
15:29:43 dansmith sean-k-mooney: I'm just saying, I think we're bordering on breaking/abusing the model... it still fits nicely, but only just
15:29:45 sean-k-mooney and what we discoverd (vendor/product id)
15:30:00 dansmith and tbc, I'm not complaining :)
15:30:27 sean-k-mooney dansmith: to b efair we have a design for numa in palcment and neutorn port in placment
15:30:41 sean-k-mooney im also hesitant to proceed with the numa part for similar reasons
15:31:17 sean-k-mooney i think part of the orginal selling point for palcemetn (moveing all or almost all the schduling work to it) didnt really pan out
15:31:41 sean-k-mooney if we are beign pargmatic and say the nova schduler will always be needed to do find grained choices liek numa
15:31:42 dansmith I have no mental space to consider that now, but a numa node with fungible inventory inside is a closer fit to the original goal I think
15:31:51 sean-k-mooney then there is an argrument to say palcment should never know about it
15:32:02 sean-k-mooney it being numa
15:32:25 sean-k-mooney ack lets leave it there for now
15:32:26 dansmith I wonder if we could introduce another data type to placement that represents actual physical allocations where needed
15:32:38 sean-k-mooney maybe
15:32:51 dansmith something that ties an allocation uuid to that other thing that represents an actual thing with an address
15:33:02 sean-k-mooney im happy to disucss that at some point just also dont have the brain power to context switch to this in detail right now
15:33:03 gibi lets add more metadata :)
15:33:16 dansmith memory has no address or serial number, but a pci device does, a disk might
15:33:32 sean-k-mooney memory kind of does
15:33:43 dansmith gibi: it would only be good if we can drop something else in favor, of course :)
15:33:54 sean-k-mooney its at least got a numa node
15:34:08 sean-k-mooney it might not be a unique idetifier across all host
15:34:18 sean-k-mooney but there are ways to talk about fungible pools of it
15:34:23 dansmith sean-k-mooney: if the numa node is a nested provider, then the allocation within has no (further) addressing
15:34:27 dansmith right
15:34:58 dansmith anyway, we're way off in the weeds, didn't mean to start digging a hole
15:35:36 sean-k-mooney https://collection.nam.ac.uk/images/960/204000-204999/204324.jpg
15:36:06 dansmith heh yeah
15:36:11 sean-k-mooney thats not actuly the meme i was loogkign for but it will do
15:36:33 gibi :)
15:50:49 bauzas folks I just saw your discussion, will review again dansmith's spec tomorrow
16:52:15 opendevreview Masahito Muroi proposed openstack/nova master: Use dict object for request_specs_dict in the _list_view https://review.opendev.org/c/openstack/nova/+/939658
16:54:37 masahito sean-k-mooney: gibi: please review this api bug fix again https://review.opendev.org/c/openstack/nova/+/939658 thank you.
18:25:05 gmann sean-k-mooney: do you remember the some missing package for pcre.h in bindep.txt for doc job running on noble. this one https://review.opendev.org/c/openstack/nova/+/935577
18:25:20 sean-k-mooney yes
18:25:30 sean-k-mooney didnt htat go away?
18:25:43 sean-k-mooney or is that still a thing
18:25:45 gmann sean-k-mooney: no, I tired some but still error

Earlier   Later