| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-01-19 | |||
| 17:19:51 | bauzas | like I said, I can just drop the doc and leave ops find the way they prefer by themselves | |
| 17:19:58 | sean-k-mooney | so i think it woudl be nice ot have other func test as well but you have enocuh to merge the patch i think | |
| 17:20:02 | sean-k-mooney | so yes its all about the doc | |
| 17:20:14 | bauzas | so, I'll split the patch | |
| 17:20:30 | bauzas | I'll just address the python fix in one change | |
| 17:20:43 | bauzas | and we'll discuss the docs things in a separate change | |
| 17:20:43 | sean-k-mooney | no | |
| 17:20:47 | sean-k-mooney | please keep it in two | |
| 17:20:57 | sean-k-mooney | you can add a thid for the doc if you like | |
| 17:21:04 | bauzas | that's what I'm suggesting | |
| 17:21:07 | bauzas | 1/ functest | |
| 17:21:08 | sean-k-mooney | but im just askign you to swap 20 lins of docs | |
| 17:21:09 | bauzas | 2/ patch | |
| 17:21:11 | bauzas | 3/ doc | |
| 17:21:44 | bauzas | so your concern is about the future and the fact that we persist flavors right? | |
| 17:21:56 | sean-k-mooney | yes | |
| 17:22:02 | bauzas | I'm just trying to understand the concern correctly | |
| 17:22:19 | bauzas | say one day another great feature would provide other nested RPs | |
| 17:22:29 | bauzas | and ops eagerly wanting it | |
| 17:22:42 | sean-k-mooney | yep | |
| 17:22:58 | sean-k-mooney | or endusers | |
| 17:23:01 | bauzas | those new children would be one separate RPs but the GPU ones, right? | |
| 17:23:03 | sean-k-mooney | it might not be the operator | |
| 17:23:12 | sean-k-mooney | that not the issue | |
| 17:23:29 | sean-k-mooney | if we use isolate you cant have 2 request_groups form the same rp | |
| 17:23:41 | sean-k-mooney | so pci in placment for example | |
| 17:23:50 | sean-k-mooney | you wont be abel to have 2 vfs form teh same hsot nic | |
| 17:24:00 | bauzas | I get it | |
| 17:24:11 | bauzas | isolate applies to all resource groups | |
| 17:24:15 | sean-k-mooney | yep | |
| 17:24:26 | bauzas | so the problem is really with prefilters then ? | |
| 17:24:32 | sean-k-mooney | no | |
| 17:24:36 | bauzas | because the flavors wouldn't express those | |
| 17:24:58 | bauzas | pci in placement is a good strawman, let's continue to use it for the example | |
| 17:25:00 | sean-k-mooney | the problem is with the placment api | |
| 17:25:19 | sean-k-mooney | well it also apples to cyborg or neturon prots with resouce requests | |
| 17:25:31 | bauzas | say I have VGPUS and PCI | |
| 17:25:43 | sean-k-mooney | sure | |
| 17:25:47 | bauzas | the problem arises when I want to ask for both, right? | |
| 17:25:54 | sean-k-mooney | yes | |
| 17:26:07 | bauzas | and what if I'm asking for a single resource ? | |
| 17:26:24 | bauzas | (resource *class) | |
| 17:26:33 | sean-k-mooney | yes | |
| 17:27:05 | sean-k-mooney | so pci in placeemnt is a less good exampel because we dont suppot that with neturon port yet. but ill give you an example | |
| 17:27:08 | bauzas | you agree that's not a problem if I'm only asking for VGPU *or* PCI, right ? | |
| 17:27:26 | sean-k-mooney | openstack flavor set vgpu_2 --property "resources1:VGPU=1" \ | |
| 17:27:28 | sean-k-mooney | --property "resources2:VGPU=1" \ | |
| 17:27:30 | sean-k-mooney | --property "group_policy=isolate" | |
| 17:28:07 | sean-k-mooney | if we add --property "hw:pci_ailias=my_vf:2" | |
| 17:28:36 | sean-k-mooney | then if we only have one defivce with my_vf aviaable | |
| 17:28:45 | bauzas | I got it | |
| 17:28:48 | sean-k-mooney | then isolate will break that | |
| 17:28:48 | bauzas | hence my warning | |
| 17:28:50 | bauzas | .. warning:: Be careful when using ``group_policy`` as this policy is global to all the resources request. If you ask for other resources but only VGPUs in your flavor, that ``isolate`` policy will also apply for other | |
| 17:28:52 | bauzas | ResourceProviders. | |
| 17:29:08 | bauzas | that will be written in solid red | |
| 17:29:26 | sean-k-mooney | and it also breadk AggregateInstanceExtraSpecsFilter and computecapaitlies filter | |
| 17:29:33 | bauzas | written too | |
| 17:29:37 | bauzas | .. note:: If you use ``AggregateInstanceExtraSpecsFilter`` filter, you also need to configure your aggregates metadata with the ``group_policy`` values. | |
| 17:29:53 | bauzas | (well, I can add the mention to computecapabilitiesFilter for sure) | |
| 17:30:04 | sean-k-mooney | and neutron QOS prots and cyborg integration if the cyborg request profile has multiple devices | |
| 17:30:16 | bauzas | that's covered by the warning section | |
| 17:30:43 | sean-k-mooney | can you see why an approch that break 5 differnt feature is not something i want to recomemnd people to use as the first option | |
| 17:30:45 | bauzas | I'm ready to bet that a very small fraction of users and ops don't care a bit about any of those stuff | |
| 17:31:20 | sean-k-mooney | sure so we can docuemnt it for them but not the first way we recomemnd | |
| 17:31:40 | bauzas | this isn't breaking the feature, this is a limitation saying "you can't and shouldn't request multiple vgpus per instance if you want to create a flavor that asks for more than vgpus" | |
| 17:31:57 | sean-k-mooney | all im asking you to do is move lines 132-156 after lines 158-181 | |
| 17:32:04 | bauzas | if you really want to mix resources, do other ways | |
| 17:32:16 | bauzas | let's talk about the alternatives | |
| 17:32:19 | bauzas | custom traits | |
| 17:32:54 | bauzas | what you're proposing requires ops to define distinct traits between GPUs | |
| 17:33:00 | bauzas | even if they share the same type | |
| 17:33:31 | bauzas | the only difference would be "I'm GPU A" while the other would be "I'm GPU B" | |
| 17:33:46 | bauzas | that's not great, right? | |
| 17:33:54 | sean-k-mooney | well since this bugfix is really a feature then yes | |
| 17:34:19 | sean-k-mooney | without intoducing a new feature to expclity supprot selecting vgpus form diffent parent pysical devices | |
| 17:34:52 | sean-k-mooney | and makign that work transparly you whave to reuse one of our supproted features to acive the same goal | |
| 17:35:12 | bauzas | the real fix is is to drop the silly warning https://review.opendev.org/c/openstack/nova/+/845757/4/nova/virt/libvirt/driver.py#b8541 and just provide a loop | |
| 17:35:34 | bauzas | we could improve that for sure in a blueprint | |
| 17:36:02 | bauzas | we discussed that years ago and we said that it would require some better placement syntax | |
| 17:36:21 | bauzas | before that, there is a quickwin | |
| 17:36:22 | sean-k-mooney | we dont supprot booting a vm with multiple gpus upstream today. it can be done ebcasue we dont block it but this is a new feature | |
| 17:37:07 | bauzas | today, there a gap that prevents you to request two allocations | |
| 17:37:23 | sean-k-mooney | right and its not supported as a result | |
| 17:37:35 | sean-k-mooney | upstream and downstream | |
| 17:38:15 | bauzas | pas-ha[m]: this is Friday but I'd appreciate your thoughts https://review.opendev.org/c/openstack/nova/+/845757/ | |
| 17:38:32 | bauzas | pas-ha[m]: we can talk on Monday | |
| 17:39:05 | bauzas | sean-k-mooney: in the meantime, I'll split https://review.opendev.org/c/openstack/nova/+/845757/ in twice | |
| 17:39:16 | bauzas | and just leave the doc discussion aside | |
| 17:44:36 | sean-k-mooney | sure im ok with merging the repoducer and fix then following up with the doc. can you also add a short release note to the fix patch | |
| 17:44:39 | opendevreview | Sylvain Bauza proposed openstack/nova master: Support multiple allocations for vGPUs https://review.opendev.org/c/openstack/nova/+/845757 | |
| 17:44:40 | opendevreview | Sylvain Bauza proposed openstack/nova master: document how to ask for more a single vGPU https://review.opendev.org/c/openstack/nova/+/906151 | |
| 17:45:33 | bauzas | sean-k-mooney: if I go with a relnotes that would be a fixes section just mentioning that now you can request vGPUs using multiple resource groups | |
| 17:45:34 | sean-k-mooney | bauzas: so another way to do this which we can disucss monday | |
| 17:45:56 | sean-k-mooney | is we could have a weigher that perferve allocatoin candates that had resouce form diffent RPs | |
| 17:46:15 | sean-k-mooney | bauzas: yep exactly just short and sweet | |
| 17:46:32 | bauzas | cool | |
| 17:46:57 | spatel | sean-k-mooney hey! | |
| 17:47:28 | spatel | quick question, does snapshot include memory footprint also ? | |
| 17:47:57 | _colby | bauzas: Thank you for your help. So I removed all the resource providers except those listed in the nova config for the device types and now nova is spinning up vgpu instances correctly. Its even creating the mdevs on its own. So I guess it was tying to create an mdev for a device not listed in the config | |