| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-11 | |||
| 13:30:35 | gibi | bauzas: thanks a lot | |
| 13:31:00 | bauzas | gibi: FWIW, I'm also thinking about your comments like https://review.openstack.org/#/c/599208/12/nova/virt/libvirt/driver.py@6718 | |
| 13:33:06 | gibi | bauzas: so if the TODO means that the operator today can change the config that nova does not handle then I think we have a problem | |
| 13:34:52 | bauzas | gibi: yeah but the problem is that we don't know yet if we need to create a new child RP | |
| 13:35:04 | bauzas | we only know about the root RP | |
| 13:35:15 | bauzas | and we don't persist the existing type | |
| 13:35:53 | bauzas | oh and FWIW, it doesn't work also atm | |
| 13:37:04 | gibi | bauzas: so, what will happen if the admin changes the enabled_vgpu_types ? | |
| 13:43:19 | bauzas | gibi: for the moment, we create an inventory by calling the number of left mdevs | |
| 13:43:38 | bauzas | gibi: if the operator changes the type, the total would be modified | |
| 13:44:07 | gibi | bauzas: so eventually the gpu inventory will be in sync with the reality? | |
| 13:44:21 | bauzas | if we have allocations, I guess placement verifies that the new total is more than the allocations value | |
| 13:44:36 | bauzas | gibi: every 60 secs, ye | |
| 13:45:37 | gibi | bauzas: so with type X the hypervisor reports 10 available mdev, then a server is created that use up 1 of that 10. Then the operator changes the config to type Y and restart the compute | |
| 13:45:51 | gibi | bauzas: from type Y the hypervisor report 5 mdevs available | |
| 13:46:03 | bauzas | right | |
| 13:46:10 | gibi | bauzas: so the compute changes the inventory from 10 to 5 which still fits, as only 1 allocated | |
| 13:46:15 | bauzas | yup | |
| 13:46:34 | bauzas | that said, I have an open question | |
| 13:46:47 | bauzas | what if we have 6 allocs for a type with 10 total | |
| 13:47:04 | bauzas | and then operator uses type Y which only has 5 as a total | |
| 13:47:11 | gibi | I think placement will not allow changing the total to 5 then | |
| 13:47:18 | bauzas | ok, then we're good | |
| 13:47:20 | gibi | but easxy to try | |
| 13:47:21 | bauzas | actually wait | |
| 13:47:31 | bauzas | if you have one pGPU | |
| 13:47:35 | bauzas | with multiple types | |
| 13:48:08 | bauzas | when you first create a mdev, then all of the other types but the one used by the mdev are having total = 0 | |
| 13:48:47 | bauzas | so, if the operator wants to change a type for the same pGPU when there are instances, then it's litterally not possible to get more instances | |
| 13:49:01 | bauzas | because the total will be the number of existing mdevs | |
| 13:49:19 | bauzas | unless the operators changes it back to the original type | |
| 13:49:26 | bauzas | so we don't have a problem atm | |
| 13:49:29 | gibi | bauzas: so the operator needs to delete / migrate away the instances first | |
| 13:49:42 | gibi | then change the tyee | |
| 13:49:43 | bauzas | yeah, or change the type back to the original | |
| 13:49:43 | gibi | type | |
| 13:49:53 | gibi | bauzas: OK, that makes it even stricter | |
| 13:50:01 | gibi | bauzas: and here strict is good, I think | |
| 13:50:02 | bauzas | no, he can change without needing to delete | |
| 13:50:23 | gibi | bauzas: yeah, if he want to change back then he can without delete | |
| 13:50:26 | bauzas | that's just only if he really wants to change from one type to the other | |
| 13:50:32 | gibi | bauzas: yeas | |
| 13:50:32 | bauzas | yeah | |
| 13:50:35 | gibi | cool | |
| 13:50:50 | gibi | maybe we want to document this behavior somehow for the deployer | |
| 13:50:51 | bauzas | now, with the change, lemme think | |
| 13:50:53 | gibi | to avoud surprise | |
| 13:51:00 | bauzas | gibi: we did it already | |
| 13:51:07 | gibi | bauzas: then I rest my case | |
| 13:51:09 | gibi | :) | |
| 13:51:46 | bauzas | oh wait, no actually | |
| 13:51:47 | bauzas | https://docs.openstack.org/nova/latest/admin/virtual-gpu.html | |
| 13:52:05 | bauzas | I could add a note in https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#enable-gpu-types-compute | |
| 13:52:54 | gibi | bauzas: making that note solves my comment in your patch | |
| 13:53:14 | bauzas | gibi: that said, I need to think on how it works with nested RPs | |
| 13:53:42 | bauzas | for the moment, it does work because allocations are made against the same RP | |
| 13:53:50 | bauzas | oh wait | |
| 13:54:05 | bauzas | even if people change types, they won't change the pGPU | |
| 13:54:22 | bauzas | so allocations will still be made against the right pGPUI | |
| 13:54:42 | bauzas | actually, something is missing :( | |
| 13:54:51 | bauzas | holy shit | |
| 13:54:59 | bauzas | we have an allocation against one pGPU | |
| 13:55:17 | fried_rice | n-sch meeting in 5 minutes in #openstack-meeting-alt | |
| 13:55:19 | bauzas | but then libvirt creates the mdev (or uses some mdev) from any pGPU | |
| 13:55:26 | bauzas | which is a problem | |
| 13:55:33 | bauzas | for correctly counting allocations | |
| 13:55:47 | bauzas | shit shit shit | |
| 13:55:53 | bauzas | gibi: thanks for making me think of it | |
| 13:59:43 | gibi | bauzas: you are welcome :) | |
| 14:12:24 | openstackgerrit | Hamdy Khader proposed openstack/os-vif master: Add VIFPortProfileSmartNIC profile https://review.openstack.org/636061 | |
| 14:21:02 | openstackgerrit | Surya Seetharaman proposed openstack/nova-specs master: Support server power state update through external event https://review.openstack.org/636132 | |
| 14:28:37 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix deps for api-samples tox env https://review.openstack.org/633620 | |
| 14:28:38 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Make VolumeAttachmentsSampleV249 test other methods https://review.openstack.org/633621 | |
| 14:28:38 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add microversion to expose virtual device tags https://review.openstack.org/631948 | |
| 14:39:32 | bauzas | mriedem: FWIW, gibi made me think of a problem with the reshape series | |
| 14:39:45 | bauzas | mriedem: that's uncommented yet, but I'll need to respin | |
| 14:41:53 | mriedem | you've yet to respin anyway right? | |
| 14:42:05 | mriedem | so respin your local respin | |
| 14:43:05 | mriedem | bauzas: gibi: are you able to summarize the issue? | |
| 14:43:10 | mriedem | so i don't have to read scrollback? | |
| 14:43:27 | bauzas | mriedem: tl;dr: | |
| 14:43:38 | bauzas | we will have inventories per pGPU | |
| 14:43:56 | bauzas | placement will choose one child RP and make an allocation to it | |
| 14:44:07 | bauzas | then it goes down to the driver | |
| 14:44:16 | bauzas | which picks randomly at the moment | |
| 14:44:34 | bauzas | the last sentence has to change | |
| 14:44:53 | bauzas | the driver has to pick one of the available mdevs from the pGPU related to the allocation | |
| 14:45:00 | mriedem | driver.spawn() gets the allocations, so we just have to reconcile the local inventory in the driver to what was allocated and pick that mdev yeah? | |
| 14:45:05 | bauzas | or allocations would diverge from actual usage | |
| 14:45:14 | bauzas | mriedem: yup that | |
| 14:45:43 | bauzas | mriedem: we need to lookup the child RP from the allocation and find an allocable mdev from it | |
| 14:45:52 | bauzas | not a big deal, but still has to be done | |
| 14:46:12 | openstackgerrit | garyk proposed openstack/nova master: Better handle live migration abort https://review.openstack.org/635440 | |
| 14:46:17 | mriedem | does that need to be done separately from the reshaper patch, like as a dependency - even though it won't do anything until the reshape is done? | |
| 14:46:23 | mriedem | because that reshaper patch is already big | |
| 14:46:31 | bauzas | yeah I was considering this | |
| 14:46:38 | mriedem | i'd split them if possible | |
| 14:46:41 | bauzas | we could drop a -2 | |
| 14:46:46 | mriedem | 1. handled nested in spawn, 2. do reshapes | |
| 14:46:48 | bauzas | and only +W once all are accepted | |