| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-15 | |||
| 13:54:12 | sean-k-mooney | or better yet the allocation summeries | |
| 13:54:12 | sean-k-mooney | or better yet the allocation summeries | |
| 13:55:29 | sean-k-mooney | we can look up the RP form which the allocation came from and then identify the partent device and use that to look up the mdev type in the config | |
| 13:55:29 | sean-k-mooney | we can look up the RP form which the allocation came from and then identify the partent device and use that to look up the mdev type in the config | |
| 13:55:42 | sean-k-mooney | bauzas: that should work right ^ | |
| 13:55:42 | sean-k-mooney | bauzas: that should work right ^ | |
| 13:55:50 | bauzas | sean-k-mooney: the problem is that the traits are optional | |
| 13:55:50 | bauzas | sean-k-mooney: the problem is that the traits are optional | |
| 13:55:58 | sean-k-mooney | we dont need the traits | |
| 13:55:58 | sean-k-mooney | we dont need the traits | |
| 13:56:20 | sean-k-mooney | use allcotion to figure out partent device use parent device to look up mdev type in nova.conf | |
| 13:56:20 | sean-k-mooney | use allcotion to figure out partent device use parent device to look up mdev type in nova.conf | |
| 13:56:23 | sean-k-mooney | then recreate it | |
| 13:56:23 | sean-k-mooney | then recreate it | |
| 13:56:51 | bauzas | sean-k-mooney: https://github.com/openstack/nova/blob/450213f/nova/virt/libvirt/driver.py#L816 | |
| 13:56:51 | bauzas | sean-k-mooney: https://github.com/openstack/nova/blob/450213f/nova/virt/libvirt/driver.py#L816 | |
| 13:57:33 | bauzas | here, we would then need to call placement | |
| 13:57:33 | bauzas | here, we would then need to call placement | |
| 13:57:40 | sean-k-mooney | yes | |
| 13:57:54 | sean-k-mooney | unless we have the allocation summeries saved somewhere | |
| 13:57:54 | sean-k-mooney | unless we have the allocation summeries saved somewhere | |
| 13:58:11 | sean-k-mooney | i.e. in the nova db but we dont as far as i know | |
| 13:58:11 | sean-k-mooney | i.e. in the nova db but we dont as far as i know | |
| 13:58:12 | bauzas | ok, but then we would see VGPU allocations | |
| 13:58:12 | bauzas | ok, but then we would see VGPU allocations | |
| 13:58:23 | bauzas | for RP | |
| 13:58:23 | bauzas | for RP | |
| 13:58:25 | bauzas | for a RP | |
| 13:58:25 | bauzas | for a RP | |
| 13:58:32 | bauzas | which is a pGPU | |
| 13:58:32 | bauzas | which is a pGPU | |
| 13:58:52 | bauzas | so then we would need to look at the conf option to know which type it uses | |
| 13:58:52 | bauzas | so then we would need to look at the conf option to know which type it uses | |
| 13:58:59 | sean-k-mooney | yep | |
| 13:59:03 | bauzas | that *could* work | |
| 13:59:03 | bauzas | that *could* work | |
| 13:59:23 | bauzas | but that's a long change I think | |
| 13:59:23 | bauzas | but that's a long change I think | |
| 13:59:55 | sean-k-mooney | you mean complex to write/test | |
| 13:59:55 | sean-k-mooney | you mean complex to write/test | |
| 14:00:19 | sean-k-mooney | i think its what is required though unless we start storing the infor in the nova db in the resouces table for example | |
| 14:00:19 | sean-k-mooney | i think its what is required though unless we start storing the infor in the nova db in the resouces table for example | |
| 14:00:52 | sean-k-mooney | those are our two options caulate it form plamcnet or record mdevs in the db like pmem or pcidevices | |
| 14:00:52 | sean-k-mooney | those are our two options caulate it form plamcnet or record mdevs in the db like pmem or pcidevices | |
| 14:01:17 | sean-k-mooney | so that we can just look it up | |
| 14:01:17 | sean-k-mooney | so that we can just look it up | |
| 14:03:12 | bauzas | I can try to help | |
| 14:03:12 | bauzas | I can try to help | |
| 14:05:01 | sean-k-mooney | mnaser: it would be a bit of a hack but you could proably fix this with a bash script executed by a systemd service file tempoarlly | |
| 14:05:01 | sean-k-mooney | mnaser: it would be a bit of a hack but you could proably fix this with a bash script executed by a systemd service file tempoarlly | |
| 14:05:29 | sean-k-mooney | basically implemented the same logic | |
| 14:05:29 | sean-k-mooney | basically implemented the same logic | |
| 14:06:37 | sean-k-mooney | loop over the domains and for each with an mdev look up the placment allocation and get the rp with the vgpu resouces | |
| 14:06:37 | sean-k-mooney | loop over the domains and for each with an mdev look up the placment allocation and get the rp with the vgpu resouces | |
| 14:07:11 | sean-k-mooney | then get the mdev type and create it with the same mdev uuid as the xml currenly has | |
| 14:07:11 | sean-k-mooney | then get the mdev type and create it with the same mdev uuid as the xml currenly has | |
| 14:08:04 | sean-k-mooney | you could use systemd's "before" and "after" requirements to ensure it runs before nova-compute and after libvirt start | |
| 14:08:04 | sean-k-mooney | you could use systemd's "before" and "after" requirements to ensure it runs before nova-compute and after libvirt start | |
| 14:08:58 | sean-k-mooney | really nova should do that but that the bug your hitting i guess. | |
| 14:08:58 | sean-k-mooney | really nova should do that but that the bug your hitting i guess. | |
| 14:16:48 | mnaser | sean-k-mooney / bauzas: i guess if i'm understanding correctly, the instance <=> mdev mapping is not stored inside nova anywhere so we rely on the state in the libvirt domain | |
| 14:16:48 | mnaser | sean-k-mooney / bauzas: i guess if i'm understanding correctly, the instance <=> mdev mapping is not stored inside nova anywhere so we rely on the state in the libvirt domain | |
| 14:17:05 | bauzas | that's right | |
| 14:17:05 | bauzas | that's right | |
| 14:17:17 | bauzas | there is an existing tool tho | |
| 14:17:17 | bauzas | there is an existing tool tho | |
| 14:17:31 | bauzas | mnaser: https://github.com/mdevctl/mdevctl | |
| 14:17:31 | bauzas | mnaser: https://github.com/mdevctl/mdevctl | |
| 14:17:47 | bauzas | mnaser: you could use it for precreating the mdevs and persist them (using systemctl) | |
| 14:17:47 | bauzas | mnaser: you could use it for precreating the mdevs and persist them (using systemctl) | |
| 14:18:05 | mnaser | "When a known parent device add udev event occurs (or, for more recent kernels, change events with MDEV_STATE values), mdevctl is called by a udev rule to create defined devices with "start": "auto" configured." interesting | |
| 14:18:05 | mnaser | "When a known parent device add udev event occurs (or, for more recent kernels, change events with MDEV_STATE values), mdevctl is called by a udev rule to create defined devices with "start": "auto" configured." interesting | |
| 14:18:06 | bauzas | (whoops, systemd) | |
| 14:18:06 | bauzas | (whoops, systemd) | |
| 14:18:10 | sean-k-mooney | you could but long term we dont want peopel to do that | |
| 14:18:10 | sean-k-mooney | you could but long term we dont want peopel to do that | |
| 14:18:38 | sean-k-mooney | bauzas: if we were to go down that route we shoudl remvoe the code for nova to do it | |
| 14:18:38 | sean-k-mooney | bauzas: if we were to go down that route we shoudl remvoe the code for nova to do it | |
| 14:18:54 | bauzas | sean-k-mooney: we said this before | |
| 14:18:54 | bauzas | sean-k-mooney: we said this before | |
| 14:18:55 | sean-k-mooney | bauzas: and track the mdevs in the pci_devices table or similar | |
| 14:18:55 | sean-k-mooney | bauzas: and track the mdevs in the pci_devices table or similar | |
| 14:19:10 | bauzas | sean-k-mooney: honestly, mdevs are like VFs | |
| 14:19:10 | bauzas | sean-k-mooney: honestly, mdevs are like VFs | |
| 14:19:14 | sean-k-mooney | yep | |
| 14:19:17 | bauzas | and we don't persist the latter | |
| 14:19:17 | bauzas | and we don't persist the latter | |
| 14:19:22 | sean-k-mooney | we do | |
| 14:19:22 | sean-k-mooney | we do | |
| 14:19:26 | bauzas | in nova ? | |
| 14:19:26 | bauzas | in nova ? | |
| 14:19:29 | sean-k-mooney | in the pci_devices table in nova | |
| 14:19:29 | sean-k-mooney | in the pci_devices table in nova | |
| 14:19:40 | bauzas | but you need to precreate them, right? | |
| 14:19:40 | bauzas | but you need to precreate them, right? | |
| 14:19:43 | sean-k-mooney | thats what the pci_tracker does | |
| 14:19:43 | sean-k-mooney | thats what the pci_tracker does | |
| 14:19:49 | sean-k-mooney | bauzas: oh yes you do | |
| 14:19:49 | sean-k-mooney | bauzas: oh yes you do | |
| 14:19:55 | bauzas | the pci trackers tracks the VFs | |