| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-15 | |||
| 13:41:52 | bauzas | mnaser: well, maybe we would need to ask the operator to create the mdevs after rebooting | |
| 13:41:52 | bauzas | mnaser: well, maybe we would need to ask the operator to create the mdevs after rebooting | |
| 13:49:12 | openstackgerrit | Qiu Fossen proposed openstack/nova-specs master: Allow migrating PMEM's data https://review.opendev.org/c/openstack/nova-specs/+/785563 | |
| 13:51:32 | sean-k-mooney | bauzas: the mdevs should be create in init_host today | |
| 13:51:32 | sean-k-mooney | bauzas: the mdevs should be create in init_host today | |
| 13:51:57 | sean-k-mooney | oh | |
| 13:51:57 | sean-k-mooney | oh | |
| 13:52:10 | bauzas | sean-k-mooney: the problem is that we don't know which mdev type they usze | |
| 13:52:10 | bauzas | sean-k-mooney: the problem is that we don't know which mdev type they usze | |
| 13:52:49 | sean-k-mooney | well we should no? | |
| 13:52:49 | sean-k-mooney | well we should no? | |
| 13:53:05 | sean-k-mooney | the mdevs should still be in the domins | |
| 13:53:05 | sean-k-mooney | the mdevs should still be in the domins | |
| 13:53:12 | sean-k-mooney | this should be running before we start the vms | |
| 13:53:12 | sean-k-mooney | this should be running before we start the vms | |
| 13:53:22 | sean-k-mooney | oh the type is not recorded | |
| 13:53:22 | sean-k-mooney | oh the type is not recorded | |
| 13:53:33 | sean-k-mooney | so if you have multiple devices that would be an issue | |
| 13:53:33 | sean-k-mooney | so if you have multiple devices that would be an issue | |
| 13:54:02 | sean-k-mooney | although we would be abel to look at the vgpu request in the flavor if you had a traits request | |
| 13:54:02 | sean-k-mooney | although we would be abel to look at the vgpu request in the flavor if you had a traits request | |
| 13:54:12 | sean-k-mooney | or better yet the allocation summeries | |
| 13:54:12 | sean-k-mooney | or better yet the allocation summeries | |
| 13:55:29 | sean-k-mooney | we can look up the RP form which the allocation came from and then identify the partent device and use that to look up the mdev type in the config | |
| 13:55:29 | sean-k-mooney | we can look up the RP form which the allocation came from and then identify the partent device and use that to look up the mdev type in the config | |
| 13:55:42 | sean-k-mooney | bauzas: that should work right ^ | |
| 13:55:42 | sean-k-mooney | bauzas: that should work right ^ | |
| 13:55:50 | bauzas | sean-k-mooney: the problem is that the traits are optional | |
| 13:55:50 | bauzas | sean-k-mooney: the problem is that the traits are optional | |
| 13:55:58 | sean-k-mooney | we dont need the traits | |
| 13:55:58 | sean-k-mooney | we dont need the traits | |
| 13:56:20 | sean-k-mooney | use allcotion to figure out partent device use parent device to look up mdev type in nova.conf | |
| 13:56:20 | sean-k-mooney | use allcotion to figure out partent device use parent device to look up mdev type in nova.conf | |
| 13:56:23 | sean-k-mooney | then recreate it | |
| 13:56:23 | sean-k-mooney | then recreate it | |
| 13:56:51 | bauzas | sean-k-mooney: https://github.com/openstack/nova/blob/450213f/nova/virt/libvirt/driver.py#L816 | |
| 13:56:51 | bauzas | sean-k-mooney: https://github.com/openstack/nova/blob/450213f/nova/virt/libvirt/driver.py#L816 | |
| 13:57:33 | bauzas | here, we would then need to call placement | |
| 13:57:33 | bauzas | here, we would then need to call placement | |
| 13:57:40 | sean-k-mooney | yes | |
| 13:57:54 | sean-k-mooney | unless we have the allocation summeries saved somewhere | |
| 13:57:54 | sean-k-mooney | unless we have the allocation summeries saved somewhere | |
| 13:58:11 | sean-k-mooney | i.e. in the nova db but we dont as far as i know | |
| 13:58:11 | sean-k-mooney | i.e. in the nova db but we dont as far as i know | |
| 13:58:12 | bauzas | ok, but then we would see VGPU allocations | |
| 13:58:12 | bauzas | ok, but then we would see VGPU allocations | |
| 13:58:23 | bauzas | for RP | |
| 13:58:23 | bauzas | for RP | |
| 13:58:25 | bauzas | for a RP | |
| 13:58:25 | bauzas | for a RP | |
| 13:58:32 | bauzas | which is a pGPU | |
| 13:58:32 | bauzas | which is a pGPU | |
| 13:58:52 | bauzas | so then we would need to look at the conf option to know which type it uses | |
| 13:58:52 | bauzas | so then we would need to look at the conf option to know which type it uses | |
| 13:58:59 | sean-k-mooney | yep | |
| 13:59:03 | bauzas | that *could* work | |
| 13:59:03 | bauzas | that *could* work | |
| 13:59:23 | bauzas | but that's a long change I think | |
| 13:59:23 | bauzas | but that's a long change I think | |
| 13:59:55 | sean-k-mooney | you mean complex to write/test | |
| 13:59:55 | sean-k-mooney | you mean complex to write/test | |
| 14:00:19 | sean-k-mooney | i think its what is required though unless we start storing the infor in the nova db in the resouces table for example | |
| 14:00:19 | sean-k-mooney | i think its what is required though unless we start storing the infor in the nova db in the resouces table for example | |
| 14:00:52 | sean-k-mooney | those are our two options caulate it form plamcnet or record mdevs in the db like pmem or pcidevices | |
| 14:00:52 | sean-k-mooney | those are our two options caulate it form plamcnet or record mdevs in the db like pmem or pcidevices | |
| 14:01:17 | sean-k-mooney | so that we can just look it up | |
| 14:01:17 | sean-k-mooney | so that we can just look it up | |
| 14:03:12 | bauzas | I can try to help | |
| 14:03:12 | bauzas | I can try to help | |
| 14:05:01 | sean-k-mooney | mnaser: it would be a bit of a hack but you could proably fix this with a bash script executed by a systemd service file tempoarlly | |
| 14:05:01 | sean-k-mooney | mnaser: it would be a bit of a hack but you could proably fix this with a bash script executed by a systemd service file tempoarlly | |
| 14:05:29 | sean-k-mooney | basically implemented the same logic | |
| 14:05:29 | sean-k-mooney | basically implemented the same logic | |
| 14:06:37 | sean-k-mooney | loop over the domains and for each with an mdev look up the placment allocation and get the rp with the vgpu resouces | |
| 14:06:37 | sean-k-mooney | loop over the domains and for each with an mdev look up the placment allocation and get the rp with the vgpu resouces | |
| 14:07:11 | sean-k-mooney | then get the mdev type and create it with the same mdev uuid as the xml currenly has | |
| 14:07:11 | sean-k-mooney | then get the mdev type and create it with the same mdev uuid as the xml currenly has | |
| 14:08:04 | sean-k-mooney | you could use systemd's "before" and "after" requirements to ensure it runs before nova-compute and after libvirt start | |
| 14:08:04 | sean-k-mooney | you could use systemd's "before" and "after" requirements to ensure it runs before nova-compute and after libvirt start | |
| 14:08:58 | sean-k-mooney | really nova should do that but that the bug your hitting i guess. | |
| 14:08:58 | sean-k-mooney | really nova should do that but that the bug your hitting i guess. | |
| 14:16:48 | mnaser | sean-k-mooney / bauzas: i guess if i'm understanding correctly, the instance <=> mdev mapping is not stored inside nova anywhere so we rely on the state in the libvirt domain | |
| 14:16:48 | mnaser | sean-k-mooney / bauzas: i guess if i'm understanding correctly, the instance <=> mdev mapping is not stored inside nova anywhere so we rely on the state in the libvirt domain | |
| 14:17:05 | bauzas | that's right | |
| 14:17:05 | bauzas | that's right | |
| 14:17:17 | bauzas | there is an existing tool tho | |
| 14:17:17 | bauzas | there is an existing tool tho | |
| 14:17:31 | bauzas | mnaser: https://github.com/mdevctl/mdevctl | |
| 14:17:31 | bauzas | mnaser: https://github.com/mdevctl/mdevctl | |
| 14:17:47 | bauzas | mnaser: you could use it for precreating the mdevs and persist them (using systemctl) | |
| 14:17:47 | bauzas | mnaser: you could use it for precreating the mdevs and persist them (using systemctl) | |
| 14:18:05 | mnaser | "When a known parent device add udev event occurs (or, for more recent kernels, change events with MDEV_STATE values), mdevctl is called by a udev rule to create defined devices with "start": "auto" configured." interesting | |
| 14:18:05 | mnaser | "When a known parent device add udev event occurs (or, for more recent kernels, change events with MDEV_STATE values), mdevctl is called by a udev rule to create defined devices with "start": "auto" configured." interesting | |
| 14:18:06 | bauzas | (whoops, systemd) | |
| 14:18:06 | bauzas | (whoops, systemd) | |
| 14:18:10 | sean-k-mooney | you could but long term we dont want peopel to do that | |
| 14:18:10 | sean-k-mooney | you could but long term we dont want peopel to do that | |
| 14:18:38 | sean-k-mooney | bauzas: if we were to go down that route we shoudl remvoe the code for nova to do it | |
| 14:18:38 | sean-k-mooney | bauzas: if we were to go down that route we shoudl remvoe the code for nova to do it | |
| 14:18:54 | bauzas | sean-k-mooney: we said this before | |