| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-15 | |||
| 11:34:16 | sean-k-mooney | although you nomally dont move host between cells. you can but its not common and it proably adviasble ot not have vms on it if you do | |
| 11:34:16 | sean-k-mooney | although you nomally dont move host between cells. you can but its not common and it proably adviasble ot not have vms on it if you do | |
| 11:34:40 | sean-k-mooney | noonedeadpunk: ya its pretty cheap | |
| 11:34:40 | sean-k-mooney | noonedeadpunk: ya its pretty cheap | |
| 11:34:54 | sean-k-mooney | noonedeadpunk: ooo i think just always runs it | |
| 11:34:54 | sean-k-mooney | noonedeadpunk: ooo i think just always runs it | |
| 11:35:11 | sean-k-mooney | noonedeadpunk: we even have the option to do it as a perodic task if you really want too | |
| 11:35:11 | sean-k-mooney | noonedeadpunk: we even have the option to do it as a perodic task if you really want too | |
| 11:35:30 | noonedeadpunk | Yeah, I know. It's just smth like 20mins or so iirc | |
| 11:35:30 | noonedeadpunk | Yeah, I know. It's just smth like 20mins or so iirc | |
| 12:07:15 | lyarwood | gibi: re https://review.opendev.org/c/openstack/nova/+/770246 I forgot to note that I wanted to land https://review.opendev.org/c/openstack/nova/+/785682 first if possible, I think it's valid to also cover that corner case in the event based flow as well. | |
| 12:07:15 | lyarwood | gibi: re https://review.opendev.org/c/openstack/nova/+/770246 I forgot to note that I wanted to land https://review.opendev.org/c/openstack/nova/+/785682 first if possible, I think it's valid to also cover that corner case in the event based flow as well. | |
| 12:24:52 | gibi | lyarwood: looking | |
| 12:24:53 | gibi | lyarwood: looking | |
| 12:26:40 | gibi | lyarwood: I will do the rebase and the adaptation to your fix either today or tomorrow | |
| 12:26:40 | gibi | lyarwood: I will do the rebase and the adaptation to your fix either today or tomorrow | |
| 12:27:38 | lyarwood | gibi: ack thanks, I'll go over the rest of the series later in more detail but clicking through it LGTM at the moment | |
| 12:27:38 | lyarwood | gibi: ack thanks, I'll go over the rest of the series later in more detail but clicking through it LGTM at the moment | |
| 12:27:59 | gibi | cool, I will +A your fix soon | |
| 12:27:59 | gibi | cool, I will +A your fix soon | |
| 13:12:28 | mnaser | bauzas: https://bugs.launchpad.net/nova/+bug/1900800 have you thought about how this can be resolved? We are running into this often :( | |
| 13:12:29 | mnaser | bauzas: https://bugs.launchpad.net/nova/+bug/1900800 have you thought about how this can be resolved? We are running into this often :( | |
| 13:12:32 | openstack | Launchpad bug 1900800 in OpenStack Compute (nova) "VGPUs is not recreated on host reboot" [Low,Confirmed] - Assigned to Sylvain Bauza (sylvain-bauza) | |
| 13:41:18 | bauzas | mnaser: sorry I was on a meeting | |
| 13:41:18 | bauzas | mnaser: sorry I was on a meeting | |
| 13:41:52 | bauzas | mnaser: well, maybe we would need to ask the operator to create the mdevs after rebooting | |
| 13:41:52 | bauzas | mnaser: well, maybe we would need to ask the operator to create the mdevs after rebooting | |
| 13:49:12 | openstackgerrit | Qiu Fossen proposed openstack/nova-specs master: Allow migrating PMEM's data https://review.opendev.org/c/openstack/nova-specs/+/785563 | |
| 13:51:32 | sean-k-mooney | bauzas: the mdevs should be create in init_host today | |
| 13:51:32 | sean-k-mooney | bauzas: the mdevs should be create in init_host today | |
| 13:51:57 | sean-k-mooney | oh | |
| 13:51:57 | sean-k-mooney | oh | |
| 13:52:10 | bauzas | sean-k-mooney: the problem is that we don't know which mdev type they usze | |
| 13:52:10 | bauzas | sean-k-mooney: the problem is that we don't know which mdev type they usze | |
| 13:52:49 | sean-k-mooney | well we should no? | |
| 13:52:49 | sean-k-mooney | well we should no? | |
| 13:53:05 | sean-k-mooney | the mdevs should still be in the domins | |
| 13:53:05 | sean-k-mooney | the mdevs should still be in the domins | |
| 13:53:12 | sean-k-mooney | this should be running before we start the vms | |
| 13:53:12 | sean-k-mooney | this should be running before we start the vms | |
| 13:53:22 | sean-k-mooney | oh the type is not recorded | |
| 13:53:22 | sean-k-mooney | oh the type is not recorded | |
| 13:53:33 | sean-k-mooney | so if you have multiple devices that would be an issue | |
| 13:53:33 | sean-k-mooney | so if you have multiple devices that would be an issue | |
| 13:54:02 | sean-k-mooney | although we would be abel to look at the vgpu request in the flavor if you had a traits request | |
| 13:54:02 | sean-k-mooney | although we would be abel to look at the vgpu request in the flavor if you had a traits request | |
| 13:54:12 | sean-k-mooney | or better yet the allocation summeries | |
| 13:54:12 | sean-k-mooney | or better yet the allocation summeries | |
| 13:55:29 | sean-k-mooney | we can look up the RP form which the allocation came from and then identify the partent device and use that to look up the mdev type in the config | |
| 13:55:29 | sean-k-mooney | we can look up the RP form which the allocation came from and then identify the partent device and use that to look up the mdev type in the config | |
| 13:55:42 | sean-k-mooney | bauzas: that should work right ^ | |
| 13:55:42 | sean-k-mooney | bauzas: that should work right ^ | |
| 13:55:50 | bauzas | sean-k-mooney: the problem is that the traits are optional | |
| 13:55:50 | bauzas | sean-k-mooney: the problem is that the traits are optional | |
| 13:55:58 | sean-k-mooney | we dont need the traits | |
| 13:55:58 | sean-k-mooney | we dont need the traits | |
| 13:56:20 | sean-k-mooney | use allcotion to figure out partent device use parent device to look up mdev type in nova.conf | |
| 13:56:20 | sean-k-mooney | use allcotion to figure out partent device use parent device to look up mdev type in nova.conf | |
| 13:56:23 | sean-k-mooney | then recreate it | |
| 13:56:23 | sean-k-mooney | then recreate it | |
| 13:56:51 | bauzas | sean-k-mooney: https://github.com/openstack/nova/blob/450213f/nova/virt/libvirt/driver.py#L816 | |
| 13:56:51 | bauzas | sean-k-mooney: https://github.com/openstack/nova/blob/450213f/nova/virt/libvirt/driver.py#L816 | |
| 13:57:33 | bauzas | here, we would then need to call placement | |
| 13:57:33 | bauzas | here, we would then need to call placement | |
| 13:57:40 | sean-k-mooney | yes | |
| 13:57:54 | sean-k-mooney | unless we have the allocation summeries saved somewhere | |
| 13:57:54 | sean-k-mooney | unless we have the allocation summeries saved somewhere | |
| 13:58:11 | sean-k-mooney | i.e. in the nova db but we dont as far as i know | |
| 13:58:11 | sean-k-mooney | i.e. in the nova db but we dont as far as i know | |
| 13:58:12 | bauzas | ok, but then we would see VGPU allocations | |
| 13:58:12 | bauzas | ok, but then we would see VGPU allocations | |
| 13:58:23 | bauzas | for RP | |
| 13:58:23 | bauzas | for RP | |
| 13:58:25 | bauzas | for a RP | |
| 13:58:25 | bauzas | for a RP | |
| 13:58:32 | bauzas | which is a pGPU | |
| 13:58:32 | bauzas | which is a pGPU | |
| 13:58:52 | bauzas | so then we would need to look at the conf option to know which type it uses | |
| 13:58:52 | bauzas | so then we would need to look at the conf option to know which type it uses | |
| 13:58:59 | sean-k-mooney | yep | |
| 13:59:03 | bauzas | that *could* work | |
| 13:59:03 | bauzas | that *could* work | |
| 13:59:23 | bauzas | but that's a long change I think | |
| 13:59:23 | bauzas | but that's a long change I think | |
| 13:59:55 | sean-k-mooney | you mean complex to write/test | |
| 13:59:55 | sean-k-mooney | you mean complex to write/test | |
| 14:00:19 | sean-k-mooney | i think its what is required though unless we start storing the infor in the nova db in the resouces table for example | |
| 14:00:19 | sean-k-mooney | i think its what is required though unless we start storing the infor in the nova db in the resouces table for example | |
| 14:00:52 | sean-k-mooney | those are our two options caulate it form plamcnet or record mdevs in the db like pmem or pcidevices | |
| 14:00:52 | sean-k-mooney | those are our two options caulate it form plamcnet or record mdevs in the db like pmem or pcidevices | |
| 14:01:17 | sean-k-mooney | so that we can just look it up | |
| 14:01:17 | sean-k-mooney | so that we can just look it up | |
| 14:03:12 | bauzas | I can try to help | |
| 14:03:12 | bauzas | I can try to help | |
| 14:05:01 | sean-k-mooney | mnaser: it would be a bit of a hack but you could proably fix this with a bash script executed by a systemd service file tempoarlly | |
| 14:05:01 | sean-k-mooney | mnaser: it would be a bit of a hack but you could proably fix this with a bash script executed by a systemd service file tempoarlly | |
| 14:05:29 | sean-k-mooney | basically implemented the same logic | |
| 14:05:29 | sean-k-mooney | basically implemented the same logic | |
| 14:06:37 | sean-k-mooney | loop over the domains and for each with an mdev look up the placment allocation and get the rp with the vgpu resouces | |
| 14:06:37 | sean-k-mooney | loop over the domains and for each with an mdev look up the placment allocation and get the rp with the vgpu resouces | |