| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-11 | |||
| 17:25:30 | stephenfin | yup, agreed | |
| 17:26:02 | gibi | lyarwood, bauzas, dansmith: we need a second core for an easy refactor patch https://review.opendev.org/c/openstack/nova/+/779851 the rest of the series is reviewd by stephen but his patch is stephen's so we need somebody else to get vdpa to land | |
| 17:26:26 | gibi | stephenfin: thanks | |
| 17:26:52 | lyarwood | looking | |
| 17:27:01 | gibi | lyarwood: thanks | |
| 17:31:58 | lyarwood | gibi: done | |
| 17:33:24 | gibi | lyarwood: thanks | |
| 17:48:09 | gibi | lyarwood: o/ | |
| 19:00:57 | openstackgerrit | sean mooney proposed openstack/nova master: block vm boot with vdpa ports https://review.opendev.org/c/openstack/nova/+/780065 | |
| 19:03:57 | sean-k-mooney | FYI that is only needed if we dont merge the VDPA inital boot support to ensure that even though neutron support the vdpa type we do not try to boot with it until we do | |
| 19:04:21 | gibi | sean-k-mooney: I'm reading vdpa right now | |
| 19:04:32 | gibi | sean-k-mooney: I have a question | |
| 19:05:14 | sean-k-mooney | ask away | |
| 19:05:21 | gibi | sean-k-mooney: do we need to make the VDPA devs unavaible when the related PF is become allocated? | |
| 19:05:38 | gibi | s/is become/becomes | |
| 19:05:43 | sean-k-mooney | yes we should | |
| 19:05:59 | sean-k-mooney | i know wehere we do that let me check if it will do that | |
| 19:06:07 | gibi | I'm looking at https://review.opendev.org/c/openstack/nova/+/778350/7/nova/pci/stats.py#215 | |
| 19:06:17 | gibi | _handle_device_dependents | |
| 19:06:29 | sean-k-mooney | yep | |
| 19:06:43 | gibi | and I see that we make the PF unavailable when we allocate a VF or a VDPA | |
| 19:06:43 | sean-k-mooney | so that will mark the parent as unavaiable if we use a vf | |
| 19:06:47 | sean-k-mooney | *vdpa dev | |
| 19:06:50 | gibi | yepp | |
| 19:06:55 | gibi | but what about the other direction | |
| 19:07:02 | sean-k-mooney | so you are saying we shoudl do the reverse too | |
| 19:07:06 | gibi | wondering | |
| 19:07:10 | gibi | if I allocated the PF | |
| 19:07:26 | gibi | then all the devs under the PF should not be used any more | |
| 19:07:34 | sean-k-mooney | ya i did not think of that edgecase because i only whitelisted the VF in the pci whitelist not the PF | |
| 19:07:52 | sean-k-mooney | gibi: correct | |
| 19:08:20 | gibi | I guess there is also the case when both the VF and the related VDPA is whitelsited and I allocate the VF | |
| 19:08:33 | sean-k-mooney | it might just work however depending on how remove_device works | |
| 19:08:56 | sean-k-mooney | gibi: no we will only ever get either the VF or VDPA divce listed | |
| 19:09:21 | sean-k-mooney | im basically taking the VF recored and changing its type | |
| 19:09:35 | gibi | sean-k-mooney: so when the admin creates the VDPA dev that consumes the VF already from the host? | |
| 19:10:10 | sean-k-mooney | when an admin create a vdpa device on the host it will be reported as vdpa instead of a VF | |
| 19:10:17 | gibi | OK | |
| 19:10:27 | gibi | so we wont see both the VF and the VDPA at the same time | |
| 19:10:36 | sean-k-mooney | not for the same VF no | |
| 19:10:39 | gibi | cool | |
| 19:10:48 | sean-k-mooney | you could have a mix of VF and VDPA devices | |
| 19:10:49 | gibi | then only the PF edge case remains | |
| 19:11:15 | gibi | when the PF is allocated we have the code somewhere that makes the VFs unavailable | |
| 19:11:16 | sean-k-mooney | if each VDPA device is from a VF not listed as a VF | |
| 19:11:38 | sean-k-mooney | yes its here https://review.opendev.org/c/openstack/nova/+/778350/7/nova/pci/stats.py#223 | |
| 19:11:49 | gibi | ahh yes | |
| 19:11:54 | gibi | so that removes the list of vfs | |
| 19:12:03 | sean-k-mooney | the VDPA devices are listed with the VF address in the tracker | |
| 19:12:15 | sean-k-mooney | i need to check if that will remove the VDPA device too or not | |
| 19:12:15 | gibi | so pci_dev.child_devices does have the VDPAs listed? | |
| 19:12:34 | gibi | OK | |
| 19:12:40 | sean-k-mooney | im not sure ill whitelist the PF on my test system and check | |
| 19:12:48 | gibi | cool | |
| 19:12:56 | gibi | I continue reading in the meantime | |
| 19:13:01 | sean-k-mooney | i plan to write some func test for this too but i need to extend our fixture support | |
| 19:17:29 | gibi | Do I understand correclty that not having the mlock extra spec in W is OK if we document to use VDPA always with huge pages? | |
| 19:17:57 | stephenfin | that's the thinking, yes | |
| 19:18:02 | stephenfin | huge pages or real-time | |
| 19:18:12 | sean-k-mooney | unfortunetly no. you would have to use hw:realtime or have a libvirt that does it for you | |
| 19:18:23 | sean-k-mooney | stephenfin: hugepages is not enough unfortunetly | |
| 19:18:31 | stephenfin | sean-k-mooney: oh, are they not locked by default? | |
| 19:18:43 | sean-k-mooney | they are unswapable but not locked | |
| 19:19:22 | sean-k-mooney | technially they can be moved if you configure your kernel to allow that | |
| 19:19:46 | stephenfin | sean-k-mooney: I've forgotten - have you a release note? | |
| 19:19:47 | sean-k-mooney | in practic they basially are but i still got the dma issue becuase qemu was not allowed lock the memory | |
| 19:20:11 | sean-k-mooney | stephenfin: not yet that was going to be in the patch that block the unsupported ops | |
| 19:20:13 | stephenfin | If not, that should be called out. I suspect libvirt will start doing this shortly since it's an obvious oversight but just in case they don't | |
| 19:20:16 | sean-k-mooney | im working on that now | |
| 19:20:19 | stephenfin | Okay, cool | |
| 19:20:27 | stephenfin | I've got a functional test almost done, btw | |
| 19:20:32 | stephenfin | lots of mocking | |
| 19:20:44 | stephenfin | artom is right - we're eventually going to reinvent libvirt :) | |
| 19:20:45 | sean-k-mooney | did you implement vdpa in the pci fixture? | |
| 19:20:48 | stephenfin | yeah | |
| 19:20:55 | gibi | yeah I wanted to ask for a reno and blockin of the unsupported ops | |
| 19:21:02 | stephenfin | I copied bauzas' pattern for mdevs | |
| 19:21:18 | stephenfin | there's cleanup needed, but I want to rework all this next cycle | |
| 19:21:29 | stephenfin | start using template files with a proper language like jinja2 | |
| 19:21:35 | stephenfin | instead of hacking them together like we're doing /o\ | |
| 19:21:47 | sean-k-mooney | gibi: so here is the testing i did yesterday https://review.opendev.org/c/openstack/nova/+/778350/7#message-0bb79aad698ad4acacfec150e80d58d7abc1d09b | |
| 19:22:17 | sean-k-mooney | im going to block all the ops that did not work with the 403 and same exception i have in that other patch | |
| 19:22:27 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/780065/1/nova/exception.py | |
| 19:23:25 | sean-k-mooney | that i was also goign to add a user dock liek we have for port resouce requests | |
| 19:23:38 | sean-k-mooney | https://docs.openstack.org/api-guide/compute/port_with_resource_request.html | |
| 19:24:34 | gibi | sean-k-mooney: nice | |
| 19:25:19 | gibi | sean-k-mooney: I finished reading the vdpa patches (I ignored mlock for now) I have nothing major blocking this. But I will not be around to review the incoming reno and ops blocking patch today | |
| 19:26:10 | sean-k-mooney | ya its getting late for you thanks for staying with it so far | |
| 19:26:22 | gibi | stephenfin, sean-k-mooney: is it OK for you if we treat this now as a possible FFE | |
| 19:26:25 | gibi | ? | |
| 19:26:42 | gibi | I can do the remaining reviews tomorrow morning | |
| 19:26:45 | sean-k-mooney | i was not sure if we were going to have any but if you would be ok with that then yes | |
| 19:26:46 | stephenfin | sure, wfm. It'd be nice to be able to get dinner :) | |
| 19:27:11 | gibi | OK, then I will check where we are tomorrow morning | |
| 19:27:15 | sean-k-mooney | i can email the list and formally request one if you like | |
| 19:27:22 | gibi | sean-k-mooney: yes please | |
| 19:27:27 | sean-k-mooney | ok will do | |
| 19:27:37 | gibi | you can refer to me and stephenfin as supporters for the FFE | |
| 19:27:51 | gibi | this is really close and it is useful | |
| 19:28:06 | gibi | so I'm willing to spend timeon this tomorrow and early next week | |