| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-02 | |||
| 14:15:58 | sean-k-mooney | FYI we may need to rework how we do memory tracking to fix a previously unknow aspect of pci passhtough | |
| 14:16:16 | bauzas | gibi: once you begin to look at the RPC API, ping me and I'll explain | |
| 14:16:34 | sean-k-mooney | ill try and file a bug for it when i have time but basically memory oversubsciption cant be done if you have pci passthough/sriov | |
| 14:16:36 | bauzas | gibi: tl;dr: I'm providing a 5.x proxy for supporting old clients | |
| 14:16:44 | sean-k-mooney | it might also affect vgpu | |
| 14:16:59 | bauzas | sean-k-mooney: ack | |
| 14:17:06 | bauzas | sean-k-mooney: vgpu or gpu ? | |
| 14:17:10 | sean-k-mooney | both | |
| 14:17:17 | bauzas | why ? | |
| 14:17:18 | sean-k-mooney | we will have to verify it | |
| 14:17:33 | bauzas | b/c vgpu is different from pci passthrough and sriov | |
| 14:17:39 | sean-k-mooney | bauzas: libivrt is locking the guest memory pages whenever we use pci passthoug | |
| 14:17:50 | sean-k-mooney | it may or may not be doing the same for vgpu | |
| 14:17:55 | bauzas | ah | |
| 14:18:05 | bauzas | test it then, yes | |
| 14:18:39 | bauzas | and like I discussed with you, I'd also like to work on pci attach/detach | |
| 14:18:50 | sean-k-mooney | generic attach ya | |
| 14:19:22 | sean-k-mooney | this came up in a call i had this morining for vdpa when i was trying to dig into why it needed the memory to be locked explictly | |
| 14:19:43 | sean-k-mooney | for vdpa libvirt does not do it implcitly which is why i got dma error | |
| 14:20:02 | sean-k-mooney | so i have not actully avalidated it myself yet | |
| 14:20:22 | sean-k-mooney | once i do ill try and write up my findings | |
| 14:21:04 | sean-k-mooney | im not sure ill have time to try and validate vgpu but when i fiture out how to check if teh vm memory is locked maybe you can try? | |
| 14:28:15 | bauzas | sean-k-mooney: sure, I can try to get some vgpu environment (hopefully) | |
| 15:00:08 | sean-k-mooney | vgpus dont work with ubuntu hosts correct at leat not the nvida version? | |
| 15:00:50 | sean-k-mooney | actully im using teh mainline 5.11 kernel so never mind it wont install on my host anyway | |
| 15:01:06 | sean-k-mooney | the vdpa host im using also has a t4 but i dont really have a way to test it there | |
| 15:02:25 | bauzas | sean-k-mooney: no, ubuntu is not supported by nvidia drivers but you can use intel gvt-g if you really want to test vgpus | |
| 15:02:44 | bauzas | and a i915 host if you have one by hand | |
| 15:03:09 | sean-k-mooney | i was just wondering if the host i have would work or not. am maybe my laptop i could test with libvirt directly i guess | |
| 15:03:21 | bauzas | that's what I did for i915 | |
| 15:03:28 | bauzas | I just used an old laptop for testing | |
| 15:03:37 | sean-k-mooney | it wont happen in the next 2 weeks so we can figure it out later. | |
| 15:04:09 | bauzas | that's actually the problem with gvt-g, you can't find good intel cards that are for production | |
| 15:04:28 | bauzas | all the i915 cards are for desktop lines | |
| 15:05:02 | bauzas | but hopefully, ROCI will help us | |
| 15:05:05 | sean-k-mooney | yes they removed integrated grapshic for the server line when they added it | |
| 15:05:27 | sean-k-mooney | im hoping that the new descreet intel gpus will support it | |
| 15:05:35 | bauzas | fwiw, devstack on rhel works okay to me | |
| 15:05:48 | bauzas | but you'd need to run another env | |
| 15:06:04 | sean-k-mooney | i need a newer kernel for the vdpa work so rhel was not an option | |
| 15:06:13 | sean-k-mooney | it does not have the vendor driver | |
| 15:06:33 | sean-k-mooney | 8.4 might but not 8.3 or even centos stream | |
| 15:07:06 | sean-k-mooney | thats why im using the mainline 5.11 kernel for development | |
| 15:08:15 | bauzas | ha | |
| 15:08:23 | bauzas | and what about fedora ? | |
| 15:08:51 | bauzas | in theory, nvidia RPMs should work but I never tested them | |
| 15:08:54 | sean-k-mooney | it has python 3.9 which breaks eventlests and openstack in general | |
| 15:09:35 | bauzas | the dependency saga. | |
| 15:09:54 | sean-k-mooney | i had to use ubuntu 20.10 as it was the only os i could fined with precomipled libvit/qemu new enough and had access to the mainlin kernel and python 3.8 | |
| 15:10:17 | sean-k-mooney | although i ended up compiling qemu form souce anyway due to a driver bug | |
| 15:11:35 | sean-k-mooney | but ya dependicies for really new hardware features is a pain. | |
| 15:13:24 | sean-k-mooney | * happy | |
| 15:41:28 | sean-k-mooney | bauzas: it look like they might have been wrong. libvirt is provide qemu the capablity to lock memory but it does not look like its using it | |
| 15:42:15 | kashyap | gibi: artom: When you get a min, I think I've addressed all the pressing concerns: https://review.opendev.org/c/openstack/nova/+/774240 | |
| 15:43:24 | gibi | kashyap: ack, I will check back | |
| 15:44:21 | kashyap | Thanks; see the small change log for PS10 ["Mar 01 12:54 PM"] -- the only diff in PS11 is to put back the line that I accidentally removed during rebase. | |
| 16:34:46 | gibi | bauzas: I left some questions in the RPC bump series https://review.opendev.org/c/openstack/nova/+/761452 but overall the patch looks good to me | |
| 16:35:02 | bauzas | thanks for the review, /me looks | |
| 16:35:18 | bauzas | gibi: tbc, we need to hold this change until FF | |
| 16:35:36 | gibi | bauzas: is there any RPC impacting change open I should be aware of? | |
| 16:35:39 | bauzas | but the more we review, the quickier we could merge it just after m-3 | |
| 16:35:57 | bauzas | gibi: some compute service bumps AFAIK | |
| 16:36:14 | gibi | true, service version will be in conflict if we merge those | |
| 16:36:29 | bauzas | https://review.opendev.org/c/openstack/nova/+/761452 has a lot of merge conflicts | |
| 16:36:44 | bauzas | so I'm prepared to rebase this change | |
| 16:37:09 | gibi | bauzas: ack. I will prioritize to land this after FF | |
| 16:37:25 | gibi | but we might get FFE requests | |
| 16:37:36 | gibi | with service bumps | |
| 16:37:45 | gibi | so it will be a tricky balance | |
| 16:38:08 | gibi | anyhow we have to be strict with FFEs due to sortness of time til RC1 | |
| 16:38:37 | gibi | brinzhang0: could you please check what would be the good accel_uuid paramter here https://review.opendev.org/c/openstack/nova/+/761452/9/nova/compute/manager.py#9481 ? | |
| 16:40:25 | bauzas | gibi: what we could discuss is whether this parameter should be mandatory | |
| 16:40:56 | sean-k-mooney | gibi: ya with one week i would say we likely dont have time in most case | |
| 16:40:58 | bauzas | same for all the other methods like rebuild | |
| 16:41:57 | gibi | bauzas: I think it should be mandatory as the impl uses accel_uuids to support cyborg devices, so if accel_uuids param is not passed then we loose cyborg supprot | |
| 16:42:00 | gibi | suport | |
| 16:42:01 | gibi | support even | |
| 16:42:12 | gibi | sean-k-mooney: yeah | |
| 16:42:55 | gibi | sean-k-mooney: we have two week between FF (Mar12) and the lat possible date of RC1 Mar26 | |
| 16:43:13 | gibi | I mean FF mar11 | |
| 16:43:17 | sean-k-mooney | gibi: the only reservatio ni would have for makign it mandatory is that not all driver need it or will support cyborg | |
| 16:43:17 | gibi | for nova | |
| 16:43:25 | sean-k-mooney | but other then that ya it could be | |
| 16:43:30 | bauzas | gibi: yeah for sure | |
| 16:43:40 | bauzas | gibi: we actually pass the parameter everytime now | |
| 16:43:41 | gibi | it is the compute manager interface not the virt driver interface | |
| 16:43:50 | bauzas | if so, it's mandatory | |
| 16:44:00 | bauzas | but if we don't, then it should continue to be optional | |
| 16:44:03 | sean-k-mooney | gibi: oh i though rc1 was the 18 cool still tight | |
| 16:44:06 | bauzas | sean-k-mooney: am i right? | |
| 16:44:30 | sean-k-mooney | bauzas: when working with cyborg instace we need to pass it | |
| 16:44:41 | sean-k-mooney | but for none cycboge isntace its not needed | |
| 16:44:55 | bauzas | sean-k-mooney: the question is, do we pass them everytime even without cyborg ? | |
| 16:45:01 | gibi | bauzas: I also think that we pass accel_uuids now via the RPC. if this is not a cyborg intance then we pass [] I hope | |
| 16:45:21 | bauzas | sean-k-mooney: meaning we could pass an empty list | |
| 16:45:34 | sean-k-mooney | we can pass an empty list yes | |
| 16:45:36 | bauzas | but we could have made it optional | |
| 16:45:47 | bauzas | instead of an empty list | |
| 16:45:54 | sean-k-mooney | we cant make kwards defualt to empty list | |
| 16:46:00 | bauzas | like, you don't use cyborg, accel_uuids be None | |