Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-03-02
14:16:34 sean-k-mooney ill try and file a bug for it when i have time but basically memory oversubsciption cant be done if you have pci passthough/sriov
14:16:36 bauzas gibi: tl;dr: I'm providing a 5.x proxy for supporting old clients
14:16:44 sean-k-mooney it might also affect vgpu
14:16:59 bauzas sean-k-mooney: ack
14:17:06 bauzas sean-k-mooney: vgpu or gpu ?
14:17:10 sean-k-mooney both
14:17:17 bauzas why ?
14:17:18 sean-k-mooney we will have to verify it
14:17:33 bauzas b/c vgpu is different from pci passthrough and sriov
14:17:39 sean-k-mooney bauzas: libivrt is locking the guest memory pages whenever we use pci passthoug
14:17:50 sean-k-mooney it may or may not be doing the same for vgpu
14:17:55 bauzas ah
14:18:05 bauzas test it then, yes
14:18:39 bauzas and like I discussed with you, I'd also like to work on pci attach/detach
14:18:50 sean-k-mooney generic attach ya
14:19:22 sean-k-mooney this came up in a call i had this morining for vdpa when i was trying to dig into why it needed the memory to be locked explictly
14:19:43 sean-k-mooney for vdpa libvirt does not do it implcitly which is why i got dma error
14:20:02 sean-k-mooney so i have not actully avalidated it myself yet
14:20:22 sean-k-mooney once i do ill try and write up my findings
14:21:04 sean-k-mooney im not sure ill have time to try and validate vgpu but when i fiture out how to check if teh vm memory is locked maybe you can try?
14:28:15 bauzas sean-k-mooney: sure, I can try to get some vgpu environment (hopefully)
15:00:08 sean-k-mooney vgpus dont work with ubuntu hosts correct at leat not the nvida version?
15:00:50 sean-k-mooney actully im using teh mainline 5.11 kernel so never mind it wont install on my host anyway
15:01:06 sean-k-mooney the vdpa host im using also has a t4 but i dont really have a way to test it there
15:02:25 bauzas sean-k-mooney: no, ubuntu is not supported by nvidia drivers but you can use intel gvt-g if you really want to test vgpus
15:02:44 bauzas and a i915 host if you have one by hand
15:03:09 sean-k-mooney i was just wondering if the host i have would work or not. am maybe my laptop i could test with libvirt directly i guess
15:03:21 bauzas that's what I did for i915
15:03:28 bauzas I just used an old laptop for testing
15:03:37 sean-k-mooney it wont happen in the next 2 weeks so we can figure it out later.
15:04:09 bauzas that's actually the problem with gvt-g, you can't find good intel cards that are for production
15:04:28 bauzas all the i915 cards are for desktop lines
15:05:02 bauzas but hopefully, ROCI will help us
15:05:05 sean-k-mooney yes they removed integrated grapshic for the server line when they added it
15:05:27 sean-k-mooney im hoping that the new descreet intel gpus will support it
15:05:35 bauzas fwiw, devstack on rhel works okay to me
15:05:48 bauzas but you'd need to run another env
15:06:04 sean-k-mooney i need a newer kernel for the vdpa work so rhel was not an option
15:06:13 sean-k-mooney it does not have the vendor driver
15:06:33 sean-k-mooney 8.4 might but not 8.3 or even centos stream
15:07:06 sean-k-mooney thats why im using the mainline 5.11 kernel for development
15:08:15 bauzas ha
15:08:23 bauzas and what about fedora ?
15:08:51 bauzas in theory, nvidia RPMs should work but I never tested them
15:08:54 sean-k-mooney it has python 3.9 which breaks eventlests and openstack in general
15:09:35 bauzas the dependency saga.
15:09:54 sean-k-mooney i had to use ubuntu 20.10 as it was the only os i could fined with precomipled libvit/qemu new enough and had access to the mainlin kernel and python 3.8
15:10:17 sean-k-mooney although i ended up compiling qemu form souce anyway due to a driver bug
15:11:35 sean-k-mooney but ya dependicies for really new hardware features is a pain.
15:13:24 sean-k-mooney * happy
15:41:28 sean-k-mooney bauzas: it look like they might have been wrong. libvirt is provide qemu the capablity to lock memory but it does not look like its using it
15:42:15 kashyap gibi: artom: When you get a min, I think I've addressed all the pressing concerns: https://review.opendev.org/c/openstack/nova/+/774240
15:43:24 gibi kashyap: ack, I will check back
15:44:21 kashyap Thanks; see the small change log for PS10 ["Mar 01 12:54 PM"] -- the only diff in PS11 is to put back the line that I accidentally removed during rebase.
16:34:46 gibi bauzas: I left some questions in the RPC bump series https://review.opendev.org/c/openstack/nova/+/761452 but overall the patch looks good to me
16:35:02 bauzas thanks for the review, /me looks
16:35:18 bauzas gibi: tbc, we need to hold this change until FF
16:35:36 gibi bauzas: is there any RPC impacting change open I should be aware of?
16:35:39 bauzas but the more we review, the quickier we could merge it just after m-3
16:35:57 bauzas gibi: some compute service bumps AFAIK
16:36:14 gibi true, service version will be in conflict if we merge those
16:36:29 bauzas https://review.opendev.org/c/openstack/nova/+/761452 has a lot of merge conflicts
16:36:44 bauzas so I'm prepared to rebase this change
16:37:09 gibi bauzas: ack. I will prioritize to land this after FF
16:37:25 gibi but we might get FFE requests
16:37:36 gibi with service bumps
16:37:45 gibi so it will be a tricky balance
16:38:08 gibi anyhow we have to be strict with FFEs due to sortness of time til RC1
16:38:37 gibi brinzhang0: could you please check what would be the good accel_uuid paramter here https://review.opendev.org/c/openstack/nova/+/761452/9/nova/compute/manager.py#9481 ?
16:40:25 bauzas gibi: what we could discuss is whether this parameter should be mandatory
16:40:56 sean-k-mooney gibi: ya with one week i would say we likely dont have time in most case
16:40:58 bauzas same for all the other methods like rebuild
16:41:57 gibi bauzas: I think it should be mandatory as the impl uses accel_uuids to support cyborg devices, so if accel_uuids param is not passed then we loose cyborg supprot
16:42:00 gibi suport
16:42:01 gibi support even
16:42:12 gibi sean-k-mooney: yeah
16:42:55 gibi sean-k-mooney: we have two week between FF (Mar12) and the lat possible date of RC1 Mar26
16:43:13 gibi I mean FF mar11
16:43:17 gibi for nova
16:43:17 sean-k-mooney gibi: the only reservatio ni would have for makign it mandatory is that not all driver need it or will support cyborg
16:43:25 sean-k-mooney but other then that ya it could be
16:43:30 bauzas gibi: yeah for sure
16:43:40 bauzas gibi: we actually pass the parameter everytime now
16:43:41 gibi it is the compute manager interface not the virt driver interface
16:43:50 bauzas if so, it's mandatory
16:44:00 bauzas but if we don't, then it should continue to be optional
16:44:03 sean-k-mooney gibi: oh i though rc1 was the 18 cool still tight
16:44:06 bauzas sean-k-mooney: am i right?
16:44:30 sean-k-mooney bauzas: when working with cyborg instace we need to pass it
16:44:41 sean-k-mooney but for none cycboge isntace its not needed
16:44:55 bauzas sean-k-mooney: the question is, do we pass them everytime even without cyborg ?
16:45:01 gibi bauzas: I also think that we pass accel_uuids now via the RPC. if this is not a cyborg intance then we pass [] I hope
16:45:21 bauzas sean-k-mooney: meaning we could pass an empty list
16:45:34 sean-k-mooney we can pass an empty list yes
16:45:36 bauzas but we could have made it optional
16:45:47 bauzas instead of an empty list
16:45:54 sean-k-mooney we cant make kwards defualt to empty list
16:46:00 bauzas like, you don't use cyborg, accel_uuids be None
16:46:04 sean-k-mooney but we could default to None
16:46:17 sean-k-mooney and then pass [] if its none

Earlier   Later