| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-02 | |||
| 11:53:52 | gibi | persistent | |
| 11:53:58 | sean-k-mooney | it is | |
| 11:54:07 | gibi | pre-up echo '4' > /sys/class/net/enp129s0f0/device/sriov_numvfs | |
| 11:54:07 | gibi | so I put | |
| 11:54:08 | sean-k-mooney | but we could also protect against it eaisly in nova | |
| 11:54:23 | gibi | hm, how? | |
| 11:54:39 | gibi | does nova know what is the number of vfs to be re-created? | |
| 11:54:54 | sean-k-mooney | gibi: when we set teh vfs from unavaiable to avaiable after freeing the PF in the db we chould check with libvirt to make sure the exist | |
| 11:55:02 | sean-k-mooney | if they dont we can delete them in the db | |
| 11:55:24 | sean-k-mooney | or i guess we could count them and recreate them with that echo but deleteing seams simpler | |
| 11:55:43 | gibi | ohh, yes, we could sync our db with reality | |
| 11:55:48 | sean-k-mooney | yep | |
| 11:55:55 | gibi | I don't want to recreate anything host level | |
| 11:56:04 | sean-k-mooney | so syncing with reality shoudl be simple | |
| 11:56:08 | gibi | I see | |
| 11:56:40 | sean-k-mooney | the other thning is that unless your udev rules are correct when you unbind the pf form the vm | |
| 11:56:52 | sean-k-mooney | the pf might not bind back to the netdev vendor driver | |
| 11:57:03 | sean-k-mooney | it can stay bound to vfio-pci | |
| 11:57:22 | sean-k-mooney | in that case the vf wont be recreated by a pre-up command | |
| 11:57:22 | songwenping_ | bauzas: hi, i'm test vGPU with VM and meet some problems. I have install nvidia-smi driver and my GPU's already set nvidia driver, but i cannot find mdev_bus at /sys/class path. | |
| 11:57:35 | sean-k-mooney | but that is definetly out of nova scope | |
| 11:57:59 | sean-k-mooney | songwenping_: what kernel are you using | |
| 11:58:10 | gibi | sean-k-mooney: ack | |
| 11:58:20 | sean-k-mooney | songwenping_: also i dont think that is where it is | |
| 11:58:32 | bauzas | songwenping_: have you installed the nvidia driver in the compute service ? | |
| 11:58:58 | songwenping_ | i use ubuntu 18.04. | |
| 11:59:09 | songwenping_ | my centos env is ok. | |
| 11:59:12 | sean-k-mooney | songwenping_: i think its /sys/bus/mdev not /sys/class/mdev but its only created after you modporbe vfio-mdev | |
| 12:00:14 | sean-k-mooney | yep sean@pop-os:~$ ls /sys/bus/mdev/ | |
| 12:00:15 | sean-k-mooney | devices drivers drivers_autoprobe drivers_probe uevent | |
| 12:00:20 | songwenping_ | sean-k-mooney: also not find at /sys/bus/mdev | |
| 12:00:45 | sean-k-mooney | yep you need to do "sudo modprobe vfio-mdev" | |
| 12:00:49 | sean-k-mooney | to create it | |
| 12:01:16 | sean-k-mooney | wait is this a ubuntu 18.04 host | |
| 12:01:19 | sean-k-mooney | or guest | |
| 12:01:27 | songwenping_ | host | |
| 12:01:40 | sean-k-mooney | nvidia annoyingly does not have ubuntu host drivers | |
| 12:01:49 | sean-k-mooney | for virtual gpus | |
| 12:02:00 | sean-k-mooney | they only suppor ubuntu guests | |
| 12:03:04 | sean-k-mooney | so unless you used alien or something to install the rhel rpm on your ubuntu host i dont see how you can install the correct driver and kernel module | |
| 12:03:29 | sean-k-mooney | mdevs will work but not nvidia grid on a ubuntu host | |
| 12:04:24 | songwenping_ | i get nvidia-* last week and create vm with vgpu success with devstack on ubuntu 18.04 host. | |
| 12:04:40 | openstackgerrit | Johannes Kulik proposed openstack/nova master: vmware tests: Support different moref backend representations https://review.opendev.org/749488 | |
| 12:05:12 | bauzas | songwenping_: what sean-k-mooney said, nvidia doesn't support ubunutu as a compute OS | |
| 12:06:49 | jkulik | stephenfin, if you find some time, please have a look at https://review.opendev.org/749488 for the first patch for the oslo.vmware SOAP library switch. sorry for the long wait, downstream needed a lot more love than usual ... | |
| 12:06:54 | songwenping_ | i really get the nivida-* is at /sys/class/mdev_bus as centos. but now i cannot get it. | |
| 12:13:15 | sean-k-mooney | songwenping_: i think that class path will only exist if you have mdevs allocated | |
| 12:13:38 | sean-k-mooney | songwenping_: but as i said you cant use ubuntu as a host os for nvida grid | |
| 12:14:03 | sean-k-mooney | i can be the guest os but nvidia does not make the host level driver availabel for ubuntu | |
| 12:17:07 | songwenping_ | sean-k-mooney: yeah, i have allocated mdevs and i reinstall vgpu-kvm package. | |
| 12:17:55 | sean-k-mooney | songwenping_: where did you get the package for ubuntu? | |
| 12:18:38 | songwenping_ | nvidia official website | |
| 12:18:49 | songwenping_ | NVIDIA-Linux-x86_64-440.107-vgpu-kvm.run | |
| 12:19:07 | sean-k-mooney | https://docs.nvidia.com/grid/10.0/product-support-matrix/index.html | |
| 12:19:19 | sean-k-mooney | according to that ubnuntu is not supported as a host os | |
| 12:20:05 | songwenping_ | the path is /sys/class/mdev_bus/0000\:af\:00.0/mdev_supported_types/ generate, but nothing in it.:( | |
| 12:20:42 | sean-k-mooney | your going to have to talk to nvidia about that | |
| 12:22:27 | sean-k-mooney | the offical nividia doc which was last update on the 21st of august says ubunut is not supported i think you install the ubunut guest driver on the host | |
| 12:22:28 | songwenping_ | sean-k-mooney, bauzas: thanks a lot. i'll try to use ubuntu guest. | |
| 12:22:57 | openstackgerrit | Yao wang proposed openstack/nova master: Use VIR_DOMAIN_XML_INACTIVE for detach volume https://review.opendev.org/749489 | |
| 12:36:59 | bauzas | songwenping__: FWIW, for nvidia specific issues like it, please look at their documentation for vGPUs https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#grid-vgpu-introduction | |
| 12:37:36 | bauzas | the openstack community is not an nvidia support team :p | |
| 12:46:04 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: Add regression tests for bug #1889108 https://review.opendev.org/749232 | |
| 12:46:04 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: compute: Don't delete the original attachment during pre LM rollback https://review.opendev.org/749233 | |
| 12:46:05 | openstack | bug 1889108 in OpenStack Compute (nova) train "failures during driver.pre_live_migration remove source attachments during rollback" [Undecided,In progress] https://launchpad.net/bugs/1889108 - Assigned to Lee Yarwood (lyarwood) | |
| 12:47:07 | stephenfin | jkulik: sure, done | |
| 13:46:04 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Resolve issues with various HTTP status codes https://review.opendev.org/749505 | |
| 13:46:43 | stephenfin | gmann, sean-k-mooney, gibi: There's a potential solution to our "no microversion in Victoria" conundrum ^ If that's not completely bonkers, I could probably finish it today | |
| 13:47:02 | stephenfin | though, in fairness, it probably is completely bonkers :) | |
| 13:47:13 | gibi | stephenfin: ack, will look. | |
| 13:47:33 | stephenfin | fair fair | |
| 13:51:35 | bauzas | gibi: sean-k-mooney: /me needs help of a network specialist | |
| 13:52:44 | gibi | bauzas: I'm not sure I can help but ask :) | |
| 13:53:14 | bauzas | gibi: for instance create, we're passing requested_networks here https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/compute/api.py#L954 | |
| 13:53:37 | bauzas | it's a NetworkRequestList object | |
| 13:54:06 | sean-k-mooney | :) | |
| 13:54:13 | bauzas | https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/objects/network_request.py#L30 | |
| 13:54:46 | bauzas | so I can try to add a new field in some object that would be NetworkRequestList | |
| 13:54:47 | sean-k-mooney | stephenfin: our you could just add a microverion for the api removal too | |
| 13:55:02 | bauzas | sean-k-mooney: gibi: for example, directly the RequestSpec | |
| 13:55:03 | stephenfin | sean-k-mooney: it would be no-op though | |
| 13:55:22 | bauzas | sean-k-mooney: gibi: but if so, I need to use the same for the other move operations | |
| 13:55:24 | sean-k-mooney | well no after the version you would get 404 for the apis | |
| 13:55:38 | sean-k-mooney | stephenfin: ill take a look at your patch after bauzas thing | |
| 13:55:45 | stephenfin | right, but before the version you'd get nothing too | |
| 13:55:48 | bauzas | sean-k-mooney: gibi: but then, we no longer have NetworkRequests, just VIF objects | |
| 13:55:53 | stephenfin | since XenAPI is gone and those things only worked with XenAPI | |
| 13:56:16 | sean-k-mooney | stephenfin: oh i guess ya | |
| 13:56:17 | gibi | bauzas: yeah, so for boot you have network request for move you have either the cache or you can ask neutron for the bound ports of the instance | |
| 13:56:19 | bauzas | sean-k-mooney: gibi: so I wonder if I can create a new NetworkRequest object by using a VIF object | |
| 13:56:40 | gmann | stephenfin: ack, I will check in afternoon | |
| 13:56:53 | bauzas | sean-k-mooney: gibi: like we do in https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/objects/request_spec.py#L547 | |
| 13:57:24 | sean-k-mooney | the vifs would have everything expction the pci_request_id i think | |
| 13:57:26 | bauzas | sean-k-mooney: gibi: tbc, what's the difference between a VIF object and a NetworkRequest one | |
| 13:58:03 | bauzas | sean-k-mooney: right, see https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/network/neutron.py#L3064-L3076 | |
| 13:58:14 | sean-k-mooney | no tags or pci requests https://github.com/openstack/nova/blob/master/nova/network/model.py#L380-L404 | |
| 13:58:17 | gibi | I never tried to diff them, but I honeslty affraid of cached data as it might be stale | |
| 13:58:24 | sean-k-mooney | but it will have the network id and port id | |
| 13:59:08 | gibi | I might try to recreate the network request from neutron | |
| 13:59:21 | bauzas | gibi: do we persist it ? | |