| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-09-18 | |||
| 07:04:58 | bauzas | that's only for passthrough | |
| 07:05:06 | pvc | also the mdev bus | |
| 07:05:09 | bauzas | the driver looks good tho | |
| 07:05:17 | bauzas | you don't see the bus ? | |
| 07:05:42 | bauzas | remember me which physical card you have ? | |
| 07:05:53 | pvc | Nvidia Tesla P100 | |
| 07:06:10 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Transform libvirt.error notification https://review.openstack.org/484851 | |
| 07:06:36 | pvc | i enable mdev using modprove mdev | |
| 07:06:37 | pvc | then reboot the server wait | |
| 07:16:26 | bauzas | pvc: you don't need to enable the mdev bus by modprobe | |
| 07:16:42 | pvc | what i need to do sir bauzas? | |
| 07:17:19 | bauzas | pvc: in theory (and what I did on my dev box), you just need to disable the 'nouveau' module, then install the nvidia driver | |
| 07:17:31 | bauzas | and then reboot and you should see the mdev structure | |
| 07:17:37 | pvc | i already disable my nouveau | |
| 07:17:39 | bauzas | pvc: do you still have 'nouveau' ? | |
| 07:17:47 | bauzas | sure? lsmod | grep nouveau ? | |
| 07:17:55 | pvc | [root@ops-gpgpu1 ~]# lsmod | grep nou [root@ops-gpgpu1 ~]# | |
| 07:18:43 | pvc | http://paste.openstack.org/show/730230/ | |
| 07:21:40 | bauzas | lemme check my own driver | |
| 07:23:45 | pvc | Verify that the libvirtd service is active and running. | |
| 07:23:52 | pvc | my libvirtd is not running laso | |
| 07:24:10 | bauzas | wait a sec | |
| 07:24:38 | bauzas | I'm not very familiar with the nvidia packaging, but IIRC the "-grid" module is for the guests, not the host | |
| 07:24:56 | bauzas | my own driver is named "-kvm" | |
| 07:26:16 | bauzas | yup, that's correct, the "-grid" driver is for graphics https://docs.nvidia.com/grid/6.0/grid-vgpu-user-guide/index.html#installing-vgpu-drivers-linux | |
| 07:26:40 | pvc | i download the -grid on the 90 days trial site | |
| 07:28:54 | bauzas | pvc: see https://docs.nvidia.com/grid/6.0/grid-vgpu-user-guide/index.html#install-vgpu-package-generic-linux-kvm for the install docs | |
| 07:29:30 | pvc | it doesnt have kvm :( NVIDIA-Linux-x86_64-367.128-grid.run | |
| 07:30:02 | pvc | i think i download the wrong driver | |
| 07:30:27 | openstackgerrit | zhufl proposed openstack/nova master: Fix some typos in nova api ref doc https://review.openstack.org/603306 | |
| 07:32:25 | pvc | ongoing download of version 384 | |
| 07:32:31 | pvc | i cant curl or wget it on my linux machine :( | |
| 07:35:34 | bauzas | ok, I have another meeting, but let me know async | |
| 07:36:59 | pvc_ | hi bauzas im downloaing again another driver | |
| 07:55:20 | openstackgerrit | Tao Li proposed openstack/python-novaclient master: Remove the unused instance-name https://review.openstack.org/602520 | |
| 08:08:00 | naichuans_ | bauzas: please ping me when your are free. | |
| 08:08:37 | naichuans_ | bauzas: About n-rp and reshaper, which one will be merged first? | |
| 08:58:41 | kashyap | johnthetubaguy: gibi: Morning, can we put this change out of its misery -- https://review.openstack.org/#/c/565517 ("libvirt: Drop MIN_LIBVIRT_PF_WITH_NO_VFS_CAP_VERSION") | |
| 08:59:04 | bauzas | naichuans_: hello :) | |
| 08:59:06 | kashyap | It's a dead-simple clean-up | |
| 08:59:21 | naichuans_ | bauzas: Hi~ | |
| 09:00:00 | naichuans_ | Anything share with me about n-rp disscussion during PTG meeting? | |
| 09:00:42 | bauzas | naichuans_: sure, we could be discussing by Bluejeans | |
| 09:00:46 | bauzas | or Hangout ? | |
| 09:01:32 | naichuans_ | haha~ | |
| 09:01:32 | pvc | hi | |
| 09:02:18 | naichuans_ | bauzas: In truth I'm interested in the name mangling and reshaper | |
| 09:02:30 | naichuans_ | bauzas: for multiple vgpu type support | |
| 09:03:28 | pvc | anyone experience in using cyborg for vgpu? | |
| 09:05:09 | bauzas | naichuans_: that's why I think we could be discussing by Hangout directly | |
| 09:05:54 | naichuans_ | bauzas: Ah~ but how I reach you? | |
| 09:06:24 | naichuans_ | bauzas: Maybe I could give you a call? | |
| 09:06:35 | bauzas | naichuans_: so, we agreed during the PTG about names, but the problem is that when discussing with dansmith, we thought about modifying the children and rather have children RP that would be physical GPUs and not GPU types | |
| 09:06:55 | bauzas | hence mriedem's -W on https://review.openstack.org/#/c/599208/ | |
| 09:08:07 | naichuans_ | bauzas: got it. Then there maybe multiple vgpu rps with same type? | |
| 09:08:39 | naichuans_ | for they may have more than one gpu with same type on a host? | |
| 09:10:22 | bauzas | naichuans_: the problem is about NUMA affinity | |
| 09:11:04 | bauzas | naichuans_: see https://review.openstack.org/#/c/552924/ | |
| 09:11:27 | naichuans_ | bauzas: checking | |
| 09:11:29 | bauzas | naichuans_: if we have NUMA children, then pGPUs should be grand-children | |
| 09:11:49 | bauzas | for knowing the type, it'd then be a trait | |
| 09:11:58 | bauzas | and tbh, I think it's better | |
| 09:12:11 | naichuans_ | Oh, understood, bauzas: | |
| 09:12:25 | bauzas | because resource capacity is about some resource provider, which is a pGPU | |
| 09:13:04 | bauzas | the type itself isn't a provider of resources, just a feature | |
| 09:13:29 | bauzas | naichuans_: so I'm working on providing a new revision of https://review.openstack.org/#/c/599208/ by creating children being pGPUs | |
| 09:13:45 | naichuans_ | bauzas: Agree, it is more natural. But I'm not sure if there would be some issues involved in. | |
| 09:17:56 | naichuans_ | bauzas: I made a patch on xen according to Matt's patch:https://review.openstack.org/#/c/599208/. reshape works well | |
| 09:18:24 | naichuans_ | bauzas: https://review.openstack.org/#/c/521041/ | |
| 09:18:38 | naichuans_ | bauzas: but only for single gpu rp support | |
| 09:19:30 | naichuans_ | bauzas: Do we have a plane for reshaper on multiple gpu rp support? | |
| 09:21:56 | openstackgerrit | Chen proposed openstack/nova stable/queens: Fix soft deleting vm fails after "nova resize" vm https://review.openstack.org/603340 | |
| 09:22:08 | naichuans_ | bauzas: about gpu rp name mangling, it make no difference on xenserver, because gpu with same type would be put into the same gpu group. So base on gpu or gpu type is same | |
| 09:22:39 | bauzas | naichuans_: I'll look at your change for xen | |
| 09:22:53 | naichuans_ | bauzas: thanks. | |
| 09:23:41 | bauzas | naichuans_: we don't need to reshape for multiple GPU types | |
| 09:23:54 | bauzas | again, see my upgrade impact section on the spec | |
| 09:24:07 | naichuans_ | bauzas: will check it | |
| 09:26:42 | naichuans_ | bauzas: you mean this patch: https://review.openstack.org/#/c/552924/? | |
| 09:27:07 | bauzas | nope, the reproposed 'multiple GPU types' one | |
| 09:28:08 | naichuans_ | bauzas: will check it | |
| 09:29:28 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Merge extended_volumes extension response into server view builder https://review.openstack.org/596285 | |
| 09:33:24 | naichuans_ | bauzas: We restrict the user scenario to avoid those troubles~ Fair enough :) | |
| 09:33:45 | bauzas | naichuans_: which troubles ? :) | |
| 09:34:19 | openstackgerrit | Wenran Xiao proposed openstack/nova-specs master: Move juno approved specs https://review.openstack.org/603344 | |
| 09:35:13 | naichuans_ | bauzas: Directly upgrade the old vgpu allocation to multiple vgpu rp support is troublesome~ | |
| 09:35:57 | naichuans_ | bauzas: As I mentioned in the email, we have difficluties to decide the old allocation belongs. | |
| 09:36:38 | bauzas | naichuans_: ah this | |
| 09:36:40 | bauzas | naichuans_: yeah | |
| 09:36:52 | bauzas | naichuans_: well, about finding which is which, I don't see it difficult | |
| 09:37:01 | bauzas | but I could be wrong | |
| 09:37:14 | bauzas | I mean, allocations are done with a consumer UUID | |
| 09:37:33 | bauzas | we know that the consumer UUID is either an instance or a migration | |
| 09:37:57 | bauzas | given we don't support migrations yet in libvirt, that simplifies | |
| 09:38:32 | bauzas | so, the idea is to lookup the VGPU allocations, find for each which instance it is, check the libvirt XML to know which card is attached to, and give it back | |
| 09:38:46 | naichuans_ | bauzas: Yes, we need to talk with hypervisor, looks not so natural for me~ | |
| 09:39:03 | bauzas | naichuans_: I already do that for rebooting the host | |
| 09:39:33 | bauzas | naichuans_: given mdevs aren't persisted on reboots, every compute restart checks the existing allocations and recreates the mdevs | |
| 09:40:14 | naichuans_ | bauzas: awesome~ | |
| 09:40:56 | bauzas | naichuans_: I need to go to the gym, but I can help you by showing you the libvirt codee | |