| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-23 | |||
| 22:27:43 | sean-k-mooney | ya which is a pain | |
| #openstack-nova - 2018-10-24 | |||
| 01:40:02 | litao | hi | |
| 01:56:08 | openstackgerrit | liuming proposed openstack/nova master: Deletes evacuated instance files when source host is ok https://review.openstack.org/605987 | |
| 02:10:49 | openstackgerrit | Vu Cong Tuan proposed openstack/nova-specs master: Switch to stestr https://review.openstack.org/581284 | |
| 02:26:45 | alex_xu | gmann: sorry, I can't join office hour today | |
| 02:27:28 | gmann | alex_xu: ok, i will skip for today then. thanks for informing. | |
| 04:00:42 | openstackgerrit | Merged openstack/nova master: Fix up compute rpcapi version for pike release https://review.openstack.org/612231 | |
| 04:11:51 | openstackgerrit | melanie witt proposed openstack/nova stable/rocky: Fix up compute rpcapi version for pike release https://review.openstack.org/612561 | |
| 06:19:21 | openstackgerrit | Merged openstack/nova stable/rocky: Fix formatting non-templated cell URLs with no config https://review.openstack.org/611327 | |
| 07:33:32 | openstackgerrit | Yongli He proposed openstack/nova-specs master: add spec "show-server-numa-topology" https://review.openstack.org/612256 | |
| 07:46:44 | pvc | hi anyone sean-k-mooney or bauzas? | |
| 07:50:31 | pvc | i already launch an instance iwth vgpu on it | |
| 07:50:57 | pvc | but which driver should i use? | |
| 07:51:05 | pvc | to run a gpu application | |
| 07:53:56 | bauzas | good morning nova | |
| 08:18:25 | bauzas | pvc: you should use the grid guest driver | |
| 08:19:00 | bauzas | which was originally planned, but not lasting so long | |
| 08:33:55 | pvc | i've successfully run the nvidia-smi bauzas | |
| 08:34:01 | pvc | Product Name : GRID P100-2B4 | |
| 08:34:08 | bauzas | cool | |
| 08:34:10 | pvc | Virtualization mode : VGPU | |
| 08:34:17 | pvc | but i cannot test the tensorflow-gpu :( | |
| 08:34:31 | pvc | do you have any prefer docs for that | |
| 08:44:29 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Support delete_on_termination in volume attach api https://review.openstack.org/612949 | |
| 08:46:40 | openstackgerrit | Matthew Booth proposed openstack/nova master: Add regression test for bug 1550919 https://review.openstack.org/591733 | |
| 08:46:40 | openstack | bug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9) | |
| 08:46:40 | openstackgerrit | Matthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate https://review.openstack.org/578846 | |
| 08:47:40 | pvc_ | are you running tensorflow on your instance bauzas | |
| 08:54:59 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Support deleting data volume when destroy instance https://review.openstack.org/580336 | |
| 09:03:56 | pvc_ | hi bauzas u using cuda | |
| 09:06:43 | pvc_ | tensorflow.python.framework.errors_impl.InternalError: failed initializing StreamExecutor for CUDA device ordinal 0: Internal: failed call to cuDevicePrimaryCtxRetain: CUDA_ERROR_UNKNOWN: unknown error :( | |
| 09:07:28 | sean-k-mooney | pvc_: are you using one of the Q series mdev-types | |
| 09:07:29 | sean-k-mooney | https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#cuda-open-cl-support-vgpu | |
| 09:08:52 | pvc_ | NVIDIA Corporation GP100GL [Tesla P100 PCIe 16GB] | |
| 09:09:11 | sean-k-mooney | for you gpu https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#vgpu-types-tesla-p100 | |
| 09:09:59 | sean-k-mooney | you need to fine the mdev-type that correspond to one of P100-1Q P100-2Q P100-4Q P100-8Q or P100-16Q | |
| 09:10:08 | pvc_ | videocard of my instance: 00:05.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:15f8] (rev a1) | |
| 09:10:44 | sean-k-mooney | pvc_: yes that likely not going to change regardless of the mdev type yoou choose | |
| 09:12:26 | pvc_ | im using nvidia-211 | |
| 09:13:57 | sean-k-mooney | sure that does not help us map to the nvidia docs. you will need to look at the vendor data reported for the mdev type and see if you can find the vgpu type | |
| 09:14:31 | pvc_ | GRID P100-2B4 | |
| 09:14:35 | pvc_ | the name of nvidia-211 | |
| 09:15:02 | sean-k-mooney | right so that is a B series vgpu and does not support compute via opencl or cuda | |
| 09:15:46 | sean-k-mooney | pvc_: find the one corresponding to P100-2Q | |
| 09:15:52 | sean-k-mooney | that will be the most similar | |
| 09:15:56 | pvc_ | okay wait | |
| 09:16:33 | pvc_ | cat nvidia-84/name GRID P100-2Q | |
| 09:17:15 | sean-k-mooney | cool so if you set that in the nova.conf and boot a new vm it should work with cuda | |
| 09:17:37 | sean-k-mooney | you shoudl look at https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#vgpu-types-tesla-p100 and determin which one meets your needs | |
| 09:17:44 | pvc_ | wait | |
| 09:17:44 | pvc_ | may i know where i can find that information? | |
| 09:17:46 | pvc_ | thank you i'll try | |
| 09:18:38 | pvc_ | what docs can i read to determine where cuda will run? | |
| 09:18:58 | sean-k-mooney | all the vGPU types that end in Q support cuda acordeing to https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#cuda-open-cl-support-vgpu. | |
| 09:19:08 | sean-k-mooney | its the same doc just later in section 1.6.2 | |
| 09:19:33 | pvc_ | wow that's so cool :) | |
| 09:19:35 | pvc_ | thank you so much | |
| 09:19:43 | pvc_ | i'll lauch again an instance | |
| 09:20:43 | pvc_ | can i use P100-1Q sean-k-mooney? | |
| 09:20:49 | sean-k-mooney | so decodeing this a bit more it appears the number before the Q in there name ing is the amound of ram so the p100-2q has 2GB of fram buffer so you can run 8 of them on the 16GB p100 | |
| 09:20:51 | sean-k-mooney | yes | |
| 09:21:11 | sean-k-mooney | that will simply have 1GB of vRAM allocated to its framebuffer instead | |
| 09:21:29 | sean-k-mooney | that will allow you to run 16 vms on each phyicl p100 | |
| 09:21:32 | pvc_ | 2Q is better right | |
| 09:21:39 | pvc_ | so i can launch 8 instances with 2Q | |
| 09:21:45 | sean-k-mooney | yep | |
| 09:21:57 | pvc_ | i have 2 tesla p100 | |
| 09:22:13 | sean-k-mooney | 8 with 2Q 4 with 4Q 2 with 8Q and 1 with 16Q | |
| 09:22:21 | pvc_ | http://paste.openstack.org/show/732950/ | |
| 09:22:41 | sean-k-mooney | so its a trade off between number of vms you can run and performcne that each vm will have | |
| 09:25:10 | sean-k-mooney | pvc_: what software license do you have by the way. | |
| 09:25:38 | sean-k-mooney | you will need the Quadro vDWS license to be able to used the higher performance Q series vgpu types | |
| 09:26:27 | pvc_ | i didnt use any license for now | |
| 09:26:29 | pvc_ | do i need to install it | |
| 09:27:11 | pvc_ | but i think i have the license | |
| 09:27:31 | sean-k-mooney | acording to https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#licensing-grid-vgpu cuda will be disabled untill you add the license key | |
| 09:28:24 | pvc_ | sean-k-mooney http://paste.openstack.org/show/732952/ | |
| 09:29:07 | pvc_ | licensing on compute node or on the instance? | |
| 09:29:47 | sean-k-mooney | you have to install the lisening server somewhere on your network and then i belive you need to point your instance at the licensing server | |
| 09:30:24 | sean-k-mooney | its all covered in https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#licensing-grid-vgpu | |
| 09:31:19 | pvc_ | can i just install in on my compute node? | |
| 09:31:26 | pvc_ | is that possible | |
| 09:31:52 | sean-k-mooney | pvc_: no not as far as i can tell | |
| 09:32:41 | sean-k-mooney | what you can do is configure the adresss of the licening server once then snapshot the vm and use the new image as your base image in glance for tenants | |
| 09:33:30 | pvc_ | wait i just install the licensing server | |
| 09:37:07 | bauzas | sean-k-mooney: pvc_: sorry was on meeting | |
| 09:37:16 | sean-k-mooney | pvc_: if you are using linux guest looks like you can also just add a config file and enable the nvidia-gridd service https://docs.nvidia.com/grid/latest/grid-licensing-user-guide/index.html#licensing-grid-software-linux-config-file | |
| 09:37:22 | sean-k-mooney | bauzas: no worries | |
| 09:37:26 | bauzas | sean-k-mooney: thanks for helping pvc_, all of what you said is valid :) | |
| 09:37:45 | bauzas | nvidia requires a specific license for CUDA, and implies a GPU profile | |
| 09:39:00 | pvc_ | im using 90days trial, may i know if it have a free license? | |
| 09:40:36 | sean-k-mooney | pvc_: the trial shoudl work for now but you will need a Quadro vDWS license | |
| 09:40:53 | sean-k-mooney | pvc_: the priceing is covered here https://images.nvidia.com/content/grid/pdf/161207-GRID-Packaging-and-Licensing-Guide.pdf | |
| 09:41:47 | pvc_ | can i continue without license? | |
| 09:41:49 | pvc_ | or do i need to buy | |
| 09:41:56 | sean-k-mooney | ouch $450 per concurrent user for perpetual license | |
| 09:42:50 | sean-k-mooney | pvc_: with out cuda suppor and a frame rate cap of 3 frames pre second and a reduced frame rate yes but realitically no you need a licese | |
| 09:47:33 | pvc_ | as per nvidia support yes, The evaluation license provides access to 128 CCU's of NVIDIA Quadro Virtual Data Center Workstation (Quadro vDWS) edition for up to 90 days. | |
| 09:49:30 | bauzas | pvc_: just to make it clear, the client licensing is in https://docs.nvidia.com/grid/6.0/grid-licensing-user-guide/index.html | |
| 09:58:54 | openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: VIF info and utils for image as template https://review.openstack.org/612974 | |