| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-24 | |||
| 09:15:52 | sean-k-mooney | that will be the most similar | |
| 09:15:56 | pvc_ | okay wait | |
| 09:16:33 | pvc_ | cat nvidia-84/name GRID P100-2Q | |
| 09:17:15 | sean-k-mooney | cool so if you set that in the nova.conf and boot a new vm it should work with cuda | |
| 09:17:37 | sean-k-mooney | you shoudl look at https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#vgpu-types-tesla-p100 and determin which one meets your needs | |
| 09:17:44 | pvc_ | may i know where i can find that information? | |
| 09:17:44 | pvc_ | wait | |
| 09:17:46 | pvc_ | thank you i'll try | |
| 09:18:38 | pvc_ | what docs can i read to determine where cuda will run? | |
| 09:18:58 | sean-k-mooney | all the vGPU types that end in Q support cuda acordeing to https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#cuda-open-cl-support-vgpu. | |
| 09:19:08 | sean-k-mooney | its the same doc just later in section 1.6.2 | |
| 09:19:33 | pvc_ | wow that's so cool :) | |
| 09:19:35 | pvc_ | thank you so much | |
| 09:19:43 | pvc_ | i'll lauch again an instance | |
| 09:20:43 | pvc_ | can i use P100-1Q sean-k-mooney? | |
| 09:20:49 | sean-k-mooney | so decodeing this a bit more it appears the number before the Q in there name ing is the amound of ram so the p100-2q has 2GB of fram buffer so you can run 8 of them on the 16GB p100 | |
| 09:20:51 | sean-k-mooney | yes | |
| 09:21:11 | sean-k-mooney | that will simply have 1GB of vRAM allocated to its framebuffer instead | |
| 09:21:29 | sean-k-mooney | that will allow you to run 16 vms on each phyicl p100 | |
| 09:21:32 | pvc_ | 2Q is better right | |
| 09:21:39 | pvc_ | so i can launch 8 instances with 2Q | |
| 09:21:45 | sean-k-mooney | yep | |
| 09:21:57 | pvc_ | i have 2 tesla p100 | |
| 09:22:13 | sean-k-mooney | 8 with 2Q 4 with 4Q 2 with 8Q and 1 with 16Q | |
| 09:22:21 | pvc_ | http://paste.openstack.org/show/732950/ | |
| 09:22:41 | sean-k-mooney | so its a trade off between number of vms you can run and performcne that each vm will have | |
| 09:25:10 | sean-k-mooney | pvc_: what software license do you have by the way. | |
| 09:25:38 | sean-k-mooney | you will need the Quadro vDWS license to be able to used the higher performance Q series vgpu types | |
| 09:26:27 | pvc_ | i didnt use any license for now | |
| 09:26:29 | pvc_ | do i need to install it | |
| 09:27:11 | pvc_ | but i think i have the license | |
| 09:27:31 | sean-k-mooney | acording to https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#licensing-grid-vgpu cuda will be disabled untill you add the license key | |
| 09:28:24 | pvc_ | sean-k-mooney http://paste.openstack.org/show/732952/ | |
| 09:29:07 | pvc_ | licensing on compute node or on the instance? | |
| 09:29:47 | sean-k-mooney | you have to install the lisening server somewhere on your network and then i belive you need to point your instance at the licensing server | |
| 09:30:24 | sean-k-mooney | its all covered in https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#licensing-grid-vgpu | |
| 09:31:19 | pvc_ | can i just install in on my compute node? | |
| 09:31:26 | pvc_ | is that possible | |
| 09:31:52 | sean-k-mooney | pvc_: no not as far as i can tell | |
| 09:32:41 | sean-k-mooney | what you can do is configure the adresss of the licening server once then snapshot the vm and use the new image as your base image in glance for tenants | |
| 09:33:30 | pvc_ | wait i just install the licensing server | |
| 09:37:07 | bauzas | sean-k-mooney: pvc_: sorry was on meeting | |
| 09:37:16 | sean-k-mooney | pvc_: if you are using linux guest looks like you can also just add a config file and enable the nvidia-gridd service https://docs.nvidia.com/grid/latest/grid-licensing-user-guide/index.html#licensing-grid-software-linux-config-file | |
| 09:37:22 | sean-k-mooney | bauzas: no worries | |
| 09:37:26 | bauzas | sean-k-mooney: thanks for helping pvc_, all of what you said is valid :) | |
| 09:37:45 | bauzas | nvidia requires a specific license for CUDA, and implies a GPU profile | |
| 09:39:00 | pvc_ | im using 90days trial, may i know if it have a free license? | |
| 09:40:36 | sean-k-mooney | pvc_: the trial shoudl work for now but you will need a Quadro vDWS license | |
| 09:40:53 | sean-k-mooney | pvc_: the priceing is covered here https://images.nvidia.com/content/grid/pdf/161207-GRID-Packaging-and-Licensing-Guide.pdf | |
| 09:41:47 | pvc_ | can i continue without license? | |
| 09:41:49 | pvc_ | or do i need to buy | |
| 09:41:56 | sean-k-mooney | ouch $450 per concurrent user for perpetual license | |
| 09:42:50 | sean-k-mooney | pvc_: with out cuda suppor and a frame rate cap of 3 frames pre second and a reduced frame rate yes but realitically no you need a licese | |
| 09:47:33 | pvc_ | as per nvidia support yes, The evaluation license provides access to 128 CCU's of NVIDIA Quadro Virtual Data Center Workstation (Quadro vDWS) edition for up to 90 days. | |
| 09:49:30 | bauzas | pvc_: just to make it clear, the client licensing is in https://docs.nvidia.com/grid/6.0/grid-licensing-user-guide/index.html | |
| 09:58:54 | openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: VIF info and utils for image as template https://review.openstack.org/612974 | |
| 09:59:48 | openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: Inventory path utils for image as template https://review.openstack.org/612976 | |
| 10:15:26 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Fail to live migration if instance has a NUMA topology https://review.openstack.org/611088 | |
| 10:19:22 | openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: OVA and StrOpt images as VM templates https://review.openstack.org/609736 | |
| 10:22:30 | BlackDex | Hello there. I'm seeing some invalid (RFC Validation) json blob data in the instance_extra table of nova. It has nested json data and also a key is not quoted. | |
| 10:22:55 | BlackDex | is this by design? | |
| 10:26:50 | BlackDex | wel not nested, but multiple root elements actually | |
| 10:53:45 | openstackgerrit | Fan Zhang proposed openstack/nova master: Retry after hitting libvirt error VIR_ERR_OPERATION_INVALID in live migration. https://review.openstack.org/612272 | |
| 11:59:43 | jangutter | sean-k-mooney: in os-vif (neutron api v3), what's going to be the names of the two sides passing objects? The docs refer to 'provider host' and 'networking host'? (respectively on compute and controller, I presume). | |
| 12:06:45 | sean-k-mooney | am which doc? i have not worked on the spec for this in like 18 months | |
| 12:07:44 | sean-k-mooney | the two entities are the neutron api (sepcificaly the ml2 drivers which handel the binding request) and the nova compute agent | |
| 12:09:44 | sean-k-mooney | the intent was that nova compute agent would pass a filtered host info object(in a serialised form) to neutron as part of port binding and the neutron ml2 driver would bind the port and respond with a serialise os-vif vif object | |
| 12:10:10 | jangutter | sean-k-mooney: I was looking at the vestigial docs in os-vif itself. and I understood it as you describe it too. | |
| 12:10:28 | sean-k-mooney | oh ok cool | |
| 12:11:16 | sean-k-mooney | i had previously debated createing a request and responce object pair in os-vif also | |
| 12:12:07 | sean-k-mooney | i think the details are something we can iterate on when we actully move forward on this. | |
| 12:15:25 | jangutter | sean-k-mooney: yep, just wanted to kinda capture the proposed intent and sequence a bit clearer. | |
| 12:16:51 | sean-k-mooney | cool | |
| 12:17:20 | sean-k-mooney | anything else you wanted to know on that topic or did i cover it above? | |
| 12:18:56 | jangutter | sean-k-mooney: I think it's sufficient, I'm not implementing the entire sequence here, basically just roughly sketching out things like sequence, the direction of filtering, and entities in the system. | |
| 12:19:43 | sean-k-mooney | cool so you pulling to geter all the required reading to write a spec to adress it :) | |
| 12:21:15 | jangutter | sean-k-mooney: heh, writing a paragraph in the doc explaining: hey, this is a stub, this is why it's a stub, and it might look like this after it's unstubbed. | |
| 12:26:56 | openstackgerrit | Daniel Abad proposed openstack/nova master: Fix ironic client ironic_url deprecation warning https://review.openstack.org/611872 | |
| 12:28:57 | openstackgerrit | Daniel Abad proposed openstack/nova master: Fix ironic client ironic_url deprecation warning https://review.openstack.org/611872 | |
| 12:39:27 | openstackgerrit | Gaudenz Steinlin proposed openstack/nova master: Ignore misleading resource updates from virt driver https://review.openstack.org/523006 | |
| 12:44:16 | jangutter | sean-k-mooney: one more clarification, when you say "filtered host-info", what's the filter that's applied? Is it just filtering based on the plugin info gathered on the compute node, filtering out out-of-range plugins? | |
| 12:45:57 | sean-k-mooney | basicaly the idea was that nova would prefilter the host info object on 2 factor | |
| 12:46:19 | sean-k-mooney | first what plugins/viftypes were supported by the hypervior on that node | |
| 12:46:37 | sean-k-mooney | and second any aspect of the guest that would resitct what vif types could be used. | |
| 12:47:16 | jangutter | sean-k-mooney: I see, only info locally available at the compute node at the time, analogous to "capabilities". | |
| 12:47:30 | sean-k-mooney | e.g. vhost-user requires hugepages to work so if the flavor did not have a hugepage request it would be removed form the list | |
| 12:47:41 | sean-k-mooney | yep | |
| 12:49:51 | sean-k-mooney | the main motivaitoin is to enable nova to say based on my knoladge of the hypervior and the instance request(flavor and image metatdata) this is the set of vif types i could support | |
| 12:50:38 | sean-k-mooney | and then neutron can say ok form that set i can support Y and select an optimal vif type to use | |
| 12:50:57 | stephenfin | np | |
| 12:51:17 | sean-k-mooney | once we have that capablity we can potainilly schedule on that in the future too | |
| 12:52:02 | jangutter | sean-k-mooney: right, makes sense and saves a round-trip with mis-scheduling or a failed portbinding. | |
| 12:52:33 | sean-k-mooney | yep or worse in the vhost user case where the vm boots with no error and no networking | |
| 12:56:05 | jangutter | sean-k-mooney: even the qemu error in libvirt is tricky to trace and very misleading in that case. | |
| 12:56:50 | sean-k-mooney | jangutter: qemu does not provide an error in that case at least it did not in the past | |
| 12:57:30 | sean-k-mooney | the only error i have ever seen for this is a debug only error in dpdk logs related to not being able to map the memory | |
| 12:58:31 | sean-k-mooney | but in anycase it would allow us on both the nova and neutron side to filter down to only inteface we think should work instead of relying on the operator/deployer to get this right | |
| 12:59:08 | jangutter | sean-k-mooney: I remember finding something in the qemu logs about "falling back on userspace virtio" when that happened. No hard error. | |
| 13:00:38 | sean-k-mooney | really well the fallback does not work so i guess its nice they tried but ya i think we have all been bit by that at some point if we have used vhost-user | |
| 13:02:13 | jangutter | sean-k-mooney: one day, there'll be unscarred users, developers and operators. | |