| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-09-17 | |||
| 14:48:50 | mriedem | <HYPERVISOR_HOSTNAME>_VGPU_<TYPE_NAME> | |
| 14:50:19 | dansmith | mriedem: yauntmeta fix up my grammar nits, or just stack something on top? | |
| 14:51:19 | mriedem | i'll fix them quick | |
| 14:52:54 | dansmith | aight | |
| 14:58:07 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Add support specify volume type when boot instance https://review.openstack.org/579520 | |
| 14:58:10 | mriedem | dansmith: done | |
| 15:00:15 | dansmith | mriedem: also done | |
| 15:01:30 | dansmith | mriedem: heading to the twitters | |
| 15:02:21 | dansmith | https://twitter.com/get_offmylawn/status/1041703857460412419 | |
| 15:02:39 | mriedem | heh | |
| 15:02:41 | mriedem | oh chet | |
| 15:10:22 | bauzas | mriedem: sorry missed your ping | |
| 15:10:31 | bauzas | mriedem: yup, I guess, lemme check notes on this | |
| 15:11:07 | bauzas | mriedem: there was an upgrade concern, but I feel this should only be written for the multiple types support | |
| 15:11:15 | bauzas | I feel => I'm sure | |
| 15:11:56 | mriedem | yes multiple gpu types can't be supported until the reshape is done | |
| 15:12:10 | bauzas | mriedem: I reproposed the spec FWIW | |
| 15:12:28 | bauzas | mriedem: https://review.openstack.org/#/c/602474/ | |
| 15:13:06 | bauzas | jaypipes: I know you had concerns on https://review.openstack.org/#/c/602474/ but you eventually got a +2 | |
| 15:13:31 | bauzas | jaypipes: my take on this is that I'm fine if someone writes something using YAML | |
| 15:13:43 | bauzas | jaypipes: I'll then just use it for multiple VGPU types | |
| 15:13:56 | bauzas | (I mean, speaking of the inventory) | |
| 15:14:38 | bauzas | if by Stein, nobody proposes it, then I could consider proposing myself for Train | |
| 15:14:54 | mriedem | resource provider name looks like it's just a string in the api schema, | |
| 15:14:55 | bauzas | for the YAML inventory recording | |
| 15:15:02 | mriedem | wonder if we should replace whitespaces with underscores just to be safe | |
| 15:15:18 | mriedem | or remove whitespaces | |
| 15:15:23 | bauzas | mriedem: I think we agreed on underscores, nope ? | |
| 15:15:36 | mriedem | the etherpad says <HYPERVISOR_HOSTNAME>_VGPU_<TYPE_NAME> | |
| 15:15:43 | mriedem | the conf docs show: | |
| 15:15:44 | mriedem | <HYPERVISOR_HOSTNAME>_VGPU_<TYPE_NAME> | |
| 15:15:46 | mriedem | oops | |
| 15:15:50 | mriedem | enabled_vgpu_types = GRID K100,Intel GVT-g,MxGPU.2,nvidia-11 | |
| 15:16:02 | bauzas | hah, that | |
| 15:16:08 | mriedem | so you could have foo.bar.something_VGPU_GRID K100 | |
| 15:16:34 | bauzas | for libvirt, there is (AFAIK) only "vendorshit-number" | |
| 15:16:52 | bauzas | the above example was for Xen | |
| 15:17:09 | bauzas | but yeah, we could try to replace | |
| 15:18:30 | bauzas | https://libvirt.org/drvnodedev.html#MDEVCap | |
| 15:18:37 | bauzas | "The element has one attribute id which holds an official vendor-supplied identifier for the type. Since 3.4.0" | |
| 15:18:56 | bauzas | I don't know what "official vendor-supplied ID" means :) | |
| 15:19:28 | bauzas | "official" and "vendor-supplied" seem opposite to me | |
| 15:23:14 | mriedem | i'm just going to pass the configured type through | |
| 15:23:31 | bauzas | mriedem: uh, I'm being told that this string is just passed thru by the vendor driver to the kernel, period | |
| 15:23:47 | bauzas | mriedem: so there is litterally no formatting | |
| 15:23:48 | mriedem | and i'm passing it through to placement | |
| 15:23:57 | bauzas | and passing to operators | |
| 15:24:05 | bauzas | that's awesome | |
| 15:24:11 | mriedem | well the operator is the one that configures nova with the type | |
| 15:24:12 | mriedem | so yeah | |
| 15:26:08 | jaypipes | bauzas: done. | |
| 15:26:55 | jaypipes | bauzas: by "done", I mean I've re-reviewed the Stein vGPU types spec. | |
| 15:28:24 | bauzas | jaypipes: gotcha, and thanks | |
| 15:28:34 | bauzas | jaypipes: yeah, I think your comment is all good with me | |
| 15:30:08 | bauzas | jaypipes: tbc, I think I can discover certain things like framebuffer size and what's described in https://docs.nvidia.com/grid/6.0/grid-vgpu-user-guide/index.html#vgpu-types-tesla-m60 and others | |
| 15:30:26 | bauzas | jaypipes: but GPU capabilities will have to be described by operators directly | |
| 15:31:56 | jaypipes | bauzas: what is the user asking for? I've only seen requests for things like "my application is built with CUDA library X and therefore can take advantage of NVIDIA Compute Capability Y hardware, so make sure I get on a GPU that has that Compute Capability". | |
| 15:32:26 | bauzas | yeah probably | |
| 15:32:41 | bauzas | jaypipes: that's where your idea of an YAML inventory could help | |
| 15:33:05 | bauzas | but for the moment, I'm not happy with nova supporting vendor-specific features by code | |
| 15:33:24 | bauzas | anyway, speaking of code is better with code | |
| 15:36:36 | openstackgerrit | Merged openstack/nova-specs master: Add support specify volume type when boot instance https://review.openstack.org/579520 | |
| 15:38:00 | mriedem | libvirt reshaper patch updated | |
| 15:38:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: implement reshaper for vgpu https://review.openstack.org/599208 | |
| 15:38:13 | mriedem | i'm pondering how useful a pre-upgrade check could be for this | |
| 15:38:30 | mriedem | as in, i'm not sure how useful it would be since it's not something you can run offline | |
| 15:39:01 | mriedem | a pre-upgrade check would mostly be for reporting - yes you have x number of providers of these things and you're going to need to upgrade those via reshaper | |
| 15:40:16 | mriedem | although...why can't we run this offline? if the vgpu inventory is already on the root compute node provider and has allocations, we can call the reshaper API - the only thing we'd need is the gpu type and we can get that from config | |
| 15:40:26 | mriedem | lyarwood: dansmith: ^ | |
| 15:40:52 | dansmith | each compute node can have different types, right? | |
| 15:41:09 | dansmith | we need the quantity from the virt driver at least | |
| 15:41:15 | dansmith | we know which types, but not how many of each | |
| 15:41:20 | dansmith | right now we expose N of one type, IIRC | |
| 15:41:20 | mriedem | you'd have to run the data migration per compute yes | |
| 15:41:29 | mriedem | like the ironic instance flavor one | |
| 15:41:37 | dansmith | but you still need data from the compute, not just the types the compute is going to use, AFAIK | |
| 15:41:45 | mriedem | isn't that already in placement? | |
| 15:41:50 | mriedem | via the inventory record? | |
| 15:42:07 | dansmith | for one type, but not the others | |
| 15:42:10 | dansmith | bauzas: right? | |
| 15:42:34 | mriedem | we don't support multiple types yet | |
| 15:42:44 | mriedem | so one vgpu inventory record at most per compute node provider | |
| 15:42:46 | dansmith | right, but this is to get us there right? | |
| 15:42:51 | mriedem | yes | |
| 15:42:52 | dansmith | or your idea is to reshape just the one inventory, | |
| 15:42:55 | mriedem | yes | |
| 15:43:00 | mriedem | that's what my patch above does | |
| 15:43:01 | dansmith | and then let the compute fill it itn? | |
| 15:43:27 | dansmith | so, that might work, | |
| 15:43:35 | sean-k-mooney | dansmith: one of the issues with vgpus is that while you can have multilpel vgpus on the same pcpu like vm flavors dependng on what you have already created you may not be able to create others since they all share the same resouces | |
| 15:43:51 | dansmith | yeah, | |
| 15:44:07 | dansmith | so can't we have created some vgpus for guests on different cards that now need to be represented separately but weren't before? | |
| 15:44:15 | dansmith | and without a view of the virt driver, that might not be doable | |
| 15:45:08 | sean-k-mooney | dansmith: perhaps i belive today bauzas has confied the compute agent to only createing 1 vgpu type per host so it might not be be an issue | |
| 15:45:16 | bauzas | dansmith: you can get the supported type by looking up the conf optionj | |
| 15:45:56 | dansmith | type per host | |
| 15:45:57 | bauzas | sean-k-mooney: that's correct, we only support one type | |
| 15:45:58 | mriedem | if you changed the configured support vgpu type after creating some inventory/allocations but before reshaping, then we'd have a problem | |
| 15:46:09 | bauzas | mriedem: yeah that's a limitation | |
| 15:46:13 | dansmith | bauzas: one type per host, but potentially across multiple cards right? | |