| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-15 | |||
| 13:35:42 | openstackgerrit | Vladyslav Drok proposed openstack/nova master: Placement: allow to set reserved value equal to total for inventory https://review.openstack.org/564838 | |
| 13:35:43 | openstackgerrit | Vladyslav Drok proposed openstack/nova master: ironic: Report resources as reserved when needed https://review.openstack.org/517921 | |
| 13:38:12 | openstackgerrit | Vladyslav Drok proposed openstack/nova master: Placement: allow to set reserved value equal to total for inventory https://review.openstack.org/564838 | |
| 13:38:23 | openstackgerrit | Vladyslav Drok proposed openstack/nova master: Ironic: report 0 for vcpus/memory_mb/disk_gb resources https://review.openstack.org/565841 | |
| 13:38:44 | openstackgerrit | Vladyslav Drok proposed openstack/nova master: ironic: Report resources as reserved when needed https://review.openstack.org/517921 | |
| 13:39:17 | openstackgerrit | Vladyslav Drok proposed openstack/nova master: Ironic: report 0 for vcpus/memory_mb/disk_gb resources https://review.openstack.org/565841 | |
| 13:43:29 | openstackgerrit | Merged openstack/osc-placement master: CLI for traits (v1.6) https://review.openstack.org/514643 | |
| 14:09:28 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Move image conversion to privsep. https://review.openstack.org/554437 | |
| 14:09:29 | openstackgerrit | Matt Riedemann proposed openstack/nova master: We don't need utils.trycmd any more. https://review.openstack.org/554439 | |
| 14:09:30 | openstackgerrit | Matt Riedemann proposed openstack/nova master: We no longer need rootwrap. https://review.openstack.org/554438 | |
| 14:11:39 | bauzas | FWIW, /me works on a one-slide about explaining Placement API :) | |
| 14:11:54 | bauzas | efried: jaypipes: cdent: edleafe: ^ ;) | |
| 14:12:29 | efried | bauzas: FYI, edleafe and I have... uhh... more than one slide explaining the Placement API. | |
| 14:12:30 | edleafe | bauzas: efried does this in 30 slides :) | |
| 14:13:03 | efried | bauzas: For what audience? | |
| 14:13:13 | bauzas | edleafe: efried: sure but it's for https://www.openstack.org/summit/vancouver-2018/summit-schedule/events/20802/call-it-real-virtual-gpus-in-nova | |
| 14:13:54 | bauzas | I started my slides with https://docs.google.com/presentation/d/1HNXbT1awinXVWVMfB1LqGAALhzYS9HNoRln24zcjNxc/edit#slide=id.g3a91510c35_0_0 | |
| 14:17:54 | bauzas | edleafe: efried: I think I'll just tell about inventories and allocations ;) | |
| 14:18:01 | bauzas | and traits | |
| 14:19:03 | bauzas | oh, and FWIW, maybe we should update https://docs.openstack.org/nova/latest/user/architecture.html | |
| 14:31:37 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Refactor _build_device_metadata https://review.openstack.org/533804 | |
| 14:31:38 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Consider hostdev devices when building metadata https://review.openstack.org/533805 | |
| 14:33:03 | jaypipes | dansmith: what benefit would having a trait called "nvidia" give us? | |
| 14:33:11 | jaypipes | give the user that is.. | |
| 14:33:24 | jaypipes | or M60, even. | |
| 14:34:12 | jaypipes | dansmith: for the record we already have lots of GPU-specific traits around CUDA, OpenCL, DirectX API versions, etc. | |
| 14:34:29 | jaypipes | dansmith: I'm really trying hard not to have plain vendor tags leaked out to the user. | |
| 14:38:47 | dansmith | jaypipes: my image doesn't have AMD drivers in it and can't work with it, so.. I kinda need to be sure to land on a host with an nvidia card | |
| 14:38:57 | dansmith | I think CUDA implies nvidia, so if we have tags for that, then that's cool, | |
| 14:39:10 | dansmith | but I don't think even just API versions gives me enough to select what I want | |
| 14:40:29 | jaypipes | dansmith: what *would* be enough to select what you want? (other than "nvidia" and "m60", since those are vendor-specific and will essentially cause our API to have vendor-lockin) | |
| 14:41:19 | dansmith | jaypipes: what would be enough, other than the things I need? I dunno :) | |
| 14:42:13 | jaypipes | dansmith: I thought you were against vendor lockin? :) | |
| 14:42:16 | dansmith | how is having CUDA any different than nvidia? | |
| 14:42:48 | dansmith | jaypipes: by asking the question like that, you're unfairly trying to put me in a place to agree with your side of the argument | |
| 14:43:11 | jaypipes | dansmith: well, yeah... that's how I roll brother :) | |
| 14:43:13 | dansmith | how is it different than saying "my image requires vmware to run" | |
| 14:43:32 | dansmith | um, alright. | |
| 14:43:42 | jaypipes | dansmith: I'm joking with you. | |
| 14:44:12 | jaypipes | dansmith: so, I may be totally wrong (feel free to correct), but the way I understand it is this: | |
| 14:44:42 | jaypipes | CUDA is basically a programming *model* but to use it, you either use OpenCL or OpenACC frameworks. | |
| 14:45:11 | dansmith | I don't think that's how it works at all | |
| 14:45:31 | jaypipes | and considering we already have traits for opencl and other frameworks (which describe what the user needs), I am questioning a need for more traits | |
| 14:45:34 | dansmith | I could totally be wrong, of course | |
| 14:47:09 | dansmith | everyone I have heard talk about this makes it sound like they tent to target a pretty specific GPU while designing their application | |
| 14:47:41 | dansmith | not that they're not speaking a generic language to it such that it could run on a different model of the same family, but that it defeats the point of hyper-optimization | |
| 14:48:25 | Kevin_Zheng | mriedem: Hi, as for your comment in https://review.openstack.org/#/c/568542 I added the loop and other logic because I was considering race conditions, like maybe when I check the existence of future obj in queue, it was there, but when I actually cancel it, it went to preparing or running so I might do also some cleanup | |
| 14:48:30 | jaypipes | dansmith: any chance you can put me in touch with an example user of these things? | |
| 14:48:36 | dansmith | it would be like saying "I want a PCI nic", and having only intel drivers in your image, but you get put on a box and handed a mellanox card | |
| 14:48:59 | dansmith | jaypipes: lol | |
| 14:49:23 | openstackgerrit | Merged openstack/nova master: __str__ methods for RequestGroup, ResourceRequest https://review.openstack.org/568353 | |
| 14:49:43 | jaypipes | dansmith: I wasn't being facetious... | |
| 14:49:55 | jaypipes | dansmith: sorry | |
| 14:50:18 | Kevin_Zheng | mriedem: the loop is because I thought if it is in “preparing” it will end up either “error” or “running” so we can cancel it when it turn to “running” or just ignore it if it turned to “error” | |
| 14:50:53 | Kevin_Zheng | mriedem: that was just my consideration, not sure it is correct | |
| 14:51:20 | dansmith | jaypipes: so you already have a cuda trait, which means you effectively already have an nvidia one, | |
| 14:51:43 | dansmith | jaypipes: what happens when I've got my image that supports m60 gpus, and then next month you add a bunch of nodes with m61s, | |
| 14:51:58 | dansmith | I start booting instances that get there and their drivers are too old to recognize and use an m61? | |
| 14:52:31 | jaypipes | dansmith: the traits indicate the version of the CUDA, OpenGL, OpenCL drivers that are supported by the vGPU type. | |
| 14:53:03 | jaypipes | dansmith: https://github.com/openstack/os-traits/blob/master/os_traits/hw/gpu/api.py | |
| 14:53:08 | dansmith | jaypipes: okay, so for cuda that might help | |
| 14:53:21 | jaypipes | dansmith: so the user would just say "hey, I wrote my app using this version of CUDA" | |
| 14:53:27 | jaypipes | dansmith: and put that trait in their image metaedata | |
| 14:53:32 | dansmith | jaypipes: what about AMD and intel? there isn't a uniform opencl api bump to indicate each one right? | |
| 14:53:44 | dansmith | and I don't have amd drivers in my instance, but you put me there | |
| 14:54:18 | jaypipes | dansmith: we wouldn't put you on an AMD instance if you asked for CUDA 7.1... | |
| 14:54:23 | dansmith | no, | |
| 14:54:41 | dansmith | I'm saying let's say I'm a non-cuda instance.. I'm using OpenCL, and I asked for v2.2 | |
| 14:54:58 | dansmith | I have intel drivers in my instance, but you put me on an AMD-only node that supports v2.2 | |
| 14:56:38 | jaypipes | dansmith: not sure? :) | |
| 14:58:20 | jaypipes | dansmith: wikipedia states this... "Programs in the OpenCL language are intended to be compiled at run-time, so that OpenCL-using applications are portable between implementations for various host devices." | |
| 14:58:35 | dansmith | um, so? | |
| 14:58:45 | dansmith | the drivers and libraries are in my instance (or not) | |
| 14:58:54 | dansmith | once again, archlinux is the best wiki on the internet: https://wiki.archlinux.org/index.php/GPGPU | |
| 14:59:02 | dansmith | see the package names you have to get installed for each type of device? | |
| 15:00:02 | jaypipes | dansmith: yeah. | |
| 15:00:25 | dansmith | those being distro packages either bundle or download proprietary binaries for that layer | |
| 15:00:26 | jaypipes | dansmith: back to hardware-defined software it seems. :( | |
| 15:00:59 | dansmith | pretty sure the nvidia ones at least are very closely-guarded and require an account with endowments in order to download them | |
| 15:02:16 | dansmith | also pretty sure most of these have pretty specific kernel versions that are required, | |
| 15:02:30 | dansmith | as they also utilize the kernel-side driver, fglrx, nvidia, etc | |
| 15:02:48 | jaypipes | dansmith: my fear is that we will get locked in to vendor-specific identifiers and behaviour in the user-facing API and when these vendors inevitably change directions, we'll be forced to change our APIs. | |
| 15:03:31 | dansmith | jaypipes: I don't think we are though, because the traits are just enums.. if nothing is exposing m60 right now, then.. nothing boots that requires it | |
| 15:04:53 | dansmith | and going back to your point of trying to shame me, | |
| 15:05:04 | jaypipes | dansmith: what does m60 give you though? I can understand a potential need for traits that represent all these vendor drivers for OpenCL and CUDA, but the model name doesn't give any more information than the resolution, display heads and API library versions, right? | |
| 15:05:18 | dansmith | I don't want vendor lock-in of course, as you know, but I don't think enumerating the things that could be out there is doing that | |
| 15:05:24 | dansmith | jaypipes: of course it does | |
| 15:05:45 | dansmith | jaypipes: max resolution and display heads means nothing to me, I need cores/shaders/memory for GPGPU | |
| 15:05:55 | dansmith | jaypipes: and of course, lineage so I know if my drivers are appropriate | |
| 15:06:14 | jaypipes | but the drivers are for a version of CUDA or OpenCL, though, right? | |
| 15:06:17 | dansmith | jaypipes: if I'm an amd instance and I have a specific version of the drivers, I may be able to support a foobar99 but not a foobar128 | |
| 15:06:25 | jaypipes | you don't have drivers for M60, drivers for M61, etc, right? | |
| 15:06:26 | dansmith | I'm not sure what the appropriate level of abstraction is exactly, | |
| 15:06:44 | dansmith | maybe m60 is too fine-grained, but I think probably not because there are cards with 1, 2, 4 m60 chips, etc | |
| 15:08:14 | openstackgerrit | Vladyslav Drok proposed openstack/nova-specs master: Allow having placement inventories with reserved value equal to total https://review.openstack.org/568613 | |
| 15:12:23 | openstackgerrit | Merged openstack/nova master: trivial: Explain how the marker works for instance-cell mapping https://review.openstack.org/567597 | |
| 15:26:46 | jaypipes | dansmith: ok, so I'll just point out one last word on this VGPU stuff... AWS' elastic GPUs don't mention any vendor at all (https://aws.amazon.com/ec2/elastic-gpus/). They only mention "supports OpenGL 4.3" and up to 8G of GPU memory. Now, I know it's just one data point, but it does seem they are trying not to get into the game of leaking out vendor information ... | |
| 15:28:39 | dansmith | jaypipes: check out this in their FAQ: Q. Do I need a driver for Elastic GPUs? | |
| 15:29:00 | dansmith | jaypipes: looks like they have network-virtualized the GPUs so they can be attached anywhere, and they consume network bandwidth | |
| 15:29:20 | dansmith | they use OpenGL as their interface, | |