| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-03 | |||
| 16:08:52 | bauzas | efried: ok, I take the challenge | |
| 16:09:20 | bauzas | efried: I'm in favor of an unified trait name | |
| 16:09:20 | efried | bauzas: Which needs to happen regardless, because at the very least 'nvidia-11' will need to be translated to 'trait:HW_VGPU_TYPE_NVIDIA_11=required' | |
| 16:09:31 | bauzas | yup | |
| 16:10:14 | efried | bauzas: What would be neat is if each hypervisor provided a script to discover the devs and list them along with their trait names. | |
| 16:10:28 | efried | that might be asking too much, though :) | |
| 16:10:31 | bauzas | efried: what I wonder if whether the mapping should be hardcoded in the virt driver code, or if we should leave operators setting that | |
| 16:10:45 | bauzas | efried: heh, I have the same wish | |
| 16:10:50 | bauzas | efried: xen does that | |
| 16:10:57 | bauzas | efried: but see my docs on libvirt usage | |
| 16:11:02 | bauzas | that's crazy | |
| 16:11:10 | bauzas | you have to lookup sysfs | |
| 16:11:21 | efried | bauzas: The operator mapping thing - that's going back to the yaml config we started brainstorming in Denver. | |
| 16:11:26 | bauzas | we don't have any libvirt API that shows you that | |
| 16:11:52 | bauzas | efried: I know | |
| 16:12:06 | efried | bauzas: BUT that mapping is going to be more for device aliasing, white/blacklisting, etc. I don't think we should be using it to define traits, unless maybe they're CUSTOM_* | |
| 16:12:19 | bauzas | efried: just a thought | |
| 16:12:26 | bauzas | efried: about custom traits | |
| 16:12:29 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec https://review.openstack.org/541290 | |
| 16:12:33 | bauzas | efried: can you set those with nested RPs ? | |
| 16:12:52 | bauzas | efried: because then, I wouldn't care a bit about that | |
| 16:13:07 | bauzas | efried: I could just ask operators to provide custom traits | |
| 16:13:19 | efried | bauzas: You can set traits on any RP. We originally talked about traits "inheriting" or "propagating" in various directions, but we're not doing that. | |
| 16:13:31 | efried | bauzas: "on any RP" including child, sharing, etc. | |
| 16:13:31 | bauzas | efried: I'm a bit rusty on traits | |
| 16:13:46 | bauzas | efried: but I guess operators have to play with placement client to set the trait ? | |
| 16:13:56 | bauzas | if so, that's perfect | |
| 16:14:44 | bauzas | jaypipes: for your sanity, don't read the vendor crazypants I wrote | |
| 16:15:08 | efried | bauzas: Well, what we talked about in Dublin is that each of various entities (nova-compute, neutron, operator) will have a set of traits it owns/controls for a given provider. It's allowed to set/unset those traits, but the other entities are not. | |
| 16:15:42 | efried | bauzas: So in this example, I would expect nova-compute (specifically the virt driver) to "own" the trait indicating the GPU type. The operator doesn't get to set that. That wouldn't make sense. | |
| 16:16:02 | bauzas | efried: keep in mind we will have one inventory per vGPU type | |
| 16:16:18 | efried | bauzas: Same entity owns the inventory. | |
| 16:16:49 | bauzas | efried: what if I want to set custom traits ? | |
| 16:17:26 | bauzas | efried: say the virt driver now provides a tree of a root RP and two children RP, each one being a vGPU type | |
| 16:17:44 | bauzas | efried: as an operator, I'd like to pick only one type based on a custom trait | |
| 16:17:50 | efried | bauzas: From this perspective, we don't really make a distinction between standard and custom traits. But I would say the op is allowed to invent and assign a random custom trait as she sees fit. | |
| 16:17:54 | bauzas | efried: exactly what we wrote in the spec, actually | |
| 16:18:13 | bauzas | https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html | |
| 16:18:14 | efried | bauzas: Legit. Basically the op just has to be careful not to stomp on a trait owned by compute. | |
| 16:18:24 | efried | bauzas: beyond that, sky's the limit. | |
| 16:18:38 | efried | bauzas: which is exactly why we made this a requirement in Dublin. | |
| 16:18:46 | efried | (dansmith and I almost came to blows over it) | |
| 16:18:51 | bauzas | yeah I remember | |
| 16:19:14 | efried | I still believe it's going to bite us, but time will tell. | |
| 16:19:36 | bauzas | efried: ok, so what's the way to set custom traits as of now ? | |
| 16:19:42 | bauzas | thru placement API ? (and the client) | |
| 16:19:55 | efried | bauzas: Sure, that works. | |
| 16:20:15 | bauzas | efried: ok, what I need is testing then | |
| 16:20:30 | bauzas | efried: my patch series is close to provide inventories using update_provider_tree() | |
| 16:20:44 | efried | bauzas: There's no change to the placement API for any of that; the only thing we did was update the u-p-t spec/docs to make it clear that a dance is required to make sure you're not stepping on each other. | |
| 16:21:12 | efried | bauzas: Oh, are you aware of https://review.openstack.org/#/c/560444/ ? | |
| 16:21:12 | bauzas | efried: ok, will look | |
| 16:21:40 | bauzas | efried: no, I wasn't | |
| 16:21:54 | bauzas | efried: my top patch in the series returns a dict of dicts | |
| 16:22:08 | bauzas | efried: I'll use that dict of dicts to populate the tree | |
| 16:22:31 | bauzas | efried: but I was considering to implement by a change the new public interface for libvirt | |
| 16:22:34 | openstackgerrit | Merged openstack/nova master: Get anchors for sharing providers https://review.openstack.org/565279 | |
| 16:22:42 | openstackgerrit | Merged openstack/nova master: Make association_refresh configurable https://review.openstack.org/565526 | |
| 16:23:00 | efried | bauzas: Have you been following the xen driver work for vgpus? | |
| 16:23:08 | bauzas | efried: barely but yes | |
| 16:23:10 | efried | bauzas: I think they're the furthest along with using upt in their driver. | |
| 16:23:30 | bauzas | efried: well, I'm pretty close too | |
| 16:23:35 | efried | bauzas: Of course, they're based on code that hasn't merged and/or won't work until the nrp-in-alloc-cands series lands. | |
| 16:23:55 | bauzas | efried: last change in my series is https://review.openstack.org/#/c/564215/ | |
| 16:24:13 | bauzas | efried: I know, and I was about to propose your series for a runway slot | |
| 16:24:42 | bauzas | I'm a bit out of review as of those days due to my patch series, but I can switch to reviewing if I'm blocked due to that series :p | |
| 16:25:06 | bauzas | anyway, I need to drop for today | |
| 16:25:08 | efried | bauzas: Granular is ready as well. (Except both series are going to bounce on merge conflict soon) | |
| 16:25:32 | bauzas | granular is already a runway candidate | |
| 16:25:32 | jaypipes | bauzas: I'm a strong -2 for having standard traits (or really, any traits) representing GPU vendor types | |
| 16:25:33 | efried | I guess I could put both of those series into the runway queue. | |
| 16:25:45 | efried | jaypipes: eh?? | |
| 16:25:50 | bauzas | jaypipes: I'm cool with that | |
| 16:25:57 | exarlos | mriedem: pastebin.com/SPpHu9Tu is the current error state of the VM. | |
| 16:26:11 | bauzas | jaypipes: I guess your concern is the os-traits explosion | |
| 16:26:14 | melwitt | looks like the gate is timing out a lot in response to me saying it had been good in the nova meeting | |
| 16:26:22 | efried | jaypipes: Oh, you're looking for GPU *features* as opposed to types. | |
| 16:26:25 | bauzas | melwitt: yup, I noticed that in the meeting too | |
| 16:26:33 | melwitt | :( sorry everyone | |
| 16:26:54 | exarlos | mriedem: I would think it's recoverable, but i'm still learning. | |
| 16:26:59 | bauzas | jaypipes: efried: anyway, I need to drop | |
| 16:27:12 | efried | bauzas: à bientôt | |
| 16:27:17 | bauzas | the first step is to make nested RPs a thing | |
| 16:27:32 | bauzas | the second step will be to test my patches using custom traits | |
| 16:27:39 | mriedem | melwitt: excellent catch on https://review.openstack.org/#/c/541420/ - i reproduced in one of the unit tests | |
| 16:28:01 | bauzas | so I'm super not concerned by having standard traits as of now, in particular for vendor purposes | |
| 16:28:11 | bauzas | jaypipes: hope that clarifies ^ | |
| 16:28:54 | jaypipes | bauzas: you don't need custom traits for any of this. | |
| 16:29:03 | jaypipes | bauzas: you don't need GPU types to be traits either. | |
| 16:29:16 | bauzas | what I want is a pony | |
| 16:29:25 | mriedem | exarlos: so you're hitting https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7450 | |
| 16:29:26 | jaypipes | bauzas: we had long conversations with Jianghua on the original VGPU spec about this. | |
| 16:29:32 | bauzas | jaypipes: what do you suggest ? | |
| 16:29:35 | mriedem | exarlos: which comes after https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7413 | |
| 16:29:49 | melwitt | mriedem: woot (though concerned the unit test was covering it up?) | |
| 16:29:51 | mriedem | exarlos: are you sure this isn't a volume-backed instance? and by volume-backed, i mean cinder volume, not local lvm disk | |
| 16:30:06 | jaypipes | bauzas: use the traits for GPUs that are already in os-traits for describing the *capabilities* of the GPU. | |
| 16:30:06 | mriedem | melwitt: well, the unit test was missing a check for the bdm turducken | |
| 16:30:12 | melwitt | a-ha | |
| 16:30:17 | jaypipes | bauzas: "nvidia-11" isn't a capability. | |