| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-27 | |||
| 13:48:28 | bauzas | ie. "for that PCI device, here is the type" | |
| 13:48:51 | bauzas | jaypipes: thoughts on that ? ^ | |
| 13:48:58 | efried | bauzas: If you want ultimate flexibility, I think you need to provide inventory of some RC that lets you represent "units of VGPU-ness", where different types consume a different number of that resource. And then your request would have to be well-behaved and ask for resources=VGPU:1,VGPU_UNITY_THINGY:4 | |
| 13:49:26 | efried | ...where the request for VGPU_UNITY_THINGY has to be correct for the type you're requesting. | |
| 13:49:53 | openstackgerrit | Merged openstack/os-traits master: Updated from global requirements https://review.openstack.org/551599 | |
| 13:50:04 | bauzas | efried: that's where I think we're over engineering | |
| 13:50:22 | efried | resources=VGPU:1,VGPU_UNITY_THINGY:4&required=VGPU_TYPE_X | |
| 13:50:22 | efried | or | |
| 13:50:22 | efried | resources=VGPU_TYPE_X:1,VGPU_UNITY_THINGY:4 | |
| 13:50:22 | efried | That would be clearer as | |
| 13:50:25 | bauzas | efried: operators want flavors like VGPU=2&trait=MY_TYPE | |
| 13:50:58 | bauzas | yeah, honestly, I feel for the short term a config option that would do a whitelist seems the most acceptable solution | |
| 13:51:00 | efried | bauzas: Yeah, I get that. This isn't the only place we've seen where it would be useful to provide a translation layer from the flavor to the actual placement request. | |
| 13:51:38 | efried | bauzas: I think perhaps we're oversimplifying/idealizing by thinking that a direct mapping of flavor to placement request is going to allow us to cover everything. | |
| 13:52:06 | bauzas | let's incrementally try to resolve the usecase | |
| 13:52:20 | bauzas | for queens, we supported a single type | |
| 13:52:30 | efried | bauzas: Basically, requiring the admin to understand the nuances and quirks of both the provider tree as modeled by the virt driver, and the syntax and semantics of the placement API query. | |
| 13:52:48 | bauzas | efried: that's where I think a whitelist could be more understandable | |
| 13:53:09 | bauzas | efried: we could have enabled_vgpu_types that would keep the existing types we agree | |
| 13:53:23 | bauzas | and then a second option that would tell for which PCI ID which type | |
| 13:53:42 | efried | bauzas: I'm okay with that idea in general, but I'm going to be watching like a hawk to make sure we retain the separation of platform-specific syntax. | |
| 13:53:51 | efried | E.g. "NO PCI ID!" | |
| 13:53:51 | bauzas | in that case, the PGPU inventory would be of one type, problem solved. | |
| 13:54:27 | bauzas | efried: the PCI ID thingy is just within the driver | |
| 13:54:35 | bauzas | no crazypants about PCI tracking | |
| 13:54:56 | bauzas | litterally ask libvirt to pick that type for that pGPU | |
| 13:55:16 | efried | bauzas: If there's a PCI ID in the file, then we have to say that the file gets parsed by virt alone. Kind of thing. | |
| 13:55:48 | bauzas | efried: yeah, zactly that | |
| 13:56:04 | efried | bauzas: So... you want to do something like this for Rocky? | |
| 13:56:08 | bauzas | I guess | |
| 13:56:17 | bauzas | if nested RPs is a thing :) | |
| 13:56:22 | efried | bauzas: Cause this is edging unequivocally into Generic Device Management territory. | |
| 13:56:34 | alex_xu_ | bauzas: when your request VGPU_thistype, how do you change total=0 for the VGPU_thosetype? | |
| 13:56:48 | bauzas | alex_xu_: that's magically done by sysfs | |
| 13:57:03 | bauzas | alex_xu_: which I'm using for getting the inventory | |
| 13:57:05 | alex_xu_ | bauzas: but how the placement to know that | |
| 13:57:20 | bauzas | alex_xu_: because we provide inventories of VGPU resouces as of queens :) | |
| 13:57:27 | efried | bauzas: Wait, you're talking about doing that at setup time, not at allocation time, right? | |
| 13:57:28 | bauzas | that are populated based on sysfs :) | |
| 13:58:12 | bauzas | efried: alex_xu_ is talking of the case where we would have pGPU types as children | |
| 13:58:14 | alex_xu_ | bauzas: if there are two requests at same time. One for VGPU_thistype, another for VGPU_thosetype. So there will be race case. Only one can successful in the host finally | |
| 13:58:18 | efried | bauzas: Bricks will be shat by several people if you start talking about modifying inventory of Y because X got allocated. | |
| 13:58:32 | bauzas | alex_xu_: yeah I considered the race condition | |
| 13:58:41 | bauzas | alex_xu_: and that's actually a good call for not doing that | |
| 13:58:52 | bauzas | but rather doing a whitelisting on the virt driver direcrtly | |
| 13:59:07 | alex_xu_ | bauzas: so limit only one type in each host? | |
| 14:00:27 | bauzas | alex_xu_: no, one type per physical GPU | |
| 14:00:41 | alex_xu_ | ah, i got it | |
| 14:00:42 | bauzas | one type per host is already here | |
| 14:00:59 | bauzas | except the fact we don't use traits atm | |
| 14:01:29 | bauzas | anyway, will just propose something soon so we could see if that requires some spec | |
| 14:01:43 | bauzas | or at least some spec amendment | |
| 14:01:50 | efried | bauzas: Right, so you're setting up the types & inventories the first time you load up. The first time update_provider_tree runs, it reads that config file and sets up the provider tree with types & inventories according to what the admin put in there. Right? | |
| 14:01:59 | bauzas | ++ | |
| 14:02:12 | efried | Cool cool | |
| 14:02:16 | alex_xu_ | the race condidates only happened one time, if people accept that, it also ok | |
| 14:02:20 | bauzas | efried: https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html describes that partially | |
| 14:02:34 | jaypipes | guh, it would have been nice to have more than a few hours of sleep before spec sprint day... :( | |
| 14:02:41 | efried | bauzas: I can also see a solution that goes halfway in the future... | |
| 14:02:55 | bauzas | jaypipes: take caffeine or drugs, either. | |
| 14:03:32 | bauzas | jaypipes: FWIW, https://review.openstack.org/#/c/541290/7/specs/rocky/approved/numa-aware-vswitches.rst will require me some paracetamol | |
| 14:03:36 | bauzas | stephenfin: ^ ;) | |
| 14:03:50 | cdent | jaypipes: start on an easy one: https://review.openstack.org/#/c/418393/ | |
| 14:03:52 | edleafe | jaypipes: have you tried the blue meth? | |
| 14:04:02 | bauzas | always take the blue pill | |
| 14:04:04 | stephenfin | I hear the blue meth is real good | |
| 14:04:13 | bauzas | never choose the red one | |
| 14:04:35 | efried | bauzas: In a fashion similar to neutron port creation, the user could create the VGPU resource before doing the boot request. The VGPU creation would eventually hit vendor-specific code (maybe virt, maybe some agent - cyborg?) which actually *would* tweak the inventories and available types. Then you would attach that VGPU to your boot request, just like you would a port. | |
| 14:05:03 | alex_xu_ | efried: any legal drugs in Florida? | |
| 14:05:26 | efried | alex_xu_: Legal schmegal. But I'm in Texas. | |
| 14:05:30 | bauzas | efried: creating the mediated devices in advance is already a possibility | |
| 14:05:32 | efried | In Florida, they wash up on the beach. | |
| 14:08:46 | kashyap | alex_xu_: Hey there | |
| 14:09:12 | kashyap | alex_xu_: About your comment here: https://review.openstack.org/#/c/534384/16/nova/tests/unit/virt/libvirt/test_config.py | |
| 14:09:21 | efried | bauzas: But having that creation impact placement inventories would be new. | |
| 14:09:34 | bauzas | efried: no, it's already the case | |
| 14:09:42 | kashyap | alex_xu_: The test you pointed out is slightly different in that, it is testing: 'obj.extra_flags' | |
| 14:09:43 | efried | oh? | |
| 14:09:57 | bauzas | efried: if you create a mediated device, libvirt will just add that to the total | |
| 14:10:11 | bauzas | I should write a blogpost on this... | |
| 14:10:34 | bauzas | because creating a mediated device doesn't mean you *use* it for a guesrt | |
| 14:10:53 | bauzas | so, placement is getting an inventory of 'available+created' as a total already | |
| 14:11:20 | efried | cool | |
| 14:12:11 | jaypipes | stephenfin: and ack on the OVS documentation being awful. | |
| 14:12:19 | jaypipes | OVS-DPDK that is. | |
| 14:14:57 | dansmith | efried: mriedem: I can never remember the moving target of translations.. we're translating exceptions now, but not translating logs _at all_ (even warning) ? | |
| 14:15:26 | bauzas | dansmith: right IIRC | |
| 14:15:28 | efried | dansmith: correct | |
| 14:15:49 | bauzas | because translating logs is too much for the i18n team | |
| 14:15:57 | bauzas | they aren't able to scale | |
| 14:16:24 | efried | Nice patch for a new contributor: remove _Lx from nova.i18n, see what breaks, fix it. | |
| 14:17:36 | dansmith | wtfever | |
| 14:18:07 | dansmith | not marking them means they _can't_ be translated which seems like a stupid step back, but whatever | |
| 14:21:09 | openstackgerrit | Patricia Domingues proposed openstack/nova master: load up the volume drivers by checking architecture https://review.openstack.org/541393 | |
| 14:23:50 | jaypipes | efried: "The first time update_provider_tree runs, it reads that config file and sets up the provider tree with types & inventories according to what the admin put in there. Right?" <-- you mean, exactly like my provider-config-file spec enabled? | |
| 14:24:56 | efried | jaypipes: could be, could be. I'm not sure if I read that before it was declared moot, but if I did, I've purged it :( | |
| 14:25:47 | efried | jaypipes: Anyway, it sounds like a lovely idea you had :) | |
| 14:26:06 | jaypipes | efried, bauzas: I'm the person that would shit a brick if you start adding dynamic inventory creation based on whatever a user requested. | |
| 14:26:34 | efried | Yes, but not the only person. | |
| 14:26:52 | bauzas | jaypipes: I'm entering the danger timezone of me needing to go get my children at school | |