| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-09 | |||
| 15:23:18 | dansmith | efried: my preference (and what I thought we were moving towards) was a future where resource (amounts) are specified in placement syntax, and topologies and the like are assigned sanely. Tweaks to how those look (like I really need this thing pinned here) would be hw:things but where the hw: is the optional | |
| 15:23:52 | dansmith | meaning, usually you just ask for the amount of stuff you want and we give it to you in a sane arrangement which is fine for most people, | |
| 15:24:07 | dansmith | and hw:tweak your way to a specific arrangement if that's important | |
| 15:24:20 | dansmith | a more clear delineation of which thing does what | |
| 15:24:40 | dansmith | with the hw: being the primary, then I'm not sure how you know when you do or don't or can't specify some resource count as well | |
| 15:25:13 | dansmith | but anyway, I grok the complexity on both sides and I'm sure both will have warts | |
| 15:25:37 | sean-k-mooney | i could see that working if we had a speartion between the placment query and the resouces*:* element in the extra specs | |
| 15:25:56 | efried | We're very close to agreeing here. I want the clear delineation to be that placement-ese syntax doesn't have side effects. That's the same delineation as you describe in *most* cases. | |
| 15:26:43 | dansmith | well, in what I said above, asking for PCPUS:2 would get you two pinned cpus, which I think you're describing as a side effect | |
| 15:26:59 | sean-k-mooney | dansmith: in your world would you see the need to supprot the number group syntax in the flavor for example | |
| 15:27:55 | dansmith | sean-k-mooney: I will totally admit that I don't (ever) have the resource group numbering quagmire swapped into my head, but I assume yeah, you might need an extension to placement syntax to group those things if that's what you mean | |
| 15:28:02 | dansmith | resources:2:PCPUS=2 I assume | |
| 15:28:53 | sean-k-mooney | the numbered sysntax means that all resouce requst in that group must come from the same RP | |
| 15:29:19 | sean-k-mooney | so if you have resources:2:PCPUS=2,MEMORY_MB=1024 | |
| 15:29:34 | dansmith | ack | |
| 15:29:39 | sean-k-mooney | then you need 2 pinned cpus and 1G of memory form the same resouce provider | |
| 15:29:43 | dansmith | yeah, I dunno, that's encoding a little bit of topology into the placement syntax, | |
| 15:29:54 | sean-k-mooney | so if you use that and we move to numa in placmenet it will break | |
| 15:30:05 | dansmith | where we should be able to parse the hw: things and alter the actual things we ask placement for inside nova I think | |
| 15:30:16 | efried | dansmith: exactly ^ | |
| 15:30:36 | sean-k-mooney | yes so if you only used the unmbered syntax | |
| 15:30:50 | sean-k-mooney | and had the hw:* things tweek the toplogy i can see it working | |
| 15:30:52 | efried | The existing (unsuffixed) placement syntax is already pretty complicated to understand. Trying to add group identifiers to that will make it worse. | |
| 15:30:56 | sean-k-mooney | as nova can alter it interally | |
| 15:31:14 | efried | and then you have to understand *how* we model numa in placement | |
| 15:31:24 | sean-k-mooney | well we already support grouping of bothe the resouce and trait requests | |
| 15:31:26 | dansmith | sean-k-mooney: right, so I'd say do that.. placement syntax for totals, hw syntax for details, let nova alter what we ask placement for based on any tweaks you asked for | |
| 15:31:29 | efried | and how group identifiers correspond to that. | |
| 15:31:44 | dansmith | efried: I totes don't understand how current placement syntax is hard to understand | |
| 15:31:54 | sean-k-mooney | dansmith: ya i could see that working as we aready alter stuff with the prefilters now | |
| 15:32:17 | efried | dansmith: maybe you're too close to it | |
| 15:32:36 | sean-k-mooney | dansmith: its complicated if you use the numbered groups | |
| 15:32:49 | sean-k-mooney | if you dont ist not that bad and its consitent | |
| 15:33:33 | dansmith | sean-k-mooney: but we don't use numbered groups in the flavor placement syntax today right? | |
| 15:33:35 | sean-k-mooney | your are forced to use the groups today if you want to apply trait requirement to specific resouces | |
| 15:33:43 | sean-k-mooney | dansmith: we do | |
| 15:33:47 | efried | I think we support it, yes | |
| 15:33:50 | sean-k-mooney | well you can use them | |
| 15:33:56 | efried | we just don't have many use cases that need it today. | |
| 15:34:01 | efried | perhaps... none? | |
| 15:34:04 | dansmith | efried: well, I've had to explain it a few times and it has always been really nice, but okay | |
| 15:34:21 | dansmith | okay, I didn't know we even supported the group numbering today | |
| 15:34:36 | sean-k-mooney | it was added so you could say these traits are for my nic and these other traits are for my cpu | |
| 15:35:10 | sean-k-mooney | but since we dont have traiats that apply to mulple resouce classes today? do we? i dont think its stictly required | |
| 15:35:18 | dansmith | sean-k-mooney: is the reason we need that an artifact of how placement works or something? | |
| 15:35:39 | dansmith | because I would think that with no traits that apply to both cpus and nics that wouldn't be a thing I need to specify | |
| 15:35:48 | efried | dansmith: I get what you're saying. For most cases, the placement-ese syntax isn't any more complicated than the hw:-ese syntax. I'm asserting that we will for sure need the latter, and understanding *both* is harder than understanding *one* of them. | |
| 15:35:57 | sean-k-mooney | well you cant do resocues:<resouce class>=X; <traits for that RC> | |
| 15:36:32 | sean-k-mooney | so you have to do resources1:PCPU traits1=AVX=required | |
| 15:36:39 | dansmith | efried: I'm saying I think placement syntax (not including groups) is simpler and more consistent than hw syntax and is easier to understand as a whole DSL on that basis | |
| 15:36:57 | sean-k-mooney | where the sufix 1 in this case pairs the trait and request group | |
| 15:37:21 | efried | sean-k-mooney: which only matters if we have multiple providers, which today is only the case for VGPU and bandwidth. | |
| 15:37:28 | efried | right? | |
| 15:37:34 | dansmith | efried: and I'm also saying that I can understand the language of placement syntax in a few minutes, but never learn the taxonomy that is hw: without docs always in front of me | |
| 15:37:35 | sean-k-mooney | yes | |
| 15:37:56 | efried | dansmith: ack | |
| 15:37:57 | sean-k-mooney | although i thory it would matter for cinder volumes too with sharing providers | |
| 15:38:14 | efried | in theory, yes. in theory it also matters for numa. | |
| 15:38:19 | sean-k-mooney | but ya its only imporant for nested resouce providres or sharing | |
| 15:38:56 | sean-k-mooney | if we have 1 RP per compute then you never need grouping | |
| 15:39:08 | sean-k-mooney | with no nesting or sharing | |
| 15:40:30 | efried | We've done all this work to support nesting and grouping primarily for numa and shared storage. Only in train did we get to a point where placement is powerful enough to do what we want for numa. And in this release we've landed a couple of patches to make nova use those placement microversions. | |
| 15:40:33 | sean-k-mooney | dansmith: if we removed supprot for the group syntax then it would make hw:* been used for congriuation and resouce: be used for quantity much simpler | |
| 15:40:37 | efried | so we're making progress. | |
| 15:40:56 | efried | but the last jump, modeling numa in placement from nova, is going to be really hard. | |
| 15:41:20 | dansmith | sean-k-mooney: ack | |
| 15:41:31 | efried | I would be fine with that fwiw. | |
| 15:41:53 | efried | though it would technically be removing functionality that | |
| 15:42:01 | efried | ...that we've already released with | |
| 15:42:14 | efried | even if it's pretty unlikely anyone is using it. | |
| 15:42:16 | sean-k-mooney | well yes but we have depreacated things in the past. | |
| 15:42:19 | efried | yeah | |
| 15:42:58 | efried | this becomes relevant for cyborg too btw. | |
| 15:43:02 | sean-k-mooney | i think there are 3 distinct things. Resocues:* for quantiy, hw:* for configuration and traits:* for capablities | |
| 15:43:10 | alex_xu | i guess nobody using group now | |
| 15:43:31 | efried | and, dansmith, is a good example of where we could (but I really don't want to) use placement-ese to have side effects. | |
| 15:43:37 | sean-k-mooney | at the moment we allow hw:* to be renderd into resouces:* and traits:* | |
| 15:43:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: DNM: Avoid PlacementFixture silently swallowing kwargs https://review.opendev.org/701754 | |
| 15:44:05 | dansmith | efried: cyborg is a good example of side effects? | |
| 15:44:05 | efried | like programming bitstreams | |
| 15:44:44 | dansmith | efried: isn't the bitstream specified in the device profile in cyborg? | |
| 15:45:03 | efried | yes. | |
| 15:45:16 | sean-k-mooney | dansmith: i think it a metadata key on the guest image | |
| 15:45:28 | efried | no (at least not yet) | |
| 15:45:36 | sean-k-mooney | the type of accelerator is definetly in the device profiel | |
| 15:45:45 | dansmith | I think the bitstream is too | |
| 15:45:55 | sean-k-mooney | the important thing is its not in the nova flavor | |
| 15:46:22 | dansmith | efried: I'm not sure what you mean exactly by cyborg would be a good example of side effects (affecting the bitstream we use?) | |
| 15:47:10 | efried | in the future we want to "prefer" hosts that have an accel with the desired bitstream already on them. In that case the placement request would us preferred=BITSTREAM_XXX. But we don't want to expose trait:BISTREAM_XXX=preferred in the flavor extra specs. | |
| 15:47:13 | dansmith | efried: are you saying that if cyborg represented possible devices that could be composed by programming a, say, gzip bitstream into a fpga as a resource in placement that asking for that thing would cause you to get one? | |
| 15:47:14 | efried | kind of thing. | |
| 15:47:41 | sean-k-mooney | well vgpus is a example of Resources:vGPU=1 haveing an effect of actully provision a gpu for the guest | |
| 15:47:46 | dansmith | that's maybe a next-level leap from what I'm saying, | |
| 15:48:03 | dansmith | but it's also a much more user/ops-friendly way of stitching high level functions together | |
| 15:48:37 | dansmith | i.e. instead of plugging low-level nova into low-level cyborg, you define an abstract thing in cyborg which becomes a high-level resource in nova that you just ask for by symbolic name, | |
| 15:48:46 | efried | YES | |
| 15:48:47 | dansmith | which is more what I think we should be shooting for, compared to the former | |
| 15:49:19 | dansmith | you say YES, but I feel like you were arguing NO a minute ago, so I'm confused | |
| 15:50:01 | efried | Today the way we model resources in placement is very similar to the way nova thinks of them | |
| 15:50:09 | sean-k-mooney | how about this. for the mix cpu case. we could say that if you do VCPU:2,PCPU:6 we will generate the topogy for you but you can override it with hw:gust_pinned_cpus | |