Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-01-09
15:29:54 sean-k-mooney so if you use that and we move to numa in placmenet it will break
15:30:05 dansmith where we should be able to parse the hw: things and alter the actual things we ask placement for inside nova I think
15:30:16 efried dansmith: exactly ^
15:30:36 sean-k-mooney yes so if you only used the unmbered syntax
15:30:50 sean-k-mooney and had the hw:* things tweek the toplogy i can see it working
15:30:52 efried The existing (unsuffixed) placement syntax is already pretty complicated to understand. Trying to add group identifiers to that will make it worse.
15:30:56 sean-k-mooney as nova can alter it interally
15:31:14 efried and then you have to understand *how* we model numa in placement
15:31:24 sean-k-mooney well we already support grouping of bothe the resouce and trait requests
15:31:26 dansmith sean-k-mooney: right, so I'd say do that.. placement syntax for totals, hw syntax for details, let nova alter what we ask placement for based on any tweaks you asked for
15:31:29 efried and how group identifiers correspond to that.
15:31:44 dansmith efried: I totes don't understand how current placement syntax is hard to understand
15:31:54 sean-k-mooney dansmith: ya i could see that working as we aready alter stuff with the prefilters now
15:32:17 efried dansmith: maybe you're too close to it
15:32:36 sean-k-mooney dansmith: its complicated if you use the numbered groups
15:32:49 sean-k-mooney if you dont ist not that bad and its consitent
15:33:33 dansmith sean-k-mooney: but we don't use numbered groups in the flavor placement syntax today right?
15:33:35 sean-k-mooney your are forced to use the groups today if you want to apply trait requirement to specific resouces
15:33:43 sean-k-mooney dansmith: we do
15:33:47 efried I think we support it, yes
15:33:50 sean-k-mooney well you can use them
15:33:56 efried we just don't have many use cases that need it today.
15:34:01 efried perhaps... none?
15:34:04 dansmith efried: well, I've had to explain it a few times and it has always been really nice, but okay
15:34:21 dansmith okay, I didn't know we even supported the group numbering today
15:34:36 sean-k-mooney it was added so you could say these traits are for my nic and these other traits are for my cpu
15:35:10 sean-k-mooney but since we dont have traiats that apply to mulple resouce classes today? do we? i dont think its stictly required
15:35:18 dansmith sean-k-mooney: is the reason we need that an artifact of how placement works or something?
15:35:39 dansmith because I would think that with no traits that apply to both cpus and nics that wouldn't be a thing I need to specify
15:35:48 efried dansmith: I get what you're saying. For most cases, the placement-ese syntax isn't any more complicated than the hw:-ese syntax. I'm asserting that we will for sure need the latter, and understanding *both* is harder than understanding *one* of them.
15:35:57 sean-k-mooney well you cant do resocues:<resouce class>=X; <traits for that RC>
15:36:32 sean-k-mooney so you have to do resources1:PCPU traits1=AVX=required
15:36:39 dansmith efried: I'm saying I think placement syntax (not including groups) is simpler and more consistent than hw syntax and is easier to understand as a whole DSL on that basis
15:36:57 sean-k-mooney where the sufix 1 in this case pairs the trait and request group
15:37:21 efried sean-k-mooney: which only matters if we have multiple providers, which today is only the case for VGPU and bandwidth.
15:37:28 efried right?
15:37:34 dansmith efried: and I'm also saying that I can understand the language of placement syntax in a few minutes, but never learn the taxonomy that is hw: without docs always in front of me
15:37:35 sean-k-mooney yes
15:37:56 efried dansmith: ack
15:37:57 sean-k-mooney although i thory it would matter for cinder volumes too with sharing providers
15:38:14 efried in theory, yes. in theory it also matters for numa.
15:38:19 sean-k-mooney but ya its only imporant for nested resouce providres or sharing
15:38:56 sean-k-mooney if we have 1 RP per compute then you never need grouping
15:39:08 sean-k-mooney with no nesting or sharing
15:40:30 efried We've done all this work to support nesting and grouping primarily for numa and shared storage. Only in train did we get to a point where placement is powerful enough to do what we want for numa. And in this release we've landed a couple of patches to make nova use those placement microversions.
15:40:33 sean-k-mooney dansmith: if we removed supprot for the group syntax then it would make hw:* been used for congriuation and resouce: be used for quantity much simpler
15:40:37 efried so we're making progress.
15:40:56 efried but the last jump, modeling numa in placement from nova, is going to be really hard.
15:41:20 dansmith sean-k-mooney: ack
15:41:31 efried I would be fine with that fwiw.
15:41:53 efried though it would technically be removing functionality that
15:42:01 efried ...that we've already released with
15:42:14 efried even if it's pretty unlikely anyone is using it.
15:42:16 sean-k-mooney well yes but we have depreacated things in the past.
15:42:19 efried yeah
15:42:58 efried this becomes relevant for cyborg too btw.
15:43:02 sean-k-mooney i think there are 3 distinct things. Resocues:* for quantiy, hw:* for configuration and traits:* for capablities
15:43:10 alex_xu i guess nobody using group now
15:43:31 efried and, dansmith, is a good example of where we could (but I really don't want to) use placement-ese to have side effects.
15:43:37 sean-k-mooney at the moment we allow hw:* to be renderd into resouces:* and traits:*
15:43:50 openstackgerrit Balazs Gibizer proposed openstack/nova master: DNM: Avoid PlacementFixture silently swallowing kwargs https://review.opendev.org/701754
15:44:05 dansmith efried: cyborg is a good example of side effects?
15:44:05 efried like programming bitstreams
15:44:44 dansmith efried: isn't the bitstream specified in the device profile in cyborg?
15:45:03 efried yes.
15:45:16 sean-k-mooney dansmith: i think it a metadata key on the guest image
15:45:28 efried no (at least not yet)
15:45:36 sean-k-mooney the type of accelerator is definetly in the device profiel
15:45:45 dansmith I think the bitstream is too
15:45:55 sean-k-mooney the important thing is its not in the nova flavor
15:46:22 dansmith efried: I'm not sure what you mean exactly by cyborg would be a good example of side effects (affecting the bitstream we use?)
15:47:10 efried in the future we want to "prefer" hosts that have an accel with the desired bitstream already on them. In that case the placement request would us preferred=BITSTREAM_XXX. But we don't want to expose trait:BISTREAM_XXX=preferred in the flavor extra specs.
15:47:13 dansmith efried: are you saying that if cyborg represented possible devices that could be composed by programming a, say, gzip bitstream into a fpga as a resource in placement that asking for that thing would cause you to get one?
15:47:14 efried kind of thing.
15:47:41 sean-k-mooney well vgpus is a example of Resources:vGPU=1 haveing an effect of actully provision a gpu for the guest
15:47:46 dansmith that's maybe a next-level leap from what I'm saying,
15:48:03 dansmith but it's also a much more user/ops-friendly way of stitching high level functions together
15:48:37 dansmith i.e. instead of plugging low-level nova into low-level cyborg, you define an abstract thing in cyborg which becomes a high-level resource in nova that you just ask for by symbolic name,
15:48:46 efried YES
15:48:47 dansmith which is more what I think we should be shooting for, compared to the former
15:49:19 dansmith you say YES, but I feel like you were arguing NO a minute ago, so I'm confused
15:50:01 efried Today the way we model resources in placement is very similar to the way nova thinks of them
15:50:09 sean-k-mooney how about this. for the mix cpu case. we could say that if you do VCPU:2,PCPU:6 we will generate the topogy for you but you can override it with hw:gust_pinned_cpus
15:50:13 efried so just understanding placement syntax is enough
15:50:50 sean-k-mooney but you could just set hw:guest_pinned_cpus too
15:51:05 efried but very soon (numa modeling, cyborg) we will have to model in placement in ways that are less intuitive
15:51:40 efried if we continue to support placement-ese in that future, it will required the user to understand that extra translation
15:51:49 efried I'm just suggesting we hide that translation in nova.
15:52:14 dansmith but then we have to invent syntax for everything no?
15:52:29 efried we already have that syntax
15:52:33 sean-k-mooney no only if we want to expose contol over that translation
15:53:06 dansmith efried: we've already invented new syntax for "please include me an accelerator by the name of X"
15:53:39 dansmith efried: if that was a placement resource, we wouldn't have had to do that, but we did and now sundar is inspecting internal flavor details of this bespoke syntax everywhere
15:53:59 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: Add recreate test for bug 1855927 https://review.opendev.org/701756
15:53:59 openstack bug 1855927 in OpenStack Compute (nova) "_poll_unconfirmed_resizes may not retry later if confirm_resize fails in API" [Low,In progress] https://launchpad.net/bugs/1855927 - Assigned to Matt Riedemann (mriedem)
15:53:59 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: Ensure source service is up before resizing/migrating https://review.opendev.org/701757
15:54:29 dansmith efried: and we will have to invent new syntax when we want to ask it to be hung on a specific numa node I think too right?
15:54:44 dansmith so we'll have two unique things for accelerators: which one and how to attach
15:54:48 sean-k-mooney dansmith: well that would only work if each resouce class is only owned by 1 project
15:55:10 sean-k-mooney nova already "owns" the VGPU resouce class

Earlier   Later