Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-01-09
15:15:16 sean-k-mooney so if you are sane you would do hw:cpu_socket=1 hw:cpu_theads=2
15:15:19 dansmith and closed source P2V'd ancient telco appliances are far far worse
15:15:24 efried dansmith: I think I see your point. It should be *possible* to say "give me $p pinned and $s shared, you go figure out how to distribute them" without *having* to say "I want CPUs 0,3,7 pinned and 1,2,4-6 shared".
15:15:44 dansmith efried: pretty much
15:15:58 sean-k-mooney efried: yes so that is what we wanted to discuss in the meeting when we said count vs mask
15:16:01 efried But I would argue that, if you care about a pinned vs shared split, you care enough about performance that having to do the extra syntax isn't a big jump.
15:16:04 dansmith there's also the device-to-numa affinity which I understand is important, so that your nic is affined to your dedicated cpus
15:16:12 efried mm
15:16:48 sean-k-mooney dansmith: that is out of scope of this spec but yes that is a factor
15:17:06 dansmith certainly you know I'm not arguing that operators of multi-billion-dollar clouds shouldn't have to write 255 characters of ascii syntax for this right?
15:17:22 dansmith sean-k-mooney: no, I know, I was just commenting that I know asking for the abstract isn't enough
15:17:30 dansmith sean-k-mooney: or rather, asking for his abstract example
15:17:50 sean-k-mooney so if we take numa as an example
15:17:55 sean-k-mooney we actully support both
15:18:04 sean-k-mooney if you do hw:numa_nodes=2
15:18:17 sean-k-mooney and say nothing esle we devied the cpu and ram evenly between the numa nodes
15:18:49 sean-k-mooney but you can also use hw:numa_cpu.0=2 .... to speciy the numa of cpus or ram per numa node
15:19:04 sean-k-mooney most peopel jsut hw:numa_nodes and let nova do it
15:19:58 sean-k-mooney i guess the diffferen with numa id they can detect the cpu and ram in the gues via the kernel jsut as if it was real hardware
15:20:40 sean-k-mooney pining vs floaing is a qualtive change ranter then quantative so if they care about that then need another openstack sepcifc way to determin that
15:22:04 sean-k-mooney the primary use case for this by the way that im aware of is for realtime guests where they want all the realtime cores to be pinned and the non realtime managment core to be floating so that they can get the most desity out of a host
15:22:08 dansmith efried: so I guess to close,
15:23:18 dansmith efried: my preference (and what I thought we were moving towards) was a future where resource (amounts) are specified in placement syntax, and topologies and the like are assigned sanely. Tweaks to how those look (like I really need this thing pinned here) would be hw:things but where the hw: is the optional
15:23:52 dansmith meaning, usually you just ask for the amount of stuff you want and we give it to you in a sane arrangement which is fine for most people,
15:24:07 dansmith and hw:tweak your way to a specific arrangement if that's important
15:24:20 dansmith a more clear delineation of which thing does what
15:24:40 dansmith with the hw: being the primary, then I'm not sure how you know when you do or don't or can't specify some resource count as well
15:25:13 dansmith but anyway, I grok the complexity on both sides and I'm sure both will have warts
15:25:37 sean-k-mooney i could see that working if we had a speartion between the placment query and the resouces*:* element in the extra specs
15:25:56 efried We're very close to agreeing here. I want the clear delineation to be that placement-ese syntax doesn't have side effects. That's the same delineation as you describe in *most* cases.
15:26:43 dansmith well, in what I said above, asking for PCPUS:2 would get you two pinned cpus, which I think you're describing as a side effect
15:26:59 sean-k-mooney dansmith: in your world would you see the need to supprot the number group syntax in the flavor for example
15:27:55 dansmith sean-k-mooney: I will totally admit that I don't (ever) have the resource group numbering quagmire swapped into my head, but I assume yeah, you might need an extension to placement syntax to group those things if that's what you mean
15:28:02 dansmith resources:2:PCPUS=2 I assume
15:28:53 sean-k-mooney the numbered sysntax means that all resouce requst in that group must come from the same RP
15:29:19 sean-k-mooney so if you have resources:2:PCPUS=2,MEMORY_MB=1024
15:29:34 dansmith ack
15:29:39 sean-k-mooney then you need 2 pinned cpus and 1G of memory form the same resouce provider
15:29:43 dansmith yeah, I dunno, that's encoding a little bit of topology into the placement syntax,
15:29:54 sean-k-mooney so if you use that and we move to numa in placmenet it will break
15:30:05 dansmith where we should be able to parse the hw: things and alter the actual things we ask placement for inside nova I think
15:30:16 efried dansmith: exactly ^
15:30:36 sean-k-mooney yes so if you only used the unmbered syntax
15:30:50 sean-k-mooney and had the hw:* things tweek the toplogy i can see it working
15:30:52 efried The existing (unsuffixed) placement syntax is already pretty complicated to understand. Trying to add group identifiers to that will make it worse.
15:30:56 sean-k-mooney as nova can alter it interally
15:31:14 efried and then you have to understand *how* we model numa in placement
15:31:24 sean-k-mooney well we already support grouping of bothe the resouce and trait requests
15:31:26 dansmith sean-k-mooney: right, so I'd say do that.. placement syntax for totals, hw syntax for details, let nova alter what we ask placement for based on any tweaks you asked for
15:31:29 efried and how group identifiers correspond to that.
15:31:44 dansmith efried: I totes don't understand how current placement syntax is hard to understand
15:31:54 sean-k-mooney dansmith: ya i could see that working as we aready alter stuff with the prefilters now
15:32:17 efried dansmith: maybe you're too close to it
15:32:36 sean-k-mooney dansmith: its complicated if you use the numbered groups
15:32:49 sean-k-mooney if you dont ist not that bad and its consitent
15:33:33 dansmith sean-k-mooney: but we don't use numbered groups in the flavor placement syntax today right?
15:33:35 sean-k-mooney your are forced to use the groups today if you want to apply trait requirement to specific resouces
15:33:43 sean-k-mooney dansmith: we do
15:33:47 efried I think we support it, yes
15:33:50 sean-k-mooney well you can use them
15:33:56 efried we just don't have many use cases that need it today.
15:34:01 efried perhaps... none?
15:34:04 dansmith efried: well, I've had to explain it a few times and it has always been really nice, but okay
15:34:21 dansmith okay, I didn't know we even supported the group numbering today
15:34:36 sean-k-mooney it was added so you could say these traits are for my nic and these other traits are for my cpu
15:35:10 sean-k-mooney but since we dont have traiats that apply to mulple resouce classes today? do we? i dont think its stictly required
15:35:18 dansmith sean-k-mooney: is the reason we need that an artifact of how placement works or something?
15:35:39 dansmith because I would think that with no traits that apply to both cpus and nics that wouldn't be a thing I need to specify
15:35:48 efried dansmith: I get what you're saying. For most cases, the placement-ese syntax isn't any more complicated than the hw:-ese syntax. I'm asserting that we will for sure need the latter, and understanding *both* is harder than understanding *one* of them.
15:35:57 sean-k-mooney well you cant do resocues:<resouce class>=X; <traits for that RC>
15:36:32 sean-k-mooney so you have to do resources1:PCPU traits1=AVX=required
15:36:39 dansmith efried: I'm saying I think placement syntax (not including groups) is simpler and more consistent than hw syntax and is easier to understand as a whole DSL on that basis
15:36:57 sean-k-mooney where the sufix 1 in this case pairs the trait and request group
15:37:21 efried sean-k-mooney: which only matters if we have multiple providers, which today is only the case for VGPU and bandwidth.
15:37:28 efried right?
15:37:34 dansmith efried: and I'm also saying that I can understand the language of placement syntax in a few minutes, but never learn the taxonomy that is hw: without docs always in front of me
15:37:35 sean-k-mooney yes
15:37:56 efried dansmith: ack
15:37:57 sean-k-mooney although i thory it would matter for cinder volumes too with sharing providers
15:38:14 efried in theory, yes. in theory it also matters for numa.
15:38:19 sean-k-mooney but ya its only imporant for nested resouce providres or sharing
15:38:56 sean-k-mooney if we have 1 RP per compute then you never need grouping
15:39:08 sean-k-mooney with no nesting or sharing
15:40:30 efried We've done all this work to support nesting and grouping primarily for numa and shared storage. Only in train did we get to a point where placement is powerful enough to do what we want for numa. And in this release we've landed a couple of patches to make nova use those placement microversions.
15:40:33 sean-k-mooney dansmith: if we removed supprot for the group syntax then it would make hw:* been used for congriuation and resouce: be used for quantity much simpler
15:40:37 efried so we're making progress.
15:40:56 efried but the last jump, modeling numa in placement from nova, is going to be really hard.
15:41:20 dansmith sean-k-mooney: ack
15:41:31 efried I would be fine with that fwiw.
15:41:53 efried though it would technically be removing functionality that
15:42:01 efried ...that we've already released with
15:42:14 efried even if it's pretty unlikely anyone is using it.
15:42:16 sean-k-mooney well yes but we have depreacated things in the past.
15:42:19 efried yeah
15:42:58 efried this becomes relevant for cyborg too btw.
15:43:02 sean-k-mooney i think there are 3 distinct things. Resocues:* for quantiy, hw:* for configuration and traits:* for capablities
15:43:10 alex_xu i guess nobody using group now
15:43:31 efried and, dansmith, is a good example of where we could (but I really don't want to) use placement-ese to have side effects.
15:43:37 sean-k-mooney at the moment we allow hw:* to be renderd into resouces:* and traits:*
15:43:50 openstackgerrit Balazs Gibizer proposed openstack/nova master: DNM: Avoid PlacementFixture silently swallowing kwargs https://review.opendev.org/701754

Earlier   Later