Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-01-09
15:35:10 sean-k-mooney but since we dont have traiats that apply to mulple resouce classes today? do we? i dont think its stictly required
15:35:18 dansmith sean-k-mooney: is the reason we need that an artifact of how placement works or something?
15:35:39 dansmith because I would think that with no traits that apply to both cpus and nics that wouldn't be a thing I need to specify
15:35:48 efried dansmith: I get what you're saying. For most cases, the placement-ese syntax isn't any more complicated than the hw:-ese syntax. I'm asserting that we will for sure need the latter, and understanding *both* is harder than understanding *one* of them.
15:35:57 sean-k-mooney well you cant do resocues:<resouce class>=X; <traits for that RC>
15:36:32 sean-k-mooney so you have to do resources1:PCPU traits1=AVX=required
15:36:39 dansmith efried: I'm saying I think placement syntax (not including groups) is simpler and more consistent than hw syntax and is easier to understand as a whole DSL on that basis
15:36:57 sean-k-mooney where the sufix 1 in this case pairs the trait and request group
15:37:21 efried sean-k-mooney: which only matters if we have multiple providers, which today is only the case for VGPU and bandwidth.
15:37:28 efried right?
15:37:34 dansmith efried: and I'm also saying that I can understand the language of placement syntax in a few minutes, but never learn the taxonomy that is hw: without docs always in front of me
15:37:35 sean-k-mooney yes
15:37:56 efried dansmith: ack
15:37:57 sean-k-mooney although i thory it would matter for cinder volumes too with sharing providers
15:38:14 efried in theory, yes. in theory it also matters for numa.
15:38:19 sean-k-mooney but ya its only imporant for nested resouce providres or sharing
15:38:56 sean-k-mooney if we have 1 RP per compute then you never need grouping
15:39:08 sean-k-mooney with no nesting or sharing
15:40:30 efried We've done all this work to support nesting and grouping primarily for numa and shared storage. Only in train did we get to a point where placement is powerful enough to do what we want for numa. And in this release we've landed a couple of patches to make nova use those placement microversions.
15:40:33 sean-k-mooney dansmith: if we removed supprot for the group syntax then it would make hw:* been used for congriuation and resouce: be used for quantity much simpler
15:40:37 efried so we're making progress.
15:40:56 efried but the last jump, modeling numa in placement from nova, is going to be really hard.
15:41:20 dansmith sean-k-mooney: ack
15:41:31 efried I would be fine with that fwiw.
15:41:53 efried though it would technically be removing functionality that
15:42:01 efried ...that we've already released with
15:42:14 efried even if it's pretty unlikely anyone is using it.
15:42:16 sean-k-mooney well yes but we have depreacated things in the past.
15:42:19 efried yeah
15:42:58 efried this becomes relevant for cyborg too btw.
15:43:02 sean-k-mooney i think there are 3 distinct things. Resocues:* for quantiy, hw:* for configuration and traits:* for capablities
15:43:10 alex_xu i guess nobody using group now
15:43:31 efried and, dansmith, is a good example of where we could (but I really don't want to) use placement-ese to have side effects.
15:43:37 sean-k-mooney at the moment we allow hw:* to be renderd into resouces:* and traits:*
15:43:50 openstackgerrit Balazs Gibizer proposed openstack/nova master: DNM: Avoid PlacementFixture silently swallowing kwargs https://review.opendev.org/701754
15:44:05 efried like programming bitstreams
15:44:05 dansmith efried: cyborg is a good example of side effects?
15:44:44 dansmith efried: isn't the bitstream specified in the device profile in cyborg?
15:45:03 efried yes.
15:45:16 sean-k-mooney dansmith: i think it a metadata key on the guest image
15:45:28 efried no (at least not yet)
15:45:36 sean-k-mooney the type of accelerator is definetly in the device profiel
15:45:45 dansmith I think the bitstream is too
15:45:55 sean-k-mooney the important thing is its not in the nova flavor
15:46:22 dansmith efried: I'm not sure what you mean exactly by cyborg would be a good example of side effects (affecting the bitstream we use?)
15:47:10 efried in the future we want to "prefer" hosts that have an accel with the desired bitstream already on them. In that case the placement request would us preferred=BITSTREAM_XXX. But we don't want to expose trait:BISTREAM_XXX=preferred in the flavor extra specs.
15:47:13 dansmith efried: are you saying that if cyborg represented possible devices that could be composed by programming a, say, gzip bitstream into a fpga as a resource in placement that asking for that thing would cause you to get one?
15:47:14 efried kind of thing.
15:47:41 sean-k-mooney well vgpus is a example of Resources:vGPU=1 haveing an effect of actully provision a gpu for the guest
15:47:46 dansmith that's maybe a next-level leap from what I'm saying,
15:48:03 dansmith but it's also a much more user/ops-friendly way of stitching high level functions together
15:48:37 dansmith i.e. instead of plugging low-level nova into low-level cyborg, you define an abstract thing in cyborg which becomes a high-level resource in nova that you just ask for by symbolic name,
15:48:46 efried YES
15:48:47 dansmith which is more what I think we should be shooting for, compared to the former
15:49:19 dansmith you say YES, but I feel like you were arguing NO a minute ago, so I'm confused
15:50:01 efried Today the way we model resources in placement is very similar to the way nova thinks of them
15:50:09 sean-k-mooney how about this. for the mix cpu case. we could say that if you do VCPU:2,PCPU:6 we will generate the topogy for you but you can override it with hw:gust_pinned_cpus
15:50:13 efried so just understanding placement syntax is enough
15:50:50 sean-k-mooney but you could just set hw:guest_pinned_cpus too
15:51:05 efried but very soon (numa modeling, cyborg) we will have to model in placement in ways that are less intuitive
15:51:40 efried if we continue to support placement-ese in that future, it will required the user to understand that extra translation
15:51:49 efried I'm just suggesting we hide that translation in nova.
15:52:14 dansmith but then we have to invent syntax for everything no?
15:52:29 efried we already have that syntax
15:52:33 sean-k-mooney no only if we want to expose contol over that translation
15:53:06 dansmith efried: we've already invented new syntax for "please include me an accelerator by the name of X"
15:53:39 dansmith efried: if that was a placement resource, we wouldn't have had to do that, but we did and now sundar is inspecting internal flavor details of this bespoke syntax everywhere
15:53:59 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: Ensure source service is up before resizing/migrating https://review.opendev.org/701757
15:53:59 openstack bug 1855927 in OpenStack Compute (nova) "_poll_unconfirmed_resizes may not retry later if confirm_resize fails in API" [Low,In progress] https://launchpad.net/bugs/1855927 - Assigned to Matt Riedemann (mriedem)
15:53:59 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: Add recreate test for bug 1855927 https://review.opendev.org/701756
15:54:29 dansmith efried: and we will have to invent new syntax when we want to ask it to be hung on a specific numa node I think too right?
15:54:44 dansmith so we'll have two unique things for accelerators: which one and how to attach
15:54:48 sean-k-mooney dansmith: well that would only work if each resouce class is only owned by 1 project
15:55:10 sean-k-mooney nova already "owns" the VGPU resouce class
15:55:26 dansmith wat
15:55:46 sean-k-mooney so that would mean we woudl have to have a seperate CYBORG_VGPU resouce class for cyborg manged VGPUs
15:55:57 efried naw
15:56:07 dansmith we're assigning ownership of *classes* now?
15:56:08 efried per-project ownership is at the RP level.
15:56:09 sean-k-mooney dansmith: to use the vgpu support in nova today you set resouce:vgpu=1
15:56:17 dansmith I thought only instances of RPs were owned
15:56:21 dansmith yeah what efried said
15:56:21 efried ^
15:56:47 openstackgerrit Merged openstack/nova master: Do not reschedule on ExternalNetworkAttachForbidden https://review.opendev.org/694179
15:56:53 openstackgerrit Merged openstack/nova master: nova-net: Remove firewall support (pt. 3) https://review.opendev.org/700511
15:56:56 sean-k-mooney well yes but how would you know in the resouce:vgpu=1 if libvirt or cyborge shoudl handel that request
15:57:00 openstack bug 1852610 in OpenStack Compute (nova) rocky "API allows source compute service/node deletion while instances are pending a resize confirm/revert" [Medium,In progress] https://launchpad.net/bugs/1852610 - Assigned to Matt Riedemann (mriedem)
15:57:00 openstackgerrit Merged openstack/nova stable/rocky: Add functional recreate test for bug 1852610 https://review.opendev.org/698108
15:57:05 openstackgerrit Merged openstack/nova stable/rocky: Add functional recreate revert resize test for bug 1852610 https://review.opendev.org/698110
15:57:34 dansmith sean-k-mooney: libvirt handles it in both cases eventually, but that's my point about all of this
15:57:55 dansmith sean-k-mooney: we should have the user ask for what they want, not what they want how they think it's plumbed today
15:58:07 dansmith because if we move vgpu handling completely to cyborg in the future,
15:58:22 dansmith the syntax shouldn't have to change, the users shouldn't have to know how the cloud is doing that behind the scenes, etc
15:58:32 dansmith we're supposed to be an abstraction, not just an API
15:58:53 sean-k-mooney yes i agree with that in principal. and yes libvirt will provide it in either case where it is invetoried by the livbrit driver or cyborg
15:59:48 efried Currently we have ways to use the placement syntax mostly-not-awkwardly, which is where we're conflicting.
15:59:48 efried So when it comes to choosing between using placement syntax in an awkward way, or inventing a new fit-for-purpose custom syntax, I vote for the latter.
15:59:48 efried So we're going to need the snowflake syntax no matter what.
15:59:48 efried IF we could reasonably use placement syntax for everything, that would be okay, but that is not going to be possible.
15:59:48 efried Yes, we had to invent syntax for cyborg, and we'll have to extend existing syntax for numa. IMO that's not a bad thing.

Earlier   Later