| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-09 | |||
| 15:35:57 | sean-k-mooney | well you cant do resocues:<resouce class>=X; <traits for that RC> | |
| 15:36:32 | sean-k-mooney | so you have to do resources1:PCPU traits1=AVX=required | |
| 15:36:39 | dansmith | efried: I'm saying I think placement syntax (not including groups) is simpler and more consistent than hw syntax and is easier to understand as a whole DSL on that basis | |
| 15:36:57 | sean-k-mooney | where the sufix 1 in this case pairs the trait and request group | |
| 15:37:21 | efried | sean-k-mooney: which only matters if we have multiple providers, which today is only the case for VGPU and bandwidth. | |
| 15:37:28 | efried | right? | |
| 15:37:34 | dansmith | efried: and I'm also saying that I can understand the language of placement syntax in a few minutes, but never learn the taxonomy that is hw: without docs always in front of me | |
| 15:37:35 | sean-k-mooney | yes | |
| 15:37:56 | efried | dansmith: ack | |
| 15:37:57 | sean-k-mooney | although i thory it would matter for cinder volumes too with sharing providers | |
| 15:38:14 | efried | in theory, yes. in theory it also matters for numa. | |
| 15:38:19 | sean-k-mooney | but ya its only imporant for nested resouce providres or sharing | |
| 15:38:56 | sean-k-mooney | if we have 1 RP per compute then you never need grouping | |
| 15:39:08 | sean-k-mooney | with no nesting or sharing | |
| 15:40:30 | efried | We've done all this work to support nesting and grouping primarily for numa and shared storage. Only in train did we get to a point where placement is powerful enough to do what we want for numa. And in this release we've landed a couple of patches to make nova use those placement microversions. | |
| 15:40:33 | sean-k-mooney | dansmith: if we removed supprot for the group syntax then it would make hw:* been used for congriuation and resouce: be used for quantity much simpler | |
| 15:40:37 | efried | so we're making progress. | |
| 15:40:56 | efried | but the last jump, modeling numa in placement from nova, is going to be really hard. | |
| 15:41:20 | dansmith | sean-k-mooney: ack | |
| 15:41:31 | efried | I would be fine with that fwiw. | |
| 15:41:53 | efried | though it would technically be removing functionality that | |
| 15:42:01 | efried | ...that we've already released with | |
| 15:42:14 | efried | even if it's pretty unlikely anyone is using it. | |
| 15:42:16 | sean-k-mooney | well yes but we have depreacated things in the past. | |
| 15:42:19 | efried | yeah | |
| 15:42:58 | efried | this becomes relevant for cyborg too btw. | |
| 15:43:02 | sean-k-mooney | i think there are 3 distinct things. Resocues:* for quantiy, hw:* for configuration and traits:* for capablities | |
| 15:43:10 | alex_xu | i guess nobody using group now | |
| 15:43:31 | efried | and, dansmith, is a good example of where we could (but I really don't want to) use placement-ese to have side effects. | |
| 15:43:37 | sean-k-mooney | at the moment we allow hw:* to be renderd into resouces:* and traits:* | |
| 15:43:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: DNM: Avoid PlacementFixture silently swallowing kwargs https://review.opendev.org/701754 | |
| 15:44:05 | dansmith | efried: cyborg is a good example of side effects? | |
| 15:44:05 | efried | like programming bitstreams | |
| 15:44:44 | dansmith | efried: isn't the bitstream specified in the device profile in cyborg? | |
| 15:45:03 | efried | yes. | |
| 15:45:16 | sean-k-mooney | dansmith: i think it a metadata key on the guest image | |
| 15:45:28 | efried | no (at least not yet) | |
| 15:45:36 | sean-k-mooney | the type of accelerator is definetly in the device profiel | |
| 15:45:45 | dansmith | I think the bitstream is too | |
| 15:45:55 | sean-k-mooney | the important thing is its not in the nova flavor | |
| 15:46:22 | dansmith | efried: I'm not sure what you mean exactly by cyborg would be a good example of side effects (affecting the bitstream we use?) | |
| 15:47:10 | efried | in the future we want to "prefer" hosts that have an accel with the desired bitstream already on them. In that case the placement request would us preferred=BITSTREAM_XXX. But we don't want to expose trait:BISTREAM_XXX=preferred in the flavor extra specs. | |
| 15:47:13 | dansmith | efried: are you saying that if cyborg represented possible devices that could be composed by programming a, say, gzip bitstream into a fpga as a resource in placement that asking for that thing would cause you to get one? | |
| 15:47:14 | efried | kind of thing. | |
| 15:47:41 | sean-k-mooney | well vgpus is a example of Resources:vGPU=1 haveing an effect of actully provision a gpu for the guest | |
| 15:47:46 | dansmith | that's maybe a next-level leap from what I'm saying, | |
| 15:48:03 | dansmith | but it's also a much more user/ops-friendly way of stitching high level functions together | |
| 15:48:37 | dansmith | i.e. instead of plugging low-level nova into low-level cyborg, you define an abstract thing in cyborg which becomes a high-level resource in nova that you just ask for by symbolic name, | |
| 15:48:46 | efried | YES | |
| 15:48:47 | dansmith | which is more what I think we should be shooting for, compared to the former | |
| 15:49:19 | dansmith | you say YES, but I feel like you were arguing NO a minute ago, so I'm confused | |
| 15:50:01 | efried | Today the way we model resources in placement is very similar to the way nova thinks of them | |
| 15:50:09 | sean-k-mooney | how about this. for the mix cpu case. we could say that if you do VCPU:2,PCPU:6 we will generate the topogy for you but you can override it with hw:gust_pinned_cpus | |
| 15:50:13 | efried | so just understanding placement syntax is enough | |
| 15:50:50 | sean-k-mooney | but you could just set hw:guest_pinned_cpus too | |
| 15:51:05 | efried | but very soon (numa modeling, cyborg) we will have to model in placement in ways that are less intuitive | |
| 15:51:40 | efried | if we continue to support placement-ese in that future, it will required the user to understand that extra translation | |
| 15:51:49 | efried | I'm just suggesting we hide that translation in nova. | |
| 15:52:14 | dansmith | but then we have to invent syntax for everything no? | |
| 15:52:29 | efried | we already have that syntax | |
| 15:52:33 | sean-k-mooney | no only if we want to expose contol over that translation | |
| 15:53:06 | dansmith | efried: we've already invented new syntax for "please include me an accelerator by the name of X" | |
| 15:53:39 | dansmith | efried: if that was a placement resource, we wouldn't have had to do that, but we did and now sundar is inspecting internal flavor details of this bespoke syntax everywhere | |
| 15:53:59 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: Add recreate test for bug 1855927 https://review.opendev.org/701756 | |
| 15:53:59 | openstack | bug 1855927 in OpenStack Compute (nova) "_poll_unconfirmed_resizes may not retry later if confirm_resize fails in API" [Low,In progress] https://launchpad.net/bugs/1855927 - Assigned to Matt Riedemann (mriedem) | |
| 15:53:59 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: Ensure source service is up before resizing/migrating https://review.opendev.org/701757 | |
| 15:54:29 | dansmith | efried: and we will have to invent new syntax when we want to ask it to be hung on a specific numa node I think too right? | |
| 15:54:44 | dansmith | so we'll have two unique things for accelerators: which one and how to attach | |
| 15:54:48 | sean-k-mooney | dansmith: well that would only work if each resouce class is only owned by 1 project | |
| 15:55:10 | sean-k-mooney | nova already "owns" the VGPU resouce class | |
| 15:55:26 | dansmith | wat | |
| 15:55:46 | sean-k-mooney | so that would mean we woudl have to have a seperate CYBORG_VGPU resouce class for cyborg manged VGPUs | |
| 15:55:57 | efried | naw | |
| 15:56:07 | dansmith | we're assigning ownership of *classes* now? | |
| 15:56:08 | efried | per-project ownership is at the RP level. | |
| 15:56:09 | sean-k-mooney | dansmith: to use the vgpu support in nova today you set resouce:vgpu=1 | |
| 15:56:17 | dansmith | I thought only instances of RPs were owned | |
| 15:56:21 | efried | ^ | |
| 15:56:21 | dansmith | yeah what efried said | |
| 15:56:47 | openstackgerrit | Merged openstack/nova master: Do not reschedule on ExternalNetworkAttachForbidden https://review.opendev.org/694179 | |
| 15:56:53 | openstackgerrit | Merged openstack/nova master: nova-net: Remove firewall support (pt. 3) https://review.opendev.org/700511 | |
| 15:56:56 | sean-k-mooney | well yes but how would you know in the resouce:vgpu=1 if libvirt or cyborge shoudl handel that request | |
| 15:57:00 | openstackgerrit | Merged openstack/nova stable/rocky: Add functional recreate test for bug 1852610 https://review.opendev.org/698108 | |
| 15:57:00 | openstack | bug 1852610 in OpenStack Compute (nova) rocky "API allows source compute service/node deletion while instances are pending a resize confirm/revert" [Medium,In progress] https://launchpad.net/bugs/1852610 - Assigned to Matt Riedemann (mriedem) | |
| 15:57:05 | openstackgerrit | Merged openstack/nova stable/rocky: Add functional recreate revert resize test for bug 1852610 https://review.opendev.org/698110 | |
| 15:57:34 | dansmith | sean-k-mooney: libvirt handles it in both cases eventually, but that's my point about all of this | |
| 15:57:55 | dansmith | sean-k-mooney: we should have the user ask for what they want, not what they want how they think it's plumbed today | |
| 15:58:07 | dansmith | because if we move vgpu handling completely to cyborg in the future, | |
| 15:58:22 | dansmith | the syntax shouldn't have to change, the users shouldn't have to know how the cloud is doing that behind the scenes, etc | |
| 15:58:32 | dansmith | we're supposed to be an abstraction, not just an API | |
| 15:58:53 | sean-k-mooney | yes i agree with that in principal. and yes libvirt will provide it in either case where it is invetoried by the livbrit driver or cyborg | |
| 15:59:48 | efried | Yes, we had to invent syntax for cyborg, and we'll have to extend existing syntax for numa. IMO that's not a bad thing. | |
| 15:59:48 | efried | IF we could reasonably use placement syntax for everything, that would be okay, but that is not going to be possible. | |
| 15:59:48 | efried | So we're going to need the snowflake syntax no matter what. | |
| 15:59:48 | efried | So when it comes to choosing between using placement syntax in an awkward way, or inventing a new fit-for-purpose custom syntax, I vote for the latter. | |
| 15:59:48 | efried | Currently we have ways to use the placement syntax mostly-not-awkwardly, which is where we're conflicting. | |
| 15:59:50 | sean-k-mooney | part of the proably however was if i wanted 2 fpga with differet bit stream i might need to select different devices to be compatible | |
| 15:59:59 | efried | I feel like we're repeating at this point. | |
| 16:00:05 | efried | I need to run to a meeting. BBIAB | |
| 16:00:55 | sean-k-mooney | that 2 devices with different requirements lead to the need for use to model each request as a seperate request group with different traits | |