| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-12 | |||
| 20:16:03 | edleafe | efried: isn't that always the case? :) | |
| 20:16:28 | edleafe | mriedem: ah, your laptop's background image | |
| 20:16:40 | mriedem | i use it as motivation | |
| 20:16:46 | mriedem | to be jerseylicious | |
| 20:17:28 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add regression test for BFV+IsolatedHostsFilter failure https://review.openstack.org/543602 | |
| 20:17:29 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Handle volume-backed instances in IsolatedHostsFilter https://review.openstack.org/543603 | |
| 20:17:29 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Fix docs for IsolatedHostsFilter https://review.openstack.org/543604 | |
| 20:18:43 | mriedem | efried: how much of the change to the spec is implementation details vs just high level design? | |
| 20:19:48 | efried | mriedem: Let me finish my initial reread and get back to you on that. But at a glance, the main thrust of the spec is implementing root_provider_uuid and parent_provider_uuid, which is done. What remains, and what we should probably give a little more detail about, is the GET /allocation_candidates changes. | |
| 20:20:59 | mriedem | then for the done stuff, say it's done in queens and update the not-yet-done-stuff parts | |
| 20:21:49 | efried | mriedem: That latter part isn't even mentioned in this spec. | |
| 20:22:21 | efried | But yeah. I'll do the needful. | |
| 20:22:33 | efried | There will be controversy over NUMA, though. | |
| 20:22:43 | mriedem | efried: well, if the NRP spec is all about stuff that's already done, then there probably isn't a need to re-propose it, | |
| 20:22:56 | mriedem | and we create a new bp/spec for the things that are pending and move all of those changes under that new bp? | |
| 20:23:11 | mriedem | but i deferred the NRP bp from queens b/c there were remaining things to use it | |
| 20:23:20 | efried | mriedem: Finished reading now, and yeah, that's kinda what I was thinking. | |
| 20:24:04 | efried | mriedem: I think that (deferring/remaining) was a reflection of "scope creep". | |
| 20:24:09 | mriedem | https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/nested-resource-providers.html#work-items | |
| 20:24:16 | mriedem | the last 2 bullets in there aren't done are they? | |
| 20:24:22 | efried | mriedem: Nor will they be. | |
| 20:24:28 | efried | Cause that's no longer the plan. | |
| 20:24:33 | mriedem | oy | |
| 20:24:50 | mriedem | ok, so the nrp spec became the api stuff and foundational plumbing, | |
| 20:25:08 | mriedem | and support for modeling numa and sriov are separate new thing snow | |
| 20:25:10 | mriedem | yeah? | |
| 20:25:30 | efried | That's part of the story, yes. | |
| 20:25:51 | mriedem | if that's the case, then maybe we should amend the queens spec, and mark the bp as done for queens, even though we don't have numa/sriov modeling with it | |
| 20:26:22 | efried | If the update_provider_tree stuff had gotten finished in Q, it would have been tucked under the umbrella of the nrp blueprint (most of it was merged under that topic) even though it was never mentioned or even hinted at in the nrp spec. | |
| 20:26:41 | edleafe | jaypipes: cdent: efried: dansmith: mriedem: wrote up something quick on my ideas on affinity: https://blog.leafe.com/modeling-affinity/ | |
| 20:26:49 | edleafe | comments welcome! | |
| 20:26:53 | mriedem | efried: that's an implementation detail though isn't it? | |
| 20:27:54 | cdent | thanks edleafe enqueued for tomorrow, currently writing my own | |
| 20:27:54 | efried | mriedem: Which, update_provider_tree? I don't think there's any possible interpretation by which that stack of work could be considered just "implementation detail". | |
| 20:28:38 | mriedem | efried: ok, i guess i need to read your provider tree spec then | |
| 20:28:51 | mriedem | which i thought you said was less of a spec and more just docs of the thing that merged in queens? | |
| 20:29:06 | efried | ...which needed to be specified in some form, yeah. | |
| 20:29:38 | efried | And which didn't quite get finished in queens, and is a big enough piece of work to warrant a bp and spec of its own (at least if it wasn't going to be described in the nrp spec) | |
| 20:31:55 | mriedem | ok, i'll get to new business some other day; was just trying to go through easy re-approvals | |
| 20:32:11 | mriedem | i will mentally note that this isn't an easy re-approval | |
| 20:32:50 | efried | mriedem: Anyway, to tie that back together, I believe update_provider_tree changes the NUMA plan as described around https://review.openstack.org/#/c/505209/7/specs/queens/approved/nested-resource-providers.rst@136 | |
| 20:34:08 | efried | mriedem: Because we're no longer going to be basing anything on get_available_resource. Rather, we'll ask virt to report NUMA topology through update_provider_tree (in some standardized format TBD? Swhat jaypipes and I are debating in https://review.openstack.org/543062) and then deal with the pinning/affinitizing from there. | |
| 20:34:42 | efried | Suspect there will be much PTGing at the PTG about this. | |
| 20:35:01 | mriedem | ok - thinking again about just amending the queens spec, making a note that the spec is just focusing on groundwork stuff, might give examples of use cases, but details for supporting other use cases (sriov/numa) are going to be found in later specs | |
| 20:35:30 | efried | mriedem: Do you want me to propose that edit? | |
| 20:35:36 | mriedem | up to you | |
| 20:35:37 | efried | Do we do that? | |
| 20:35:49 | mriedem | sometimes if things changed quite a bit | |
| 20:35:59 | mriedem | the thing i wouldn't want to say is, | |
| 20:36:16 | mriedem | the nrp spec is marked 'implemented' for queens and it says 'we'll model sriov with this in queens' and then we didn't | |
| 20:36:44 | mriedem | if we aren't going to re-propose the nrp spec for rocky because it's basically done and the rest is some other new spec, then we should say it was just done in queens | |
| 20:36:48 | mriedem | and amend accordingly | |
| 20:36:58 | efried | mriedem: Okay, well, the spec does say the NUMA/SR-IOV stuff won't be done in Q. | |
| 20:37:09 | efried | But it also asserts pretty forcefully what that will look like when it does get implemented. | |
| 20:37:51 | mriedem | the goal in queens was sriov | |
| 20:37:55 | mriedem | https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/nested-resource-providers.html#work-items | |
| 20:38:09 | mriedem | "Please note that not all of this spec is expected to be implemented in a single release cycle. At the Queens PTG we agreed that fully suppporting NUMA will probably have to be deferred to the next release." just came out of the ptg saying we wouldn't try to get numa done | |
| 20:38:15 | mriedem | which implies sriov would be supported in queens | |
| 20:47:05 | efried | which is kind of a wow, because we're not even close to supporting SR-IOV. It's more than just NRP that needs to be wrapped up to make that happen. | |
| 20:48:12 | efried | mriedem: Softening those statements jeeest a leetle bit would make them more or less true: "...the ability to *model* NUMA and SR-IOV..." | |
| 20:48:33 | efried | You can model it. You just can't *do* anything with that model. | |
| 20:49:03 | efried | And you can't actually model it without writing more code on the nova side; but the placement side will handle it. | |
| 20:49:20 | efried | Except for the GET /allocation_candidates thing, which is required if it's going to be usable at all. | |
| 20:49:41 | mriedem | https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/nested-resource-providers.html#rest-api-impact | |
| 20:49:42 | efried | So yeah, it's a mess. | |
| 20:49:45 | mriedem | "The filter parameter root={uuid} will not be added to GET /allocation_candidates, as this call is for a specific use case for the Nova scheduler, and there is no use case for it." | |
| 20:50:07 | efried | Still true. | |
| 20:50:19 | mriedem | i think the nrp spec is mostly just about the modeling and placement api bits, | |
| 20:50:40 | mriedem | i think it's fine to say nova's usage of it for certain use cases (sriov/numa) are separate efforts that will build on this later | |
| 20:50:47 | efried | But GET /allocation_candidates - the algorithm on the back end - needs to be updated to understand and handle nested providers. Today it doesn't. If you have a tree model, you get no candidates. | |
| 20:51:12 | mriedem | sure, that's a new work effort | |
| 20:51:15 | efried | That's something that's not mentioned at all, but needs to be, and needs to be completed in Rocky ASAP. | |
| 20:53:03 | mriedem | so skimming over the spec again, it seems like if one just removed (or somehow re-worded) the parts about the scheduler reporting client in nova, it would explain what was completed in queens | |
| 20:53:11 | mriedem | agree? | |
| 20:53:32 | mriedem | add root_provider_uuid and parent_provider_uuid to GET /resource_providers, fin | |
| 20:54:04 | mriedem | .. note:: Future work will be needed client-side to model things like SR-IOV and NUMA for nova to leverage the Placement API for nested resource providers. | |
| 20:54:24 | efried | and ?tree= to the latter (written as ?root= in the spec, was renamed during impl) | |
| 20:54:57 | mriedem | you mean ?in_tree? | |
| 20:55:00 | mriedem | bazinga! | |
| 20:55:33 | mriedem | yeah so amend the ?in_tree thingy (we still haven't defined how that filter works i don't think) | |
| 20:55:46 | mriedem | remove/reword the parts about the scheduler client - say those are done separately later | |
| 20:55:49 | efried | fusque | |
| 20:55:55 | mriedem | and i think we say the rest is done in queens | |
| 20:56:08 | mriedem | with caveats like the GET /allocation_candidates thing | |
| 20:56:09 | efried | Oh, we nailed down in_tree. It's implemented. | |
| 20:56:36 | mriedem | i know it's in the api | |
| 20:56:46 | mriedem | but the semantics of how that works is not documented anywhere as far as i know | |
| 20:56:56 | mriedem | i remember bringing this up in a review or the ML or something at one point | |
| 20:57:22 | mriedem | https://review.openstack.org/#/c/520663/9/nova/tests/unit/scheduler/client/test_report.py@1400 | |
| 20:57:34 | efried | https://developer.openstack.org/api-ref/placement/#list-resource-providers | |
| 20:57:50 | mriedem | yeah i know about that, all it says is "A UUID of a resource provider. The returned resource providers will be in the same “provider tree” as the specified provider." | |
| 20:58:23 | jaypipes | what's unclear about that? | |
| 20:58:26 | mriedem | based on your reply to https://review.openstack.org/#/c/520663/9/nova/tests/unit/scheduler/client/test_report.py@1400 it means, in_tree with *any* node in the tree returns the entire tree | |
| 20:58:48 | jaypipes | ah I see | |
| 20:59:10 | mriedem | in my example, B and D are in the same tree because they share a common ancestor | |
| 20:59:18 | mriedem | but i wasn't sure about that | |
| 20:59:24 | mriedem | i mean on first reading of the api-ref | |
| 21:00:11 | efried | It doesn't say "subtree" or "below". I mean, we could add words to explain it more fully, including an example like the one in the comment. | |
| 21:00:24 | mriedem | http://lists.openstack.org/pipermail/openstack-dev/2017-December/125453.html | |
| 21:00:28 | mriedem | yeah that's what i asked for in ^ | |