Posted	Nick	Remark
#openstack-nova - 2017-09-05
21:17:41	jaypipes	guys, can I hit pause on this conversation for 5 minutes so I can finish my braindump?
21:17:48	efried	...with some kind of "generic device manager" which talks to Placement
21:17:50	efried	sure.
21:19:26	dansmith	efried: definitely, to the extent we can
21:20:22	openstackgerrit	Dan Smith proposed openstack/nova-specs master: Add migration-allocations spec https://review.openstack.org/498510
21:59:53	cfriesen__	when calling a remotable object function, does nova-conductor do a keystone token validation?
22:01:40	dansmith	no
22:21:40	openstackgerrit	Matt Riedemann proposed openstack/nova-specs master: Spec for flavor description https://review.openstack.org/501017
22:21:42	mriedem	Kevin_Zheng: ^
22:23:43	jaypipes	efried, dansmith: sorry, dinner got in the way..
22:23:55	jaypipes	efried, dansmith: dumped some thoughts here: http://paste.openstack.org/show/620456/
22:24:46	efried	jaypipes ...
22:25:54	jaypipes	efried: yes?
22:26:18	efried	jaypipes Sorry, that's "Message received, processing..."
22:26:30	jaypipes	efried: ack
22:29:19	efried	jaypipes Yes, just so. But I thought this was the part we were all in sync with. The part we hadn't yet come to grips with was the hand-waving on L35-6.
22:30:14	jaypipes	efried: no, I can come up with an *internal* representation for the request. what I can't figure out is how to map that to a *single* REST API call.
22:30:24	efried	Right.
22:31:48	efried	jaypipes Don't kill me don't kill me but... what if I numbered the query param keys?
22:32:10	efried	resources1=...&traits1=...&resources2=...&traits2=...
22:32:34	jaypipes	efried: uhm, eww? :)
22:32:56	efried	Well, yeah it's eww, that's a given. But it's clear that it could work.
22:33:07	efried	It could also conceivably answer the thing where it matters what order we attach devices in...
22:33:18	efried	...which some people care about.
22:33:34	jaypipes	efried: not sure I follow why the attachment order matters to the placement API.
22:33:55	efried	It doesn't, but when the request comes in, don't those keys come through? Mebbe not, sorry.
22:35:21	efried	Anyway, for the general case, non-numbered querystring keys work just fine; in fact they don't even have to be numbers - we just parse the prefix and group by the suffix, whatever it is.
22:36:14	jaypipes	efried: the problem with adding this to the placement REST API is that then we're adding a bunch of orchestration-like logic into the placement service.
22:36:48	jaypipes	efried: instead of keeping that out of placement and relying on the caller to placement understanding that the request contains a number of "subrequests" that are for specific complex types.
22:37:07	efried	jaypipes Yeah, I admit I don't understand what is meant by "orchestration" and where that line is drawn.
22:37:36	jaypipes	efried: dealing with lots of resources as a group is orchestration.
22:38:08	jaypipes	efried: where the relationship between the resources, the dependencies between them, the constraints that each imposes on each other, etc. that's orchestration.
22:38:39	jaypipes	efried: whereas placement is designed to be a fast, efficient method of determining providers of resources for a given simple request for capacity.
22:41:21	cdent	jaypipes: can you clarify something for me: I’m not fully grokking why, in your paste, we need to make multiple requests to /a_c ? Is it is because at the point we don’t have a good way to express (in one request) what we want or is there more than that?
22:42:23	efried	're having trouble expressing it under the current design, not because it inherently crosses the line to "orchestration".
22:42:23	efried	jaypipes I'm probably being obtuse again, but I still don't really see how what we're talking about here doesn't still fall into that description. Placement is designed to grab resources from multiple resource providers. From a way-zoomed-out view, saying "you can only get stuff from a given resource class from a single provider" seems like an arbitrary limitation. Zoom back in, and it's one that seems imposed because we
22:42:47	cdent	jaypipes: on line 50 it is GROUP_A on the first go round and GROUP_B on the second?
22:43:37	efried	cdent Yeah; and/or L49 could be different egress bandwidth numbers
22:43:44	cdent	thanks
22:44:01	jaypipes	query parameters.
22:44:01	jaypipes	cdent: a couple reasons for that. the first is that it's difficult for me to map the concept of related sub-requests in a single REST API call (again, without resorting to some DSL-ish request payload) and b) because I suspect these types of use cases are just the tip of the iceberg and that the logic that relates various things with each other (GPUs, FPGAs, NIC HA groups, etc) will be different and not possible to cleanly express with simple
22:44:51	jaypipes	cdent: and yes, it would be GROUP_A on the first go around and GROUP_B on the second.
22:45:08	cdent	jaypipes: next question: step 4 sounds like a pretty big step backwards for the goals we originally expressed for allocation_candidates
22:45:21	jaypipes	cdent: could also have different amounts of egress bandwidth, different affinity policies, different all sorts of things, frnakly
22:45:32	jaypipes	cdent: Yupppp!
22:45:37	cdent	why can’t we have a complete allocation_requests (just to be explicit)
22:45:48	jaypipes	cdent: it removes the whole opaqueness aspect of the allocation request :()
22:45:57	cdent	yes much :(
22:46:07	efried	Is it because the allocation request is keyed by resource class?
22:46:15	efried	(haven't looked at that guy yet)
22:46:41	jaypipes	cdent: we can't have a complete allocation_requests unless placement is passed all of the logic about relationship between sub-resources
22:47:03	cdent	I thought that’s what nested was providing us?
22:47:04	jaypipes	efried: the allocation request is not keyed by resource class, no.
22:47:23	jaypipes	efried: the allocation request is essentially the request payload to PUT /allocations/{consumer_uuid}
22:47:29	openstackgerrit	Michael Still proposed openstack/nova master: Add release note for requiring shred 8.22 or above. https://review.openstack.org/501022
22:48:19	cdent	jaypipes: related, have you yet seen: http://lists.openstack.org/pipermail/openstack-dev/2017-September/121824.html from avolkov ?
22:48:21	efried	jaypipes And it must be able to contain resources from different RPs. So why would there be a limitation that RCs therein be unique?
22:48:32	efried	cdent That's what started this whole thing :)
22:48:38	cdent	ah, good
22:48:50	efried	cdent Back around http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-09-05.log.html#t2017-09-05T15:37:39
22:49:01	cdent	presumably the response to option 2 is “no”
22:49:04	cdent	as that’s not "real"
22:49:12	jaypipes	cdent: nested resource providers allows the placement API to understand whether, say, a PF that provides some VFs is on a particular compute node. Nested providers doesn't, however, solve the problem of how do we model the *request* for resources when the user doesn't know that there is a nested relationship between things.
22:49:40	jaypipes	efried: there is no such limitation. I'm not sure what you're getting at.
22:50:19	mikal	mriedem: I am a bad man and realized once that shred patch had merged that it probably should have had a reno, so I've added one in https://review.openstack.org/#/c/501022/
22:51:14	efried	jaypipes What you and cdent were saying about breaking opaqueness? And a step backwards for original goals of allocation_candidates. I didn't catch where that came from.
22:51:15	cdent	jaypipes: I get that the request modeling is a limitation, but if were to set that aside for a moment and we could express the request well then the we could present a complete set of allocations in response, right? The limitation as described in your paste is because the current request doesn’t have all the state.
22:52:03	cdent	efried: ideally it would be possible to take the first item in the allocation_request list and send that to /allocations/{consumer_uuid} without modifications to make a “claim”
22:52:23	efried	cdent Why doesn't that still work in this scenario?
22:52:34	jaypipes	efried: the items in the "allocation_requests" part of the HTTP response for the GET /allocation_candidates placement API call is intended to be able to pass as-is (i.e. opaquely without the caller needing to know the structure of the HTTP payload) to the PUT /allocations/{consumer_uuid} call.'
22:52:38	cdent	but if you’ve used multple requests to construct the set of rps, we don’t have enough info to construction all the pieces of the allocation
22:53:09	jaypipes	cdent: correct.
22:53:25	cdent	feh
22:53:43	efried	Okay, I don't understand that, but I'm sure it's because I haven't read everything yet.
22:54:17	cdent	efried: if you haven’t got it after some cogitation, ask me again a bit later and I can try to explain it using different words
22:54:49	cdent	we’re making some shortcuts in our explanations that aren’t really helping matters
22:54:53	jaypipes	cdent: thus my earlier comment that "fuck it..." we will probably end up needing yet another REST API call to placement that takes as a payload some crazypants HOT template|TOSCA YAML|CloudFormation template thing that describes all the various components of the instance that the user wants Nova to atomically claim and spawn.
22:54:54	efried	cdent Thanks - I don't think it's words; it's background.
22:55:25	cdent	jaypipes: i believe that’s where I feel like borrowing dansmith’s gun
22:55:44	jaypipes	cdent: yup. and the reason I keep bringing up that I hate Nova being an orchestrator.
22:55:46	dansmith	cdent: what kind you want? semi-auto? large bore? hollow points?
22:56:04	cdent	dansmith: I’m away for home, so would feel bad for making a mess
22:56:08	cdent	from
22:56:14	efried	Every time this happens, though, I go away and study some more, and next time I come back and read the eavesdrop or whatever, I actually get it. I'm hoping to be good enough by the PTG not to get lost when this stuff is being discussed live.
22:56:15	dansmith	cdent: okay so hollow-points then
22:56:17	jaypipes	dansmith: shotty. it's got a "good spread".
22:56:41	jaypipes	efried: no worries, man. these conversations are important to have.
22:56:45	cfriesen__	.700 nitro express
22:56:55	openstackgerrit	Dan Smith proposed openstack/nova master: Split out the core of the ironic flavor migration https://review.openstack.org/501024
22:56:56	openstackgerrit	Dan Smith proposed openstack/nova master: Add nova-manage db command for ironic flavor migrations https://review.openstack.org/501025
22:57:01	dansmith	mriedem: dtantsur|afk ^
22:57:07	jaypipes	efried: as much as they just inevitably end up reinforcing my annoyance with orchestration.
22:57:13	dansmith	needs a reno but I'm out of brain power and the smoke is cutting my oxygen supply
22:57:49	efried	cdent jaypipes So what we're talking about here is that we made an architectural call to be able to take a chunk of the placement response and just blat it into a (single) allocation request; but if we've made multiple calls to placement we'll have multiple such chunks, and there's currently no semantic for "combining" them into a single allocation request.
22:58:00	efried	Did I get that right?
22:59:03	cdent	jaypipes: sadly, somewhere is going to have to have a model for that tosca thing for an instance doign nested rp stuff and it is going to need to land on a compute (so the instance can build correctly). we planned ourselves into this corner, is just the way the world is for now :(
22:59:40	cdent	efried: yes, pretty much. we’d need to build that “reassembler” in the scheduler and in a perfect world wouldn’t have to
23:00:10	efried	In practical terms, the "opaque" allocation request is just a list of things, and we would just append those lists together and be fine. We just didn't wanna have to do that.
23:00:12	cdent	we now need to be smart in at least two spots
23:00:53	cdent	efried: not exactly.

Earlier   Later