| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-22 | |||
| 18:05:24 | efried | Right, "be able to answer" doesn't help me understand what will actually *happen* | |
| 18:05:58 | efried | so, detect whether A+B+C is copacetic and... what? | |
| 18:06:17 | sean-k-mooney | so really kashyap is trying to turn the a possible runtime failure when you boot the first vm into hopefully a startup failure of the agent with a detailed error message | |
| 18:06:28 | efried | when you say "agent", you mean n-cpu? | |
| 18:06:43 | efried | ...by blowing up the virt driver init, I guess | |
| 18:06:43 | sean-k-mooney | ya at least that what i hope is the plan | |
| 18:06:50 | sean-k-mooney | yep | |
| 18:06:54 | efried | Cool. Definitely needs to be spelled out in the spec. | |
| 18:07:02 | efried | That is *not* an implementation detail :) | |
| 18:07:31 | sean-k-mooney | ya kasap has a second spec that he inherited that is related | |
| 18:07:54 | sean-k-mooney | i think the agent stoping might be in that one | |
| 18:08:16 | sean-k-mooney | https://review.opendev.org/#/c/642030/ | |
| 18:09:17 | sean-k-mooney | ya line 71 https://review.opendev.org/#/c/642030/6/specs/train/approved/cpu-model-selection.rst@71 | |
| 18:14:13 | openstackgerrit | sean mooney proposed openstack/nova master: [WIP] Libvirt: add nfv job https://review.opendev.org/652197 | |
| 18:15:08 | efried | stephenfin: mriedem: I don't see a bp for nova-console removal. Who was going to be spearheading that? | |
| 18:16:03 | sean-k-mooney | efried: fyi its a bankholiday here in ireland so stephenfin proably wont see that until tomorrow | |
| 18:16:34 | efried | sean-k-mooney: ack, thx | |
| 18:16:37 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add placement request filter for disabled computes https://review.opendev.org/654596 | |
| 18:16:39 | mriedem | efried: sean-k-mooney: dansmith: belmoreira: ^ here is a PoC for the disabled compute pre-filtering stuff we talked about in berlin | |
| 18:16:43 | mriedem | melwitt: ^ | |
| 18:16:44 | efried | mdbooth_: nudge, dud you see http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005366.html ? | |
| 18:16:59 | mriedem | efried: i don't think there is a blueprint or a driver | |
| 18:17:19 | efried | ack | |
| 18:17:20 | sean-k-mooney | oh cool | |
| 18:17:32 | mriedem | efried: i wouldn't probably move forward with nova-console removal without a definitive ack from the citrix team | |
| 18:17:37 | dansmith | isn't jaypipes going to lose his mind about that trait? | |
| 18:17:54 | mriedem | why? | |
| 18:18:04 | sean-k-mooney | metadata in placement | |
| 18:18:10 | mriedem | that's what traits are | |
| 18:18:43 | dansmith | because it's not a capability | |
| 18:19:14 | efried | mriedem: That's a pretty big "bug fix" | |
| 18:19:20 | sean-k-mooney | well if a compute service is disable its not capable of booting a node | |
| 18:19:23 | sean-k-mooney | *vm | |
| 18:20:03 | sean-k-mooney | the other way to do it if traits were not used would be a aggregate | |
| 18:20:24 | sean-k-mooney | just add the root rps of all disabled nodes to an aggreate and frobid it | |
| 18:20:26 | mriedem | efried: see the todo | |
| 18:20:30 | sean-k-mooney | but a trait works too | |
| 18:20:57 | mriedem | so, this is a poc hacked together for discussion at the ptg | |
| 18:21:05 | mriedem | i would write a spec | |
| 18:21:11 | mriedem | but wanted to post this first after working on it one morning | |
| 18:21:26 | efried | dig | |
| 18:21:53 | sean-k-mooney | an active service aggrate + member of is likely less of a sticking point in that an aggrate is just a bunch of resouce providers | |
| 18:32:51 | mriedem | efried: i thought i had something in the ptg etherpad for this bug but can't find it now - what would you prefer? nova general etherpad or the xp one with placement? | |
| 18:33:20 | efried | mriedem: "this bug" - disabled CPU prefiltering? | |
| 18:33:32 | mriedem | yes | |
| 18:33:34 | efried | Doesn't require any placement changes (I don't count os-traits really) so nova etherpad. | |
| 18:33:55 | efried | but please ask yourself: does it need PTG time? | |
| 18:33:56 | mriedem | alright | |
| 18:34:17 | efried | I haven't read your patch yet | |
| 18:34:19 | mriedem | given the questions i just got after posting this... | |
| 18:34:48 | efried | would | |
| 18:34:48 | efried | sure, questions that would be answered in a spec; I guess I'm asking whether discussion in the spec wolud suffice. | |
| 18:35:14 | sean-k-mooney | am it might help to do some of it in real time too | |
| 18:35:30 | mriedem | yeah idk, i guess i was thinking i'd rather crank it out in 15 minutes rather than waffle on a spec for 2 months | |
| 18:35:33 | sean-k-mooney | but i think the idea of adressing this in placment in some way is a good one | |
| 18:36:12 | efried | sean-k-mooney: Do you mean you think there's potential placement API changes needed? | |
| 18:36:13 | mriedem | sean-k-mooney: it has to be if cern (and huawei public cloud) want to grab more allocation candidates from placement during scheduling, cern is currently hacking around this | |
| 18:36:47 | sean-k-mooney | efried: no this should not need any api changes in placment | |
| 18:37:25 | sean-k-mooney | but we need to either use a trait or aggrage of some kind to track it in placement so placment can use its existing apis to filter out disabled hosts | |
| 18:37:37 | sean-k-mooney | mriedem: right if you use limit its going to be a huge pain | |
| 18:38:24 | mriedem | cern limits their allocation candidate results to like 20 and as a result were only getting back disabled computes in some cases | |
| 18:38:25 | sean-k-mooney | i left a quick comment https://review.opendev.org/#/c/654596/1/nova/scheduler/request_filter.py@104 by the way ill review it properly tomorow | |
| 18:38:42 | efried | Yeah, so the concept seems pretty simple to me, but if you think we need PTG time... | |
| 18:39:00 | mriedem | efried: i just assume that like all things placement related there will be a holy war | |
| 18:39:48 | efried | I really hope we're getting close to having an established process for these GET /a_c-based filters | |
| 18:40:53 | efried | to the point where soon we'll be able to have specless bps for them. "Add trait called X. Set trait X on compute RP when Y. Add filter for X a) in flavor, b) in image, c) when condition Z is present. Done." | |
| 18:41:56 | sean-k-mooney | that would make my "port num instance filter to placemtn" personal backlog item simpler :) | |
| 18:42:05 | efried | melwitt, lbragstad: I seem to recall this session https://www.openstack.org/summit/denver-2019/summit-schedule/events/23712/migrating-nova-apis-to-keystone-scope-types being merged with another - link please? | |
| 18:43:15 | efried | d'oh, never mind. | |
| 18:45:54 | jaypipes | mriedem: https://review.opendev.org/#/c/623558/ addresses the *reason* why CERN limits to 20. That's a more appropriate solution IMHO than making placement into another servicegroup API where we will need to sync active status between the nova cell DB tables and placement. | |
| 18:46:54 | jaypipes | mriedem: it's not a holy war. it's just fundamentally you'll get into a situation where there are two sources of truth for whether a provider is active or not. | |
| 18:47:05 | jaypipes | I'd prefer to avoid that if possilbe. | |
| 18:47:42 | mriedem | we already have 2 sources of truth for aggregates yeah? | |
| 18:47:58 | mriedem | i thought one of the goals of placement was to do as much filtering in sql as possible to avoid the slower post-filtering in python we have | |
| 18:48:03 | mriedem | so this would also align with that | |
| 18:48:32 | mriedem | i realize the downside of this is nova and placement are out of sync on the status of the thing - we could potentially heal that in our compute update_available_resource periodic | |
| 18:48:38 | efried | Maybe eventually placement can be the sole source of truth | |
| 18:48:39 | jaypipes | erm, not really if it doesn't usually filter anything at all... | |
| 18:49:13 | mriedem | i imagine in a large cloud the ComputeFilter gets used a lot | |
| 18:49:16 | mriedem | during maintenance and such | |
| 18:50:29 | jaypipes | mriedem: we could run the servicegroup API ahead of time and pass a parameter like ?uuids=!in:<DISABLED_SET> | |
| 18:50:41 | jaypipes | which actually *would* be an efficient filter. | |
| 18:50:51 | jaypipes | but I digress. | |
| 18:51:11 | sean-k-mooney | that could be a long list of uuids | |
| 18:51:22 | jaypipes | sean-k-mooney: rarely. | |
| 18:51:22 | sean-k-mooney | which is why an aggregate would make more sense | |
| 18:51:45 | jaypipes | sean-k-mooney: you're going to keep an aggregate membership sync'd with yet another periodic task? :( | |
| 18:52:22 | sean-k-mooney | no on servcei startup have the compute node remove itself form the aggrrate if its currently listed | |
| 18:52:50 | mriedem | sean-k-mooney: i don't think you'd want an aggregate for all non-disabled computes if you have 14K computes | |
| 18:52:51 | sean-k-mooney | then we can add it if we miss a heartbeat or someone use force down | |
| 18:53:29 | jaypipes | mriedem: I think he's saying the opposite. an agg for disabled providers. | |
| 18:53:30 | sean-k-mooney | mriedem: sure but you can do it the other way around too as you said in the review | |
| 18:53:50 | mriedem | sean-k-mooney: if we did aggregates for this yes that's what i'd do, an aggregate for only disabled computes | |
| 18:53:57 | jaypipes | oh look, a case for forbidden aggregates :) | |
| 18:54:05 | mriedem | and yeah that's what we'd use ^ | |
| 18:54:07 | sean-k-mooney | jaypipes: orgiginly in the code is said an aggrate for enabled because i fogot negitive member of was merged | |
| 18:54:23 | jaypipes | sean-k-mooney: it's the forbidden fruit. | |
| 18:55:00 | sean-k-mooney | from a db perspecitve this really would not be that expensice either way | |
| 18:55:27 | sean-k-mooney | the negitive case would be better but it will be similar | |
| 18:55:49 | jaypipes | so, I suppose, in order of preference, I would personally go with: a) finish up https://review.opendev.org/#/c/623558/ and the other cell-specific-service-listing thing mentioned in there, b) use an aggregate for disabled computes and member_of=! |
|