| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-15 | |||
| 15:07:28 | stephenfin | jaypipes: I'll let you call that | |
| 15:07:57 | jaypipes | mriedem: it's just a small refactor. if you want to keep it in, that's OK with me. but the series is associated with a blueprint that is not approved, as you know. | |
| 15:08:18 | jaypipes | efried: sure. hangout or IRC? | |
| 15:08:33 | mriedem | i haven't been tracking it, just saw you mention that | |
| 15:08:36 | efried | cdent, jaypipes, edleafe: If y'all are okay with limiting based on the MISC_SHARES_VIA_AGGREGATE trait, at least for now, I'll make it so. If I'm the only one who's uncomfortable about that, and I can't give a good reason/counterexample, then I'm okay to let it ride. | |
| 15:08:50 | jaypipes | mriedem: for some reason I thought -W would prevent it from gating... | |
| 15:08:56 | dansmith | jaypipes: nay | |
| 15:08:57 | efried | jaypipes, mriedem: A -2 might | |
| 15:09:02 | jaypipes | ah | |
| 15:09:09 | jaypipes | well I don't want to do that... | |
| 15:09:09 | mriedem | no, the best thing with stuff that's not bp approved right now, is don't +2 it | |
| 15:09:11 | dansmith | a -2 will, but it will reset the gate at the last moment | |
| 15:09:22 | bauzas | dansmith: oh, saw https://review.openstack.org/#/c/544585/ | |
| 15:09:28 | mriedem | +1 if you wanted to review it and said lgtm but the bp isn't approved yet | |
| 15:10:26 | dansmith | bauzas: came out of discussing some concerns with CERN | |
| 15:10:41 | bauzas | okay, I need to review it carefully then | |
| 15:10:57 | bauzas | I wasn't really paying attention to specs yet | |
| 15:11:39 | jaypipes | stephenfin: I -W'd the following patch. | |
| 15:11:45 | dansmith | bauzas: it's pretty simple, and you can see my prototype code which is probably easier to grok | |
| 15:11:47 | jaypipes | stephenfin: and left the one small refactoring in the gate. | |
| 15:11:56 | bauzas | dansmith: yeah, will look | |
| 15:12:04 | stephenfin | jaypipes: Cool. I had concerns about that one anyway (though it's also a refactoring change) | |
| 15:12:36 | jaypipes | efried: let me comment on the spec. | |
| 15:12:56 | efried | jaypipes: ack, thx. I'd like to answer the question in the text before it's published. | |
| 15:13:14 | jaypipes | ya | |
| 15:13:18 | efried | edleafe: btw, we're talking about https://review.openstack.org/#/c/540111/4/specs/rocky/approved/update-provider-tree.rst@48 | |
| 15:16:40 | dansmith | mriedem: you're going to let me know when I can un -W this right? https://review.openstack.org/#/c/543580/ | |
| 15:18:17 | mriedem | preferably after rc2 | |
| 15:18:55 | jaypipes | efried: done | |
| 15:29:02 | edleafe | efried: was getting coffee. Makes sense to limit the tree by that trait. | |
| 15:29:10 | mriedem | melwitt: i thought about this earlier today for some reason: if anyone, new contributor, stephen, whoever :) wants to start converting mox to mock in nova, they should definitely either way for cellsv1 and nova-net removal first, or be sure to stear clear from converting any tests that touch those code paths | |
| 15:29:24 | mriedem | *wait | |
| 15:29:44 | efried | edleafe: Rgr, thx | |
| 15:34:43 | bauzas | dansmith: question for you, why can't we leave Placement return all the hosts and only select the ones from that or this in a filter ? | |
| 15:35:07 | bauzas | dansmith: you described that in https://review.openstack.org/#/c/544585/6/specs/rocky/approved/placement-req-filter.rst@115 IIRC but it's a bit confusing for me | |
| 15:35:39 | dansmith | bauzas: because cern would have to process 9000 hosts that couldn't possibly work before they get to the first one that would | |
| 15:36:14 | dansmith | bauzas: if you have a 10k node deployment with lots of space, and you boot a small instance, you get back a ridiculous number of hosts that you're not allowed to use and we have to run filters on all of them | |
| 15:36:23 | bauzas | dansmith: it ties to a Placement performance question, right? | |
| 15:36:28 | dansmith | no | |
| 15:36:36 | dansmith | placement is fast at doing that, | |
| 15:36:47 | dansmith | scheduler is slow at processing the result | |
| 15:36:48 | bauzas | dansmith: because for that kind of purpose, we always say to use the more important filters first | |
| 15:36:53 | mriedem | melwitt: i've started a technical debt / cleanup section in https://etherpad.openstack.org/p/nova-ptg-rocky L157 | |
| 15:37:09 | dansmith | bauzas: at CERN-level scale, that is still very wasteful | |
| 15:37:10 | bauzas | dansmith: since each filter is processed one after the other, depending on the ordering of the list | |
| 15:37:36 | bauzas | dansmith: did they found some bottleneck ? | |
| 15:37:49 | dansmith | bauzas: did you read the irc conversation? | |
| 15:37:55 | bauzas | because the last benches I had from the scheduler, the filtering process itself was way fast | |
| 15:38:18 | bauzas | dansmith: from the cells meeting ? unfortunately no | |
| 15:38:25 | dansmith | bauzas: we also only request 1000 hosts from placement, so if your hosts are after the first thousand, we wouldn't get back any that work for us | |
| 15:38:31 | dansmith | bauzas: no, linked in the spec | |
| 15:38:47 | mriedem | bauzas: cern ties tenants to cells, | |
| 15:38:56 | mriedem | so top level cells v1 scheduler will pick the cell for the tenant, | |
| 15:39:01 | mriedem | but with cells v2, we have a flat scheduler, | |
| 15:39:12 | dansmith | bauzas: so if you have more than 1000 hosts that could fit your size flavor, but the first one you're allowed to use is after that, you could never schedule, or have to bump that limit higher | |
| 15:39:20 | mriedem | so rather than a request from that tenant being just the computes in the chosen child cell, now it's all hosts from all cells, | |
| 15:39:21 | mriedem | as dan pointed out | |
| 15:39:48 | dansmith | and, any hard requirement we (nova) knows about that it can communicate to placement ahead of time will only improve performance, and ensure we get back a richer set of hosts to choose from | |
| 15:40:03 | dansmith | and "can never be on hosts in aggregate X" is a pretty concrete requirement :) | |
| 15:40:08 | bauzas | dansmith: mriedem: I understand that concern | |
| 15:40:20 | bauzas | I just wonder how big it can become | |
| 15:40:25 | mriedem | i like to also think of this as a way to eventually get rid of these other post-placement filters | |
| 15:40:34 | dansmith | yeah | |
| 15:40:37 | bauzas | the fact we said placement was efficient is that because it doesn't care of anything specific | |
| 15:40:37 | dansmith | that is also nice | |
| 15:40:38 | mriedem | we have already said don't use ram/disk/core filter b/c placement | |
| 15:40:49 | mriedem | this would also remove the aggregate tenant isolation filter | |
| 15:41:06 | dansmith | bauzas: placement already does aggregates, this is well within its scope | |
| 15:41:16 | mriedem | bauzas: i'm not aware of anyone that has done benchmarking before and after placement at scale | |
| 15:41:19 | mriedem | to say that we're way better now | |
| 15:41:39 | dansmith | bauzas: performance aside, you get the point about the request limit right? | |
| 15:41:43 | bauzas | mriedem: no, the benches were pre-placement, but saw the filtering process very fast | |
| 15:41:49 | bauzas | dansmith: sure | |
| 15:41:51 | dansmith | if we'reonly getting back 1000 hosts, then we could get back none | |
| 15:42:26 | openstackgerrit | Claudiu Belu proposed openstack/nova master: compute: Cleans up allocations after failed resize https://review.openstack.org/543971 | |
| 15:42:36 | bauzas | well, I don't know what to say | |
| 15:42:42 | dansmith | bauzas: performance aside, we want to get back a list of hosts that are candidates, and hosts that are in aggregates we're not allowed to be on are not candidates, so they're just busy work and they defeat the point of getting a solid fast set of canidates from placement based on things it knows about | |
| 15:43:47 | bauzas | dansmith: sure | |
| 15:43:52 | bauzas | anyway, here is my take | |
| 15:43:58 | bauzas | I'm not opposed at that | |
| 15:44:05 | bauzas | I just need to consider all the implications | |
| 15:44:20 | bauzas | because filters are like the nova things the most tweaked | |
| 15:44:25 | bauzas | by ops | |
| 15:44:41 | mriedem | dansmith: i'm assuming the pre-placement filters would also be pluggable? | |
| 15:45:01 | dansmith | mriedem: modular, like our scheduler filters, not necessarily pluggable | |
| 15:45:06 | bauzas | if we're about to introduce a new hook for cheating placement to only return the subset we want, chances are that in the future ops will come with solid use-cases | |
| 15:45:24 | dansmith | bauzas: cheating placement? I can't understand how this is that :) | |
| 15:45:30 | bauzas | and at the end of the day, we'd just introduce more and more complexity to placement | |
| 15:45:39 | dansmith | this doesn't change placement at all | |
| 15:45:41 | dansmith | not a single bit | |
| 15:45:48 | mriedem | these pre-placement filters are restricted to what placement api supports | |
| 15:45:55 | bauzas | dansmith: for your proposal, I agree | |
| 15:45:56 | mriedem | which is resource classes, traits and aggregates | |
| 15:46:00 | dansmith | they operate on request spec | |
| 15:46:20 | bauzas | dansmith: again, I'm not opposed to *your* use case which is tenant filtering | |
| 15:46:41 | mriedem | i also have a use case for this | |
| 15:46:46 | mriedem | if you care | |
| 15:46:49 | bauzas | that said, we could do that without using aggregateds | |
| 15:46:54 | bauzas | I guess | |
| 15:46:56 | bauzas | anyway | |