| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-08-07 | |||
| 13:52:26 | jaypipes | we're not covering a large portion of our functional tests for no good reason. | |
| 13:53:22 | gibi | bauzas: do you know where the ChanceScheduler is set in the functional test env? | |
| 13:53:22 | bauzas | jaypipes: if we run only one compute and want to boot one instance, why should we run filterscheduler and placement ? | |
| 13:53:45 | jaypipes | bauzas: because placement is now required. | |
| 13:53:45 | bauzas | gibi: just look at self.flags(driver='chance_scheduler', group='scheduler') | |
| 13:54:34 | jaypipes | bauzas: we will never be able to get rid of the code in the compute node that does "claiming" if the chance scheduler is allowed to keep on existing. same for caching scheduler, frankly. | |
| 13:54:56 | gibi | bauzas: thanks. I always forget about the fact that it is chance_scheduler in the conf not ChanceScheduler | |
| 13:54:59 | bauzas | jaypipes: that's 2 different concerns IMHO | |
| 13:55:15 | bauzas | jaypipes: the main concerning one is that not all our drivers use placement | |
| 13:55:22 | bauzas | jaypipes: and that, I fully agree, should be fixed | |
| 13:55:41 | dansmith | we're going to fix that by deprecating caching scheduler | |
| 13:55:50 | jaypipes | bauzas: no, I don't think the chance scheduler should be fixed at all. it should be removed. it does nothing at all. | |
| 13:55:50 | bauzas | jaypipes: the second concern you express is that a large portion of our tests are not using filter_scheduler, but rather a dummy driver | |
| 13:56:01 | bauzas | jaypipes: that second concern is not a big deal to me | |
| 13:56:03 | dansmith | I dunno about chance scheduler, but I'd kinda expect the same | |
| 13:56:29 | jaypipes | bauzas: why is that second concern not a big deal to you? we're missing a huge % of coverage because of this. | |
| 13:56:37 | bauzas | dansmith: jaypipes: well, I personnally think that having different drivers is good for nova | |
| 13:56:52 | jaypipes | bauzas: yeah, I don't. | |
| 13:56:54 | bauzas | dansmith: jaypipes: and I expressed my idea of having the same input data for all drivers | |
| 13:57:12 | bauzas | just the scheduler algorithm should be different | |
| 13:57:27 | dansmith | having a completely ridiculous scheduler (chance) is not useful just to have an alternate option, IMHO | |
| 13:57:57 | bauzas | dansmith: well, I know some operators that use it :) | |
| 13:58:03 | dansmith | and the reason to have the cachingscheduler should go away when we're claiming in the scheduler, so... no real reason to keep tht either | |
| 13:58:14 | dansmith | bauzas: and? :) | |
| 13:58:27 | jaypipes | bauzas: the caching scheduler doesn't implement a different "algorithm". it implements a different storage mechanism and a different behaviour in responding to the instance update events. | |
| 13:58:44 | bauzas | I mean, filter scheduler being O(n2), some people tend to prefer other in-tree | |
| 13:59:09 | bauzas | jaypipes: I agree | |
| 13:59:53 | bauzas | jaypipes: let me rephrase, I just think that all drivers should by default get the list of hosts to verify by calling placement | |
| 13:59:56 | jaypipes | bauzas: what do you think the caching scheduler's big O notation is? | |
| 14:00:03 | bauzas | the same | |
| 14:00:07 | jaypipes | right. | |
| 14:00:15 | bauzas | but chance isn't | |
| 14:00:29 | jaypipes | chance doesn't do anything at all. | |
| 14:00:35 | edleafe | Scheduler subteam running now in #openstack-meeting-alt | |
| 14:00:43 | mriedem | o/ | |
| 14:00:48 | jaypipes | bauzas: it doesn't even consider if a compute has room for the workload. | |
| 14:00:58 | bauzas | jaypipes: it does ONE thing, picking a host | |
| 14:01:04 | bauzas | jaypipes: the rest is done by the compute claim | |
| 14:01:16 | bauzas | that's a quick scheduler at least | |
| 14:01:25 | bauzas | if you are enough spaced | |
| 14:01:26 | dansmith | heh | |
| 14:01:38 | jaypipes | bauzas: err, you and I have different ideas of efficiency I guess. | |
| 14:01:46 | bauzas | yeah, I know, it's orthogonal to what we do | |
| 14:02:03 | bauzas | I'm just describing *why* people use it | |
| 14:02:03 | jaypipes | bauzas, dansmith: sched meeting in meeting-alt | |
| 14:03:07 | dansmith | bauzas: can you find any reference to support your claim that some people use chance successfully and recommend it in anything other than toy environments? | |
| 14:03:31 | bauzas | dansmith: unfortunately, only hallways talks | |
| 14:03:39 | dansmith | the only such mention on the first page of google is a RAX presentation from 2013 that says "filter scheduler is the only legit one" | |
| 14:04:22 | bauzas | anyway, I'm not particularly attached to chance_scheduler | |
| 14:04:29 | bauzas | it's even broken if you use ironic | |
| 14:04:33 | stephenfin | cdent: https://review.openstack.org/#/c/490952/ is pretty much done, yeah. All that needs to be checked if if I've missed anything from the import process | |
| 14:04:59 | dansmith | bauzas: oh okay, seems like you're arguing to keep maintaining it | |
| 14:05:01 | bauzas | I'm just trying to explain that I want to make sure we keep a clear separation in between what placement gives to drivers, and how driver work | |
| 14:05:28 | bauzas | dansmith: sorry, I'm unclear, I'm just giving food for thoughts about why we have it in-tree | |
| 14:05:47 | bauzas | we could outsource chance_scheduler | |
| 14:06:08 | bauzas | if we make clear the fact that it gets a list of RPs provided by Placement | |
| 14:06:20 | bauzas | it == the scheduler driver interface | |
| 14:06:28 | mriedem | the chance scheduler shouldn't even be asking placement for anything | |
| 14:06:37 | mriedem | it sets a flag for that (or doesn't set it rather) | |
| 14:06:40 | dansmith | I don't know what outsource means in this context | |
| 14:06:46 | bauzas | dansmith: out-of-tree | |
| 14:06:51 | dansmith | mriedem: well, the problem is it won't claim | |
| 14:07:02 | dansmith | mriedem: so if we have it in tree it needs to at least do that | |
| 14:07:14 | mriedem | dansmith: claim where? the scheduler? | |
| 14:07:20 | mriedem | neither does the caching scheduler | |
| 14:07:25 | dansmith | mriedem: right, same issue | |
| 14:07:37 | mriedem | why is that a problem? that was intentional | |
| 14:07:53 | dansmith | mriedem: because if we remove the claiming on the compute, we have no way of knowing if something will fit before sending it | |
| 14:08:06 | dansmith | which isn't so much a problem for chance since chance doesn't care about that at all, | |
| 14:08:06 | mriedem | we aren't removing claims in the compute in pike | |
| 14:08:33 | dansmith | mriedem: I think we're talking about the long-term strategy here | |
| 14:08:35 | bauzas | I really 'd love to have a whiteboard | |
| 14:08:47 | mriedem | dansmith: ok - not sure why it's a problem or fire drill for this week | |
| 14:08:49 | mriedem | maybe it's not | |
| 14:08:55 | mriedem | sounds like a distraction | |
| 14:08:55 | bauzas | I just see drivers as a black box from a placement perspective | |
| 14:08:58 | dansmith | mriedem: I don't think it is | |
| 14:09:17 | bauzas | #1 scheduler manager passes a list of candidates to the driver | |
| 14:09:31 | bauzas | #2 driver does its black magic to find the perfect candidate | |
| 14:09:46 | bauzas | #3 scheduler manager would claim the allocation | |
| 14:09:58 | bauzas | that is the loved interface I'd like | |
| 14:10:32 | bauzas | so, one day, we could just do better things than just looping over the list of instances which loops over the list of hosts | |
| 14:12:06 | dansmith | filter scheduler with no filters or weights configured is O(n) right? | |
| 14:12:28 | dansmith | and is effectively chance, but without the pathological sending of instances to full computes | |
| 14:12:40 | openstackgerrit | Merged openstack/nova-specs master: Amend spec for "Allow custom resource classes in flavor extra specs" https://review.openstack.org/481748 | |
| 14:13:39 | bauzas | dansmith: in theory, we also loop over each filter | |
| 14:13:59 | bauzas | dansmith: but since we have far less filters than hosts, I'm just keeping it O(n2) | |
| 14:14:13 | dansmith | bauzas: well (a) that's O(1) and (b) that's irrelevant if there are no filters :) | |
| 14:14:15 | bauzas | dansmith: the point is, what would be the interest of filter scheduler if we don't filter anything ? | |
| 14:14:33 | dansmith | bauzas: there is zero interest of the chance scheduler, so I'm not sure what your point is :) | |
| 14:14:40 | bauzas | dansmith: LOL | |
| 14:15:26 | dansmith | filter scheduler with no filters gives you at least selection by "would fit at all" in constant time (per instance) with proper claiming and random selection of a host within the possible set | |
| 14:15:58 | bauzas | dansmith: I see your point | |
| 14:16:13 | bauzas | if you want to deprecate chance, that would the solution, I agree | |
| 14:16:15 | dansmith | I'd be willing to bet a number of people assume chance does at least "would fit" and then once they read more and realize it doesn't, move on to something sane as they work through their POC | |
| 14:16:18 | bauzas | would be | |
| 14:20:02 | gibi | 20% of func tests are failing after change chance_scheduler to filter_scheduler | |
| 14:20:22 | gibi | let's see if it is something that easy to fix | |
| 14:21:02 | cdent | gibi++ | |
| 14:26:59 | mriedem | mikal: http://eavesdrop.openstack.org/meetings/nova/2017/nova.2017-07-27-14.00.log.html#l-67 | |