| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-06-27 | |||
| 13:36:06 | artom | Doh | |
| 13:36:26 | alex_xu | artom: https://review.openstack.org/#/c/376021/72/nova/tests/unit/api/openstack/compute/test_attach_interfaces.py@516 this should use self.req | |
| 13:36:30 | artom | alex_xu, can you leave a comment and I'll fix | |
| 13:36:37 | alex_xu | artom: ok | |
| 13:37:20 | artom | alex_xu, thanks :) | |
| 13:37:47 | edleafe | jaypipes: I read ahead to the other PSs in the series to understand. It appears that select_destinations will return num_instances hosts to the manager - without claiming first. Right? | |
| 13:37:54 | edleafe | jaypipes: That means that there aren't any backup choices to select. The big gain we were hoping to get is when there is a race and the claim fails, the scheduler can simply select the next host in the weighed list. Having the manager make the claim means we lose that. | |
| 13:38:10 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Use ``assert_has_calls`` to check function calls https://review.openstack.org/477469 | |
| 13:39:29 | alex_xu | artom: done | |
| 13:45:32 | mriedem | artom: couple easy things to do in https://review.openstack.org/#/c/435082/ | |
| 13:45:34 | mriedem | and then i'm +2 | |
| 13:45:44 | artom | mriedem, sweet, thanks :) | |
| 13:46:14 | artom | alex_xu, I don't think we can remove those mocks though | |
| 13:46:47 | artom | Wait | |
| 13:46:59 | artom | Dammit, that's twice now I say a thing and change my mind immediately after | |
| 13:47:03 | artom | Sorry :( | |
| 13:47:51 | alex_xu | artom: no worries :) | |
| 13:59:13 | openstackgerrit | Chason Chan proposed openstack/nova master: switch to openstackdocstheme https://review.openstack.org/477751 | |
| 14:01:38 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add abstract OVSDB API https://review.openstack.org/476612 | |
| 14:04:56 | artom | mriedem, in the interest of reducing latency, is https://review.openstack.org/#/c/435082/50/nova/tests/unit/compute/test_compute_mgr.py@2537 satisfactory to you, or do you want to remove the tag and the call assertion entirely? | |
| 14:15:25 | mriedem | artom: replied | |
| 14:15:32 | mriedem | but now i'm shutting it down | |
| 14:16:15 | artom | mriedem, posthumous thanks :) | |
| 14:20:53 | jaypipes | edleafe: I thought we needed to return the alternate selections to the conductor anyway? | |
| 14:26:29 | edleafe | jaypipes: we need to pick a good host, so that we know what cell the alternates must be in | |
| 14:27:20 | jaypipes | edleafe: right, and those alternatives can be used to perform claim retries in the manager, no? | |
| 14:28:43 | edleafe | jaypipes: sure, but now you've kind of defeated the whole purpose of alternatives. If the setting is for 3 retires, and you burn 2 of them before you even get to the build... | |
| 14:30:05 | jaypipes | edleafe: it sounds like you're pretty gung-ho to just have a giant _schedule() method just do everything. | |
| 14:30:21 | jaypipes | I'm trying to add some separation of concerns into the codebase. | |
| 14:31:03 | edleafe | jaypipes: No, I'm "gung-ho" about having the _schedule() method return a good host | |
| 14:31:41 | jaypipes | edleafe: my patch series doesn't change _schedule()./ | |
| 14:31:54 | edleafe | jaypipes: your separation is functionally the same as claims in the conductor. | |
| 14:32:05 | jaypipes | edleafe: what? | |
| 14:32:31 | edleafe | jaypipes: you're making a claim after the HostState objects are gone | |
| 14:33:11 | jaypipes | edleafe: what do you mean by "after the HostState objects are gone"? | |
| 14:33:48 | edleafe | jaypipes: in _schedule() we have a list of filtered, weighed HostState objects. | |
| 14:34:05 | edleafe | When you return they are no longer accessible | |
| 14:34:27 | jaypipes | edleafe: what do we need the host state objects for after the scheduler driver has filtered and weighed them? | |
| 14:34:38 | edleafe | So whether you're in the scheduler manager or the conductor manager, you're limited to the objects returned from _schedule() | |
| 14:34:39 | jaypipes | edleafe: we just need the resource provider UUIDs. | |
| 14:34:58 | jaypipes | edleafe: exactly! have the scheduler driver do one thing and do it well. | |
| 14:34:59 | edleafe | jaypipes: in a race-free world, sure | |
| 14:35:26 | edleafe | jaypipes: we had this discussion weeks ago | |
| 14:35:29 | jaypipes | edleafe: alternately, let's change the select_destinations() *driver* method to return the HostState list instead of a list of dicts. | |
| 14:35:40 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Delete disk metadata when detaching volume https://review.openstack.org/435082 | |
| 14:35:41 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: API support for tagged device attachment https://review.openstack.org/376021 | |
| 14:35:56 | edleafe | dansmith was arguing for the conductor doing claiming, and you and I were arguing for the scheduler doing the claiming, using the exact thing I'm talking about now | |
| 14:36:25 | jaypipes | edleafe: I'm not doing claiming in the conductor. I'm simply doing it in the scheduler manager not the scheduler driver. | |
| 14:36:35 | edleafe | jaypipes: yes, I know that | |
| 14:36:52 | edleafe | jaypipes: which is why I wrote "functionally" equivalent | |
| 14:36:58 | jaypipes | but it isn't. | |
| 14:37:10 | edleafe | jaypipes: because you no longer have access to the weighed host list | |
| 14:37:27 | jaypipes | edleafe: so have the select_destinations() return a weighed host list, then. | |
| 14:37:35 | edleafe | jaypipes: and if you want to return the full list, well, there goes your separation goal | |
| 14:37:36 | dansmith | edleafe: claiming in scheduler manager is functionally equivalent to claiming in conductor? | |
| 14:37:56 | edleafe | dansmith: as far as having access to the weighed list of hosts, yes | |
| 14:38:09 | dansmith | o.O | |
| 14:38:11 | jaypipes | edleafe: if we return the weighted list to the manager, how does that mean "there goes my separation goal"? | |
| 14:38:58 | edleafe | jaypipes: because HostState is a driver concept | |
| 14:39:07 | edleafe | jaypipes: they don't exist in the manager | |
| 14:39:24 | jaypipes | I'm not saying return the list of host state objects. I'm saying return a weighted list of hosts. | |
| 14:39:45 | jaypipes | in other words, what my patch is doing -- minus the returning alternatives part. | |
| 14:40:24 | dansmith | even still, HostState is not a driver concept, IMHO | |
| 14:40:28 | dansmith | it's defined in host_manager even | |
| 14:40:32 | edleafe | jaypipes: http://p.anticdent.org/3900 | |
| 14:40:46 | dansmith | I see no reason to keep that hidden from scheduler manager if it makes things more convenient | |
| 14:40:51 | jaypipes | or even if we did return a list of host state objects, I actually wouldn't care, as long as the manager isn't modifying those objects. | |
| 14:41:09 | jaypipes | or even if we did return a list of host state objects, I actually wouldn't care, as long as the manager isn't modifying those objects. | |
| 14:41:20 | jaypipes | edleafe: ^ | |
| 14:43:36 | edleafe | jaypipes: one of your stated concerns was the chance scheduler. Chance scheduler doesn't use HostState. It's specific to filter scheduler | |
| 14:44:05 | edleafe | jaypipes: so I'm not sure how this is separating things more cleanly | |
| 14:44:08 | jaypipes | edleafe: right, which is why I changed the returned value from select_destinations() to be a list of dicts that included the UUID. | |
| 14:45:05 | edleafe | 09:40 < jaypipes>| or even if we did return a list of host state objects, I actually wouldn't care, as long as the manager isn't modifying those objects. | |
| 14:45:10 | jaypipes | edleafe: because the manager will loop through the allocation requests and attempt to claim resources on the destinations returned by select_destinations(). If select_destinations() doesn't return something with that UUID, the manager doesn't have the ability to match an allocation request to a host. | |
| 14:46:19 | edleafe | jaypipes: well, if keeping claiming out of the filter scheduler is worth it, then go for it | |
| 14:46:21 | jaypipes | edleafe: the alternative is to have the _schedule() method of the filter scheduler driver do everything under the sun: filter, weigh, get allocation requests, and claim. | |
| 14:47:10 | edleafe | jaypipes: and IYO it's *so* much cleaner to have the _schedule method do filter, weigh, and get allocation requests | |
| 14:47:34 | jaypipes | edleafe: no, I have the _schedule() method only do filter and weigh. | |
| 14:47:44 | jaypipes | edleafe: the manager does the claiming. | |
| 14:47:47 | edleafe | jaypipes: In my mind, the claim is a final filter. _schedule() isn't doing its job if it returns an invalid host | |
| 14:47:52 | jaypipes | after receiving the set of destinations from the driver. | |
| 14:48:20 | jaypipes | edleafe: what do you mean by "returns an invalid host"? I lost you on that one. | |
| 14:48:29 | jaypipes | and I don't see the claim as a final filter at all. | |
| 14:48:34 | jaypipes | it's a completely different operation. | |
| 14:48:37 | dansmith | definitely not | |
| 14:48:42 | dansmith | if you mean a filterscheduler filter | |
| 14:48:46 | edleafe | jaypipes: in the case of horizontal scaling of the scheduler, when there could be races for the same host | |
| 14:49:11 | edleafe | jaypipes: a filter is logic that determines if a host is acceptable. | |
| 14:49:12 | dansmith | how does claiming in a filter make that better? | |
| 14:49:23 | edleafe | dansmith: not "in a filter". | |
| 14:49:28 | jaypipes | edleafe: there definitely *will* be races for the same host. And the manager will be the thing responsible for retrying those claim attempts on the same or an alternate matching host. | |
| 14:49:37 | dansmith | jaypipes: right | |
| 14:49:39 | edleafe | dansmith: it's the final step in the process | |
| 14:49:47 | dansmith | edleafe: in the manager. | |
| 14:50:00 | edleafe | ok, fine, fine | |
| 14:50:26 | edleafe | it just looks to me like you're trading one entanglement for another.. | |
| 14:50:32 | edleafe | But don't let that slow you down | |
| 14:50:44 | jaypipes | .. | |
| 14:50:51 | dansmith | I don't even know how to respond to that | |
| 14:50:58 | jaypipes | I'm not going to. | |