| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-22 | |||
| 15:57:55 | dansmith | jaypipes: wat. really? | |
| 15:58:03 | jaypipes | dansmith: yes. | |
| 15:58:17 | dansmith | jaypipes: we totally said we were doing them in the conductor in boston :) | |
| 15:58:32 | bauzas | jaypipes: I explained why the scheduler sould be a wrong possibilty in some email | |
| 15:58:34 | bauzas | lemme find it | |
| 15:58:53 | jaypipes | dansmith, bauzas: reschedule issues go away if we do the claims in the scheduler. | |
| 15:59:02 | dansmith | jaypipes: eh? | |
| 15:59:16 | bauzas | jaypipes: http://lists.openstack.org/pipermail/openstack-dev/2017-May/117039.html | |
| 15:59:20 | dansmith | jaypipes: if we do it in the scheduler, then we end up with scheduler claiming first, then conductor claiming on reschedules | |
| 15:59:24 | dansmith | jaypipes: which I don't like | |
| 15:59:25 | bauzas | "There are multiple reasons why we chose to use conductor for that :" | |
| 15:59:30 | jaypipes | dansmith: you remember when I told operators that "95% or more of the reasons for retries go away when the scheduler does claims"? | |
| 15:59:51 | jaypipes | dansmith: there is no reason to do reschedules at all if we do claims in the scheduler. | |
| 15:59:52 | dansmith | jaypipes: sure, but we're talking about claiming during the scheduling phase | |
| 16:00:10 | dansmith | I can't believe we're having this conversation again :) | |
| 16:00:20 | mriedem | the problem is no one is defining what 'reschedule' means here | |
| 16:00:26 | jaypipes | dansmith: I thought you were cool with getting rid of retries entirely? | |
| 16:00:33 | dansmith | lol | |
| 16:00:34 | mriedem | i think what jaypipes is talking about is calling select_destinations again | |
| 16:00:37 | bauzas | that's at least a bit unfortunate I spent my week on something I thought we had a consensus :) | |
| 16:00:50 | bauzas | without doing any reviews (sorry folks!) | |
| 16:01:00 | mriedem | i think dansmith is talking about deleting failed allocations and looping to the next host in our list, which is not the same | |
| 16:01:13 | mriedem | which we could do in conductor, | |
| 16:01:19 | mriedem | if the scheduler gives a list from select_destinations | |
| 16:01:30 | bauzas | mriedem: I implemented that in https://review.openstack.org/#/c/460455/15/nova/conductor/manager.py | |
| 16:01:31 | jaypipes | mriedem: no, I'm talking about the scheduler doing the "retry" of a claim/allocation. | |
| 16:01:33 | mriedem | "reschedule" is a broad ass term | |
| 16:01:51 | bauzas | jaypipes: please see at http://lists.openstack.org/pipermail/openstack-dev/2017-May/117039.html | |
| 16:01:58 | dansmith | how about a hangout at the bottom of the hour? | |
| 16:02:09 | bauzas | jaypipes: there are 4 reasons why I disagree with doing the allocation within the scheduler worker process | |
| 16:02:22 | openstackgerrit | Stephen Finucane proposed openstack/nova master: utils: Replace 'utf8' with oslo.utils equivalent https://review.openstack.org/466799 | |
| 16:02:22 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Remove dead code https://review.openstack.org/460662 | |
| 16:02:29 | mriedem | dansmith: yeah need a hangout, so 1PM ET? | |
| 16:02:57 | jaypipes | dansmith, bauzas: I strongly think we should be calling placement's POST /allocations from the scheduler, not the conductor. Sorry if this surprises people. | |
| 16:03:03 | dansmith | mriedem: mriedem sure | |
| 16:03:26 | dansmith | jaypipes: yes that's very surprising.. we all had this conversation in boston and I was sure we had consensus | |
| 16:03:33 | jaypipes | I specifically remember saying "doing claims in the scheduler gets rid of the retry issues (and all associated "upcall shenanigans") | |
| 16:03:50 | bauzas | jaypipes: well, I do remember exatly operators nodding at dansmith's proposal of passing down a list of possible acceptable hosts | |
| 16:03:53 | dansmith | jaypipes: so, we also agreed we can't fully get rid of retries right away, which means conductor _will_ have to do allocation calls as well | |
| 16:03:56 | dansmith | right | |
| 16:03:57 | mriedem | note we've also been saying "claims in the scheduler*" | |
| 16:04:16 | dansmith | jaypipes: I'm opposed to doing it in scheduler first and conductor second.. I think that's a mess | |
| 16:04:23 | melwitt | mriedem: thanks. short answer is, you're right. replying on the review now | |
| 16:04:35 | jaypipes | dansmith: I see no reason to "do it in the conductor second". | |
| 16:04:45 | mriedem | melwitt: ok - so basically wtf for how this behaves, but it's not backward incompatible so ok | |
| 16:04:52 | jaypipes | dansmith: if the scheduler does the claim, there is no reason for anything else to do the claim. | |
| 16:05:00 | dansmith | jaypipes: reschedule | |
| 16:05:06 | mriedem | jaypipes: retries from the computes go up to the cell conductor | |
| 16:05:10 | jaypipes | dansmith: we should *not* support reschedule. period. | |
| 16:05:16 | dansmith | jaypipes: there was almost zero support in the room for eliminating reschedules | |
| 16:05:30 | bauzas | I thought we had that discussion in Boston :/ | |
| 16:05:33 | dansmith | jaypipes: which was the whole point of the "send extra hosts down to the cell" discussion, which you were definitely there for | |
| 16:05:36 | dansmith | bauzas: we did | |
| 16:05:36 | jaypipes | dansmith: yes there was when I said "95% of the causes of reschedules currently will be eliminated" | |
| 16:05:44 | dansmith | *facepalm* | |
| 16:06:03 | melwitt | mriedem: yeah ... I'm trying to make all of the formats of count as dict consistent. before, I had some dict and some int so that only things that could have project and user would be dict, but people found that more confusing | |
| 16:06:05 | bauzas | anyway, I'm litterally 24 hours away of taking a long break | |
| 16:06:42 | jaypipes | dansmith: perhaps this is the session I had to leave early (I was standing in the back and right before I left I said "recschedules should not occur and the reason for them will be gone when claining in the scheduler"? | |
| 16:06:45 | bauzas | I'm trying hard to make sure we're all on the same page before I leave | |
| 16:06:58 | dansmith | jaypipes: bauzas: mriedem: I'm good for a hangout any time starting now, whenever you want, until 11:30 PDT | |
| 16:07:08 | bauzas | I can | |
| 16:07:09 | jaypipes | I can do a hangout whenever. | |
| 16:07:13 | mriedem | i can also do now | |
| 16:07:17 | jaypipes | ++ | |
| 16:07:23 | dansmith | jaypipes: yeah, you left, but I thought you were still around for the consensus on reschedules.. however you _were_ there for the scheduling claims session where we talked about the extra hosts thing | |
| 16:07:30 | melwitt | eliminating reschedules would make things really suck for Ironic users in the absence of a porcelain API that does retries | |
| 16:07:47 | jaypipes | melwitt: you mean "standalone Ironic". | |
| 16:07:53 | melwitt | no, Ironic + Nova | |
| 16:08:15 | mriedem | plus i think numa came up | |
| 16:08:20 | dansmith | I'll get a link, just a sec | |
| 16:08:22 | mriedem | in the claims in the scheduler session | |
| 16:08:40 | jaypipes | mriedem: NUMA is handled in the existing NUMAtopologyFilter. | |
| 16:08:57 | dansmith | https://hangouts.google.com/call/e6jtzndusjf6rezqvwdatascvuu | |
| 16:09:12 | bauzas | joining | |
| 16:09:16 | jaypipes | mriedem: claims would occur after the NUMATopologyFilter selected a host. We could have the NUMAtopology that was selected (and currently thrown away by the filter) written to the compute_nodes table. | |
| 16:10:26 | bauzas | dansmith: you're not there in the hangout | |
| 16:11:15 | jaypipes | melwitt: honestly, I don't really care about use cases where an IPMI boot failure results in an ERROR state for an instance | |
| 16:12:30 | sajeesh | ping sdague | |
| 16:14:37 | sdague | sajeesh: what's up? | |
| 16:14:58 | sajeesh | ruleset looks like, so it can be turned into a worked example | |
| 16:14:58 | sajeesh | Hi sdague ..you have written in the mail Get someone from the CERN team to write down what their ideal | |
| 16:15:12 | sdague | sajeesh: yes | |
| 16:15:20 | sajeesh | by ruleset do you mean all use cases ? | |
| 16:15:51 | sdague | not use cases, how they think they would want hierarchical quotas to work | |
| 16:16:06 | melwitt | jaypipes: :\ I don't know the particulars on what is different about scheduling onto baremetal directly, but when you have enough nodes, enough of them could be flaky and if we fail and go to error, we're not absorbing any of that for the end user | |
| 16:16:06 | sajeesh | ok | |
| 16:17:03 | sdague | sajeesh: because the use cases here don't build the hierarchical model all that well, most people set a bunch of use cases together that actually are conflicting | |
| 16:17:33 | sdague | so, I want the model algorithm to be proposed by groups that are interested, then we'll work it through | |
| 16:18:00 | sajeesh | sdague:+1 | |
| 16:21:44 | ayoung | Since we are rewriting how you guys do policy enforcement, maybe y'all should take a look: https://review.openstack.org/#/c/384148/ | |
| 16:22:50 | ayoung | alaski, it has changed a bit since you last looked at it...in October | |
| 16:25:23 | cdent | jaypipes are you guys still on the hangout? | |
| 16:25:30 | jaypipes | cdent: yes | |
| 16:25:37 | sdague | ayoung: so, my concern with is_admin on project is that now if you give a user authority to add other users to a project, they also get to see the physical topology of your cloud, and perform live migrations. Because while is_admin is no longer fully global, it isn't service scoped, right? | |
| 16:26:41 | alaski | ayoung: I'm no longer actively involved in openstack and likely won't get a chance to look too closely at that change. | |
| 16:26:54 | ayoung | alaski, understood | |
| 16:27:06 | ayoung | sdague, let me parse what you are asking. | |
| 16:28:19 | ayoung | sdague, "my concern with" == "the reason I don't like this approach." " if you give a user authority to add other users to a project" should read " if you give a user authority to add other users to the admin project" | |
| 16:28:45 | ayoung | sdague, is that what you mean? | |
| 16:28:52 | sdague | ayoung: well, I'm trying to understand based on what's there | |