Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-02-03
21:48:28 mriedem i thought the cells scheduler was at the top level cell in cells v1?
21:48:35 superdan mriedem: yeah, in cellsv1 the scheduler is at the lower levels
21:48:43 superdan mriedem: no there's a special scheduler up there, regular scheduler down below
21:48:50 mriedem right
21:49:03 superdan melwitt: won't be able to have a cell-by-cell conversion from v1 to v2
21:49:18 superdan melwitt: but it'll be more of a "turning stuff off" kind of procedure I think, if we get the mappings in place ahead of time
21:49:50 superdan these are not friday afternoon questions
21:50:01 superdan ask me shit like "is it beer o clock yet" or something
21:50:12 melwitt superdan: heh, sorry. I blame dtp for asking me to think about stuff by asking about cells v1 step-by-step
21:50:18 mriedem scheduler is global in cells v2 world right?
21:50:22 superdan mriedem: yeah
21:50:33 mriedem so computes in multiple cells v2 cells are reporting into placement,
21:50:41 mriedem scheduler asks placement for an RP, gets one, sends that to conductor,
21:50:45 mriedem which is mapped to a cell via hostmappings,
21:50:53 mriedem and then conductor creates the instance in that cell and builds it there
21:51:10 superdan with pure v2 yes, but with v1,
21:51:15 superdan that all happens inside a cellv1 cell
21:51:31 melwitt superdan: I was thinking it doesn't matter if they set up all their cells now (technically) and that they could do them later. but I think that would make it more confusing
21:51:41 mriedem what's preventing the low-level filter scheduler in cells v1 not picking an RP from another v1 cell?
21:52:05 superdan mriedem: because they don't have visibility of each other? they'll all need their own placement, just like they need their own api
21:52:24 mriedem ok i thought placement was still global in cells v1
21:52:24 superdan mriedem: otherwise we have to teach placement about cellsv1, which would be ungood I think
21:52:28 mriedem right
21:52:36 mriedem we don't want cells v1 specific garbage in placement
21:53:00 mriedem maybe we want a note of that somewhere so when i ask it 5 more times in 12 months, you can point me
21:53:03 superdan placement is kinda part of the regular scheduler, which is replicated per v1 cell, which means placement is too
21:53:40 superdan melwitt: are you doing a general "what to make of the v1 mess" section? you could put that ^ in there if so
21:54:57 melwitt superdan: I'm doing a step-by-step for cells v1 peeps. what's this note about, just telling why not to worry about scheduling working right?
21:55:09 mriedem melwitt: that placement is per cells v1 cell
21:55:12 mriedem not global in cells v1
21:55:13 superdan that ^
21:55:29 melwitt oh ... hm
21:55:38 mriedem in other words, placement only has knowledge of the computes in the cells v1 cell that it's running in
21:55:50 superdan probably needs to go into some placement doc
21:55:56 mriedem yeah, which we have
21:56:10 melwitt yeah, I dunno anything about how to deploy placement for cells v1
21:56:17 mriedem just like nova-api
21:56:25 mriedem except in the child cell
21:56:31 superdan mriedem: I'll add it to placement.rst
21:56:43 mriedem ok http://docs.openstack.org/developer/nova/placement.html
21:56:47 melwitt so more like nova-scheduler then?
21:56:48 mriedem not sure what the best place is
21:56:54 mriedem melwitt: yeah
21:57:02 superdan melwitt: right, goes with scheduler
21:57:03 mriedem don't we have nova-scheduler in the top cell too?
21:57:03 melwitt k, that makes sense
21:57:07 mriedem or is that the cells service?
21:57:12 mriedem with the cell scheduler
21:57:27 melwitt it's the cells scheduler, I dunno if it's the same code as nova-scheduler or not
21:57:28 mriedem cells-region or whatever
21:57:29 superdan hmm, you know
21:57:54 superdan because placement is all service-catalog-y, I wonder if people will be able to do that
21:59:29 openstackgerrit Dan Smith proposed openstack/nova master: Clarify the deployment of placement for cellsv1 users https://review.openstack.org/429056
21:59:55 superdan mriedem: no it's different I think
22:00:15 superdan nova/cells/scheduler.py
22:01:40 melwitt ah, cool
22:15:03 superdan mriedem: your rc1 todo etherpad says we have until monday to do rc1 but the announce already went out right?
22:15:19 mriedem yeah, i'll fix
22:15:25 superdan okay
22:23:38 owalsh superdan, mriedem: hola... TL;DR what is the suspected problem with https://review.openstack.org/#/c/427534/?
22:25:19 superdan owalsh: none that I know of
22:25:20 mriedem owalsh: not totally sure, i dug into the compute logs a bit to see instances not building on ironic nodes because the disk on the nodes was fully consumed by another instance
22:25:45 mriedem which i think those things just get rescheduled,
22:26:00 mriedem but i didn't trace through any failures in real detail
22:26:14 superdan but really, there's no reason for that patch to affect that kind of thing
22:26:18 openstackgerrit John Perkins proposed openstack/nova master: Integrate oslo.config validator https://review.openstack.org/428974
22:26:46 superdan and before it, no nodes were even registered
22:26:47 owalsh ok, can't see how they could be related... that change just double checks nova has the ironic nodes in the RT before running discover_hosts
22:27:03 superdan yep
22:27:31 owalsh peeling an onion me thinks... my fix just got us to the next layer
22:27:35 superdan yep
22:27:44 superdan I haven't even heard that it happened more than once
22:27:50 superdan mriedem: did you see a bunch of these failures?
22:30:23 mriedem i don't think i actually saw any error messages, just that resources couldn't be claimed on a given node because the disk was fully consumed on that node at the time of the instance build
22:30:41 mriedem like i said, i didn't trace a job failure all the way through,
22:30:53 mriedem was just pointed at a debug log message and was asked what it meant, and i poked around that abit
22:30:56 owalsh ok https://review.openstack.org/428974 couldn't be related I think...
22:31:02 melwitt superdan: sanity check me here but I don't think cells v1 peeps can use the discover_hosts periodic task because their nova-schedulers run inside cells, right?
22:31:08 ttx mriedem: I don't have final hours yet but yes 9am start sounds like a safe bet. Up until 6pm in the afternoon I'd say. Probably wrap up slightly earlier on Friday (5pm or so)
22:31:15 mriedem i also added this because of that debugging https://review.openstack.org/#/c/428985/
22:31:21 mriedem ttx: ok
22:31:22 superdan melwitt: why not?
22:31:40 mriedem melwitt: it would discover the computes in that cell
22:31:45 superdan right
22:31:45 mriedem which is what we want
22:31:46 melwitt superdan: writes to the api database? okay
22:31:55 mriedem writes the host mappings to the api db
22:32:12 melwitt they just have to configure it per cell
22:32:26 mriedem so do they have the api_database/connection in their nova-cells.conf for the low-level scheduler to do that?
22:32:35 mriedem they're going to have to for any of the nova-manage commands
22:32:40 superdan yeah,
22:32:53 mriedem we should probably point that out somewhere too
22:32:55 melwitt I think they do in that I think mgagne found out by experience that he had to configure it per cell (the api db connection)
22:33:03 mriedem was just going to mention mgagne
22:33:18 melwitt okay, cool. I shall continue my writings
22:33:18 superdan melwitt: or did he just say it worked that way?
22:33:22 openstackgerrit John Perkins proposed openstack/nova master: Integrate oslo.config validator https://review.openstack.org/428974
22:33:23 superdan melwitt: either way, doesn't matter that much
22:33:47 melwitt superdan: it matters in that the discover_hosts needs for it to be there for to write the host mappings
22:33:51 melwitt right?

Earlier   Later