Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-01-13
16:07:24 mriedem after that change i mean
16:07:52 melwitt yes
16:08:23 amoralej melwitt, we were getting return 0 in second execution
16:08:28 mwhahaha are you sure we're just not failing from line 1226
16:08:29 amoralej because we retry it
16:08:39 mriedem melwitt: and that's because they got to https://review.openstack.org/#/c/420051/1/nova/cmd/manage.py@1379 ?
16:08:45 mriedem i.e. no computes yet so it would return 1
16:08:47 mriedem well,
16:08:51 alaski bauzas: EmilienM https://review.openstack.org/#/c/369634/
16:08:54 mriedem return None from that method, and simple_cell_setup would return 1
16:08:59 mriedem which then broke tripleo, right?
16:09:25 bauzas alaski: yeah I remember now our discussions
16:09:54 alaski that was just an idea I had though, there had been no real discussion or consensus on the direction
16:10:08 bauzas yup, what I told to EmilienM and amoralej
16:10:56 bauzas EmilienM: amoralej: mwhahaha: melwitt: mriedem: sorry, was in a meeting
16:11:06 melwitt alaski: that requires that a first cell was ever created though, right? they're in a situation where they have no compute hosts and no cell mapping other than cell0
16:11:32 amoralej but, why can't create a cell without compute nodes?
16:11:47 melwitt I'm guessing to avoid creating orphaned cells
16:11:52 bauzas melwitt: we could simple setup cells with no computes, and populate the mappings when adding the nodes
16:11:58 alaski melwitt: right. that wouldn't solve their issue. I wasn't exactly clear what's being discussed now.
16:12:08 mwhahaha shouldn't the cell be created first then nodes added to it?
16:12:14 mwhahaha like it seems weird to do the other way around
16:12:14 melwitt alaski: is that why we don't create cell mappings with no hosts? to avoid orphaned cells?
16:12:16 amoralej from a functional, operational point of view, creating the cell first and registering compute nodes later make sense imo
16:12:36 amoralej in fact compute nodes should be automatically added
16:12:43 amoralej i'd say
16:13:01 melwitt they could, but they'd have to know what cell to automatically map to
16:13:06 mriedem the compute nodes self registering with a cell mapping requires an upcall from the compute to the api which we were trying to avoid,
16:13:15 mriedem but is what alaski's WIP patch was for i think
16:13:31 mwhahaha but if you have the transport url, isn't it just a message?
16:13:35 melwitt if you have several, they'd have to automatically go to some 'staging cell' before you move them to where you actually want them. I think that's why it's not straightforward to be automatic. unless you're talking single cell
16:13:40 mwhahaha i mean whats the point of adding the transport url to the cell config
16:13:47 alaski mriedem: yep, but I was going for an HTTP call which sidesteps it a bit. it's assuming there's an admin API for registering a node in a cell
16:14:17 alaski melwitt: I'm not sure there was a conscious decision to not allow mapping an empty cell, at least not that I'm aware of.
16:14:54 melwitt mwhahaha: that's for the api cell to call into a compute cell. when the cell mapping is created in the api, the compute cell's transport_url is part of the mapping so the api cell can talk to it
16:15:04 alaski melwitt: it is a bit strange to stand up a cell if there are no ready computes, but I can see why people would want that
16:15:22 melwitt alaski: okay. all the code is written that way so I think I assumed there was some consensus I missed :P
16:15:38 melwitt yeah. it only makes sense in the fresh install case
16:15:50 alaski melwitt: it's also possible I'm forgetting some discussion that happened
16:15:51 mwhahaha alaski: i would respectfully disagree because as you deploy, people don't configure comptues first
16:16:11 mwhahaha alaski: controllers (and their config) generally are first which is why this is a problem
16:16:19 mriedem honestly i think we probably just always thought about cells v2 in an upgrade scenario, not fresh install
16:16:22 mriedem for the most part
16:16:28 alaski yeah
16:16:49 alaski for the initial deploy an empty cell makes more sense, but standing up a second cell wouldn't typically happen if there are no computes for it
16:17:07 melwitt okay. well, I saw andrey has a review up for creating an empty cell
16:17:08 mwhahaha no it still makes sense
16:17:23 mwhahaha because you're growing your cloud, so you create the cell config and populate it with the new nodes
16:17:50 alaski mwhahaha: I'm not saying it doesn't make sense, but it also makes sense to say don't stand up a new cell if there are no computes for it yet
16:18:41 cfriesen mdbooth: are you there?
16:18:53 melwitt typically I think we've thought of adding cells as something that you do while the cloud is running, not during code deploy. right?
16:19:10 alaski yes
16:19:13 melwitt it's just this initial state problem
16:20:56 mdbooth cfriesen: Yep
16:21:24 mwhahaha alaski: so i guess i'm not clear on what you mean 'to say don't stand up a new cell if there are no computes for it'
16:21:41 cfriesen re: my question at https://review.openstack.org/#/c/337160/17/nova/virt/libvirt/driver.py. I'm trying to figure out why we unconditionally call root_disk.create_snap(). Why is it needed for resize/migration to a different host?
16:21:43 mwhahaha alaski: do you mean as a recommendation to operators that you shouldn't have an empty cell running?
16:21:55 EmilienM I agree, compute nodes are usually deployed at the very end
16:22:17 mwhahaha alaski: what's the impact of the empty cell? would break something?
16:22:20 EmilienM I think part of the reason is because in the past it used to requires some API to be running
16:22:28 mdbooth cfriesen: It's used for rollback of Rbd disks.
16:22:33 alaski mwhahaha: I'm saying that as a deployer I would not add a new cell to the system unless I had computes ready to add to that cell. I'm not going to stand up a new cell that won't do anything yet
16:22:58 mdbooth cfriesen: IIRC it's a no-op for all other backends.
16:23:24 mdbooth However, cold migration/resize have a resize->confirm/rollback workflow
16:23:24 mwhahaha alaski: well if i know i'm goign to have computes, i'd want to create the cell but wait to add the computes until i get to that phase of my install
16:23:49 mwhahaha alaski: that's the thing, fuel, kolla, tripleo we all do controller configs first then add computes later because we need to have keystone and stuff running first
16:23:51 mdbooth For disk-based local storage, we copy the instance dir, whether single host or migrating host
16:24:10 mdbooth For rbd, they're the same disks, so if we didn't snapshot we wouldn't be able to rollback.
16:24:11 alaski mwhahaha: an empty cell should not break anything afaik, but I've been out of cells for a bit now so I may be mistaken.
16:24:19 cfriesen mdbooth: okay, that makes sense. missed the fact it was a no-op for most cases.
16:24:26 mwhahaha alaski: sure, just trying to understand this better
16:24:26 mriedem alaski: is your perspective on that model biased to how it was done with cells v1?
16:25:06 mriedem an empty cell shouldn't break anything, no
16:25:07 mdbooth cfriesen: It's a reasonable hook. Possible other storage could be cleverer under some circumstances, but Rbd requires it.
16:25:17 alaski mriedem: very likely, I'm irreparably damaged from v1
16:25:27 mriedem alaski: you're safe now, it's OK :)
16:25:38 openstackgerrit Sven Anderson proposed openstack/nova: Fix suggested database migration command https://review.openstack.org/420060
16:25:43 cfriesen mdbooth: agreed, it's fine. I just didn't understand what was going on.
16:26:01 mriedem ok so i've updated https://wiki.openstack.org/wiki/Nova-Cells-v2#Nova_Cells_v2 with bugs, todos and open questoins
16:26:03 mriedem *questions
16:26:27 alaski note that I'm not trying to argue for not allowing an empty cell, just explaining why that assumption was in my head when writing code which likely affected the direction
16:27:11 dtp dansmith melwitt i'm at your disposal today
16:27:35 mriedem alaski: yup
16:28:02 dims mriedem : bauzas : https://review.openstack.org/#/c/419476/ gets the full cinder/rally test working. that test does a nova boot/attach/detach/delete etc. should be easy to review as well (almost all code protected with six.PY3 checks so wont regress existing stuff too)
16:28:03 melwitt dtp: dansmith is on PTO today FYI
16:28:10 dtp ok
16:29:44 alaski o/
16:29:54 melwitt so things used to work prior to that patch because the first run of simple_cell_setup created cell0 and then failed when it found no compute nodes, then the retry found cell0 and returned 0
16:30:15 melwitt so it continued on without creating any cell mapping
16:30:54 bauzas melwitt: yup
16:33:04 dtp melwitt i'll just be reviewing your and dan's changes under your blueprints. lemme know if there's something else i should do
16:35:14 dtp if i don't respond immdediately, i'm trapped in a meeting
16:36:13 openstackgerrit Balazs Gibizer proposed openstack/nova: handle uninited fields in notification payload https://review.openstack.org/415857
16:36:44 mriedem melwitt: so the question right now is shoud this be a 1 or a 0 https://github.com/openstack/nova/blob/3ec43d81c324d8229bbb4b5db301175886a049b6/nova/cmd/manage.py#L1114
16:37:42 mriedem that's also called from the map_cell_and_hosts CLI https://github.com/openstack/nova/blob/3ec43d81c324d8229bbb4b5db301175886a049b6/nova/cmd/manage.py#L1316
16:37:56 mriedem note the comment in there about what the return codes mean (0 everything is done, 1 means you have more to do)
16:38:13 melwitt yeah. I think the 1 is supposed to be a signal that "hey, I didn't create any cell mappings other than cell0"
16:38:17 mriedem i think i took issue with that a bit during a review because i consider >0 == error
16:38:34 sdague which is the standard unixism
16:38:38 melwitt yeah

Earlier   Later