| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-13 | |||
| 15:38:37 | melwitt | so really it would be you have to run map_cell_and_hosts I think | |
| 15:38:48 | melwitt | after compute hosts show up later | |
| 15:38:49 | mriedem | but if there are major things that people have run into in practice trying to get this setup, it would be helpful to post those in the mailing list for posterity rather than a dump in IRC which hits maybe 1% of people right now on a friday | |
| 15:39:01 | mriedem | EmilienM: mwhahaha: amoralej: ^ | |
| 15:39:13 | mriedem | because those are the things i want to make sure are addressed in a doc | |
| 15:39:14 | melwitt | what I'm wondering is whether the cells v2 upgrade check runs on a fresh install or not? | |
| 15:39:22 | mwhahaha | it doesn't | |
| 15:39:32 | mwhahaha | the upgrade problem is something else | |
| 15:39:42 | mwhahaha | the problem is that you're adding requirements that no one know sabout | |
| 15:39:50 | mriedem | melwitt: which cells v2 upgrade check? the db schema migration? | |
| 15:39:51 | openstack | Launchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged] | |
| 15:39:51 | bauzas | melwitt: https://bugs.launchpad.net/nova/+bug/1656276 refers to a fresh install | |
| 15:40:06 | mwhahaha | oh yea,that's new as of today | |
| 15:40:08 | melwitt | mriedem: the thing that raises an error if there aren't two cell mappings | |
| 15:40:16 | bauzas | melwitt: and the proposed fix to amend https://github.com/openstack/puppet-nova/blob/master/manifests/db/sync_cell_v2.pp#L32 with return code 1 as valid would fix $ | |
| 15:40:22 | bauzas | would fix ^ (I mean) | |
| 15:40:40 | bauzas | what mwhahaha is discussing is another issue, related to the upgrade path | |
| 15:40:40 | mriedem | melwitt: mwhahaha: so the db schema migration runs on a fresh install by first checking to see if there are any flavors, if there aren't then it noops | |
| 15:40:43 | mwhahaha | https://bugs.launchpad.net/tripleo/+bug/1649341 was teh upgrade bug | |
| 15:40:44 | openstack | Launchpad bug 1649341 in tripleo "Undercloud upgrade fails with "Cell mappings are not created, but required for Ocata"" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | |
| 15:40:57 | bauzas | so another bug | |
| 15:40:58 | melwitt | mriedem: the (_('Cells v2'), _check_cellsv2), in nova/cmd/status.py | |
| 15:41:05 | mriedem | melwitt: so that's different | |
| 15:41:07 | melwitt | I see | |
| 15:41:18 | mriedem | and we haven't integrated the nova-status upgrade check stuff into our CI pipeline yet | |
| 15:41:21 | mriedem | another TODO of mine | |
| 15:41:41 | openstack | Launchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged] | |
| 15:41:41 | mwhahaha | which the problem is to solve https://bugs.launchpad.net/tripleo/+bug/1649341 i need to know how to properly tackle https://bugs.launchpad.net/nova/+bug/1656276 | |
| 15:41:49 | mwhahaha | where i didn't until today | |
| 15:41:59 | mwhahaha | so now we're just adding complexity on complexity | |
| 15:42:31 | mwhahaha | and adding accept 1 doesn't solve the upgrade case | |
| 15:42:45 | melwitt | yeah, I don't think accepting 1 helps anything really | |
| 15:42:52 | mwhahaha | i had a solution for the upgrade until the requirement of compute nodes was added | |
| 15:43:01 | mriedem | mwhahaha: EmilienM: so can we hold on a minute, | |
| 15:43:08 | mriedem | i understand you're frustrated, | |
| 15:43:31 | bauzas | melwitt: it only helps their CI to be green | |
| 15:43:51 | mriedem | i need help with posting the list of issues you have in a form that doesn't require me to be chasing 3 conversations at once | |
| 15:43:54 | melwitt | I'm not sure it'll run correctly after that if there are no cell mappings created though (nova) | |
| 15:43:56 | EmilienM | it's not about a green CI | |
| 15:44:05 | openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova: [py35] Fixes to get rally scenarios working https://review.openstack.org/419476 | |
| 15:44:05 | EmilienM | it's about how people outside devstack deploy your software | |
| 15:44:17 | mriedem | mwhahaha: EmilienM: so can you guys help me out by starting a ML thread with the current known issues you have so we can work through it there? | |
| 15:44:23 | EmilienM | mriedem: of course | |
| 15:44:58 | mriedem | b/c right now i'm not keeping up with this fire hose of stuff | |
| 15:45:03 | mriedem | and not everyone is around right now | |
| 15:45:18 | mriedem | so we need to start a list of items in the ML and tackle those | |
| 15:45:31 | mwhahaha | yea we can move to the ML | |
| 15:45:46 | bauzas | EmilienM: I don't disagree, I'm just trying to reduce the criticity so we can sit down and discuss without having the tense of urgency | |
| 15:46:26 | mriedem | we could start an etherpad too if that is helpful | |
| 15:46:41 | mriedem | hell i'm pretty sure we already have an etherpad for cells v2 | |
| 15:46:54 | bauzas | EmilienM: I'm really sorry that you folks are experiencing bugs against our recent improvements and I think your insights are definitely valid | |
| 15:47:54 | mriedem | i can't find the etherpad, only the wiki page | |
| 15:48:04 | mriedem | melwitt: do you know if we had a generic cells v2 etherpad for work items? | |
| 15:48:43 | melwitt | mriedem: we've had a number of etherpads, let me see | |
| 15:49:04 | mriedem | melwitt: yeah this is the older wiki https://wiki.openstack.org/wiki/Nova-Cells-v2 | |
| 15:49:08 | mriedem | but not seeing anything generic there | |
| 15:49:28 | EmilienM | bauzas: let's get this thing fixed | |
| 15:50:05 | melwitt | https://etherpad.openstack.org/p/nova-multiple-cells https://etherpad.openstack.org/p/cellsV2-remaining-work-items are old ones | |
| 15:51:44 | mriedem | melwitt: thanks i added those to the wiki | |
| 15:54:47 | mriedem | trying to update the TODOs in there also | |
| 15:56:20 | melwitt | thanks mriedem | |
| 15:57:19 | mriedem | also https://review.openstack.org/#/c/420051/ | |
| 15:58:13 | mriedem | EmilienM: melwitt: bauzas: mwhahaha: ok i've updated https://wiki.openstack.org/wiki/Nova-Cells-v2#Nova_Cells_v2 a bit - linked in those older etherpads, freshed up the TODOs, and added those bugs | |
| 15:58:18 | mriedem | let me know if i missed something | |
| 15:59:32 | EmilienM | mriedem: thanks for helping | |
| 16:00:00 | EmilienM | mriedem: are we sure to backport https://review.openstack.org/#/c/420051/ ? | |
| 16:00:05 | EmilienM | it broke us on master :/ | |
| 16:01:59 | mriedem | EmilienM: i thought that change was made on master specifically for you guys | |
| 16:02:04 | mwhahaha | no | |
| 16:02:09 | mwhahaha | it's not for us, we were working prior to that | |
| 16:02:10 | melwitt | I think it was sort of a coincidence that it broke you, because simple_cell_setup has always returned 1 in the case of not finding compute nodes. the reason you got past it is because you ran map_cell0 separately before running simple_cell_setup, and the old version of simple_cell_setup returned 0 if it found cell0 i.e. it assumed it had already run in hte past | |
| 16:02:24 | mwhahaha | we don't run map_cell0 | |
| 16:02:36 | mwhahaha | cause we didn't know that was a thing | |
| 16:04:28 | mriedem | ok i don't have the context on how that change broke you, is that in a bug somewhere? | |
| 16:04:30 | openstack | Launchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged] | |
| 16:04:30 | melwitt | in https://bugs.launchpad.net/nova/+bug/1656276 amoralej said you were mapping cell0 separately? | |
| 16:04:37 | openstack | Launchpad bug 1649341 in tripleo "Undercloud upgrade fails with "Cell mappings are not created, but required for Ocata"" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | |
| 16:04:37 | mriedem | is that https://bugs.launchpad.net/tripleo/+bug/1649341 ? | |
| 16:04:55 | melwitt | also from the code it's the only way you could have gotten simple_cell_setup to return 0 without any compute nodes | |
| 16:05:01 | melwitt | unless I've missed something | |
| 16:05:12 | mriedem | sorry i meant https://bugs.launchpad.net/nova/+bug/1656276 | |
| 16:05:48 | mwhahaha | so we're relying on puppet, and all we knew is that we had to run the simple setup (thats what we were told last time we came here with problems) | |
| 16:05:54 | mriedem | ok i've -W'd the backport until we can sort it out | |
| 16:05:55 | mwhahaha | so i went and implemented just the simple setup | |
| 16:06:03 | melwitt | mriedem: I *think* it's because they are running 'map_cell0' and then 'simple_cell_setup', before the idempotence patch it would all return 0, after the patch it returns 1 because it gets to the 'checking for hosts' part | |
| 16:06:06 | mwhahaha | cause i didn't know those other commands were required | |
| 16:06:36 | melwitt | the other commands aren't required | |
| 16:06:37 | amoralej | melwitt, we get cell0 by simple_cell_setup command | |
| 16:06:50 | mwhahaha | melwitt: evidently they are | |
| 16:06:55 | mwhahaha | melwitt: they weren't until that patch | |
| 16:07:03 | alaski | EmilienM: bauzas I've been out of discussions for a bit, but I had a PoC up at one point for computes to self register with a cell. It put an entry in the db that they could pull which told them which cell they were in, and then I was adding an HTTP API call for them to hit for registration. | |
| 16:07:21 | mriedem | melwitt: so you mean they got here https://review.openstack.org/#/c/420051/1/nova/cmd/manage.py@1240 ? | |
| 16:07:24 | mriedem | after that change i mean | |
| 16:07:52 | melwitt | yes | |
| 16:08:23 | amoralej | melwitt, we were getting return 0 in second execution | |
| 16:08:28 | mwhahaha | are you sure we're just not failing from line 1226 | |
| 16:08:29 | amoralej | because we retry it | |
| 16:08:39 | mriedem | melwitt: and that's because they got to https://review.openstack.org/#/c/420051/1/nova/cmd/manage.py@1379 ? | |
| 16:08:45 | mriedem | i.e. no computes yet so it would return 1 | |
| 16:08:47 | mriedem | well, | |