| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-27 | |||
| 22:13:23 | superdan | it's a really small window | |
| 22:13:31 | superdan | sure | |
| 22:13:47 | diana_clarke | so return code 1 is an error case, right? | |
| 22:13:50 | superdan | I mean if you like less awesome comments | |
| 22:14:08 | diana_clarke | superdan: not changing your comment, trying to write the man page | |
| 22:14:10 | diana_clarke | ;) | |
| 22:14:20 | superdan | they both mean that the instance isn't functional, | |
| 22:14:48 | superdan | and 1 could be a transient case as well I think, let me think about it | |
| 22:14:50 | superdan | I know, I meant "awesome" in the snarky way | |
| 22:15:35 | openstackgerrit | Dan Smith proposed openstack/nova: Scheduler calling the Placement API https://review.openstack.org/417961 | |
| 22:15:36 | openstackgerrit | Dan Smith proposed openstack/nova: Block starting compute unless placement conf is provided https://review.openstack.org/425806 | |
| 22:16:33 | superdan | diana_clarke: okay, so from here forward, 1 is always an error, but not one we expect to happen, because: | |
| 22:16:33 | diana_clarke | Or asking differently (like johnthetubaguy), what action should be taken for return codes 1 & 2? | |
| 22:17:15 | superdan | in the past, we didn't have instance mappings, so if you have an instance in the database and no mapping, it was created on an older release and you didn't do your homework to patch up all the mappings yet | |
| 22:17:31 | superdan | so the point of that tool is really to verify that you created instance mappings for all of them | |
| 22:17:52 | superdan | however, if you run that while things are going on, you could get retval=2 due to a transient luck-based event | |
| 22:18:12 | superdan | but doing the online migrations should fix both cases up for any non-booting instances | |
| 22:18:29 | superdan | so the answer to "what to do" is run the map command | |
| 22:19:13 | superdan | map_instances for 1 and map_cell_and_hosts for 2 (modulo transientness) | |
| 22:19:17 | superdan | clear as mud? | |
| 22:21:13 | diana_clarke | superdan: I going to re-read that 10 times in a row, and then hopefully it will be ;) much appreciated! | |
| 22:21:45 | superdan | let me say it again linearly now that I have it in my head... | |
| 22:22:06 | superdan | If you get retval=1, that means you need to run map_instances as some instances (which were created with a previous version) don't have instance mappings yet | |
| 22:22:48 | superdan | if you get retval=2, that may be because you're racing with instances that are booting, or it may mean that you are missing cell mappings, which means you need to run map_cell_and_hosts | |
| 22:23:24 | superdan | someone could modify that code to not return 2 if the instance has a building-related task state such that we know it's the former case for retval=2 | |
| 22:24:49 | diana_clarke | nice, thx! | |
| 22:43:57 | openstackgerrit | Ed Leafe proposed openstack/nova: placement: RT now adds proper Ironic inventory https://review.openstack.org/404472 | |
| 22:44:29 | figleaf | johnthetubaguy: cdent: ^^ adds the workarounds for the problem we identified earlier. | |
| 22:48:26 | mriedem_beach | hola | |
| 22:48:55 | mriedem | donde esta el progresso? | |
| 22:49:12 | superdan | where is the progress? | |
| 22:49:21 | mriedem | si | |
| 22:49:25 | mriedem | grenade patch is +W | |
| 22:49:32 | superdan | is it? | |
| 22:49:38 | mriedem | i thought you were going to +W it | |
| 22:49:51 | superdan | I will when I see the nova one yeah | |
| 22:49:57 | mriedem | oh | |
| 22:50:04 | superdan | I think the nova patch was running against a stale version of the grenade one, | |
| 22:50:08 | superdan | so it's going again now | |
| 22:50:16 | superdan | john fixed up a bunch of the unit and pep8 issues | |
| 22:50:34 | mriedem | yeah i see the series must have been rebased | |
| 22:50:38 | mriedem | i'll review that all again | |
| 22:51:34 | superdan | still getting NoValidHost on 11 tests on regular grenade ... | |
| 22:51:56 | superdan | jeez | |
| 22:52:05 | superdan | compute is still not starting because of detecting the placement config | |
| 22:52:11 | mriedem | i thought things were peachy this morning? | |
| 22:52:18 | mriedem | or was that not valid b/c of bad deps? | |
| 22:52:48 | superdan | yeah, I dunno now, I thought we were past that | |
| 22:53:36 | mriedem | maybe changing the grenade patch messed something up? | |
| 22:53:55 | superdan | PS3 worked. but that was the last time | |
| 22:54:11 | openstackgerrit | Pushkar Umaranikar proposed openstack/nova: Adopts keystoneauth with glance client. https://review.openstack.org/412634 | |
| 22:54:21 | superdan | but I see no reason why, there's been almost no change since | |
| 22:56:53 | superdan | yeah no placement config in nova.conf | |
| 22:56:55 | superdan | grr | |
| 22:58:00 | mriedem | hmm, | |
| 22:58:13 | mriedem | we're now deploying placement before nova in grenade right? | |
| 22:58:18 | superdan | yeah | |
| 22:58:24 | mriedem | which should configure nova.conf on the primary node, | |
| 22:58:33 | mriedem | which would actually be newton at that point | |
| 22:58:36 | mriedem | since it's pre-upgrade | |
| 22:58:44 | mriedem | 55 is updating a newton compute i mean | |
| 22:58:57 | superdan | but we still call the configure bit | |
| 22:59:03 | mriedem | sure, | |
| 22:59:13 | mriedem | and that shouldn't get wiped out or anything by the 60_nova part | |
| 22:59:19 | openstackgerrit | Pushkar Umaranikar proposed openstack/nova: Adopts keystoneauth with glance client. https://review.openstack.org/412634 | |
| 23:00:46 | mriedem | huh, so now both of the nova changes depend on the grenade change | |
| 23:00:50 | mriedem | doesn't hurt anything | |
| 23:01:02 | superdan | I did that because it ran against a stale grenade patch for some reason | |
| 23:01:06 | superdan | so when I rebased I just added it on | |
| 23:01:11 | superdan | didn't know why | |
| 23:01:32 | superdan | ahhh | |
| 23:01:47 | superdan | I wonder if we still need to do configure_placement_nova_compute in the 60_nova bit | |
| 23:02:02 | superdan | not sure why it wouldn't survive, but that's the part that gets us placemetn config | |
| 23:02:35 | superdan | also, what about this? http://logs.openstack.org/06/425806/6/check/gate-grenade-dsvm-neutron-ubuntu-xenial/fb281f1/logs/grenade.sh.txt.gz#_2017-01-27_20_02_44_142 | |
| 23:02:38 | mriedem | https://review.openstack.org/#/c/424730/15/projects/55_placement/from-newton/upgrade-placement@17 | |
| 23:02:47 | mriedem | ^ should configure nova-compute on the primary node for placement | |
| 23:02:57 | superdan | I know | |
| 23:03:02 | superdan | but | |
| 23:03:09 | superdan | after the run it's not set | |
| 23:03:18 | superdan | oh wait | |
| 23:03:26 | superdan | it doesn't have $NOVA_CONF in the run | |
| 23:04:16 | mriedem | because we didn't source the nova stuff yet? | |
| 23:04:22 | superdan | yeah | |
| 23:04:25 | mriedem | teehee | |
| 23:04:30 | mriedem | i believe i called this ~6 hours ago | |
| 23:04:35 | superdan | yes you did | |
| 23:05:18 | superdan | it's definitely weird doing this out of body but still part of nova | |
| 23:05:53 | mriedem | superdan: so do we no longer need the thing you added where it saves off the number of computes before and checks for that after? | |
| 23:06:09 | superdan | mriedem: that is being challenged, so I split it out | |
| 23:06:49 | superdan | with this set we're introducing a very specific race because the scheduler checks the min vers at startup | |
| 23:06:50 | superdan | which is why I think we need it | |
| 23:06:51 | superdan | but we haven't hit it yet, so ... | |
| 23:07:49 | superdan | mriedem: this is the standalone patch: https://review.openstack.org/#/c/426310/ | |
| 23:08:09 | superdan | which I see I have tabs in.. gdi | |
| 23:17:56 | mriedem | ok the 55_placement thing looks good now | |
| 23:19:18 | mriedem | now we play the waiting game | |
| 23:21:29 | superdan | cool | |
| 23:21:34 | superdan | personally, | |
| 23:21:38 | superdan | I hate the effing waiting game | |
| 23:21:45 | superdan | I've been playing it all week and I'm done with it | |