| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-24 | |||
| 13:49:54 | superdan | the scheduler will start keeping an instance in the same az it was booted in, | |
| 13:50:00 | superdan | even if they didn't ask for one | |
| 13:50:12 | superdan | that's because of the cruft still left to cleanup with reqspec | |
| 13:50:29 | superdan | reqspec has what they requested, but instance stores what it was booted in, and we're not restoring the reqspec at the right place yet | |
| 13:50:43 | superdan | now, I think in practice this is fine, and we even discussed it prior to merging | |
| 13:50:51 | superdan | however, I think what is happening is, | |
| 13:51:11 | superdan | tempest creates a test az and adds the one compute to it in another test, | |
| 13:51:36 | superdan | and if that instance happens to land there while the az is in place, and is then removed, the instance won't schedule anywhere because we have now told nova that the az it wants doesn't exist anymore | |
| 13:51:38 | superdan | we == tempest | |
| 13:51:54 | superdan | if the tests run such that we don't get lucky there, then we're fine | |
| 13:52:11 | superdan | bauwser: ^ | |
| 13:52:22 | mriedem | let me see what the test does | |
| 13:52:25 | bauwser | sorry was distracted by some internal bug triaging :) | |
| 13:52:27 | mriedem | the az is random btw | |
| 13:52:38 | superdan | mriedem: right | |
| 13:52:51 | superdan | mriedem: the az comes from compute/admin/test_aggregates | |
| 13:53:09 | mriedem | http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/compute/admin/test_migrations.py#n42 | |
| 13:53:38 | superdan | right, no az stuff there | |
| 13:54:58 | mriedem | superdan: oh so the instance lands on the single compute which, because of a concurrently running test, is now in an AZ, so the instance in the test that fails is now in that temp AZ which on cleanup of the other unrelated test is removed | |
| 13:55:00 | bauwser | mriedem: superdan: so if there is no AZ asked when booting, the default is not set | |
| 13:55:09 | mriedem | so when we move the instance, it's in an AZ that is now gone b/c of the tempest cleanup? | |
| 13:55:17 | superdan | mriedem: that's what I'm thinking yeah | |
| 13:55:23 | bauwser | mriedem: that could be a reason yes | |
| 13:55:52 | mriedem | look here http://logs.openstack.org/62/449362/1/check/gate-tempest-dsvm-py35-ubuntu-xenial/ec959b4/console.html#_2017-03-24_00_00_55_563868 | |
| 13:55:58 | mriedem | there is an AZ test right at the same time | |
| 13:56:24 | superdan | we need this cleanup to be done in order to ask for what they actually wanted: https://github.com/openstack/nova/blob/master/nova/conductor/tasks/migrate.py#L49-L55 | |
| 13:56:37 | bauwser | I was planning to do so next week | |
| 13:57:00 | bauwser | -ish | |
| 13:57:42 | bauwser | I could try to tackle that now | |
| 13:58:15 | superdan | hmm, | |
| 13:58:26 | superdan | looks like we got request_spec at init and we just overwrite it from_compontents? | |
| 13:58:29 | superdan | is that right? | |
| 13:58:39 | sdague | mriedem or others, anyone want to merge in i18n deletes. I'm trying to keep no more than 2 open at a time just to not waste time on rebasing - https://review.openstack.org/#/c/448082/ | |
| 13:59:23 | mriedem | superdan: i tried to summarize in the bug | |
| 13:59:26 | superdan | yeah, I think we should have that right there | |
| 13:59:32 | superdan | mriedem: okay I was going to, sorry | |
| 13:59:51 | bauwser | superdan: yup, I actually think we can directly use self.request_spec without needing to rehydrate | |
| 14:00:06 | superdan | I'll try | |
| 14:00:06 | bauwser | because it's useless to regeneate ity | |
| 14:00:11 | bauwser | lemme fire the drill | |
| 14:00:28 | bauwser | because some other move operations can have the same stuff | |
| 14:00:36 | bauwser | and here, it's resize | |
| 14:00:42 | bauwser | not live-mig | |
| 14:00:46 | bauwser | superdan: ^ | |
| 14:01:02 | bauwser | superdan: I'm cooking the patch | |
| 14:01:11 | superdan | we get it from the same place in both cases | |
| 14:01:12 | superdan | bauwser: about to submit | |
| 14:01:36 | bauwser | superdan: okay, so if you cook, don't forget to add the resize case | |
| 14:01:47 | superdan | eh? | |
| 14:02:00 | superdan | it's in migratetask, same for both | |
| 14:02:56 | bauwser | oh snpa | |
| 14:03:09 | bauwser | I was thinking we were loooking at the *live* migrate Task | |
| 14:03:24 | bauwser | nvm then | |
| 14:03:39 | bauwser | but I think we probably have the same for live-migration and rebuild | |
| 14:04:09 | bauwser | I'm just looking at that | |
| 14:04:10 | superdan | let me put this up to see if it breaks anything and/or appears to fix | |
| 14:05:08 | bauwser | okay | |
| 14:05:41 | bauwser | FWIW, live-migration is not having the same useless rehydratation | |
| 14:05:50 | bauwser | only if we have a not found spec | |
| 14:05:57 | bauwser | but we can remove the conditional | |
| 14:06:10 | superdan | yeah | |
| 14:06:15 | bauwser | so, I'm gonna fire a patch for that given we now have 100% of chances to have it | |
| 14:06:22 | superdan | alright | |
| 14:06:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: remove i18n log markers from nova.api.* https://review.openstack.org/448082 | |
| 14:07:22 | mriedem | yay my sump pump is running every other minute | |
| 14:07:24 | mriedem | weeee | |
| 14:07:55 | superdan | mriedem: http://imgur.com/DWMGBJW | |
| 14:08:27 | mriedem | which is which? | |
| 14:08:29 | mriedem | blue is rain? | |
| 14:08:34 | superdan | seconds on the left, inches on the right | |
| 14:08:44 | superdan | runtime seconds per hour, that is | |
| 14:09:11 | superdan | oh and green is rain | |
| 14:09:13 | bauwser | interesting | |
| 14:09:19 | bauwser | we don't build houses this way | |
| 14:09:30 | bauwser | is that for collecting rain ? | |
| 14:10:10 | bauwser | because I have a drain running all around my house but the gravity does the rest :) | |
| 14:10:25 | bauwser | anyway | |
| 14:10:37 | superdan | bauwser: if you're below grade that doesn't work :) | |
| 14:10:55 | sdague | superdan: or that there is elevation changes | |
| 14:11:11 | bauwser | superdan: ah correct then | |
| 14:11:19 | sdague | the midwest water table being 1' below the surface and everything flat is quite a thing | |
| 14:11:23 | superdan | yeah | |
| 14:12:01 | bauwser | do you collect that into pipes, or do you spill the water in your garden ? | |
| 14:12:31 | openstackgerrit | Dan Smith proposed openstack/nova master: Remove legacy regeneration of RequestSpec in MigrationTask https://review.openstack.org/449640 | |
| 14:12:36 | mriedem | in my neighborhood, you wash it down the hill to the suckers at the bottom, | |
| 14:12:36 | superdan | bauwser: mine goes into the storm drain | |
| 14:12:37 | leakypipes | bauwser: spill. :) | |
| 14:12:43 | mriedem | which unfortunately is me and another guy after me | |
| 14:12:46 | bauwser | interesting | |
| 14:13:08 | bauwser | because yet another regulation here is to defend us spillinh | |
| 14:13:16 | mriedem | supposed to be graded to drain into catch baskets that pump it into a pond further down the way | |
| 14:13:29 | bauwser | but rather collecting all the rain into pipes that are transfered to a specific collecting tank | |
| 14:13:36 | bauwser | that's a rain network | |
| 14:13:45 | superdan | bauwser: == storm drain | |
| 14:14:08 | mriedem | sometimes i hear you get a drunken frenchman drown in one of those and then he clogs up the pipes | |
| 14:14:15 | superdan | hah | |
| 14:14:28 | mriedem | stories to scare your kids, | |
| 14:14:33 | mriedem | like ringing doorbells | |
| 14:14:51 | bauwser | hah | |
| 14:17:40 | bauwser | superdan: what will be the name of your branch for cleaning up the migration task ? I'd ideally put it on the same gerrit topic | |