| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-27 | |||
| 15:37:12 | melwitt | this is new, gate failure RETRY_LIMIT | |
| 15:37:15 | melwitt | great | |
| 15:37:19 | mriedem | melwitt: yes same | |
| 15:37:34 | openstackgerrit | Chris Dent proposed openstack/nova master: WIP: funtional test with sharing providers https://review.openstack.org/586589 | |
| 15:37:36 | mriedem | #status alert A zuul config error slipped through and caused a pile of job failures with retry_limit - a fix is being applied and should be back up in a few minutes | |
| 15:37:36 | mriedem | infra just posted a status | |
| 15:37:44 | mriedem | so don't recheck | |
| 15:38:15 | cdent | dansmith: ^ that gets the test actually making reasonable requests, but no more that that | |
| 15:38:27 | cdent | not sure if you care given the earlier discussion, but in case you do... | |
| 15:38:52 | dansmith | cdent: yeah, probably don't care now that I found this other one | |
| 15:38:57 | dansmith | but thanks for setting me straight | |
| 15:40:57 | fried_rice | cdent: All I care about is that you misspelled funtional | |
| 15:40:59 | mriedem | i can write the bug if no one has started yet | |
| 15:41:27 | fried_rice | Do it. And let the English see you do it. | |
| 15:41:40 | mriedem | alright | |
| 15:42:00 | cdent | fried_rice: that was dansmith in this case | |
| 15:42:16 | cdent | but I can see how it being me would be unsurprising | |
| 15:42:16 | dansmith | I was rushing | |
| 15:42:34 | cdent | is my new excuse | |
| 15:42:40 | fried_rice | cdent: If it had been three weeks ago, and it had been fuctional, I would have totally known it was you. | |
| 15:43:42 | cdent | I fuctional tests all the time | |
| 15:44:16 | fried_rice | Is there a way to mark a normal funtional test as an xfail? | |
| 15:45:46 | fried_rice | oh, shit, bhagyashri's test still succeeds with that bit commented out :( | |
| 15:46:32 | fried_rice | ignore me, phew. | |
| 15:47:02 | cdent | fried_rice: https://docs.python.org/3/library/unittest.html#unittest.expectedFailure | |
| 15:47:22 | cdent | https://docs.python.org/3/library/unittest.html#skipping-tests-and-expected-failures | |
| 15:47:24 | fried_rice | thanks cdent | |
| 15:47:54 | fried_rice | only py3? Are we running func tests on only py3 these days? | |
| 15:48:28 | sean-k-mooney | fried_rice: i think we have both. still | |
| 15:48:55 | fried_rice | yup sean-k-mooney | |
| 15:49:02 | sean-k-mooney | you can proably do a version check and jsut skip on 2 and expect failure on 3 | |
| 15:50:10 | openstack | Launchpad bug 1784020 in OpenStack Compute (nova) "Shared storage providers are not supported and will break things if used" [High,Triaged] | |
| 15:50:10 | mriedem | here you go https://bugs.launchpad.net/nova/+bug/1784020 | |
| 15:50:21 | mriedem | dansmith: fried_rice: melwitt: ^ | |
| 15:50:27 | dansmith | oh thanks | |
| 15:50:39 | cdent | fried_rice: https://docs.python.org/2.7/library/unittest.html#skipping-tests-and-expected-failures | |
| 15:50:48 | melwitt | now that's a bug report | |
| 15:51:03 | dansmith | mine would have been 5% of that | |
| 15:51:11 | mriedem | fried_rice: testtools has an expectedFailure thing | |
| 15:51:14 | melwitt | hah, I know | |
| 15:51:22 | mriedem | you said i take pride in it... | |
| 15:51:26 | dansmith | "s'broken, kthx" | |
| 15:51:34 | mriedem | mostly because if i don't put those details in there, i'll totally forget wtf we talked about a year from now | |
| 15:51:57 | melwitt | yeah. the details are super helpful | |
| 15:52:18 | sean-k-mooney | mriedem: i think that bug also falls into the catagory of "we dont have ci for it so its broken by default" | |
| 15:52:28 | mriedem | well, | |
| 15:52:32 | mriedem | we don't have CI for a lot of things | |
| 15:52:50 | mriedem | and we still support them, <cough>evacuate</cough> | |
| 15:53:28 | sean-k-mooney | yes and i assume they are broken by default unless proven otherwise by it working when i use it and being happy | |
| 15:53:31 | dansmith | evacuate is hard to test for legit reasons, but this shared thing is not | |
| 15:53:43 | dansmith | and it's also often broken | |
| 15:54:10 | mriedem | yup | |
| 15:54:57 | mriedem | btw, yes, forced host live migrate/evacuate will drop the DISK_GB allocation on the shared provider | |
| 15:56:09 | dansmith | mriedem: from your test? | |
| 15:57:18 | mriedem | no just looking at teh code | |
| 15:57:27 | mriedem | https://github.com/openstack/nova/blob/6be7f7248fb1c2bbb890a0a48a424e205e173c9c/nova/scheduler/utils.py#L500 | |
| 15:57:34 | mriedem | we only get the allocations for the instance against the source node | |
| 15:57:39 | dansmith | oh | |
| 15:57:40 | mriedem | and copy those to the dest node for the instance | |
| 15:57:42 | mriedem | double up | |
| 15:58:00 | mriedem | doesn't put anything on the migration record in the force cas | |
| 15:58:03 | mriedem | *case | |
| 15:58:53 | mriedem | hmm, which makes me wonder if we ever cleanup the dest host allocations on a failed live migration | |
| 15:58:55 | mriedem | that is forced | |
| 16:00:14 | mriedem | looks like post_live_migration will give you a warning but remove the doubled allocation | |
| 16:00:46 | mriedem | https://github.com/openstack/nova/blob/6be7f7248fb1c2bbb890a0a48a424e205e173c9c/nova/compute/manager.py#L6638L6669 | |
| 16:00:49 | mriedem | oops | |
| 16:01:21 | mriedem | i'll write a functional test for the rollback forced live migration case | |
| 16:06:08 | openstack | Launchpad bug 1784022 in OpenStack Compute (nova) "Failed forced live migration does not rollback doubled up allocations in placement" [High,Triaged] | |
| 16:06:08 | mriedem | https://bugs.launchpad.net/nova/+bug/1784022 | |
| 16:07:32 | mriedem | looks like we regressed that in queens | |
| 16:08:44 | mriedem | blarg https://review.openstack.org/#/c/507638/25/nova/compute/manager.py@6252 | |
| 16:11:21 | dansmith | mriedem: are you saying we don't have a migration record if we do a forced? | |
| 16:11:30 | mriedem | dansmith: we do, but we don't put the allocations on it | |
| 16:11:38 | mriedem | b/c we don't go through the scheduler for forced | |
| 16:11:46 | dansmith | um | |
| 16:11:51 | mriedem | this is just one of the many reasons for the dreaded -5 in dublin | |
| 16:12:30 | mriedem | dansmith: forced live migration calls this method to double up the allocations from the source to the forced dest https://github.com/openstack/nova/blob/6be7f7248fb1c2bbb890a0a48a424e205e173c9c/nova/scheduler/utils.py#L473 | |
| 16:12:40 | mriedem | that's from pike when doubling was all the rage | |
| 16:12:49 | dansmith | okay, so you're saying on forced we don't do the migration allocations, we just allocate against the newhost, then if we have to revert, we don't have the migration allocations to revert to the instance? | |
| 16:12:55 | melwitt | is it safe to recheck yet? I didn't see another status update | |
| 16:13:00 | mriedem | dansmith: correct | |
| 16:13:04 | mriedem | melwitt: yeah i just did | |
| 16:13:19 | melwitt | ok | |
| 16:13:27 | mriedem | dansmith: i'll write a functional test for it when i'm back from lunch | |
| 16:13:36 | dansmith | mriedem: okay but the doubling is not intentional, just incidental since we didn't replace the instance allocs with the migration one yeah? | |
| 16:13:51 | mriedem | it's intentional | |
| 16:13:59 | mriedem | it mimics the behavior of doubling in the scheduler from before quens | |
| 16:14:00 | mriedem | *queens | |
| 16:14:08 | dansmith | right, but we shouldn't be doing any doubling anymore | |
| 16:14:17 | mriedem | sure, | |
| 16:14:19 | mriedem | but we are :) | |
| 16:14:20 | mriedem | for forced | |
| 16:14:24 | mriedem | b/c forced is FUN | |
| 16:14:26 | mriedem | -20! | |
| 16:14:33 | dansmith | I'm saying we shouldn't intend to be doing that, | |
| 16:14:40 | mriedem | not anymore no | |
| 16:14:44 | dansmith | which means it's a case we missed in converting to non-doubling | |
| 16:14:47 | mriedem | but we missed it in queens with your bp | |
| 16:14:49 | mriedem | yes | |