Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-07-27
15:34:41 mriedem dansmith: so we should likely start with a bug saying this stuff will nuke your DISK_GB allocations on failure or revert at least
15:34:51 mriedem fried_rice: melwitt: ^
15:34:53 fried_rice dansmith: Taking that line out of the libvirt driver.
15:35:09 dansmith mriedem: for sure
15:35:24 dansmith fried_rice: sure, I'm happy to do it as well, either way
15:35:28 mriedem and we can track the various bugs in a spec in stein if we're going to go full on and support this
15:36:01 fried_rice dansmith: You want to write up the bug, I'll do the patch?
15:36:02 mriedem b/c we need a spec for the upgrade impacts obviously, and how to deploy the thing, plus CI requirements (which i'm already half-way done with)
15:36:03 melwitt sounds like a plan
15:36:21 dansmith fried_rice: if mriedem isn't going to
15:36:34 mriedem go ahead
15:36:43 dansmith fried_rice: I think mriedem really wants to do it
15:36:46 dansmith I heard him say earlier
15:36:53 dansmith so I don't want to step on his toes
15:36:57 mriedem i'm cleaning up stephenfin's last 2 changes in his vswitc hseries
15:37:04 dansmith because I think he measures his weekly progress by bugs reported
15:37:07 dansmith mriedem: more?
15:37:08 mriedem plus, zuul just f'ed my ceph ci run that was almost done!
15:37:12 melwitt this is new, gate failure RETRY_LIMIT
15:37:15 melwitt great
15:37:19 mriedem melwitt: yes same
15:37:34 openstackgerrit Chris Dent proposed openstack/nova master: WIP: funtional test with sharing providers https://review.openstack.org/586589
15:37:36 mriedem #status alert A zuul config error slipped through and caused a pile of job failures with retry_limit - a fix is being applied and should be back up in a few minutes
15:37:36 mriedem infra just posted a status
15:37:44 mriedem so don't recheck
15:38:15 cdent dansmith: ^ that gets the test actually making reasonable requests, but no more that that
15:38:27 cdent not sure if you care given the earlier discussion, but in case you do...
15:38:52 dansmith cdent: yeah, probably don't care now that I found this other one
15:38:57 dansmith but thanks for setting me straight
15:40:57 fried_rice cdent: All I care about is that you misspelled funtional
15:40:59 mriedem i can write the bug if no one has started yet
15:41:27 fried_rice Do it. And let the English see you do it.
15:41:40 mriedem alright
15:42:00 cdent fried_rice: that was dansmith in this case
15:42:16 cdent but I can see how it being me would be unsurprising
15:42:16 dansmith I was rushing
15:42:34 cdent is my new excuse
15:42:40 fried_rice cdent: If it had been three weeks ago, and it had been fuctional, I would have totally known it was you.
15:43:42 cdent I fuctional tests all the time
15:44:16 fried_rice Is there a way to mark a normal funtional test as an xfail?
15:45:46 fried_rice oh, shit, bhagyashri's test still succeeds with that bit commented out :(
15:46:32 fried_rice ignore me, phew.
15:47:02 cdent fried_rice: https://docs.python.org/3/library/unittest.html#unittest.expectedFailure
15:47:22 cdent https://docs.python.org/3/library/unittest.html#skipping-tests-and-expected-failures
15:47:24 fried_rice thanks cdent
15:47:54 fried_rice only py3? Are we running func tests on only py3 these days?
15:48:28 sean-k-mooney fried_rice: i think we have both. still
15:48:55 fried_rice yup sean-k-mooney
15:49:02 sean-k-mooney you can proably do a version check and jsut skip on 2 and expect failure on 3
15:50:10 openstack Launchpad bug 1784020 in OpenStack Compute (nova) "Shared storage providers are not supported and will break things if used" [High,Triaged]
15:50:10 mriedem here you go https://bugs.launchpad.net/nova/+bug/1784020
15:50:21 mriedem dansmith: fried_rice: melwitt: ^
15:50:27 dansmith oh thanks
15:50:39 cdent fried_rice: https://docs.python.org/2.7/library/unittest.html#skipping-tests-and-expected-failures
15:50:48 melwitt now that's a bug report
15:51:03 dansmith mine would have been 5% of that
15:51:11 mriedem fried_rice: testtools has an expectedFailure thing
15:51:14 melwitt hah, I know
15:51:22 mriedem you said i take pride in it...
15:51:26 dansmith "s'broken, kthx"
15:51:34 mriedem mostly because if i don't put those details in there, i'll totally forget wtf we talked about a year from now
15:51:57 melwitt yeah. the details are super helpful
15:52:18 sean-k-mooney mriedem: i think that bug also falls into the catagory of "we dont have ci for it so its broken by default"
15:52:28 mriedem well,
15:52:32 mriedem we don't have CI for a lot of things
15:52:50 mriedem and we still support them, <cough>evacuate</cough>
15:53:28 sean-k-mooney yes and i assume they are broken by default unless proven otherwise by it working when i use it and being happy
15:53:31 dansmith evacuate is hard to test for legit reasons, but this shared thing is not
15:53:43 dansmith and it's also often broken
15:54:10 mriedem yup
15:54:57 mriedem btw, yes, forced host live migrate/evacuate will drop the DISK_GB allocation on the shared provider
15:56:09 dansmith mriedem: from your test?
15:57:18 mriedem no just looking at teh code
15:57:27 mriedem https://github.com/openstack/nova/blob/6be7f7248fb1c2bbb890a0a48a424e205e173c9c/nova/scheduler/utils.py#L500
15:57:34 mriedem we only get the allocations for the instance against the source node
15:57:39 dansmith oh
15:57:40 mriedem and copy those to the dest node for the instance
15:57:42 mriedem double up
15:58:00 mriedem doesn't put anything on the migration record in the force cas
15:58:03 mriedem *case
15:58:53 mriedem hmm, which makes me wonder if we ever cleanup the dest host allocations on a failed live migration
15:58:55 mriedem that is forced
16:00:14 mriedem looks like post_live_migration will give you a warning but remove the doubled allocation
16:00:46 mriedem https://github.com/openstack/nova/blob/6be7f7248fb1c2bbb890a0a48a424e205e173c9c/nova/compute/manager.py#L6638L6669
16:00:49 mriedem oops
16:01:21 mriedem i'll write a functional test for the rollback forced live migration case
16:06:08 openstack Launchpad bug 1784022 in OpenStack Compute (nova) "Failed forced live migration does not rollback doubled up allocations in placement" [High,Triaged]
16:06:08 mriedem https://bugs.launchpad.net/nova/+bug/1784022
16:07:32 mriedem looks like we regressed that in queens
16:08:44 mriedem blarg https://review.openstack.org/#/c/507638/25/nova/compute/manager.py@6252
16:11:21 dansmith mriedem: are you saying we don't have a migration record if we do a forced?
16:11:30 mriedem dansmith: we do, but we don't put the allocations on it
16:11:38 mriedem b/c we don't go through the scheduler for forced
16:11:46 dansmith um
16:11:51 mriedem this is just one of the many reasons for the dreaded -5 in dublin
16:12:30 mriedem dansmith: forced live migration calls this method to double up the allocations from the source to the forced dest https://github.com/openstack/nova/blob/6be7f7248fb1c2bbb890a0a48a424e205e173c9c/nova/scheduler/utils.py#L473
16:12:40 mriedem that's from pike when doubling was all the rage
16:12:49 dansmith okay, so you're saying on forced we don't do the migration allocations, we just allocate against the newhost, then if we have to revert, we don't have the migration allocations to revert to the instance?
16:12:55 melwitt is it safe to recheck yet? I didn't see another status update
16:13:00 mriedem dansmith: correct

Earlier   Later