| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-06 | |||
| 15:08:46 | gibi | dansmith: I put it in front of the legacy code and hit the same tests you hit | |
| 15:09:01 | gibi | dansmith: it doesn't make any sense | |
| 15:09:27 | dansmith | hrm. | |
| 15:09:40 | dansmith | oh, well, | |
| 15:09:46 | dansmith | that can happen in the legit case, | |
| 15:10:01 | dansmith | because we're just checking for allocs there. if we find none, we will fall through to the legacy path as expected | |
| 15:10:53 | gibi | dansmith: ahh. When I question the existence of an if condition above it was the if allocs | |
| 15:11:13 | dansmith | oh | |
| 15:11:37 | gibi | as we are querying alloctions held by the migration.uuid on the _dest_ host | |
| 15:11:47 | gibi | but that is always empty | |
| 15:11:55 | gibi | as migration holds allocation on the source host only | |
| 15:12:01 | dansmith | right right | |
| 15:12:19 | dansmith | except for same-host, but that should never hit this because of the source check above | |
| 15:12:30 | dansmith | I'll move the exception and re-run to confirm | |
| 15:12:47 | gibi | I did that and got green results | |
| 15:13:34 | gibi | so my theory is that in case of revert on dest we always see empty allocs and don't return so we hit the legacy path below | |
| 15:14:22 | dansmith | okay | |
| 15:14:53 | dansmith | hmm, which I guess isn't a problem, | |
| 15:15:11 | dansmith | if we're deleting dest allocs and restoring source allocs, | |
| 15:15:22 | dansmith | except for the case of if we miss the migration allocs we need to delete | |
| 15:15:37 | gibi | it seams it doesn't casue a leak, but I have to leak a bit more closer | |
| 15:15:41 | dansmith | gibi: still not sure where this is coming from though -- are you chasing a bug or writing a test or what? | |
| 15:16:44 | gibi | dansmith: trying implement placement 1.28 support in report client (consumer_generation) and writing consumer gen conflict tests and I hit this legacy path that I thought I should not hit any more | |
| 15:16:57 | dansmith | okay | |
| 15:17:07 | dansmith | we should be able to clean all this up at this point anyway I think | |
| 15:17:16 | dansmith | perhaps we should make a point of ripping this all out for stein? | |
| 15:17:29 | dansmith | anyone with an unconfirmed migration between pike and stein probably has other problems :) | |
| 15:19:13 | gibi | dansmith: to avoid hitting the legacy and later rip it out I need to change https://github.com/openstack/nova/blob/8688b25ca7379391cba28fab30b5a628957e673e/nova/compute/manager.py#L3952-L3955 to query the migration allocation from the source host. Does that make sense to you? | |
| 15:19:45 | dansmith | well.. | |
| 15:20:08 | dansmith | the point of that is to make the destination node not do anything at all (i.e. not run the legacy path) if the new-style allocations were used | |
| 15:21:04 | dansmith | so, I guess, but you'll have to do a lookup of the source uuid I think in order to use that method | |
| 15:21:16 | gibi | dansmith: yes that is the goal. if that code sees the allocation held by the migration on the source host it returns and let the finish_revert_resize do the work | |
| 15:21:18 | dansmith | which is a little bit icky | |
| 15:22:14 | gibi | dansmith: ohh. Then we just assume there is no legacy migration any more and unconditioanlly not doing anython on the dest host | |
| 15:22:18 | dansmith | gibi: ah, just change that call to get_allocations_by_consumer() and if the migration has any allocations, then we must be doing new-style | |
| 15:22:32 | gibi | dansmith: good point, we can do that | |
| 15:23:14 | gibi | then I will file a bug and this small change as a bugfix. then later when Stein is open I can remove the whole legacy path in a separate patch | |
| 15:23:29 | dansmith | yar | |
| 15:23:48 | gibi | dansmith: thanks for the brainpower | |
| 15:24:29 | dansmith | np :) | |
| 15:30:28 | mdbooth | Incidentally, my pbr test problem was that I wasn't explicitly installing python-testrepository. This resulted in <unfathomable pbr-related weirdness> ultimately resulting in the tests attempting to run functional rather than just unit. | |
| 15:35:06 | efried | jaypipes, mriedem, dansmith: What is it going to take to have sufficient confidence in any of the various attempts to reduce the number of redundant placement calls per periodic to actually merge code? | |
| 15:36:36 | efried | and are both showing green across the board test-wise. | |
| 15:36:36 | efried | https://review.openstack.org/#/c/587050/ (removes the "last" _update) | |
| 15:36:36 | efried | https://review.openstack.org/#/c/588091/ (removes the "first" _update) | |
| 15:36:36 | efried | e.g. these two are quite similar | |
| 15:37:46 | jaypipes | efried: which one do you prefer? | |
| 15:38:41 | efried | jaypipes: It makes little difference to me. The first one removes more lines of code. At the moment it's sitting on top of the reshaper series, but could be extracted easily. | |
| 15:42:11 | mriedem | idk | |
| 15:42:27 | mriedem | stuff like that likely won't show a side effect until we've merged it and run it awhile and found some weird failures, | |
| 15:42:33 | mriedem | which will likely be non-trivial to debug | |
| 15:42:38 | efried | just so. | |
| 15:42:43 | mriedem | so definitely not rocky | |
| 15:43:15 | mriedem | and i'd expect what you'd see are weird scheduling failures | |
| 15:43:20 | mriedem | due to some timing issue | |
| 15:43:34 | dansmith | and maybe not at all at gate-level scale | |
| 15:44:39 | mriedem | would be nice to have some company with a stress test lab kick around either of those to see what falls out | |
| 15:44:47 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Delete instance_group_member records from API DB during archive https://review.openstack.org/588943 | |
| 15:45:07 | mriedem | but i'm pretty sure at this point in my time working on openstack, no company has a stress test lab that shares results publicly :) | |
| 15:45:32 | mriedem | if we don't see it in the gate, | |
| 15:45:44 | mriedem | we'll see it 18-24 months from now when someone actually upgrades to use it and finds problems | |
| 15:45:50 | efried | right | |
| 15:46:02 | jaypipes | mriedem: 36-64 months from now. | |
| 15:46:11 | mriedem | well, cern would be the first to hit it probably | |
| 15:46:13 | efried | But even that is a big "maybe". | |
| 15:46:56 | mriedem | i also haven't had the time to read cdent's write up on this either yet | |
| 15:47:09 | efried | mriedem: TL;DR we're calling _update twice every periodic. | |
| 15:47:16 | mriedem | yeah i knew that much :) | |
| 15:47:24 | efried | which calls all the placement things to refresh the cache etc. | |
| 15:47:33 | efried | that's really all there is to it. | |
| 15:47:42 | cdent | mriedem: there's not much more than that what efried says, must more words to indicate the code path | |
| 15:47:43 | efried | Do we need both calls for some reason? | |
| 15:48:05 | mriedem | also https://review.openstack.org/#/c/520024/ | |
| 15:48:20 | mriedem | given ^ you should probably see if the ovh gang is interested in testing out either of those changes | |
| 15:48:50 | mriedem | when did we start doing the 2 calls? | |
| 15:48:56 | mriedem | b/c ovh might not have a region running that yet | |
| 15:49:30 | cdent | >18 months ago | |
| 15:49:39 | cdent | it was there when I did the first version of the post | |
| 15:50:26 | efried | oh look, https://review.openstack.org/#/c/520024 is exactly the same as https://review.openstack.org/#/c/588091/ | |
| 15:50:29 | mriedem | looking at ^ from ovh they opted to leave in the _update call at the end | |
| 15:50:55 | mriedem | the +1 from minho on that was also because they had done a duplicate of the same patch | |
| 15:51:03 | openstackgerrit | Eric Fried proposed openstack/nova master: Update resources once in update_available_resource https://review.openstack.org/520024 | |
| 15:52:44 | mriedem | ok i have voted | |
| 15:52:54 | openstackgerrit | Chen proposed openstack/nova master: Trivial fix on migration doc https://review.openstack.org/589028 | |
| 15:52:55 | mriedem | if it's a coin toss, go with what ovh is already using | |
| 15:53:13 | mriedem | and drop the duplicate change from efried and coalesce on the ovh patch | |
| 15:53:26 | efried | Note that there's also https://review.openstack.org/#/c/588094/ which is failing tox (expected because I haven't updated those tests yet) but also live migration consistently. | |
| 15:53:53 | efried | mriedem: The coalesce is a no-op; the extras in mine are only there because of the reshaper series. I've abandoned it. | |
| 15:54:05 | mriedem | maciejjozefczyk: are you guys running with this in production now? https://review.openstack.org/#/c/520024/ | |
| 15:54:16 | mriedem | maciejjozefczyk: any side effects or issues with that patch? | |
| 16:00:06 | mriedem | jaypipes: has cfriesen talked with you about how they account for hosting shared and pinned cpus on the same host by making VCPU inventory a fraction? | |
| 16:00:37 | mriedem | i assume it would have come up during https://review.openstack.org/#/c/555081/ | |
| 16:01:25 | jaypipes | mriedem: nope. | |
| 16:01:46 | jaypipes | mriedem: and I'm 100% against making amount a non-integer value. | |
| 16:03:55 | cdent | It was briefly glossed over in Dublin, but the response then was "placement can't do that" | |
| 16:04:05 | cdent | or maybe s/can't/won't/ | |
| 16:04:18 | dansmith | I'm also 100% against that :) | |
| 16:04:24 | mriedem | i'm not suggesting we do that, | |
| 16:04:35 | mriedem | but i thought there was an alternative way to model it via nested providers, or some other kind of inventory | |
| 16:04:38 | mriedem | PCPUs? | |