| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-16 | |||
| 15:12:51 | mriedem | i'm entering the roundtable danger zone, bbiab | |
| 15:13:23 | owalsh | dansmith: yea, assuming it's not too late to rename ocata migrations | |
| 15:13:47 | mriedem | we can rename it and backport it | |
| 15:13:52 | dansmith | owalsh: so, tbh, I'm not sure we've ever actually done this | |
| 15:14:02 | dansmith | mriedem: that won't really fix people in the field, actually | |
| 15:14:36 | dansmith | mriedem: if they have ocata now, and then update to this, they'll get two migrations for #21 | |
| 15:14:48 | owalsh | other option is to remove the 021 placeholder in ocata and turn 028 into a no-op | |
| 15:14:50 | dansmith | I guess if they use a proper package manager then it should remove the old one and install the new one | |
| 15:15:00 | dansmith | owalsh: I don't think that helps | |
| 15:15:23 | dansmith | owalsh: and we definitely don't want to do that in case someone was at #27 right now | |
| 15:15:42 | lyarwood | mdbooth: sorry just reading up, so iirc negative totals are valid right? BFV instances using flavors with local disk space etc? That's why I asked that the author add the examples where we can hit this into the commit message. | |
| 15:15:56 | dansmith | owalsh: so I think you have to do the rename in ocata, add a release note about it, and we will have to depend on packaging to properly clean up the two #21s | |
| 15:16:22 | lyarwood | mdbooth: ah but that's going to be > 0, I see your point | |
| 15:17:14 | mdbooth | lyarwood: That's only a working assumption, btw. Plenty of scope to be entirely wrong :) | |
| 15:20:26 | cfriesen | what's the process for documenting a known issue in stable/newton? (It'll be affected by bug 1644248 but we're not fixing it.) | |
| 15:20:26 | openstack | bug 1644248 in OpenStack Compute (nova) ocata "Nova incorrectly tracks live migration progress" [High,In progress] https://launchpad.net/bugs/1644248 - Assigned to Matt Riedemann (mriedem) | |
| 15:22:41 | owalsh | dansmith: hmm, anyone on #27 will need #28 so it needs to remain, but it also has to tolerate the case where the migration was already performed in newton #21 | |
| 15:22:52 | dansmith | owalsh: right, it has to be in both places | |
| 15:23:05 | dansmith | owalsh: but it should be idempotent so that's fine right? otherwise a backport of it doesn't work at all | |
| 15:23:52 | owalsh | dansmith: ok, then the only Q is where packaging does the right thing here. I'll take a look | |
| 15:24:01 | owalsh | s/where/whether/ | |
| 15:24:42 | dansmith | owalsh: yeah, so I'm pretty sure we've done this at least once in the far past, but I'm not positive.. so if that did happen, packaging must have done the right thing as I don't remember ever having to consider this rename issue before | |
| 15:28:36 | mriedem | dansmith: can't we just have the same migration in ocata at 21 and 28? | |
| 15:28:43 | mriedem | it's idempotent | |
| 15:28:45 | dansmith | mriedem: yes | |
| 15:28:57 | mriedem | ok | |
| 15:29:18 | dansmith | mriedem: but anyone with ocata installed would get two different #21s if the packaging didn't remove the 21_placeholder and replace with with 21_do_stuff | |
| 15:29:19 | dansmith | that's he concern | |
| 15:29:27 | mriedem | sure | |
| 15:29:40 | mriedem | fwiw i don't expect much of that to be the case right now, but i know it's a concern | |
| 15:30:06 | dansmith | sure, but have to account for it | |
| 15:30:43 | mriedem | so grenade didn't fail b/c it installed newton up to 21, and then when syncing ocata on the new side it saw it was already past 21 so didn't care? | |
| 15:31:02 | dansmith | no | |
| 15:31:20 | dansmith | I think grenade didn't catch it because it installs a new nova into /opt/stack/new and doesn't have the migrate_repo overlay with the old one | |
| 15:31:46 | dansmith | even if it did, it would use git as its package manager and it would remove the old one during an update I think | |
| 15:32:19 | mriedem | gah | |
| 15:32:39 | mriedem | should fix that gap at some point but i don't have an easy solution off the top of my head | |
| 15:32:59 | dansmith | yeah, it's actually weird that the same thing didn't happen with their packages already, actually | |
| 15:33:20 | sdague | dansmith: did we ever actually have a backported migration? | |
| 15:33:42 | sdague | I know the facility has existed for a while, but it's not regularly used | |
| 15:33:44 | dansmith | sdague: well, that's what I was trying to think ... I feel like we must've had one at least, but I'm not positive | |
| 15:33:50 | dansmith | sdague: yeah, very infrequently if ever | |
| 15:34:23 | sdague | so, I'm trying to understand the bug - https://bugs.launchpad.net/nova/+bug/1665370 | |
| 15:34:23 | openstack | Launchpad bug 1665370 in tripleo "api db_sync fails on upgrade due to duplicate migration files" [Critical,Triaged] - Assigned to Steven Hardy (shardy) | |
| 15:34:46 | dansmith | sdague: if you use python setup.py install, | |
| 15:34:56 | dansmith | sdague: you'll end up with two 21_ migrations, you get that right? | |
| 15:35:39 | mdbooth | bauzas: You around? | |
| 15:35:45 | sdague | dansmith: i'm not sure I do | |
| 15:35:57 | sdague | because won't the new rpm file remove the old one? | |
| 15:36:04 | mriedem | sdague: different names | |
| 15:36:13 | shardy | hey folks, thanks for looking into this - I'm around if you need any more info from my environment | |
| 15:36:22 | shardy | and/or to test any fixes | |
| 15:36:22 | mriedem | 021_placeholder in ocata, 021_build_something_fix in newton | |
| 15:36:24 | dansmith | sdague: not rpm, I said setup.py | |
| 15:36:35 | sdague | dansmith: ok, how do you get the filesystem | |
| 15:36:43 | sdague | maybe that's where I'm trying to understand | |
| 15:36:48 | dansmith | sdague: the confusing thing is that I think the rpm should be removing the old one even in their case | |
| 15:37:10 | sdague | I don't understand how you create a filesystem where they both are there | |
| 15:37:20 | dansmith | sdague: I'm just saying, take a system, setup.py install newton, then ocata, you'll end up with two 21_ files because you don't have a package manager in the mix | |
| 15:37:28 | mdbooth | bauzas: I'm trying to join some dots and I'm missing something. When ComputeManager calls update_instance_info(instance), I can see that the scheduler squirrels that instance away in host_state. However, I can't see anywhere that the information is actually used. I'm assuming that something somewhere uses it to keep resource usage up to date in between complete syncs, but I'm being blind. | |
| 15:37:36 | mriedem | we don't pip install nova during grenade? | |
| 15:37:41 | mriedem | i.e. pip install . | |
| 15:37:48 | sdague | mriedem: we do | |
| 15:37:49 | dansmith | mriedem: we "develop" install it don't we? | |
| 15:37:58 | dansmith | which is an egg link | |
| 15:38:02 | mriedem | to put the files in /var/lib/python27/site-packages/nova/.. | |
| 15:38:16 | sdague | mriedem: oh, we do do dev install | |
| 15:38:20 | bauzas | mriedem: I think we pip develop | |
| 15:38:22 | dansmith | yeah so dev install won't catch this | |
| 15:38:23 | sdague | that might be isolating | |
| 15:38:26 | dansmith | right | |
| 15:38:30 | mriedem | shart | |
| 15:38:46 | dansmith | the thing is, | |
| 15:38:47 | bauzas | devstack too | |
| 15:38:50 | dansmith | they install from rpms, | |
| 15:38:59 | bauzas | mdbooth: not sure I get your problem | |
| 15:38:59 | dansmith | so they should be removing the old 21 and installing the new one, AFAIK | |
| 15:39:04 | sdague | dansmith: right | |
| 15:39:13 | sdague | dansmith: the thing I could imagine is stale pyc files | |
| 15:39:20 | dansmith | sdague: yeah, could be that | |
| 15:39:23 | dansmith | shardy: owalsh ^ | |
| 15:39:38 | owalsh | dansmith: well, rpm should be clearing out .pyc/pyo files too | |
| 15:39:49 | mdbooth | bauzas: I'm assuming that when ComputeManager calls update_instance_info(instance), it somehow updates the host_state data which is ultimately used by DiskFilter | |
| 15:39:52 | bauzas | mdbooth: oh gotcha | |
| 15:39:56 | mdbooth | bauzas: Is that correct? | |
| 15:39:58 | BobBall | mriedem johnthetubaguy: I'd be quite happy to deprecate the vif driver config option - BUT - is it used by Rackspace with Quark? | |
| 15:40:02 | dansmith | owalsh: so maybe before we go tweaking stuff in tree we should try to figure out why the cleaning isn't happening? | |
| 15:40:05 | bauzas | mdbooth: yeah, some filters | |
| 15:40:08 | owalsh | dansmith: grabbing the bits to create a local rpm now... | |
| 15:40:14 | mdbooth | I'm just trying to follow the whole code path, because I suspect there's a bug somewhere in it | |
| 15:40:18 | shardy | dansmith, owalsh: actually this could be a packaging issue: | |
| 15:40:20 | shardy | http://paste.openstack.org/show/599261/ | |
| 15:40:21 | bauzas | mdbooth: not sure about the DiskFilter, but some others | |
| 15:40:21 | mdbooth | But I can't even find the code path :) | |
| 15:40:34 | bauzas | mdbooth: sec then | |
| 15:40:35 | shardy | the new python-nova doesn't seem to have obsoleted the old one | |
| 15:40:36 | sdague | dansmith: yeh, because I can't imagine there is another way to fix this, those placement backports are important | |
| 15:40:39 | BobBall | mriedem: johnthetubaguy: Also very happy to change the default to the OVS driver | |
| 15:40:48 | dansmith | shardy: maybe building the rpm again in an unclean space or something? | |
| 15:40:54 | sdague | I guess the only thing you could do is not change 21_ to be the new name | |