| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-09-04 | |||
| 18:45:17 | cdent | should we: a) make a devstack bug to make it be more unhappy when doing the migrations, or b) simply fix the exist code on the script? | |
| 18:46:13 | prometheanfire | imo the exit code being fixed would should make devstack unhappy when it should be unhappy | |
| 18:46:40 | cdent | yes | |
| 18:47:06 | cdent | Until you mentioned the exit code my b was going to be something else that how now passed out of my mind... | |
| 18:47:31 | prometheanfire | :D | |
| 18:47:39 | cdent | so two fixes, in nova: exit code handling on that command, intialize the placement db properly | |
| 18:47:39 | prometheanfire | fixing the error is still nice though | |
| 18:47:49 | cdent | that's backportable then | |
| 18:48:17 | prometheanfire | is that what's happening (api_db sync not init'ing the placement stuff)? | |
| 18:48:20 | mriedem | i'll hack some stuff up | |
| 18:48:28 | mriedem | this isn't api_db sync | |
| 18:48:48 | cdent | then on top of that we need address that an online placement db migration probably shouldn't be in nova's db migrations (even if placement wasn't being extracted) | |
| 18:48:48 | mriedem | but yes the online_data_migraitons command isn't configuring a global properly | |
| 18:49:06 | prometheanfire | ok | |
| 18:49:34 | prometheanfire | I'll be around if you want me to test, I should be able to take a snapshot of my master for rollback type testing too if you want | |
| 18:49:42 | prometheanfire | zfs <3 | |
| 18:49:52 | cdent | mriedem: please ping me if you can't get around to it, or if you do add me to the review please | |
| 18:50:24 | mriedem | sure | |
| 18:55:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Swallow fewer exceptions in _run_migration https://review.openstack.org/599744 | |
| 18:56:59 | mriedem | fwiw that blanket try/except has been around since the command was added https://review.openstack.org/#/c/278078/ | |
| 18:58:01 | prometheanfire | mriedem: want a paste with my error with that patch | |
| 18:58:04 | prometheanfire | it's longer | |
| 18:58:14 | mriedem | sure | |
| 18:58:43 | prometheanfire | mriedem: seems to fix the exit code problem too | |
| 18:58:56 | mriedem | yeah b/c you're not getting a nova exception | |
| 18:59:04 | mriedem | so it kills the command | |
| 18:59:07 | mriedem | i expect my patch to make devstack fail | |
| 18:59:16 | prometheanfire | I don't think it's that helpful (my paste) | |
| 18:59:17 | prometheanfire | http://paste.openstack.org/show/729440/ | |
| 18:59:27 | prometheanfire | but it's there anyway | |
| 18:59:42 | mriedem | yeah it's what i'd expect | |
| 18:59:52 | mriedem | i'll push the real fix in a follow up and then squash them | |
| 19:00:00 | mriedem | so we can see the failure in devstack and the fix in the follow up | |
| 19:01:10 | prometheanfire | wfm, going to lunch | |
| 19:01:12 | prometheanfire | you should too | |
| 19:04:29 | mriedem | eating while typing | |
| 19:04:32 | mriedem | the healthy way | |
| 19:06:53 | prometheanfire | :D | |
| 19:08:19 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Standardize CPU resource tracking https://review.openstack.org/555081 | |
| 19:19:35 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: allow transferring ownership of instance https://review.openstack.org/599598 | |
| 19:33:51 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: allow transferring ownership of instance https://review.openstack.org/599598 | |
| 19:41:43 | openstackgerrit | melanie witt proposed openstack/nova stable/queens: Fix the request context in ServiceFixture https://review.openstack.org/599762 | |
| 19:41:44 | openstackgerrit | melanie witt proposed openstack/nova stable/queens: Honor availability_zone hint via placement https://review.openstack.org/599763 | |
| 19:41:45 | openstackgerrit | melanie witt proposed openstack/nova stable/queens: Improve NeutronFixture and remove unncessary stubbing https://review.openstack.org/599764 | |
| 19:41:46 | openstackgerrit | melanie witt proposed openstack/nova stable/queens: Add functional test for affinity with multiple cells https://review.openstack.org/599765 | |
| 19:41:47 | openstackgerrit | melanie witt proposed openstack/nova stable/queens: Make scheduler.utils.setup_instance_group query all cells https://review.openstack.org/599766 | |
| 19:56:02 | mriedem | melwitt: rather than https://review.openstack.org/#/c/599763/ i'd probably just add that small 5 LOC to whatever you need that uses it and mention it in the commit message | |
| 19:57:01 | melwitt | mriedem: ok, can do | |
| 19:57:45 | melwitt | mriedem: it's a similar deal with the NeutronFixture changes. I could alternatively add a stub_network_* method call to my test. I didn't need it on master because of the NeutronFixture changes | |
| 19:59:59 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Configure placement DB context manager for online_data_migrations https://review.openstack.org/599822 | |
| 20:00:50 | mriedem | how big would the stub be on the patch in queens? | |
| 20:01:01 | mriedem | but https://review.openstack.org/#/c/599764/ is a bit gross yeah | |
| 20:02:17 | mriedem | prometheanfire: https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/nova/44/599744/1/check/tempest-dsvm-full-xenial/9691036/devstacklog.txt.gz shows that failure as expected | |
| 20:02:21 | mriedem | 2018-09-04 19:18:56.723 | ProgrammingError: (pymysql.err.ProgrammingError) (1146, u"Table 'nova_cell0.projects' doesn't exist") [SQL: u'SELECT projects.id \nFROM projects \nWHERE projects.external_id = %(external_id_1)s'] [parameters: {u'external_id_1': '00000000-0000-0000-0000-000000000000'}] (Background on this error at: http://sqlalche.me/e/f405) | |
| 20:03:10 | prometheanfire | :D | |
| 20:05:19 | cdent | mriedem: is it normal for there to be NovaExceptions that we would want to only print? | |
| 20:05:27 | mriedem | i'd say no | |
| 20:05:28 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Configure placement DB context manager for online_data_migrations https://review.openstack.org/599744 | |
| 20:05:31 | mriedem | i've removed it | |
| 20:05:36 | cdent | huzza | |
| 20:05:37 | cdent | h | |
| 20:05:37 | mriedem | ^ is the old squasharoo | |
| 20:06:52 | mriedem | oops guess i missed that in the squash | |
| 20:08:55 | melwitt | mriedem: I mean I could add a fake_network.set_stub_network_methods(self) all to my test instead of backporting the changes that removed all of those calls | |
| 20:08:59 | melwitt | *call | |
| 20:12:44 | mriedem | melwitt: you mean like *add* to your backport test what was removed from here right? https://review.openstack.org/#/c/599764/1/nova/tests/functional/db/test_archive.py | |
| 20:12:48 | mriedem | melwitt: if so, then yes just do that | |
| 20:12:56 | mriedem | the neutron fixture backport is very weird otherwise | |
| 20:13:25 | melwitt | mriedem: yes, add fake_network.set_stub_network_methods(self) to my backport test | |
| 20:13:29 | melwitt | k | |
| 20:13:40 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Configure placement DB context manager for online_data_migrations https://review.openstack.org/599744 | |
| 20:14:18 | mriedem | prometheanfire: ^ should make your dreams finally come true | |
| 20:15:01 | prometheanfire | mriedem: mostly, 'nova-status upgrade check' still shows a warning | |
| 20:15:11 | mriedem | paste me | |
| 20:15:12 | prometheanfire | but online migrations work (with nothing migrated) | |
| 20:15:42 | prometheanfire | mriedem: http://paste.openstack.org/show/729448/ | |
| 20:17:39 | prometheanfire | mriedem: I thought sean-k-mooney mentioned that being a false positive or something though | |
| 20:19:49 | mriedem | hmm, how many nodes are shown with nova hypervisor-list? | |
| 20:22:41 | prometheanfire | 2 | |
| 20:24:32 | mriedem | and how many in openstack resource provider list? | |
| 20:26:03 | prometheanfire | 2 | |
| 20:28:25 | mriedem | hmm, looks like we have a bug then | |
| 20:28:35 | mriedem | you want to write that up? | |
| 20:29:00 | prometheanfire | sure | |
| 20:29:14 | prometheanfire | I think this one is not pike only, I think I saw this in queens too | |
| 20:29:20 | prometheanfire | maybe more, not sure | |
| 20:29:46 | prometheanfire | s/pike/rocky | |
| 20:29:53 | prometheanfire | I keep on calling rocky pike | |
| 20:32:55 | cdent | criminey | |
| 20:34:02 | mriedem | prometheanfire: and list_cells shows cell0 and cell1 right? | |
| 20:34:28 | prometheanfire | https://bugs.launchpad.net/nova/+bug/1790721 | |
| 20:34:28 | openstack | Launchpad bug 1790721 in OpenStack Compute (nova) "nova-status upgrade check shows warnings when it shouldn't" [Undecided,New] | |
| 20:34:40 | prometheanfire | yep, shows both cells | |
| 20:35:28 | mriedem | i think it's the same bug | |
| 20:35:39 | mriedem | we're hitting the api db using this placement context manager, but it's not configured for the api db | |
| 20:35:44 | mriedem | so it's hitting cell0 looking for resource providers | |
| 20:36:50 | cdent | blargh. I thought nova-status use the api for placement checks? | |
| 20:36:51 | prometheanfire | ok, so partial fix so far then (I think) | |
| 20:37:10 | mriedem | cdent: there is a TODO from me in that same coe | |
| 20:37:12 | mriedem | *code | |