| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-26 | |||
| 12:16:34 | amrith | just that one change | |
| 12:16:42 | amrith | this is the change to get around the nova pass through API's | |
| 12:16:55 | amrith | it is the first time we're seeing 8.0.0 | |
| 12:17:06 | amrith | and whatever else nova backend brings to the party | |
| 12:17:24 | amrith | to repro it just recheck that change https://review.openstack.org/#/c/467447/ | |
| 12:17:29 | amrith | and watch the multi job | |
| 12:17:50 | amrith | it'll get to the wait_for_non_affinity_repl_to_fail thing | |
| 12:17:51 | amrith | and hang | |
| 12:18:00 | amrith | because nova lists the instance (in nova list) as scheduling | |
| 12:18:04 | amrith | while it has actually failed | |
| 12:18:10 | sdague | mriedem: it's not just that change | |
| 12:18:21 | sdague | the entire trove gate looks like this is busted | |
| 12:18:27 | sdague | it's the multinode for mariadb | |
| 12:18:29 | amrith | in my laptop now ... http://paste.openstack.org/show/610724/ | |
| 12:18:51 | amrith | you can see that scheduler gave up, conductor errored but nova list says scheduling | |
| 12:18:52 | amrith | and yes to what sdague said | |
| 12:18:54 | amrith | currnetly the trove gate is borked | |
| 12:18:54 | mriedem | so, how does the trove CI setup cells v2? is it using devstack? | |
| 12:18:55 | sdague | https://review.openstack.org/#/c/468102/ is a change to CONTRIBUTING.rst that has the same pattern | |
| 12:19:13 | mriedem | did this just start? | |
| 12:19:18 | amrith | yes | |
| 12:19:31 | amrith | I thought it was related to the 8.0.0 stuff | |
| 12:19:34 | amrith | but not true | |
| 12:19:59 | amrith | this may have been the last successful run https://review.openstack.org/#/c/457223/ | |
| 12:20:24 | amrith | certainly didnt happen > 4 days ago | |
| 12:20:42 | mriedem | http://logs.openstack.org/47/467447/14/check/gate-trove-scenario-dsvm-mariadb-multi-ubuntu-xenial-nv/6efec50/logs/screen-n-sch.txt.gz#_May_26_03_24_31_029544 | |
| 12:20:52 | mriedem | the anti-affinity filter kicked it out, which is why you're getting NoValidHost, | |
| 12:20:57 | mriedem | also, the scheduler is only seeing 1 host | |
| 12:21:02 | sdague | mriedem: right, that's expected | |
| 12:21:06 | mriedem | are you using devstack? | |
| 12:21:08 | sdague | that's actually the point of the test | |
| 12:21:28 | amrith | mriedem it is devstack driven through trovestack (formerly redstack) | |
| 12:21:49 | amrith | i can send you my local confs if that will help | |
| 12:22:01 | amrith | but you can see the confs in the CI as well | |
| 12:22:02 | sdague | the real question is actually why nova-conductor fails to actually error the the node | |
| 12:22:16 | sdague | not that it fails to schedule | |
| 12:22:28 | mriedem | yes i realize | |
| 12:22:33 | sdague | ok, cool | |
| 12:24:39 | mriedem | odd that i can't get hits on this in logstash | |
| 12:24:51 | sdague | is conductor indexed? | |
| 12:24:58 | sdague | or these jobs? | |
| 12:25:11 | mriedem | conductor is certainly indexed, | |
| 12:25:21 | mriedem | but i didn't think it was per job | |
| 12:25:24 | mriedem | it's per file | |
| 12:26:26 | mriedem | i think i see the problem, maybe | |
| 12:26:28 | mriedem | patch incoming | |
| 12:28:48 | sdague | cool | |
| 12:28:51 | sdague | thanks | |
| 12:28:59 | mriedem | amrith: got a bug? | |
| 12:30:23 | sdague | mriedem: I don't think he does, and he just had to head off for holiday weekend | |
| 12:31:08 | mriedem | ok, i'll put up a trove patch with depends-on to test this out | |
| 12:31:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Use targeted context when burying instances in cell0 https://review.openstack.org/468401 | |
| 12:31:38 | mriedem | dansmith: ^ | |
| 12:32:38 | openstackgerrit | Sean Dague proposed openstack/python-novaclient master: Have python-novaclient support global_request_id https://review.openstack.org/468126 | |
| 12:33:36 | mriedem | aha https://bugs.launchpad.net/nova/+bug/1693438 | |
| 12:33:36 | openstack | Launchpad bug 1693438 in OpenStack Compute (nova) "error instances remain in "Build" status and can't delete it" [Undecided,New] - Assigned to Tetsuro Nakamura (tetsuro0907) | |
| 12:33:42 | mriedem | already reported yesterday by someone doing a good job at CD | |
| 12:34:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Use targeted context when burying instances in cell0 https://review.openstack.org/468401 | |
| 12:35:51 | sdague | mriedem: nice | |
| 12:39:47 | mriedem | amrith: fyi https://review.openstack.org/#/c/468402/ | |
| 12:40:45 | mriedem | and now i'm afraid i must get cleaned up | |
| 12:42:08 | sdague | mriedem: what was the dogpile cache issue with nova? | |
| 12:44:24 | amrith | mriedem your trove change will fail because of the nova client 8.0.0. if you'd like please dep my change (which aims to fix that) on your proposed fix | |
| 12:44:28 | amrith | sdague ^^ | |
| 12:44:47 | sdague | amrith: yeh, I can update that | |
| 13:00:39 | leakypipes | morning folks | |
| 13:02:56 | sdague | leakypipes: in reading through test logs of my failing patches, I saw a lot of sqla warnings are being emitted around the traits tests | |
| 13:03:03 | sdague | because of unicode mismatch | |
| 13:03:17 | cdent | sfinucan: leakypipes put his +2 on https://review.openstack.org/#/c/461494/ so it is safe for you to raise your +1 to +2 if you like. or anyone else if they are feeling like a quick win | |
| 13:03:43 | sfinucan | cdent: Yup, done | |
| 13:03:44 | leakypipes | sdague: ya. I've been meaning to ask alex_xu to submit fixups for those... | |
| 13:03:50 | cdent | sfinucan: yay! | |
| 13:04:26 | sdague | leakypipes: ok, cool | |
| 13:04:31 | leakypipes | sdague: but since it's now the weekend for alex_xu I'll put the word out to get that cleaned up. | |
| 13:04:43 | leakypipes | efried: feel like an easy one? :) | |
| 13:04:53 | sdague | yeh, that's fine, I just wanted to make sure it was known | |
| 13:05:15 | leakypipes | sdague: yup, thank you sir :) | |
| 13:06:21 | sdague | cross service request id tracking to cinder is Jenkins +1 - https://review.openstack.org/#/c/468355/ if anyone wants to push that in | |
| 13:11:18 | openstackgerrit | Sean Dague proposed openstack/nova master: Migrate to oslo request_id middleware - mv 2.46 https://review.openstack.org/467998 | |
| 13:22:39 | mriedem | sdague: i don't think a bug was reported for the dogpile.cache test failure | |
| 13:27:17 | mriedem | sdague: tsk tsk https://review.openstack.org/#/c/468355/1 | |
| 13:28:23 | sdague | mriedem: yeh, I was going to swing around with minimums at the end | |
| 13:28:44 | sdague | because otherwise it's another couple of days per patch | |
| 13:28:55 | mriedem | we're using a new thing on a minimum version, | |
| 13:28:59 | mriedem | we should require the minimum version | |
| 13:29:01 | sdague | and if you are CDing and not using upper constraints | |
| 13:29:03 | mriedem | regardless of u-c | |
| 13:29:04 | sdague | you are crazy | |
| 13:29:16 | mriedem | if i'm packaging and picking up when minimums bump... | |
| 13:29:20 | mriedem | at the least | |
| 13:29:26 | mriedem | anyway, can we just do that? it's trivial | |
| 13:29:31 | mriedem | and it's the right thing | |
| 13:29:53 | sdague | mriedem: we have to do g-r change to and Depends-On, that's fine, but I need to swing around on it | |
| 13:30:07 | mriedem | sure | |
| 13:30:13 | mriedem | or i can quick | |
| 13:30:46 | sdague | mriedem: I'll at least state for my defense, even trivial individual patches aren't so much when you have 12 of them outstanding in 7 repos, with depends-on :) | |
| 13:31:12 | sdague | I'll do it | |
| 13:31:39 | mriedem | just pushed the reqs change | |
| 13:31:42 | sdague | ok | |
| 13:31:51 | mriedem | https://review.openstack.org/468417 | |