| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-15 | |||
| 17:15:42 | mordred | mriedem: so - looking further - I _think_ the problem is that novaclient passes its own logger to keystoneauth - so in this case my config of "hey keystoneauth, please log at debug" is not matching the logger named "novaclient" | |
| 17:15:54 | mordred | mriedem: I'm now trying also adding novaclient to the list of things I log at debug level | |
| 17:16:02 | mordred | and we'll see where that gets me | |
| 18:01:47 | openstackgerrit | Anusha Unnam proposed openstack/nova master: [WIP]Remove mox from nova/tests/unit/test_service.py https://review.openstack.org/434437 | |
| 18:07:16 | openstackgerrit | Sujitha proposed openstack/nova master: Change tags to default field in Instance object. https://review.openstack.org/415298 | |
| 18:10:20 | dims | jaypipes : oomichi : can you please take a look at this https://review.openstack.org/#/c/434264/ - will help get a bunch of upper constraints changes into requirements (https://review.openstack.org/#/c/434434/) | |
| 18:11:02 | dims | jaypipes : oomichi : basically checks if the exception message is from the old library or new library and asserts on both formats | |
| 18:11:06 | jaypipes | dims: on it. | |
| 18:11:21 | jaypipes | finally done with conf calls for the day. | |
| 18:12:50 | dims | :) | |
| 18:13:29 | jaypipes | dims: got a link to some output that shows the problem with this? | |
| 18:13:35 | jaypipes | dims: in the absense of a bug report... | |
| 18:13:55 | dims | one sec | |
| 18:16:54 | dims | jaypipes : test_validate_patternProperties_fails in http://logs.openstack.org/84/433484/2/check/gate-cross-nova-python27-ubuntu-xenial/dcdc886/testr_results.html.gz | |
| 18:18:07 | jaypipes | dims: got it. thanks! | |
| 18:21:25 | jaypipes | dims: done | |
| 18:21:32 | dims | thanks jay | |
| 18:39:25 | oomichi | dims: Thanks for doing that :-) This kind of message change is I imagined when implementing it at the first time. Maybe we will need this kind of change again and again. | |
| 18:40:05 | dims | oomichi : cross my fingers, it has held up for 2 years since your last change :) so maybe in 2 more years we revisit | |
| 18:41:15 | oomichi | dims: Haha, then I need to take care of them 25 times before the end of my life | |
| 18:41:26 | dims | LOL oomichi | |
| 18:46:39 | mordred | dims: Patch Set 1: Verified-1 Build failed (check pipeline). For information on how to proceed, see http://docs.openstack.org/infra/manual/developers.html#automated-testing - gate-shade-docs-ubuntu-xenial http://docs-draft.openstack.org/28/434428/1/check/gate-shade-docs-ubuntu-xenial/0d76e6f//doc/build/html/ : SUCCESS in 1m 55s - gate-shade-pep8-ubuntu-xenial | |
| 18:46:40 | mordred | http://logs.openstack.org/28/434428/1/check/gate-shade-pep8-ubuntu-xenial/56ea0fa/ : FAILURE in 1m 54s - gate-shade-python27-ubuntu-xenial http://logs.openstack.org/28/434428/1/check/gate-shade-python27-ubuntu-xenial/f4e4e9f/ : SUCCESS in 2m 55s - gate-shade-python35 http://logs.openstack.org/28/434428/1/check/gate-shade-python35/0792130/ : SUCCESS in 1m 24s - shade-coverage-ubuntu-xenial | |
| 18:46:42 | mordred | http://logs.openstack.org/28/434428/1/check/shade-coverage-ubuntu-xenial/e55e969/ : SUCCESS in 2m 21s - gate-shade-dsvm-functional-legacy http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-legacy/9297a44/ : POST_FAILURE in 26m 36s - gate-shade-dsvm-functional-legacy-libs-nv http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-legacy-libs-nv/aac8988/ : POST_FAILURE | |
| 18:46:44 | mordred | in 27m 46s (non-voting) - gate-shade-dsvm-functional-libs-nv http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-libs-nv/9a98b97/ : FAILURE in 33m 18s (non-voting) - gate-shade-dsvm-functional-magnum-nv http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-magnum-nv/1ecf057/ : POST_FAILURE in 25m 20s (non-voting) - gate-shade-dsvm-functional-neutron | |
| 18:46:46 | mordred | http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-neutron/3a5b115/ : POST_FAILURE in 32m 26s - gate-shade-dsvm-ansible http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-ansible/cc31729/ : POST_FAILURE in 28m 11s - gate-shade-dsvm-ansible-dev-nv http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-ansible-dev-nv/99f5d9e/ : POST_FAILURE in 25m 11s (non-voting) - | |
| 18:46:48 | mordred | gate-dsvm-nodepool-src http://logs.openstack.org/28/434428/1/check/gate-dsvm-nodepool-src/54b5fd4/ : SUCCESS in 36m 54s12:31 | |
| 18:46:50 | mordred | gah | |
| 18:46:52 | mordred | sorry everyone | |
| 18:47:02 | dims | oops :) | |
| 18:47:10 | mordred | diana_clarke: http://logs.openstack.org/04/434404/1/check/gate-shade-dsvm-functional-neutron/2e8773b/console.html#_2017-02-15_17_39_38_245587 | |
| 18:47:30 | mordred | diana_clarke: I'm getting back an empty tenant_usage with novaclient latest microversion | |
| 18:47:45 | mordred | diana_clarke: setting microversion to 2.0 seems to at least fix that test | |
| 18:48:10 | mordred | in any case - not sure if that's useful information or if it's just a thing I should know and deal with as I add in more explicit microversion support | |
| 18:48:13 | mordred | but thought I'd report back | |
| 18:50:01 | mordred | mriedem: is there anything in the new hotness we'd be excited about? | |
| 18:50:55 | mriedem | mordred: new hotness = ocata? we = ? | |
| 18:51:45 | mordred | mriedem: oh - you're learning about ocata in an auditorium at work? I was hoping you were learning what was going to be cool after containers or something | |
| 18:52:03 | mriedem | mordred: oh no, definitely not openstack related :) | |
| 19:26:04 | diana_clarke | mordred: That's worrisome.... if there are instances, you should see usage stats. | |
| 19:27:11 | mordred | diana_clarke: should the payload be there and have 0s in the total slots even with no instances? | |
| 19:27:41 | diana_clarke | mordred: I don't think so. Let me review the code quick. | |
| 19:28:07 | mordred | diana_clarke: oh - also - check it out - it's sending a start and end time | |
| 19:28:47 | mordred | that are only different by microseconds | |
| 19:30:36 | diana_clarke | mordred: nope, no zeros. you have to have at least one instance to get any aggregate totals | |
| 19:31:02 | mordred | diana_clarke: gotcha | |
| 19:33:08 | mordred | diana_clarke: cool - so I can fix our handling of that shade side - also, pinning back to 2.0 for now seems to have fixed the tests - so I'm good now and also know what it wants to be next | |
| 19:33:11 | mordred | thanks! | |
| 19:34:43 | diana_clarke | mordred: The fact that the results seem to differ between microversions is what's worrisome though.... In both cases, I would expect an empty payload. | |
| 19:35:14 | diana_clarke | (if there are indeed no instances in that time window in both cases) | |
| 19:38:29 | mordred | well - there should almost never be instances - since we're passing in start=now(), end=now() ... (it may not be a very good test) | |
| 19:44:12 | mriedem | BobBall: johnthetubaguy: are you still around? question on this xenapi live migration fix | |
| 19:44:53 | mriedem | https://review.openstack.org/#/c/424428/ | |
| 19:45:03 | mriedem | from what i can tell, that will blow up if using XenAPIBridgeDriver | |
| 19:45:38 | mriedem | which is also the default vif_driver | |
| 19:46:21 | diana_clarke | mordred: That doesn't explain how it ever passed though... which is what is nagging me. | |
| 19:46:28 | mordred | diana_clarke: yah | |
| 19:47:02 | mordred | diana_clarke: it definitely passes with version='2.0' - just verified https://review.openstack.org/#/c/434428/ | |
| 19:47:15 | mordred | diana_clarke: so a payload existing with 2.0 seems to be reproducible | |
| 19:47:16 | diana_clarke | mordred: I have a hunch that if you ignore the usage error and fix the other failing 26 tests, that it will pass again. | |
| 19:47:55 | diana_clarke | mordred: Are you printing the payload in the 2.0 case? Does it differ? | |
| 19:48:34 | mordred | diana_clarke: I'm not - we only print that on test failure. I could push up a patch to cause that test to fail so that we can see it in the 2.0 case pretty easily though | |
| 19:50:00 | mriedem | sdague: can you drop the -1 on this ocata backport for rc2? https://review.openstack.org/#/c/432954/ | |
| 19:50:02 | mordred | diana_clarke: https://review.openstack.org/434482 | |
| 19:50:05 | mriedem | my fix is backported on top | |
| 19:50:24 | sdague | mriedem: can we just merge them? or you don't want to for record keeping? | |
| 19:50:45 | mriedem | sdague: i was waiting to approve the bottom patch until the top is approved | |
| 19:50:49 | mriedem | but go ahead and +2 | |
| 19:52:37 | diana_clarke | mordred: My hunch is that instance creation is failing in other tests and that the usage test has only been passing by chance (side effect of there being instances from other tests). | |
| 19:52:54 | diana_clarke | mordred: But, as always I could be wrong. Curious to see that payload. | |
| 19:54:14 | mordred | diana_clarke: me too! | |
| 19:54:48 | diana_clarke | (but now() for the time window also makes me think I'm wrong) | |
| 20:00:45 | mriedem | dansmith: i have to run out for a bit, but can you hit https://review.openstack.org/#/c/434316/ and the one below it for rc2 when you get a chance? | |
| 20:01:03 | dansmith | okay | |
| 20:26:26 | diana_clarke | mordred: That's a non-zero payload (it has instances). | |
| 20:27:47 | diana_clarke | mordred: http://logs.openstack.org/82/434482/1/check/gate-shade-dsvm-functional-legacy/e884ee5/console.html#_2017-02-15_20_14_05_828079 | |
| 20:31:12 | diana_clarke | mordred: So, I'm sticking with my initial hunch: side effects between tests. That usage test is relying on other tests to have first created instances in a very small time window. | |
| 20:31:25 | mordred | diana_clarke: cool | |
| 20:31:47 | mordred | diana_clarke: fwiw, morgan is going to make a fun fixture for us to do a project create for each test so that we're completely isolated and not bleeding over - but until then, I think reworking this test to be less flaky is a good plan :) | |
| 20:32:26 | morgan | yes i am | |
| 20:32:34 | morgan | just... dealing with unwinding some other insane ick atm | |
| 20:33:03 | morgan | may be done with that unwind today... only another... 300-400 lines of code restructure to do | |
| 20:33:48 | diana_clarke | And I'm going to go back to teaching myself tripleo. I must have been a very bad person in a past life :( | |
| 20:42:00 | morgan | diana_clarke: i'm sorry, so sorry | |
| 20:50:11 | mtreinish | morgan, mordred: fwiw, we've been planning on moving tempest's project isolation bits into the lib for the past few cycles. We could work on that together and you could reuse it in the shade tests :) | |
| 20:50:36 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/432001 | |
| 20:51:13 | mtreinish | the isolation problem is something we had to solve a long time ago when I first enabled tempest to run in parallel workers | |
| 21:00:10 | openstackgerrit | Chris Friesen proposed openstack/nova master: show flavor info in server details https://review.openstack.org/434514 | |
| 21:01:20 | cfriesen | ^ preliminary code for displaying a subset of the flavor details directly embedded in the server details. | |
| 21:13:03 | mriedem | dansmith: did you want to get this into the next newton release (tomorrowish)? https://review.openstack.org/#/c/428671/ | |
| 21:13:10 | mriedem | sounded like a nasty bug | |
| 21:14:05 | mriedem | mtreinish: can you push this over the line? https://review.openstack.org/#/c/420051/ | |
| 21:15:02 | mtreinish | mriedem: +A | |
| 21:38:14 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove dead code: _safe_destroy_instance_residue https://review.openstack.org/434528 | |
| 21:41:55 | mriedem | coreycb: do you have a fix up yet for that update_cell issue? | |
| 21:42:09 | coreycb | mriedem, i will in about a minute | |
| 21:53:50 | openstackgerrit | Corey Bryant proposed openstack/nova master: Enable defaults for cell_v2 update_cell command https://review.openstack.org/434533 | |
| 21:55:01 | coreycb | mriedem, ok submitted that. i accidentally submitted that to stable/ocata first. | |
| 21:56:28 | mriedem | coreycb: commented inline | |
| 21:58:02 | mriedem | it should just be the =None thing for the kwargs | |