| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-15 | |||
| 16:55:29 | mordred | diana_clarke: but unfortunately there is no nova api in the log there ... I'm making a patch now to try to capture more things | |
| 17:05:32 | jroll | hey nova, this refactor has been sitting for a while, if you'd like an easy-ish review :) https://review.openstack.org/#/c/399022/ | |
| 17:05:41 | jroll | +13, -285 btw :D | |
| 17:06:53 | mriedem | jroll: that reminds me, how do we ci test the hashring stuff? weren't there ideas on that with a multinode job? | |
| 17:07:26 | openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: remove *_topic config opts https://review.openstack.org/433924 | |
| 17:07:41 | jroll | mriedem: yeah, we've been working on a multinode job, which I believe has two computes | |
| 17:07:54 | jroll | idk if we want to do anything fancy like bring on compute host down | |
| 17:08:18 | mriedem | probably not if yo'ure running tempest on it at the same time | |
| 17:08:24 | jroll | right | |
| 17:08:28 | mriedem | jroll: the ironic networking one | |
| 17:08:32 | openstackgerrit | Sujitha proposed openstack/nova master: Adding tags field to InstancePayload https://review.openstack.org/407228 | |
| 17:08:38 | jroll | we could do something in setup, though | |
| 17:08:42 | mriedem | jroll: you could write your own integration tests for it i guess... | |
| 17:08:53 | jroll | or that yeah | |
| 17:08:57 | mriedem | ironic-ha-dsvm-integration | |
| 17:09:12 | jroll | ya | |
| 17:10:28 | mriedem | sdague: i've got the backport on top of https://review.openstack.org/#/c/432954/ for the CONF.placement.os_interface fixes so can you change that -1 to a +2? | |
| 17:10:32 | mordred | morgan, mriedem: either of you know what, if anyhting, I need to do with the novaclient Client constructor or python logging to get http debug tracing from novaclient commands? I don't seem to be getting them just from setting keystoneauth logger to debug | |
| 17:11:33 | mriedem | mordred: python logging is always greek to me, i did this in nova for enabling debug logging with glanceclient https://github.com/openstack/nova/blob/master/nova/config.py#L38 | |
| 17:11:54 | jroll | mriedem: I don't believe that ironic networking job is multinode, fwiw, unless you think this might affect it some other way | |
| 17:12:00 | mriedem | i assume there is a logging.conf or something somewhere where you should be able to just set this stuff | |
| 17:12:29 | mriedem | jroll: i'm just buying time brothaman. i'm in an auditorium at work right now hearing about the new hotness | |
| 17:12:33 | mriedem | so can't review things atm | |
| 17:12:37 | mriedem | but bs is free | |
| 17:12:38 | openstackgerrit | Eric Brown proposed openstack/nova master: Add RPC version aliases for Ocata https://review.openstack.org/425958 | |
| 17:12:49 | jroll | mriedem: heh, right on | |
| 17:15:42 | mordred | mriedem: so - looking further - I _think_ the problem is that novaclient passes its own logger to keystoneauth - so in this case my config of "hey keystoneauth, please log at debug" is not matching the logger named "novaclient" | |
| 17:15:54 | mordred | mriedem: I'm now trying also adding novaclient to the list of things I log at debug level | |
| 17:16:02 | mordred | and we'll see where that gets me | |
| 18:01:47 | openstackgerrit | Anusha Unnam proposed openstack/nova master: [WIP]Remove mox from nova/tests/unit/test_service.py https://review.openstack.org/434437 | |
| 18:07:16 | openstackgerrit | Sujitha proposed openstack/nova master: Change tags to default field in Instance object. https://review.openstack.org/415298 | |
| 18:10:20 | dims | jaypipes : oomichi : can you please take a look at this https://review.openstack.org/#/c/434264/ - will help get a bunch of upper constraints changes into requirements (https://review.openstack.org/#/c/434434/) | |
| 18:11:02 | dims | jaypipes : oomichi : basically checks if the exception message is from the old library or new library and asserts on both formats | |
| 18:11:06 | jaypipes | dims: on it. | |
| 18:11:21 | jaypipes | finally done with conf calls for the day. | |
| 18:12:50 | dims | :) | |
| 18:13:29 | jaypipes | dims: got a link to some output that shows the problem with this? | |
| 18:13:35 | jaypipes | dims: in the absense of a bug report... | |
| 18:13:55 | dims | one sec | |
| 18:16:54 | dims | jaypipes : test_validate_patternProperties_fails in http://logs.openstack.org/84/433484/2/check/gate-cross-nova-python27-ubuntu-xenial/dcdc886/testr_results.html.gz | |
| 18:18:07 | jaypipes | dims: got it. thanks! | |
| 18:21:25 | jaypipes | dims: done | |
| 18:21:32 | dims | thanks jay | |
| 18:39:25 | oomichi | dims: Thanks for doing that :-) This kind of message change is I imagined when implementing it at the first time. Maybe we will need this kind of change again and again. | |
| 18:40:05 | dims | oomichi : cross my fingers, it has held up for 2 years since your last change :) so maybe in 2 more years we revisit | |
| 18:41:15 | oomichi | dims: Haha, then I need to take care of them 25 times before the end of my life | |
| 18:41:26 | dims | LOL oomichi | |
| 18:46:39 | mordred | dims: Patch Set 1: Verified-1 Build failed (check pipeline). For information on how to proceed, see http://docs.openstack.org/infra/manual/developers.html#automated-testing - gate-shade-docs-ubuntu-xenial http://docs-draft.openstack.org/28/434428/1/check/gate-shade-docs-ubuntu-xenial/0d76e6f//doc/build/html/ : SUCCESS in 1m 55s - gate-shade-pep8-ubuntu-xenial | |
| 18:46:40 | mordred | http://logs.openstack.org/28/434428/1/check/gate-shade-pep8-ubuntu-xenial/56ea0fa/ : FAILURE in 1m 54s - gate-shade-python27-ubuntu-xenial http://logs.openstack.org/28/434428/1/check/gate-shade-python27-ubuntu-xenial/f4e4e9f/ : SUCCESS in 2m 55s - gate-shade-python35 http://logs.openstack.org/28/434428/1/check/gate-shade-python35/0792130/ : SUCCESS in 1m 24s - shade-coverage-ubuntu-xenial | |
| 18:46:42 | mordred | http://logs.openstack.org/28/434428/1/check/shade-coverage-ubuntu-xenial/e55e969/ : SUCCESS in 2m 21s - gate-shade-dsvm-functional-legacy http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-legacy/9297a44/ : POST_FAILURE in 26m 36s - gate-shade-dsvm-functional-legacy-libs-nv http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-legacy-libs-nv/aac8988/ : POST_FAILURE | |
| 18:46:44 | mordred | in 27m 46s (non-voting) - gate-shade-dsvm-functional-libs-nv http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-libs-nv/9a98b97/ : FAILURE in 33m 18s (non-voting) - gate-shade-dsvm-functional-magnum-nv http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-magnum-nv/1ecf057/ : POST_FAILURE in 25m 20s (non-voting) - gate-shade-dsvm-functional-neutron | |
| 18:46:46 | mordred | http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-functional-neutron/3a5b115/ : POST_FAILURE in 32m 26s - gate-shade-dsvm-ansible http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-ansible/cc31729/ : POST_FAILURE in 28m 11s - gate-shade-dsvm-ansible-dev-nv http://logs.openstack.org/28/434428/1/check/gate-shade-dsvm-ansible-dev-nv/99f5d9e/ : POST_FAILURE in 25m 11s (non-voting) - | |
| 18:46:48 | mordred | gate-dsvm-nodepool-src http://logs.openstack.org/28/434428/1/check/gate-dsvm-nodepool-src/54b5fd4/ : SUCCESS in 36m 54s12:31 | |
| 18:46:50 | mordred | gah | |
| 18:46:52 | mordred | sorry everyone | |
| 18:47:02 | dims | oops :) | |
| 18:47:10 | mordred | diana_clarke: http://logs.openstack.org/04/434404/1/check/gate-shade-dsvm-functional-neutron/2e8773b/console.html#_2017-02-15_17_39_38_245587 | |
| 18:47:30 | mordred | diana_clarke: I'm getting back an empty tenant_usage with novaclient latest microversion | |
| 18:47:45 | mordred | diana_clarke: setting microversion to 2.0 seems to at least fix that test | |
| 18:48:10 | mordred | in any case - not sure if that's useful information or if it's just a thing I should know and deal with as I add in more explicit microversion support | |
| 18:48:13 | mordred | but thought I'd report back | |
| 18:50:01 | mordred | mriedem: is there anything in the new hotness we'd be excited about? | |
| 18:50:55 | mriedem | mordred: new hotness = ocata? we = ? | |
| 18:51:45 | mordred | mriedem: oh - you're learning about ocata in an auditorium at work? I was hoping you were learning what was going to be cool after containers or something | |
| 18:52:03 | mriedem | mordred: oh no, definitely not openstack related :) | |
| 19:26:04 | diana_clarke | mordred: That's worrisome.... if there are instances, you should see usage stats. | |
| 19:27:11 | mordred | diana_clarke: should the payload be there and have 0s in the total slots even with no instances? | |
| 19:27:41 | diana_clarke | mordred: I don't think so. Let me review the code quick. | |
| 19:28:07 | mordred | diana_clarke: oh - also - check it out - it's sending a start and end time | |
| 19:28:47 | mordred | that are only different by microseconds | |
| 19:30:36 | diana_clarke | mordred: nope, no zeros. you have to have at least one instance to get any aggregate totals | |
| 19:31:02 | mordred | diana_clarke: gotcha | |
| 19:33:08 | mordred | diana_clarke: cool - so I can fix our handling of that shade side - also, pinning back to 2.0 for now seems to have fixed the tests - so I'm good now and also know what it wants to be next | |
| 19:33:11 | mordred | thanks! | |
| 19:34:43 | diana_clarke | mordred: The fact that the results seem to differ between microversions is what's worrisome though.... In both cases, I would expect an empty payload. | |
| 19:35:14 | diana_clarke | (if there are indeed no instances in that time window in both cases) | |
| 19:38:29 | mordred | well - there should almost never be instances - since we're passing in start=now(), end=now() ... (it may not be a very good test) | |
| 19:44:12 | mriedem | BobBall: johnthetubaguy: are you still around? question on this xenapi live migration fix | |
| 19:44:53 | mriedem | https://review.openstack.org/#/c/424428/ | |
| 19:45:03 | mriedem | from what i can tell, that will blow up if using XenAPIBridgeDriver | |
| 19:45:38 | mriedem | which is also the default vif_driver | |
| 19:46:21 | diana_clarke | mordred: That doesn't explain how it ever passed though... which is what is nagging me. | |
| 19:46:28 | mordred | diana_clarke: yah | |
| 19:47:02 | mordred | diana_clarke: it definitely passes with version='2.0' - just verified https://review.openstack.org/#/c/434428/ | |
| 19:47:15 | mordred | diana_clarke: so a payload existing with 2.0 seems to be reproducible | |
| 19:47:16 | diana_clarke | mordred: I have a hunch that if you ignore the usage error and fix the other failing 26 tests, that it will pass again. | |
| 19:47:55 | diana_clarke | mordred: Are you printing the payload in the 2.0 case? Does it differ? | |
| 19:48:34 | mordred | diana_clarke: I'm not - we only print that on test failure. I could push up a patch to cause that test to fail so that we can see it in the 2.0 case pretty easily though | |
| 19:50:00 | mriedem | sdague: can you drop the -1 on this ocata backport for rc2? https://review.openstack.org/#/c/432954/ | |
| 19:50:02 | mordred | diana_clarke: https://review.openstack.org/434482 | |
| 19:50:05 | mriedem | my fix is backported on top | |
| 19:50:24 | sdague | mriedem: can we just merge them? or you don't want to for record keeping? | |
| 19:50:45 | mriedem | sdague: i was waiting to approve the bottom patch until the top is approved | |
| 19:50:49 | mriedem | but go ahead and +2 | |
| 19:52:37 | diana_clarke | mordred: My hunch is that instance creation is failing in other tests and that the usage test has only been passing by chance (side effect of there being instances from other tests). | |
| 19:52:54 | diana_clarke | mordred: But, as always I could be wrong. Curious to see that payload. | |
| 19:54:14 | mordred | diana_clarke: me too! | |
| 19:54:48 | diana_clarke | (but now() for the time window also makes me think I'm wrong) | |
| 20:00:45 | mriedem | dansmith: i have to run out for a bit, but can you hit https://review.openstack.org/#/c/434316/ and the one below it for rc2 when you get a chance? | |
| 20:01:03 | dansmith | okay | |