| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-20 | |||
| 18:24:26 | sdague | why would the compute node be auto disabled in the middle of a tempest run? | |
| 18:24:38 | mriedem | b/c libvirt crashes | |
| 18:24:43 | mriedem | that's an old gate bug for xenial | |
| 18:24:55 | mriedem | the libvirt driver auto-disables the service when the host connection drops | |
| 18:25:07 | mriedem | http://status.openstack.org/elastic-recheck/#1643911 | |
| 18:25:12 | mriedem | http://status.openstack.org/elastic-recheck/#1646779 | |
| 18:25:19 | mriedem | http://status.openstack.org/elastic-recheck/#1638982 | |
| 18:27:37 | sdague | mriedem: ok, well interesting, the libvirt fail is on a different test where the guests are irrelevant | |
| 18:28:05 | mriedem | but if libvirt fails and disables the only compute service in a single node run... | |
| 18:28:11 | mriedem | it's going to break any test that needs to create a vm right? | |
| 18:28:17 | sdague | it recovers | |
| 18:28:23 | mriedem | yeah eventually | |
| 18:28:29 | sdague | well, actually pretty quick | |
| 18:29:29 | sdague | I think we're building new guests within 12 seconds | |
| 18:29:42 | sdague | only a single tempest test failed in this case | |
| 18:32:26 | mriedem | johnthetubaguy: should we really not expect hostnames on compute services to be unique across cells? | |
| 18:34:52 | johnthetubaguy | mriedem: seems like a bad assumption, given its not the FQDN, its just the hostname, but I might just be paranoid | |
| 18:36:40 | johnthetubaguy | mriedem: given we have a rabbit bus per cell, there is no restriction on that, so far, I might be overthinking that | |
| 18:36:53 | johnthetubaguy | mriedem: I just figured if we started over, all those APIs would just use uuids | |
| 18:36:55 | mriedem | i figured the hostname was fqdn | |
| 18:37:20 | johnthetubaguy | I thought it was conf.host, which isn't usually fqdn | |
| 18:37:21 | mriedem | johnthetubaguy: yeah i don't disagree with starting over, see my latest reply https://review.openstack.org/#/c/447149/1/specs/pike/approved/service-hyper-uuid-in-api.rst@148 | |
| 18:37:51 | mriedem | it is CONF.host | |
| 18:38:28 | mriedem | Hostname, FQDN or IP address of this host. Must be valid within AMQP key. | |
| 18:38:43 | mriedem | so i guess it doesn't have to be fqdn | |
| 18:39:00 | mriedem | sdague: you might have input on ^ | |
| 18:39:14 | mriedem | i don't know why the services actions apis are different from the server actions apis in this regard | |
| 18:39:39 | mriedem | and why they are PUTs rather than POSTs | |
| 18:39:56 | johnthetubaguy | mriedem: yeah, it does all seem a bit crufty in there | |
| 18:40:37 | johnthetubaguy | mriedem: I quite like the idea of adding the uuid in the URL in the new micro-version, to make it more normal | |
| 18:44:05 | mriedem | i can throw it in the weekly api subteam meeting for discussions | |
| 18:46:59 | sdague | mriedem: I'm confused, you pointed at https://github.com/openstack/nova/blob/6f66119153a36ab2ee7d63c98418af3a56fd1f1e/nova/compute/manager.py#L6812 for that logging issue | |
| 18:47:06 | sdague | except, that line is printed fine in the logs | |
| 18:47:31 | mriedem | sdague: it's blowing up trying to process the sub vars for the message | |
| 18:47:53 | mriedem | i'm assuming because event is None | |
| 18:48:02 | sdague | well, are you sure? | |
| 18:48:11 | mriedem | although it shouldn't be | |
| 18:48:17 | sdague | Received unexpected event network-vif-unplugged-8ed92137-a81a-4afd-a391-854848cd2916 for instance | |
| 18:48:18 | mriedem | because if it was https://github.com/openstack/nova/blob/6f66119153a36ab2ee7d63c98418af3a56fd1f1e/nova/compute/manager.py#L6867 | |
| 18:48:35 | mriedem | ah yeah | |
| 18:48:37 | mriedem | my mistake | |
| 18:48:45 | mriedem | when i originally looked at it, i thought it was using _event | |
| 18:50:01 | mriedem | https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/processutils.py#L355 ? | |
| 18:50:03 | sdague | and there are tons of these other places | |
| 18:50:12 | mriedem | https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/processutils.py#L385 | |
| 18:50:29 | sdague | you think that's blowing up? | |
| 18:50:38 | mriedem | "Logged from file processutils.py, line 385" | |
| 18:51:33 | jaypipes | sean-k-mooney, sfinucan, alex_xu, edleafe, Yingxin, moshele: https://review.openstack.org/447643 | |
| 18:53:47 | mriedem | but yeah they are everywhere https://goo.gl/sPpPN0 | |
| 18:55:23 | sdague | mriedem: right | |
| 18:56:51 | mriedem | https://github.com/openstack/oslo.log/blob/master/oslo_log/formatters.py#L273 | |
| 18:57:02 | mriedem | so what sets the log record.args? | |
| 18:57:50 | sdague | most of these stack traces include a daemon.py reference | |
| 18:57:56 | sdague | I wonder if it's this change - https://review.openstack.org/#/c/360252/ | |
| 18:58:43 | mriedem | data['args'] = None | |
| 18:59:25 | sdague | so... yes, it would be that change then | |
| 18:59:38 | sdague | it's one that angus seems to have written over the summer, and dims just merged | |
| 19:00:23 | sdague | I'm going to see what a revert does there in the run | |
| 19:00:23 | mriedem | started using that version 2 weeks ago https://review.openstack.org/#/c/442067/ | |
| 19:00:26 | sdague | yeh | |
| 19:00:28 | dims | sdague : https://review.openstack.org/#/c/447334/ | |
| 19:00:57 | dims | does that help? | |
| 19:01:15 | sdague | dims: I think so | |
| 19:01:41 | mriedem | message:"TypeError: 'NoneType' object is not iterable" AND message:"Logged from file" AND build_change:"447334" | |
| 19:02:38 | sdague | mriedem: yeh, though mostly at debug, so it wouldn't always show there | |
| 19:03:04 | sdague | but, http://logs.openstack.org/34/447334/1/check/gate-tempest-dsvm-neutron-src-oslo.privsep-ubuntu-xenial/c147be6/logs/screen-n-cpu.txt.gz looks clean | |
| 19:06:51 | openstackgerrit | Eric Brown proposed openstack/python-novaclient master: Remove py34 tox env and pypi classifier https://review.openstack.org/442847 | |
| 19:13:50 | wally | Hello, all, i got some question about the Openstack Ocata nova placement api,. | |
| 19:14:22 | wally | I tried installed ocata on ubuntu 1604, and also configure the nova placement api, but still not wokring, | |
| 19:14:31 | wally | can anyone help? | |
| 19:14:56 | wally | there is nothing about this on the installation doc | |
| 19:15:01 | jaypipes | wally: please see /topic. It may be best to send an email to the openstack@ mailing list. | |
| 19:15:17 | jaypipes | wally: also "but still not working" doesn't exactly provide us much information :) | |
| 19:16:02 | wally | on the controller node, i install the apt install nova-placement-api | |
| 19:16:40 | wally | and create the placement endpoint, but when i run the nova-status upgrade check | |
| 19:16:47 | wally | Check: Placement API | Result: Failure | Details: Placement API endpoint not found. | |
| 19:17:03 | jaypipes | wally: did you add the placement endpoint to the Keystone service catalog? | |
| 19:17:09 | wally | i followed this link https://ask.openstack.org/en/question/102256/how-to-configure-placement-service-for-compute-node-on-ocata/ | |
| 19:17:33 | jaypipes | wally: openstack service create --name placement --description "OpenStack Placement" placement | |
| 19:17:46 | wally | yeah i create this as well | |
| 19:17:54 | wally | openstack service create --name placement --description "OpenStack Placement" placement | |
| 19:18:30 | wally | dont know what is wrong? | |
| 19:18:44 | jaypipes | wally: and you did the endpoint create steps as well? | |
| 19:19:08 | wally | openstack service create --name placement --description "OpenStack Placement" placement openstack endpoint create --region RegionOne placement public http://controller:8778 openstack endpoint create --region RegionOne placement admin http://controller:8778 openstack endpoint create --region RegionOne placement internal http://controller:8778 | |
| 19:19:16 | wally | is the enpoint good?>> | |
| 19:19:29 | jaypipes | wally: what does `openstack endpoint list` show? | |
| 19:19:44 | wally | one sec | |
| 19:19:53 | jaypipes | wally: also, please read /topic :) | |
| 19:19:54 | wally | how to paste entire log | |
| 19:20:03 | jaypipes | do not paste entire log. | |
| 19:20:08 | jaypipes | use paste.openstack.org | |
| 19:21:17 | wally | here is it | |
| 19:21:18 | wally | http://paste.openstack.org/show/603483/ | |
| 19:21:32 | wally | @jaypipes, could you please help | |
| 19:21:58 | wally | i followed this doc to do https://ask.openstack.org/en/question/102256/how-to-configure-placement-service-for-compute-node-on-ocata/ | |
| 19:22:33 | mriedem | jroll: what do people use for ironic cli? osc or ironicclient? | |
| 19:22:39 | jroll | mriedem: yes | |
| 19:23:01 | jroll | mriedem: pip install python-ironicclient gets you ironic cli and the osc plugin | |
| 19:23:17 | jroll | we're deprecating the former soon(TM) | |
| 19:23:25 | jaypipes | wally: can you hit http://10.10.10.11:8778 from your compute node? | |