| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-07-27 | |||
| 15:09:23 | sdague | each node is waiting for it's own hostname to show up | |
| 15:09:33 | sdague | stack.sh doesn't complete until it has | |
| 15:10:10 | dansmith | and something else waits for stack.sh on all the nodes before we run tempest/ | |
| 15:10:37 | sdague | yes, stack.sh executions are linear | |
| 15:10:43 | dansmith | okay | |
| 15:10:50 | sdague | otherwise tempest would run before services were setup | |
| 15:13:50 | mriedem | dansmith: devstack-gate is waiting for the subnode stacks to be done | |
| 15:13:56 | mriedem | before calling discover_hosts | |
| 15:14:00 | mriedem | so yeah this looks ok | |
| 15:14:03 | dansmith | ack | |
| 15:14:07 | mriedem | http://logs.openstack.org/46/487246/2/experimental/gate-tempest-dsvm-neutron-dvr-ha-multinode-full-ubuntu-xenial-nv/6cd2a5b/logs/devstack-gate-discover-hosts.txt.gz | |
| 15:14:11 | mriedem | ^ is the 3 node job on that change | |
| 15:14:21 | dansmith | I hadn't scrolled right enough to see it was querying its own record, so assumed this was waiting for _a_ compute | |
| 15:15:06 | mriedem | sdague: dansmith: btw, would like to get rid of those ugly ass DEBUG outputs for oslo.concurrency from nova-manage https://review.openstack.org/#/c/487179/ | |
| 15:15:22 | mriedem | ^ is probably backportable if you want me to open a bug | |
| 15:18:07 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add oslo_concurrency=INFO to default log levels for nova-manage https://review.openstack.org/487179 | |
| 15:21:53 | s-dean | Hi, I got past the Cells issue yesterday and managed to list hypervisors and display nodes and services, then i cocked up on install neutron, and now im back to square one. cant list hypervisors | |
| 15:22:13 | s-dean | <class 'oslo_messaging.exceptions.MessagingTimeout'> __call__ /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:1039 | |
| 15:22:32 | mriedem | installing neutron shouldn't do anything with nova | |
| 15:22:47 | s-dean | i swear all services are connected to rabbit so why cant I list hypervisors and services | |
| 15:22:54 | s-dean | MessagingTimeout: Timed out waiting for a reply to message ID e65f06c471cc4e75858f936cf4dff041 | |
| 15:23:06 | s-dean | is there a database that i can check to see the transport URL | |
| 15:23:31 | s-dean | i had to start a fresh again | |
| 15:23:46 | mriedem | s-dean: nova-manage cell_v2 list_cells --verbose | |
| 15:23:55 | mriedem | will dump the db and mq urls for the cell mappings | |
| 15:24:00 | mriedem | which should just be cell0 and cell1 | |
| 15:24:04 | s-dean | ive ran that command | |
| 15:24:09 | s-dean | transport is fine | |
| 15:24:16 | s-dean | to my eye | |
| 15:24:28 | s-dean | hell conductor can even see my compute node | |
| 15:24:36 | mriedem | and the cell1 transport url is the same as the transport url in nova.conf? | |
| 15:24:43 | s-dean | yes | |
| 15:24:57 | s-dean | =INFO REPORT==== 27-Jul-2017::16:17:08 === | |
| 15:24:57 | s-dean | Connection <0.1768.0> (10.30.0.2:32936 -> 10.30.0.2:5672) has a client-provided name: nova-api:4929:dec90db5-2cda-44ea-8ce7-e007fe5280a2 | |
| 15:24:58 | mriedem | are the computes in the nova_api.host_mappings table? | |
| 15:25:08 | s-dean | not yet | |
| 15:25:18 | s-dean | tried to list the hypervisor | |
| 15:25:20 | mriedem | that's probably your problem | |
| 15:25:29 | mriedem | run: nova-manage cell_v2 discover_hosts | |
| 15:25:54 | mriedem | --verbose option on that too | |
| 15:26:09 | s-dean | Found 1 computes in cell: f2852cfb-ae7f-47d0-bef2-d00069bb57ca | |
| 15:26:46 | s-dean | and again ERROR 500 on nova compute api | |
| 15:26:57 | s-dean | just times out | |
| 15:27:10 | s-dean | i can see it connecting to rabbit | |
| 15:28:01 | s-dean | ive had this issue at least 4 times now | |
| 15:28:33 | s-dean | only yesterday, dont know what i did different but it was happy and let me list hypervisors and services | |
| 15:30:33 | s-dean | this is the rabbit mq driver | |
| 15:30:55 | s-dean | oslo_messaging _drivers amqpdriver.py | |
| 15:33:46 | mriedem | does the cell_mapping in the host_mappings table entries match cell1 in the cell_mappings table? | |
| 15:33:54 | mriedem | i.e. are the computes mapped to cell1 properly? | |
| 15:34:21 | sdague | mriedem: was just talking with dhellman, the sitemap fix isn't going to fix the 404s | |
| 15:34:30 | sdague | we're going to need to build a redirect list up ourselves | |
| 15:36:20 | s-dean | in that table, this is what i have | 2017-07-27 14:35:05 | NULL | 1 | 2 | compute01 | | |
| 15:36:22 | s-dean | | 2017-07-27 14:35:05 | NULL | 1 | 2 | compute01 | | |
| 15:36:38 | mriedem | sdague: ok :( | |
| 15:37:50 | mriedem | what are the columns for that? | |
| 15:38:07 | s-dean | created at updated at id cell_id host | |
| 15:38:38 | mriedem | ok so 1 entry | |
| 15:38:42 | s-dean | yeah | |
| 15:38:46 | mriedem | and the id for cell1 in the cell_mappings table is 2? | |
| 15:38:51 | s-dean | yes | |
| 15:39:04 | s-dean | corresponds to cell_mappings table | |
| 15:39:14 | s-dean | cell1 = id 2 | |
| 15:39:28 | mriedem | and this is ocata code or mater (pike)? | |
| 15:39:31 | mriedem | *master | |
| 15:39:36 | s-dean | ocata | |
| 15:40:36 | mriedem | and nova hypervisor-list fails yes? | |
| 15:40:39 | mriedem | does nova service-list work? | |
| 15:41:03 | s-dean | checking | |
| 15:41:37 | s-dean | no | |
| 15:41:50 | s-dean | just hangs | |
| 15:42:27 | s-dean | let me turn tls off and recreate the endpoint with http | |
| 15:43:42 | openstackgerrit | Merged openstack/nova master: Ensure we unshelve in the cell the instance is mapped https://review.openstack.org/486208 | |
| 15:44:03 | s-dean | sdague replied to one of my bugs yesterday saying that i had a rabbitmq misconfiguration, and then out of the blue it worked but now im back to square one. | |
| 15:48:50 | s-dean | where does os-hypervisors | |
| 15:48:56 | s-dean | live in the DB | |
| 15:48:57 | s-dean | ? | |
| 15:52:48 | bauzas | s-dean: compute_nodes | |
| 15:54:10 | mriedem | sdague: bauzas: dansmith: should we get this in before we release novaclient for pike? https://review.openstack.org/#/c/487513/ | |
| 15:54:17 | mriedem | we could also backport since it's latent | |
| 15:54:45 | bauzas | mriedem: dunno, your thoughts ? | |
| 15:55:00 | bauzas | honestly it's for the CLI | |
| 15:55:04 | bauzas | so a backport looks good to me | |
| 15:55:36 | s-dean | ok well that table has data inside it, and references my compute node | |
| 15:57:20 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Allow tuple as for nics value https://review.openstack.org/475816 | |
| 15:58:15 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: doc: Remove Makefile https://review.openstack.org/480096 | |
| 16:01:44 | openstackgerrit | Ed Leafe proposed openstack/nova master: Handle hash ring rebalancing in ironic flavor migration https://review.openstack.org/487925 | |
| 16:02:05 | s-dean | does setting cell type in the nova.conf screw things up ? | |
| 16:02:39 | s-dean | is that option only for V1 | |
| 16:02:40 | mriedem | s-dean: cell type is only for cells v1 | |
| 16:02:45 | s-dean | right ok | |
| 16:02:56 | ildikov | mriedem: I know it's code freeze week, but can you come to the Cinder-Nova meeting? | |
| 16:03:04 | s-dean | thats my issue | |
| 16:03:44 | mriedem | s-dean: you shouldn't touch anything in the [cells] group in nova.conf | |
| 16:04:10 | s-dean | ok, man have cell disabled makes it sound like you should set it to enabled | |
| 16:04:21 | s-dean | but that activates cells v1 right | |
| 16:04:23 | mriedem | the naming is unfortunate and confusing yes | |
| 16:04:30 | s-dean | omg | |
| 16:04:33 | s-dean | alright | |
| 16:04:34 | mriedem | [cells]/enable=True is not what you want | |