| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-24 | |||
| 18:23:02 | mriedem | cdent might know but he's gone | |
| 18:23:21 | smcginnis | mriedem: If I find anything I'll try to update the docs. Thanks. | |
| 18:23:25 | smcginnis | :q | |
| 18:30:23 | mriedem | sdague: can we get this into stable/ocata to unblock the LM job there? https://review.openstack.org/#/c/458843/ | |
| 18:35:27 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: De-complicate some of the instance delete path https://review.openstack.org/458636 | |
| 18:35:27 | openstackgerrit | Dan Smith proposed openstack/nova master: Make server groups api aware of multiple cells for membership https://review.openstack.org/457338 | |
| 18:35:28 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Move to proper target_cell calling convention https://review.openstack.org/458634 | |
| 18:35:28 | openstackgerrit | Dan Smith proposed openstack/nova master: Make target_cell() yield a new context https://review.openstack.org/458635 | |
| 18:35:29 | openstackgerrit | Dan Smith proposed openstack/nova master: Sort CellMappingList.get_all() for safety https://review.openstack.org/443174 | |
| 18:35:29 | openstackgerrit | Dan Smith proposed openstack/nova master: Clean up ClientRouter debt https://review.openstack.org/444487 | |
| 18:35:30 | openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack https://review.openstack.org/458537 | |
| 18:35:36 | openstackgerrit | Dan Smith proposed openstack/nova master: Add workaround to disable group policy check upcall https://review.openstack.org/442736 | |
| 18:43:41 | smcginnis | mriedem: FYI, Ubuntu packages put the placement stuff under port 8778. | |
| 18:44:01 | smcginnis | mriedem: I seem to have that resolved now, but it's failing to find resource types when updating, etc. | |
| 18:44:23 | smcginnis | mriedem: Is there an installation step that needs to be run again to get resource info populated in placement? | |
| 18:44:45 | mriedem | resource types? | |
| 18:44:52 | mriedem | got an error message? | |
| 18:44:54 | mriedem | or log | |
| 18:45:04 | smcginnis | Failed to retrieve filtered list of resource providers from placement API for filters | |
| 18:45:09 | smcginnis | Resource could not be found. | |
| 18:45:43 | dansmith | mriedem: <3 | |
| 18:45:59 | mriedem | dansmith: that's my way of paying you back for bugging you all morning | |
| 18:46:09 | dansmith | heh | |
| 18:46:45 | mriedem | smcginnis: hmm, you might not have the resource classes, or any resource providers, loaded into placement | |
| 18:46:49 | mriedem | resource provider == compute node | |
| 18:47:04 | mriedem | smcginnis: so now that placement is working, are you seeing things populating it in the nova-compute logs? | |
| 18:47:23 | mriedem | smcginnis: the update_available_resource periodic task in the compute service will update placement | |
| 18:47:27 | mriedem | every 1 min | |
| 18:47:37 | smcginnis | Failed to create resource provider record in placement API for UUID e3c2178a-8fc6-47d7-9ee1-c01a067213aa. Got 404: 404 Not Found | |
| 18:48:44 | mriedem | hmm wtf | |
| 18:49:02 | mriedem | smcginnis: nothing in the placement logs? | |
| 18:50:08 | smcginnis | mriedem: Just a bunch of 404s. | |
| 18:50:19 | mriedem | i don't see where we'd ever return a 404 when trying to POST /resource_providers | |
| 18:50:28 | mriedem | unless you're not making it to placement? | |
| 18:50:39 | mriedem | can you make a curl request against the placement endpoint in the service catalog? | |
| 18:50:57 | mriedem | i wonder if you're getting an endpoint not found from keystone or something | |
| 18:51:10 | smcginnis | I get : {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}} | |
| 18:51:36 | mriedem | ok check your nova.conf [placement] section | |
| 18:51:42 | mriedem | on the compute host | |
| 18:51:58 | mriedem | well, you'll need a token to curl to placement | |
| 18:52:02 | mriedem | did you pass a token? | |
| 18:52:28 | smcginnis | mriedem: No, was just seeing if I could even hit it. | |
| 18:52:55 | mriedem | source the credentials that you have in nova.conf under the [placement] group using the normal OS_* variables | |
| 18:53:04 | mriedem | get a token and pass that to the curl request headers | |
| 18:54:45 | smcginnis | mriedem: Happen to have an example of that curl command line? | |
| 18:54:58 | mriedem | yeah sec | |
| 18:56:29 | mriedem | hmm seems the docs have changed for the api guide and now i've lost my reference | |
| 18:57:17 | mriedem | here we go https://developer.openstack.org/api-guide/quick-start/api-quick-start.html | |
| 18:57:43 | mriedem | OS_TOKEN=$(openstack token issue -c id -f value) | |
| 18:58:01 | mriedem | is what i use to get the token and store into a variable | |
| 18:58:27 | smcginnis | OK, I see X-Auth-Token... | |
| 18:58:34 | mriedem | curl -s -H "X-Auth-Token: $OS_TOKEN" <whatever your placement endpoint is in the service catalog> | |
| 18:59:01 | mriedem | "openstack catalog list" to get the service catalog | |
| 18:59:41 | smcginnis | mriedem: Still 404: The resource could not be found. | |
| 19:00:39 | mriedem | are you trying to hit it on port 8778 ? | |
| 19:01:46 | smcginnis | yes | |
| 19:02:32 | smcginnis | mriedem: I wonder if there is something else wrong with the apache config the Ubuntu package has. | |
| 19:02:37 | mriedem | i assume there is | |
| 19:02:47 | mriedem | but i'm like a baby lamb when it comes to apache config | |
| 19:03:02 | mriedem | but i see cdent snuck back in.... | |
| 19:03:35 | cdent | uh oh | |
| 19:04:00 | cdent | that placement not found business is usually the result of a missing <Directory /path/to/the/wsgi/script> clause | |
| 19:04:03 | smcginnis | cdent: Do you happen to know of issues with the way Ubuntu packages set up placement? | |
| 19:04:22 | smcginnis | cdent: They set up a virtual host on *:8778 | |
| 19:04:45 | cdent | yeah, that's okay as long as the service catalog is right but without the Directory clause the script is not accessible | |
| 19:04:51 | cdent | lemme locate something, one sec | |
| 19:05:15 | smcginnis | cdent: They have WSGIScriptAlias / /usr/bin/nova-placement-api | |
| 19:05:24 | smcginnis | cdent: Then <Directory /usr/bin> | |
| 19:05:37 | cdent | is that inside our outside the virtual host? | |
| 19:05:43 | cdent | s/our/or/ | |
| 19:05:45 | smcginnis | Inside. | |
| 19:06:02 | cdent | try moving it outside | |
| 19:06:32 | cdent | I seem to recall that being one of several fixes (there's been a slew of bugs and changes related to this and I don't think the actually released package has caught up) | |
| 19:06:55 | mriedem | https://docs.openstack.org/ocata/install-guide-ubuntu/nova-controller-install.html doesn't mention anything about known issues | |
| 19:07:09 | mriedem | or apache config at all, i guess because the install guide relies on the packages being ok | |
| 19:07:27 | cdent | mriedem: exactly, there's been some gaps | |
| 19:08:13 | smcginnis | Here's the full current Apache config that the package includes if you're interested: http://paste.openstack.org/show/607726/ | |
| 19:08:52 | cdent | yeah, other than the directory clause needing to be top level that looks okay | |
| 19:09:40 | mriedem | i only see 2 reported bugs for placement in the nova ubuntu packges on LP | |
| 19:09:47 | mriedem | https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1671509 | |
| 19:09:47 | openstack | Launchpad bug 1671509 in Ubuntu Cloud Archive "placement api configuration in apache configure CustomLog incorrectly" [High,Fix committed] - Assigned to Corey Bryant (corey.bryant) | |
| 19:09:53 | mriedem | https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1674465 | |
| 19:09:54 | openstack | Launchpad bug 1674465 in nova (Ubuntu) "wsgi scripts shouldn't grant on /usr/bin" [Undecided,Confirmed] | |
| 19:10:09 | mriedem | neither of those sound like what you're hitting | |
| 19:11:10 | cdent | strange I swear I had talked to corey about this, but I suppose it could have been one of the rdo people as they have had similar problems | |
| 19:11:12 | smcginnis | Nope, still getting 404's after moving the Directory piece out of the VirtualServer. | |
| 19:11:25 | mriedem | smcginnis: did you restart it? | |
| 19:11:30 | smcginnis | s/VirtualServer/VirtualHost/ | |
| 19:11:40 | smcginnis | yeah, restarted apache, then tried restarting the host. | |
| 19:14:05 | cdent | smcginnis: what's your curl? | |
| 19:15:48 | smcginnis | curl -s -H "X-Auth-Token: $OS_TOKEN" http://192.168.1.230:8778/placement | |
| 19:16:02 | cdent | smcginnis: oh, yeah, drop the 'placement' | |
| 19:16:08 | cdent | when it is on a port, there's no prefix | |
| 19:16:26 | cdent | or change the WSGIScriptAlias from / to /placement | |
| 19:16:37 | smcginnis | Shernuf: {"versions": [{"min_version": "1.0", "max_version": "1.4", "id": "v1.0"}]} | |
| 19:17:14 | cdent | there's a docs bug which discusses that the service catalog entry is wrong when on a port | |
| 19:17:15 | smcginnis | cdent: So if I just change my catalog endpoint to not include /placement, are there going to be an issues with code expecting that in the path? | |
| 19:17:17 | cdent | lemme see if I can find that | |
| 19:17:43 | cdent | the code always uses the service catalog, so should be no problems | |
| 19:17:43 | mriedem | smcginnis: we don't put that in code anywhere | |