Earlier  
Later
New Search
Posted Nick Remark
#openstack-sdks - 2018-11-16
15:03:24 larsks ...but it's doing that because it doesn't seem to be able to determine the api version for the octavia service, and I'm wondering if someone here has thoughts on what might be going on. The openstack cli works fine.
15:05:37 frickler larsks: I fear lots of folks are still recovering from the summit, I'm off for the weekend now, too. probably best create a story at https://storyboard.openstack.org/#!/project/openstack/openstacksdk , include some error logs and mention the story here
15:06:01 larsks Curses! But sure, I'll do that.
15:06:19 frickler usually mordred might want to take a look, but he also seems to be busy recovering ;)
15:33:03 gtema larsks: what's your endpoint url for octavia? Do you also set/change octavia api version in your clouds.yaml?
15:36:20 gtema UnsupportedVersionWarning should be either created in exceptions.py or replaced with UnsupportedServiceVersion. Not sure what mordred would prefer
15:36:46 larsks gtema: The endpoint url does not hav a version (https://my.public.hostname:13876). I did try setting load_balancer_api_version: 2 in the clouds.yaml file but that didn't appear to resolve the issue.
15:38:04 gtema under "cli" you mean octaviaclient, right?
15:39:06 larsks gtema: I mean "openstack loadbalancer ...", which uses octaviaclient, yes.
15:39:47 gtema ok
15:41:00 johnsom larsks What version of Octavia do you have deployed?
15:42:10 larsks johnsom: 'octavia-api --version' tells me 2.0.2.dev42
15:44:33 johnsom Queens. Ok, you should be good with the API versioning. It's v2. If you do an "openstack versions show" you should see lines like this:
15:44:45 johnsom https://www.irccloud.com/pastebin/uQXXdGAU/
15:46:10 larsks johnsom: ah, there's a problem. The same command here shows: http://termbin.com/8rlw
15:46:45 larsks If I curl the endpointa address I get: http://termbin.com/om4q
15:48:32 johnsom Hmm, yeah, I have more versions because I'm running Rocky, but I'm not sure why they aren't being listed with the queens version that has one v2
15:51:49 larsks johnsom: apparently 'openstack versions' doesn't like the output from octavia? http://termbin.com/bc0i
15:52:11 johnsom Ah, yeah, ok. Queens doesn't have the "links" stuff, so that openstack versions show is correct. It still shows the v2 API though, so you should be good to go. As you saw with the OpenStack client
15:53:40 larsks johnsom: here is the specific behavior I see: http://termbin.com/mlid
15:57:08 amito Hey all, I'm working on the integration for manila (shared file systems) and openstack SDK. It's in progress and I guess a patch will be uploaded sometime in the next few weeks.
15:57:50 gtema amito: cool
15:59:08 gtema larsks: isn't it working further? the proxy should be fine, it seems to just throw a warning
15:59:44 gtema can you add `conn.load_balancer.load_balancers()`
15:59:47 larsks gtema: the os_loadbalancer attempts to call cloud.load_balancer.find_load_balancer, which fails, since that's not a REST pass-through.
16:00:26 gtema larsks: how it fails?
16:00:35 larsks gtema: re: your question, AttributeError: 'Proxy' object has no attribute 'load_balancers'
16:00:45 larsks And the attempt to use find_load_balancer fails the same way.
16:00:47 gtema ok, so empty proxy
16:01:54 johnsom larsks I know a bunch of changes were made to API version discovery around that time in openstacksdk. I would try adding an explicit version "v2.0" to your cloud definition for the load-balancer service.
16:02:24 larsks johnsom: like this? load_balancer_api_version: "v2.0"
16:02:45 johnsom It might be v2 though. load-balancer_api_version I think
16:03:03 johnsom The service type has a hyphen in it
16:03:05 larsks I'm pretty sure that all "-" get convereted to "_" for use as keys, but I can put both in.
16:06:12 larsks johnsom: I've tried this: http://termbin.com/byhkv I've set the api version to "2", "2.0", "v2", and "v2.0", all seem to result in the same error.
16:10:36 gtema larsks: would you try adding `openstack.enable_logging(debug=True, http_debug=True)` into your test to see which requests are being sent? I'm extending SDK with own services, which also do not properly support discovery but I have not faced such behaviour so far
16:11:04 larsks gtema: sure, just a second.
16:13:50 larsks gtema: there are a bunch of "Skipping invalid version data. Missing links" in the debug output: http://termbin.com/tgff
16:20:23 gtema larsks: hmm. The version discovery response is certanly bad. In each version item there should be "links" attribute
16:21:46 gtema larsks: i guess if no response would have been sent - it would be better than that. To tell the truth - I have no clue so far. Parsing of this document is not done in SDK, but keystoneauth1 (I guess)
16:22:33 gtema but I would rather ask - why is this response wrong?
16:22:52 gtema https://wiki.openstack.org/wiki/VersionDiscovery#links
16:22:58 larsks As far as I know, that's just what this version of octavia spits out.
16:23:21 larsks johnsom: ^^^^ is that correct?
16:23:32 gtema sure, but I mean why is it spitting this out
16:24:28 gtema there might be bug in octavia (this version) or configuration. Since I do not have Octavia - I do not know for sure
16:24:45 larsks That I don't know. I just wish there were some way to skip the version discovery and *tell* openstacksdk what to use.
16:25:51 johnsom Well, the version discovery in openstacksdk changed, this is why octavia changed and added the links in a later version. Still, openstacksdk can use Octavia, it's how the CLI and horizon dashboard work. We just have some configuration issue in openstacksdk that is causing it to not be happy for some reason.
16:26:47 johnsom Yeah, there definitely is a way to tell the SDK to use a specific version, but you tried the way I know. Sadly, I haven't spent much time with openstacksdk in over a year.
16:26:56 gtema there is a way, but just temporarily - `openstacksdk:openstack.load_balancer.v2._proxy.Proxy.skip_discovery = True`
16:27:08 gtema but you will not get it landed in SDK
16:28:33 gtema johnsom: the problem is that octavia in this case does not return a valid version discovery doc, so I might try to process this correctly. But the root should be fixed
16:29:05 johnsom gtema It returned a valid discovery doc for the Queens timeframe
16:29:20 johnsom gtema He is running Queens
16:29:55 gtema are you sure links was not mandatory field? I am using Mitaka and services return this attribute
16:30:09 johnsom Since the CLI is working, the SDK works against the queens Octavia API
16:30:26 johnsom Yep, it was not required until later
16:30:44 gtema CLI is working because octaviaclient is not using SDK (I guess)
16:31:41 gtema yesm octaviaclient is not using SDK as of now
16:32:25 johnsom Ah, it was just octavia-dashboard that used the SDK at that time
16:32:59 larsks Bleargh.
16:33:18 larsks Okay, I'm going to punt on this right now and just right my playbook using the openstack cli rather than os_loadbalancer for now.
16:33:38 johnsom We just need monty or someone that knows about that version discovery change to tell us what the configuration magic is for the SDK
16:34:04 larsks I'm going to spend a little time stepping through the code with pdb and see if maybe it's obvious...
16:34:30 gtema I know that you can disable version discovery per service (from the code, not configurable), but I am quite sure for Octavia it will not be switched off that easy
16:34:49 gtema `openstack.load_balancer.v2._proxy.Proxy.skip_discovery = True`
16:35:45 larsks gtema: isn't that at the wrong place? The problem right now is that it doesn't know to use the load_balancer.v2 proxy because of the version discovery issue...
16:36:48 gtema larsks: if the discovery is skipped - it should not even try to get it and return a "proper" proxy
16:37:18 johnsom larsks Try one more thing, looking at the code it bypasses that if both the version and the endpoint of overriden in the config
16:38:15 larsks gtema: setting openstack.load_balancer.v2._proxy.Proxy.skip_discovery=True doesn't seem to impact the issue.
16:38:18 larsks johnsom: let me try that...
16:38:33 johnsom Since you are using the old port way of doing endpoints, append "/v2" to what you have in keystone and try that as an override
16:38:39 larsks Actually, what's the syntax for that...load_balanacer_endpoint?
16:38:50 larsks ...but spelled correctly.
16:39:33 johnsom my "openstack endpoints" gives http://10.21.21.53/load-balancer and uses "http://10.21.21.53/load-balancer/v2" for the sdk endpoint.
16:40:26 larsks johnsom: success!
16:40:36 gtema great
16:41:06 larsks (although apparently octavia lives at /v2.0)
16:41:21 johnsom Yeah, so something might have changed in the newer sdk that broke the old funky stuff that was happening in queens. Monty did a ton of work here to fix the SDK version discovery stuff.
16:41:55 gtema api-ref says /v2.0 would be an alias to /v2
16:42:16 johnsom Yeah, /v2.0 is good and will work
16:42:40 larsks Aaaaand now os_loadbalancer is happy, and I can rewind my stack back a couple of hours and continue :). Thanks johnsom , gtema !
16:43:26 gtema larsks: congrats for "funny" friday ;-)
17:21:18 gtema mordred: have you arrived already?
17:24:50 gtema mordred or anyone else who might know: all latest pathes fail in `bifrost-integration-tinyipa-ubuntu-xenial` with `Unable to establish connection to http://127.0.0.1:5050: HTTPConnectionPool(host='127.0.0.1', port=5050): Max retries exceeded with url`
17:24:59 gtema I was not able to figure out the reason yet
18:18:47 frickler gtema: TheJulia: ^^ that looks like something in ironic may have broken http://logs.openstack.org/98/618198/1/check/bifrost-integration-tinyipa-ubuntu-xenial/14c183a/logs/ironic-inspector.log
#openstack-sdks - 2018-11-17
00:58:18 openstackgerrit Merged openstack/keystoneauth master: Add py36 tox environment https://review.openstack.org/615845
02:52:09 openstackgerrit zhouxinyong proposed openstack/keystoneauth master: Replacing the HTTP protocal with HTTPS in using-sessions.rst. https://review.openstack.org/617811
03:45:33 openstackgerrit Yang JianFeng proposed openstack/openstacksdk master: Add support for octavia's resuorces quota https://review.openstack.org/590193
05:45:08 openstackgerrit Yang JianFeng proposed openstack/openstacksdk master: Add support for octavia's resuorces quota https://review.openstack.org/590193
13:24:30 openstackgerrit Lars Kellogg-Stedman proposed openstack/openstacksdk master: syntax errors and undefined exceptions in service_description.py https://review.openstack.org/618519
16:53:06 isears I'm running python-openstackclient v3.17.0 and I don't seem to have access to the 'stack' sub commands anymore (e.g. openstack stack list). Was support for those commands removed recently or is my install somehow broken?
16:56:36 isears lol nvm I'm an idiot. Had to pip install python-heatclient.
20:13:41 openstackgerrit Merged openstack/openstacksdk master: Advancing the protocal of the website to HTTPS in compute.rst. https://review.openstack.org/617762
#openstack-sdks - 2018-11-18
06:06:44 openstackgerrit Yang JianFeng proposed openstack/openstacksdk master: Add support for octavia's resuorces quota https://review.openstack.org/590193
07:17:01 openstackgerrit Yang JianFeng proposed openstack/openstacksdk master: Add support for octavia's resuorces quota https://review.openstack.org/590193
19:56:40 openstackgerrit Merged openstack/keystoneauth master: Replacing the HTTP protocal with HTTPS in using-sessions.rst. https://review.openstack.org/617811
#openstack-sdks - 2018-11-19
07:22:12 openstackgerrit wangxiyuan proposed openstack/openstacksdk master: Add missing properties for role https://review.openstack.org/614705
07:33:55 openstackgerrit zhufl proposed openstack/openstacksdk master: Add missing seperator between words https://review.openstack.org/618704

Earlier   Later