| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-sdks - 2019-11-01 | |||
| 15:11:31 | efried | cool, ksa is a wrapper around requests that handles the auth and service catalog pieces. And sdk is a wrapper around ksa that lets you use a yaml file for config. | |
| 15:12:04 | efried | (sdk does much more than that obviously for other services, but not yet for placement) | |
| 15:12:38 | umbSublime | Ohhh I see now, so I could just use ksa and at least get the same interface as conn.compute.<HTTP METHOD> instead of re-implementing it ! | |
| 15:13:41 | efried | instead of re-implementing the auth and service catalog pieces, yes | |
| 15:13:59 | efried | umbSublime: Here's an example of oslo.config style setup to talk to placement: https://zuul.opendev.org/t/openstack/build/8d219a6777234f349caf02883b9e0a4d/log/logs/etc/nova/nova.conf.txt.gz#104-112 | |
| 15:14:14 | efried | (from a random CI run on a random recent nova patch) | |
| 15:17:02 | efried | umbSublime: So if you wanted to go that route, you could | |
| 15:17:02 | efried | - steal nova's get_ksa_adapter https://opendev.org/openstack/nova/src/branch/master/nova/utils.py#L969 | |
| 15:17:02 | efried | - register those conf opts like this https://opendev.org/openstack/nova/src/branch/master/nova/conf/placement.py#L19-L30 | |
| 15:17:02 | efried | - set up a .conf file like that one | |
| 15:17:02 | efried | and then you should be able to do: | |
| 15:17:02 | efried | adap = get_ksa_adapter('placement') | |
| 15:17:02 | efried | adap.get(...) etc. | |
| 15:19:29 | efried | I would be interested to know more about how the sdk is failing though. I'm not sure "Skipping over invalid version data. No stability status in version." should actually be preventing sdk from creating a proxy for you... | |
| 15:20:31 | efried | However, that ^ probably means that queens placement wasn't producing a good version document. cdent can you corroborate? | |
| 15:20:52 | umbSublime | I can paste the full stacktrace. It's something about parsing the endpoint | |
| 15:21:13 | cdent | efried, umbSublime : it's likely it was not, at that time | |
| 15:21:36 | cdent | because there was never any plan to have other versions, so for a while it was not a list, if I remember right | |
| 15:21:38 | umbSublime | https://hastebin.com/qezivuxivu.sql | |
| 15:21:49 | cdent | yeah, that | |
| 15:22:21 | umbSublime | Is there a version of the SDK where this would work though ? (prior to train release maybe?) | |
| 15:22:33 | cdent | umbSublime: can we wind this back a bit more: what's your end goal? | |
| 15:22:52 | umbSublime | We have some issues with allocations that don't exist | |
| 15:23:05 | umbSublime | I'm just making a little tool to help show and clean them | |
| 15:23:37 | cdent | ah, okay, yeah, that's a common thing, enough that in newer versions nova has some tooling for it | |
| 15:23:45 | umbSublime | I have all my code working right now, and the tool works. I'm just trying to make it more "standard" I guess | |
| 15:24:24 | cdent | a good inclination but I'd say probably not worth it in this case | |
| 15:24:41 | cdent | no "standard" was set with regard to placement around queens time | |
| 15:24:53 | umbSublime | agreed, it was also a bit of an exercise to get more familiar with the SDK | |
| 15:25:11 | efried | unfortunate that you tried with this particular combination :( | |
| 15:25:30 | efried | We didn't have sdk talking to placement until mid-train, and iirc we had to make some tweaks to sdk to make that happen more smoothly. | |
| 15:25:38 | umbSublime | I appreciate all your help though. efried I think I'll give that technique you shared a try | |
| 15:26:02 | cdent | umbSublime: you can run placement standalone, if you want something to talk to using modern techniques, for educational purposes | |
| 15:26:40 | umbSublime | Ohh you mean on a devstack or something ? | |
| 15:27:01 | cdent | no, much more standalone, let me find you a ref | |
| 15:27:41 | cdent | https://docs.openstack.org/placement/latest/contributor/quick-dev.html as well as https://github.com/cdent/placedock | |
| 15:27:50 | umbSublime | One day I'll try and get more involved into dev in openstack currently I'm just an operator trying to make my life easier :) | |
| 15:28:19 | cdent | that would be great, openstack badly needs more people with field experience working on it to make it easier to manage | |
| 15:29:56 | umbSublime | to be honest I was wondering is there is some type of portal where openstack operators (which don't necesseraly have all the dev experience) could share and discuss some future features/bugs etc. | |
| 15:31:56 | cdent | the openstack-discuss mailing list is supposed to be, in part, for that | |
| 15:34:39 | umbSublime | What is the actual goal the SDK is trying to fill (not historically, but nowadays) ? Is it meant to be a one stop shop for os services needing to talk to another ? Or is it more dev/ops oriented to standardize talking to all the different APIs | |
| 15:35:46 | mriedem | umbSublime: #openstack-operators channel | |
| 15:35:52 | mriedem | though it's generally quiet | |
| 15:35:58 | efried | umbSublime: That's really a mordred (sitting shirtless on a mountain with supplicants bringing him burnt offerings) question. | |
| 15:36:11 | efried | but I'll give it a shot... | |
| 15:36:25 | mriedem | umbSublime: if you send to the ML, tag ops stuff with [ops] to filter properly in people's mailbox | |
| 15:37:01 | efried | Yes, one stop shop. Originally I think for cloud operators writing tooling that talks to their cloud. Yes standardize how we talk to the different APIs. | |
| 15:37:25 | efried | Only more recently have we started to want to use it as a way for one service to talk to another. | |
| 15:37:34 | umbSublime | hence the merge of shade into the sdk in the bigining | |
| 15:38:12 | efried | We made some inroads on that in nova in train: we're most of the way through replacing nova's use of python-ironicclient with sdk; and we already swapped out placement (which had been using ksa directly) as you saw. | |
| 15:54:30 | umbSublime | efried: One last question concerning using get_ksa_adapter. Here https://opendev.org/openstack/nova/src/branch/master/nova/conf/placement.py#L28 what is conf ? I know it's what you showed with the .ini style config. But what type of object do I use to generate it ? | |
| 15:59:24 | efried | umbSublime: it's something like: | |
| 15:59:25 | efried | from oslo_config import cfg | |
| 15:59:25 | efried | register_ksa_options(..) # This should register the opts onto cfg.CONF | |
| 15:59:25 | efried | cfg.CONF(args=['--conf-file', '/path/to/your.conf']) # This loads up your config file into cfg.CONF | |
| 15:59:25 | efried | get_ksa_adapter... | |
| 15:59:36 | efried | gotta run to a meeting, that ^ is a bit rough but hopefully gets you pointed in the right direction. | |
| 16:00:19 | umbSublime | efried: thanks that's exactly what I need :) | |
| 16:16:40 | openstackgerrit | Pedro Henrique Pereira Martins proposed openstack/keystoneauth master: Fixes OIDC authentication with multiple IdPs https://review.opendev.org/692140 | |
| 17:25:57 | openstackgerrit | Merged openstack/python-openstackclient master: versions: Fix 'versions show' help message https://review.opendev.org/686408 | |
| 17:29:13 | umbSublime | I must be missing something https://hastebin.com/utuhuyinur.python I don't see what could be a duplicate. I'm trying to dig in the code, but i'm a bit lost | |
| 17:35:27 | efried | umbSublime: You're just running this code in isolation? | |
| 17:38:09 | openstackgerrit | Pierre Prinetti proposed openstack/python-openstackclient master: Fix 'versions show' helps & docs https://review.opendev.org/691989 | |
| 17:38:15 | efried | umbSublime: It looks right to me, but I was able to repro locally. Investigating... | |
| 17:40:37 | umbSublime | efried: yes, only the code in hastebin in a venv | |
| 17:43:26 | efried | umbSublime: oh, I see what's happening. I didn't notice you were importing from nova. That import is already registering the conf opts via nova.conf.__init__. | |
| 17:43:58 | efried | If you want to be able to use this cleanly, I suggest just copying in the guts of register_ksa_options | |
| 17:44:08 | efried | ...so you don't need to rely on nova existing at all. | |
| 17:44:17 | umbSublime | Ohh I see what you mean. | |
| 17:45:08 | efried | umbSublime: Um, except you can simplify a lot further | |
| 17:45:21 | efried | you don't need to remove and re-add the _ADAPTER_VERSION_OPTS for your purposes. | |
| 17:45:38 | efried | though I guess you can, shrug. | |
| 17:46:05 | umbSublime | same for get_ksa_adapter (I just use the guts of the function) ? | |
| 17:48:40 | openstackgerrit | Pierre Prinetti proposed openstack/python-openstackclient master: Fix 'versions show' helps & docs https://review.opendev.org/691989 | |
| 17:50:14 | openstackgerrit | Pierre Prinetti proposed openstack/python-openstackclient master: Fix 'versions show' helps & docs https://review.opendev.org/691989 | |
| 18:04:15 | umbSublime | got it working. Thanks again efried ! | |
| 18:04:24 | efried | \o/ | |
| 18:13:47 | umbSublime | Here is what I end up with https://hastebin.com/huhajekiso.py | |
| 18:15:48 | efried | umbSublime: Some of that stuff is nova-specific and/or legacy gorp you're dragging along. If this is for more than just playing around, I would suggest simplifying. | |
| 18:16:59 | umbSublime | efried: yup. I'm doing that right now :) but being unfamiliar with the codebase i'm taking my time :) | |
| 18:17:01 | efried | but yeah, this looks good. | |
| 18:17:12 | efried | umbSublime: okay, let me know if you need help | |
| 18:17:31 | efried | for example, if this is always going to be a placement-specific tool, you can remove all the stuff about service types and just hardcode 'placement'. | |
| 18:17:51 | efried | and as I said before, you probably don't care to scrub those version opts for your purposes. | |
| 18:17:57 | umbSublime | Well I'm baking this into my more general purpuse lib module, so I'll try and keep it generic | |
| 18:18:05 | umbSublime | just in case I need it for another service at some point | |
| 18:18:14 | efried | mmkay | |
| 18:18:30 | umbSublime | efried: I tried without and I get the same stack-trace as when using the nova register_ksa_opts | |
| 18:18:52 | efried | you would need to remove that hackup from both places | |
| 18:19:43 | efried | so 1) get_ksa_adapter_opts can go away and just use ks_loading.get_adapter_opts directly; and 2) kill L36-8 | |
| 18:20:17 | umbSublime | Ohhh I see, remove lines for "Removing version opts" and the ones that "add dummy opts for version" | |
| 18:20:47 | efried | actually, it gets simpler. | |
| 18:21:32 | efried | I only did this conf.register(get_opts) thing because I had to hack out those version things. | |
| 18:22:07 | efried | There ought to be a ks_loading method like ks_loading.register_adapter_conf_options which you can call directly. | |
| 18:22:54 | umbSublime | Ok, I'll look into that | |
| 18:30:48 | efried | umbSublime: like this http://paste.openstack.org/show/785725/ | |
| 18:37:58 | umbSublime | For some reason i'm getting "keystoneauth1.exceptions.catalog.EndpointNotFound: Could not find requested endpoint in Service Catalog." with that code (adapted to my env of course) | |
| 18:39:14 | efried | add to your conf: | |
| 18:39:15 | efried | service_type = placement | |
| 18:39:39 | efried | I simplified out the bit that detects that :P | |