Earlier  
Later
New Search
Posted Nick Remark
#openstack-sdks - 2019-01-08
14:08:15 mordred uhoh :)
14:09:19 lbragstad that's failing because the user making the request in those tests isn't defined as a system administrator
14:09:35 lbragstad they probably only have a role assignment on the project
14:09:56 lbragstad and in keystone we've changed those policies to require a system admin in order to access the registered limits APIs
14:13:31 lbragstad cmurphy mordred https://review.openstack.org/#/c/621016/ was probably the breaking change
14:17:53 mordred lbragstad: ok. so we should likely update the osc tests for that to use an admin auth context rather than an end-user one
14:21:16 mordred lbragstad: hrm. it looks like osc is already using the devstack-admin account (it's doing domain deletion operations already and stuff)
14:23:22 Shrews mordred: the osc-func-tips job is pretty sad in 380
14:23:48 Shrews non-voting but probably worth looking into?
14:33:46 mordred Shrews: yah - that's the convo with lbragstad
14:34:07 mordred lbragstad: perhaps the devstack-admin account in devstack isn't being granted the system admin role?
14:36:29 Shrews mordred: pfffft... who has time to read scrollback??
14:37:10 lbragstad mordred it should have it
14:37:19 lbragstad keystone-manage bootstrap does that for you
14:37:30 lbragstad which i'm pretty sure devstack executes
14:37:47 lbragstad but, i think the tests need to ask for a system-scoped token instead of a project-scoped one
14:37:54 lbragstad and use that for those requests instead
14:39:17 lbragstad https://developer.openstack.org/api-ref/identity/v3/index.html?expanded=token-authentication-with-scoped-authorization-detail#id30
14:40:19 mordred oh goodie
14:41:34 mordred lbragstad: I honestly have no idea how to get a system-scoped token with osc - or if it's even possible
14:41:47 lbragstad does osc use ksa?
14:42:24 mordred yeah - but I don't know if we've got 'scope' plumbed through in either config or cli args
14:42:39 mordred lbragstad: how does one get a system scoped token with ksa?
14:43:42 lbragstad looking for an example
14:44:57 mordred cool. fwiw, we're still using keystoneclient for keystone things
14:47:49 lbragstad mordred if you set system_scope=all with password authentication, it should work https://docs.openstack.org/keystoneauth/latest/plugin-options.html#password
14:49:05 lbragstad :(
14:49:20 lbragstad we did overhaul some docs to try and make things more clear
14:49:24 lbragstad https://docs.openstack.org/keystone/latest/admin/tokens-overview.html
14:51:25 frickler "openstack --os-system-scope system registered limit create ..." works for me with the devstack-admin user and unsetting all OS_PROJECT/TENANT env
14:52:18 mordred ah - --os-system-scope
14:52:20 mordred neat
14:53:12 frickler would be nice to give a warning when it still does project-scope, instead of silently ignoring the system-scope request in that case
14:54:31 mordred I think it would be better if --os-system-scope overrode the project settings -- unsetting env vars is a bit of a pita
14:54:45 dtantsur lbragstad: this actually helps. I don't think i saw it when I started rust-openstack.
14:56:08 mordred frickler: because the osc test cases are all run with env vars set ...
14:57:37 frickler mordred: yeah, having cli options override other env vars would be neat, too, but not so easy I fear
14:57:43 lbragstad frickler it does.. but since the registered limit and limit apis are still experimental, we didn't deprecate the policies
14:58:03 lbragstad https://gist.github.com/lbragstad/ad744f1a5acb4f4dc3387383514ab33f is an example of defining different scopes in clouds.yaml
14:58:27 lbragstad if you had different users with different roles on the system (one of them being a read-only role)
14:58:33 frickler mordred: maybe setting things like --os-project-name="" to set overrides to the env via cli might work in the meantime
14:59:00 lbragstad dtantsur if you have questions about how all that works, you should bug me :) I'm happy to help
14:59:02 mordred frickler: yeah, I'll try that
14:59:21 dtantsur lbragstad: thanks! I will when I start extending authentication support there
14:59:30 frickler lbragstad: I'm not talking about policies, a simple "openstack --os-system-scope system token issue" will still issue a project-scope token without warning when e.g. OS_TENANT_NAME is set
14:59:44 lbragstad ahh
14:59:53 lbragstad got it - i misread that
15:00:11 cmurphy lbragstad: it seems like this is pointing out an issue in our backwards compatibility?
15:00:37 lbragstad with the policies or with osc?
15:00:48 cmurphy with the policies
15:01:22 lbragstad they are backwards compatible when we deprecate them - due to the logic in the oslo.policy library
15:01:42 lbragstad the only reason we didn't with the registered limit and limit policies is because the api is experimental
15:01:53 cmurphy oh I see
15:02:08 cmurphy I assumed we went through the same process with those
15:02:30 openstackgerrit Monty Taylor proposed openstack/python-openstackclient master: Set system-scope parameter for registered_limits tests https://review.openstack.org/629203
15:03:08 mordred ok - there is a stab ... I'm not sure that'll fix it or not
15:03:12 lbragstad cmurphy wxy brought it up here - https://review.openstack.org/#/c/621016/2
15:03:40 cmurphy got it
15:03:45 mordred what we might need to do is add support for registered_limit in sdk and have the sdk force the system-scope auth regardless of what parameters are otherwise set then update osc to use sdk for these calls
15:04:19 mordred but - let's see if that works first
15:07:48 frickler mordred: doesn't work in my local tests, you cannot use an empty string at that location
15:08:00 mordred blerg
15:08:44 mordred a potentially easier fix would be to update the test runner to not source the env vars but instead to add an --os-cloud command line option to self.openstack in the tests
15:09:18 mordred frickler: do you have the ability to test if --os-project-id='' works if the project-id is coming from clouds.yaml instead of from env vars?
15:09:23 mordred oh - you know what?
15:09:32 frickler mordred: but that sounds ... modern
15:09:45 mordred we could make a new devstack user in clouds.yaml - devstack-system - that would not have project stuff set in teh first place and have system-scoped: all
15:09:55 frickler mordred: also, you could simple use a different cloud ... yeah
15:10:04 mordred I think that sounds like the best bet to me
15:10:19 mordred I'm going to start by changing the func test setup to use --os-cloud instead of env var
15:10:44 mordred and then work on getting an additional clouds.yaml entry for devstack
15:10:51 frickler mordred: the different cloud user should probably be in devstack by default
15:10:54 mordred yah
15:10:57 mordred totally agree
15:11:08 mordred I think it'll be a nice addition to devstack for folks
15:11:54 frickler o.k., I'm away for a bit, I can do reviews later
15:14:07 lbragstad frickler mordred yeah - that's what i do
15:14:18 lbragstad super handy for switching profiles
15:15:24 lbragstad i used it a bunch in a demo, trying to explain scopes https://youtu.be/A1vP6be1U5U?t=353
15:19:05 openstackgerrit Merged openstack/openstacksdk master: Skip block storage v2 functional tests for a minute https://review.openstack.org/629162
15:22:03 mordred gtema: did you see dtantsur's comment on https://review.openstack.org/#/c/621153/ - it came on a +A and was a suggestion for followup, just wanted to make sure you didn't miss it
15:22:41 gtema mordred: yupp, have seen but have a "disaster" at work which I need to take care of
15:22:59 mordred gtema: ugh. "disasters" are the worst. have fun with that
15:23:09 gtema thanks
15:23:22 mordred gtema: is the "disaster" caused by someone else's lack of planning? I really love those
15:24:02 gtema no, it is caused by someone leaving the company and others on a long vacation and suddenly certificates stoped working
15:31:04 mordred gtema: oh double ugh
15:31:19 mordred lbragstad, frickler: remote: https://review.openstack.org/629235 Add a devstack-system cloud for system scoped actions
15:37:35 lbragstad mordred nice, thanks
15:46:06 openstackgerrit Monty Taylor proposed openstack/python-openstackclient master: Use devstack-system cloud for limit tests https://review.openstack.org/629203
15:46:07 openstackgerrit Monty Taylor proposed openstack/python-openstackclient master: Use os-cloud instead of OS env vars for functional tests https://review.openstack.org/629243
15:46:33 mordred lbragstad, frickler: ^^ and there's the OSC update to use clouds.yaml - and then to use devstack-system for the limits tests
15:48:25 mordred mmm. donuts. is there nothing they can't do?
16:01:53 openstackgerrit Bruno HALEBLIAN proposed openstack/openstacksdk master: implement identity v3 Proxy "unassign_project_role_from_user" https://review.openstack.org/629073
16:09:11 dtroyer to temporarily unset an env var
16:52:24 frickler dtroyer: I've been testing whether that works, so the idea sounds plausible, but I'm not sure whether it would be worth the code complexity. maybe a global --ignore-environment option instead? but then, using all these options should be deprecated in favor of --os-cloud anyway, so probably let's just work with the latter
17:07:20 dtroyer frickler: my reasoning for the precedence we have (cli options > env vars > clouds.yaml) is because of exactly this case, where you generally want your environment to contain the "normal" setup and can override on the command line for the exceptions.
17:07:46 dtroyer I'm not sure a global 'ignore env' is right, I might only want to change one or two things, project being the one I run into myself most
17:09:42 mordred dtroyer: I thinkm --no-project would be a great addition
17:10:39 mordred dtroyer: although ultimately I'd like for people to be able to use these limit commands without needing to always set --os-system-scope --no-os-project-id --no-os-project-name --no-domain-id --no-domain-name ... because that just seems like we're abusing people :)

Earlier   Later