Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-05-22
16:34:27 ayoung the reason for that is, as alaski and I discussed in his comments, we built a hack into oslo-context
16:34:40 ayoung sdague, it only takes effect once the config option is actively set in Keystone
16:35:09 ayoung at that point, only users with roles on the project that Keystone has configured as the admin_project would be able to execute these APIs
16:35:32 ayoung We need the change in Nova, Glance, Cinder, Neutron, and Keystone before the fix enforces policy across the board
16:35:33 sdague ayoung: sure, I guess I don't understand why Nova should ever know that big
16:35:38 sdague that bit
16:35:43 sdague like, that's the thing I don't get
16:35:55 sdague there are multiple ways now for a keystone setup to say "you are admin"
16:36:01 sdague so, that's fine
16:36:12 ayoung sdague Nova is the policy enforcement point, not Keystone
16:36:15 sdague why doesn't keystonemiddleware / oslo.context process that
16:36:27 sdague and just set the is_admin flag based on that logic
16:36:31 ayoung so....
16:36:48 ayoung Did you go to my RBAC in middleware talk?
16:36:50 sdague it doesn't seem useful to have to teach every service what the current keystone configuration is to do that
16:36:53 sdague nope
16:37:45 ayoung sdague, ok, so the conccet there is that RBAC could be enforced on just the URL, as it does not need to know the scope of the object out of the database. Unlike, say, neutron netwokrs, or glance images, which are project scoped. The role check is only on the URL
16:37:53 ayoung is_admin_project is a scope check.
16:38:15 ayoung and, for all the listed resources, we need to fetch an object our of the database first
16:38:39 ayoung OK...you say...but these APIs are not scoped, so we don't need an object from the database, right?
16:38:45 ayoung and...I agree with you
16:38:59 ayoung but we do not have enough information to determine what policy rule we need to enforce
16:39:09 ayoung only the policy.json file knows that, as of now
16:39:56 sdague ayoung: honestly, I think I need worked examples showing the set of cli flows for it to make sense. The policy bits get really abstract really quickly. And while I realize you have most of it your head, I can't really do that without pictures and flows
16:39:56 ayoung In the "RBAC in middleware" proposal, we specify an inventory of routes, with a role assigned to each. The routes look a lot like the Compute API doc
16:40:35 dansmith jaypipes: feel free to cc me directly on that mail if you want to further implicate me in the act of bring it back up
16:40:40 ayoung sdague, the short answer to your question is, without additional policy enforcement, we need to know, at policy time, whether a specific token is supposed to be global scoped or not
16:41:31 ayoung forget my RBAC proposal, cus that is not approved yet. JUst to get a fix for bug 968696, we need a way for Nova to say "yes, this token is good for global operations" or "no it is not"
16:41:31 openstack bug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] https://launchpad.net/bugs/968696 - Assigned to Adam Young (ayoung)
16:41:58 ayoung sdague, determining which operations are global vs scoped is a decision for the Nova engineering team to make
16:42:08 ayoung as is figuring out how to enforce the scoping
16:42:22 ayoung what Keystione is providing is the answer to "is this token valid for global operations"
16:42:44 ayoung this patch is using that answer to start differentiating between scoped and global operationws
16:42:57 ayoung could it be done in middleware? Not today
16:43:03 ayoung Not without a lot more tooling.
16:43:35 sdague ayoung: so, I'll repeat, this isn't going to make any sense to me without those other diagrams and flows. So I'll stay opted out of the conversation if those don't exist.
16:43:54 ayoung sdague, huh?
16:44:08 ayoung I...I don't think I can make it any more simple than that.
16:44:22 ayoung In order to perform global admin operations, you need a token scoped to the admin project.
16:46:18 openstackgerrit Sylvain Bauza proposed openstack/nova master: Scheduler returning alternatives per destination https://review.openstack.org/465175
16:46:18 openstackgerrit Sylvain Bauza proposed openstack/nova master: Delete existing allocations when retrying https://review.openstack.org/460433
16:46:19 openstackgerrit Sylvain Bauza proposed openstack/nova master: Don't delete allocation if instance being scheduled https://review.openstack.org/460177
16:46:19 openstackgerrit Sylvain Bauza proposed openstack/nova master: WIP: Allocate instance by the conductor https://review.openstack.org/460455
16:46:20 openstackgerrit Sylvain Bauza proposed openstack/nova master: Fix ChanceScheduler returning a wrong node https://review.openstack.org/466725
16:54:23 ayoung sdague, what exactly would you require in order to move this forward? Considering that a comparable change needs to go into every openstack project, I'm guessing that whatever we provided for the Nova team would help move it forward across the board. The current implementations are really broken, due to Keystone not having provided a way to distinguish between global and scoped tokens. But that mistake goes back to pre-Ke
16:54:23 ayoung ystone Nova's token system as well.
16:54:42 ayoung Just because it has been broken since Cactus does not mean it is not broken.
17:04:09 bauzas jaypipes: mriedem: dansmith: cdent: melwitt: edleafe: FWIW, given the outcome of our hangout, I'm going to stop working on my series, in particular given I'll need to disappear starting tomorrow
17:04:23 bauzas edleafe: feel free to takeover any change you want
17:04:57 bauzas edleafe: I just removed my -2s that I gave you previously
17:05:09 bauzas tl;dr: all is clear for iterating fast
17:05:32 bauzas I'll spend my day tomorrow for bug triaging and reviews
17:09:21 ayoung sdague, here is my RBAC talk. https://www.openstack.org/videos/boston-2017/per-api-role-based-access-control please watch it when you have the time. In the meanwhile, I will try to put together the explanation of why we are approach the fix to 968696 the way we are.
17:15:24 bauzas mriedem: melwitt: do you want me to publish a new rev for the ReqSpec delete where I would call the deletion in https://review.openstack.org/#/c/391060/9/nova/compute/api.py@2046 for example ?
17:15:33 bauzas I can do that quickly so you both could review
17:16:24 cdent bauzas: ack on the all clear for iterating
17:16:57 mriedem bauzas: i don't think that's the place to do it
17:17:05 mriedem _local_delete is the place to do it in the api
17:17:20 bauzas mriedem: I saw you wanted to restrict for local deletes
17:17:20 mriedem bauzas: but i won't be reviewing anything else for awhile, i'm on patch 1 of the quota series
17:17:37 mriedem bauzas: the _local_delete method is where local delete happens
17:17:41 bauzas mriedem: like I said in the comment, I think it's pretty harmless
17:18:08 bauzas mriedem: well, I think we would loose a huge number of deletions if we only do that for local instance deletes
17:18:15 bauzas right?
17:18:16 melwitt bauzas: I didn't know about the on-the-fly reqspec it can come up with if it doesn't find one, so it's not catastrophic in the case we described
17:19:36 melwitt it would still be something like "user can see and do actions on instance but instance may not honor original scheduling spec"
17:19:57 mriedem we are still going to need something to cleanup api db orphans, like instance_mappings
17:20:05 bauzas melwitt: eg. https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L294
17:20:11 mriedem idk, i can't context switch on this right now
17:20:53 bauzas okay, let's postpone that for later
17:20:58 bauzas it's a bugfix, we can land it later
17:21:15 bauzas I'll review jaypipes's series for shared-rps
17:36:17 edmondsw johnthetubaguy can you point me to the patch you mentioned at the summit to swap network routing before pause/resume on live migrate?
17:44:43 mriedem melwitt: just so you're aware, i posted comments in the base patch but i'm still working through the rest
17:45:46 gyee mriedem: can you please take a look whenever you have a chance? https://review.openstack.org/#/c/466143/
17:45:48 melwitt mriedem: cool, thanks
17:46:24 openstackgerrit Steve Noyes proposed openstack/nova master: WIP: update live migration to use v3 cinder api https://review.openstack.org/463987
17:54:40 jaypipes mriedem, dansmith, melwitt, cdent, bauzas: ok, ML post sent about getting rid of automated reschedule behaviour.
17:55:15 dansmith ack
17:56:35 jaypipes mriedem, dansmith, melwitt, cdent, bauzas: pls let me know if I accurately explained the issue and the request.
17:56:48 melwitt out of curiosity I checked how kubernetes scheduling works and whether it retries, to be aware of what others are doing,
17:57:22 melwitt and it sounds like it does retry forever "remains in pending state until suitable machine becomes available" according to this post. I didn't find a doc about it on k8s site yet https://stackoverflow.com/questions/28857993/how-does-kubernetes-scheduler-work/28874577#28874577
17:57:41 jaypipes melwitt: well, yeah, because k8s is an orchestrator :)
17:57:56 dansmith yeah, totally different
17:58:06 jaypipes melwitt: which fits nicely into my "nova shouldn't be an orchestrator" point ;)
17:58:28 melwitt but openstack doesn't have an orchestrator at all, does it?
17:58:57 melwitt so our thing just performs worse
17:59:25 jaypipes melwitt: heat?
17:59:27 dansmith heat
17:59:28 dansmith heh
17:59:36 jaypipes melwitt: or tacker for NFV.
18:00:45 melwitt I think heat is a manual orchestrator, you (the end user) have to tell it everything to do, I thought
18:01:28 jaypipes melwitt: well, k8s is the same. you declare a pod definition and k8s tries constantly to make your pod definition a reality.
18:01:51 jaypipes melwitt: or you just meant heat doesn't constantly try to do things?
18:03:04 melwitt jaypipes: more the latter. I'm not a heat expert, maybe there's a way to tell it to keep retrying a definition
18:03:29 jaypipes melwitt: yeah, I'm not a Heat expert either.
18:17:08 cdent jaypipes, melwitt: if want to play the odds, it is more likely that a "keep retrying orchestrator" (that actually does the job well) will come into existence if we do not include bad orchestration in nova.
18:17:36 jaypipes cdent: ++
18:17:54 cdent it is also the case that the odds of "good orchestration" merging sooner (in nova) than something else trying to do it (outside of nova's constraints) are low
18:20:43 melwitt cdent: maybe. I'm not sure how many people would want to use an orchestrator-less system, but that works as long as no other viable options are available

Earlier   Later