Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-05-17
17:03:05 levy14 My org found a lot of code that runs in VMs contains credentials in code or config files. I would like to be able to make calls (HTTP requests) to a secret store (Barbican) without providing credentials, and Nova to add them automatically, based on the VM's identity. Much like AWS instance profiles and Azure managed identities work. How should I proceed with this?
17:03:14 levy14 will try to add it to teh agenda for the next time
17:03:29 bauzas levy14: feel free to add it by yourself in https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
17:03:41 bauzas you only need LP creds
17:04:15 levy14 so no need to create a blueprint for it?
17:04:28 bauzas levy14: that's what we'll figure out at the meeting :)
17:04:59 levy14 cool, will add it. if any feedback here, will try to capture that too.
17:05:56 Uggla bauzas, leaving IRC, see you in 1h call me on my cell if needed.
17:06:09 sean-k-mooney levy14: that kind of sound like something that was propsoed before
17:06:12 bauzas Uggla: bien sûr et à toute :)
17:06:32 sean-k-mooney related to jwt tokenes or something like that
17:07:11 levy14 sean-k-moonehy: any idea why it is not yet available? any serious blocker? as we see it as a huge security improvement.
17:07:13 sean-k-mooney levy14: so you would lke to have the ablity to request that a set of barbican securest are automaticly injected wehn a vm is created
17:07:32 sean-k-mooney levy14: well you woudl have to opt into this behavior on a pervm basis
17:07:48 sean-k-mooney and that woudl requrie an api change to request it on boot
17:08:00 sean-k-mooney to say which secret or secrets to inject
17:08:14 levy14 nope, I would like to be able to write code that just calls barbican to get a secret, without having to deal with any credentials. nor inject them.
17:08:18 sean-k-mooney levy14: so basically becacuse no oen has asked for it or step up to do the work
17:08:51 sean-k-mooney wehre would that code run
17:09:00 levy14 in the vm
17:09:06 sean-k-mooney well you can do that today
17:09:07 levy14 in whatever language
17:09:13 sean-k-mooney call barbical from the vm
17:09:28 sean-k-mooney but you would need an applciation credental or simialr in the vm to make that call
17:09:53 sean-k-mooney levy14: nova cant run code in the vm for you by the way
17:10:03 levy14 that is what I want to avoid. people are putting those in code or in config files. or injecting them, without properly handling it. and it's a mess.
17:10:49 levy14 how about not needint that at all. all all https calls going of from a vm to (at least) barbican get added a (temp) access token of the vm's identity
17:11:37 sean-k-mooney that would be a security risk
17:11:49 sean-k-mooney since by default nova is not allows to access the users secrets
17:11:50 levy14 it is not about running code in the vm by nova. it is about intercepting outbound http(s) calls and if the destination is barbican, add the token
17:11:58 sean-k-mooney even admin cannot get them by default
17:12:05 dmendiza[m] 👀
17:12:31 sean-k-mooney levy14: you would need to intercept the request using a man in the midel proxy
17:12:43 sean-k-mooney kind of like the metadta proxy
17:13:06 sean-k-mooney levy14: i think what you want is more like this https://review.opendev.org/c/openstack/keystone/+/784558
17:13:23 levy14 well, can we use the vm's identity (I don't even know if there is such a think in nova)
17:13:30 sean-k-mooney its nto what you asked for but providign a jwt token to the vm via metadta
17:13:49 sean-k-mooney levy14: vms dont have identity's
17:14:12 sean-k-mooney a vm is a resouce that is owned by a project not a user
17:15:01 levy14 ok, is there a good reason why it cannot get an identity? for this specific purpose. to allow the vm to impersonate a user and access secrets?
17:15:39 sean-k-mooney you mean grab the token form the request that was used to boot the vm and then use that to call barbican
17:15:52 levy14 that would be an option, yes
17:16:09 sean-k-mooney well that would be a giant red flag form an auti perspetivce
17:16:19 levy14 although e.g. what aws does is generate short lived temp tokens on request for this purpose
17:17:13 levy14 ut in the end, any solution that allows users to grab secrets from bbarbican with a well-known identity, and without being a security risk, would be fine
17:17:31 levy14 I just want to end this cancer of credentials in code repos
17:18:15 sean-k-mooney usign jwt is proably trhe way to go
17:18:50 sean-k-mooney there was someone looking at extening keystone to supprot just json web tokens to be able to issue a new keystone token
17:19:11 sean-k-mooney and then useing vendor data or nova metadta to provide the token ot the vm
17:19:18 levy14 I am not sure I understand how those would help
17:19:18 sean-k-mooney via the metadtat servie
17:19:51 sean-k-mooney the vm could get a jwt token form the metadata endpoing without auth based in the vm mac and ip via curl
17:20:03 sean-k-mooney and then use that to issue a full keystone token
17:20:09 sean-k-mooney and make requesuts
17:21:12 levy14 and how would the metadata endpoint know what identity to issue a jwt token for?
17:21:45 levy14 I mean where is the ip/mac <-> identity info coming from?
17:22:27 levy14 so that vm creators/users can control it (so that that user gets access to the ecrets the code in the vm will try to fetch)
17:22:42 sean-k-mooney levy14: that comes for nova/neutron
17:23:12 levy14 so in the end nova/neutron does have an identity for each vm?
17:23:19 sean-k-mooney no
17:23:26 sean-k-mooney it used an external service
17:23:47 sean-k-mooney nova suport external dynimc vender data
17:24:13 sean-k-mooney service like nova-join can use that to add extra info into the metadata aviable to the vm
17:25:04 levy14 interesting
17:25:13 sean-k-mooney https://review.opendev.org/q/topic:bp%252Fjson-web-tokens
17:25:19 sean-k-mooney i think that is the keystone feature
17:25:25 sean-k-mooney let me see if i can find any more info
17:25:26 levy14 I am very curious in any solution that gets me closer to this
17:25:31 sean-k-mooney its been a while since this came up
17:25:36 sean-k-mooney like 2 years or more
17:26:50 sean-k-mooney levy14: https://specs.openstack.org/openstack/keystone-specs/specs/keystone/stein/json-web-tokens.html
17:27:43 levy14 thanks for the links, will do some reading
17:27:54 levy14 and add this to the next meeting's agenda
17:29:33 sean-k-mooney levy14: this likely woudl have to be implemted outsdie fo nova. if it was in nova we would need a very detailed spec of how to do this safely
17:29:41 sean-k-mooney just to set expecations
17:30:19 levy14 I understand
17:31:13 levy14 we're a federation of openstack sites, so solutions that require each site to config this would be problematic. I was hoping for a nova-native solution. even if it's going to take a while.
17:31:59 sean-k-mooney levy14: jrosser was asking about this before
17:32:21 sean-k-mooney levy14: just checked my irc logs let me get you a link
17:32:39 sean-k-mooney they did a poc of injecting the token https://paste.opendev.org/show/798996/
17:33:25 sean-k-mooney https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.2020-10-13.log.html#t2020-10-13T15:30:05
17:33:36 sean-k-mooney levy14: it was based on https://cloud.google.com/compute/docs/instances/verifying-instance-identity
17:33:46 jrosser sean-k-mooney: I totally ran out of time to take that further
17:34:18 jrosser but still interesting if it can be made to work with an external thing like step-ca (that was my use case)
17:34:34 sean-k-mooney jrosser: maybe you can expalin it to levy14 and they could find resouce to work on it
17:35:45 sean-k-mooney jrosser: i have not been following keystoen recently but i belive tehy are also working on some oath/openid connect supprot this cycle
17:35:52 sean-k-mooney not sure if that is related
17:36:09 sean-k-mooney this https://review.opendev.org/q/topic:bp%252Foauth2-client-credentials-ext
17:37:01 levy14 in any case, adding it to the agenda and we'll see next time
17:37:11 levy14 thanks for the links
17:42:11 jrosser levy14: I have POC code to insert a signed jwt into the metadata
17:44:17 jrosser https://github.com/bbc/nova/commit/c0e9a98fb96d0dff73d0a5c5e7ebad8078fd85a1
17:45:03 sean-k-mooney jrosser: didnt knwo you worked for/with the bbc
17:45:10 sean-k-mooney or that they used openstack
17:45:18 sean-k-mooney thats cool
17:45:30 jrosser we use it in the R&D labs
17:45:59 sean-k-mooney even still that is nice to hear
17:46:08 sean-k-mooney even if its only for R&D
17:46:12 sean-k-mooney and not production
17:47:30 sean-k-mooney jrosser: your poc is that integrated with keystone in any way. can the jwt token be sued to auth with keystone

Earlier   Later