| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-03-06 | |||
| 13:09:39 | sean-k-mooney | am sure | |
| 13:23:51 | stblatzheim | thank you sean-k-mooney :) | |
| 15:32:30 | dansmith | melwitt: yeah so without the creator role (or whatever) in barbican, I can't boot.. and nova dumps a double-tall stack trace into the logs | |
| 15:33:00 | dansmith | especially since that is likely to be a very common thing, surely we should catch and LOG.error that big mess instead right? | |
| 15:33:45 | dansmith | it also seems like it's being handled as a "meh try it elsewhere" sort of thing because it hits NoHost, but this should, AFAICT, be a fatal do-not-retry sort of thing right? no reason to try it on three hosts when it will just fail everywhere | |
| 15:40:18 | dansmith | from the outside, I don't think I can see anything about why this failed other than "not enough hosts" | |
| 15:41:46 | dansmith | maybe we should have a "set up encryption" action event or something so we can see that's what actually failed? | |
| 15:50:14 | opendevreview | Vlad Gusev proposed openstack/nova stable/zed: Unbind port when offloading a shelved instance https://review.opendev.org/c/openstack/nova/+/908694 | |
| 16:00:02 | melwitt | dansmith: yeah, I agree that should be handled better. I'll put this on a list of changes to make. an instance action event is an interesting idea | |
| 16:00:31 | dansmith | cool | |
| 16:57:58 | melwitt | sean-k-mooney: this is how vtpm secrets get created, it's pretty similar https://github.com/openstack/nova/blob/master/nova/crypto.py#L171 | |
| 17:24:40 | dansmith | melwitt: just to restate what I said with more context: | |
| 17:25:08 | dansmith | I think the exception addition stuff should be as much before the create patch as possible because the create patch is already still bordering on overweight (like me) | |
| 17:25:16 | dansmith | so adding fixes to previous stuff in there just makes it bigger | |
| 17:26:22 | melwitt | dansmith: yeah, I agree that makes sense | |
| 17:27:21 | sean-k-mooney | by the way the reason i was testing with admin demo beside the fact i was creating flavors was mainly because i tought tempst was testing with a normal user so i really was not expecting the creator role requirement | |
| 17:28:59 | sean-k-mooney | i wont have time to test this again for likely a week or two but next time i test it ill try an swap beteen users betwen admin and non admin actions | |
| 17:30:33 | melwitt | sean-k-mooney: this is the thing I was trying to say about the tempest roles https://github.com/openstack/barbican/blob/master/devstack/lib/tempest | |
| 17:50:18 | melwitt | oh, interesting. the creator role requirement is a deprecated "legacy" policy https://github.com/openstack/barbican/blob/master/barbican/common/policies/secrets.py#L50 and the current policy only requires member. I'm checking why is it that devstack is defaulting to the deprecated "legacy" thing. I guess for compat with older stuff | |
| 17:57:49 | melwitt | here's where it defaults to ENFORCE_SCOPE = False https://github.com/openstack/devstack/blob/5e837d1f0d9078c58bc634474a1adf311bc2b491/stackrc#L175 so if we set that to true before ./stack.sh then it would not require the creator role | |
| 17:58:44 | melwitt | either way, we have to handle it gracefully but I didn't realize that the creator role thing was (thankfully) changed in the past | |
| 18:02:54 | dansmith | yeah, definitely still want to handle it, but good if there's not going to be that hoop for admins to have to do | |
| 18:04:23 | melwitt | ++ | |
| 18:44:10 | sean-k-mooney | bauzas: i replied to your comment in https://review.opendev.org/c/openstack/nova/+/902687/4 by the way regarding the lock | |
| 19:17:00 | dansmith | sean-k-mooney: melwitt confirmed with TPM: https://paste.openstack.org/show/bYK59AaAMr67tn9p56zP/ | |
| 19:17:24 | sean-k-mooney | ack so that only works today because scope enforcement is disabled | |
| 19:17:29 | sean-k-mooney | right | |
| 19:17:53 | sean-k-mooney | well that or users need to have the creator role | |
| 19:18:13 | melwitt | dansmith: ok, cool, thanks | |
| 19:18:18 | sean-k-mooney | oh re reading melwitt comment | |
| 19:18:35 | sean-k-mooney | so createor shoudl not be required now and member shoudl be enouch with new policy is that correct | |
| 19:18:42 | melwitt | sean-k-mooney: yeah it's if scope enforcement is disabled, you need the creator role. if scope enforcement is enabled, you only need the member role | |
| 19:19:02 | sean-k-mooney | ok so that also explains why this worked when we tested it with the new isntaller | |
| 19:19:10 | sean-k-mooney | we are already enabeld the new SRBAC stuff there | |
| 19:19:12 | melwitt | and scope enforcement is enabled by default. it's just in devstack it defaults to False | |
| 19:19:29 | sean-k-mooney | ack | |
| 19:19:58 | melwitt | ah, yeah | |
| 19:20:02 | melwitt | yeah, I'll add these details to the spec too in regards to the role, like when it's needed | |
| 19:20:33 | sean-k-mooney | cool gibi manually checked vtpm when we were adding barbican supprot. weill have automatic testing in the next few weeks | |
| 19:21:06 | sean-k-mooney | but as i said we are using the new default and i dont know if he tested with admin but it worked there | |
| 19:21:26 | sean-k-mooney | our downstrema docs for vtpm dont mention this as far as im aware but ill check quickly | |
| 19:22:22 | sean-k-mooney | https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17.1/html/configuring_the_compute_service_for_instance_creation/assembly_configuring-instance-security_vgpu#assembly_configuring-compute-nodes-to-provide-emulated-TPM-devices-for-instances_TPM | |
| 19:22:49 | sean-k-mooney | ya so we dont which makes me think that creator has not been created for a while | |
| 19:23:52 | sean-k-mooney | well that or there are two posiblities i guess tripleo deployed wallaby such that member was enouch | |
| 19:24:02 | sean-k-mooney | or the same tempest change was copied into our jobs | |
| 19:26:16 | sean-k-mooney | ah our job has the barbican plugin so we would not have seen it either way | |
| 19:26:17 | sean-k-mooney | https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/.zuul.yaml#L114 | |
| 19:26:39 | dansmith | not sure why you say that.. I'm just using the barbican plugin | |
| 19:27:05 | sean-k-mooney | our whitebox test are using the default roles created for tempest | |
| 19:27:31 | sean-k-mooney | dansmith: and as melwitt noted https://github.com/openstack/barbican/blob/master/devstack/lib/tempest#L9-L12 | |
| 19:27:50 | sean-k-mooney | the tempest config tempest_roles key gets creator added | |
| 19:28:35 | sean-k-mooney | https://github.com/openstack/tempest/blob/master/tempest/config.py#L64-L66 | |
| 19:28:50 | sean-k-mooney | so all user created by tempest woudl hae the creator role | |
| 19:28:52 | dansmith | oh you mean that's why or job works because it only needs the tempest users to have that permission, which it configures to happen | |
| 19:28:53 | dansmith | yes | |
| 19:29:12 | dansmith | sorry I thought you were asserting that because of the plugin we should be in the mode where it's not required | |
| 19:29:29 | sean-k-mooney | ya so either we have the new defualts in 17.1 downstream or we have duplicated this tempst config in our downstream jobs | |
| 19:30:21 | sean-k-mooney | my guess is 17.1 is configuring barbica so that member works | |
| 19:30:34 | dansmith | yeah that I don' tknow | |
| 19:35:59 | melwitt | sean-k-mooney: oh also the legacy barbican policy rule is "admin" or "creator". so if you have the admin role you don't need creator role | |
| 19:38:55 | sean-k-mooney | it looks like the barbican user in the service ocnfigs has the creator roles https://github.com/openstack-archive/tripleo-heat-templates/blob/stable/wallaby/deployment/barbican/barbican-api-container-puppet.yaml#L259-L274 | |
| 19:39:34 | sean-k-mooney | anyway with that in mind do we still want ot have a check for this in the api | |
| 19:39:53 | sean-k-mooney | given we shoudl be using the new defaults in 2024.1 across all services | |
| 19:41:42 | sean-k-mooney | looking at https://github.com/openstack/governance/blob/48d090643e7faf138b8f4e19f4a2915a20e1a873/goals/selected/consistent-and-secure-rbac.rst#20231-release-timeline | |
| 19:42:11 | sean-k-mooney | barbica had supprot for enforce_new_defaults=True in antelope | |
| 19:42:26 | sean-k-mooney | and in 2024.1 oslo is ment to remvoe enforce_scope | |
| 19:42:29 | sean-k-mooney | https://github.com/openstack/governance/blob/48d090643e7faf138b8f4e19f4a2915a20e1a873/goals/selected/consistent-and-secure-rbac.rst#20241-release-timeline | |
| 19:43:23 | sean-k-mooney | so as of this release and certenly for 2024.1 we shoudl be able to assume barbican has the behavior of enforce_scope=true | |
| 19:46:01 | sean-k-mooney | *2024.2 | |
| 19:48:16 | melwitt | sean-k-mooney: yeah.. I guess, I'm thinking of the upgrade scenario of a deployment that's been around for awhile and has enforce_scope = False set explicitly in policy? not sure how common that may be | |
| 19:48:40 | sean-k-mooney | melwitt: it looks like oslo has not deprecate the option or change the default yet | |
| 19:48:42 | sean-k-mooney | https://github.com/openstack/oslo.policy/blob/master/oslo_policy/opts.py#L27-L36 | |
| 19:48:44 | JayF | I suspect it's reasonably common | |
| 19:48:57 | melwitt | same.. | |
| 19:48:58 | JayF | Ironic was configured that way (enforce_scope=False) by default until this cycle | |
| 19:49:38 | sean-k-mooney | JayF: right so it looks like we did not make progress on changing the default and removing the option this cycle | |
| 19:49:48 | melwitt | sean-k-mooney: yeah oslo_policy hasn't yet but barbican defaulting it https://github.com/openstack/barbican/blob/master/barbican/common/config.py#L357-L361 | |
| 19:49:51 | sean-k-mooney | i remember this commign up in the last ptg | |
| 19:50:51 | sean-k-mooney | melwitt: so we can do soemthing like this | |
| 19:51:09 | sean-k-mooney | we can either use our new default value to infer barbicans | |
| 19:51:28 | sean-k-mooney | and check for creator based on that or just catch the excption late | |
| 19:51:48 | sean-k-mooney | once htis cant be turned off in barbican | |
| 19:51:52 | sean-k-mooney | we can drop the check in nova | |
| 19:51:59 | sean-k-mooney | and just rely on member | |
| 19:52:23 | melwitt | I was gonna say that we could check [oslo_policy]enforce_scope and if it's False do the check to barbican API but that would be assuming barbican is deployed on the same host as nova-api. which may not be a safe assumption | |
| 19:52:24 | sean-k-mooney | what that will actully mean is by default we wont need to check | |
| 19:53:13 | sean-k-mooney | it might pass the api check when it shoudl not but we will catch the api error in the compute manager when we try to create a key | |
| 19:53:25 | melwitt | er, or it would be assuming that barbican is configured the same. which it may not | |
| 19:53:47 | sean-k-mooney | so its better then nothign for the small subset of pepole who diverge form the defaults | |
| 19:54:08 | sean-k-mooney | right alghough you are not really ment to mix and match enfoce_scope | |
| 19:54:48 | sean-k-mooney | i mean i guess it depends on how expensive the check is | |
| 19:55:01 | sean-k-mooney | i dont know if oslo midelware gives us a list of all the roles on the token already | |
| 19:55:08 | sean-k-mooney | if it does and its just a map lookup | |
| 19:55:14 | sean-k-mooney | then it cheap and we can alwasy do it | |
| 19:55:39 | sean-k-mooney | if its an api call to keystone on every spawn with this feature that when we might want to optimise | |
| 19:57:04 | melwitt | I was thinking the same thing. I had been thinking it would have to be an api call but maybe we could do a lookup in the RequestContext or something | |
| 19:59:03 | sean-k-mooney | ya im hoping that oslo.midelway or keystone auth is just giving use a list of roles when it doing the token validation in keystone and we can jsut check in that | |
| 20:42:03 | gmann | sean-k-mooney: melwitt: we are not at the stage to remove/change defaults of enforce_scope in oslo.policy as many other project might be impacted but this config default can be overriden per service | |
| 20:42:22 | gmann | for nova, it is enabled by default, this is where we override the default https://github.com/openstack/nova/blob/13ccaf75f61a19ba11f39b63052bcef5dfbce91f/nova/policy.py#L51 | |
| 20:43:52 | sean-k-mooney | gmann: yep i know we have it enabled | |