| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-05-20 | |||
| 18:13:58 | sean-k-mooney | melwitt: basiclly when you use this all the test that dont need to run serially acriore the reasder lock and hte ones that do aquire the writer lock | |
| 18:14:18 | sean-k-mooney | and hten the filesysem lock syncoised across the workers | |
| 18:14:29 | melwitt | makes sense | |
| 18:14:31 | sean-k-mooney | but in addtion ot that we run the serial tests last | |
| 18:14:40 | sean-k-mooney | becuase of how we order testes base don name | |
| 18:14:49 | sean-k-mooney | at least in upstream tempest | |
| 18:15:22 | sean-k-mooney | melwitt: https://github.com/openstack/tempest/commit/dfb304355b46882696ef26386637836577be8db7 | |
| 18:15:42 | sean-k-mooney | it was an optimisation that gibi help imeplemnta few years ago | |
| 18:16:39 | gmaan | yeah, that resolved the aggregates tests issue | |
| 18:16:40 | melwitt | very cool | |
| 18:16:45 | sean-k-mooney | hehe https://github.com/openstack/tempest/commit/73ba33773daf1df1be792b616842dd389fd325bc | |
| 18:16:50 | sean-k-mooney | looks like you have used it before | |
| 18:17:04 | melwitt | uhhh lol | |
| 18:17:06 | gmaan | so for now, concurrency=1 will work for VTPM case as they are from same test class | |
| 18:17:37 | sean-k-mooney | yep also we run like 8 tests in that job | |
| 18:17:44 | sean-k-mooney | so concrance 1 is not a big issue | |
| 18:17:50 | melwitt | well, I'm not gonna forget it again now haha | |
| 18:18:16 | melwitt | I put it at its own job bc it needs nested virt and was wary to tie up those machines too much with other tests that don't need it | |
| 18:18:26 | sean-k-mooney | for whiht box in general iw touls be nicer to not need that obvioulsy as many of the test can run in parralel | |
| 18:18:35 | gmaan | yeah | |
| 18:19:02 | sean-k-mooney | well mystery solved | |
| 18:19:13 | sean-k-mooney | feel free to pin and i can review the zuul change quickly | |
| 18:19:51 | melwitt | yeah. this is good. I was struggling to write the code comment for needing concurrency=1 so I was looking at details again. bc I was not seeing why. ugh. | |
| 18:29:29 | opendevreview | melanie witt proposed openstack/nova master: Use tempest_concurrency=1 for nova-vtpm job https://review.opendev.org/c/openstack/nova/+/984864 | |
| 18:30:41 | melwitt | gmaan, sean-k-mooney: ^ | |
| 18:39:15 | gmaan | melwitt: ack, thanks | |
| 18:46:53 | sean-k-mooney | looks good to me | |
| 19:41:33 | melwitt | I'm sure someone already said this but https://bugs.launchpad.net/tempest/+bug/2153382 is affecting nova-next too Details: {'code': 401, 'title': 'Unauthorized', 'message': 'The request you have made requires authentication.'} | |
| 19:53:48 | melwitt | (just replied on the ML) | |
| 19:54:23 | gmaan | ohk, I did not realize nova-next also run that test | |
| 19:55:07 | gmaan | anyways fix is in gate, I will merge and updaet ML once test results are out https://review.opendev.org/c/openstack/tempest/+/938766 | |
| 19:56:24 | melwitt | thanks gmaan | |
| 19:57:18 | melwitt | nova-multi-cell too. sent my reply too hastily | |
| 20:01:00 | sean-k-mooney | you know our gate would pass more frequetly if we just turned off cidner :P | |
| 20:01:39 | sean-k-mooney | gmaan: was the change form a 403 to a 401 intentioanly by the way | |
| 20:02:40 | gmaan | it was due to change in default value of service_token_roles_required in keystonemiddlwware | |
| 20:02:57 | sean-k-mooney | its not really a 401 Unauthorized Error as i understnad it. the service token is a valid token but it does not have the requrie roles right? | |
| 20:02:58 | gmaan | and tempest test did not handle it beacuse it was disable it in CI. | |
| 20:03:07 | gmaan | yes | |
| 20:03:08 | sean-k-mooney | so it really shoudl be a s 403 Forbidden Error | |
| 20:03:29 | sean-k-mooney | becuase it authrization issue not authentication | |
| 20:04:12 | sean-k-mooney | so the real fix woudl be in cidner to make ti return a 403 right? | |
| 20:05:14 | gmaan | I am not sure, if service token does not have a required token then it does not come to cinder itself that if operation is forbidden or not. | |
| 20:05:45 | gmaan | so that is why 401 seems valid one as passed token and required roles are not valid | |
| 20:06:05 | sean-k-mooney | but 401 is not about roles or permeison | |
| 20:06:13 | sean-k-mooney | its about is the token exprie/valid | |
| 20:06:37 | melwitt | I thought that would include roles too though, no? | |
| 20:06:52 | sean-k-mooney | a 401 is an indtication to the client that it should reauthicate | |
| 20:06:59 | sean-k-mooney | adn retry | |
| 20:07:05 | sean-k-mooney | but you do not retyr a 403 | |
| 20:07:29 | sean-k-mooney | form https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/401 | |
| 20:07:40 | sean-k-mooney | A 401 Unauthorized is similar to the 403 Forbidden response, except that a 403 is returned when a request contains valid credentials, but the client does not have permissions to perform a certain action. | |
| 20:08:26 | sean-k-mooney | and form https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/403 """Clients that receive a 403 response should expect that repeating the request without modification will fail with the same error. Server owners may decide to send a 404 response instead of a 403 if acknowledging the existence of a resource to clients with insufficient privileges is not | |
| 20:08:28 | sean-k-mooney | desired.""" | |
| 20:09:16 | sean-k-mooney | melwitt: in general no | |
| 20:09:38 | sean-k-mooney | it can | |
| 20:09:44 | gmaan | so in this case, token is considered as invalid right | |
| 20:09:47 | sean-k-mooney | but it more the retry behvior diffent | |
| 20:09:58 | sean-k-mooney | the token is valid | |
| 20:10:04 | sean-k-mooney | but it has insufffenct privldages | |
| 20:10:19 | melwitt | 401 maybe you need to re-auth as a different project or role | |
| 20:11:24 | sean-k-mooney | i guess a client could choose to do that | |
| 20:12:01 | sean-k-mooney | eveutlly nova shoudl just be callign cidner with its own token with the service role on the user token | |
| 20:12:05 | gmaan | I think it is different from service token perspective. what keystonemiddlware consider a serviec token a valid if it has all requried role | |
| 20:12:23 | sean-k-mooney | and we shoudl not be lookign at the service_token at all for permsions in this case | |
| 20:12:28 | gmaan | otherwise a user token which is valid can always be a valid token but that is not the case if we see that as a serviec token | |
| 20:13:03 | sean-k-mooney | maybe | |
| 20:13:05 | gmaan | it just validate if service token send has required role to be consider as 'service' and if not then it is invalid "SERVICE TOKEN" | |
| 20:13:26 | sean-k-mooney | well | |
| 20:13:27 | gmaan | from suer tokjen perspective i agree on 401 vs 403 | |
| 20:13:43 | sean-k-mooney | it need to do 2 things, validate it has the expecte roles adn that it has not expired | |
| 20:13:56 | sean-k-mooney | if both are ture the the service token is valid | |
| 20:14:04 | gmaan | yes | |
| 20:14:44 | sean-k-mooney | anyway did i understnad the fix in tempest is to accpet either 401 or 403 | |
| 20:14:53 | sean-k-mooney | so that you can tolerate either behvior | |
| 20:14:54 | gmaan | yes | |
| 20:15:22 | gmaan | it was only 403 previously bcz service token roles were not validated before | |
| 20:15:30 | sean-k-mooney | that fair you have a few whit space issue by th ewya | |
| 20:15:52 | sean-k-mooney | well yes and no it was optional and off by default | |
| 20:15:55 | sean-k-mooney | but for the cve | |
| 20:16:08 | sean-k-mooney | we added validation in the code spereate from the policy layer | |
| 20:16:28 | gmaan | i thought of adding a new config option to check 401 but that will be unnecessary as service_token_roles_required os a temp config option and should be removed at the end | |
| 20:16:55 | gmaan | it was added for migration purpose and never got moment to be default to True and then removed | |
| 20:17:06 | gmaan | same as enforce_scope in RBAC | |
| 20:17:21 | sean-k-mooney | same as the fallback for threaing pcpus as vcpus | |
| 20:17:39 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/975779 | |
| 20:17:43 | gmaan | yeah, i do not know if it is good to be less aggressive on those things or bad :) | |
| 20:18:57 | sean-k-mooney | well ^ was deprecated in nova 20.0.0 | |
| 20:19:57 | sean-k-mooney | which was trian | |
| 20:20:18 | sean-k-mooney | i think that one os more then overdue to be disabel by default and we proably shoudl delete it :) | |
| 20:21:25 | sean-k-mooney | we have a habbit of not cleaing up these migrtion path for many many many years after they were ment to be removed | |
| 20:23:59 | gmaan | yeah | |
| 20:24:36 | gmaan | and by then they become a valid thing for operators than just migration path :) | |
| 20:25:19 | gmaan | I will be more than happy if i can remove enforce_scope this cycle but I know it will fail many project tests and they will not fix it on time | |
| 20:25:44 | gmaan | and breaking them might be the only option to proceed | |
| 20:35:03 | sean-k-mooney | gmaan: if you do it kinder to do it early | |
| 20:35:15 | sean-k-mooney | i.e. at m1 or m2 | |
| 20:37:37 | gmaan | m1 was too early to fix the things so i sent m2 as deadline in ML | |
| 20:38:30 | gmaan | doing it in m1 could be better but then it would not give time for projects to fix tests or change default who still disable it | |
| 20:40:21 | sean-k-mooney | thats fair. on the otherhadn the default was change in oslo a few cycles ago right | |