| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-08-26 | |||
| 13:03:34 | sean-k-mooney | it reminds me of time wehre adding a print statement to debug code stop the bug form happening :) | |
| 13:13:24 | sean-k-mooney | am i being stupid "nova.exception.ServiceTooOld: This service is older (v72) than the minimum (v73) version of the rest of the deployment. Unable to continue." | |
| 13:13:36 | sean-k-mooney | so i deployed master yesterday | |
| 13:14:03 | sean-k-mooney | and i just checked out mel's cybrog change now | |
| 13:14:08 | sean-k-mooney | oh | |
| 13:14:14 | sean-k-mooney | i ownder if it need a rebase | |
| 13:14:26 | sean-k-mooney | we might have merged a new version in between? | |
| 13:17:09 | sean-k-mooney | ya ok it works after a rebase | |
| 13:18:34 | gibi | sean-k-mooney: with native threading, task can be switched any time. with eventlet they switch at certain places only (sleep and io) | |
| 13:18:47 | gibi | so our test are relied on those eventlet run to completion guarantees | |
| 13:18:58 | gibi | but it cannot rely on it any more | |
| 13:19:18 | gibi | I'm pretty happy that these are only test instabilities and we haven't found any new code races yet | |
| 13:20:09 | sean-k-mooney | ya, in the real code we were better about treating shared memory access as if it was real thread and using locks | |
| 13:20:27 | sean-k-mooney | i for one alwasy preetened it was a real thread for the most part hwen reviewing | |
| 13:20:52 | sean-k-mooney | but i woudl be surppriesed if we never mised a case where we relised on the eventlet behavior | |
| 13:21:05 | gibi | :) | |
| 13:39:03 | opendevreview | Balazs Gibizer proposed openstack/nova master: Stabilize live migration tests https://review.opendev.org/c/openstack/nova/+/1002493 | |
| 13:39:09 | gibi | the follow up :) ^^ | |
| 14:08:36 | melwitt | sean-k-mooney, stephenfin: from what I understand, the ServiceTokenAuthWrapper is required in order to send user+service token https://github.com/openstack/keystoneauth/blob/9e083fd5ae63aef66349ca980cdd315f5df4c767/keystoneauth1/service_token.py#L26 so that has to be passed in somehow | |
| 14:09:03 | opendevreview | Daniel Schulz proposed openstack/nova master: Prevent orphaned attachments by preemptive delete https://review.opendev.org/c/openstack/nova/+/1002439 | |
| 14:10:12 | melwitt | there's already a helper in nova to create one https://github.com/openstack/nova/blob/bbe520dd911be598273dd3d6d5a09fd35a56ddca/nova/service_auth.py#L69 so you get an auth plugin | |
| 14:11:41 | melwitt | and have to pass that to sdk somehow | |
| 14:17:22 | melwitt | ok I am seeing the rest of the review comments now | |
| 14:40:56 | gibi | gmaan: filed one more instabilities https://bugs.launchpad.net/nova/+bug/2165149 and assigned it to myself | |
| 14:46:36 | gibi | aaand file one more https://bugs.launchpad.net/nova/+bug/2165150 | |
| 14:50:26 | gibi | Uggla: I confirmed that your shelve offload / unshelve race is the same as we see on the gate so I reviewed your repor and fix and left comments | |
| 14:51:00 | Uggla | gibi cool ! I will try to jump on it ASAP | |
| 14:51:03 | sean-k-mooney | melwitt: so i think we can proceed with your change as is | |
| 14:51:34 | sean-k-mooney | melwitt: i think we can tidy things up in a followup and make manilla interaction use the service user tokes ectra based on this infra | |
| 14:51:35 | Uggla | Right now I'm testing cold migration of shares and found couple of issues. | |
| 14:51:36 | gibi | and marked https://bugs.launchpad.net/nova/+bug/2163552 as duplicate of https://bugs.launchpad.net/nova/+bug/2120927 (cc gmaan ) | |
| 14:52:01 | melwitt | sean-k-mooney: ok that is good news! | |
| 14:52:50 | sean-k-mooney | i held +w in case you wanted to look at refactoring the ksa plugin creation in this patch but i think its better to defer that | |
| 14:53:07 | sean-k-mooney | so i can add +w now if you dont intend to make any other changes | |
| 14:53:28 | sean-k-mooney | melwitt: we are expcting to update this code again after the sdk release too right | |
| 14:53:44 | sean-k-mooney | so we can defer those topic to that patch or a followup | |
| 14:54:23 | melwitt | sean-k-mooney: you mean the keystoneauth release? yeah although I think I see that got released this morning | |
| 14:55:24 | melwitt | it's not in upper-constraints yet though so I think we can't use yet | |
| 14:57:20 | melwitt | sean-k-mooney: yeah I mean AFAICT we do need 3 options in get_sdk_adapter (I replied on the review) so I'm not sure there is a cleaner way to do it | |
| 14:59:27 | melwitt | for manila, do we know we want to send service tokens along with it too? I wondered if it is intentional that it does not today | |
| 14:59:37 | sean-k-mooney | yes | |
| 14:59:45 | sean-k-mooney | infact orginally we didnt want to merge it without it | |
| 14:59:54 | melwitt | oh, interesting | |
| 15:00:31 | sean-k-mooney | there were issue with doing that at the teim becuase there was no end ot end example of using service tokens wiht the sdk | |
| 15:01:02 | melwitt | ah, gotcha | |
| 15:01:02 | sean-k-mooney | basiclly we shoudl always send service user tokens if we can to prevent token exeriation issues | |
| 15:02:50 | gmaan | gibi: ack and thanks | |
| 15:03:06 | gmaan | gibi: no eventlet meeting today right/? | |
| 15:03:20 | gmaan | i did not find in my calendar this week | |
| 15:04:16 | stephenfin | melwitt: I'm only reading the scrollback now, but https://review.opendev.org/c/openstack/requirements/+/1002461 is winding it's way through the gate | |
| 15:04:35 | melwitt | stephenfin: a-ha cool | |
| 15:05:06 | stephenfin | Also, you probably saw this in the scrollback but these are the docs I wrote on using service token with SDK a few years back https://review.opendev.org/c/openstack/openstacksdk/+/893505 | |
| 15:05:20 | melwitt | so yeah when that lands IIUC we could update the sdk patch to require 5.16.0 and remove the workaround wrapper | |
| 15:05:36 | melwitt | oh, yes thanks | |
| 15:05:59 | stephenfin | As I noted, I recall Uggla wasn't able to get it working. I did some testing and it turns out your KSA patch was the missing part 😄 | |
| 15:06:13 | stephenfin | *And I also noted that | |
| 15:06:14 | melwitt | ahh yeah | |
| 15:07:49 | stephenfin | and yeah, once the requirements patch merges, we could tweak your patch to bump the ksa version and drop the local workaround | |
| 15:08:19 | melwitt | sean-k-mooney: thanks for the +W. I think that is safer to merge as-is first and then I do a follow on patch to bump the ksa version and drop the workaround JUST in case it does not work | |
| 15:08:21 | stephenfin | or we can just merge it as-is now and fix it in a follow-up. I'm okay with either. If you or sean-k-mooney want to do the former, feel free to carry my +2 forward | |
| 15:08:25 | gibi | gmaan: right no eventlet meeting today | |
| 15:08:35 | stephenfin | okay well that solves that :) | |
| 15:08:47 | gmaan | ok | |
| 15:09:37 | sean-k-mooney | stephenfin: no i +w'd as is just noting that 3 probaly is an existing bug | |
| 15:09:51 | sean-k-mooney | stephenfin: but being an exsting bug we should fix it as such later | |
| 15:11:32 | gmaan | gibi: melwitt: I am not sure if dansmith will be able to re-review it but we need to merge it as FF is approaching. melwitt if you can check this? or gibi can +w https://review.opendev.org/c/openstack/nova/+/1000211 | |
| 15:12:02 | gmaan | gibi: melwitt and doc change which i updated as per feedback and replied on other comments, please check https://review.opendev.org/c/openstack/nova/+/997141/13 | |
| 15:13:35 | melwitt | gmaan: yeah I think dansmith will not be able to re-review today, I can do it | |
| 15:13:45 | gmaan | melwitt: thanks | |
| 15:13:57 | gibi | I will check those tomorrow my morning | |
| 15:14:08 | gmaan | ack, thanks | |
| 15:14:09 | gibi | and add some +w if needed | |
| 15:14:15 | gmaan | ok | |
| 15:14:43 | gmaan | thanks for review on the bug fixes, let me check your comment today and fix but I see things are merging at least | |
| 15:15:06 | gibi | yeah we merged some TDX patches | |
| 15:15:27 | gmaan | yeah i saw those very first thing in morning :) | |
| 15:15:45 | sean-k-mooney | about half | |
| 15:15:57 | sean-k-mooney | i saw gibi already rechecked the failures | |
| 15:16:02 | gibi | yep | |
| 15:16:07 | gibi | I'm keeping an eye of that series | |
| 15:16:22 | gibi | a/of/on/ | |
| 15:16:25 | gmaan | yeah, will continue in my TZ | |
| 15:17:07 | sean-k-mooney | i assume we will apply our normal rules, if its +2w'd before FF we can recheck without a FFE until monday if requried | |
| 15:17:18 | sean-k-mooney | then talk about it in the meeting | |
| 15:18:04 | melwitt | that is my understanding | |
| 15:18:23 | sean-k-mooney | as an aside obviously wont be around friday but ill be off on monday as well | |
| 15:18:38 | melwitt | same for me | |
| 15:18:44 | gibi | I can do some recheck on Friday too | |
| 15:20:50 | Uggla | Upstream bug triage in ~10mn (I not prepared) | |
| 15:20:53 | stephenfin | sean-k-mooney: melwitt: Is there any time we *shouldn't* be sending service tokens when talking to other services? | |
| 15:21:09 | stephenfin | I assume not, but just checking in case I've missed something | |
| 15:21:30 | sean-k-mooney | stephenfin: my understanding is only if its not configured in the config or explcitly disabled in teh config | |
| 15:21:53 | sean-k-mooney | stephenfin: but nova requries them to be able to talks to cidner and manage attachment | |
| 15:21:57 | sean-k-mooney | and to cybrog | |
| 15:22:12 | sean-k-mooney | so i think we shoudl expect that its more or less alwasy configured | |
| 15:22:45 | melwitt | yeah I remember we had to require it for cinder, for deleting attachments | |
| 15:22:46 | gmaan | stephenfin: sean-k-mooney we should always, i know we had a temporary config option not to send it but that was added only for transition purpose and suppose to be removed (which did not happen) | |
| 15:22:53 | Uggla | link: meet.google.com/zjr-rxus-hzj (triage) | |
| 15:23:11 | gmaan | Uggla: in horizon meeting, might be little late | |
| 15:23:17 | sean-k-mooney | melwitt: ya so eventully we shoudl not require the service token to talk to cinder and cybrog | |