| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-08-31 | |||
| 09:23:39 | sean-k-mooney | ya it looks like wal mode replaces it | |
| 09:23:49 | sean-k-mooney | and shared_cache mdoe is the legacy version | |
| 09:23:51 | sean-k-mooney | https://sqlite.org/wal.html | |
| 09:24:21 | gibi | we do set WAL | |
| 09:24:30 | sean-k-mooney | There is an additional quasi-persistent "-wal" file and "-shm" shared memory file associated with each database, | |
| 09:24:39 | sean-k-mooney | i was confusing it with the shared memory file | |
| 09:24:49 | gibi | but I don't see the WAL file either :) | |
| 09:25:11 | gibi | maybe it is deleted when the connection closed | |
| 09:27:06 | gibi | yepp | |
| 09:27:10 | gibi | they are there | |
| 09:27:14 | gibi | temporarily | |
| 09:27:18 | gibi | nova-test-api-5dclxobp.sqlite-wal | |
| 09:27:24 | gibi | nova-test-api-5dclxobp.sqlite-shm | |
| 09:27:35 | gibi | but that also means it is a unique file per DB | |
| 09:28:08 | gibi | now if two thread like nova-api and nova-conductor connects to the same api DB I assume they are sharing these | |
| 09:28:31 | sean-k-mooney | ack well if you find something that helps let me knwo https://sqlite.org/pragma.html#pragma_locking_mode might be intersting | |
| 09:28:52 | sean-k-mooney | gibi: they could but i thinkt hat is intended to be supproted | |
| 09:29:02 | gibi | anyhow I will push a patch with a PRAGMA busy_timeout = milliseconds; | |
| 09:29:12 | gibi | that is small and easy | |
| 09:29:24 | gibi | and functional test specific | |
| 09:32:02 | gibi | hm, maybe not. The WAL PRAGMA is persistent for the DB, but the busy_timeout is per connection so I cannot just add it to the same place :/ | |
| 09:32:18 | gibi | looking at oslo.db retry :) | |
| 09:36:51 | sean-k-mooney | the other thing you could try is set the lockign mode to exclusivne and back to normal in our currnet lock code https://sqlite.org/pragma.html#pragma_locking_mode but im not sure if that will really help or not | |
| 09:37:50 | sean-k-mooney | i feel like that hsoudl not be required but ya im goign to go get coffee and step away for a bit | |
| 09:40:19 | gibi | the hardness of this that I have no reproducer so whatever I try I have no proof it actually helps other than merging it and letting is run in the gate | |
| 09:41:12 | gibi | so trying random things is expensive | |
| 09:42:04 | gibi | the oslo.db retry code is pluged in a horrible way. I'm not even sure we use any retry at all by default | |
| 09:43:02 | gibi | we have decorators @oslo_db_api.wrap_db_retry(max_retries=5, retry_on_deadlock=True) in some places | |
| 09:43:09 | gibi | so we do retry in some cases | |
| 09:43:11 | gibi | not all | |
| 09:43:28 | gibi | so even if I change what to retry on I would need to decorate everything | |
| 09:43:31 | gibi | that is uggly | |
| 09:48:51 | gibi | OK, looking at differently. We have the write lock per DB file. That should serialize all the writes. If I assume it works well, then the only thing that can happen is that the SQLite DB lock is not released when our functional fixture lock is released (as you said at the start) so we could try to flush, close, whatever the connection to help with that DB level lock | |
| 10:12:35 | opendevreview | Merged openstack/nova master: tests: Remove errant CONF mocks https://review.opendev.org/c/openstack/nova/+/1002852 | |
| 10:15:46 | tkajinam | could anyone give 2nd +2 to https://review.opendev.org/c/openstack/nova/+/995370 and https://review.opendev.org/c/openstack/nova/+/995370 to reduce deprecation warnings ? | |
| 10:52:36 | gibi | tkajinam: thanks I +Ad the https://review.opendev.org/c/openstack/nova/+/995370 but you linked that patch twice above so I assuem you have one more patch to look at | |
| 11:03:31 | tkajinam | oh sorry I've bee struggling to make my copy-paste work correctly | |
| 11:03:35 | tkajinam | gibi, https://review.opendev.org/c/openstack/nova/+/1002937 this is the 2nd one | |
| 11:03:47 | tkajinam | gibi, and thanks a lot ! | |
| 11:05:16 | tkajinam | oh wait I found a few remaining warnings | |
| 11:05:39 | tkajinam | will address these in follow-up | |
| 11:20:00 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_clas argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 11:20:26 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_cls argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 11:34:13 | gibi | thanks | |
| 12:25:52 | opendevreview | Merged openstack/nova master: Replace deprecated remotable_classmethod https://review.opendev.org/c/openstack/nova/+/995370 | |
| 12:41:11 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_cls argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 13:39:55 | clif | sean-k-mooney: is there anything I can do to help that perf patch merge? | |
| 14:11:59 | sean-k-mooney | did it fail after the rebase? im on pto today so not really paying attaentio, gibi is lookign at some stablity impovment | |
| 14:12:17 | sean-k-mooney | i think you just geting unlucky with the test failures | |
| 14:12:48 | sean-k-mooney | the db locked error is oen of the knwo issues withthe func tests under threading mode currently | |
| 14:13:07 | sean-k-mooney | and it was in an entirly differnt area of the code so not related to that patch | |
| 14:40:54 | clif | gotcha | |
| 14:41:05 | clif | sorry to ping you on your day off, enjoy | |
| 14:41:34 | sean-k-mooney | its fien i would not be on irc if it bothered me | |
| 15:30:10 | Uggla | Reminder: Upstream meeting in ~30mn. | |
| 16:02:47 | Uggla | #startmeeting nova | |
| 16:02:47 | opendevmeet | Meeting started Mon Aug 31 16:02:47 2026 UTC and is due to finish in 60 minutes. The chair is Uggla. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 16:02:47 | opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 16:02:47 | opendevmeet | The meeting name has been set to 'nova' | |
| 16:02:56 | Uggla | Hello everyone | |
| 16:02:59 | bauzas | o/ | |
| 16:03:02 | gmaan | o/ | |
| 16:03:42 | elodilles | o/ | |
| 16:04:37 | lajoskatona | o/ | |
| 16:05:14 | Uggla | Let's start | |
| 16:05:22 | Uggla | #topic Bugs (stuck/critical) | |
| 16:05:32 | Uggla | #info No Critical bug | |
| 16:05:43 | Uggla | #topic Gate status | |
| 16:05:46 | tkajinam | o/ | |
| 16:05:52 | Uggla | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | |
| 16:06:01 | Uggla | #link https://etherpad.opendev.org/p/nova-ci-failures-minimal | |
| 16:06:12 | Uggla | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status | |
| 16:06:23 | Uggla | #info Please look at the gate failures and file a bug report with the gate-failure tag. | |
| 16:06:32 | Uggla | #info Please try to provide a meaningful comment when you recheck | |
| 16:06:33 | gmaan | we ,merged a lot of fixes last week, I will remove the bugs from that etherpad one we alrady fixed | |
| 16:07:13 | Uggla | #info last week we had gate instability. Any progress? | |
| 16:07:54 | Uggla | gmaan 👍 | |
| 16:08:00 | tkajinam | I found a few strange errors this evening like dead workers or zombie process causing job failure, but I'm not sure if that appears consistently | |
| 16:08:11 | gmaan | we merged almost all the fixes up for the frequent bugs happening but I dod not check today how gate is behaving | |
| 16:08:12 | tkajinam | (these are not specific to nova but look like infra problems | |
| 16:08:23 | gmaan | seems like, no change in CI pipeline | |
| 16:09:03 | tkajinam | gmaan, https://review.opendev.org/c/openstack/nova/+/1002223 is the one which is supposed to fix unstable threading job, right ? | |
| 16:09:05 | gmaan | tkajinam: ack, if you see them agian, please log the bug and we can check that if it happens more frequently | |
| 16:09:29 | gmaan | tkajinam: yes, this is one of that. but there were 5-6 totals | |
| 16:09:35 | tkajinam | gmaan, yeah or I'll report it to infra team, because that's a infra problem rather than a nova bug I believe. | |
| 16:09:38 | tkajinam | ok. good | |
| 16:09:43 | gmaan | cool, thanks | |
| 16:09:52 | tkajinam | will be careful about threading unit tests this week | |
| 16:10:53 | Uggla | I think we can move on | |
| 16:11:12 | Uggla | #topic Release Planning | |
| 16:11:18 | Uggla | #link https://releases.openstack.org/hibiscus/schedule.html | |
| 16:11:20 | tkajinam | yup | |
| 16:11:29 | Uggla | #info Nova deadlines are set in the above schedule | |
| 16:11:37 | Uggla | #info Feature freeze was last week. | |
| 16:12:09 | Uggla | #topic Review priorities | |
| 16:12:22 | Uggla | #info Starting: https://etherpad.opendev.org/p/nova-2026.2-status#L16 interesting bugs to review. | |
| 16:13:32 | Uggla | virtiofs cold migration needs focus too. I focused on testing it last week. | |
| 16:14:08 | Uggla | #topic Stable Branches | |
| 16:14:25 | elodilles | thx | |
| 16:14:29 | elodilles | #info stable gates should be OK | |