| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-08-27 | |||
| 19:14:35 | melwitt | gmaan: haha thanks. yes that makes sense that it's necessary for the restart. I was confused by your last comment about tests failing without the wait because we were commenting at the setUp location in the code :) | |
| 19:15:20 | gmaan | yeah, i commented that in wrong place. let me remove it from setUp to avoid confusion in future | |
| 19:15:31 | melwitt | I kept thinking about it like, that doesn't make sense at this location. I understand now, you meant at the restart location | |
| 19:25:42 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Fix graceful shutdown functional unstable tests https://review.opendev.org/c/openstack/nova/+/1002223 | |
| 19:26:29 | gmaan | melwitt: ^^ fixed it and also added notes | |
| 19:26:59 | melwitt | cool thanks! | |
| 21:33:58 | opendevreview | Merged openstack/nova master: perf(ironic): eliminate O(N²) ProviderTree deepcopy at startup https://review.opendev.org/c/openstack/nova/+/980676 | |
| 21:43:03 | opendevreview | Merged openstack/nova master: Add shebang to mark bash files https://review.opendev.org/c/openstack/nova/+/995213 | |
| #openstack-nova - 2026-08-28 | |||
| 00:23:08 | opendevreview | Merged openstack/nova master: Add functional test for cleanup_host cancelling queued live migrations https://review.opendev.org/c/openstack/nova/+/986477 | |
| 01:24:00 | opendevreview | Merged openstack/nova master: Replace OVN Metadata agent with OVN agent https://review.opendev.org/c/openstack/nova/+/987042 | |
| 06:20:22 | opendevreview | Merged openstack/nova master: Fix graceful shutdown functional unstable tests https://review.opendev.org/c/openstack/nova/+/1002223 | |
| 09:24:04 | opendevreview | Merged openstack/nova master: Remove tempest-pg-full https://review.opendev.org/c/openstack/nova/+/996887 | |
| 12:57:02 | tkajinam | bauzas, hi. could you review this test addition for tdx ? I had to update it according to some changes made in the tdx support implementation https://review.opendev.org/c/openstack/nova/+/998525 | |
| 13:23:11 | opendevreview | Daniel Schulz proposed openstack/nova master: Prevent orphaned attachments by preemptive delete https://review.opendev.org/c/openstack/nova/+/1002439 | |
| 13:30:23 | opendevreview | Ivan Anfimov proposed openstack/nova master: Replace outdated usage of egrep by grep https://review.opendev.org/c/openstack/nova/+/1002754 | |
| 13:31:43 | opendevreview | Ivan Anfimov proposed openstack/nova master: Replace outdated usage of egrep by grep https://review.opendev.org/c/openstack/nova/+/1002754 | |
| 15:25:28 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron networks https://review.opendev.org/c/openstack/nova/+/928022 | |
| 15:28:15 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron networks https://review.opendev.org/c/openstack/nova/+/928022 | |
| 15:49:03 | opendevreview | Daniel Schulz proposed openstack/nova master: Prevent orphaned attachments by preemptive delete https://review.opendev.org/c/openstack/nova/+/1002439 | |
| 18:15:31 | opendevreview | Merged openstack/nova master: Shutdown thread pool executors during graceful shutdown https://review.opendev.org/c/openstack/nova/+/1000211 | |
| 18:15:43 | opendevreview | Merged openstack/nova master: Add doc and release notes for the graceful shutdown task tracking https://review.opendev.org/c/openstack/nova/+/997141 | |
| #openstack-nova - 2026-08-29 | |||
| 10:38:12 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Remove errant CONF mocks https://review.opendev.org/c/openstack/nova/+/1002852 | |
| 11:30:45 | opendevreview | Merged openstack/nova master: libvirt: Add functional tests for TDX capability report https://review.opendev.org/c/openstack/nova/+/998525 | |
| 16:47:03 | opendevreview | Udayendu Kar proposed openstack/python-novaclient master: Fixed max-servers handling in host-evacuate-live https://review.opendev.org/c/openstack/python-novaclient/+/1002874 | |
| #openstack-nova - 2026-08-30 | |||
| 00:12:39 | opendevreview | Khoi proposed openstack/nova-specs master: Add availability-zone-scoped quota spec https://review.opendev.org/c/openstack/nova-specs/+/1002901 | |
| 04:59:23 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_clas argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 05:01:43 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_cls argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 05:20:23 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_cls argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 06:00:02 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_clas argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 15:15:40 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_clas argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| #openstack-nova - 2026-08-31 | |||
| 08:28:25 | gibi | looking at the functional threding job results we are definitely in a better shape now. There is one thing to monitor. I see TIMED_OUT results still after our stabilization fixes landed. And the timeout happnes during the functional test execution. It seems like we have a potentially hanging test case somewhere | |
| 08:29:07 | gibi | I'm not sure why the normal test case timeout does not hit and kill the test case | |
| 08:29:35 | gibi | but given that native threads are not killable from outside I'm not that surprised | |
| 08:29:35 | bauzas | excellent news, thanks gibi | |
| 08:30:55 | stephenfin | sean-k-mooney: Uggla: Would I be able to get some eyes on https://review.opendev.org/c/openstack/nova/+/1002852 today? It's blocking the u-c bump of ksa. gibi has already taken a look (thanks 🙏) | |
| 08:34:25 | sean-k-mooney | done | |
| 08:35:16 | sean-k-mooney | gibi: ya i rechecekd a patch twice over the weekend and it timed out in the functional tests both times | |
| 08:39:41 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron networks https://review.opendev.org/c/openstack/nova/+/928022 | |
| 08:40:54 | gibi | sean-k-mooney: I will take a look | |
| 08:52:05 | sean-k-mooney | test_live_migrate_vifs_from_info_cache | |
| 08:52:11 | sean-k-mooney | failed in the later run | |
| 08:52:17 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/980679 | |
| 08:52:57 | sean-k-mooney | but the job in general timesed out | |
| 08:53:12 | sean-k-mooney | the proir run just timed out in general | |
| 08:53:57 | sean-k-mooney | those were a day and ahalf apart so it may have been before some of the stablisation patches landed | |
| 08:54:34 | sean-k-mooney | sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked | |
| 08:55:03 | sean-k-mooney | i dont knwo if your per db lock tweak landed before that | |
| 08:55:53 | sean-k-mooney | hum it shoudl have | |
| 08:55:59 | stephenfin | sean-k-mooney++ ty | |
| 08:58:00 | gibi | https://bugs.launchpad.net/nova/+bug/2165149 | |
| 08:58:26 | gibi | there are new hits in check pipeline | |
| 08:58:44 | gibi | it might or might not mean that the patches are not rebased yet | |
| 08:59:04 | sean-k-mooney | they have not been rebvased but have been rechecked after the merge | |
| 08:59:45 | opendevreview | Clif Houck proposed openstack/nova master: Parallelize per-node resource updates https://review.opendev.org/c/openstack/nova/+/980679 | |
| 08:59:48 | gibi | does the check pipeline rebases the patch before running the tests? I don't think so | |
| 08:59:58 | sean-k-mooney | i clicked the rebase button in the ui we can see | |
| 09:00:03 | gibi | cool | |
| 09:00:19 | sean-k-mooney | gibi: am it does a speculitive merge with master yes | |
| 09:00:33 | sean-k-mooney | so it wont rebase but it should have the same effect | |
| 09:00:52 | gibi | even the check pipeline? not just the gate? | |
| 09:01:11 | sean-k-mooney | correct | |
| 09:01:31 | gibi | the the locking fix did not fully solved the issue reported in that bug | |
| 09:01:51 | sean-k-mooney | ya so the fix | |
| 09:02:16 | sean-k-mooney | is that we shoudl add database locked to the list of excation that oslo.deb treads as a db deadlock excption | |
| 09:02:26 | sean-k-mooney | then our existing decorators will handel that properly | |
| 09:03:07 | gibi | I'm not sure we want to change the production code due to functional test DB limitations | |
| 09:03:14 | sean-k-mooney | my guess is unless we explcitly flush the db or add a small delay it might take a second for sqlachemy to finsih unlocking the db after the write | |
| 09:03:28 | gibi | could be | |
| 09:03:44 | gibi | sqlite also has some config to retry internall for hits | |
| 09:03:47 | sean-k-mooney | well sqlite is used in production when you use bifrost or standalone ironic | |
| 09:03:51 | gibi | /hits/this/ | |
| 09:04:24 | gibi | they are broken then as our locking is only for functional test | |
| 09:04:44 | sean-k-mooney | well yes but our lockign is for nova :) | |
| 09:04:55 | gibi | ahh so they not using nova with sqlite | |
| 09:04:57 | sean-k-mooney | i mean ironic proper not ironic virt driver | |
| 09:05:16 | gibi | but then I'm not sure how this is relevant :) | |
| 09:05:25 | sean-k-mooney | ya so if you use biforst to manage yoru hardware inventory it uses or can use sqlite as the db | |
| 09:05:51 | sean-k-mooney | the same way that ironic suprot there own jsonrpc implation instead of rabbit for a lighter weight deployment if you want that | |
| 09:06:23 | sean-k-mooney | oh just that the oslo.db change should benift them too | |
| 09:06:26 | gibi | OK, so you say we change oslo.db's list of exceptions to retry on | |
| 09:06:37 | gibi | :) | |
| 09:07:07 | gibi | I still cannot really justify that from nova perspective and the ironic case is something I cannot reproduce | |
| 09:07:08 | sean-k-mooney | i think that would be good in the long term yes but i think we shoudl be able to work aroudn this in the futnctionl tests | |
| 09:07:55 | sean-k-mooney | its just somethign i looked into when sqlite was segfaulting in cybrog | |
| 09:08:44 | sean-k-mooney | as a cleaner way then jsut hte locking | |
| 09:10:08 | sean-k-mooney | but ya if there are other tuneable or we can specficly tell sqlachemy to flush/release the db or something that might be better | |
| 09:10:39 | gibi | https://sqlite.org/c3ref/busy_timeout.html | |
| 09:10:50 | gibi | this could be an SQLite native retry | |
| 09:11:24 | gibi | https://sqlite.org/c3ref/busy_timeout.html | |
| 09:11:33 | gibi | https://sqlite.org/pragma.html#pragma_busy_timeout | |
| 09:13:41 | sean-k-mooney | """The SQLITE_LOCKED result code indicates that a write operation could not continue because of a conflict within the same database connection or a conflict with a different database connection that uses a shared cache. """ | |
| 09:14:20 | sean-k-mooney | so the vs """The SQLITE_BUSY result code indicates that the database file could not be written (or in some cases read) because of concurrent activity by some other database connection, usually a database connection in a separate process. """ | |
| 09:14:30 | opendevreview | Balazs Gibizer proposed openstack/nova master: [functional]Do not patch req-id generation https://review.opendev.org/c/openstack/nova/+/1003024 | |
| 09:14:45 | sean-k-mooney | we are not using processes and we have a per db lock | |
| 09:15:08 | sean-k-mooney | so im no t sure the busy tunabels will help | |
| 09:15:14 | gibi | we use threads | |
| 09:15:18 | sean-k-mooney | are we using the cache file | |
| 09:15:36 | sean-k-mooney | i wonder if we are sharign the cache betwen dbs? | |
| 09:17:18 | gibi | I think it is a separate file per DB but I can probably check by injecting a long see into our tests and see the dir structure | |
| 09:17:35 | sean-k-mooney | ack | |