| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-08-27 | |||
| 19:10:13 | gmaan | let me know if you prefer to remove it from setUp, i can respin it quickly | |
| 19:12:42 | gmaan | basically there is difference in service status until first heartbeat arrives for start fresh service vs restart existing service. where in former case, service initial status is 'up' and latter one is 'down'. once first heartbeat arrives it is as per the heartbeat and service_down_time | |
| 19:14:35 | melwitt | gmaan: haha thanks. yes that makes sense that it's necessary for the restart. I was confused by your last comment about tests failing without the wait because we were commenting at the setUp location in the code :) | |
| 19:15:20 | gmaan | yeah, i commented that in wrong place. let me remove it from setUp to avoid confusion in future | |
| 19:15:31 | melwitt | I kept thinking about it like, that doesn't make sense at this location. I understand now, you meant at the restart location | |
| 19:25:42 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Fix graceful shutdown functional unstable tests https://review.opendev.org/c/openstack/nova/+/1002223 | |
| 19:26:29 | gmaan | melwitt: ^^ fixed it and also added notes | |
| 19:26:59 | melwitt | cool thanks! | |
| 21:33:58 | opendevreview | Merged openstack/nova master: perf(ironic): eliminate O(N²) ProviderTree deepcopy at startup https://review.opendev.org/c/openstack/nova/+/980676 | |
| 21:43:03 | opendevreview | Merged openstack/nova master: Add shebang to mark bash files https://review.opendev.org/c/openstack/nova/+/995213 | |
| #openstack-nova - 2026-08-28 | |||
| 00:23:08 | opendevreview | Merged openstack/nova master: Add functional test for cleanup_host cancelling queued live migrations https://review.opendev.org/c/openstack/nova/+/986477 | |
| 01:24:00 | opendevreview | Merged openstack/nova master: Replace OVN Metadata agent with OVN agent https://review.opendev.org/c/openstack/nova/+/987042 | |
| 06:20:22 | opendevreview | Merged openstack/nova master: Fix graceful shutdown functional unstable tests https://review.opendev.org/c/openstack/nova/+/1002223 | |
| 09:24:04 | opendevreview | Merged openstack/nova master: Remove tempest-pg-full https://review.opendev.org/c/openstack/nova/+/996887 | |
| 12:57:02 | tkajinam | bauzas, hi. could you review this test addition for tdx ? I had to update it according to some changes made in the tdx support implementation https://review.opendev.org/c/openstack/nova/+/998525 | |
| 13:23:11 | opendevreview | Daniel Schulz proposed openstack/nova master: Prevent orphaned attachments by preemptive delete https://review.opendev.org/c/openstack/nova/+/1002439 | |
| 13:30:23 | opendevreview | Ivan Anfimov proposed openstack/nova master: Replace outdated usage of egrep by grep https://review.opendev.org/c/openstack/nova/+/1002754 | |
| 13:31:43 | opendevreview | Ivan Anfimov proposed openstack/nova master: Replace outdated usage of egrep by grep https://review.opendev.org/c/openstack/nova/+/1002754 | |
| 15:25:28 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron networks https://review.opendev.org/c/openstack/nova/+/928022 | |
| 15:28:15 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron networks https://review.opendev.org/c/openstack/nova/+/928022 | |
| 15:49:03 | opendevreview | Daniel Schulz proposed openstack/nova master: Prevent orphaned attachments by preemptive delete https://review.opendev.org/c/openstack/nova/+/1002439 | |
| 18:15:31 | opendevreview | Merged openstack/nova master: Shutdown thread pool executors during graceful shutdown https://review.opendev.org/c/openstack/nova/+/1000211 | |
| 18:15:43 | opendevreview | Merged openstack/nova master: Add doc and release notes for the graceful shutdown task tracking https://review.opendev.org/c/openstack/nova/+/997141 | |
| #openstack-nova - 2026-08-29 | |||
| 10:38:12 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Remove errant CONF mocks https://review.opendev.org/c/openstack/nova/+/1002852 | |
| 11:30:45 | opendevreview | Merged openstack/nova master: libvirt: Add functional tests for TDX capability report https://review.opendev.org/c/openstack/nova/+/998525 | |
| 16:47:03 | opendevreview | Udayendu Kar proposed openstack/python-novaclient master: Fixed max-servers handling in host-evacuate-live https://review.opendev.org/c/openstack/python-novaclient/+/1002874 | |
| #openstack-nova - 2026-08-30 | |||
| 00:12:39 | opendevreview | Khoi proposed openstack/nova-specs master: Add availability-zone-scoped quota spec https://review.opendev.org/c/openstack/nova-specs/+/1002901 | |
| 04:59:23 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_clas argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 05:01:43 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_cls argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 05:20:23 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_cls argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 06:00:02 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_clas argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| 15:15:40 | opendevreview | Takashi Kajinami proposed openstack/nova master: ovo: Drop deprecated item_clas argument https://review.opendev.org/c/openstack/nova/+/1002937 | |
| #openstack-nova - 2026-08-31 | |||
| 08:28:25 | gibi | looking at the functional threding job results we are definitely in a better shape now. There is one thing to monitor. I see TIMED_OUT results still after our stabilization fixes landed. And the timeout happnes during the functional test execution. It seems like we have a potentially hanging test case somewhere | |
| 08:29:07 | gibi | I'm not sure why the normal test case timeout does not hit and kill the test case | |
| 08:29:35 | gibi | but given that native threads are not killable from outside I'm not that surprised | |
| 08:29:35 | bauzas | excellent news, thanks gibi | |
| 08:30:55 | stephenfin | sean-k-mooney: Uggla: Would I be able to get some eyes on https://review.opendev.org/c/openstack/nova/+/1002852 today? It's blocking the u-c bump of ksa. gibi has already taken a look (thanks 🙏) | |
| 08:34:25 | sean-k-mooney | done | |
| 08:35:16 | sean-k-mooney | gibi: ya i rechecekd a patch twice over the weekend and it timed out in the functional tests both times | |
| 08:39:41 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron networks https://review.opendev.org/c/openstack/nova/+/928022 | |
| 08:40:54 | gibi | sean-k-mooney: I will take a look | |
| 08:52:05 | sean-k-mooney | test_live_migrate_vifs_from_info_cache | |
| 08:52:11 | sean-k-mooney | failed in the later run | |
| 08:52:17 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/980679 | |
| 08:52:57 | sean-k-mooney | but the job in general timesed out | |
| 08:53:12 | sean-k-mooney | the proir run just timed out in general | |
| 08:53:57 | sean-k-mooney | those were a day and ahalf apart so it may have been before some of the stablisation patches landed | |
| 08:54:34 | sean-k-mooney | sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked | |
| 08:55:03 | sean-k-mooney | i dont knwo if your per db lock tweak landed before that | |
| 08:55:53 | sean-k-mooney | hum it shoudl have | |
| 08:55:59 | stephenfin | sean-k-mooney++ ty | |
| 08:58:00 | gibi | https://bugs.launchpad.net/nova/+bug/2165149 | |
| 08:58:26 | gibi | there are new hits in check pipeline | |
| 08:58:44 | gibi | it might or might not mean that the patches are not rebased yet | |
| 08:59:04 | sean-k-mooney | they have not been rebvased but have been rechecked after the merge | |
| 08:59:45 | opendevreview | Clif Houck proposed openstack/nova master: Parallelize per-node resource updates https://review.opendev.org/c/openstack/nova/+/980679 | |
| 08:59:48 | gibi | does the check pipeline rebases the patch before running the tests? I don't think so | |
| 08:59:58 | sean-k-mooney | i clicked the rebase button in the ui we can see | |
| 09:00:03 | gibi | cool | |
| 09:00:19 | sean-k-mooney | gibi: am it does a speculitive merge with master yes | |
| 09:00:33 | sean-k-mooney | so it wont rebase but it should have the same effect | |
| 09:00:52 | gibi | even the check pipeline? not just the gate? | |
| 09:01:11 | sean-k-mooney | correct | |
| 09:01:31 | gibi | the the locking fix did not fully solved the issue reported in that bug | |
| 09:01:51 | sean-k-mooney | ya so the fix | |
| 09:02:16 | sean-k-mooney | is that we shoudl add database locked to the list of excation that oslo.deb treads as a db deadlock excption | |
| 09:02:26 | sean-k-mooney | then our existing decorators will handel that properly | |
| 09:03:07 | gibi | I'm not sure we want to change the production code due to functional test DB limitations | |
| 09:03:14 | sean-k-mooney | my guess is unless we explcitly flush the db or add a small delay it might take a second for sqlachemy to finsih unlocking the db after the write | |
| 09:03:28 | gibi | could be | |
| 09:03:44 | gibi | sqlite also has some config to retry internall for hits | |
| 09:03:47 | sean-k-mooney | well sqlite is used in production when you use bifrost or standalone ironic | |
| 09:03:51 | gibi | /hits/this/ | |
| 09:04:24 | gibi | they are broken then as our locking is only for functional test | |
| 09:04:44 | sean-k-mooney | well yes but our lockign is for nova :) | |
| 09:04:55 | gibi | ahh so they not using nova with sqlite | |
| 09:04:57 | sean-k-mooney | i mean ironic proper not ironic virt driver | |
| 09:05:16 | gibi | but then I'm not sure how this is relevant :) | |
| 09:05:25 | sean-k-mooney | ya so if you use biforst to manage yoru hardware inventory it uses or can use sqlite as the db | |
| 09:05:51 | sean-k-mooney | the same way that ironic suprot there own jsonrpc implation instead of rabbit for a lighter weight deployment if you want that | |
| 09:06:23 | sean-k-mooney | oh just that the oslo.db change should benift them too | |
| 09:06:26 | gibi | OK, so you say we change oslo.db's list of exceptions to retry on | |
| 09:06:37 | gibi | :) | |
| 09:07:07 | gibi | I still cannot really justify that from nova perspective and the ironic case is something I cannot reproduce | |
| 09:07:08 | sean-k-mooney | i think that would be good in the long term yes but i think we shoudl be able to work aroudn this in the futnctionl tests | |
| 09:07:55 | sean-k-mooney | its just somethign i looked into when sqlite was segfaulting in cybrog | |
| 09:08:44 | sean-k-mooney | as a cleaner way then jsut hte locking | |
| 09:10:08 | sean-k-mooney | but ya if there are other tuneable or we can specficly tell sqlachemy to flush/release the db or something that might be better | |
| 09:10:39 | gibi | https://sqlite.org/c3ref/busy_timeout.html | |
| 09:10:50 | gibi | this could be an SQLite native retry | |
| 09:11:24 | gibi | https://sqlite.org/c3ref/busy_timeout.html | |
| 09:11:33 | gibi | https://sqlite.org/pragma.html#pragma_busy_timeout | |
| 09:13:41 | sean-k-mooney | """The SQLITE_LOCKED result code indicates that a write operation could not continue because of a conflict within the same database connection or a conflict with a different database connection that uses a shared cache. """ | |
| 09:14:20 | sean-k-mooney | so the vs """The SQLITE_BUSY result code indicates that the database file could not be written (or in some cases read) because of concurrent activity by some other database connection, usually a database connection in a separate process. """ | |
| 09:14:30 | opendevreview | Balazs Gibizer proposed openstack/nova master: [functional]Do not patch req-id generation https://review.opendev.org/c/openstack/nova/+/1003024 | |
| 09:14:45 | sean-k-mooney | we are not using processes and we have a per db lock | |
| 09:15:08 | sean-k-mooney | so im no t sure the busy tunabels will help | |
| 09:15:14 | gibi | we use threads | |
| 09:15:18 | sean-k-mooney | are we using the cache file | |
| 09:15:36 | sean-k-mooney | i wonder if we are sharign the cache betwen dbs? | |