| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-11-20 | |||
| 19:32:56 | dansmith | melwitt: like, compute was down, user did a delete, when we come backup and finish the local delete we don't have any creds to talk to barbican other than the service user | |
| 19:33:21 | melwitt | ok I see | |
| 19:33:55 | dansmith | probably also for soft/delayed delete | |
| 19:34:20 | melwitt | I'll check through that and add test coverages for it. I remember covering all that when I worked on ephemeral encryption but I'm not sure if I remembered it for this | |
| 19:34:34 | dansmith | ack | |
| 19:35:37 | melwitt | I think the deferred delete is taken care of already but yeah would be best to test specifically for that as well. I'll work on covering all of the late delete possibilities | |
| 19:36:13 | dansmith | okay, I didn't see where that would be, but we have loooots of delete indirection going on, so I'm sure I missed it :) | |
| 19:54:57 | melwitt | dansmith: for the deferred delete stuff it's compartmentalized into the "complete_deletion" methods, like deleting secrets and all that happens in "complete deletion" which does not run until the reclaim periodic task runs which checks for deferred delete expiry and "completes deletion" if the time has expired | |
| 19:57:14 | melwitt | (deferred delete meaning the soft delete API) | |
| 20:21:53 | dansmith | melwitt: right, but if we're in host or user mode we don't have a context with user creds to do the delete in barbican right? | |
| 20:23:39 | dansmith | like if we arrive at that code synchronously with a user- or admin-initiated delete, then we will | |
| 20:24:09 | dansmith | but if we're in that code because we're completing a local delete we won't unless I'm missing something | |
| 20:41:27 | melwitt | dansmith: ok yeah I see what you're saying now | |
| 20:43:47 | melwitt | the problem must be already present with plain vTPM as it is today.. since it's all user owned secrets | |
| 21:23:37 | Zhan[m] | hey team I'm wondering when is the next spec freeze? and it's for 2026.1 or 2026.2? I saw in the agenda that it writes "Soft spec freeze November 20th !". | |
| 21:29:05 | melwitt | Zhan[m]: the soft spec freeze today is for 2026.1 and IIUC it means the last day to propose a new spec to openstack/nova-specs: "After 20 November 2025 (23:59 UTC), no new specs will be accepted." https://releases.openstack.org/gazpacho/schedule.html#g-nova-spec-soft-freeze | |
| 21:30:46 | melwitt | and then Dec 4 is the hard spec freeze meaning the deadline for spec approvals i.e. no more approvals after Dec 4 | |
| 21:31:31 | Zhan[m] | melwitt: Thanks for explaining and the link! I guess my spec needs to go to 2026.2 :P | |
| 21:36:14 | melwitt | realistically yes :) if you have it ready to go and can upload within the next 2 hours before 23:59 UTC then you can try | |
| 21:36:21 | dansmith | melwitt: right exactly, nothing new with this series, just thinking we can now fix it since we have your use-the-service-user stuff | |
| 21:37:22 | melwitt | yeah, gotcha | |
| 23:22:46 | opendevreview | Jay Faulkner proposed openstack/nova master: [ironic] Ensure unprovision happens for new states https://review.opendev.org/c/openstack/nova/+/967941 | |
| 23:24:25 | opendevreview | Jay Faulkner proposed openstack/nova master: [ironic] Ensure unprovision happens for new states https://review.opendev.org/c/openstack/nova/+/967941 | |
| 23:35:29 | opendevreview | Merged openstack/nova master: api: Add response body schemas for security group APIs https://review.opendev.org/c/openstack/nova/+/952973 | |
| #openstack-nova - 2025-11-21 | |||
| 08:15:35 | gokhan | Hello folks, We are experiencing an intermittent issue affecting some virtual machines running database workloads in our OpenStack environment. Occasionally, these instances temporarily lose access to their attached volumes, resulting in “disk I/O errors” at the guest OS level. The affected volumes are provided via shared storage using Fibre Channel with multipath enabled. We do not see any related errors in the Nova or Cinder logs—onl | |
| 08:15:35 | gokhan | y the VM’s kernel logs report disk I/O errors. In most cases, the issue is resolved by performing a hard reboot of the instance; however, in some cases the filesystem becomes corrupted and requires manual repair (fsck) afterwards. We are looking for guidance on possible root causes and recommended troubleshooting steps for environments using FC-backed Cinder volumes with multipath. | |
| 10:03:41 | ykarel | Hi is the kernel panic with test_rebuil_server a known thing? seeing this quite frequently, example https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_034/openstack/034dfb1649dc475c913e56927363ffd2/testr_results.html | |
| 10:04:22 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: nova.virt.libvirt.host: introduce Host.supports_mem_encryption https://review.opendev.org/c/openstack/nova/+/967969 | |
| 10:04:22 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: fix SEV-specific requirement checks for VMs https://review.opendev.org/c/openstack/nova/+/967970 | |
| 10:04:23 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: remove explicit SEV dependency from nova.scheduler.utils https://review.opendev.org/c/openstack/nova/+/967971 | |
| 10:04:23 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: Make RP creation independent of specific encryption models https://review.opendev.org/c/openstack/nova/+/967972 | |
| 10:04:24 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: fix code that assumes SEV in configuring guests https://review.opendev.org/c/openstack/nova/+/967973 | |
| 10:04:24 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: fix requirement checks for mem_encryption guests https://review.opendev.org/c/openstack/nova/+/967974 | |
| 10:10:26 | jkulik | Just stumbled over https://review.opendev.org/c/openstack/nova/+/911374 again. It's a docs correction regarding the default quota driver switch by melwitt. Why didn't we merge that? | |
| 10:50:43 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make RBD Tpool usage conditional https://review.opendev.org/c/openstack/nova/+/956089 | |
| 10:50:44 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make guestfs' Tpool usage optional https://review.opendev.org/c/openstack/nova/+/965821 | |
| 10:50:44 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make image upload tpool usage conditional https://review.opendev.org/c/openstack/nova/+/966505 | |
| 10:50:45 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove tpool_execute as it is unused https://review.opendev.org/c/openstack/nova/+/966552 | |
| 10:50:45 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix ProviderTree copying with threading Lock https://review.opendev.org/c/openstack/nova/+/956091 | |
| 10:50:46 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]Further categorization of disabled unit tests https://review.opendev.org/c/openstack/nova/+/956092 | |
| 10:50:46 | opendevreview | Balazs Gibizer proposed openstack/nova master: Do not fork compute workers in native threading mode https://review.opendev.org/c/openstack/nova/+/965466 | |
| 10:50:47 | opendevreview | Balazs Gibizer proposed openstack/nova master: Compute manager to use thread pools selectively https://review.opendev.org/c/openstack/nova/+/966016 | |
| 10:50:47 | opendevreview | Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949 | |
| 10:50:49 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 10:50:49 | opendevreview | Balazs Gibizer proposed openstack/nova master: WIP:Move eventlet specific libvirt event handling https://review.opendev.org/c/openstack/nova/+/967178 | |
| 10:52:22 | gibi | stephenfin: thanks for the reviews in the eventlet removal series. I fixed / answered them ^^ | |
| 10:53:14 | gibi | stephenfin: wait a bit | |
| 10:53:21 | gibi | I made a rebase mistake :/ | |
| 10:54:19 | stephenfin | gibi: ack. Also, replied on https://review.opendev.org/c/openstack/nova/+/956089/comment/caec94ed_4fdb16bf/ | |
| 10:58:01 | stephenfin | gibi: also, on https://review.opendev.org/c/openstack/nova/+/966505/comment/b6cfa860_58bb6342/, I'm still confused, sorry :) | |
| 10:58:28 | stephenfin | My point was that previously we were calling 'tpool.execute(target)' and now we're calling 'tpool.Proxy(target)' (but only if running in eventlet mode). I'm asking if tpool.execute($thing) is the same as tpool.Proxy($thing) | |
| 10:59:01 | stephenfin | ...and not what the rationale for only wrapping it it something in eventlet mode was. Sorry if I was unclear | |
| 10:59:42 | gibi | what we do now is tpool.Proxy($thing)(args, kwargs) | |
| 10:59:50 | gibi | so we wrap and call the wrapped | |
| 11:00:11 | gibi | and this has the same effect as the execute in the past | |
| 11:00:56 | gibi | because any call on the Proxy object will be done on the native thread of the tpool. Similarly how the tpool.execute executed the call directly on the tpool native thread | |
| 11:01:16 | stephenfin | Sorry for being slow /o\ but we seem to be calling tpool.execute, not tpool.Proxy? | |
| 11:01:19 | stephenfin | a: https://github.com/openstack/nova/blob/master/nova/image/glance.py#L590 | |
| 11:01:24 | stephenfin | b: https://github.com/openstack/nova/blob/master/nova/utils.py#L634 | |
| 11:02:14 | stephenfin | oh, wait, so tpool.Proxy had a __call__ dunder, and tpool.execute is an alias? | |
| 11:02:16 | gibi | yes. In the past we passed a func to the tpool.execute. That run the func on native thread of the tpool | |
| 11:02:27 | gibi | I'm not sure about the impl | |
| 11:02:39 | gibi | but any call on the Proxy object will be run in the tpool | |
| 11:02:45 | gibi | that is the contract | |
| 11:03:10 | stephenfin | okay, I'm still not getting it but you seem to understand my confusion and I trust you've done the right thing. Sorry for the noise! | |
| 11:04:05 | gibi | https://github.com/eventlet/eventlet/blob/8cb20296f3455a1836cdbcbf1d3545666ee7f867/eventlet/tpool.py#L207-L211 | |
| 11:04:13 | gibi | indeed it has a __call__ | |
| 11:04:42 | gibi | so yes on the impl question as well | |
| 11:04:48 | stephenfin | yup, okay, that checks out. Thanks for confirming 🙏 | |
| 11:05:20 | gibi | thanks for the review. I will refine the commit message based on this | |
| 11:05:33 | gibi | (I just have to redo one unintended squash in my rebase) | |
| 11:13:05 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make RBD Tpool usage conditional https://review.opendev.org/c/openstack/nova/+/956089 | |
| 11:13:05 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make libvirt Tpool proxying conditional https://review.opendev.org/c/openstack/nova/+/956090 | |
| 11:13:06 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make guestfs' Tpool usage optional https://review.opendev.org/c/openstack/nova/+/965821 | |
| 11:13:06 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make image upload tpool usage conditional https://review.opendev.org/c/openstack/nova/+/966505 | |
| 11:13:07 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove tpool_execute as it is unused https://review.opendev.org/c/openstack/nova/+/966552 | |
| 11:13:07 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix ProviderTree copying with threading Lock https://review.opendev.org/c/openstack/nova/+/956091 | |
| 11:13:08 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]Further categorization of disabled unit tests https://review.opendev.org/c/openstack/nova/+/956092 | |
| 11:13:09 | opendevreview | Balazs Gibizer proposed openstack/nova master: Do not fork compute workers in native threading mode https://review.opendev.org/c/openstack/nova/+/965466 | |
| 11:13:09 | opendevreview | Balazs Gibizer proposed openstack/nova master: Compute manager to use thread pools selectively https://review.opendev.org/c/openstack/nova/+/966016 | |
| 11:13:11 | opendevreview | Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949 | |
| 11:13:11 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 11:13:13 | opendevreview | Balazs Gibizer proposed openstack/nova master: WIP:Move eventlet specific libvirt event handling https://review.opendev.org/c/openstack/nova/+/967178 | |
| 11:13:53 | gibi | OK I think I managed to restore sanity in the seriese | |
| 11:20:38 | jkulik | hm ... I don't seem to be able to run unit-tests (tox -e py) on nova master anymore unless I specify OS_NOVA_DISABLE_EVENTLET_PATCHING (with either true or false) | |
| 11:24:06 | jkulik | https://paste.opendev.org/show/b8uHX1hkkuwNJnQhni4L/ is the error I get | |
| 11:29:02 | opendevreview | Johannes Kulik proposed openstack/nova master: Enhance performance of get instance groups from DB https://review.opendev.org/c/openstack/nova/+/967977 | |
| 11:30:42 | jkulik | sean-k-mooney: ^ the other commit for server group performance. I'd appreciate a review. | |
| 11:37:32 | sean-k-mooney | jkulik: ah cool and for tox its "tox -e py3" | |
| 11:37:56 | sean-k-mooney | tox -e unit might also work i dont recally if i did that in nova or not | |
| 11:39:00 | jkulik | oh, -e py3 works. thanks. -e py worked until a couple of weeks ago | |
| 11:39:07 | sean-k-mooney | https://github.com/openstack/nova/blob/master/tox.ini#L59 | |
| 11:40:02 | sean-k-mooney | jkulik: -e py was never officlaly supported, that ahs not changed in our config in a long time but if a env starts with py | |
| 11:40:03 | jkulik | yeah, took a look already. not sure why I thought it should be py. env variable is set when using one of these testenvs, yes | |
| 11:40:15 | sean-k-mooney | tox will invent it nd run the command form the base target | |
| 11:40:28 | sean-k-mooney | https://github.com/openstack/nova/blob/master/tox.ini#L54-L57 | |
| 11:40:33 | sean-k-mooney | which happens to be our unit tests | |
| 11:40:45 | sean-k-mooney | my guess is this behvior has changed in a new version of tox | |
| 11:41:12 | sean-k-mooney | we coudl trivially add py | |
| 11:41:40 | jkulik | I can easily adapt :D | |