| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-11-20 | |||
| 18:24:55 | dansmith | we won't use it, AFAIK | |
| 18:25:00 | sean-k-mooney | oh ok | |
| 18:25:07 | dansmith | ideally it won't forever though | |
| 18:25:35 | sean-k-mooney | maybe we coudl to make it consitent in all cases. we wont have admin you mean? just service right? | |
| 18:25:51 | dansmith | correct | |
| 18:25:54 | sean-k-mooney | allowing the service role to delete a secret might make sense | |
| 18:26:07 | sean-k-mooney | but that is not in there policy today | |
| 18:26:31 | dansmith | eh, idk, if we just make service do everything admins can do that also kinda defeats the point | |
| 18:27:05 | sean-k-mooney | ya i guess im not sure which side fo the ling this falls under | |
| 18:28:01 | sean-k-mooney | in any case it sounds like we cant guarentee cleanup in most cases | |
| 18:28:28 | sean-k-mooney | althogh it will work in a subset so users or admins will need to discuvoer and remove the secrets perodicly | |
| 18:28:28 | dansmith | melwitt: so when you read the backscroll, ignore my line about losing the TPM key only under deployment mode, but we might want to consider/test/fix late deletes and make them use your nova credential to do the delete in cases where we don't have local creds anymore | |
| 19:31:39 | melwitt | dansmith: what do you mean by local creds? libvirt secrets store on the compute in the case of 'host' mode? the proposed code tries to lookup the libvirt secret first and if it doesn't find it, it falls back to pulling it from barbican | |
| 19:31:53 | melwitt | *stored | |
| 19:32:08 | dansmith | melwitt: I meant the nova service user | |
| 19:32:35 | melwitt | ok, the word "local" is confusing me | |
| 19:32:56 | dansmith | melwitt: like, compute was down, user did a delete, when we come backup and finish the local delete we don't have any creds to talk to barbican other than the service user | |
| 19:33:21 | melwitt | ok I see | |
| 19:33:55 | dansmith | probably also for soft/delayed delete | |
| 19:34:20 | melwitt | I'll check through that and add test coverages for it. I remember covering all that when I worked on ephemeral encryption but I'm not sure if I remembered it for this | |
| 19:34:34 | dansmith | ack | |
| 19:35:37 | melwitt | I think the deferred delete is taken care of already but yeah would be best to test specifically for that as well. I'll work on covering all of the late delete possibilities | |
| 19:36:13 | dansmith | okay, I didn't see where that would be, but we have loooots of delete indirection going on, so I'm sure I missed it :) | |
| 19:54:57 | melwitt | dansmith: for the deferred delete stuff it's compartmentalized into the "complete_deletion" methods, like deleting secrets and all that happens in "complete deletion" which does not run until the reclaim periodic task runs which checks for deferred delete expiry and "completes deletion" if the time has expired | |
| 19:57:14 | melwitt | (deferred delete meaning the soft delete API) | |
| 20:21:53 | dansmith | melwitt: right, but if we're in host or user mode we don't have a context with user creds to do the delete in barbican right? | |
| 20:23:39 | dansmith | like if we arrive at that code synchronously with a user- or admin-initiated delete, then we will | |
| 20:24:09 | dansmith | but if we're in that code because we're completing a local delete we won't unless I'm missing something | |
| 20:41:27 | melwitt | dansmith: ok yeah I see what you're saying now | |
| 20:43:47 | melwitt | the problem must be already present with plain vTPM as it is today.. since it's all user owned secrets | |
| 21:23:37 | Zhan[m] | hey team I'm wondering when is the next spec freeze? and it's for 2026.1 or 2026.2? I saw in the agenda that it writes "Soft spec freeze November 20th !". | |
| 21:29:05 | melwitt | Zhan[m]: the soft spec freeze today is for 2026.1 and IIUC it means the last day to propose a new spec to openstack/nova-specs: "After 20 November 2025 (23:59 UTC), no new specs will be accepted." https://releases.openstack.org/gazpacho/schedule.html#g-nova-spec-soft-freeze | |
| 21:30:46 | melwitt | and then Dec 4 is the hard spec freeze meaning the deadline for spec approvals i.e. no more approvals after Dec 4 | |
| 21:31:31 | Zhan[m] | melwitt: Thanks for explaining and the link! I guess my spec needs to go to 2026.2 :P | |
| 21:36:14 | melwitt | realistically yes :) if you have it ready to go and can upload within the next 2 hours before 23:59 UTC then you can try | |
| 21:36:21 | dansmith | melwitt: right exactly, nothing new with this series, just thinking we can now fix it since we have your use-the-service-user stuff | |
| 21:37:22 | melwitt | yeah, gotcha | |
| 23:22:46 | opendevreview | Jay Faulkner proposed openstack/nova master: [ironic] Ensure unprovision happens for new states https://review.opendev.org/c/openstack/nova/+/967941 | |
| 23:24:25 | opendevreview | Jay Faulkner proposed openstack/nova master: [ironic] Ensure unprovision happens for new states https://review.opendev.org/c/openstack/nova/+/967941 | |
| 23:35:29 | opendevreview | Merged openstack/nova master: api: Add response body schemas for security group APIs https://review.opendev.org/c/openstack/nova/+/952973 | |
| #openstack-nova - 2025-11-21 | |||
| 08:15:35 | gokhan | y the VM’s kernel logs report disk I/O errors. In most cases, the issue is resolved by performing a hard reboot of the instance; however, in some cases the filesystem becomes corrupted and requires manual repair (fsck) afterwards. We are looking for guidance on possible root causes and recommended troubleshooting steps for environments using FC-backed Cinder volumes with multipath. | |
| 08:15:35 | gokhan | Hello folks, We are experiencing an intermittent issue affecting some virtual machines running database workloads in our OpenStack environment. Occasionally, these instances temporarily lose access to their attached volumes, resulting in “disk I/O errors” at the guest OS level. The affected volumes are provided via shared storage using Fibre Channel with multipath enabled. We do not see any related errors in the Nova or Cinder logs—onl | |
| 10:03:41 | ykarel | Hi is the kernel panic with test_rebuil_server a known thing? seeing this quite frequently, example https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_034/openstack/034dfb1649dc475c913e56927363ffd2/testr_results.html | |
| 10:04:22 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: fix SEV-specific requirement checks for VMs https://review.opendev.org/c/openstack/nova/+/967970 | |
| 10:04:22 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: nova.virt.libvirt.host: introduce Host.supports_mem_encryption https://review.opendev.org/c/openstack/nova/+/967969 | |
| 10:04:23 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: Make RP creation independent of specific encryption models https://review.opendev.org/c/openstack/nova/+/967972 | |
| 10:04:23 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: remove explicit SEV dependency from nova.scheduler.utils https://review.opendev.org/c/openstack/nova/+/967971 | |
| 10:04:24 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: fix requirement checks for mem_encryption guests https://review.opendev.org/c/openstack/nova/+/967974 | |
| 10:04:24 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: fix code that assumes SEV in configuring guests https://review.opendev.org/c/openstack/nova/+/967973 | |
| 10:10:26 | jkulik | Just stumbled over https://review.opendev.org/c/openstack/nova/+/911374 again. It's a docs correction regarding the default quota driver switch by melwitt. Why didn't we merge that? | |
| 10:50:43 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make RBD Tpool usage conditional https://review.opendev.org/c/openstack/nova/+/956089 | |
| 10:50:44 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make image upload tpool usage conditional https://review.opendev.org/c/openstack/nova/+/966505 | |
| 10:50:44 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make guestfs' Tpool usage optional https://review.opendev.org/c/openstack/nova/+/965821 | |
| 10:50:45 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix ProviderTree copying with threading Lock https://review.opendev.org/c/openstack/nova/+/956091 | |
| 10:50:45 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove tpool_execute as it is unused https://review.opendev.org/c/openstack/nova/+/966552 | |
| 10:50:46 | opendevreview | Balazs Gibizer proposed openstack/nova master: Do not fork compute workers in native threading mode https://review.opendev.org/c/openstack/nova/+/965466 | |
| 10:50:46 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]Further categorization of disabled unit tests https://review.opendev.org/c/openstack/nova/+/956092 | |
| 10:50:47 | opendevreview | Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949 | |
| 10:50:47 | opendevreview | Balazs Gibizer proposed openstack/nova master: Compute manager to use thread pools selectively https://review.opendev.org/c/openstack/nova/+/966016 | |
| 10:50:49 | opendevreview | Balazs Gibizer proposed openstack/nova master: WIP:Move eventlet specific libvirt event handling https://review.opendev.org/c/openstack/nova/+/967178 | |
| 10:50:49 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 10:52:22 | gibi | stephenfin: thanks for the reviews in the eventlet removal series. I fixed / answered them ^^ | |
| 10:53:14 | gibi | stephenfin: wait a bit | |
| 10:53:21 | gibi | I made a rebase mistake :/ | |
| 10:54:19 | stephenfin | gibi: ack. Also, replied on https://review.opendev.org/c/openstack/nova/+/956089/comment/caec94ed_4fdb16bf/ | |
| 10:58:01 | stephenfin | gibi: also, on https://review.opendev.org/c/openstack/nova/+/966505/comment/b6cfa860_58bb6342/, I'm still confused, sorry :) | |
| 10:58:28 | stephenfin | My point was that previously we were calling 'tpool.execute(target)' and now we're calling 'tpool.Proxy(target)' (but only if running in eventlet mode). I'm asking if tpool.execute($thing) is the same as tpool.Proxy($thing) | |
| 10:59:01 | stephenfin | ...and not what the rationale for only wrapping it it something in eventlet mode was. Sorry if I was unclear | |
| 10:59:42 | gibi | what we do now is tpool.Proxy($thing)(args, kwargs) | |
| 10:59:50 | gibi | so we wrap and call the wrapped | |
| 11:00:11 | gibi | and this has the same effect as the execute in the past | |
| 11:00:56 | gibi | because any call on the Proxy object will be done on the native thread of the tpool. Similarly how the tpool.execute executed the call directly on the tpool native thread | |
| 11:01:16 | stephenfin | Sorry for being slow /o\ but we seem to be calling tpool.execute, not tpool.Proxy? | |
| 11:01:19 | stephenfin | a: https://github.com/openstack/nova/blob/master/nova/image/glance.py#L590 | |
| 11:01:24 | stephenfin | b: https://github.com/openstack/nova/blob/master/nova/utils.py#L634 | |
| 11:02:14 | stephenfin | oh, wait, so tpool.Proxy had a __call__ dunder, and tpool.execute is an alias? | |
| 11:02:16 | gibi | yes. In the past we passed a func to the tpool.execute. That run the func on native thread of the tpool | |
| 11:02:27 | gibi | I'm not sure about the impl | |
| 11:02:39 | gibi | but any call on the Proxy object will be run in the tpool | |
| 11:02:45 | gibi | that is the contract | |
| 11:03:10 | stephenfin | okay, I'm still not getting it but you seem to understand my confusion and I trust you've done the right thing. Sorry for the noise! | |
| 11:04:05 | gibi | https://github.com/eventlet/eventlet/blob/8cb20296f3455a1836cdbcbf1d3545666ee7f867/eventlet/tpool.py#L207-L211 | |
| 11:04:13 | gibi | indeed it has a __call__ | |
| 11:04:42 | gibi | so yes on the impl question as well | |
| 11:04:48 | stephenfin | yup, okay, that checks out. Thanks for confirming 🙏 | |
| 11:05:20 | gibi | thanks for the review. I will refine the commit message based on this | |
| 11:05:33 | gibi | (I just have to redo one unintended squash in my rebase) | |
| 11:13:05 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make libvirt Tpool proxying conditional https://review.opendev.org/c/openstack/nova/+/956090 | |
| 11:13:05 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make RBD Tpool usage conditional https://review.opendev.org/c/openstack/nova/+/956089 | |
| 11:13:06 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make image upload tpool usage conditional https://review.opendev.org/c/openstack/nova/+/966505 | |
| 11:13:06 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make guestfs' Tpool usage optional https://review.opendev.org/c/openstack/nova/+/965821 | |
| 11:13:07 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix ProviderTree copying with threading Lock https://review.opendev.org/c/openstack/nova/+/956091 | |
| 11:13:07 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove tpool_execute as it is unused https://review.opendev.org/c/openstack/nova/+/966552 | |
| 11:13:08 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]Further categorization of disabled unit tests https://review.opendev.org/c/openstack/nova/+/956092 | |
| 11:13:09 | opendevreview | Balazs Gibizer proposed openstack/nova master: Compute manager to use thread pools selectively https://review.opendev.org/c/openstack/nova/+/966016 | |
| 11:13:09 | opendevreview | Balazs Gibizer proposed openstack/nova master: Do not fork compute workers in native threading mode https://review.opendev.org/c/openstack/nova/+/965466 | |
| 11:13:11 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 11:13:11 | opendevreview | Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949 | |
| 11:13:13 | opendevreview | Balazs Gibizer proposed openstack/nova master: WIP:Move eventlet specific libvirt event handling https://review.opendev.org/c/openstack/nova/+/967178 | |
| 11:13:53 | gibi | OK I think I managed to restore sanity in the seriese | |