Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-11-20
15:26:32 opendevreview Balazs Gibizer proposed openstack/nova master: WIP:Move eventlet specific libvirt event handling https://review.opendev.org/c/openstack/nova/+/967178
15:26:32 opendevreview Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467
15:44:08 opendevreview Balazs Gibizer proposed openstack/nova master: DNM:Test with oslo.vmware eventlet removal patches https://review.opendev.org/c/openstack/nova/+/967487
18:06:20 dansmith melwitt: I just had a thought as I start looking at deployment mode
18:06:43 dansmith melwitt: if I create an instance, snapshot it, and then delete it.. I'm going to lose my TPM key
18:07:05 dansmith since we don't snapshot the TPM with the instance, that's fine now
18:07:35 dansmith but I guess I'm not sure what the long-term intent here is.. maybe we won't ever want to save the TPM with a snapshot for the cloud-fork case, but for the "I snapshot for backup reasons" case it's more relevant...
18:09:10 sean-k-mooney well that woudl be requried if we ever supprot shelve
18:09:17 sean-k-mooney but yes that was one of the opens
18:09:31 sean-k-mooney do we ever supprot snapthosting the tpm and wehre to put the data
18:11:41 sean-k-mooney we brifly dicussed if you woudl also need a api chagne to allwo requesting it to be snapshot and or restored in rebuild because you mihgt want to rebuidl the root disk without regenerating the tpm
18:12:09 dansmith shelve doesn't suffer from us deleting the tpm key in deployment mode, which is what I'm focusing on here
18:13:33 sean-k-mooney well at the moment sicne we dont store the tpm data the fact we delet the tpm key when the vm is deleted is consitent
18:14:22 sean-k-mooney its a problem for the boot, setupm snapshot and boot many form snapshot workflow
18:14:53 sean-k-mooney but that not really in scope of the live mitration feature is it
18:15:52 sean-k-mooney dansmith: are you suggeting in the curernt proposed code rebuilting woudl delete the tpm secrete?
18:16:16 sean-k-mooney i.e. just dong boot, snapshot, rebuild?
18:16:24 sean-k-mooney without the delete step?
18:18:16 dansmith I'm just saying that we're deleting the secret on instance delete in deployment mode but not the others (of course) and that may be surprising for cases like boot, snapshot, delete, boot-from-snapshot, for whatever reason you might do that (cross-nova rebuilding, boot from backup, etc)
18:18:40 dansmith obviously we're not storing the TPM so that's to be expected now, I'm just noting the difference
18:19:09 sean-k-mooney ack that because the secret is owned by nova
18:19:22 sean-k-mooney is the tpm secrete ever deleted in the user case
18:19:36 sean-k-mooney i woudl expect it to be deleted with the vm in the case also
18:19:53 dansmith it can't be
18:20:02 sean-k-mooney why we have the users token
18:20:10 sean-k-mooney for the delete unless an admin did it
18:20:31 dansmith not always.. admin delete, local delete, init_host() finish delete, etc
18:20:51 sean-k-mooney so instead of clenaing it up in the case we can we leak it alwasy?
18:21:06 sean-k-mooney im not saying that a bug but its surprisng to me
18:21:26 dansmith actually, you're right, we are deleting it in those cases, we'll just fail to actually do it in the cases I'm noting
18:21:38 sean-k-mooney so we proably need to document the key deletion semantics clearly for all 3 cases in any event
18:21:40 dansmith melwitt: ^
18:22:43 sean-k-mooney so i know admin cant read the secret
18:22:58 sean-k-mooney but i tought he admin user can list and delete them in barbcan
18:23:04 sean-k-mooney did i imagine that
18:24:18 sean-k-mooney https://github.com/openstack/barbican/blob/master/barbican/common/policies/secrets.py#L41-L49
18:24:23 dansmith maybe? but in local_delete and init_host cleanup we won't even have (or use) that
18:24:41 sean-k-mooney well we do via nova credital
18:24:55 sean-k-mooney i.e. the nova user shoudl have service and admin
18:24:55 dansmith we won't use it, AFAIK
18:25:00 sean-k-mooney oh ok
18:25:07 dansmith ideally it won't forever though
18:25:35 sean-k-mooney maybe we coudl to make it consitent in all cases. we wont have admin you mean? just service right?
18:25:51 dansmith correct
18:25:54 sean-k-mooney allowing the service role to delete a secret might make sense
18:26:07 sean-k-mooney but that is not in there policy today
18:26:31 dansmith eh, idk, if we just make service do everything admins can do that also kinda defeats the point
18:27:05 sean-k-mooney ya i guess im not sure which side fo the ling this falls under
18:28:01 sean-k-mooney in any case it sounds like we cant guarentee cleanup in most cases
18:28:28 sean-k-mooney althogh it will work in a subset so users or admins will need to discuvoer and remove the secrets perodicly
18:28:28 dansmith melwitt: so when you read the backscroll, ignore my line about losing the TPM key only under deployment mode, but we might want to consider/test/fix late deletes and make them use your nova credential to do the delete in cases where we don't have local creds anymore
19:31:39 melwitt dansmith: what do you mean by local creds? libvirt secrets store on the compute in the case of 'host' mode? the proposed code tries to lookup the libvirt secret first and if it doesn't find it, it falls back to pulling it from barbican
19:31:53 melwitt *stored
19:32:08 dansmith melwitt: I meant the nova service user
19:32:35 melwitt ok, the word "local" is confusing me
19:32:56 dansmith melwitt: like, compute was down, user did a delete, when we come backup and finish the local delete we don't have any creds to talk to barbican other than the service user
19:33:21 melwitt ok I see
19:33:55 dansmith probably also for soft/delayed delete
19:34:20 melwitt I'll check through that and add test coverages for it. I remember covering all that when I worked on ephemeral encryption but I'm not sure if I remembered it for this
19:34:34 dansmith ack
19:35:37 melwitt I think the deferred delete is taken care of already but yeah would be best to test specifically for that as well. I'll work on covering all of the late delete possibilities
19:36:13 dansmith okay, I didn't see where that would be, but we have loooots of delete indirection going on, so I'm sure I missed it :)
19:54:57 melwitt dansmith: for the deferred delete stuff it's compartmentalized into the "complete_deletion" methods, like deleting secrets and all that happens in "complete deletion" which does not run until the reclaim periodic task runs which checks for deferred delete expiry and "completes deletion" if the time has expired
19:57:14 melwitt (deferred delete meaning the soft delete API)
20:21:53 dansmith melwitt: right, but if we're in host or user mode we don't have a context with user creds to do the delete in barbican right?
20:23:39 dansmith like if we arrive at that code synchronously with a user- or admin-initiated delete, then we will
20:24:09 dansmith but if we're in that code because we're completing a local delete we won't unless I'm missing something
20:41:27 melwitt dansmith: ok yeah I see what you're saying now
20:43:47 melwitt the problem must be already present with plain vTPM as it is today.. since it's all user owned secrets
21:23:37 Zhan[m] hey team I'm wondering when is the next spec freeze? and it's for 2026.1 or 2026.2? I saw in the agenda that it writes "Soft spec freeze November 20th !".
21:29:05 melwitt Zhan[m]: the soft spec freeze today is for 2026.1 and IIUC it means the last day to propose a new spec to openstack/nova-specs: "After 20 November 2025 (23:59 UTC), no new specs will be accepted." https://releases.openstack.org/gazpacho/schedule.html#g-nova-spec-soft-freeze
21:30:46 melwitt and then Dec 4 is the hard spec freeze meaning the deadline for spec approvals i.e. no more approvals after Dec 4
21:31:31 Zhan[m] melwitt: Thanks for explaining and the link! I guess my spec needs to go to 2026.2 :P
21:36:14 melwitt realistically yes :) if you have it ready to go and can upload within the next 2 hours before 23:59 UTC then you can try
21:36:21 dansmith melwitt: right exactly, nothing new with this series, just thinking we can now fix it since we have your use-the-service-user stuff
21:37:22 melwitt yeah, gotcha
23:22:46 opendevreview Jay Faulkner proposed openstack/nova master: [ironic] Ensure unprovision happens for new states https://review.opendev.org/c/openstack/nova/+/967941
23:24:25 opendevreview Jay Faulkner proposed openstack/nova master: [ironic] Ensure unprovision happens for new states https://review.opendev.org/c/openstack/nova/+/967941
23:35:29 opendevreview Merged openstack/nova master: api: Add response body schemas for security group APIs https://review.opendev.org/c/openstack/nova/+/952973
#openstack-nova - 2025-11-21
08:15:35 gokhan y the VM’s kernel logs report disk I/O errors. In most cases, the issue is resolved by performing a hard reboot of the instance; however, in some cases the filesystem becomes corrupted and requires manual repair (fsck) afterwards. We are looking for guidance on possible root causes and recommended troubleshooting steps for environments using FC-backed Cinder volumes with multipath.
08:15:35 gokhan Hello folks, We are experiencing an intermittent issue affecting some virtual machines running database workloads in our OpenStack environment. Occasionally, these instances temporarily lose access to their attached volumes, resulting in “disk I/O errors” at the guest OS level. The affected volumes are provided via shared storage using Fibre Channel with multipath enabled. We do not see any related errors in the Nova or Cinder logs—onl
10:03:41 ykarel Hi is the kernel panic with test_rebuil_server a known thing? seeing this quite frequently, example https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_034/openstack/034dfb1649dc475c913e56927363ffd2/testr_results.html
10:04:22 opendevreview Taketani Ryo proposed openstack/nova master: mem-enc: fix SEV-specific requirement checks for VMs https://review.opendev.org/c/openstack/nova/+/967970
10:04:22 opendevreview Taketani Ryo proposed openstack/nova master: mem-enc: nova.virt.libvirt.host: introduce Host.supports_mem_encryption https://review.opendev.org/c/openstack/nova/+/967969
10:04:23 opendevreview Taketani Ryo proposed openstack/nova master: mem-enc: Make RP creation independent of specific encryption models https://review.opendev.org/c/openstack/nova/+/967972
10:04:23 opendevreview Taketani Ryo proposed openstack/nova master: mem-enc: remove explicit SEV dependency from nova.scheduler.utils https://review.opendev.org/c/openstack/nova/+/967971
10:04:24 opendevreview Taketani Ryo proposed openstack/nova master: mem-enc: fix requirement checks for mem_encryption guests https://review.opendev.org/c/openstack/nova/+/967974
10:04:24 opendevreview Taketani Ryo proposed openstack/nova master: mem-enc: fix code that assumes SEV in configuring guests https://review.opendev.org/c/openstack/nova/+/967973
10:10:26 jkulik Just stumbled over https://review.opendev.org/c/openstack/nova/+/911374 again. It's a docs correction regarding the default quota driver switch by melwitt. Why didn't we merge that?
10:50:43 opendevreview Balazs Gibizer proposed openstack/nova master: Make RBD Tpool usage conditional https://review.opendev.org/c/openstack/nova/+/956089
10:50:44 opendevreview Balazs Gibizer proposed openstack/nova master: Make image upload tpool usage conditional https://review.opendev.org/c/openstack/nova/+/966505
10:50:44 opendevreview Balazs Gibizer proposed openstack/nova master: Make guestfs' Tpool usage optional https://review.opendev.org/c/openstack/nova/+/965821
10:50:45 opendevreview Balazs Gibizer proposed openstack/nova master: Fix ProviderTree copying with threading Lock https://review.opendev.org/c/openstack/nova/+/956091
10:50:45 opendevreview Balazs Gibizer proposed openstack/nova master: Remove tpool_execute as it is unused https://review.opendev.org/c/openstack/nova/+/966552
10:50:46 opendevreview Balazs Gibizer proposed openstack/nova master: Do not fork compute workers in native threading mode https://review.opendev.org/c/openstack/nova/+/965466
10:50:46 opendevreview Balazs Gibizer proposed openstack/nova master: [test]Further categorization of disabled unit tests https://review.opendev.org/c/openstack/nova/+/956092
10:50:47 opendevreview Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949
10:50:47 opendevreview Balazs Gibizer proposed openstack/nova master: Compute manager to use thread pools selectively https://review.opendev.org/c/openstack/nova/+/966016
10:50:49 opendevreview Balazs Gibizer proposed openstack/nova master: WIP:Move eventlet specific libvirt event handling https://review.opendev.org/c/openstack/nova/+/967178
10:50:49 opendevreview Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467

Earlier   Later