| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-08-13 | |||
| 13:48:10 | sean-k-mooney | while we do not have the ablity to report upstream form our downstrema host we do have the ablity to runa devstack job there in thory | |
| 13:48:21 | sean-k-mooney | so may be we can figoure out a way to do that testing later | |
| 13:48:52 | sean-k-mooney | evertually we might eb able to make that public but we would have to sanitze the logs | |
| 13:49:03 | sean-k-mooney | that the mian blocker currenlty if its a post merge ci | |
| 13:49:11 | sean-k-mooney | i.e. a weekly perodic | |
| 13:50:39 | gibi | I noted the os_secure_boot=optional thing in the doc patch in the TDX series | |
| 13:55:23 | opendevreview | Kamil Sambor proposed openstack/nova master: Replace StaticallyDelayingWrapper with futurist DelayingExecutor https://review.opendev.org/c/openstack/nova/+/997410 | |
| 13:56:47 | gibi | OK I'm cool with the TDX series now. I run some local tests as well with the latest series. | |
| 13:57:01 | gibi | /run/ran/ | |
| 14:29:06 | opendevreview | Anton Iacobaeus proposed openstack/nova master: doc: Add documentation for Intel TDX https://review.opendev.org/c/openstack/nova/+/1000081 | |
| 14:29:07 | opendevreview | Anton Iacobaeus proposed openstack/nova master: libvirt: Add os_firmware_secure https://review.opendev.org/c/openstack/nova/+/1000472 | |
| 14:29:07 | opendevreview | Anton Iacobaeus proposed openstack/nova master: libvirt: reject_mem_enc_instances https://review.opendev.org/c/openstack/nova/+/1000082 | |
| 14:29:08 | opendevreview | Anton Iacobaeus proposed openstack/nova master: libvirt: Check only x86_64 arch for Intel TDX https://review.opendev.org/c/openstack/nova/+/1000805 | |
| 15:32:16 | dansmith | unit tests are giving me "no such option enforce_scope in group [oslo_policy]" errors on several.. gmaan that was a recent change right? tried rebuilding venv, etc but still get it.. what am I missing? | |
| 15:34:05 | gmaan | dansmith: yes, this option is removed. i think recreate venv should fix it but is it nova giving error? | |
| 15:34:30 | gmaan | most of projects are fixed with this but not all. nova/placement etc are fixed alreadt | |
| 15:39:59 | dansmith | yeah, venv rebuild seemed to not.. I figured it was a library version thing | |
| 15:40:00 | dansmith | hrm | |
| 15:41:08 | dansmith | gmaan: is it a version of oslo.policy I need to have or something? | |
| 15:42:10 | gmaan | dansmith: oslo.policy 6.0.0 removed it but nova should not be using it anywhere so it should work any version of oslo.policy | |
| 15:43:00 | dansmith | gmaan: this is an example (one of many) https://paste.opendev.org/show/bdPguy48EU0zoQuirlrk/ | |
| 15:44:00 | dansmith | oslo.policy in the venv is indeed 6.0.0 | |
| 15:44:56 | gmaan | dansmith: ohk and nova seems old one not master https://github.com/openstack/nova/blob/master/nova/tests/unit/policies/base.py#L66-L70 | |
| 15:45:08 | gmaan | is it master and you updated the repo or stable branch? | |
| 15:45:23 | gmaan | old nova with oslo.policy 6.0.0 will not work | |
| 15:45:46 | dansmith | I'm on 3c23390cc8 | |
| 15:45:53 | dansmith | wait a damn second | |
| 15:46:41 | dansmith | lol, okay I did a local clone for a new working tree and I think it doesn't know what actual origin/master is so I'm based on something way old.. wtf | |
| 15:47:40 | gmaan | dansmith: yeah that's too old Jan 26 | |
| 15:47:53 | dansmith | yeah, stupid local shallow clone | |
| 15:47:56 | dansmith | sorry for the noise | |
| 15:50:03 | gmaan | np! I used to have same issue in my multiple local copies and forgot to rebase before change and finds it during git review which adds more work. I tried to stick to have 1 or 2 local working repo now :) | |
| 15:52:07 | dansmith | yeah I almost always work out of one repo, but for something destructive I'm using another.. I did a local clone and so it's (long stale) master branch becomes origin/master in the new clone so I thought I was rebased on master but wasn't | |
| 15:52:37 | dansmith | which is stupid rookie mistake, I just rarely do this so I was on autopilot I guess | |
| 17:04:26 | gmaan | gibi: i updated the thread pool executor refactoring, i replied to this comment, I think there are more chance that different executors creation can happen in parallel compare to before this change where most of them were created during init https://review.opendev.org/c/openstack/nova/+/998571/comment/0f4554fd_8780afaf/ | |
| 17:05:44 | gmaan | that is why I think a another lock is worth here to make them parallel but i do not have much performance data/impact so we can leave that with single lock (serial creation) and can change if any issue we see. let me know and i can update it accordingly | |
| 18:18:35 | opendevreview | melanie witt proposed openstack/nova master: Reproducer for bug 2016173 https://review.opendev.org/c/openstack/nova/+/946222 | |
| 18:18:36 | opendevreview | melanie witt proposed openstack/nova master: Call volume detach rollback API if detach fails https://review.opendev.org/c/openstack/nova/+/880399 | |
| 19:40:09 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Shutdown thread pool executors during graceful shutdown https://review.opendev.org/c/openstack/nova/+/1000211 | |
| 19:55:11 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Add doc and release notes for the graceful shutdown task tracking https://review.opendev.org/c/openstack/nova/+/997141 | |
| 20:03:31 | gmaan | melwitt: i saw you +w on the 2nd change but this base change is also ready, just in case you missed it, but if you are still reviewing that, please ignore this ping :) https://review.opendev.org/c/openstack/nova/+/996299/ | |
| 20:04:23 | melwitt | gmaan: yes, sorry I am mid review still. thanks for checking :) | |
| 20:04:35 | gmaan | melwitt: ohk, thanks | |
| 20:08:55 | gmaan | dansmith: there are other two changes are also ready to complete the graceful shutdown series, https://review.opendev.org/c/openstack/nova/+/1000211 and doc/releasenotes on top of it. | |
| 20:09:52 | gmaan | dansmith: Those are dependent on thread pool executor refactoring which makes the executors shudown easy to handle during graceful shutdown, you are welcome to review the same but just to clarify that it is part of eventlet-removal and not directly to graceful shutdown BP https://review.opendev.org/c/openstack/nova/+/998571 | |
| 20:10:24 | gmaan | that is why i set the different topic for those even they are in same series | |
| 20:10:38 | dansmith | gmaan: yeah I thought gibi had a -1 on the next patch.. is that resolved? | |
| 20:11:14 | gmaan | dansmith: I updated it but gibi have not checked the updated one yet | |
| 20:11:54 | gmaan | i resolved his feedback except one which i replied and waiting for him to check before we need any update there | |
| 20:18:15 | dansmith | ack | |
| 21:37:02 | melwitt | seems like the nova-graceful-shutdown job fails intermittently "Timed out waiting for compute service on npb4f32edc7a464 to be inactive (current: failed)" https://zuul.opendev.org/t/openstack/build/065f1dfcb6604871b71c9b7f84392275 but I don't see any gate-failure bugs open about it | |
| 21:39:21 | gmaan | melwitt: i have one but that is race between some test stopping service and other need it but this looks new to me, checking | |
| 22:03:40 | gmaan | it seems two issue here 1. graceful shutdown is timeout andit seems lot of periodic tasks were running, this is solved in task tracking system where we stop running any new periodic tasks during shutdown | |
| 22:04:22 | gmaan | Aug 13 19:11:12.544060 npb4f32edc7a464 systemd[1]: devstack@n-cpu.service: Failed with result 'exit-code'. | |
| 22:04:22 | gmaan | Aug 13 19:11:12.543995 npb4f32edc7a464 systemd[1]: devstack@n-cpu.service: Main process exited, code=exited, status=1/FAILURE | |
| 22:04:22 | gmaan | Aug 13 19:11:12.515025 npb4f32edc7a464 nova-compute[92418]: INFO oslo_service.backend._threading.service [None req-e086152e-d33a-491b-8053-3d300bc9a11e None None] Graceful shutdown timeout exceeded, instantaneous exiting | |
| 22:04:22 | gmaan | 2, when timeout it seems privsep-helper still running | |
| 22:04:24 | gmaan | Aug 13 19:11:12.544562 npb4f32edc7a464 systemd[1]: devstack@n-cpu.service: Unit process 94748 (privsep-helper) remains running after unit stopped. | |
| 22:04:24 | gmaan | Aug 13 19:11:12.544388 npb4f32edc7a464 systemd[1]: devstack@n-cpu.service: Unit process 93426 (privsep-helper) remains running after unit stopped. | |
| 22:04:26 | gmaan | Aug 13 19:11:12.545602 npb4f32edc7a464 systemd[1]: devstack@n-cpu.service: Consumed 12.751s CPU time, 277.1M memory peak, 0B memory swap peak. | |
| 22:04:42 | gmaan | this seems something new | |
| 22:06:07 | gmaan | thats does not seems the issue, i can see "Unit process 82937 (privsep-helper) remains running after unit stopped." in successful case also | |
| 22:11:26 | gmaan | not sure why service status is updated in DB, because after 180 sec of graceful shutdown, test wait for another 180 sec timeout for serviec to be inactive | |
| 22:17:01 | gmaan | i see, I think its the status update interval time report_interval and service_down_time is racing with test timeout. report_interval = 120 service_down_time = 720 | |
| 22:17:02 | gmaan | https://zuul.opendev.org/t/openstack/build/065f1dfcb6604871b71c9b7f84392275/log/controller/logs/screen-n-cpu.txt#101-115 | |
| 22:17:46 | gmaan | i think i need to update the test script to consider the report_interval and service_down_time for service status update timeout | |
| 22:28:21 | gmaan | melwitt: reported the bug, I will propose fix sometime tomorrow otherwise next week https://bugs.launchpad.net/nova/+bug/2163448 | |
| 22:44:02 | melwitt | thanks for doing that gmaan | |
| #openstack-nova - 2026-08-14 | |||
| 08:29:56 | opendevreview | Rico Lin proposed openstack/nova master: compute: add skip_inventory_in_use_errors opt https://review.opendev.org/c/openstack/nova/+/985750 | |
| 09:12:26 | opendevreview | Bhavana proposed openstack/nova-specs master: Add spec for MIG-aware PCI Host Weigher https://review.opendev.org/c/openstack/nova-specs/+/998753 | |
| 10:08:33 | opendevreview | Kamil Sambor proposed openstack/nova master: Fix pickling error that forces oslo.service to fall back to fork https://review.opendev.org/c/openstack/nova/+/1000956 | |
| 12:09:44 | opendevreview | Pierre Libeau proposed openstack/nova master: libvirt: Add reproducer for unstable vNIC PCI addresses on hard reboot https://review.opendev.org/c/openstack/nova/+/1000961 | |
| 12:09:45 | opendevreview | Pierre Libeau proposed openstack/nova master: libvirt: Preserve vNIC guest PCI address on domain XML regeneration https://review.opendev.org/c/openstack/nova/+/1000963 | |
| 12:09:45 | opendevreview | Pierre Libeau proposed openstack/nova master: libvirt: Render guest interface PCI address in config https://review.opendev.org/c/openstack/nova/+/1000962 | |
| 12:43:40 | plibeau | hi — opinions wanted on https://bugs.launchpad.net/nova/+bug/2163417 : vNIC guest PCI slots get reshuffled on hard reboot (multi-NIC appliances see port1/port2 swapped). nova never pins the VIF guest PCI address, libvirt renumbers from XML order; MAC is the only invariant. reproduced on clean master | |
| 12:45:19 | plibeau | I have a WIP opt-in (3 commits above) [libvirt] preserve_vif_pci_addresses re-pinning by MAC from old_guest (hard reboot/power_on/resume only). is that the right direction, or should the slot be persisted per-VIF? or is "use device role tagging" the intended answer? | |
| 12:47:34 | gibi | plibeau: I think we never promised the guest that the pci slot ordering is stable. Tagging devices are the way to map ports to devices afaik | |
| 12:53:59 | plibeau | yes and linux by default manage that correctly because it use mac address and not pci slot to manage NIC but it's not the case of some and specially appliance. What do you think to try to improve that for this type of use case? | |
| 12:56:21 | gibi | plibeau: I mean you can tag the nic and block devices in the nova api when add it to your server, then the VM can read the tagging from the metadata | |
| 12:56:28 | gibi | that should be OS independent | |
| 12:57:43 | amorin | yes, but some OS are not able to do that, the proposition from plibeau is to try to keep the order as much as possible to avoid order change when it's possible | |
| 13:01:26 | gibi | amorin: you mean there are OSes that are not able to read the metadata API? | |
| 13:01:47 | gibi | or read the metadat from the config drive? | |
| 13:02:00 | amorin | yup | |
| 13:02:10 | amorin | some of our customers complain about this | |
| 13:02:49 | gibi | is it possible that they could read but don't want to? Sorry I'm suspicious here as it is a very basic fuctionality to read a file from an attached disk or do http request | |
| 13:03:40 | amorin | I totally agree with you :) They usually rely on providers to give them images and they seems not able to do so in an easy way | |
| 13:03:54 | amorin | plibeau will have more info, I dont remember the name of the image? | |
| 13:05:11 | gibi | amorin: you can propose a feature (or ask this on the IRC meeting) to start providing a stable order of PCI devices to guests to get the oppinion of others but this was rejected before with the above argument | |
| 13:05:52 | plibeau | Fortinet is the name :) but not only and the customer can't fix also because it's appliance, he don't have root access on it | |
| 13:07:48 | gibi | can this be brougth up to Frotinet then? to integrate with cloud VMs by reading the metadata? :) | |
| 13:09:58 | gibi | I feel like either Fortinet supports to run in OpenStack or isn't :) | |
| 13:15:21 | plibeau | yes, fix on Fortinet side is a solution but we need to fix also probably other case. I think yes Fortinet is supported on openstack I see some documentation how to deploy it above openstack | |
| 13:38:29 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Unify the thread pool executors into a central factory https://review.opendev.org/c/openstack/nova/+/998571 | |
| 13:58:28 | gmaan | gibi: ^^ added the test coverage | |
| 13:59:31 | gibi | gmaan: thanks. I will check it soon | |
| 14:01:33 | gmaan | thanks | |
| 14:01:39 | gibi | +2 | |
| 14:01:42 | gibi | thanks | |
| 14:01:50 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Shutdown thread pool executors during graceful shutdown https://review.opendev.org/c/openstack/nova/+/1000211 | |