| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-10-08 | |||
| 09:01:07 | cw0306-lee[m] | ||
| 09:01:56 | gibi | bauzas: I'm not aware of that | |
| 09:02:02 | bauzas | ack | |
| 09:02:23 | bauzas | my first triage of the last failure was incorrect, I'll add a better gerrit comment | |
| 09:03:06 | bauzas | good news is that I failed three times with three different causes, none of them being related to threading apparently :) | |
| 09:03:32 | gibi | bauzas: you might remember this https://review.opendev.org/c/openstack/nova/+/966446 that prevented the libvirt events to arrive. but your last failure was due to slow re-image and nova timing out waiting for the event from cinder not from libvirt | |
| 09:04:01 | gibi | btw re-image succeeded but after novas timeout | |
| 09:08:40 | bauzas | yeah that's what I figured | |
| 09:09:01 | bauzas | cinder took a while to reimage, nova timed out | |
| 10:33:00 | frickler | fyi just saw a gate failure with two rebuild tests in reqs gate, didn't look into the details, but might be related? https://zuul.opendev.org/t/openstack/build/69448ba8006646e2857f42c6449434e3 | |
| 11:21:18 | gibi | frickler: at least the symptomps are different, in your case the ssh to the guest failed after nova declared the rebuild successfull, in bauzas case nova failed to finish the rebuild | |
| 11:22:37 | gibi | for some reason in your case the tempest test does not dump the guest console log after the failure that would help diagnosing if it is a guest slow startup / kernel crash or something else | |
| 11:24:13 | gibi | I don't see your failure happening frequently in that job on the req repo so I assume it is a one off slowness the worker node | |
| 11:24:32 | gibi | let us know if it repeates | |
| 11:27:56 | frickler | yeah, usually I just recheck these and ignore if they don't repeat, I just though this might match the earlier discussion. thx for checking | |
| 11:29:09 | gibi | thanks for reporting | |
| 11:44:20 | sean-k-mooney | frickler: as an asied there has been no feedback or movement of any kind on https://github.com/cirros-dev/cirros/pull/139 or the other prs/issue i opened a few months ago | |
| 11:45:01 | sean-k-mooney | frickler: its possibel rebuild issue was related to some of those filesystem copy and sync bugs | |
| 11:45:10 | sean-k-mooney | but ya if there isnt a console log its hard to tell | |
| 11:45:53 | sean-k-mooney | for example https://github.com/cirros-dev/cirros/pull/139/changes/d1266a2d73b23fea2d30bb00ef21b645953039e1 | |
| 11:46:02 | sean-k-mooney | may have helped. | |
| 11:47:44 | sean-k-mooney | rebuild shoudl cause all the the cirror first both logic to run again so that not a smoking gun in this case but ya had to say unless we can actully see what happend | |
| 12:03:52 | opendevreview | Nathan proposed openstack/nova master: libvirt: Attach disk.config.rescue in stable device rescue https://review.opendev.org/c/openstack/nova/+/1009444 | |
| 12:28:22 | frickler | sean-k-mooney: yeah, sadly no real feedback from smoser. I'm not sure yet how to proceed, either simply publish on github and ignore the download site or do a hard fork with a new name | |
| 13:49:40 | stephenfin | sean-k-mooney: melwitt: You previously reviewed my "migrate cinder client config to ksa opts" patch. I've since broken it up into a few smaller patches after review from Mel. I'd appreciate it if you could take a second spin through it before PTG chaos next week, since it unblocks future work to replace cinderclient (and helps with the neutronclient removal also: see last patch) | |
| 13:50:09 | stephenfin | the series starts here https://review.opendev.org/c/openstack/nova/+/1003174/ There are a lot of patches but they're all tightly focused. Happy to trade reviews, as always | |
| 15:17:36 | opendevreview | Konrad Gube proposed openstack/nova master: Add assisted volume extend API. https://review.opendev.org/c/openstack/nova/+/1009077 | |
| 16:28:51 | opendevreview | Merged openstack/nova master: [test]Ensure shutdown thread is waited on https://review.opendev.org/c/openstack/nova/+/1003245 | |
| 17:17:29 | Uggla | Hi dansmith, I would like you thoughts on https://review.opendev.org/c/openstack/nova/+/1006036/comment/316111f8_64b2c4aa if you have a moment. | |
| 17:23:41 | opendevreview | Dan Smith proposed openstack/nova-specs master: Add cross-cell-live-migration spec https://review.opendev.org/c/openstack/nova-specs/+/1008081 | |
| 17:38:43 | dansmith | Uggla: done | |
| 17:47:48 | frickler | weird uwsgi coredump when starting up n-api ... https://zuul.opendev.org/t/openstack/build/98635793ea6c461592a3953ff768fdb2 | |
| 19:19:18 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Add tests for race between RPC thread executor and manager executors https://review.opendev.org/c/openstack/nova/+/1009630 | |
| 19:19:38 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Add tests for race between RPC thread executor and manager executors https://review.opendev.org/c/openstack/nova/+/1009630 | |
| 20:11:32 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Add tests for race between RPC thread executor and manager executors https://review.opendev.org/c/openstack/nova/+/1009630 | |
| 20:11:52 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Add tests for race between RPC thread executor and manager executors https://review.opendev.org/c/openstack/nova/+/1009630 | |
| #openstack-nova - 2026-10-09 | |||
| 11:59:56 | lajoskatona | Uggla: Hi, our downstream team recently run into this (or to a very similar issue): https://redhat.atlassian.net/browse/RHEL-48801 (thanks gibi for pointing to it) | |
| 12:00:47 | lajoskatona | Uggla: I am not a nova->libvirt->qemu chain specialist but with some reading I started to believe that there is a fix: https://gitlab.com/qemu-project/qemu/-/commit/3214bec13d8d4c40f707d21d8350d04e4123ae97 in qemu | |
| 12:01:36 | lajoskatona | at least Qemu folks did something they believe is related to the paused state issues (fix for assertions during paused migrations; rework of late-block-activate logic (#2395, #686)) | |
| 12:02:36 | lajoskatona | It was released in Qemu v10.0.0, but currently Nova runs between 6.2.0 and 8.2.2 | |
| 12:04:08 | lajoskatona | Uggla: after the long story: wehat do you think about opening a Nova bug, "fix it" with a doc mentioning that it is a qemu bug, and when qemu 10.0.0 is in the nova support matrix, remove the warning from the doc? | |
| 12:04:33 | lajoskatona | I checked this support page: https://docs.openstack.org/nova/latest/reference/libvirt-distro-support-matrix.html | |
| 12:24:45 | opendevreview | Lucas HENRY proposed openstack/nova master: Shut down executors after the RPC servers drain https://review.opendev.org/c/openstack/nova/+/1009164 | |
| 12:29:02 | na-te | hi, could someone take a look at https://review.opendev.org/c/openstack/nova/+/1009444 (libvirt rescue fix), CI is green | |
| 12:36:42 | na-te | * hi, could someone take a look at https://review.opendev.org/c/openstack/nova/+/1009444 (libvirt rescue fix), CI is green, thanks in advance ! | |
| 13:13:12 | Uggla | Hi lajoskatona, reading back but that sounds good. | |
| 13:27:42 | lajoskatona | Uggla: so shall I open a bug to track it in Nova launchpad? as I see with ubuntu26 (https://review.opendev.org/c/openstack/governance/+/996925/1/reference/runtimes/2027.1.rst ) we will have qemu 10+ for Indri, but I also have to test it myselv to see if the find in gitlog is really fixing the issue with pause | |
| 13:36:12 | Uggla | lajoskatona, yes please. | |
| 13:36:27 | lajoskatona | Uggla: ack | |
| 14:16:38 | opendevreview | sean mooney proposed openstack/os-vif stable/2026.1: ovs: Pass ``multi_queue`` inside ``ifr`` dict for tuntap devices https://review.opendev.org/c/openstack/os-vif/+/1009778 | |
| 14:19:07 | dansmith | lajoskatona: do you know for sure the fix won't be backported by distros? | |
| 14:19:54 | dansmith | it seems weird to me for us to maintain a bug pointer like that, unless it's a fundamental pitfall | |
| 15:25:40 | lajoskatona | dansmith: good question, I am not sure, for example in my test ubuntu24.04 it seems it is not backported. | |
| 15:26:59 | dansmith | point being we could mirror all the potential bugs in our docs, but it seems like a maintenance issue for us and I'm not sure why this one would warrant it necessarily | |
| 15:27:02 | lajoskatona | dansmith: I wait anyway as on the above mentioned devstack I was not able to reproduce, so I have to check how our downstream team run into it and understand better the bugzilla also | |
| 15:27:46 | dansmith | we previously say things like "multifd with postcopy is not supported in <10.0" but that's sort of a thing we are telling them not to turn on in nova until it's supported.. a bug like this could be different across distros | |
| 17:32:09 | sean-k-mooney | i guess they have left but the other thing to note is that nova does not have a max libvir/qemu or a supproted range | |
| 17:32:16 | sean-k-mooney | we only have a min and next min | |
| 17:32:39 | sean-k-mooney | where next min is just telling distro and oepratroat what we wil move too for plannign reaons | |
| 17:33:16 | sean-k-mooney | so qemu 10 is supproted by nova today | |
| 17:33:35 | sean-k-mooney | and it was since it was relesaed and shiped in a supproted distro | |
| 17:36:58 | sean-k-mooney | lajoskatona ^ if you read scroll back later vai the logs. basiclly the idea that "Nova supprot between 6.2.0 and 8.2.2" is invliad | |
| 17:43:05 | sean-k-mooney | so tl;dr i was invovled in digagnoing that oring issue and advicsed agaisn opening a nova bug for this or mentioning it in our docs as this was a qemu issue nto a nova one and i dont think we shoudl include it in our docs | |
| 17:43:53 | eduardomorais[m] | Hey everyone, I'm having an issue with my OpenStack. I'm using kolla-ansible 2024.1, and when I try to create a VM through the Horizon dashboard, it gets stuck in an infinite 'spawning' state. However, when I create it via the CLI, it works perfectly fine. Has anyone seen anything like this before? | |
| 17:46:51 | sean-k-mooney | are they both created as the saem user with the same parmaters. i have had issue where for exmple the default in horizon diveraged form nova i..e using BFV by defualt or simialr | |
| 17:47:24 | sean-k-mooney | in general no i the request is the saem there shoudl be 0 delta as nova does not actully know if its coming form horizon or not | |
| 17:47:36 | eduardomorais[m] | same user, flavor, network and image | |
| 17:48:16 | sean-k-mooney | and both are using boot form image using nova provisioned storage? | |
| 17:49:36 | sean-k-mooney | as i said there is a config option to make booting form a cinder volume created form a glance image the default in horizon and i think that is the default at least in devstack when cinder is deployed | |
| 17:51:53 | eduardomorais[m] | I created both of them using ephemeral disks. When I try to create them through the dashboard using a volume, Cinder times out; however, it is working for everything else and there are no error logs, which makes me believe it is not the problem. | |
| 17:53:04 | sean-k-mooney | os that the only casie i have had that happen thace comes to mind. horizong cna have some other defult liek cofnig driver or inlcuding a ssh key by default but neither shoudl cause wahat you describe | |
| 17:53:17 | sean-k-mooney | eduardomorais[m]: i woudl propably set the nova api/service into debug mode | |
| 17:53:39 | sean-k-mooney | and then create a vm with horizon then find the request id and see if you can compare that to a cli request | |
| 17:54:03 | eduardomorais[m] | ok, i will try that, thx | |
| 17:54:40 | sean-k-mooney | if its getting stuck in spawning its proably gettin gstuck in the comptue agent somewhere but again that only makes sense to me if the api request were somehow differnt | |
| 17:55:32 | eduardomorais[m] | ok | |
| 19:01:00 | opendevreview | Merged openstack/placement master: Replace license classifier https://review.opendev.org/c/openstack/placement/+/1004422 | |
| 22:37:41 | opendevreview | Stephen Finucane proposed openstack/nova master: Replace use of datetime.timezone.UTC https://review.opendev.org/c/openstack/nova/+/989515 | |
| 22:37:41 | opendevreview | Stephen Finucane proposed openstack/nova master: trivial: Fix outstanding flake8 issues https://review.opendev.org/c/openstack/nova/+/999714 | |
| 22:37:42 | opendevreview | Stephen Finucane proposed openstack/nova master: Add ruff-check https://review.opendev.org/c/openstack/nova/+/974441 | |
| 22:37:42 | opendevreview | Stephen Finucane proposed openstack/nova master: Enable G (flake8-logging-format) rules https://review.opendev.org/c/openstack/nova/+/998557 | |
| 22:37:43 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Fix DB migration URL test for SQLAlchemy 2.1+ https://review.opendev.org/c/openstack/nova/+/1009877 | |
| 22:52:01 | opendevreview | Merged openstack/nova master: Add reproducer for bug #2164869 https://review.opendev.org/c/openstack/nova/+/1006717 | |
| #openstack-nova - 2026-10-10 | |||
| 00:05:54 | opendevreview | Merged openstack/nova master: Fix metadata subpath queries returning 500 instead of 404 https://review.opendev.org/c/openstack/nova/+/1006718 | |
| 00:06:29 | opendevreview | Merged openstack/nova master: api-ref: Add new lines not allowed for server description https://review.opendev.org/c/openstack/nova/+/928758 | |
| 00:07:11 | opendevreview | Merged openstack/nova master: Fix inconsistent description of volume attachment id in Nova API docs https://review.opendev.org/c/openstack/nova/+/979378 | |
| 00:07:33 | opendevreview | Merged openstack/nova master: Fix misleading description of assisted volume snapshots https://review.opendev.org/c/openstack/nova/+/991649 | |
| 00:08:07 | opendevreview | Merged openstack/nova master: Update the api-ref for unshelve https://review.opendev.org/c/openstack/nova/+/938054 | |
| 00:25:13 | opendevreview | Merged openstack/nova master: Cfg: document devname reject in [pci]device_spec https://review.opendev.org/c/openstack/nova/+/1009083 | |
| 01:37:05 | opendevreview | Merged openstack/nova master: Remove experimental warning from UnifiedLimitsDriver https://review.opendev.org/c/openstack/nova/+/989102 | |
| 02:02:01 | opendevreview | Merged openstack/nova stable/2026.1: Fix deny_share host locality for NFS https://review.opendev.org/c/openstack/nova/+/1000212 | |
| 02:02:12 | opendevreview | Merged openstack/nova master: Remove [workarounds] disable_libvirt_livesnapshot https://review.opendev.org/c/openstack/nova/+/997492 | |
| 02:02:26 | opendevreview | Merged openstack/nova stable/2025.2: [novnc]Only log traffic if debug=True https://review.opendev.org/c/openstack/nova/+/995255 | |
| 02:02:38 | opendevreview | Merged openstack/nova stable/2026.1: Add regression tests for interface attach MTU bug https://review.opendev.org/c/openstack/nova/+/995257 | |
| 02:47:18 | opendevreview | Merged openstack/nova stable/2026.1: Fix network mtu metadata disappears after attaching https://review.opendev.org/c/openstack/nova/+/995258 | |
| 02:47:28 | opendevreview | Merged openstack/nova master: doc: Fix [service_user] auth_url in install guides https://review.opendev.org/c/openstack/nova/+/995528 | |
| 02:47:41 | opendevreview | Merged openstack/nova master: Fix KeyError on assisted snapshot call https://review.opendev.org/c/openstack/nova/+/900783 | |
| 02:47:51 | opendevreview | Merged openstack/nova master: libvirt: Omit options in cephfs mount when empty https://review.opendev.org/c/openstack/nova/+/986257 | |
| 03:20:27 | opendevreview | Merged openstack/nova master: Remove an unncessary LOG https://review.opendev.org/c/openstack/nova/+/912661 | |
| 03:20:42 | opendevreview | Merged openstack/nova master: Fix UEFI config test https://review.opendev.org/c/openstack/nova/+/978894 | |