| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-07-31 | |||
| 21:47:55 | melwitt | sean-k-mooney: something to do with nova rebasing the image. there's something where when you delete a volume snapshot while the instance is stopped, nova will end up rebasing the image which corrupts the metadata and then attempt to delete the last snapshot fails on the cinder side | |
| 21:49:27 | melwitt | so the repro is, boot from volume, create two volume snapshots, stop instance, delete the newest volume snapshot (works), delete the oldest volume snapshot (fails) | |
| 21:49:50 | melwitt | this is what I uploaded to do ^ https://review.opendev.org/c/openstack/tempest/+/956302/1/tempest/scenario/test_volume_boot_pattern.py | |
| 21:54:51 | sean-k-mooney | melwitt: so nova is not ment to allow that | |
| 21:55:07 | sean-k-mooney | when the instnace is stopped nova returns a 400 to cinder | |
| 21:55:30 | sean-k-mooney | when it calls the assisted voluem snapshot delete api | |
| 21:56:47 | sean-k-mooney | ya that tesss looks resonable | |
| 21:57:11 | gmaan | sean-k-mooney: that is only case (nova reject based on instance state )when merge_target_file is None https://github.com/openstack/nova/blob/master/nova/compute/api.py#L5926C34-L5926C51 | |
| 21:57:39 | gmaan | NOT None | |
| 21:57:59 | gmaan | when i looked into this bug, i also got confused with when nova reject based on instance state and when not | |
| 21:58:02 | sean-k-mooney | hum | |
| 21:58:25 | sean-k-mooney | well that is proably too narrorw | |
| 21:58:42 | gmaan | yeah | |
| 21:58:45 | sean-k-mooney | i mean we could make it work offline | |
| 21:58:57 | sean-k-mooney | but for right now there are obviously other cases where it can fail | |
| 21:59:29 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1919487 | |
| 21:59:36 | sean-k-mooney | that was the bug lee was tryign to mitigate | |
| 22:00:22 | sean-k-mooney | ok so thinking about this | |
| 22:00:57 | sean-k-mooney | if (delete_info.get('merge_target_file') is not None and | |
| 22:00:59 | sean-k-mooney | instance.vm_state != vm_states.ACTIVE | |
| 22:01:01 | sean-k-mooney | ): | |
| 22:01:26 | melwitt | not supposed to allow it? I did not know that | |
| 22:01:26 | sean-k-mooney | so if there is a merge_target_file | |
| 22:01:32 | sean-k-mooney | and the vm is stopped | |
| 22:01:34 | melwitt | coincidence I guess that it works fine with relative paths | |
| 22:01:35 | sean-k-mooney | then we raise | |
| 22:02:01 | melwitt | well, I guess let me wait for my test first. maybe it won't pass | |
| 22:02:05 | sean-k-mooney | melwitt: well i was thinkign ti was not inteded | |
| 22:02:19 | sean-k-mooney | becuase the logs i was looking at had a 400 saying it was not supported | |
| 22:02:28 | melwitt | oh, huh. | |
| 22:02:31 | sean-k-mooney | and if we added it then it was not inteded | |
| 22:02:39 | sean-k-mooney | so i think that was hitting this check | |
| 22:04:03 | sean-k-mooney | there are too bugs downstream for the issue your looking at fixing | |
| 22:05:03 | sean-k-mooney | this is in the cinder log of the other one | |
| 22:05:05 | sean-k-mooney | Call to Nova delete snapshot failed: novaclient.exceptions.BadRequest: Instance 9b578630-6c54-4481-8c49-6e0dd5d7e1fa in vm_state stopped. Cannot volume_snapshot_delete while the instance is in this state. (HTTP 400) | |
| 22:05:33 | melwitt | oh 🫤 | |
| 22:07:08 | sean-k-mooney | in the othe rbug they had 3 snapshot | |
| 22:07:22 | sean-k-mooney | the first one deleted fine | |
| 22:08:49 | melwitt | it just looked like such an easy simple thing to put it back to using relative path for the qemu-img rebase | |
| 22:09:35 | sean-k-mooney | the scond one the invalid backing file | |
| 22:10:20 | sean-k-mooney | the third once got that error above | |
| 22:10:36 | sean-k-mooney | melwitt: well it is | |
| 22:10:41 | sean-k-mooney | we shoud fix that first | |
| 22:11:17 | sean-k-mooney | when you have 3 snapshot deliting the most recnet breaked deletin the next oldest becuase of the backing file | |
| 22:11:18 | melwitt | I had no idea the depths of this issue 🙂 | |
| 22:11:42 | melwitt | yeah. that seems to be what happens with the 2 snapshots | |
| 22:11:44 | sean-k-mooney | melwitt: do you remember the last time you fix an nfs snapshot issue.... | |
| 22:11:53 | melwitt | lol | |
| 22:12:09 | melwitt | twas a journey | |
| 22:12:47 | sean-k-mooney | so i dont know if the backing file issue is having sidefect | |
| 22:13:20 | sean-k-mooney | i.e. if that is fixed i dont knwo if the other isssue will happen | |
| 22:13:35 | melwitt | oh, I see what you mean | |
| 22:13:50 | melwitt | hells yeah the test passes | |
| 22:14:01 | sean-k-mooney | oh with your chage | |
| 22:14:03 | sean-k-mooney | cool | |
| 22:14:20 | sean-k-mooney | can you try 3 snapshot? | |
| 22:14:25 | melwitt | yeah. I thought it would but I wanted to have proof | |
| 22:14:51 | melwitt | sean-k-mooney: yeah I can try that. in what order do you want to delete? newest then middle then oldest or? | |
| 22:16:21 | sean-k-mooney | yep that what they did | |
| 22:16:27 | melwitt | k | |
| 22:16:54 | sean-k-mooney | although i dont know if they waited. in the down stream bug they are suing the cli and issue the 3 delete back to back then list the snapshot | |
| 23:30:47 | melwitt | sean-k-mooney: for your tomorrow - test just passed with 3 snapshots also | |
| #openstack-nova - 2025-08-01 | |||
| 00:00:04 | opendevreview | Merged openstack/nova master: Implement USB controller extra spec for libvirt. https://review.opendev.org/c/openstack/nova/+/950643 | |
| 00:35:12 | gmaan | melwitt nice, I can see same error in cinder log also https://zuul.opendev.org/t/openstack/build/c68c0479054f47d08d47535f2625a7b8/log/controller/logs/screen-c-vol.txt#9067 | |
| 00:35:25 | gmaan | invalid backing file. and with your fix it poass | |
| 00:35:51 | gmaan | let me know once you are ready with tempest test if anything more you want to update and I can revierw | |
| 03:47:33 | mikal | sean-k-mooney: thank you. | |
| 09:38:24 | sean-k-mooney | Uggla: when you arbout can you sett implemention to complete https://blueprints.launchpad.net/nova/+spec/libvirt-spice-vdi | |
| 09:39:00 | Uggla | sean-k-mooney ah cool ! gonna do that. | |
| 09:39:00 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/950643 was the last patch | |
| 09:39:34 | mikal | sean-k-mooney: turns out as the author I can do that, so I've done it just now... Hopefully that's not a paddling. | |
| 09:39:54 | sean-k-mooney | mikal: not that fine | |
| 09:40:08 | sean-k-mooney | iw as goign to do it for you but im not in the corerct group in launch pad | |
| 09:40:33 | sean-k-mooney | *no thats fine | |
| 09:41:01 | sean-k-mooney | mikal: you will have to let use know how all this works out for kerbside once folks that using it | |
| 09:41:16 | sean-k-mooney | i.e. are there more vdi enhancments that are needed ectra | |
| 09:41:25 | sean-k-mooney | mikal: i assume kolla-ansible is your next priority | |
| 09:42:00 | mikal | sean-k-mooney: yeah, I've spent the last few weeks working on a kolla-ansible role. I figure if I do one example that will make it easier for others. At some point I should figure out how to get onto that team's radar, although I assume that's a G Release problem at this point. | |
| 09:42:38 | mikal | I can't think of any further VDI patches for now, but I think you're right that there will likely be more as people play with it. | |
| 10:00:11 | sean-k-mooney | mikal: well they used to use spec and they alwasy have ptg sessions, so i woudl see if they can give you a slot at a not horible time | |
| 10:00:54 | sean-k-mooney | they used to be pretty active on irc in the past too but its been a while since i did anything with kolla directly | |
| 10:10:21 | mikal | sean-k-mooney: I figure I'll finish the role and upload it, and then that will give something tangle to talk about. That said, the PTG does sound like a good idea. I have a mate who thinks I should shoot some demo videos as well, but baby steps and all that. | |
| 10:17:57 | opendevreview | Balazs Gibizer proposed openstack/nova master: Document native threading mode and tuneables https://review.opendev.org/c/openstack/nova/+/949364 | |
| 10:17:58 | opendevreview | Balazs Gibizer proposed openstack/nova master: Allow services to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | |
| 10:17:59 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-next with n-sch in threading mode https://review.opendev.org/c/openstack/nova/+/948450 | |
| 10:17:59 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-api and -metadata in threaded mode https://review.opendev.org/c/openstack/nova/+/951957 | |
| 10:18:00 | opendevreview | Balazs Gibizer proposed openstack/nova master: Do not yield in threading mode https://review.opendev.org/c/openstack/nova/+/950994 | |
| 10:18:00 | opendevreview | Balazs Gibizer proposed openstack/nova master: Allow to start unit test without eventlet https://review.opendev.org/c/openstack/nova/+/953436 | |
| 10:18:01 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run unit test with threading mode https://review.opendev.org/c/openstack/nova/+/953475 | |
| 10:18:01 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]RPC using threading or eventlet selectively https://review.opendev.org/c/openstack/nova/+/953815 | |
| 10:18:02 | opendevreview | Balazs Gibizer proposed openstack/nova master: [CI]Make nova-tox-py312-threading voting https://review.opendev.org/c/openstack/nova/+/955791 | |
| 10:18:02 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make RBD Tpool usage conditional https://review.opendev.org/c/openstack/nova/+/956089 | |
| 10:18:04 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make libvirt Tpool proxying conditional https://review.opendev.org/c/openstack/nova/+/956090 | |
| 10:18:04 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix ProviderTree copying with threading Lock https://review.opendev.org/c/openstack/nova/+/956091 | |
| 10:18:06 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]Further categorization of disabled unit tests https://review.opendev.org/c/openstack/nova/+/956092 | |
| 10:18:06 | opendevreview | Balazs Gibizer proposed openstack/nova master: [vncproxy]Handle ssl.wrap_socket removal in py312 https://review.opendev.org/c/openstack/nova/+/955915 | |
| 10:18:08 | opendevreview | Balazs Gibizer proposed openstack/nova master: Warn on long task wait time for executor https://review.opendev.org/c/openstack/nova/+/952666 | |
| 10:50:59 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Simplify servers views (2/3) https://review.opendev.org/c/openstack/nova/+/956232 | |
| 10:51:00 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Simplify servers views (3/3) https://review.opendev.org/c/openstack/nova/+/956233 | |
| 10:51:00 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for servers APIs (1/6) https://review.opendev.org/c/openstack/nova/+/956234 | |