| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-04-01 | |||
| 17:29:11 | sean-k-mooney | we also have force image convertion to raw in this job | |
| 17:29:30 | sean-k-mooney | which could be a diffent issue | |
| 17:29:50 | sean-k-mooney | its not failing in galnce however so its not the current issue | |
| 17:32:10 | supamatt | :wave: hi folks this change broke security groups https://review.opendev.org/c/openstack/nova/+/811521 , if any security groups with the same name exist in the project (whether the VM is using that sec group or not). Results in VMs not being able to be built with an error saying "Multiple security groups found matching |
|
| 17:33:08 | sean-k-mooney | if you relying on passign the name instead of the uuid | |
| 17:33:54 | sean-k-mooney | it woudl also only fail if that conflicting named security group was shared with you | |
| 17:34:37 | sean-k-mooney | supamatt: im not really sure how to adress the conflicting requirements | |
| 17:34:53 | sean-k-mooney | prefering the local one seams incorrect | |
| 17:35:11 | sean-k-mooney | if there is a name conflict it should be raised as an error by nova | |
| 17:36:05 | masahito | gmann: gibi: hi please review this api bug fix https://review.opendev.org/c/openstack/nova/+/939658 The v2.100 follows the same fix, so it might be ok for W+1. | |
| 17:36:48 | supamatt | sean-k-mooney: that's the thing, I am passing the uuid to nova. But nova is errors if any security has the same name in a project, regardless if I use that sec group to provision a vm. | |
| 17:37:30 | sean-k-mooney | supamatt: on ok | |
| 17:37:34 | sean-k-mooney | *oh | |
| 17:37:41 | sean-k-mooney | that simpler to fix | |
| 17:37:58 | sean-k-mooney | supamatt: have you filed a bug for this yet | |
| 17:38:17 | supamatt | I was checking lp now to see if there was an existing one | |
| 17:38:32 | supamatt | but it seems I am the first to see this, so I will file one | |
| 17:39:08 | sean-k-mooney | if you pass a name then the current behavior is arguabel correct. we could prefer the one from the current project but that might eb surprisign, if you pass a uuid we shoudl use the one you said even if there are duplicate names so that a ligit bug | |
| 17:40:38 | sean-k-mooney | supamatt: so taht a new feature in epoxy which release tomorrow | |
| 17:41:10 | sean-k-mooney | we wont have a fix for it in the release but we likely will do another release after the ptg | |
| 17:51:11 | sean-k-mooney | supamatt: https://review.opendev.org/c/openstack/nova/+/811521/13/nova/network/neutron.py#851 that where the but is | |
| 17:51:34 | sean-k-mooney | supamatt: the current algorthim is raising based on the aviabel secuirty groups to the user not the requsted one | |
| 17:52:26 | sean-k-mooney | we need to change the data stuctures and the algorthim slightly | |
| 17:53:41 | gmann | masahito: ack, will check soon | |
| 17:57:51 | gmann | dansmith: sean-k-mooney: elodilles: on test_volume_upload ceph failure, I found the cinder fix which is being backported to stable branches one by one. https://review.opendev.org/q/I32b505aa69c71b62e7e3a52d65d38165d34e97d8 | |
| 17:58:31 | sean-k-mooney | gmann ack | |
| 17:58:34 | gmann | we just need to wait for that to be merged on all stable | |
| 17:58:38 | dansmith | gotcha | |
| 17:58:59 | dansmith | I'm also a bit surprised (not really) they have not converted to oslo format inspector | |
| 17:59:08 | dansmith | I even prototyped the patch for them | |
| 17:59:49 | sean-k-mooney | hum ya the failure was on stable/2025.1 | |
| 18:00:05 | sean-k-mooney | that means they are also missing some of the mutliiple format things | |
| 18:00:18 | sean-k-mooney | for iso supprot and vfat supprot | |
| 18:00:26 | dansmith | yeah | |
| 18:00:33 | dansmith | not sure if that's an issue for them or not | |
| 18:00:51 | sean-k-mooney | well it mean you wont be able to create volumes form some isos presumable | |
| 18:01:09 | sean-k-mooney | that a pretty neich usecase given you can only use them properly with hacks | |
| 18:01:10 | dansmith | I'm still a bit surprised that fix that gmann found is not deterministic in its fail pattern | |
| 18:01:18 | elodilles | gmann: ah, cool, sounds great! thanks! | |
| 18:01:43 | sean-k-mooney | dansmith: ya that odd to me too | |
| 18:04:51 | sean-k-mooney | dansmith: that unlink https://review.opendev.org/c/openstack/cinder/+/945616/2/cinder/volume/drivers/rbd.py#2126 bug | |
| 18:05:22 | sean-k-mooney | upload can raise so there should at least be a try finally block here | |
| 18:06:05 | dansmith | unlink you mean? | |
| 18:06:20 | dansmith | it also looks like they're using two strategies to unlink the temp file no? | |
| 18:06:39 | sean-k-mooney | ya the os.unlink of the temp export of the volume | |
| 18:06:41 | dansmith | oh, I guess the context only unlinks on error | |
| 18:06:54 | dansmith | but yeah, that could definitely fail | |
| 18:07:09 | sean-k-mooney | oh they are using fileutils.remove_path_on_error(tmp_file): | |
| 18:07:16 | sean-k-mooney | to handel the excption case | |
| 18:07:26 | sean-k-mooney | and then the explcit unlink for happy path | |
| 18:07:34 | dansmith | right, seems like they should use try..finally: unlink() | |
| 18:07:41 | sean-k-mooney | that works... but not how i would have done that | |
| 18:07:46 | dansmith | yeah | |
| 18:08:08 | sean-k-mooney | dont we use context lib.exit_stack or something like that to handel it in nova/oslo | |
| 18:08:34 | dansmith | in some places yeah but I don't think it's really necessary here | |
| 18:08:35 | sean-k-mooney | we might not even need that | |
| 18:08:54 | sean-k-mooney | right that really only needed if we have multipel resocue we need to clean up | |
| 18:10:27 | sean-k-mooney | gmann: are we sure this is not just hard failing but it looks like its only failign sometimes due to the backport fo that patch | |
| 18:10:44 | sean-k-mooney | gmann: given the "fix" id dont see how this could sometiems work and sometiems not | |
| 18:11:28 | sean-k-mooney | enabling the qcow image type and this abckprot have all been happening over the last week | |
| 18:11:55 | gmann | sean-k-mooney: after this fix merged in master and stable/2025.1, I have not seen failure. in my devstack change on stable/2025.1 it started passing once this fix backport merged on stable/2025.1. I checked the timing to make sure that | |
| 18:12:27 | sean-k-mooney | ack | |
| 18:12:37 | sean-k-mooney | well let just wait for the final patch to land i guess | |
| 18:12:40 | gmann | stable/2024.2 backport still not merged so there we have 100% | |
| 18:12:44 | gmann | 100% failing | |
| 18:12:45 | gmann | yeah | |
| 18:13:47 | sean-k-mooney | ack that makes sense | |
| 18:16:13 | sean-k-mooney | gmann: the default chagne of image formats was in tempest yes? | |
| 18:16:56 | gmann | yes | |
| 18:17:35 | sean-k-mooney | i wonder if it would make sense to replace https://github.com/openstack/tempest/blob/80c0477f78c71a2bd2e1a324c41cd2f50329b200/zuul.d/project.yaml#L113-L115 with the nova ceph multistore job | |
| 18:18:39 | sean-k-mooney | that would provide the same level fo ceph coverage but allso test this configuration where upload go to a glance backed by file in this case. | |
| 18:20:01 | sean-k-mooney | that might be an over reaction but that would ahve caught this | |
| 18:20:16 | sean-k-mooney | ye already have quite a lot in check | |
| 18:20:27 | sean-k-mooney | so i would be hesitent to suggest adding more jobs | |
| 18:20:27 | gmann | sean-k-mooney: +1, that make sense. we do have nova-ceph-multistore in tempest but as experimental which is what we forget to run on related changes | |
| 18:20:44 | sean-k-mooney | its in devstack too | |
| 18:20:55 | sean-k-mooney | https://github.com/openstack/devstack/blob/master/.zuul.yaml#L933-L934 | |
| 18:21:12 | sean-k-mooney | that how we are testing ceph integration on devstack changes | |
| 18:21:30 | gmann | yes, I think replacing devstack-plugin-ceph-tempest-py3->nova-ceph-multistore in tempest will cover more cases | |
| 18:22:03 | gmann | catching in devstack is late where tempest things get merged | |
| 18:23:15 | gmann | sean-k-mooney: might be late for you but tomorrow or later if you can propose that in tempest I can merge. if not then I will do sometime later this week | |
| 18:24:06 | sean-k-mooney | i can do it now. its 19:23 but ill be around until at least the top of the hour | |
| 18:24:14 | gmann | thanks | |
| 18:27:06 | sean-k-mooney | gmann: am what will i do about https://github.com/openstack/tempest/blob/80c0477f78c71a2bd2e1a324c41cd2f50329b200/zuul.d/project.yaml#L159-L160 | |
| 18:27:49 | sean-k-mooney | it was temporaly disabled in gate 3 years ago | |
| 18:28:04 | sean-k-mooney | when i replace it will i add nova-ceph-multinode or just remove that comment | |
| 18:28:16 | sean-k-mooney | https://github.com/openstack/tempest/commit/b1ea4327108cbbd518dfc75482dff79493b4edc9 | |
| 18:28:26 | gmann | sean-k-mooney: yeah, mostly due to stability. but you can keep nova ceph as voting and remove from experimental queue https://github.com/openstack/tempest/blob/80c0477f78c71a2bd2e1a324c41cd2f50329b200/zuul.d/project.yaml#L176 | |
| 18:28:30 | gmann | yeah | |
| 18:28:46 | sean-k-mooney | ok ill add it to both | |
| 18:29:08 | sean-k-mooney | its voting on nova and devstack and stable so i dont see a reason to skip in gate | |
| 18:29:30 | gmann | yeah | |
| 18:31:00 | sean-k-mooney | gmann: last question will i put the curernt devstack-ceph job in experimental. i think no but i can replace nova-ceph-multistore with it if its useful to have there | |
| 18:31:48 | sean-k-mooney | experiental quite a lot in it too and we can easilly add a dnm to run it if needed | |
| 18:31:57 | supamatt | sean-k-mooney: here's the lp bug for the security group problem, https://bugs.launchpad.net/nova/+bug/2105896 | |
| 18:32:16 | sean-k-mooney | thanksi htink i figured out how to fix it and commented in the orginal patch | |
| 18:40:00 | sean-k-mooney | gmann: i think https://review.opendev.org/c/openstack/tempest/+/946076 should be good. i will respin it tomorrow if you have any issues | |
| 18:40:46 | gmann | sean-k-mooney: looks good, thanks | |
| 19:28:14 | opendevreview | Matthew Heler proposed openstack/nova master: Fix creating virtual servers with multiple security groups https://review.opendev.org/c/openstack/nova/+/946079 | |
| 19:42:38 | opendevreview | Goutham Pacha Ravi proposed openstack/nova stable/2025.1: DNM: test dependency on devstack-plugin-ceph changes https://review.opendev.org/c/openstack/nova/+/946082 | |