Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-04-01
16:34:11 gibi go for it
16:34:23 Uggla sean-k-mooney, I will set the topic for the agenda, maybe it will be quick if it is crystal clear.
16:34:40 sean-k-mooney ack
16:35:00 Uggla something else you want to discuss for the PTG prep ?
16:35:47 sean-k-mooney am just an fyi im double booked with watcher
16:36:00 sean-k-mooney so ill attend where i can
16:36:12 sean-k-mooney ping if im not there and you would like my input
16:36:21 sean-k-mooney i tried to leave comment in the doc already
16:36:29 Uggla sean-k-mooney, ok sure thx.
16:36:44 Uggla I close this topic by saying we will not have this meeting next week due to the vPTG.
16:36:56 Uggla #topic Stable Branches
16:36:57 masahito i'm not sure i mentioned it before. if my topic is selected for the ptg topics, i prefer 14utc or later.
16:37:23 Uggla masahito, yes, this is somewhere in my mind.
16:37:30 masahito thanks.
16:37:35 Uggla #info stable/2024.* gates broken with nova-ceph-multistore job failure (test case test_volume_upload fails - No image found with ID...)
16:37:41 Uggla #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci
16:38:08 Uggla elodilles, the floor is yours
16:38:16 elodilles yeah, i couldn't identify the root cause yet for that job failure yet ^^^ :(
16:38:42 elodilles so if anyone has any idea, it is appreciated o:)
16:39:24 elodilles stable/2023.2 gate is fine, and afaik stable/2025.1, too, but not 100% on that
16:39:57 elodilles and that is all i can say now
16:41:14 sean-k-mooney the iso patch has been rechecked again...
16:41:22 sean-k-mooney still hiting that volume bug
16:41:37 sean-k-mooney i havent looked at it yet either ill admit
16:42:10 sean-k-mooney i assume cinder of glance shoudl be seeing the same failure?
16:42:48 gmann I think I saw it on stable/2025.1 also (have not checked if it is same or different?) https://review.opendev.org/c/openstack/devstack/+/945239/comments/231d655b_52020b94
16:43:46 sean-k-mooney that still the nova job
16:44:00 sean-k-mooney i was hopign that was a non nova one showing the same issue
16:44:30 elodilles gmann: looks like same
16:44:53 Uggla looks the same yes.
16:45:09 sean-k-mooney dansmith: do you know off the top of your head if there is anything special about the nova-ceph-multistore job that woudl cause no image found on volume upload?
16:45:30 dansmith volume upload to glance?
16:45:49 sean-k-mooney presumable yes
16:46:09 dansmith not really, that seems sort of impossible,
16:46:16 dansmith since you create first and then upload
16:46:19 elodilles (and the devstack patch merged successfully on 27th March, so it either not fully blocking or it was fixed on stable/2025.1 somehow)
16:46:30 gmann seems like glance fail to import image?
16:46:32 gmann #link https://zuul.opendev.org/t/openstack/build/54c93553fc6c4797b25077fb29a89338/log/controller/logs/screen-g-api.txt#12807
16:47:33 sean-k-mooney 953cc449-8f49-4d56-ab3e-49b54f49f937 failed to import image 833ccb36-83b8-41a7-b932-86884e4cdfc0 to the filesystem.: NoneType: None
16:47:47 dansmith is this a multinode setup?
16:47:53 dansmith with glance-remote configured?
16:48:13 dansmith that sort of looks like it can't do import because it can't look up the configured self-reference url of a node that did the stage, maybe?
16:48:31 sean-k-mooney its multinode and multi store but im note sure if that also means multiple glance apis
16:48:50 sean-k-mooney oh no
16:48:53 sean-k-mooney its single node
16:48:57 sean-k-mooney but multiple backends
16:49:03 dansmith yeah, single node and no g-api-r service
16:50:05 dansmith that must be the import-from-http test and not volume upload then
16:50:44 sean-k-mooney its empest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_upload[id-d8f1ca95-3d5b-44a3-b8ca-909691c9532d,image]
16:50:52 sean-k-mooney https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_54c/openstack/54c93553fc6c4797b25077fb29a89338/testr_results.html
16:51:57 sean-k-mooney its a post to POST https://10.0.18.121/volume/v3/volumes/5726aafd-b7f7-4dbd-be64-265076f0efbb/action so asking cidner to upload the voluem to glance
16:51:59 sean-k-mooney Body: {"os-volume_upload_image": {"image_name": "tempest-VolumesActionsTest-Image-2109450379", "disk_format": "raw"}}
16:52:12 dansmith I also don't expect volume upload to use import
16:53:03 dansmith sean-k-mooney: I meant the failed task that gmann linked to
16:53:30 dansmith let's not debug it here
16:53:30 sean-k-mooney oh ok
16:53:31 gibi Uggla: I think we can close the meeting and let the folks continue troubleshooting
16:53:41 sean-k-mooney ya
16:53:42 gmann I think there are some image delete request in log
16:53:47 gmann #link https://zuul.opendev.org/t/openstack/build/54c93553fc6c4797b25077fb29a89338/log/controller/logs/screen-g-api.txt#19062
16:53:51 gmann yeah let's debug later
16:54:08 Uggla yep can we looks at that after the meeting ?
16:54:24 sean-k-mooney sure
16:54:25 Uggla I skip the vmwareapi 3rd-party CI efforts Highlights as fwiesel is ooo this week.
16:54:37 Uggla Latest topic
16:54:43 Uggla #topic Open discussion
16:55:00 Uggla If there is not, I'll close the meeting.
16:55:21 Uggla anything more to discuss ?
16:55:51 Uggla 3...
16:56:13 Uggla 2...
16:56:38 Uggla 1...
16:56:51 bauzas .
16:57:14 Uggla thanks all
16:57:16 Uggla #endmeeting
16:57:16 opendevmeet Meeting ended Tue Apr 1 16:57:16 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
16:57:16 opendevmeet Minutes: https://meetings.opendev.org/meetings/nova/2025/nova.2025-04-01-16.01.html
16:57:16 opendevmeet Minutes (text): https://meetings.opendev.org/meetings/nova/2025/nova.2025-04-01-16.01.txt
16:57:16 opendevmeet Log: https://meetings.opendev.org/meetings/nova/2025/nova.2025-04-01-16.01.log.html
16:57:21 elodilles thanks Uggla o/
16:57:30 bauzas thanks Uggla
16:57:34 masahito thank you
16:57:35 gibi thanks
16:58:16 Uggla I thought @dansmith would like to discuss brainfuck formating. :)
16:58:16 sean-k-mooney dansmith: gmann: i have a thoey but im not sure if it makes sense. looking at https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_54c/openstack/54c93553fc6c4797b25077fb29a89338/controller/logs/etc/glance/glance-api_conf.txt the default backend is file(cheap), this is a volume upload test and the job has ceph. so i would assome that
16:58:18 sean-k-mooney cinder is using ceph. could the issue be that cidner is expectign to upload to the robust(ceph) store but we are trying to upload to the default file store
16:58:49 sean-k-mooney its called black
16:59:03 sean-k-mooney with its one line per function argument
17:00:17 sean-k-mooney dansmith: what im wondering is did cinder create a volume snapshot on ceph and then try and use the interoperable imporat flow to import from the ceph store to file store or somethign odd like that
17:00:34 dansmith you can't use import like that
17:00:38 dansmith so I don't think so
17:00:56 dansmith you're thinking of adding the location directly
17:01:21 sean-k-mooney perhaps
17:01:23 sean-k-mooney https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_54c/openstack/54c93553fc6c4797b25077fb29a89338/controller/logs/etc/cinder/cinder_conf.txt
17:01:28 sean-k-mooney cidner is using ceph
17:01:48 sean-k-mooney im just wondering if cidner ig getting confused about how to upload the image
17:02:23 dansmith maybe best to ask a cinder person to look
17:03:06 sean-k-mooney ya im going to very quieckly try and find the chagne id in there logs
17:03:15 sean-k-mooney *request id
17:04:58 elodilles gmann sean-k-mooney : it looks like this patch landed around the time when things started to break: https://review.opendev.org/c/openstack/tempest/+/938592
17:05:24 gmann elodilles: yeah, I was about to debug that and someone rechecked and it merged and then I forgot

Earlier   Later