| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-04-13 | |||
| 16:44:31 | Uggla | #topic vmwareapi 3rd-party CI efforts Highlights | |
| 16:44:37 | Uggla | fwiesel anything for us ? | |
| 16:45:07 | fwiesel | Hi, so I can confirm that the bugfix in oslo.vmware would fix the image upload timeout observed by gibi | |
| 16:45:12 | fwiesel | #link https://review.opendev.org/c/openstack/oslo.vmware/+/982614 | |
| 16:45:31 | Uggla | \o/ thanks | |
| 16:45:46 | fwiesel | That's from my side for now... I spend the calmer days looking into some random errors, but nothing to report now. | |
| 16:46:17 | gibi | yeah I'm +1 on the fix | |
| 16:46:28 | Uggla | fwiesel thank you. | |
| 16:46:36 | Uggla | #topic Gibi's news about eventlet removal | |
| 16:46:45 | gibi | o/ | |
| 16:48:01 | gibi | no much to report | |
| 16:48:20 | gibi | work ongoing on the console proxies and testing | |
| 16:48:51 | gibi | the PTG etherpad is up to date | |
| 16:49:04 | gibi | we will discuss status and plans there | |
| 16:49:08 | gibi | EOM | |
| 16:49:15 | sean-k-mooney | https://review.opendev.org/c/openstack/oslo.vmware/+/982614 looks indepent of the executor issue we saw in the vmware job so this is just a general urllib3 issue not an eventet oen right | |
| 16:50:16 | sean-k-mooney | im asking because it does not have a bug filed for it and im wodnering if eventlet was masking it | |
| 16:51:20 | gibi | the executor issue was the deadlock on sending dependent tasks to the parent's executor we fixed that in general | |
| 16:51:28 | gibi | this bug apperad after | |
| 16:51:44 | gibi | and was visible as hanging in image tasks in the vmware driver | |
| 16:52:13 | sean-k-mooney | ack so it may have been there but not visable before | |
| 16:52:17 | gibi | yeah | |
| 16:52:25 | sean-k-mooney | i know evently also monkey patches urllib3 | |
| 16:52:35 | sean-k-mooney | so that also complciates things | |
| 16:52:43 | sean-k-mooney | ok we can likely move on | |
| 16:52:53 | Uggla | #topic Bug scrubbing | |
| 16:52:58 | Uggla | #info up to 197 (-2) | |
| 16:53:10 | Uggla | #link: https://etherpad.opendev.org/p/nova-bug-triage-roster | |
| 16:53:19 | Uggla | #link: https://truc.uggla.fr/ to follow the trend. | |
| 16:53:25 | Uggla | Uggla: This week [public] Upstream bug triage. Wednesday, April 15 · 15:30 – 16:00 UTC. Video call link: https://meet.google.com/kmq-eedg-auj | |
| 16:53:47 | Uggla | oops forget to say : Skipping Layos topic as he looks not around and we are a bit late on schedule | |
| 16:54:17 | Uggla | #topic Open discussion | |
| 16:54:25 | Uggla | Uggla: Suggest no meeting next week. 20th April vPTG week. | |
| 16:54:38 | gibi | +1 | |
| 16:54:54 | elodilles | +1 | |
| 16:54:54 | Uggla | I will notify on the ML too | |
| 16:55:36 | Uggla | any last 5 minutes topic to discuss ? | |
| 16:57:08 | phildree[m] | Hello guys, I am not sure if this is the right place and time to bring this up because i am new to OpenStack, but I have created a spec and am looking for some feedback: https://review.opendev.org/c/openstack/nova-specs/+/984384 | |
| 16:58:07 | Uggla | phildree[m], can you add a topic in the ptg etherpad ? then I'll define a slot for you next week. | |
| 16:58:32 | phildree[m] | Ok, thank you :) | |
| 16:58:54 | Uggla | any "prefered" timing ? | |
| 16:59:59 | phildree[m] | Nope | |
| 17:00:22 | Uggla | ok | |
| 17:00:27 | Uggla | so I think we are good. | |
| 17:01:00 | Uggla | time to close, thanks for joining this meeting. Have a nice day/evening and see you next week for the vPTG. | |
| 17:01:07 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2026/nova.2026-04-13-16.02.log.html | |
| 17:01:07 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2026/nova.2026-04-13-16.02.txt | |
| 17:01:07 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2026/nova.2026-04-13-16.02.html | |
| 17:01:07 | opendevmeet | Meeting ended Mon Apr 13 17:01:07 2026 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 17:01:07 | Uggla | #endmeeting | |
| 17:01:11 | fwiesel | o/ thanks everyone | |
| 17:01:17 | elodilles | see you next week o/ | |
| 17:01:24 | sean-k-mooney | phildree[m]: have you talked to oslo about his | |
| 17:01:52 | phildree[m] | No, this IRC is my first point of contact for now | |
| 17:01:56 | sean-k-mooney | phildree[m]: in generally iw oudl prefer if we removed the if we didnt have to have any aplciation level handchakes | |
| 17:02:24 | gibi | thanks | |
| 17:03:21 | sean-k-mooney | we have some existign rpc pings but anything that makes the conductor comptue messageing nosyier by addign background messages direct hurts scalableity fo nova | |
| 17:03:46 | phildree[m] | sean-k-mooney: Ok, I understand. Maybe handshake is a bit much for what i have proposed, its more like an echo | |
| 17:04:11 | sean-k-mooney | right im saying even adding a echo/ping is protically a scaling issues | |
| 17:04:18 | phildree[m] | sean-k-mooney: I have seen the existing ping but it is not applicable to our usecase (I explained why in the spec) | |
| 17:05:00 | phildree[m] | sean-k-mooney: yes you are totally right, I get that. Any other ideas on how to solve the problem we have been facing? | |
| 17:05:30 | opendevreview | Merged openstack/nova stable/2025.1: Fix fill_metadata usage for the ImagePropertiesWeigher https://review.opendev.org/c/openstack/nova/+/964291 | |
| 17:05:36 | sean-k-mooney | i have not read your spec yet but we gerneally perfer to start with the usecase/probelm | |
| 17:05:47 | sean-k-mooney | before going to a feature/design chagne | |
| 17:07:01 | sean-k-mooney | phildree[m]: the prviosu solution which we never implmetned was per binary healthcheck endponts on each nova service | |
| 17:08:21 | sean-k-mooney | phildree[m]: i think teh issue your having is something the outbount rabbit conenction is fucntional | |
| 17:08:28 | sean-k-mooney | but the inbound one is not correct | |
| 17:08:49 | sean-k-mooney | i.e. the compute can call the conductor but the conductor cannot sucessfully call the computes | |
| 17:09:14 | phildree[m] | Yes, we faced this exact issue. | |
| 17:09:33 | sean-k-mooney | do you have the guareented message delivery flag enabled in oslo | |
| 17:09:36 | phildree[m] | I will think about the other approach with individual per binary healthcheck endpoints | |
| 17:10:13 | phildree[m] | That is a good question, which I am not able to answer right now | |
| 17:10:13 | sean-k-mooney | well we didnt impment that or at least we didnt merge it | |
| 17:11:11 | melwitt | bauzas: hi, there are two vtpm related patches that are simple and need a second review if you could get a chance plz https://review.opendev.org/c/openstack/nova/+/978836 and https://review.opendev.org/c/openstack/nova/+/983504 | |
| 17:11:19 | sean-k-mooney | phildree[m]: https://docs.openstack.org/nova/latest/configuration/config.html#oslo_messaging_rabbit.direct_mandatory_flag | |
| 17:11:25 | sean-k-mooney | i woudl check you have that set to true | |
| 17:11:32 | phildree[m] | My issues with the per binary healthcheck is that I still need a way to ensure that full communication from conductor to compute (and back) is up and running, which i cannot check without any actual communication, I think | |
| 17:11:51 | sean-k-mooney | phildree[m]: what that will detect is fi the per compute queue is present | |
| 17:12:02 | phildree[m] | Ok very nice, I will look into it, thank you for the infos :) | |
| 17:12:18 | sean-k-mooney | phildree[m]: well the whole point of the per binary healthcheck si to not use rabbit or the conductor | |
| 17:12:38 | sean-k-mooney | it to move the health moditoring out fo that code path to an external http endpoint | |
| 17:12:51 | sean-k-mooney | that you can have prometheous or whatever scrape instead | |
| 17:14:45 | phildree[m] | I get it, and it makes sense, but then there is no way to check the health of the communication (and the underlying infrastructure) between conductor and compute (only the individual binaries) | |
| 17:14:45 | sean-k-mooney | phildree[m]: the direct_mandatory_flag tuns a slient failrue (the messag eis encued to a new queue that nothign is listenting too) into a driect rpc call failure and failure in teh api | |
| 17:15:41 | sean-k-mooney | right today we do that via the resouce update whcih is a push model | |
| 17:16:18 | sean-k-mooney | its worth a dicussion i guess but my main concern is we dont really run perodic form the conductor to monitor the satues of other agents | |
| 17:16:49 | sean-k-mooney | and we try not to do monitoring in general in nova. so this woudl likely need to be confiugrable and off by defualt | |
| 17:16:57 | sean-k-mooney | at least until we know the overhad of it | |
| 17:17:58 | phildree[m] | I think I have to read up on what you have sent first before I may rethink the idea. Thanks a lot for the infos so far, that was very helpfull and a lot less complicated then i thought :) | |
| 17:17:58 | phildree[m] | I have touched many points of what you have brought up, but as I said, I am not yet very familiar with OpenStack so maybe there are some issues I did not anticipate. | |
| 17:19:14 | sean-k-mooney | phildree[m]: one thing we are currently changing | |
| 17:19:32 | sean-k-mooney | is we will nolonger have a slngle rpc listter per compute agent | |
| 17:19:47 | sean-k-mooney | we will now have two becase fo the graceful shutdown work | |
| 17:20:06 | sean-k-mooney | so gmaan woudl likely need to review your prosal in the context fo that work | |
| 17:20:24 | phildree[m] | Ok, that is something very important as this may have implications on my idea | |
| 17:20:42 | sean-k-mooney | with graceful shutdown we will be shutting down 1 of the 2 lstenter until the inprogress operation complete | |
| 17:21:02 | sean-k-mooney | so any kind fo monitoring would likely have to be on the one we dont shutdown | |
| 17:22:08 | gmaan | sean-k-mooney: reading chat | |
| 17:22:20 | phildree[m] | At least for the shutdown period, yes | |
| 17:22:20 | sean-k-mooney | gmaan: context is https://review.opendev.org/c/openstack/nova-specs/+/984384/1/specs/2026.2/approved/bidirectional-rpc-liveness-handshake-for-compute-nodes.rst | |