| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-03-25 | |||
| 16:39:27 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2025/nova.2025-03-25-16.00.log.html | |
| 16:39:27 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2025/nova.2025-03-25-16.00.txt | |
| 16:39:27 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2025/nova.2025-03-25-16.00.html | |
| 16:39:27 | opendevmeet | Meeting ended Tue Mar 25 16:39:27 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 16:39:27 | Uggla | #endmeeting | |
| 16:39:41 | elodilles | thanks Uggla o/ | |
| 16:39:48 | masahito | thank you o/ | |
| 16:39:50 | Uggla | elodilles, ty | |
| 16:42:10 | bauzas | Uggla: bravo for your first meeting :) | |
| 16:44:28 | Uggla | bauzas, ty | |
| 16:46:23 | gibi | Uggla: thanks | |
| 17:33:24 | simondodsley | elodilles: not sure if you saw the conversation yesterday, but are the stable cores open to backport these 3 XS patches to resolve the SCSI page 83 passthrough issue: https://review.opendev.org/q/topic:%22lun-device-serial%22 | |
| 18:10:10 | gboutry | Hello, I'm currently splitting my hair around https://bugs.launchpad.net/ubuntu/+source/nova/+bug/2103413 | |
| 18:10:13 | gboutry | I'm currently running nova 31.0.0RC1 tag on python3.13, and I can't create a new instance. | |
| 18:10:17 | gboutry | When I'm trying to, I'm getting the error described in the ticket. | |
| 18:10:22 | gboutry | At around nova/nova/network/neutron.py:1233 (allocate_for_instance), admin_client.httpclient.dict seems to get totally wiped | |
| 18:10:26 | gboutry | going from {'session': <keystoneauth1.session.Session object at 0x7e1a2c601950>,... 'endpoint_override': None, ...} to {} (empty dict) | |
| 18:10:31 | gboutry | Thus failing with the error in the lp bug. the line number it happens varies but always in the function, so this looks like a finalizer going off the rails? | |
| 18:10:35 | gboutry | This seems python 3.13 related, but I'm not sure | |
| 18:11:30 | gboutry | I would appreciate any advice :) | |
| 18:13:16 | sean-k-mooney | gboutry: 3.13 is not offically supproted yet | |
| 18:13:24 | sean-k-mooney | it might work but its not actully tested | |
| 18:13:50 | sean-k-mooney | https://github.com/openstack/governance/blob/master/reference/runtimes/2025.1.rst is the supproted python verisons | |
| 18:14:26 | sean-k-mooney | gboutry: initall testing for 3.13 will be part of 2025.2 and it will only become requried in 2026.1 | |
| 18:16:22 | gboutry | sean-k-mooney: thanks for the info, Epoxy is targeted to be released in Ubuntu Plucky (25.04) which is using Python 3.13, we'll most likely need to carry a patch downstream if I can find the root cause of this issue | |
| 18:16:32 | sean-k-mooney | gboutry: i wonder if this could be eventlet related. teh error is coming form keystone authg via neutron client | |
| 18:17:26 | gboutry | I am getting some eventlet assertion errors in the background: https://pastebin.ubuntu.com/p/WhKxDrvQHt/ | |
| 18:18:30 | gboutry | https://github.com/eventlet/eventlet/commit/4d48d10b990910cc87ec6bbeeaea63b295fe314d python 3.13 has been added in this commit | |
| 18:18:37 | gboutry | I'll continue poking in this direction, thanks | |
| 18:19:11 | sean-k-mooney | so that after fork thing has come up before | |
| 18:19:27 | sean-k-mooney | we had to workaround a behvior change i think in 3.11 or 3.12 | |
| 18:19:41 | sean-k-mooney | to make sure we got the right id | |
| 18:19:49 | sean-k-mooney | perhasp that broken on 3.13 again | |
| 18:22:10 | sean-k-mooney | gboutry: https://github.com/eventlet/eventlet/issues/592 | |
| 18:22:51 | sean-k-mooney | we remvoed that workaroudn in epoxy https://github.com/openstack/nova/commit/cd980cdd1ea957a9aa2220014046ea53079a623f | |
| 18:23:02 | sean-k-mooney | because it has been fixed for a long time in eventlet | |
| 18:23:53 | sean-k-mooney | gboutry: you could try reverting that again locally and see if it fixes that issue on 3.13 but its really a eventlet bug and a regression fo something they previosly fixed | |
| 18:24:20 | gboutry | Thanks for the insight! I'll comeback later (need to take care of something), after reverting that, I'll report back! | |
| 18:24:20 | sean-k-mooney | hum well not quite | |
| 18:24:30 | sean-k-mooney | its a slightly diffent error | |
| 18:24:39 | sean-k-mooney | btu realted to the same code | |
| 18:24:57 | sean-k-mooney | cool | |
| 18:25:04 | gboutry | yes, this happens in the new code specific for python 3.13 support haha | |
| 18:30:23 | sean-k-mooney | gboutry: if you figure it out let us know. we will be trying to enable 3.13 this cycle and dont have issue with backproting non invaisive change to supprot it in 2025.1 but it wont officlaly be supproted for a while | |
| 19:01:27 | mikal | sean-k-mooney: sorry I missed the meeting, I was asleep. I am not entirely sure I know what a "specless blueprint" is. Is that just a blueprint in launchpad? There is already one for the SPICE work, are you saying I need a second one for the extra specs stuff? | |
| 19:02:35 | sean-k-mooney | mikal: a specless blueprint is just a launchpad blueprint that we agree does not require a spec so we approch based on the description of the bluepint in the irc meeting | |
| 19:02:52 | sean-k-mooney | i.e. if we belive it not contoverioal enough of big enough to need a spec | |
| 19:03:07 | sean-k-mooney | the current blueprint for the spice direct work will be closed as completed shortly | |
| 19:03:23 | sean-k-mooney | we should have a new blueprint for the vdi part fo 2025.2 | |
| 19:03:30 | mikal | Ahhh, ok. | |
| 19:03:52 | sean-k-mooney | mikal: i dont know if folks will want a mini spec for the vdi part or be fine with a detailed blueprint | |
| 19:04:12 | sean-k-mooney | given it basicaly code complete already and we discussed the design last cycle | |
| 19:05:15 | sean-k-mooney | mikal: regarding sleep you can add the bleuprint to the team meeting wiki or just ask async on the mailing list for it to be dicussed at the next one | |
| 19:06:00 | sean-k-mooney | mikal: i ment to bring it up today i just didnt have a link to the vdi work | |
| 19:07:25 | mikal | Huh. There are a surprisingly large number of SPICE blueprints on launchpad. | |
| 19:07:46 | sean-k-mooney | ya and a couple of video/audio/vdi ones | |
| 19:07:58 | mikal | Yeah, this is news to me. I shall have to read them. | |
| 19:08:05 | sean-k-mooney | like https://blueprints.launchpad.net/nova/+spec/libvirt-audio-device | |
| 19:08:18 | sean-k-mooney | mikal: I dont think any of them are relevnet anymore | |
| 19:08:30 | sean-k-mooney | mikal: im ot aware of anyoen actully working on them | |
| 19:08:35 | mikal | And https://blueprints.launchpad.net/nova/+spec/libvirt-spice-video-sound-driver | |
| 19:09:04 | sean-k-mooney | ya that the really old one | |
| 19:09:23 | sean-k-mooney | there is also a usb redirection one | |
| 19:09:46 | mikal | It would be nice to just close all of these (either as addressed or not) as part of landing the VDI stuff. | |
| 19:09:52 | sean-k-mooney | honestly its proably cleaner if you just file one for libvirt-vdi or similar that covers all yoru usecases | |
| 19:10:07 | sean-k-mooney | ya | |
| 19:10:12 | sean-k-mooney | it would be good to clean them up | |
| 19:10:13 | mikal | Oh for sure, but I should _read_ the ancient ones too | |
| 19:10:21 | mikal | There might be wisdom of the ancients there or something. | |
| 19:10:38 | sean-k-mooney | maybe most look pretty empty but not all | |
| 19:13:02 | sean-k-mooney | mikal: if nothing else it shows there si some interest form other to improve vdi on openstack | |
| 19:13:27 | mikal | Well, there once was. Presumably 2015 guy has gone and done something else now. | |
| 19:13:47 | mikal | I feel like https://blueprints.launchpad.net/nova/+spec/libvirt-spice-vdi is a bit sparse, but I am really not sure what else to put in it. | |
| 19:14:38 | sean-k-mooney | i would copy past some of the old spec that descibe the image properies/extra specs you plan to add | |
| 19:16:42 | sean-k-mooney | mikal: by the way while your usecase is mainly spice the vdi enchnemnt wont sticrtly need spice at least for the audio devce | |
| 19:17:00 | sean-k-mooney | not that it will be very useful with vnc via the proxy | |
| 19:18:02 | mikal | That is true. I presume "direct VNC" would be possible if there was a suitable proxy and would use these features too, but that's forward looking. | |
| 19:19:15 | sean-k-mooney | ya if you ever extend kerbside for that we could add it. i think you can get a better vnc conneciotn if you tunnel websocket back into a tcp connecton and use a better clinet but we dont ereally ahve anythignfor that out of the box | |
| 19:20:35 | mikal | I have this underlying suspicion that I should be feasible to transcode between SPICE and RDP in Kerbside, and I think that might interest some people. However, the RDP specs are at least partially locked down in telco standards which cost money, and its not something I am likely to find the time for anytime soon. | |
| 19:20:47 | mikal | s/that I should be/that it should be/ | |
| 19:21:29 | mikal | I tweaked https://blueprints.launchpad.net/nova/+spec/libvirt-spice-vdi, but its not an essay. | |
| 19:22:39 | sean-k-mooney | ya that looks ok. | |
| 19:22:57 | sean-k-mooney | so the process is bascilly see if peopel think its "simple enough to not need a spec" | |
| 19:23:21 | sean-k-mooney | if the other cores say sure we apprve it for the cycle if not we will ask for a spec and just do that dance | |
| 19:24:11 | mikal | Ok. Should I send an email to the list, or is this conversation sufficient to get it onto an agenda somewhere? | |
| 19:25:11 | sean-k-mooney | ideally you would add it to https://wiki.openstack.org/wiki/Meetings/Nova#Weekly_Nova_team_meeting in open dicssion section | |
| 19:25:39 | sean-k-mooney | if you want to also add a link to the irc logs fo this converstaion that fine. | |
| 19:26:14 | sean-k-mooney | https://meetings.opendev.org/irclogs/%23openstack-nova/latest.log.html#t2025-03-25T19:01:27 | |
| 19:39:15 | mikal | Done. | |
| 19:52:44 | opendevreview | mitya-eremeev-2 proposed openstack/nova master: Add ability to preserve ephemeral in evacuation https://review.opendev.org/c/openstack/nova/+/938235 | |
| 20:00:45 | opendevreview | mitya-eremeev-2 proposed openstack/nova master: Handle ephemeral root disk in evacuation https://review.opendev.org/c/openstack/nova/+/938235 | |
| 20:37:09 | opendevreview | melanie witt proposed openstack/nova stable/2025.1: unified limits: discover service ID and region ID https://review.opendev.org/c/openstack/nova/+/945525 | |
| 20:56:08 | gboutry | sean-k-mooney: I've tried multiple things, including reverting that patch, without much success. | |
| 20:56:24 | gboutry | I have a small reproducer, but not sure if it is relevant in nova context? | |
| 20:56:25 | gboutry | https://pastebin.ubuntu.com/p/t8pDTvdNqB/ | |
| 20:56:56 | gboutry | Or even if it makes much sense, but it gets the same error, I'll dig in more tomorrow, thanks for your support! | |
| 20:57:19 | sean-k-mooney | so what i think is happening is when we monkeypatch thread its not properly replacing the tread id with the grenthread id | |
| 20:57:28 | sean-k-mooney | which is what casueing the assert to fail | |
| 20:57:33 | sean-k-mooney | or somehting along those lines | |
| 20:57:44 | sean-k-mooney | but ya that seam like an eventlet bug not a nova one | |