| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-05-06 | |||
| 16:30:25 | gibi | good news both mysql server and pymysql client can be configured with timeouts to avoid handing gather threads | |
| 16:30:47 | gibi | I will add a new doc about it in tree | |
| 16:31:02 | gibi | bad news, the oslo.service threading backend uses forks | |
| 16:31:23 | gibi | which does not play nice with out global threadpools initialized before the service workers are forked off | |
| 16:31:32 | gibi | we have workarounds | |
| 16:31:41 | gibi | details are in https://review.opendev.org/c/openstack/oslo.service/+/945720/comments/fb5d6632_f0eaf102 | |
| 16:31:51 | gibi | there are two patches to look at | |
| 16:32:09 | gibi | https://review.opendev.org/c/openstack/nova/+/948064?usp=search now with test coverage | |
| 16:32:15 | gibi | and | |
| 16:32:16 | gibi | https://review.opendev.org/c/openstack/nova/+/948437?usp=search | |
| 16:32:40 | gibi | this week I will work on cleaning up the long service to make more patches ready to review | |
| 16:32:43 | gibi | that is all | |
| 16:32:46 | gibi | Uggla: back to you | |
| 16:33:16 | sean-k-mooney | gmaan: Uggla: just a quick update on the pyproject.toml change if i may | |
| 16:33:28 | sean-k-mooney | https://etherpad.opendev.org/p/pep-517-and-pip-23 is my old ether pad to track that work | |
| 16:33:33 | Uggla | thx gibi | |
| 16:33:37 | Uggla | sean-k-mooney, sure go ahead | |
| 16:33:40 | sean-k-mooney | and only the os-vif change above is not mergd for nova | |
| 16:33:47 | sean-k-mooney | so i rebased and appvoed that | |
| 16:33:55 | gmaan | just saw, ++ | |
| 16:33:57 | sean-k-mooney | once that is landed we shoudl be good | |
| 16:34:07 | Uggla | \o/ | |
| 16:34:43 | sean-k-mooney | anyway that was all on that topic | |
| 16:35:09 | Uggla | gibi, just a question you have said : the oslo.service threading backend uses forks it means real processes and not threads ? | |
| 16:35:24 | gibi | Uggla: no we want worker processes with thread pool | |
| 16:35:26 | gibi | pools | |
| 16:35:44 | gibi | but it uses os.fork to create the worker from the main process | |
| 16:35:44 | sean-k-mooney | context is oslo service | |
| 16:35:55 | sean-k-mooney | allows you to have multiple workers | |
| 16:36:00 | gibi | fork copies the state of the process | |
| 16:36:01 | sean-k-mooney | we use that in the schduler and conductor | |
| 16:36:12 | sean-k-mooney | but not nava-comptue or the api | |
| 16:36:17 | gibi | we would need os.spawn to get a totally new worker process | |
| 16:36:22 | gibi | without inheriting state | |
| 16:37:04 | Uggla | ok I think I have understood the pb. | |
| 16:37:04 | gibi | the wa is to reset the problematic state | |
| 16:37:13 | gibi | after the fork | |
| 16:37:25 | gibi | but I think the real solution would be os.spawn | |
| 16:37:48 | gibi | but changing to that is not super simple as oslo.config has non pickleable lambdas :/ | |
| 16:37:56 | gibi | details are in the linked gerrit comment | |
| 16:38:07 | gibi | and are up in the today IRC log | |
| 16:38:16 | dansmith | we can reset after the fork (as you're doing) or reset before the fork, as I suspect will also work and be less janky | |
| 16:38:23 | gibi | yepp | |
| 16:38:30 | gibi | I will try dansmith's suggestion next | |
| 16:38:40 | gibi | while we wait for the oslo folks to respond | |
| 16:38:47 | sean-k-mooney | https://github.com/openstack/oslo.config/blob/d6e5c96d6dbeec0db974dfb8afc8e508b74861e5/oslo_config/cfg.py#L1387 | |
| 16:38:52 | dansmith | I agree the real solution is spawn, but we probably don't want to wait | |
| 16:38:55 | sean-k-mooney | that is there only use of lambda i think by the way | |
| 16:39:23 | gibi | dansmith: yepp we won't wait, we will go with the WA, and adapt when oslo makes the move | |
| 16:39:52 | dansmith | +1 | |
| 16:40:39 | Uggla | moving on | |
| 16:40:49 | Uggla | #topic Open discussion | |
| 16:41:30 | Uggla | fwiesel, wants to propose something about Cross hypervisor resize. | |
| 16:41:45 | gibi | I guess that was last week's topic | |
| 16:42:07 | Uggla | yep but I understood you wanted to discuss it more. | |
| 16:42:09 | gibi | or maybe a continuation | |
| 16:42:21 | fwiesel | Yes, but there was limited feedback. I mean, if no one feels strongly about it, then I can go forward with a blueprint. | |
| 16:43:39 | fwiesel | We would like to allow mobility between two hypervisors, and I was thinking the cross-cell migration might already cover it to a large degree. | |
| 16:44:05 | fwiesel | The question is though, if that is a use-case you would feel worthwhile supporting or rather not. | |
| 16:44:50 | dansmith | I think it _has_ to be more like cross-cell migration than regular | |
| 16:45:10 | dansmith | I'm a bit mixed on whether or not I think this is worth it, because I suspect there will be a lot of gotchas in the image properties | |
| 16:45:24 | dansmith | and because you can pretty much do this with snapshot yourself now | |
| 16:45:37 | dansmith | and because we won't really be able to test it regularl I don't think | |
| 16:46:27 | gibi | yeah testing this will be painful | |
| 16:46:32 | fwiesel | Okay, got it. That's fine. | |
| 16:46:55 | sean-k-mooney | dansmith: specicly like updating the hw_vif_model for say the vmware one to virtio if they dont happen to have common values already | |
| 16:46:55 | dansmith | yeah, all that kind of stuff | |
| 16:47:21 | fwiesel | Well, we would set those in the flavours and that would override that. | |
| 16:47:39 | sean-k-mooney | so general question. is there anyting we know off that would prevent you form shelving, modifying them in glance and unshleving? | |
| 16:47:54 | fwiesel | No, it is more a usability issue. | |
| 16:48:02 | fwiesel | For our users. | |
| 16:48:08 | dansmith | sean-k-mooney: if the flavor you booted from was vmware-specific you can't unshelve to a libvirt-y one right? | |
| 16:48:18 | sean-k-mooney | fwiesel: so historically flavors are for amounts and images propereis are for changing how the devices are presented | |
| 16:48:31 | fwiesel | And we were thinking of using shared NFS shares to avoid going through image upload, etc... | |
| 16:48:36 | sean-k-mooney | fwiesel: so in the past the precendet was dont put device models in flavors | |
| 16:48:49 | sean-k-mooney | dansmith: correct | |
| 16:49:01 | sean-k-mooney | dansmith: what im thinkign is we dicussed allowing resize in the sheleved state | |
| 16:49:08 | dansmith | to me, snapshot, tweak, boot fresh is the best pattern here | |
| 16:49:18 | sean-k-mooney | so the workflow would be shelve. resize , update image properties and then unshleve | |
| 16:49:22 | dansmith | sean-k-mooney: that's a whole other conversation | |
| 16:49:48 | fwiesel | But I got it... hard to test so, no takers... Perfectly understandable. | |
| 16:49:49 | sean-k-mooney | ya im just confirming if there is a way to do that today with the api we have and i think the gaps are resize while shleved | |
| 16:50:23 | opendevreview | Balazs Gibizer proposed openstack/nova master: DNM:Run nova-next with n-sch in threading mode https://review.opendev.org/c/openstack/nova/+/948450 | |
| 16:50:36 | fwiesel | Thanks for the feedback. | |
| 16:50:36 | sean-k-mooney | but ya snapshot. detach volumes/ports and boot new vms with thos is a valid workflow too | |
| 16:52:49 | Uggla | Shall I say it is ok for fwiesel to propose something that goes in that direction ? | |
| 16:53:44 | fwiesel | My understanding is, I won't propose things as it is doable with the existing API and implementing it within Nova is hard to test and maintain. | |
| 16:53:48 | dansmith | I think anyone can propose anything they want :) | |
| 16:54:02 | dansmith | fwiesel: ++ | |
| 16:54:32 | Uggla | oh ok good. | |
| 16:56:11 | Uggla | We are almost at the top of the hour. Are you ok triaging a couple of bugs or would you prefer to do it next week ? | |
| 16:58:00 | Uggla | sean-k-mooney, dansmith ^^ | |
| 16:59:48 | Uggla | ok no answer so you might be busy. So closing the meeting. | |
| 16:59:59 | Uggla | thanks all | |
| 17:00:02 | gibi | Uggla: thanks | |
| 17:00:06 | fwiesel | thanks everyone | |
| 17:00:10 | Uggla | #endmeeting | |
| 17:00:10 | opendevmeet | Meeting ended Tue May 6 17:00:10 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 17:00:10 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2025/nova.2025-05-06-16.02.html | |
| 17:00:10 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2025/nova.2025-05-06-16.02.txt | |