| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-11-03 | |||
| 16:25:39 | sean-k-mooney | i fell like we never got around to merging all of them but i coudl be wrgon | |
| 16:27:29 | gibi | nova-sch, -api, -metadata is now proposed to be switched to native threading by default as agreed on the PTG. Patch series https://review.opendev.org/c/openstack/nova/+/965924 | |
| 16:27:33 | gibi | nova-compute can boot VMs now in native threading mode with https://review.opendev.org/c/openstack/nova/+/965467/3 . Libvirt event handling does not work yet so live migration and device attach is failing. | |
| 16:28:01 | gibi | Uggla: this is the two news I had | |
| 16:28:28 | dansmith | gibi: nice | |
| 16:28:38 | Uggla | gibi 👍 | |
| 16:28:43 | fwiesel | sean-k-mooney: To be honest, I am not sure I submitted it to review yet. It was depending on the bug in cinder getting fixed, | |
| 16:29:13 | sean-k-mooney | fwiesel: ack | |
| 16:29:27 | sean-k-mooney | fwiesel: just wanted to make sure you were not blocked on reviwe on the nova side | |
| 16:29:41 | sean-k-mooney | gibi: cool i briefly looked last week but not in any detail | |
| 16:30:39 | fwiesel | sean-k-mooney: Thanks | |
| 16:31:57 | Uggla | #topic Open discussion | |
| 16:32:07 | Uggla | No agenda topic today. | |
| 16:32:23 | Uggla | Does someone want to share about something ? | |
| 16:32:28 | gibi | - | |
| 16:33:03 | bauzas | . | |
| 16:33:15 | sean-k-mooney | ` | |
| 16:33:23 | gibi | :) | |
| 16:33:32 | sean-k-mooney | i think that is a no | |
| 16:33:40 | gibi | ∅ | |
| 16:33:46 | sean-k-mooney | do we want to do some bug triage or give everyone back some time | |
| 16:34:19 | Uggla | I think we gonna keep this one short. Especially after last long week. | |
| 16:34:35 | sean-k-mooney | +1 | |
| 16:35:00 | Uggla | I will restart the rooster stuff for triage, along the week. it should be ready next week. | |
| 16:35:47 | fwiesel | Then, thanks everyone | |
| 16:35:49 | Uggla | anyway, time to close. Thanks for joining this meeting. Have a nice day/evening. | |
| 16:36:03 | Uggla | #endmeeting | |
| 16:36:03 | opendevmeet | Meeting ended Mon Nov 3 16:36:03 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 16:36:03 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2025/nova.2025-11-03-16.02.html | |
| 16:36:03 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2025/nova.2025-11-03-16.02.txt | |
| 16:36:03 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2025/nova.2025-11-03-16.02.log.html | |
| 16:36:06 | gibi | thakmks | |
| 16:36:11 | gibi | thanks | |
| 16:51:04 | opendevreview | Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949 | |
| 18:36:49 | dansmith | melwitt: so... I redeployed the vTPM series at the host patch this morning | |
| 18:37:39 | dansmith | and I'm able to create an instance in host mode as a user, stop it, and then restart it as the admin | |
| 18:37:59 | dansmith | which shows that we've actually improved things before we even get to the deployment mode and migration part of the series | |
| 18:38:02 | dansmith | that's expected right? | |
| 18:38:22 | melwitt | dansmith: yes that is expected | |
| 18:38:43 | melwitt | it was a small change in the 'host' support patch to make restart as admin work | |
| 18:38:50 | dansmith | confirming because I'm not really testing the tpm part, but the instance does have a tpm and it's not complaining when the admin does it | |
| 18:38:55 | dansmith | yeah, cool | |
| 18:39:39 | dansmith | so I'm thinking it would be good, since we have several +2s (some of which are stale) to just go ahead and merge up to the host patch to reduce the size of the series, claim some success and focus on the later bits | |
| 18:39:44 | dansmith | any problem with that? | |
| 18:39:51 | dansmith | I wish bauzas was here but I think he's already EOD | |
| 18:40:48 | melwitt | sounds cool to me | |
| 18:41:25 | dansmith | okay I'll make sure my +2s are good down the line from there and then we can pounce on him tomorrow | |
| 18:43:43 | melwitt | ++ | |
| #openstack-nova - 2025-11-04 | |||
| 06:47:01 | tonyb | review.o.o is down. When I check it was in shutoff state. That's the second time recently. One guess is that the hypervisor host's OOM killer is ... well killing the VM (it is a memory hog). | |
| 06:48:08 | tonyb | If that happened, and nova notices does nova take any action? Like maybe logging a fake/minimal server event | |
| 07:12:11 | tonyb | Maybe it looks like a "power-update" to nova.compute.api::external_instance_event and emits and event here: https://opendev.org/openstack/nova/src/commit/32f58e8ad6a7ff896cc6ae8a361e3a18f5b35c9a/nova/compute/api.py#L5994 ? | |
| 08:00:59 | gibi | tonyb: nova logs the following from a periodic tasks if detects that the power state of the VM is different than the power state of the domain on the hypervisor https://github.com/openstack/nova/blob/32f58e8ad6a7ff896cc6ae8a361e3a18f5b35c9a/nova/compute/manager.py#L11134 | |
| 08:01:11 | gibi | then aligns the power state in the DB | |
| 08:02:03 | tonyb | gibi: In this case i don't have access to the nova logs | |
| 08:02:49 | gibi | I assume after the OOM kill libvirt will report that the VM is stopped. If so then nova will use the normal VM stop api to stop the VM properly https://github.com/openstack/nova/blob/32f58e8ad6a7ff896cc6ae8a361e3a18f5b35c9a/nova/compute/manager.py#L11190 | |
| 08:02:50 | tonyb | gibi: I do see a server event: https://paste.opendev.org/show/bvD4k4YfaLIc7qYkc3dc/ | |
| 08:04:17 | gibi | during the normal stop we record events to the DB and send notification on the message bus https://github.com/openstack/nova/blob/32f58e8ad6a7ff896cc6ae8a361e3a18f5b35c9a/nova/compute/manager.py#L3465-L3470 | |
| 08:04:49 | gibi | tonyb: yepp that server event seems like the one happening if nova detects that the VM is in stopped state on the hypervisor | |
| 08:04:54 | gibi | but active in the DB | |
| 08:04:58 | gibi | and aligns the DB | |
| 08:05:03 | tonyb | gibi: Thanks | |
| 08:06:04 | gibi | note that the same would happen if somebody logs into the VM and shutdowns it from within the guest | |
| 08:06:27 | gibi | nova does not really know why the VM is suddenly stopped on the hypervisor | |
| 08:06:34 | tonyb | Ah okay. I can check if that was it but given it's review.o.o I doubt it | |
| 08:07:36 | tonyb | Yup. I was just wanting to verify that the event "mostly agrees" with the theory the the OOM-killer fired | |
| 08:09:50 | gibi | OK | |
| 08:09:58 | gibi | sounds like it agrees | |
| 08:36:04 | tonyb | gibi: perfect thanks | |
| 09:26:06 | LarsErikP | Hi. I reported this, after talking to melwitt back in september. Could anyone take a look? https://bugs.launchpad.net/nova/+bug/2125730 | |
| 09:46:50 | opendevreview | Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949 | |
| 09:46:51 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 09:46:51 | opendevreview | Balazs Gibizer proposed openstack/nova master: Compute manager to use native thread pools https://review.opendev.org/c/openstack/nova/+/966016 | |
| 09:49:49 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 12:43:20 | sean-k-mooney | LarsErikP: i think dansmith fixed that but i dont knwo in which release | |
| 12:43:45 | sean-k-mooney | my guess is this si because nova in carical does not have the fix | |
| 12:44:53 | sean-k-mooney | caracal is now unmaintaiend so this bug is liekly not valid form a core team point of view but the fix might be backprotable | |
| 12:45:19 | sean-k-mooney | assuimg the cahge we merged for mast resolve the issue | |
| 12:46:07 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/930754 | |
| 12:46:31 | sean-k-mooney | that in epoxy but not in older release | |
| 13:16:30 | LarsErikP | sean-k-mooney: ah ok. that makes sense. thanks | |
| 13:22:11 | jlejeune | hello guys, can I have some reviews please for my topics: https://review.opendev.org/q/topic:%22bug/2085135%22 and https://review.opendev.org/q/topic:%22bug-2044235%22 ? | |
| 13:22:34 | jlejeune | just matter of backports | |
| 14:25:15 | opendevreview | Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949 | |
| 14:25:16 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 14:27:15 | dansmith | bauzas: in case you didn't see the mention, can you see my proposal here please? https://review.opendev.org/c/openstack/nova/+/941795/comments/fc49c07d_c06afef1 | |
| 14:51:21 | bauzas | dansmith: nope sorry, working hard on something else, f*** agile :) | |
| 14:51:42 | bauzas | dansmith: I definitely want to review again the vtpm series, will do it hopefully today | |
| 14:52:59 | dansmith | bauzas: alright, well, that link is to a specific proposal for for merging the bottom part now, which I think you've mostly reviewed already | |
| 14:54:53 | gibi | dansmith: thanks for the review on switching the default to native threading. Did you missed that it has two pre-req patch in the series? | |
| 14:55:32 | dansmith | gibi: oh, no I saw it, but I got distracted before I followed that down, sorry about that | |
| 14:55:42 | gibi | no worries. Take you time :) | |
| 14:57:16 | dansmith | I had skimmed them looking to see if the api/__init__ was being added in this series or was already in place | |
| 14:58:54 | gibi | it was in place. Actually I needed to improve and move from from cmd/__init__ to cmd/<entrypoint2025-11-03 14:29:01.144038 | ubuntu-noble | oslo_service.backend.exceptions.BackendAlreadySelected: Backend already set to 'eventlet', cannot reinitialize with 'threading'> | |
| 14:59:00 | gibi | oops | |
| 14:59:08 | gibi | copy past buffer grrr | |
| 14:59:26 | gibi | it was in place. Actually I needed to improve and move from from cmd/__init__ to cmd/<entrypoint>.py | |
| 14:59:53 | gibi | so I can make the default different per entry point | |
| 15:00:04 | dansmith | you're planning to do that for api? | |
| 15:00:09 | gibi | but the wsgi entry point is differnt than the cmd one | |
| 15:00:37 | dansmith | right, but we _could_ have an entry point somewhere else for api/wsgi that does the patching before it imports... | |