Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-11-04
08:02:49 gibi I assume after the OOM kill libvirt will report that the VM is stopped. If so then nova will use the normal VM stop api to stop the VM properly https://github.com/openstack/nova/blob/32f58e8ad6a7ff896cc6ae8a361e3a18f5b35c9a/nova/compute/manager.py#L11190
08:02:50 tonyb gibi: I do see a server event: https://paste.opendev.org/show/bvD4k4YfaLIc7qYkc3dc/
08:04:17 gibi during the normal stop we record events to the DB and send notification on the message bus https://github.com/openstack/nova/blob/32f58e8ad6a7ff896cc6ae8a361e3a18f5b35c9a/nova/compute/manager.py#L3465-L3470
08:04:49 gibi tonyb: yepp that server event seems like the one happening if nova detects that the VM is in stopped state on the hypervisor
08:04:54 gibi but active in the DB
08:04:58 gibi and aligns the DB
08:05:03 tonyb gibi: Thanks
08:06:04 gibi note that the same would happen if somebody logs into the VM and shutdowns it from within the guest
08:06:27 gibi nova does not really know why the VM is suddenly stopped on the hypervisor
08:06:34 tonyb Ah okay. I can check if that was it but given it's review.o.o I doubt it
08:07:36 tonyb Yup. I was just wanting to verify that the event "mostly agrees" with the theory the the OOM-killer fired
08:09:50 gibi OK
08:09:58 gibi sounds like it agrees
08:36:04 tonyb gibi: perfect thanks
09:26:06 LarsErikP Hi. I reported this, after talking to melwitt back in september. Could anyone take a look? https://bugs.launchpad.net/nova/+bug/2125730
09:46:50 opendevreview Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949
09:46:51 opendevreview Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467
09:46:51 opendevreview Balazs Gibizer proposed openstack/nova master: Compute manager to use native thread pools https://review.opendev.org/c/openstack/nova/+/966016
09:49:49 opendevreview Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467
12:43:20 sean-k-mooney LarsErikP: i think dansmith fixed that but i dont knwo in which release
12:43:45 sean-k-mooney my guess is this si because nova in carical does not have the fix
12:44:53 sean-k-mooney caracal is now unmaintaiend so this bug is liekly not valid form a core team point of view but the fix might be backprotable
12:45:19 sean-k-mooney assuimg the cahge we merged for mast resolve the issue
12:46:07 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/930754
12:46:31 sean-k-mooney that in epoxy but not in older release
13:16:30 LarsErikP sean-k-mooney: ah ok. that makes sense. thanks
13:22:11 jlejeune hello guys, can I have some reviews please for my topics: https://review.opendev.org/q/topic:%22bug/2085135%22 and https://review.opendev.org/q/topic:%22bug-2044235%22 ?
13:22:34 jlejeune just matter of backports
14:25:15 opendevreview Balazs Gibizer proposed openstack/nova master: Libvirt event handling without eventlet https://review.opendev.org/c/openstack/nova/+/965949
14:25:16 opendevreview Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467
14:27:15 dansmith bauzas: in case you didn't see the mention, can you see my proposal here please? https://review.opendev.org/c/openstack/nova/+/941795/comments/fc49c07d_c06afef1
14:51:21 bauzas dansmith: nope sorry, working hard on something else, f*** agile :)
14:51:42 bauzas dansmith: I definitely want to review again the vtpm series, will do it hopefully today
14:52:59 dansmith bauzas: alright, well, that link is to a specific proposal for for merging the bottom part now, which I think you've mostly reviewed already
14:54:53 gibi dansmith: thanks for the review on switching the default to native threading. Did you missed that it has two pre-req patch in the series?
14:55:32 dansmith gibi: oh, no I saw it, but I got distracted before I followed that down, sorry about that
14:55:42 gibi no worries. Take you time :)
14:57:16 dansmith I had skimmed them looking to see if the api/__init__ was being added in this series or was already in place
14:58:54 gibi it was in place. Actually I needed to improve and move from from cmd/__init__ to cmd/<entrypoint2025-11-03 14:29:01.144038 | ubuntu-noble | oslo_service.backend.exceptions.BackendAlreadySelected: Backend already set to 'eventlet', cannot reinitialize with 'threading'>
14:59:00 gibi oops
14:59:08 gibi copy past buffer grrr
14:59:26 gibi it was in place. Actually I needed to improve and move from from cmd/__init__ to cmd/<entrypoint>.py
14:59:53 gibi so I can make the default different per entry point
15:00:04 dansmith you're planning to do that for api?
15:00:09 gibi but the wsgi entry point is differnt than the cmd one
15:00:37 dansmith right, but we _could_ have an entry point somewhere else for api/wsgi that does the patching before it imports...
15:00:46 gibi as both wsgi service we have are switching defaults at the same time I did not needed to touch the api part
15:01:31 dansmith yeah, I'm more concerned about just something else importing something from api to get a constant or something and screwing up
15:04:16 gibi I cannot just delay monkey_patch call in api wsgi as other imports might start using primitives that should have been monkey patched
15:05:26 gibi if another service imports from api then we assume that the another service already went through monkey patching. And oslo.service does not allow changing the backend type once it set so we will catch it
15:05:49 dansmith you can if you control the order of the importing in the entry point
15:05:51 gibi when the second monkey patch call tries to change the backend due to differning default
15:06:30 gibi dansmith: that is what we do, we added the monkey patch as early as possible in the import list
15:06:37 gibi that is our control
15:07:44 dansmith I'm not sure if you're not understanding what I'm saying, or don't think it's possible, or think it's a bad idea.. either way, I'm not arguing...just commenting about my opinion
15:07:52 gibi import wsgi from api first execute __init__ so we need to be in __init__ if __init__ has other imports like oslo.log that creates a lock
15:08:22 gibi I guess I'm missing what would you like to see in case of api/__init__
15:08:44 gibi s/import wsgi from api/from api import wsgi/
15:10:05 gibi I think it is not possible to move the monkey_patch to api/wsgi from api/__init__
15:10:32 gibi as api/__init__ has other imports
15:11:56 sean-k-mooney you could monkey patch before the other import
15:12:08 dansmith this ^
15:12:08 sean-k-mooney you might need to add a noqa or similar comment
15:12:11 dansmith but I don't want to argue
15:12:17 sean-k-mooney but we can if we need too
15:12:35 gibi but I monkey patch before all the imports
15:12:36 gibi :)
15:13:05 sean-k-mooney i have not been folloing the converstaion by they way
15:13:21 gibi the first line of nova/api/openstack/__init__.py is the monkey patch
15:13:47 gibi I think dansmith would like that to be moved somewhere else
15:13:51 sean-k-mooney ah yes i remoember dicussign the #noqa comment for that
15:14:38 gibi at the moment I'm not sure where to move our nova.api.openstack.compute.wsgi:init_application to be able to move the monkey_patch
15:14:41 sean-k-mooney so ya https://github.com/openstack/nova/blob/master/nova/api/__init__.py and https://github.com/openstack/nova/blob/master/nova/__init__.py are empty today
15:15:24 sean-k-mooney well the entry point ot that is https://github.com/openstack/nova/blob/master/nova/wsgi/osapi_compute.py
15:15:55 sean-k-mooney across openstack we are ment ot have <package>.wsgi.<api endpoint)
15:16:23 sean-k-mooney so you could monkey patch here https://github.com/openstack/nova/blob/master/nova/wsgi/__init__.py but im not sure it is required
15:16:25 gibi hm but our setup.cfg ponts to nova-api-wsgi = nova.api.openstack.compute.wsgi:init_application
15:16:49 sean-k-mooney that in wsgi script
15:16:52 sean-k-mooney but that is being deleted
15:17:05 sean-k-mooney it will generate the same script effectivly
15:17:42 sean-k-mooney we agreed at the ptg to finally delete https://github.com/openstack/nova/blob/master/setup.cfg#L87-L89
15:17:51 gibi but until that setup.cfg is a thing I need to make sure that works. So today I cannot move the monkey patch to nova/wsgi because that will not be executed if the setup.cfg defined entry point is used
15:18:35 gibi I agree that once the setup.cfg does not have such entry point defined and only the nova/wsgi is used then I can move the monkey_patch
15:18:37 sean-k-mooney right but lest just find and merge stephesn patch
15:18:44 sean-k-mooney then you can move it if needed
15:18:50 gibi OK
15:19:14 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/902688
15:19:40 sean-k-mooney he updated the docs too
15:20:22 sean-k-mooney so its a slightly longer review then i was plannign but ill re review that today/tomorrow
15:20:53 gibi so that patch suggest the deployers do the right thing and use nova/wsgi
15:21:17 sean-k-mooney yep which is how devstack has worked for 2 releases now
15:25:33 gibi I need to respin my defaulting series anyhow so I can rebase on top of stephenfin and move the monkey patch accordingly
15:37:29 stephenfin sean-k-mooney: if you have time, would you be able to continue and grab the other 3 patches on top of that one? https://review.opendev.org/c/openstack/nova/+/953703
16:00:05 sean-k-mooney my 1:1 just finished so sure.
16:10:09 gibi stephenfin: could you quickly fix the versioning issue in https://review.opendev.org/c/openstack/nova/+/902688/9#message-72481479904c69ce6f3adbcdfb26143c6f85756a then we can land that
16:13:33 sean-k-mooney gibi: ah the versionchanged line
16:13:39 sean-k-mooney ya that shoudl eb 33.0.0
16:14:14 opendevreview Stephen Finucane proposed openstack/nova master: setup: Remove pbr's wsgi_scripts https://review.opendev.org/c/openstack/nova/+/902688
16:14:15 opendevreview Stephen Finucane proposed openstack/nova master: Migrate mypy configuration to pyproject.toml https://review.opendev.org/c/openstack/nova/+/953703
16:14:15 opendevreview Stephen Finucane proposed openstack/nova master: Migrate codespell configuration to pyproject.toml https://review.opendev.org/c/openstack/nova/+/953704

Earlier   Later