| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-07-29 | |||
| 15:22:10 | ralonsoh | I think this is working by accident | |
| 15:22:26 | opendevreview | Philipp Dreesens proposed openstack/nova-specs master: Add spec for bidirectional RPC liveness handshake https://review.opendev.org/c/openstack/nova-specs/+/984384 | |
| 15:22:28 | ralonsoh | --> The network-vif-plugged during live migration was sent by the SOURCE agent calling update_device_up (for the existing binding that's still active on the source), NOT by the destination agent. | |
| 15:23:11 | sean-k-mooney | that shoudl not happen becuse we set migrating_to | |
| 15:23:14 | sean-k-mooney | in the port profiel | |
| 15:23:23 | ralonsoh | I know, I changed this code | |
| 15:24:39 | sean-k-mooney | and only the host named there is ment to send it with yes you updated that code | |
| 15:25:56 | ralonsoh | sean-k-mooney, thanks a lot, I need to recheck the recent changes in the OVS agent | |
| 15:26:11 | ralonsoh | (done by me, trying to fix other issues) | |
| 15:32:18 | sean-k-mooney | as an aside "``os_vif_delegation=True`` and ``create_port=True``" will mena the tap is create before we add the ovs port | |
| 15:32:37 | sean-k-mooney | so in that case we wont see -1 for kernel ovs but we still will for dpdk | |
| 15:32:54 | sean-k-mooney | os we shoudl not have divergent code paths if we can aovid it and just alwasy supprot -1 | |
| 15:38:45 | ralonsoh | but isn't supporting ofport=-1 something wrong? | |
| 15:39:14 | ralonsoh | with ``create_port=True``, when the port is created? who creates it? | |
| 15:40:45 | ralonsoh | I need to leave now, I'll check this chat later. I'll also review the current OVS agent code | |
| 15:42:22 | sean-k-mooney | not in general. with create_port=True its not required | |
| 15:42:34 | sean-k-mooney | for kernel ovs | |
| 15:44:52 | sean-k-mooney | this is not the first time we have had to revert skipping prot wot ofprot=-1 https://github.com/openstack/neutron/commit/f1fa9529ee6753e2a3f47e4a6eb0286118520d25 | |
| 21:44:41 | opendevreview | Goutham Pacha Ravi proposed openstack/nova master: Allow rebuild for instances with virtiofs shares https://review.opendev.org/c/openstack/nova/+/999048 | |
| 22:12:20 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Task tracking mechanism for graceful shutdown https://review.opendev.org/c/openstack/nova/+/996299 | |
| 22:21:48 | gmaan | melwitt: thanks for review, ^^ updated the wrapper cache feedback. To make revert_resize better in shutdown, we need more work on some RPC redesign or so. I added the details in gerrit and reported a separate bug for tracking it for graceful shutdown part-3 or something to fix as a separate bug https://bugs.launchpad.net/nova/+bug/2162090 | |
| 22:22:35 | gmaan | we need to make revert_resize more sync between source and dest compute like live migration or other operations which involves two compute | |
| 22:45:30 | melwitt | gmaan: ack thanks | |
| 22:46:55 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Call manager graceful_shutdown while RPC server waits https://review.opendev.org/c/openstack/nova/+/996583 | |
| #openstack-nova - 2026-07-30 | |||
| 06:28:42 | opendevreview | Goutham Pacha Ravi proposed openstack/nova master: Allow rebuild for instances with virtiofs shares https://review.opendev.org/c/openstack/nova/+/999048 | |
| 07:39:49 | ralonsoh | sean-k-mooney, please, ping me whenever you have some time (https://bugs.launchpad.net/nova/+bug/2161894/comments/5) | |
| 07:40:13 | opendevreview | Joan Gilabert proposed openstack/nova master: Report OWNER_NOVA trait on resource providers https://review.opendev.org/c/openstack/nova/+/994299 | |
| 07:40:13 | opendevreview | Joan Gilabert proposed openstack/nova master: Add libvirt driver MDEV ARQ support for Cyborg https://review.opendev.org/c/openstack/nova/+/995840 | |
| 09:25:18 | opendevreview | Anton Iacobaeus proposed openstack/nova master: libvirt: Generalize memory encryption config https://review.opendev.org/c/openstack/nova/+/998466 | |
| 09:25:18 | opendevreview | Anton Iacobaeus proposed openstack/nova master: libvirt: Add memory encryption config for TDX https://review.opendev.org/c/openstack/nova/+/998608 | |
| 09:25:19 | opendevreview | Anton Iacobaeus proposed openstack/nova master: libvirt: Launch security Intel TDX https://review.opendev.org/c/openstack/nova/+/999249 | |
| 10:38:59 | opendevreview | Kamil Sambor proposed openstack/nova master: Add regression test for BFV evacuate + Cinder timeout https://review.opendev.org/c/openstack/nova/+/999261 | |
| 13:28:08 | gibi | bauzas: have you seen compile_mdev_samples step hanging in devstack? It seems to me it causing the nova-whitebox job to time out here https://zuul.opendev.org/t/openstack/build/db30a8569196402898d754be9ce85426/log/job-output.txt#33470 | |
| 13:28:23 | gibi | (or I'm not sure how knows more about that step) | |
| 13:28:41 | bauzas | hmmm, nope | |
| 13:30:15 | gibi | so far this is a single data point. I will report a bug if I see it happening multiple times | |
| 13:31:39 | bauzas | gibi: hmmm, when looking at the build logs, I see the samples created correctly | |
| 13:33:10 | bauzas | I can read " | |
| 13:33:10 | bauzas | 2026-07-30 09:14:57.345940 | controller | [3695 Async compile_mdev_samples:43255]: finished compile_mdev_samples with result 0 in 78 seconds | |
| 13:33:10 | bauzas | " | |
| 13:35:10 | gibi | are we looking at the same logs? | |
| 13:35:11 | gibi | 2026-07-30 09:24:02.142618 | compute-host | [3806 Async compile_mdev_samples:29935]: Waiting for completion of compile_mdev_samples running on PID 29935 (2 other jobs running) | |
| 13:35:16 | gibi | 334712026-07-30 09:24:02.182165 | compute-host | [3806 Async compile_mdev_samples:29935]: Signaling child to exit | |
| 13:35:19 | gibi | 334722026-07-30 09:24:03.318037 | compute-host | + inc/async:async_inner:64 : rc=0 | |
| 13:35:22 | gibi | 334732026-07-30 09:24:03.363702 | compute-host | [35342 Async configure_neutron_nova:35342]: finished successfully | |
| 13:35:25 | gibi | 334742026-07-30 12:01:31.411526 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tempest/playbooks/devstack-tempest.yaml@master] | |
| 13:35:34 | gibi | this shows at 09:24:02 that it starts compiling | |
| 13:35:45 | bauzas | I'm looking at https://zuul.opendev.org/t/openstack/build/db30a8569196402898d754be9ce85426/log/job-output.txt#33470 | |
| 13:35:49 | gibi | so this is not the same compilation that finishes at 09:14 | |
| 13:36:09 | bauzas | ah I see | |
| 13:36:20 | gibi | so on the controller it finished ot 09:14 but on the compute-host it just start at 09:24 | |
| 13:37:29 | bauzas | it was queued | |
| 13:37:36 | bauzas | but not started AFAICS | |
| 13:39:27 | bauzas | with the controller, we see the thread awaken up here https://zuul.opendev.org/t/openstack/build/db30a8569196402898d754be9ce85426/log/job-output.txt#16760 | |
| 15:04:59 | jgilaber | melwitt, sean-k-mooney hi I'm seeing some failures on the functional test from my patch adding the OWNER_NOVA trait that I don't quite understand | |
| 15:05:33 | jgilaber | the tests worked fine in patchset 10, and after what I thought were mostly cosmetic change they are failing apparently randomly https://review.opendev.org/c/openstack/nova/+/994299/10..13 | |
| 15:05:52 | jgilaber | e.g https://zuul.opendev.org/t/openstack/build/46982818a76243dfa64f66aaa34c2ef3 only one test fails but I see no error | |
| 15:06:34 | jgilaber | and in the same buildset the py313 job failed with different tests https://zuul.opendev.org/t/openstack/build/46982818a76243dfa64f66aaa34c2ef3 | |
| 15:07:49 | jgilaber | sorry wrong link, I wanted to paste https://zuul.opendev.org/t/openstack/build/1b76d14e3a7a48278e4d8b84027e546b | |
| 15:07:55 | melwitt | jgilaber: I was just starting to look at that. so far I am suspecting something with database access (perhaps improper transaction context manager nesting or something of that nature). I began to look through the bottom patch and I'm not sure about the location of the minimum service version check and trait addition, my inclination is it shouldn't be in the request spec object there, | |
| 15:08:15 | sean-k-mooney | jgilaber: you removed resting the cach | |
| 15:08:28 | sean-k-mooney | in https://review.opendev.org/c/openstack/nova/+/994299/10..13/nova/tests/functional/integrated_helpers.py | |
| 15:08:37 | melwitt | and there is a possibility that moving that part might change whatever is going wrong. BUT this is just my preliminary thoughts and I need to continue to look at it | |
| 15:08:45 | sean-k-mooney | you were clearign the cash but with the decoreateo you need ot call clear on the funtion | |
| 15:09:32 | jgilaber | right, but even then, that should only skip using the filter | |
| 15:09:49 | jgilaber | the scheduler should not hang because of that, right? | |
| 15:10:00 | sean-k-mooney | jgilaber: also what melwitt said but i woudl try clearing the cash like thi https://github.com/openstack/nova/blob/master/nova/test.py#L319 | |
| 15:10:36 | sean-k-mooney | well the filter is adding som eaddtional db calls | |
| 15:10:49 | sean-k-mooney | to get the min version | |
| 15:11:21 | sean-k-mooney | actully that is a bad exampel that using the run_once decortor not cache | |
| 15:12:02 | sean-k-mooney | https://docs.python.org/3/library/functools.html#:~:text=cache%5Fclear%28%29 | |
| 15:12:23 | sean-k-mooney | but ya we normally clear thsoe | |
| 15:12:25 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/test.py#L201 | |
| 15:13:45 | jgilaber | ack, thanks I'll try that | |
| 15:17:41 | sean-k-mooney | so lookign at the fialures more closely | |
| 15:18:28 | jgilaber | melwitt, I'm trying to understand your comment, I'm still not too familiar with most of the nova code | |
| 15:18:28 | sean-k-mooney | the failrue seam to be in in the clean up | |
| 15:18:45 | jgilaber | so you think that I should not add the version check in the request spec object? | |
| 15:19:56 | melwitt | jgilaber: I think it should be elsewhere yes but I will add detailed comments to the review on this once I go through it properly. I just wanted to give you a quick heads up, it is ok to ignore for now | |
| 15:20:19 | sean-k-mooney | melwitt: perhaps in the resouce translation logic | |
| 15:20:21 | melwitt | mainly I wanted to mention that there is a possibility it could be related to the random failures if there is something with the database session nesting, that's all | |
| 15:20:38 | jgilaber | ack, thanks, I'll respin with the cache_clear for now to see if that helps | |
| 15:20:48 | melwitt | sounds good | |
| 15:21:00 | jgilaber | thanks for looking! | |
| 15:22:47 | opendevreview | Joan Gilabert proposed openstack/nova master: Report OWNER_NOVA trait on resource providers https://review.opendev.org/c/openstack/nova/+/994299 | |
| 15:22:47 | opendevreview | Joan Gilabert proposed openstack/nova master: Add libvirt driver MDEV ARQ support for Cyborg https://review.opendev.org/c/openstack/nova/+/995840 | |
| 15:22:53 | sean-k-mooney | so what failing initally is a server show request | |
| 15:23:08 | sean-k-mooney | GET /v2.1/servers/9cb3f749-f071-450f-aa74-122d940bd364" status: 500 | |
| 15:24:23 | sean-k-mooney | but that is just us pooling for the delete | |
| 15:24:25 | sean-k-mooney | 2026-07-30 08:06:21,544 INFO [nova.api.openstack.requestlog] 127.0.0.1 "DELETE /v2.1/servers/9cb3f749-f071-450f-aa74-122d940bd364" status: 204 len: 0 microversion: 2.104 time: 0.071912 | |
| 15:24:28 | sean-k-mooney | to complete | |
| 15:24:52 | melwitt | on one of the HTTP 500 errors I saw that it got stuck in a sql connection code and then timed out | |
| 15:25:05 | melwitt | that's what made me think about database | |
| 15:25:11 | sean-k-mooney | yes | |
| 15:25:19 | sean-k-mooney | its coming form | |
| 15:25:21 | sean-k-mooney | File "/home/zuul/src/opendev.org/openstack/nova/nova/api/openstack/compute/views/servers.py", line 469, in show | |
| 15:25:23 | sean-k-mooney | server['server']['scheduler_hints'] = self._get_scheduler_hints( | |
| 15:25:25 | sean-k-mooney | ^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 15:25:27 | sean-k-mooney | File "/home/zuul/src/opendev.org/openstack/nova/nova/api/openstack/compute/views/servers.py", line 256, in _get_scheduler_hints | |
| 15:25:29 | sean-k-mooney | req_spec = objects.RequestSpec.get_by_instance_uuid( | |