| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-08-04 | |||
| 13:56:00 | sean-k-mooney | tkajinam: are you runing on an mad cpu | |
| 13:56:04 | sean-k-mooney | *amd | |
| 14:00:37 | sean-k-mooney | im running the test now locally to confirm but i also didnt see the failures on my intel cpu | |
| 14:01:37 | sean-k-mooney | and if i cat the file | |
| 14:01:39 | sean-k-mooney | [15:01:17]> cat /sys/module/kvm_amd/parameters/sev | |
| 14:01:41 | sean-k-mooney | [bat error]: '/sys/module/kvm_amd/parameters/sev': No such file or directory (os error 2) | |
| 14:02:20 | sean-k-mooney | i get an error so it would fail on this laptop if the mock didnt work | |
| 14:02:22 | sean-k-mooney | Ran: 17007 tests in 214.5597 sec. | |
| 14:02:24 | sean-k-mooney | - Passed: 16940 | |
| 14:02:26 | sean-k-mooney | - Skipped: 65 | |
| 14:02:28 | sean-k-mooney | - Expected Fail: 2 | |
| 14:02:30 | sean-k-mooney | - Unexpected Success: 0 | |
| 14:02:32 | sean-k-mooney | - Failed: 0 | |
| 14:02:34 | sean-k-mooney | Sum of execute time for each test: 3262.8758 sec. | |
| 14:02:43 | sean-k-mooney | ill try recreating the env just to make sure but on py 3.14 it seams to be working | |
| 14:03:29 | sean-k-mooney | oh | |
| 14:03:45 | sean-k-mooney | maybe this only fails on amd cpus | |
| 14:04:06 | sean-k-mooney | because we may be skiping earlier on intel before we try and read the path | |
| 14:10:03 | dansmith | succeeds on macos arm64, FWIW, so it'd have to be AMD_specific | |
| 14:10:04 | frickler | I have an VM on amd where I can reproduce this, I can give you access if that helps | |
| 14:16:10 | frickler | does this imply that all of our zuul nodes are on intel? | |
| 14:17:10 | frickler | oh, or does it need kvm enabled? maybe we can try running a tox job on a big kvm node | |
| 14:19:23 | sean-k-mooney | frickler: thre are a couple of diffent checks for sev supprot so i would not be surprised if we side step the fialign check on the amd vms as well | |
| 14:19:47 | sean-k-mooney | from pi | |
| 14:19:49 | sean-k-mooney | I found no unpatched access to /sys/module/kvm_amd/parameters/sev in the current checkout. | |
| 14:19:51 | sean-k-mooney | - nova/tests/fixtures/libvirt.py:2727 forces the SEV parameter to appear absent in LibvirtFixture. | |
| 14:19:53 | sean-k-mooney | - SEV-specific tests explicitly patch os.path.exists and open. | |
| 14:19:55 | sean-k-mooney | - SysFsPoisonFixture raises if a unit test actually opens an unmocked /sys path. | |
| 14:19:57 | sean-k-mooney | Validation: | |
| 14:19:59 | sean-k-mooney | - test_driver: 1,186 tests passed | |
| 14:20:01 | sean-k-mooney | - test_host: 127 tests passed | |
| 14:20:03 | sean-k-mooney | One unrelated potential typo exists: the fixture patches sev-es, while the production path is sev_es. | |
| 14:20:23 | sean-k-mooney | tkajinam: ^ | |
| 14:23:01 | sean-k-mooney | im checkign what that actully is now but also in an irc meeting | |
| 14:24:36 | sean-k-mooney | clif: just to confirm were the fialing teest sev specific or unrelated | |
| 14:24:59 | opendevreview | Dr. Jens Harbott proposed openstack/nova master: DNM: Test unit tests on kvm https://review.opendev.org/c/openstack/nova/+/999729 | |
| 14:25:43 | frickler | sean-k-mooney: one test was nova.tests.unit.virt.test_virt_drivers.LibvirtConnTestCase.test_unplug_vifs_with_destroy_vifs_true , doesn't look related to me | |
| 14:25:45 | sean-k-mooney | we may have mocked it in all the sev paths an missed it in the genric tests paths | |
| 14:25:54 | sean-k-mooney | ack | |
| 14:29:40 | opendevreview | Dr. Jens Harbott proposed openstack/nova master: DNM: Test unit tests on kvm https://review.opendev.org/c/openstack/nova/+/999729 | |
| 14:31:39 | clif | sean-k-mooney: every test failure I still have in traceback is failing due to the SEV exception | |
| 14:31:54 | clif | I can re-run and save all output | |
| 14:36:51 | clif | https://gist.github.com/ClifHouck/b7186501897e8276afcc9a7827e21789 here's a full traceback of one error | |
| 14:38:54 | clif | yep, I get 217 failures and they're all due to this SEV exception | |
| 14:41:42 | frickler | ok, my test triggers the same failures. maybe add that permanently at least as experimental or periodic job? | |
| 14:42:31 | frickler | https://zuul.opendev.org/t/openstack/build/1af158b6480d4783b0211c4f4d890fed | |
| 14:45:23 | tkajinam | ohhk let me check that | |
| 14:46:54 | sean-k-mooney | frickler: ok so its related to kvm? | |
| 14:48:01 | frickler | either that or it is the mad cpu and we only have that in vexxhost, not 100% sure yet | |
| 14:48:53 | clif | I assume this check is only triggered if an amd cpu is detected | |
| 14:49:43 | sean-k-mooney | clif: yes the exact check is complicated as we are also checking if libvirt has supprot too | |
| 14:50:11 | tkajinam | I suspect that there are something patched properly in the tests which eventually let the driver code to read that path | |
| 14:50:18 | tkajinam | I've not yet identified that yet though | |
| 14:52:29 | sean-k-mooney | File "/home/zuul/src/opendev.org/openstack/nova/nova/virt/libvirt/driver.py", line 7913, in _get_mem_encryption_config | |
| 14:52:31 | sean-k-mooney | if not self._host.supports_mem_encryption: | |
| 14:52:42 | sean-k-mooney | so that the trigger right | |
| 14:52:53 | sean-k-mooney | _get_guest_config | |
| 14:53:03 | sean-k-mooney | is calling _get_mem_encryption_config | |
| 14:53:16 | sean-k-mooney | whic eventully calls the sev check | |
| 14:53:24 | sean-k-mooney | but that happing alwasy now | |
| 14:54:04 | sean-k-mooney | so a way to fix this is in the libvirt fixutre alwasy mock _kernel_supports_amd_sev | |
| 14:54:14 | sean-k-mooney | and save it in self | |
| 14:54:26 | sean-k-mooney | then you can set the return value as needed in the sev tests | |
| 14:55:30 | sean-k-mooney | most unit test shoudl not get as far as calling _kernel_supports_amd_sev but if they do just returning False by defualt shoudl be fine | |
| 14:55:56 | sean-k-mooney | and you can either disbale the mockign with a class var or save the refence to the mock so you can modify it for the test that do | |
| 14:57:30 | sean-k-mooney | https://github.com/openstack/nova/blob/39dc667a1f9a7c12814f5892b900462560b1a229/nova/virt/libvirt/host.py#L2097 | |
| 14:57:35 | sean-k-mooney | i think that is the new call | |
| 14:57:43 | sean-k-mooney | added in https://github.com/openstack/nova/commit/7dd0d6106bdcf9babb81d7b55fd775b543a6e9ad | |
| 15:02:12 | sean-k-mooney | tkajinam: https://github.com/openstack/nova/commit/7dd0d6106bdcf9babb81d7b55fd775b543a6e9ad#diff-319aeacd4767563ecfa85b0690ccb4dfbdc9ce39173d08058f14cdce764c5040L2728-L2741 | |
| 15:02:32 | sean-k-mooney | so actully it this fixutre change i think | |
| 15:03:30 | sean-k-mooney | the libvirt fixure used ot alwasy mock the calls to read those files | |
| 15:04:47 | sean-k-mooney | self.useFixture(fixtures.MonkeyPatch('os.path.exists', fake_exists)) was overlly broad | |
| 15:05:01 | sean-k-mooney | but you can do a much more targeted mock in the same place | |
| 15:06:32 | sean-k-mooney | self.kernel_sev = self.useFixture(fixtures.MockPatch('nova.virt.libvirt.host.Host._kernel_supports_amd_sev', return_value=False).mock | |
| 15:30:44 | tkajinam | sean-k-mooney, I avoided that fixture in the base class so that I don't have to un-patch it later in test cases where these should be detected properly, but I may have to recheck that approach. | |
| 15:31:28 | tkajinam | but I have to revisit that approach, I agree | |
| 15:31:52 | sean-k-mooney | you wont need to unpatch it just modify the mock that is saved in self | |
| 15:32:13 | sean-k-mooney | but ya i get the intent of remvoign the broad mock | |
| 15:40:05 | tkajinam | yup | |
| 15:41:07 | tkajinam | I'm too tired to dig into it now but will take a further look tomorrow. | |
| 15:41:52 | tkajinam | I guess open is not triggered in intel machine with kvm due to absence of that path (and os.path.exists is not poison-pilled | |
| 16:10:10 | opendevreview | Joan Gilabert proposed openstack/nova master: Add libvirt driver MDEV ARQ support for Cyborg https://review.opendev.org/c/openstack/nova/+/995840 | |
| 16:10:10 | opendevreview | Joan Gilabert proposed openstack/nova master: Report OWNER_NOVA trait on resource providers https://review.opendev.org/c/openstack/nova/+/994299 | |
| 16:11:54 | opendevreview | Elod Illes proposed openstack/nova stable/2025.1: [stable-only] Set skip-level grenade as voting https://review.opendev.org/c/openstack/nova/+/999742 | |
| 16:58:06 | gmaan | gibi: did you see me reply on thread pool executor unifying idea, https://review.opendev.org/c/openstack/nova/+/998571/comment/23f68a57_21843d10 | |
| 16:58:31 | gmaan | i mentioned 3 options to unify those, if you can check and which one is more alligned with what you are thinking about iut | |
| 18:44:14 | opendevreview | Joan Gilabert proposed openstack/nova master: Add libvirt driver MDEV ARQ support for Cyborg https://review.opendev.org/c/openstack/nova/+/995840 | |
| 18:44:14 | opendevreview | Joan Gilabert proposed openstack/nova master: Report OWNER_NOVA trait on resource providers https://review.opendev.org/c/openstack/nova/+/994299 | |
| 19:48:53 | opendevreview | Sebastian Krott proposed openstack/nova master: conductor: Clean up port-bindings for live migrations https://review.opendev.org/c/openstack/nova/+/978121 | |
| 19:48:54 | opendevreview | Sebastian Krott proposed openstack/nova master: network: Add host filtering to `cleanup_instance_network_on_host` https://review.opendev.org/c/openstack/nova/+/978122 | |
| 21:37:14 | opendevreview | Merged openstack/nova master: Stabilize test_failed_count_with_anti_affinity_violation https://review.opendev.org/c/openstack/nova/+/998001 | |
| 21:39:24 | opendevreview | Merged openstack/nova master: (2/2)Stabilize test_create_images_negative_invalid_state https://review.opendev.org/c/openstack/nova/+/997719 | |
| 23:08:20 | opendevreview | Goutham Pacha Ravi proposed openstack/nova master: DNM: Add virtiofs share attachment CI jobs https://review.opendev.org/c/openstack/nova/+/999772 | |
| #openstack-nova - 2026-08-05 | |||
| 03:12:00 | opendevreview | Goutham Pacha Ravi proposed openstack/nova master: DNM: Add virtiofs share attachment CI jobs https://review.opendev.org/c/openstack/nova/+/999772 | |
| 03:12:30 | opendevreview | Goutham Pacha Ravi proposed openstack/nova master: DNM: Add virtiofs share attachment CI jobs https://review.opendev.org/c/openstack/nova/+/999772 | |
| 04:15:10 | opendevreview | Takashi Kajinami proposed openstack/nova master: Stub kernel support check of AMD-SEV in tests https://review.opendev.org/c/openstack/nova/+/999780 | |
| 04:16:00 | opendevreview | Dr. Jens Harbott proposed openstack/nova master: DNM: Test unit tests on kvm https://review.opendev.org/c/openstack/nova/+/999729 | |
| 04:26:32 | opendevreview | Takashi Kajinami proposed openstack/nova master: DNM: Test unit tests on kvm https://review.opendev.org/c/openstack/nova/+/999729 | |
| 04:55:43 | opendevreview | Goutham Pacha Ravi proposed openstack/nova master: DNM: Add virtiofs share attachment CI jobs https://review.opendev.org/c/openstack/nova/+/999772 | |
| 04:56:57 | opendevreview | Goutham Pacha Ravi proposed openstack/nova master: DNM: Add virtiofs share attachment CI jobs https://review.opendev.org/c/openstack/nova/+/999772 | |