| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-08-27 | |||
| 07:59:30 | ralonsoh | so if we have a VF bond inside the VM (with trusted VFs), it is possible to change the port MAC. But that will make the SR-IOV agent to fail. When the MAC-PCI address pair doesn't match, it returns a warning and forces a full resync | |
| 07:59:58 | ralonsoh | it will repeat this every cycle (that means, every 2 seconds, by default) | |
| 08:00:24 | ralonsoh | did we considered somehow this scenario?? from the Neutron point of view | |
| 08:00:47 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2024.2: [doc]Clarify where to set pci_in_placement https://review.opendev.org/c/openstack/nova/+/958623 | |
| 08:27:21 | gibi | sean-k-mooney: dansmith: bauzas: we have both green nova-next and unit test results for the nova-conductor eventlet removal change. See my comment in https://review.opendev.org/c/openstack/nova/+/957088/6#message-5d92177333ca556fc30c6934603d37ab55cd161c | |
| 08:27:25 | gibi | sambork: ^^ | |
| 08:27:41 | bauzas | gibi: that's on my today's duties | |
| 08:27:52 | bauzas | I'm finishing up the AMD SEV-ES series | |
| 08:27:59 | gibi | I will do some local manual testing now and report back on the same patch | |
| 08:30:39 | bauzas | ++ | |
| 08:32:08 | gibi | sambork: as far as I see this is the only remaining ask form yesterday https://review.opendev.org/c/openstack/nova/+/957088/6#message-cd81d9ce4afbc379e730f15d8df8334a5303e42f | |
| 08:32:27 | sambork | gibi, ack, thanks | |
| 08:32:33 | gibi | sambork: hm | |
| 08:32:36 | gibi | sambork: wait a sec | |
| 08:33:45 | gibi | sambork: I have a small thing in the parent of that https://review.opendev.org/c/openstack/nova/+/957424 so if we anyhow want to touch the top then I would like to fix the bottom as well in the same run | |
| 08:34:22 | gibi | sambork: I guess it is easier if I just go and do both changes in the same push | |
| 08:34:46 | sambork | gibi, ok thanks a lot! | |
| 08:34:57 | gibi | sambork: also sean-k-mooney is happy that the doc/reno/zuul patch is separate so I will not squash that in https://review.opendev.org/c/openstack/nova/+/958575/1 | |
| 08:35:28 | gibi | bauzas: ^^ hold you reviews until I re-spin | |
| 08:36:02 | bauzas | ack, anyway, I was aiming to review only by the next 30 mins, no worries | |
| 08:36:39 | opendevreview | Merged openstack/nova stable/2024.1: Reproduce bug/2098496 https://review.opendev.org/c/openstack/nova/+/945100 | |
| 08:42:51 | bauzas | tkajinam: I finished reviewing your series, nothing holding (I mostly +2d) except one breaking change you introduce by requiring a SEV trait | |
| 08:43:55 | bauzas | I overlooked the spec and nothing was mentioned in the upgrade section, but I do feel that if we start requiring a trait, then we need to mention to the operator that he has to fully upgrade his computes then | |
| 08:44:05 | bauzas | or he wouldn't get old computes supporting SEV too | |
| 08:44:45 | bauzas | that's what I called a behavioural change | |
| 08:45:13 | bauzas | sean-k-mooney[m]: when you're up, I'd like your thoughts on the rolling upgrade scenario with SEV nodes | |
| 08:51:52 | opendevreview | Balazs Gibizer proposed openstack/nova master: Switch nova-conductor to use ThreadPoolExecutor https://review.opendev.org/c/openstack/nova/+/957088 | |
| 08:51:52 | opendevreview | Balazs Gibizer proposed openstack/nova master: Ask for pre-prod testing for native threading https://review.opendev.org/c/openstack/nova/+/957424 | |
| 08:51:53 | opendevreview | Balazs Gibizer proposed openstack/nova master: Allow to start unit test without eventlet https://review.opendev.org/c/openstack/nova/+/953436 | |
| 08:51:53 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-conductor in native threading mode https://review.opendev.org/c/openstack/nova/+/958575 | |
| 08:51:54 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]RPC using threading or eventlet selectively https://review.opendev.org/c/openstack/nova/+/953815 | |
| 08:51:54 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run unit test with threading mode https://review.opendev.org/c/openstack/nova/+/953475 | |
| 08:51:55 | opendevreview | Balazs Gibizer proposed openstack/nova master: [CI]Make nova-tox-py312-threading voting https://review.opendev.org/c/openstack/nova/+/955791 | |
| 08:51:56 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make RBD Tpool usage conditional https://review.opendev.org/c/openstack/nova/+/956089 | |
| 08:51:56 | opendevreview | Balazs Gibizer proposed openstack/nova master: Do not yield in threading mode https://review.opendev.org/c/openstack/nova/+/950994 | |
| 08:51:58 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix ProviderTree copying with threading Lock https://review.opendev.org/c/openstack/nova/+/956091 | |
| 08:51:58 | opendevreview | Balazs Gibizer proposed openstack/nova master: Make libvirt Tpool proxying conditional https://review.opendev.org/c/openstack/nova/+/956090 | |
| 08:52:00 | opendevreview | Balazs Gibizer proposed openstack/nova master: [vncproxy]Handle ssl.wrap_socket removal in py312 https://review.opendev.org/c/openstack/nova/+/955915 | |
| 08:52:00 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]Further categorization of disabled unit tests https://review.opendev.org/c/openstack/nova/+/956092 | |
| 08:52:02 | opendevreview | Balazs Gibizer proposed openstack/nova master: Warn on long task wait time for executor https://review.opendev.org/c/openstack/nova/+/952666 | |
| 08:53:03 | gibi | bauzas: done. you can go wild now :) | |
| 09:03:54 | bauzas | gibi: ack | |
| 09:06:11 | tkajinam | bauzas, the HW_CPU_X86_AMD_SEV trait is added even in compute nodes. https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L13223 | |
| 09:06:53 | tkajinam | bauzas, the trait hasn't been unused really in scheduling but it has been added to compute nodes with sev enabled. so it does not break scheduling to old compute nodes with SEV | |
| 09:06:54 | bauzas | oh, you're right, and in the reshape, you moved it to the nested RP | |
| 09:06:59 | tkajinam | yes | |
| 09:07:15 | bauzas | my brain forgot that while I was still reviewing it | |
| 09:07:20 | bauzas | sorry | |
| 09:07:23 | tkajinam | no, np | |
| 09:07:30 | bauzas | changing my vote then | |
| 09:07:49 | bauzas | tkajinam: I also have a slight concern, about some deprecation release value you change | |
| 09:07:55 | bauzas | not related to SEV | |
| 09:08:07 | tkajinam | I'll revert that change and push a new version | |
| 09:08:08 | tkajinam | one sec | |
| 09:08:19 | tkajinam | (the others are just rebased so +2 should be kept | |
| 09:09:30 | bauzas | cool | |
| 09:09:41 | opendevreview | Takashi Kajinami proposed openstack/nova master: Detect AMD SEV-ES support https://review.opendev.org/c/openstack/nova/+/925685 | |
| 09:09:42 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | |
| 09:09:42 | opendevreview | Takashi Kajinami proposed openstack/nova master: Add hw_mem_encryption_model image property https://review.opendev.org/c/openstack/nova/+/927706 | |
| 09:09:43 | opendevreview | Takashi Kajinami proposed openstack/nova master: Add functional test scenario for mixed SEV RPs https://review.opendev.org/c/openstack/nova/+/958562 | |
| 09:09:44 | bauzas | and thanks for being present | |
| 09:10:41 | tkajinam | np and thank you for the review ! | |
| 09:17:09 | gibi | sambork: bauzas: sean-k-mooney: dansmith: local testing of the image cache shows that we have a bug somewhere around executor shutdown as only one of the two hosts get the image if the executor size is 1, If I increase the executor size to 2 then both gets it. See logs in https://review.opendev.org/c/openstack/nova/+/957088/7#message-a1b56a536b88c1f4372a2d986364a1f61d5c75af | |
| 09:19:36 | bauzas | hmmm | |
| 09:19:52 | sambork | gibi, ack, checking | |
| 09:20:20 | gibi | If I have to guess then it is a difference in behavior in executor shutdown when there are queued tasks between eventlet and threading executors | |
| 09:20:42 | gibi | but I have to prove that... | |
| 09:24:30 | bauzas | tkajinam: saw my other concern about the nested RP deletion when the config changes ? | |
| 09:25:32 | bauzas | we should mention that in the docs | |
| 09:29:24 | opendevreview | Takashi Kajinami proposed openstack/nova master: Purge nested SEV RPs when SEV is disabled https://review.opendev.org/c/openstack/nova/+/958626 | |
| 09:29:34 | tkajinam | bauzas, yes I saw it... and think we can try ^^^ | |
| 09:30:33 | sambork | gibi, it maybe logic is a little bit different and in case of threadPoolExecutor we are not using work_queue in shutdown logic | |
| 09:30:36 | bauzas | I had that approach first when fixing the GPU SRIOV RPs | |
| 09:30:41 | tkajinam | bauzas, I noticed one difference between vGPU RP and SEV RP. The former is named according to definition in nova.conf while the other is named statically so purging SEV rp should be possible | |
| 09:30:59 | bauzas | tkajinam: yes, you're correct | |
| 09:31:16 | bauzas | we statically define the list of RPs based on what the op provided | |
| 09:31:30 | bauzas | while here, we automatically create the RPs based on what libvirt driver detects | |
| 09:32:11 | tkajinam | yeah | |
| 09:32:22 | bauzas | so yeah, we could do what you wrote (reporting total=0 so ensure_rps() will wipe it) | |
| 09:32:33 | bauzas | nice catch | |
| 09:32:37 | tkajinam | one challenge with vGPU is that it loose the name of RP when a record is completely removed from nova.conf | |
| 09:32:45 | tkajinam | I mean nova loose * | |
| 09:32:50 | bauzas | yes, this is config-driven | |
| 09:33:12 | tkajinam | s/loose/lose/ | |
| 09:33:13 | tkajinam | yeah | |
| 09:33:27 | tkajinam | but for sev we know the exact names, as you said | |
| 09:33:48 | bauzas | anyway, I'll quickly review your patch and then I'll see how much I can help gibi and sambork on their eventlet conductor patches | |
| 09:33:49 | tkajinam | (I updated the functional tests to prove it, too | |
| 09:33:56 | bauzas | tkajinam: perfect, thanks | |
| 09:35:31 | gibi | I confirmed it is the executor.sshutdow's behavior that is different https://paste.opendev.org/show/bmZZr707naiWYHMt7qim/ | |
| 09:36:38 | gibi | I feel this is a bug in futurist as python says all pending tasks should finish https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.shutdown | |
| 09:39:42 | gibi | yepp python's own concurrent.future.ThreadPoolExecutor works as expected so this is a futuris bug. I will file it | |
| 09:40:52 | bauzas | gibi: excuse my ignorance but I'm getting hard to find how you found that only one host got the image when comparing paste.opendev.org/show/b1TPrh7ESmcIqWuTZx3D/ and paste.opendev.org/show/bGwpu8My66GcyR8T12sW/ | |
| 09:41:31 | bauzas | I see that compute1 emitted a notification saying it cached the image but I have no clue where the other compute said nooope | |
| 09:41:42 | sambork | ok, so I assume that we are block here (creating some class witch will overwrite shutdown behaviour sounds like to much) | |
| 09:41:53 | gibi | Aug 27 09:07:07 aio nova-conductor[117076]: INFO nova.conductor.manager [None req-8fc723e3-ecc0-4ee3-a4aa-fa33aa2eaf94 admin admin] Preparing to request pre-caching of image(s) 342349b7-351a-470d-92ee-c99f581e55e7 on 2 hosts across 1 cells. | |
| 09:41:58 | bauzas | I mean, I trust you but I want to be able to recognize that in logs | |
| 09:41:59 | gibi | Aug 27 09:07:11 aio nova-conductor[117076]: INFO nova.conductor.manager [None req-8fc723e3-ecc0-4ee3-a4aa-fa33aa2eaf94 admin admin] Image pre-cache operation for image(s) 342349b7-351a-470d-92ee-c99f581e55e7 completed in 4.54 seconds; 1 cached, 0 existing, 0 errors, 0 unsupported, 0 skipped (down) hosts | |
| 09:42:24 | bauzas | oh the "1 cached, gotcha" | |
| 09:42:34 | bauzas | thanks | |