| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-02-05 | |||
| 12:18:10 | opendevreview | Max proposed openstack/nova master: fix: _get_guest_disk_device UnboundLocalError https://review.opendev.org/c/openstack/nova/+/975783 | |
| #openstack-nova - 2026-02-19 | |||
| 02:57:44 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Use 2nd RPC server in compute operations https://review.opendev.org/c/openstack/nova/+/975588 | |
| 04:34:26 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Prepare resize/cold migration for graceful shutdown https://review.opendev.org/c/openstack/nova/+/977182 | |
| 05:02:16 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Use 2nd RPC server in compute operations https://review.opendev.org/c/openstack/nova/+/975588 | |
| 05:46:46 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Prepare resize/cold migration for graceful shutdown https://review.opendev.org/c/openstack/nova/+/977182 | |
| 06:14:06 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Use 2nd RPC server in compute operations https://review.opendev.org/c/openstack/nova/+/975588 | |
| 06:24:03 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Prepare resize/cold migration for graceful shutdown https://review.opendev.org/c/openstack/nova/+/977182 | |
| 07:03:52 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Use 2nd RPC server in compute operations https://review.opendev.org/c/openstack/nova/+/975588 | |
| 07:11:09 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Prepare resize/cold migration for graceful shutdown https://review.opendev.org/c/openstack/nova/+/977182 | |
| 08:54:08 | opendevreview | Dominik proposed openstack/nova master: NUMA Topology with Resource Providers: Libvirt NUMA Migrate https://review.opendev.org/c/openstack/nova/+/971177 | |
| 09:22:42 | gibi | sean-k-mooney: hi, there is another iothreads fix from lajoskatona that ready to land https://review.opendev.org/c/openstack/nova/+/975934 | |
| 09:44:46 | opendevreview | Silvan Kaiser proposed openstack/nova master: libvirt: partial revert, Quobyte driver supported again https://review.opendev.org/c/openstack/nova/+/977300 | |
| 09:46:11 | opendevreview | Silvan Kaiser proposed openstack/nova master: libvirt: partial revert, Quobyte driver supported again https://review.opendev.org/c/openstack/nova/+/977300 | |
| 10:31:42 | sean-k-mooney | gibi: ah yes i didnt pull it donw since they respon it but ill take a look shortly | |
| 10:31:53 | sean-k-mooney | that was on my todo list thatnks for the reminder | |
| 10:42:53 | sean-k-mooney | gibi: ill have finsihed testing this in about 15 minute and then ill submit it but how do you feel about adding whitebox to check? we talked about it a few times but it actully caght this bug | |
| 10:43:57 | sean-k-mooney | i mentioned it offhand to jparker too but i recnectly became aware that the linux kernle has the ablity to emulate/report multiple numa nodes as well | |
| 10:44:31 | sean-k-mooney | with not much effort we coulf properly test live migration with and without cpu pinning and hugepages and numa in the first part ci | |
| 10:44:58 | sean-k-mooney | we have most of it already in whitebox we just need to run it and tweak the job slichtly if we want ot add numa | |
| 10:45:57 | sean-k-mooney | i need to tweak my old patch that we reverted for the basic testing of cpu_share_set and repo-spe that for nova-alt-config | |
| 10:51:19 | opendevreview | ribaudr proposed openstack/nova master: FUP Add HW_PCI_LIVE_MIGRATABLE trait to PCI resource providers https://review.opendev.org/c/openstack/nova/+/977310 | |
| 11:13:41 | sean-k-mooney | lajoskatona: gibi tested and approved it will hopefully merge later today and unblock https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/975225 | |
| 11:14:20 | sean-k-mooney | it has also been tested trasitivly via that change | |
| 11:17:47 | lajoskatona | sean-k-mooney, gibi: thanks | |
| 11:24:01 | priteau | Lots of jobs hitting post_failure in 2025.1 :( ModuleNotFoundError: No module named 'typing_extensions' | |
| 11:25:15 | priteau | That's the testtools issue | |
| 11:26:00 | priteau | Is there a patch for this already? | |
| 11:30:55 | gibi | priteau: as far as I know fix landed in testtools and we are waiting for a new release from testtools to get it | |
| 11:35:52 | gibi | priteau: https://github.com/testing-cabal/testtools/pull/570 | |
| 11:36:26 | priteau | https://pypi.org/project/testtools/ | |
| 11:36:29 | priteau | Released: 2 minutes ago | |
| 11:39:12 | priteau | But we would need a bump of upper-constraints? | |
| 11:39:35 | priteau | 2024.1 has testtools===2.7.1 | |
| 11:40:48 | tkajinam | priteau, no | |
| 11:41:00 | tkajinam | priteau, the failing task was common for all branches and doesn't use u-c | |
| 11:41:06 | tkajinam | s/was/is/ | |
| 11:43:25 | priteau | OK | |
| 11:43:30 | priteau | So recheck should be enough? | |
| 11:52:15 | frickler | likely we will need to build new images, as IIUC the broken venv is preconfigured there | |
| 12:02:38 | opendevreview | ribaudr proposed openstack/nova master: FUP Add HW_PCI_LIVE_MIGRATABLE trait to PCI resource providers https://review.opendev.org/c/openstack/nova/+/977310 | |
| 12:25:01 | tkajinam | frickler, seems so, looking at the failure still appearing | |
| 12:52:28 | opendevreview | ribaudr proposed openstack/nova master: FUP Add HW_PCI_LIVE_MIGRATABLE trait to PCI resource providers https://review.opendev.org/c/openstack/nova/+/977310 | |
| 12:56:26 | priteau | frickler: ah, this is why I couldn't see a fresh installation of testtools | |
| 13:08:31 | gibi | dansmith: gmaan: after I implemented the singel long task executor I realized that we have a logical problem https://review.opendev.org/c/openstack/nova/+/977251/1#message-ff7b23fb4a19eee42289ff428220461ae3df2da8 The recommended limit for the concurrent live migration is wildly different from the number of parallel builds | |
| 13:09:25 | gibi | I don't think deplolyers will accept this approach to have 1 lm - 1 build or 10 lm - 10 build, config. | |
| 13:09:47 | gibi | we recommend a single lm but that is not a useful value for concurrent builds | |
| 13:10:22 | gibi | this experiment shows me that we might not able to avoid the complexity in https://review.opendev.org/c/openstack/nova/+/975924/1 | |
| 13:12:38 | sean-k-mooney | gibi: ya those need to be split | |
| 13:13:05 | sean-k-mooney | we could still sue a semapor for teh ratelimiting | |
| 13:13:20 | sean-k-mooney | and just have the pool limit be seperate | |
| 13:13:33 | sean-k-mooney | we can warn if the semephor config option exceed the pool size | |
| 13:13:43 | sean-k-mooney | but i think that an ok compromise | |
| 13:33:37 | opendevreview | ribaudr proposed openstack/nova master: FUP Add HW_PCI_LIVE_MIGRATABLE trait to PCI resource providers https://review.opendev.org/c/openstack/nova/+/977310 | |
| 13:34:59 | gibi | sean-k-mooney: we cannot really take the semaphore approach for live migration as we rely on cancellability of live migrations. If I use the shared executor and take a semaphore within the lm task then lm task waiting for the semaphore becomes non cancellable | |
| 13:35:23 | gibi | on lm tasks waiting in an executor queue is cancellable | |
| 13:35:51 | sean-k-mooney | it depend on how we do cancelation i guess | |
| 13:35:59 | gibi | we cancel futures | |
| 13:36:16 | sean-k-mooney | right but live migation supprot direct cancelation via the api as well | |
| 13:36:25 | gibi | sure we can re-engineer the lm abort logic but then that is complexity | |
| 13:36:26 | sean-k-mooney | so even if its in progess we can actuly cancel it | |
| 13:36:44 | sean-k-mooney | ya so for now i guess we coudl keep those executor seperate | |
| 13:37:02 | sean-k-mooney | but i dont think we want a seprate executor per config optionright | |
| 13:37:11 | sean-k-mooney | at elast not unless the idel at 0 | |
| 13:37:42 | gibi | ahh there is different abort modes of cancellation of lm, but we basically loose the abort mode for lms that are not yet running due to the sempahore (limit'l | |
| 13:37:47 | gibi | (limit) | |
| 13:38:05 | sean-k-mooney | yes you can force compelte or abort | |
| 13:38:20 | gibi | if the lm is really executing the we abort via the driver | |
| 13:38:21 | sean-k-mooney | sorry i have not had time to properly load context | |
| 13:38:51 | sean-k-mooney | you are stating https://review.opendev.org/c/openstack/nova/+/975924/1 wont work right | |
| 13:39:01 | sean-k-mooney | or are you saying it will and we cant avoid the complexity | |
| 13:39:40 | gibi | I'm saying I think we probably need the complexity from https://review.opendev.org/c/openstack/nova/+/975924/1 as a shared executor in https://review.opendev.org/c/openstack/nova/+/977251/1#message-ff7b23fb4a19eee42289ff428220461ae3df2da8 does not work well | |
| 13:39:53 | sean-k-mooney | ack | |
| 13:40:28 | sean-k-mooney | i have not reviewd iether patch yet hence why im asking for your gut feeling on which one is more likely to work | |
| 13:41:30 | gibi | having a shared executor for build and snapshot might work, but we would need a separate for lm. Or we need the complexity from the wrapper that implements limits per task type. | |
| 13:41:54 | sean-k-mooney | i dont hate the idea of TaskTypeLimiterExecutorWrapper | |
| 13:42:14 | sean-k-mooney | i actully think we will want to have a task/proirty aware executro in the future | |
| 13:42:36 | sean-k-mooney | i think that is nessisary complexity rather then overengeinging given our usecases | |
| 13:42:53 | sean-k-mooney | but i ahve only really read the doc strings at this point | |
| 13:46:51 | sean-k-mooney | gibi: honestly i dont think https://review.opendev.org/c/openstack/nova/+/975924/1/nova/utils.py is that complex and i woudl personlly extend it to add a priorty filed to the task or type stuct or both | |
| 13:47:42 | sean-k-mooney | proirtyon type woudl mean all tasks fo this type has a default priorty we regestry when we registrer the type | |
| 13:48:22 | sean-k-mooney | that woudl allow use to still work properly if the concurnace on teh executor is less then the limtis for all the indiviual types | |
| 13:48:56 | sean-k-mooney | we can technially do that in your version as well as proposed | |
| 13:49:34 | sean-k-mooney | but obvioulsy we might want to express some prefence beyond arivle time in when the next task si enquened | |
| 13:50:12 | sean-k-mooney | gibi: if this is don correctly by the way this can just live en futureist | |
| 13:50:23 | sean-k-mooney | but im ok with the idea of building this out in nova first | |
| 13:53:39 | gibi | yeah | |
| 13:58:06 | gibi | given the closeness of FF I'm trying to make a way forward with a split approach. build and snapshot in a shared executor, and lm having its own executor. | |
| 13:58:22 | gibi | but I think the right approach is the TaskTypeLimiterExecutorWrapper | |
| 13:58:27 | gibi | we just don't have time | |
| 13:58:47 | sean-k-mooney | looking at it breifly i woudl be ok with proceeding in that direction too | |
| 13:59:22 | sean-k-mooney | that obviously not a rigours code review but directionally i think the trade offs you are makign make sense | |
| 14:01:14 | gibi | yeah if I would have 2 weeks I would go all in on the wrapper | |
| 14:01:33 | gibi | but if I want to be realistic and want nova-compute with threading in G then the split approach is less risky | |
| 14:11:29 | opendevreview | Balazs Gibizer proposed openstack/nova master: [compute]Use single long task executor https://review.opendev.org/c/openstack/nova/+/977251 | |
| 14:11:30 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 14:42:53 | dansmith | gibi: you're saying you don't think it's reasonable to lift the live migration limit to match what is probably the higher build limit? | |
| 14:43:45 | dansmith | live migrations are controlled by the admins, so I guess I don't see that as a fatal compromise for the time being, | |
| 14:44:59 | dansmith | but even if you go with the semaphore approach, I think you can still handle cancelation if the tasks each immediately check to see if they've been canceled after they acquire the semaphore and exit if so, no? the operation will appear to be a bit sticky until the (or a) current one finishes, but I imagine that's happening today... | |
| 15:35:20 | Uggla | FYI: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/VTLDDSXUHPKON3WOKNMATFL7ERHFPCYB/ | |
| 15:36:05 | sean-k-mooney | Uggla: my understandign was the bug scrub was previously ment ot happen in teh channel after the meeting | |