| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-09-01 | |||
| 11:07:19 | sean-k-mooney | but at this point the conductor changes are targeting next cycle yes? | |
| 11:26:58 | gibi | sean-k-mooney: yes, conductor change is for G | |
| 11:27:37 | gibi | I did not imagine we will bump the global reqs so late, but it seems we did | |
| 11:28:17 | gibi | so the only thing left from eventlet in F is making the threading unit test job voting | |
| 11:28:42 | gibi | I will write up a list of topics regarding eventlet for the PTG to do the planning for G | |
| 11:29:38 | gibi | but we don't have https://etherpad.opendev.org/p/nova-2026.1-ptg yet | |
| 11:29:46 | gibi | cc Uggla ^^ :) | |
| 11:30:29 | opendevreview | Merged openstack/nova master: Replace remaining usage of Ubuntu Jammy https://review.opendev.org/c/openstack/nova/+/958989 | |
| 11:30:42 | opendevreview | Merged openstack/nova master: Drop CentOS 8 Stream https://review.opendev.org/c/openstack/nova/+/930445 | |
| 11:43:06 | sean-k-mooney | gibi: i added the boiler plate to that if you want ot start adding things | |
| 11:52:22 | Uggla | gibi, sean-k-mooney, I was going to create it, but sean-k-mooney did it already (thx). Please go ahead and add what you need. I will organize it later with the timeslot, owners etc... | |
| 11:54:59 | gibi | cool thanks folks | |
| 11:59:28 | opendevreview | Merged openstack/nova master: Follow-up of AMD SEV-ES support https://review.opendev.org/c/openstack/nova/+/958822 | |
| 12:00:11 | sean-k-mooney | Uggla: with ^ you can mark the sev blueprint as implmented just an fyi | |
| 12:00:23 | tkajinam | \o/ | |
| 12:03:17 | sean-k-mooney | this might be a nice feature for the cycle highlisths | |
| 12:03:24 | sean-k-mooney | i need to start on those for watcher | |
| 12:04:00 | sean-k-mooney | it proably does nto need to be in release note prelude but its a nice marketable feature for the highlights | |
| 12:05:01 | sean-k-mooney | the eventlet changes is kind of the reverse it would be good to note in the prelude but not really needed for the cycle highlights as it should be transparent ot end users | |
| 12:08:26 | Uggla | sean-k-mooney, I think that even though Eventlet is transparent for end users, it’s important to include it in the highlights and mention it, especially if we want users to test it and provide feedback. | |
| 12:08:48 | sean-k-mooney | Uggla: the cycle highlight are ment to aid the marketing team | |
| 12:08:54 | sean-k-mooney | so that why i was suggesting not | |
| 12:13:52 | Uggla | sean-k-mooney what do you suggest instead og highlights ? (note that maybe it is something we could discuss at the meeting tomorrow). | |
| 12:41:07 | bauzas | sean-k-mooney: I was about to discuss with Uggla on how to create cycle highlights and prelude indeed | |
| 12:41:33 | bauzas | the foundation asks for highlights but we also need the prelude note | |
| 13:30:44 | sean-k-mooney | so we have releasenotes for the eventlet stuff in general with a call to arms for operator to try it an provide feedback. i dont think that makes sense in the cycle highlight since they are emnt to be high level and marketing focused rather then technial. | |
| 13:30:55 | sean-k-mooney | but im not to pushsed either way | |
| 14:07:22 | stephenfin | idk if it's worth calling out the OpenAPI stuff. It's still not complete, but its damn close now and it's likely very relevant from a marketing perspective (particularly with an AI slant: machine readability etc. etc.) | |
| 14:32:53 | sean-k-mooney | stephenfin: all the request schemas are there right and most of the respocne schemas | |
| 14:33:12 | sean-k-mooney | so maybe | |
| 14:35:56 | opendevreview | Merged openstack/os-vif master: Remove remaining job with Ubuntu Jammy (22.04) https://review.opendev.org/c/openstack/os-vif/+/959019 | |
| 15:00:02 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Add runtime check for general additionalProperties https://review.opendev.org/c/openstack/nova/+/958858 | |
| #openstack-nova - 2025-09-02 | |||
| 06:53:16 | opendevreview | Rajesh Tailor proposed openstack/nova master: Fix 'nova-manage image_property set' command https://review.opendev.org/c/openstack/nova/+/946733 | |
| 08:39:14 | gibi | sean-k-mooney: Uggla: I've added the eventlet stuff to the PTG etherpad https://etherpad.opendev.org/p/nova-2026.1-ptg | |
| 09:34:45 | Uggla | gibi 👍 | |
| 11:02:43 | sean-k-mooney | elodilles: thank you for approving https://review.opendev.org/c/openstack/requirements/+/959051 in my head i was willing to grant the watcher-dashbaord review until our next watcher irc meeting to merge but that last of the 3 needed that release | |
| 11:04:30 | sean-k-mooney | its one of those things where its nice to complete the feature by including the horizon supprot this cycle but i also am not willing ot keep merging thing too much past FF even if we are just wait on release ectra | |
| 11:08:43 | opendevreview | Merged openstack/nova master: Reproduce that only half of the PCI devs are removed https://review.opendev.org/c/openstack/nova/+/953971 | |
| 11:55:57 | elodilles | sean-k-mooney: no problem. the python-watcherclient release happened yesterday, and new-release patches (upper constraint version bumps) should land as soon as possible anyway | |
| 11:57:38 | elodilles | sean-k-mooney: in the future: you could propose releases anytime if that is needed in other repository, no need to wait for the generated milestone release patches :) | |
| 11:58:13 | sean-k-mooney | gibi: i finished reviweeing https://review.opendev.org/q/topic:%22bug/2115905%22 the first 2 im +2 on although there are some nits related to the commit mesaage that would be nice to fix. https://review.opendev.org/c/openstack/nova/+/954149 i have a question/suggeetion on how to clsoe the schduer race for you to consider but im +1 over all on that | |
| 11:58:52 | sean-k-mooney | elodilles: actully that not what happend in this case, we only merged the api change on FF | |
| 11:59:17 | elodilles | (on the other hand watcher-dashboard is cycle-with-rc release typed project, so it still has time until the release, and FF is mostly discussed within the given project teams -> if the team decides that it's not risky for the release, then things can be merged after FF) | |
| 11:59:22 | sean-k-mooney | so we had to wait for that to merge before merging the clint change and doing the release | |
| 12:00:15 | sean-k-mooney | elodilles: the main issue with watcher-dashboard is there is almost no testing or test infra in that and we need to figure out how to resolve that going forward. | |
| 12:00:31 | elodilles | :S | |
| 12:00:34 | sean-k-mooney | elodilles: that means we need to carfully manually tests all changes so that takes time | |
| 12:00:44 | elodilles | i see | |
| 12:01:45 | sean-k-mooney | i want to eventully explore using playwrite (instead of creating selenium tests) to see if we can get some end to end test coverage but we also coudl use django unit test framework more then we do | |
| 12:02:07 | sean-k-mooney | horizon has historically never relaly facilated testing plugins well | |
| 12:02:23 | sean-k-mooney | so as a resutl most plugins were nto proeprly tested | |
| 12:02:53 | sean-k-mooney | fixing that is on our todo list just not as high as it proably shoudl be | |
| 12:06:13 | elodilles | maybe next cycle ;) | |
| 12:11:27 | grami[m] | Has anyone run into the problem with Nvidia GPU and requiring pci-expander-bus and vswitch-pcis? | |
| 12:19:56 | sean-k-mooney | grami[m]: it should not require them however they recommend them | |
| 12:20:01 | sean-k-mooney | for performance reasons | |
| 12:20:23 | sean-k-mooney | vswitch-pcis is not something im famialr with | |
| 12:21:20 | sean-k-mooney | nvidia recomemtn in one of there kvm docs that you attache thre gpus and nics to a virutal pcie switch in the vm | |
| 12:21:48 | sean-k-mooney | nova cant do that but its too hlep with device to device comuication within the vm | |
| 12:26:49 | sean-k-mooney | instead of conencting the gpu to a pcie-root-port you add a pcie-switch-upstream-port contoler instead and you can then group multipel gpus or other device as downstream ports on that switch. | |
| 12:27:18 | sean-k-mooney | there is very little upstream libvirt docs on how this all works https://libvirt.org/formatdomain.html#controllers | |
| 12:27:38 | sean-k-mooney | and hte nvidia docs on this topic are relivitly new. | |
| 12:27:54 | sean-k-mooney | but equally hard to actuly find | |
| 12:31:24 | grami[m] | sean-k-mooney: thanks yeah I'm in the process of passing h200 with the normal pci-root-port performance is worse then the CPU. I guess at some point I will have to right a patch but the nova code is confusing to me and need a starting point. Do you know what line in nova or the file to start in? | |
| 12:35:15 | sean-k-mooney | honestly this sould liek a nivida hardware bug as this really shoudl not have any impact if tis only for 1 h200 device | |
| 12:35:43 | sean-k-mooney | grami[m]: changign this is nto going to be easy in nova. there is an upgrade impact to this change and it will likely require a spec. | |
| 12:36:33 | sean-k-mooney | it may be possibel to jsut alwasy use a pcie-switch-upstream-port instead of pcie-root-port but i susspect that will increase teh qemu memory usage | |
| 12:36:46 | opendevreview | Rajesh Tailor proposed openstack/nova master: Fix duplicate words https://review.opendev.org/c/openstack/nova/+/959136 | |
| 12:36:50 | sean-k-mooney | so that may not be accpatabel to do in general | |
| 12:37:47 | grami[m] | sean-k-mooney: thanks for the info will do some digging on the spec and look to do one. | |
| 12:38:43 | sean-k-mooney | before gettign that far have you done any basic performace turning like ensuring numa affintiy between the vm and pci device | |
| 12:38:56 | sean-k-mooney | by defualt nova only does that if the guest has a numa toplgoy | |
| 12:39:25 | sean-k-mooney | i.e. addh hw:mem_page_size=small (exiplcit small 4k numa local pages) | |
| 12:39:46 | sean-k-mooney | will significnatly boost your vm performance if it was previosul going across numa | |
| 12:39:51 | grami[m] | sean-k-mooney: The memory issue is already a problem with 256gb as the BAR doesn't have enough so you have to increase the this just to get the GPU to boot. | |
| 12:40:25 | grami[m] | Already have numa dedicated CPU and huge page | |
| 12:41:02 | sean-k-mooney | that not what i was referint too. so each pci_root_port add a couple of megabyte of overhead even if un used | |
| 12:41:51 | sean-k-mooney | so qhen you enable the q35 machine type with the max number of pcie-root-ports we supprot today it has doble the qemu overhad as the pc machien type | |
| 12:42:21 | sean-k-mooney | usign a pcie-swithc might actully help with that but that one of the thing we woudl need to consier | |
| 12:42:40 | sean-k-mooney | a long time ago i wanted to allcoate 1 pci switch per guest numa node by default | |
| 12:42:59 | sean-k-mooney | and then attach the passhtough device to the relevnet virtual numa node pcie switch | |
| 12:43:14 | sean-k-mooney | so that the guess coudl make proper schdulign decisions | |
| 12:43:30 | sean-k-mooney | today all device in teh guest have nuam node -1 i.e. not numa info aviable | |
| 12:43:45 | sean-k-mooney | so the gest kernel assume its on virtual numa ndoe 0 effectvily | |
| 12:44:11 | sean-k-mooney | which can degreed performance so if you have not already tried expliclty having only 1 numa node in the guest that is wroth testing | |
| 12:44:33 | grami[m] | Ah yeah right now I'm going to see if I can get the instance to work by editing the xml then will look to see what needs to change. | |
| 12:45:10 | grami[m] | Yeah made sure it's only one numa zone for testing and will go from that. | |
| 12:46:53 | sean-k-mooney | right now i dont know of a way to change the bar space allocated to the viruatlised device. i assuem that need qemu supprot and im not sure if livbirt and qemu are hooked up such that it can be adjusted | |
| 12:47:48 | sean-k-mooney | there is memReserve | |
| 12:47:56 | sean-k-mooney | ```memReserve | |
| 12:47:58 | sean-k-mooney | Some PCI devices have non-prefetchable memory bar larger than 2MiB. Use this attribute to override value computed by firmware and thus make controller reserve more memory (in KiB) so that such PCI device can be hot plugged. For cold plugged PCI devices, the firmware will automatically reserve the correct amount of memory``` | |
| 12:48:25 | sean-k-mooney | but i dont think that is correect | |
| 12:59:59 | sean-k-mooney | grami[m]: you not trying to use resizable bar ar you? | |
| 13:04:16 | sean-k-mooney | just doing a little quick resarch qemu/libvirt wont do the resize for you you have to do that manually today before bindign the device to vfio-pci. second there is a limisation in ovmf by default where the adress apce resverd sames to default to 32GB for mmio mappings https://edk2.groups.io/g/discuss/topic/59340711 | |
| 13:05:37 | sean-k-mooney | we do not allow any use of <qemu:commandline> in nova so you woudl need that to be supproted in libvirt nativly before turnign the mmio apature via "fw_cfg name=opt/ovmf/X-PciMmio64Mb,string=65536" coudl be supproted | |
| 13:05:56 | gibi | sean-k-mooney: | |
| 13:06:07 | gibi | thanks for the review in https://review.opendev.org/c/openstack/nova/+/954149 | |
| 13:06:42 | gibi | I responded inline. bottom line your reserved idea is probably correct, but I'm afraid of the complexity of the patch already | |
| 13:07:08 | sean-k-mooney | gibi: that not a blocker | |
| 13:07:16 | sean-k-mooney | we can consider it in a followup | |