| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-02-06 | |||
| 16:11:14 | sean-k-mooney | it would also look at the failing job | |
| 16:11:35 | sean-k-mooney | this happens because of very high load | |
| 16:11:58 | sean-k-mooney | so the job may be usign too high a concurancy or have other isseus that is triggering it | |
| 16:12:25 | ykarel | hmm underlying hypervisor where it ran might also be overloaded | |
| 16:13:44 | sean-k-mooney | i belive libvirt is now going to wait at least 40s instead of 15 before retuing EBUSY | |
| 16:22:38 | ykarel | yes as per logs i see 40s b/w requesting destroy and returned error | |
| 17:50:04 | opendevreview | Doug Szumski proposed openstack/nova master: Stop corrupting ephemeral volumes during live migration https://review.opendev.org/c/openstack/nova/+/940900 | |
| 17:53:52 | dougszu | Any opinions on that would be great ^. I've got another fix for cold migration using block pull to share if it's of interest. And of course, the final bit to close the bug would be not making file systems on the ephemerals at all. | |
| 19:04:10 | sean-k-mooney | dougszu: i see you tried removing the backing file | |
| 19:04:36 | sean-k-mooney | longterm definlty the direction we shoudl go | |
| 19:05:08 | sean-k-mooney | we likely would need to cahnge intal boot/reboot and cold migration or evacuate too | |
| 19:05:12 | sean-k-mooney | if we were to do this. | |
| 19:06:02 | sean-k-mooney | i.e procive a way for nova, on the next lifeccyle operation on the guest to move or redifien the domain, to remove the backing file for ephmeral and swap disk | |
| 19:07:23 | mikal | sean-k-mooney: yeah, I am using Debian containers on Kolla-Ansible, so that's why I didn't notice the Ubuntu behaviour. | |
| 19:07:49 | sean-k-mooney | i need to push my pathc to try the job with debian | |
| 19:08:05 | sean-k-mooney | i have been gettign distracted for like the last 3 hours | |
| 19:08:21 | mikal | Heh, its ok, I didn't manage to get to tempest for similar reasons yesterday. | |
| 19:09:03 | mikal | I find people dropping SPICE quite frustrating to be entirely honest. The public excuse seems to be H.264 requirements, but RDP has the same requirements and there is a patent license from Cisco for the codec. | |
| 19:09:28 | opendevreview | sean mooney proposed openstack/nova master: [WIP] move nova-ovs-hybrid-plug to deploy with spice and fix qxl default https://review.opendev.org/c/openstack/nova/+/940835 | |
| 19:09:28 | opendevreview | sean mooney proposed openstack/nova master: Dont deploy n-spice on compute nodes. https://review.opendev.org/c/openstack/nova/+/940873 | |
| 19:09:52 | sean-k-mooney | mikal: spice is the better console. vnc is catching up but spice is still better | |
| 19:10:15 | sean-k-mooney | mikal: i dont think it h.264 related honestly | |
| 19:10:25 | mikal | sean-k-mooney: yeah, RDP is probably about as good as SPICE in terms of features, but qemu doesn't support it at all and the protocol is much more complicated as best as I can tell. | |
| 19:13:24 | sean-k-mooney | whats funney is as far as i know its the default in gnome boxes and a few other things | |
| 19:13:36 | sean-k-mooney | so its not like spice was unsued | |
| 19:14:03 | mikal | oVirt used it a bit too, as did proxmox. | |
| 19:14:23 | mikal | Also... its pretty stable. It hasn't changed much if at all in years. | |
| 19:14:30 | sean-k-mooney | i think its still an option in proxmox given that derived form rhel | |
| 19:14:32 | mikal | So there's no feature chasing etc. | |
| 19:14:34 | sean-k-mooney | *debian | |
| 19:15:20 | sean-k-mooney | speaking of stable i have updated the patch to the latest debian stable (12/bookworm) | |
| 19:16:37 | sean-k-mooney | actully we do not need grenade and some of the other jobs ill quickly drop those out whiel we are testign | |
| 19:17:37 | opendevreview | sean mooney proposed openstack/nova master: [WIP] move nova-ovs-hybrid-plug to deploy with spice and fix qxl default https://review.opendev.org/c/openstack/nova/+/940835 | |
| 19:17:37 | opendevreview | sean mooney proposed openstack/nova master: Dont deploy n-spice on compute nodes. https://review.opendev.org/c/openstack/nova/+/940873 | |
| 19:18:46 | sean-k-mooney | ok better its just runnign one job now | |
| 19:18:54 | opendevreview | Michael Still proposed openstack/nova master: Don't calculate the minimum compute version repeatedly. https://review.opendev.org/c/openstack/nova/+/940848 | |
| 19:18:55 | opendevreview | Michael Still proposed openstack/nova master: libvirt: Add extra spec for sound device. https://review.opendev.org/c/openstack/nova/+/926126 | |
| 19:18:55 | opendevreview | Michael Still proposed openstack/nova master: Protect older compute managers from sound model requests. https://review.opendev.org/c/openstack/nova/+/940770 | |
| 19:18:56 | opendevreview | Michael Still proposed openstack/nova master: libvirt: Add extra specs for USB redirection. https://review.opendev.org/c/openstack/nova/+/927354 | |
| 19:19:07 | mikal | ^--- fixes a one character error in the first patch in that series | |
| 19:20:42 | sean-k-mooney | ah you have split out the sound/usb ones and the performance improvment, nice | |
| 19:22:55 | mikal | Yeah, I wrote up a decoder ring on the etherpad because its getting complicated -- https://etherpad.opendev.org/p/nova-2025.1-status | |
| 19:23:20 | mikal | But there's basically two independent series now -- the API changes, and the "VDI changes" (sound, usb, compute version bumps etc). | |
| 19:23:53 | mikal | I agree we should focus on the API changes first, but that's blocking right now on me getting some quality time with tempest. | |
| 19:27:27 | sean-k-mooney | in its current form both can be merged in either order and in parallel so that will help i think. | |
| 20:14:21 | sean-k-mooney | mikal: https://zuul.opendev.org/t/openstack/build/842a66f2cfe54eb39d856a3bf311bb5c | |
| 20:14:34 | sean-k-mooney | mikal: debian works and the job passes with it | |
| 20:15:04 | sean-k-mooney | so ill revert the commetned out jobs but we should be able to test in that job | |
| 20:27:39 | opendevreview | sean mooney proposed openstack/nova master: move nova-ovs-hybrid-plug to deploy with spice and fix qxl default https://review.opendev.org/c/openstack/nova/+/940835 | |
| 20:27:39 | opendevreview | sean mooney proposed openstack/nova master: Dont deploy n-spice on compute nodes. https://review.opendev.org/c/openstack/nova/+/940873 | |
| 20:34:21 | mikal | sean-k-mooney: cool, thank you. I assume we still want to do a tempest test, which I will try to actually get around to attempting today. | |
| 20:35:21 | sean-k-mooney | i think so but those should help with a job we can run the test in | |
| 20:35:56 | mikal | Agreed | |
| 20:36:05 | mikal | Thanks for chasing this bit for me. | |
| 20:36:09 | sean-k-mooney | i just added those to the etherpad too with some explanations | |
| 20:36:28 | mikal | Cool | |
| 20:36:45 | sean-k-mooney | mikal: if nothing else, we have not deprecated spice supprot and there is litrally no spice job anywhere so just for that i consider those to be a win | |
| #openstack-nova - 2025-02-07 | |||
| 06:11:58 | mikal | sean-k-mooney / dansmith / melwitt: https://review.opendev.org/c/openstack/tempest/+/940943 is an attempt at a tempest test for the spice-direct stuff. I think it likely needs to be entangled with sean-k-mooney's devstack and zuul config changes to actually work though. | |
| 06:13:14 | mikal | I've put it on the etherpad as well at the end of the API series. | |
| 07:37:58 | opendevreview | benlei proposed openstack/nova master: Abort live migration task when stop nova compute service https://review.opendev.org/c/openstack/nova/+/938223 | |
| 10:34:48 | opendevreview | Balazs Gibizer proposed openstack/nova master: Bump InstanceNUMACell version after data migration https://review.opendev.org/c/openstack/nova/+/940604 | |
| 10:34:49 | opendevreview | Balazs Gibizer proposed openstack/nova master: Bump InstanceNUMACell version after half migration https://review.opendev.org/c/openstack/nova/+/940953 | |
| 10:55:37 | Uggla | Tentative vGPU vfio discussing latest open points. Friday, February 7 · 14:00 – 14:45 UTC link: https://meet.google.com/hia-iodh-fcz | |
| 11:10:35 | opendevreview | Taketani Ryo proposed openstack/nova-specs master: Add a spec for libvirt driver launching Arm CCA-encrypted instances https://review.opendev.org/c/openstack/nova-specs/+/938276 | |
| 11:20:13 | opendevreview | Balazs Gibizer proposed openstack/nova master: Update InstanceNUMACell version after data migration https://review.opendev.org/c/openstack/nova/+/940604 | |
| 11:20:13 | opendevreview | Balazs Gibizer proposed openstack/nova master: Update InstanceNUMACell version in more cases https://review.opendev.org/c/openstack/nova/+/940953 | |
| 11:20:14 | opendevreview | Balazs Gibizer proposed openstack/nova master: Refactor InstanceNUMA* data migrations https://review.opendev.org/c/openstack/nova/+/940958 | |
| 11:30:01 | opendevreview | Balazs Gibizer proposed openstack/nova master: Update InstanceNUMACell version in more cases https://review.opendev.org/c/openstack/nova/+/940953 | |
| 12:40:58 | opendevreview | Balazs Gibizer proposed openstack/nova master: Update InstanceNUMACell version in more cases https://review.opendev.org/c/openstack/nova/+/940953 | |
| 12:41:25 | opendevreview | Rajesh Tailor proposed openstack/nova master: Add support for showing image properties in server show response https://review.opendev.org/c/openstack/nova/+/939649 | |
| 12:47:49 | gibi | dansmith: the pcpuset fix is now split and hopefully satisfies CI https://review.opendev.org/q/topic:%22bug/2097359%22 | |
| 12:52:29 | sean-k-mooney | mikal: ill be back on monday, im on pto today but we should be able to add a patch to nova on top of my series that depend on that and your api change and that will demonstrate it workign end to end, you can have depneds on in the same repo we just normally dont becuase its harder to debug if there is a merge conflcit, but it will mean we do not need to stack the patches. | |
| 15:09:44 | opendevreview | ribaudr proposed openstack/nova master: Bump os-traits to 3.3.0 in requirements https://review.opendev.org/c/openstack/nova/+/940975 | |
| 15:16:27 | opendevreview | ribaudr proposed openstack/placement master: Bump os-traits to 3.3.0 in requirements https://review.opendev.org/c/openstack/placement/+/940976 | |
| 16:05:36 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]generic OVO version update tests https://review.opendev.org/c/openstack/nova/+/940984 | |
| 16:07:23 | gibi | dansmith: ^^ this is my fist stab about the generic ovo version update testing | |
| 16:08:04 | dansmith | ack | |
| 16:08:09 | gibi | it already able to catch the pcpuset bug while showing that the same bug is not reproducible via request_spec.numa_topology just via instance.numa_topology | |
| 16:10:52 | dansmith | cool | |
| 16:14:24 | cardoe | hoping to get https://review.opendev.org/c/openstack/nova/+/940836 approved as a backport. I'll also propose it to 2024.1 which is where the issue was first introduced but my present use case is 2024.2 with ironic | |
| 16:25:01 | opendevreview | Sylvain Bauza proposed openstack/nova master: Add a new ImagePropertiesWeigher https://review.opendev.org/c/openstack/nova/+/940642 | |
| 16:30:01 | opendevreview | Doug Szumski proposed openstack/nova master: Stop corrupting ephemeral volumes during live migration https://review.opendev.org/c/openstack/nova/+/940900 | |
| #openstack-nova - 2025-02-10 | |||
| 10:54:36 | sean-k-mooney | gibi: i forgot to hit send, the numa code and pci code expect that its workign with an upleveled object at least for the instance numa objectss | |
| 10:55:05 | sean-k-mooney | so i disagre that that how nova was designed | |
| 10:55:22 | sean-k-mooney | it might have been how dansmith intenetd we use ovo in nova | |
| 10:55:51 | sean-k-mooney | but in practice its not the mental model i or others had when writing the numa features | |
| 10:58:05 | sean-k-mooney | we always wote the code assuming the obejct would be version matched to the current processes class defintion | |
| 11:00:45 | gibi | sean-k-mooney: yeah I would like to use our ovos in a way that if a field is defined in the `fields` dict of the class then i) the ovo has that field always ii) the ovo hides all the necessary version forward (or backward) migration logic from the caller | |
| 11:01:17 | gibi | This is not the first time such assumption of mine gets invalidated | |
| 11:01:18 | sean-k-mooney | we should never loose data by upleveling, (unless we stop supporting a feature) | |
| 11:01:33 | sean-k-mooney | i dont think its invlaide | |
| 11:01:45 | sean-k-mooney | i think dansmith has been overcome by events | |
| 11:02:09 | sean-k-mooney | i.e. even if there view poitn was the orginal intent it not how we have been using them in the code for years | |
| 11:02:50 | sean-k-mooney | migration is fine by the way becasue we pass data with dedicated objects for live migration | |
| 11:03:01 | sean-k-mooney | so we not passing an instance numa toplogy object form one compute to another | |
| 11:03:21 | sean-k-mooney | were are always gettign it form teh conductor | |
| 11:03:30 | sean-k-mooney | or constucting it locally | |
| 11:04:30 | sean-k-mooney | the conductor will backlevel it if the compute is older but the compute can use the latest version it knows about in all cases | |
| 11:04:32 | gibi | I see | |
| 11:05:58 | sean-k-mooney | oh i did push my comments on teh follow up https://github.com/openstack/nova/blob/e27bbe72e0d293e55c30d4f90ca0afcf47427419/nova/compute/manager.py#L955-L958 | |