| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-02-02 | |||
| 18:24:03 | gibi | I just wanted to be sure we will do it | |
| 18:24:09 | gibi | my +2 stands :) | |
| 18:24:16 | gmaan | yeah, thanks | |
| 18:52:51 | gmaan | gibi: checked the STOPPED events change, its looks nice now. I left one comment about calculating the remaining_delay while it wait for shutdown condition https://review.opendev.org/c/openstack/nova/+/974445/comment/597be334_afaa6a5f/ | |
| 19:26:09 | gibi | gmaan: thanks you have a good point. I will fix it tomorrow | |
| 19:28:02 | sean-k-mooney | gibi: would you mind adding https://review.opendev.org/c/openstack/nova/+/975171/1 and https://review.opendev.org/c/openstack/nova/+/973149 to your review list for tomorrow | |
| 19:30:38 | gibi | sean-k-mooney: done :) | |
| 19:31:24 | gibi | I mean the first one is done | |
| 19:31:53 | gibi | the second will need my fresh brain tomorrow | |
| 19:33:49 | sean-k-mooney | thanks :) | |
| 19:38:14 | dansmith | gibi: can I have you for a sec? | |
| 19:38:20 | dansmith | fresh brain probably not required | |
| 19:51:30 | gmaan | gibi: ack | |
| 19:53:17 | dansmith | nevermind I think | |
| 20:07:49 | opendevreview | Merged openstack/nova master: Add regression test to repoduce bug 2139351 https://review.opendev.org/c/openstack/nova/+/975171 | |
| #openstack-nova - 2026-02-03 | |||
| 01:26:53 | opendevreview | melanie witt proposed openstack/nova master: testing: Run functional tests under [testenv:cover] https://review.opendev.org/c/openstack/nova/+/975346 | |
| 01:26:54 | opendevreview | melanie witt proposed openstack/nova master: unified limits: Fix openstacksdk usage for endpoint discovery https://review.opendev.org/c/openstack/nova/+/975106 | |
| 01:36:40 | opendevreview | melanie witt proposed openstack/nova master: TPM: prepare to bump service version for live migration https://review.opendev.org/c/openstack/nova/+/962051 | |
| 01:36:41 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `host` secret security https://review.opendev.org/c/openstack/nova/+/941483 | |
| 01:36:41 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `deployment` secret security https://review.opendev.org/c/openstack/nova/+/925771 | |
| 01:36:42 | opendevreview | melanie witt proposed openstack/nova master: TPM: test live migration between hosts with different security https://review.opendev.org/c/openstack/nova/+/952629 | |
| 01:36:42 | opendevreview | melanie witt proposed openstack/nova master: TPM: add late check for supported TPM secret security https://review.opendev.org/c/openstack/nova/+/956975 | |
| 01:36:43 | opendevreview | melanie witt proposed openstack/nova master: TPM: opt-in to new TPM secret security via resize https://review.opendev.org/c/openstack/nova/+/962052 | |
| 01:36:43 | opendevreview | melanie witt proposed openstack/nova master: DNM vtpm tempest https://review.opendev.org/c/openstack/nova/+/957477 | |
| 03:35:09 | melwitt | I have noticed nova-next has been failing a lot on the master branch today https://zuul.opendev.org/t/openstack/builds?job_name=nova-next&project=openstack%2Fnova&branch=master&skip=0 | |
| 03:37:34 | melwitt | error like this "Failed to start libvirt guest: libvirt.libvirtError: internal error: process exited while connecting to monitor: 2026-01-30T20:30:26.821021Z qemu-system-x86_64: -device {"driver":"scsi-block","bus":"scsi0.0","channel":0,"scsi-id":0,"lun":0,"share-rw":true,"drive":"libvirt-1-format","id":"ua-409f3368-f41a-4915-aa22-29dff53bf25b","bootindex":1,"logical_block_size":512,"physical_block_size":512}: Property | |
| 03:37:34 | melwitt | 'scsi-block.physical_block_size' not found" | |
| 08:51:36 | gibi | dansmith: sorry I was away | |
| 09:05:27 | opendevreview | Merged openstack/nova stable/2024.2: Reproduce bug/2130881 https://review.opendev.org/c/openstack/nova/+/975166 | |
| 10:46:55 | gibi | sean-k-mooney: left two questions in https://review.opendev.org/c/openstack/nova/+/973149 | |
| 10:56:05 | opendevreview | Balazs Gibizer proposed openstack/nova master: Use an executor to delay STOPPED events https://review.opendev.org/c/openstack/nova/+/974445 | |
| 10:56:05 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove spawn_after https://review.opendev.org/c/openstack/nova/+/975396 | |
| 10:56:06 | opendevreview | Balazs Gibizer proposed openstack/nova master: Cleanup libvirt driver at service stop https://review.opendev.org/c/openstack/nova/+/975128 | |
| 10:56:06 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix full executor warning on noname executor https://review.opendev.org/c/openstack/nova/+/975172 | |
| 10:56:07 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-compute in native threading mode https://review.opendev.org/c/openstack/nova/+/965467 | |
| 10:56:21 | gibi | gmaan: ^^ fixed up the wrapper based on your comments | |
| 11:14:30 | sean-k-mooney | gibi: thanks ill take a look shortly | |
| 11:31:42 | sean-k-mooney | gibi: replied inline | |
| 12:01:38 | opendevreview | sean mooney proposed openstack/nova master: enable tap creation in nova-live-migration https://review.opendev.org/c/openstack/nova/+/975500 | |
| 12:09:33 | sean-k-mooney | melwitt: so nova-next is using debian 13 to work around some qemu img issues with encypted volumes so it and the nova-alt-config and ceph jobs actully using a newer qemu then say tempest-integrated-comptue | |
| 12:09:59 | sean-k-mooney | the ‘Property scsi-block.physical_block_size not found’ might be related to that | |
| 12:10:27 | sean-k-mooney | i.e. some chagne in qemu/libvirt or perhaps we are testing a combination there that we dont have enabel elsehwere | |
| 12:10:37 | sean-k-mooney | im not sure what that might be in this case | |
| 12:19:02 | opendevreview | Merged openstack/nova stable/2024.2: Init virt driver before use https://review.opendev.org/c/openstack/nova/+/975167 | |
| 13:01:41 | sean-k-mooney | melwitt: here is the relevent data https://paste.opendev.org/show/b6sWRx6VMAdktImClzCD/ ill ask about it on the virt channel and see if it rigns a bell. it looks like there is a incompatiablty between the libvirt xml we gereate and the qemu command line args | |
| 13:01:59 | sean-k-mooney | the code on teh nova side for this looks like it has not chaged in 2+ years so this looks liek a qemu regression to me | |
| 13:31:05 | gibi | sean-k-mooney: OK, all looks good to me | |
| 14:25:22 | sean-k-mooney | melwitt: so this is a latent nova bug exposed by a regression in libvirt. https://github.com/libvirt/libvirt/commit/718721f0f9d33aa8abafd71bfb9ac1cfef95dc3e assuems physical_block_size is supproted for disk type = lun but it is not | |
| 14:25:24 | sean-k-mooney | https://issues.redhat.com/browse/RHEL-145937 | |
| 14:25:47 | sean-k-mooney | tracks fixing the libvirt validation to reject it. | |
| 14:26:12 | sean-k-mooney | but what that means for nova is we need to update the code to not generate <blockio logical_block_size='512' physical_block_size='512'/> for dev type = lun | |
| 14:27:10 | sean-k-mooney | when we use lun mode we are directly passhtihg though the underlying lun form the stroage server and we cant change thing like the physical_block_size the same way we cant set the serial of the exported lun. | |
| 14:27:41 | sean-k-mooney | so we will need to make this conditoinal just as we did for serial in https://review.opendev.org/c/openstack/nova/+/918089 | |
| 14:28:00 | sean-k-mooney | melwitt: ill write this up in a bug report adn add the gate blocker flag | |
| 14:42:31 | opendevreview | Balazs Gibizer proposed openstack/nova master: DNM: Troubleshoot deadlocks in threading mode https://review.opendev.org/c/openstack/nova/+/975515 | |
| 14:50:10 | sean-k-mooney | melwitt: it looks like the xml we generate has also changed. | |
| 14:51:08 | sean-k-mooney | the last succesfull nova-next run the the xml looked like thsi https://paste.opendev.org/show/bSZXkYi0lSv491NEjhW0/ | |
| 14:51:52 | sean-k-mooney | so this could be a bug in a recent nova or os-brick change | |
| 14:54:21 | sean-k-mooney | os-brick was bumped 3 weesk ago https://github.com/openstack/requirements/commit/d2639f76ce38c8a14ff7bd00d913601963646b1d | |
| 14:54:32 | sean-k-mooney | the lass succesfuly nova-next was 2026-01-30 21:02:10 | |
| 14:54:44 | sean-k-mooney | so that implies somethign changed in nova | |
| 15:07:00 | sean-k-mooney | fun i foudn the code in nova that sets this and that has not chagne since it was intoduced 13 years ago :) the joys of latent bugs | |
| 15:07:16 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Use firmware auto-selection by libvirt https://review.opendev.org/c/openstack/nova/+/969132 | |
| 15:07:46 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/volume/volume.py#L55-L62 | |
| 15:09:01 | sean-k-mooney | ok foudn the root cause | |
| 15:09:03 | sean-k-mooney | https://github.com/openstack/cinder/commit/47107b4c84b060be834d6f430d32c3ebcac7d35a | |
| 15:10:35 | yuval_ | gibi: the iothreads merge https://review.opendev.org/c/openstack/nova/+/939254 - is it happening for any vms I create? | |
| 15:11:09 | sean-k-mooney | yuval_: that on by default yes | |
| 15:11:16 | yuval_ | I see | |
| 15:11:16 | sean-k-mooney | for all new vms | |
| 15:11:39 | sean-k-mooney | its technially on unconditionally for all vms now | |
| 15:11:42 | yuval_ | so, I want to control how many cpu's are allocated for iothreading | |
| 15:11:51 | sean-k-mooney | we are not suprpoting that this cycle | |
| 15:11:55 | yuval_ | that is not possible | |
| 15:12:02 | yuval_ | I see | |
| 15:12:05 | sean-k-mooney | but that was going to be a disucssion topic of next steps | |
| 15:12:16 | yuval_ | I see | |
| 15:12:22 | sean-k-mooney | we will likely add a flavor extra spec of similar next cycle | |
| 15:12:38 | sean-k-mooney | there isnt currently an agreement on how that shoudl look | |
| 15:12:59 | sean-k-mooney | yuval_: do you have a prefered ux/workflow for this? | |
| 15:13:51 | yuval_ | so, currently what I done to make it work, I would edit the xml and choose which cpu allocated for it. but its nightmare. | |
| 15:14:07 | yuval_ | unmanageable | |
| 15:14:24 | sean-k-mooney | and also unsupproted upsteam | |
| 15:14:29 | yuval_ | yes yes | |
| 15:14:36 | sean-k-mooney | the 3 proposal are as follows | |
| 15:14:51 | sean-k-mooney | 1 add a simple hw:io_threads=n extra specs | |
| 15:15:09 | sean-k-mooney | 2 add the ablityto add an addtionl n io thread per cidner voume | |
| 15:15:35 | sean-k-mooney | 3 use libvirt native iothread pooling feature and add hw:io_thread_max extra spec instead | |
| 15:15:43 | sean-k-mooney | 4 somethimg more complicated | |
| 15:16:19 | sean-k-mooney | in all cases we would have nova select the cores for the tread to run on automaticly for you using the cpu_shared_set cores | |
| 15:16:33 | yuval_ | so 1 - we choose which cpu's will be allocated for iothreading. 3 - there is a libvirt feature that does that? | |
| 15:16:55 | sean-k-mooney | no 1 and 3 the value is just an integer so you chosoe the count | |
| 15:16:58 | sean-k-mooney | not the cores | |
| 15:17:14 | sean-k-mooney | https://libvirt.org/formatdomain.html#iothreads-allocation | |
| 15:17:34 | sean-k-mooney | and yes libvit has a feature fo iothread pooling | |
| 15:17:36 | sean-k-mooney | <iothread id="8" thread_pool_min="2" thread_pool_max="32"> | |
| 15:17:57 | sean-k-mooney | so it can be static or dynimc and you can hotplug/unplug iothread at runtime | |
| 15:18:46 | sean-k-mooney | so we could just add 1 per cidner voluem, even on attach/detach, or it could be a static value in teh flavor or a max value in the falvor with a min of 1 | |
| 15:19:20 | sean-k-mooney | for this cycle we just hardcode 1 and we wanted operator feedback before doing anything more complex | |
| 15:23:08 | fwiesel | gibi: I changed the vmwareapi CI to set the OS_NOVA_DISABLE_EVENTLET_PATCHING environment variable when there is a change under the eventlet-removal topic. | |