| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-12-15 | |||
| 13:44:06 | stephenfin | Tes. It's listed on the Relation chain pane on the right | |
| 13:44:09 | stephenfin | *Yes | |
| 14:06:36 | zigo | Ah, right ... | |
| 16:34:33 | opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Proposes mdev live-migration support in libvirt https://review.opendev.org/c/openstack/nova-specs/+/900636 | |
| 16:43:03 | kashyap | bauzas: Hi, unrelated: I think this can be a specless-bp, agree? - https://blueprints.launchpad.net/nova/+spec/allow-disabling-ephemeral-disk-formatting | |
| 16:46:22 | kashyap | It came out of a discussion w/ Sean. No need to respond right away; can talk on Mon. | |
| 16:49:34 | bauzas | kashyap: you left but lemme log this : we can discuss this on a next nova meeting, like Tuesday | |
| #openstack-nova - 2023-12-16 | |||
| 03:08:57 | opendevreview | melanie witt proposed openstack/nova master: WIP Support rescue with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/873675 | |
| 03:08:58 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for qcow2 with LUKS https://review.opendev.org/c/openstack/nova/+/772273 | |
| 03:08:58 | opendevreview | melanie witt proposed openstack/nova master: WIP libvirt: Introduce support for raw with LUKS https://review.opendev.org/c/openstack/nova/+/884313 | |
| 03:08:59 | opendevreview | melanie witt proposed openstack/nova master: WIP libvirt: Introduce support for rbd with LUKS https://review.opendev.org/c/openstack/nova/+/889912 | |
| 03:23:18 | opendevreview | melanie witt proposed openstack/nova master: DNM test ephemeral encryption + resize: qcow2, raw, rbd https://review.opendev.org/c/openstack/nova/+/862416 | |
| 03:41:58 | opendevreview | melanie witt proposed openstack/nova master: DNM test ephemeral encryption + resize: qcow2, raw, rbd https://review.opendev.org/c/openstack/nova/+/862416 | |
| 13:08:43 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: [spec] Add Cross-AZ scheduling blueprint https://review.opendev.org/c/openstack/nova-specs/+/900296 | |
| 17:54:53 | opendevreview | Artom Lifshitz proposed openstack/nova master: Call Neutron immediately upon _post_live_migration() start https://review.opendev.org/c/openstack/nova/+/883682 | |
| 17:54:54 | opendevreview | Artom Lifshitz proposed openstack/nova master: Parallelize calls to Neutron and Cinder in post_live_migration https://review.opendev.org/c/openstack/nova/+/883678 | |
| 17:54:54 | opendevreview | Artom Lifshitz proposed openstack/nova master: Allow best effort sending of notifications https://review.opendev.org/c/openstack/nova/+/903807 | |
| #openstack-nova - 2023-12-17 | |||
| 02:05:08 | opendevreview | Artom Lifshitz proposed openstack/nova master: Call Neutron immediately upon _post_live_migration() start https://review.opendev.org/c/openstack/nova/+/883682 | |
| 02:05:09 | opendevreview | Artom Lifshitz proposed openstack/nova master: Allow best effort sending of notifications https://review.opendev.org/c/openstack/nova/+/903807 | |
| 02:05:09 | opendevreview | Artom Lifshitz proposed openstack/nova master: Parallelize calls to Neutron and Cinder in post_live_migration https://review.opendev.org/c/openstack/nova/+/883678 | |
| 13:37:23 | opendevreview | Artom Lifshitz proposed openstack/nova master: Allow best effort sending of notifications https://review.opendev.org/c/openstack/nova/+/903807 | |
| 13:37:23 | opendevreview | Artom Lifshitz proposed openstack/nova master: Parallelize calls to Neutron and Cinder in post_live_migration https://review.opendev.org/c/openstack/nova/+/883678 | |
| #openstack-nova - 2023-12-18 | |||
| 08:37:33 | opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Proposes mdev live-migration support in libvirt https://review.opendev.org/c/openstack/nova-specs/+/900636 | |
| 12:38:57 | opendevreview | sean mooney proposed openstack/nova master: add initial healthcheck support https://review.opendev.org/c/openstack/nova/+/825015 | |
| 13:24:55 | opendevreview | sean mooney proposed openstack/nova master: add initial healthcheck support https://review.opendev.org/c/openstack/nova/+/825015 | |
| 13:24:55 | opendevreview | sean mooney proposed openstack/nova master: add healthcheck manager to manager base https://review.opendev.org/c/openstack/nova/+/827844 | |
| 13:24:56 | opendevreview | sean mooney proposed openstack/nova master: add healthcheck tracker to nova context https://review.opendev.org/c/openstack/nova/+/829468 | |
| 13:39:50 | opendevreview | sean mooney proposed openstack/nova master: add healthcheck utils and constants https://review.opendev.org/c/openstack/nova/+/829469 | |
| 13:39:51 | opendevreview | sean mooney proposed openstack/nova master: add healthcheck endpoint to proxy commands https://review.opendev.org/c/openstack/nova/+/830703 | |
| 14:02:44 | bauzas | sean-k-mooney: gibi: fwiw, I'm about to start writing the series for the mdev live-migration RFE | |
| 14:04:55 | gibi | bauzas: good luck! | |
| 14:11:17 | opendevreview | Artom Lifshitz proposed openstack/nova master: Allow live migrate paused instance when post copy is enabled https://review.opendev.org/c/openstack/nova/+/444517 | |
| 14:38:35 | artom | melwitt, sean-k-mooney, gibi, btw, I updated https://review.opendev.org/c/openstack/nova/+/883682/ to hopefully reflect what was agreed on in the review comments | |
| 14:38:48 | artom | So now there's the notifications best effort patch on top: https://review.opendev.org/c/openstack/nova/+/903807/3 | |
| 15:07:29 | gibi | artom: thanks. Both looks good to me | |
| 15:08:28 | sean-k-mooney | artom: ack ill look at it shortly | |
| 15:09:30 | artom | Thank _you_! | |
| 16:29:02 | gibi | sean-k-mooney: if you have a moment later, then I verified https://review.opendev.org/c/openstack/nova/+/444517 and it solves the live-migration of paused instance when post-copy is configured in nova.conf. | |
| 16:29:13 | gibi | artom: ^^ \o/ | |
| 16:33:14 | kashyap | gibi: Also the "double live migration of a paused instance" thing - did you see that bz too? | |
| 16:33:30 | kashyap | Although, why would one want to run a double LM, though | |
| 16:34:11 | artom | gibi, ~~o~~ | |
| 16:34:19 | kashyap | "If a VM is in paused, and it live-migrated twice, it is lost" | |
| 16:34:21 | kashyap | https://bugs.launchpad.net/nova/+bug/1947725 | |
| 16:34:31 | kashyap | "Lost"? | |
| 16:35:04 | artom | Here instance instance, here boy! | |
| 16:46:47 | gibi | kashyap: I can try | |
| 16:55:33 | gibi | kashyap: nope, that but is still reproducible https://paste.opendev.org/show/bswQVIMJddVkrUtC1Pk0/ | |
| 16:56:55 | gibi | kashyap: but it make sens as the related qemu bug is also open https://gitlab.com/qemu-project/qemu/-/issues/686 | |
| 16:58:49 | kashyap | gibi: Ohh, I missed the attached QEMU bug | |
| 17:06:17 | artom | sean-k-mooney, so, tracing our snapshot upload call, we actually call it with a file handler for image_file/data | |
| 17:07:00 | artom | https://opendev.org/openstack/nova/src/branch/master/nova/virt/libvirt/driver.py#L3182 | |
| 17:07:09 | artom | https://opendev.org/openstack/nova/src/branch/master/nova/virt/libvirt/driver.py#L3176 | |
| 17:09:00 | artom | The thing is, on the sender side (Nova) passing a file handler to Python requests results in a streaming upload: https://docs.python-requests.org/en/latest/user/advanced.html#streaming-uploads | |
| 17:09:33 | artom | For chunks we have to use a generator or iterator: https://docs.python-requests.org/en/latest/user/advanced.html#chunk-encoded-requests | |
| 17:10:44 | sean-k-mooney | ack so we likely need a [glance]uplod_type=streaming|chunked config option | |
| 17:11:11 | sean-k-mooney | default to streamign since that is ths current behavior and allow opting into chunked | |
| 17:11:19 | sean-k-mooney | and we can consider changing the default in the future | |
| 17:20:36 | sean-k-mooney | melwitt: bauzas gibi can ye look at https://review.opendev.org/c/openstack/nova/+/903530/2 and the patch below to unblock mypy in teh requiremetns job for stephenfin | |
| 17:22:00 | sean-k-mooney | seperatly this https://review.opendev.org/c/openstack/nova/+/897218/2 and the 3 patches below it will finish the codespell and sphinx-lint series | |
| 17:22:35 | sean-k-mooney | if we can get the mypy codespell and sphinx-lint serise merged this week it would be great | |
| 17:27:06 | sean-k-mooney | gibi: how did you test https://review.opendev.org/c/openstack/nova/+/444517 in the openstack-k8s-operators CI | |
| 17:27:11 | sean-k-mooney | did you submit a pr? | |
| 17:27:28 | sean-k-mooney | i know they were working on makign the conent provide supprot buildign with upstream patches | |
| 17:27:34 | sean-k-mooney | btu i didnt think that was done yet | |
| 17:28:21 | melwitt | sean-k-mooney: sure, I can look (if others don't get to it first) | |
| 17:29:01 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: attempting glance chunked uploads https://review.opendev.org/c/openstack/nova/+/903611 | |
| 17:42:13 | gibi | sean-k-mooney: sorry I was missleading I applied the patch locally | |
| 17:42:27 | sean-k-mooney | oh ok | |
| 17:42:40 | sean-k-mooney | that is something we will be able to do eventually | |
| 17:42:43 | sean-k-mooney | with a depends on | |
| 17:42:55 | sean-k-mooney | but not for a few months yet | |
| 17:43:40 | sean-k-mooney | its on the todolist and somethign ill be supprotign with rhex-ci too eventually | |
| 18:14:54 | melwitt | sean-k-mooney: hrm.. I notice that artom's patch is going to fail CI for a guest kernel panic 🫤 and it looks to be using the split image | |
| 18:14:54 | melwitt | sean-k-mooney: hrm.. I notice that artom's patch is going to fail CI for a guest kernel panic 🫤 and it looks to be using the split image | |
| 18:15:26 | artom | :( | |
| 18:17:41 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: attempting glance chunked uploads https://review.opendev.org/c/openstack/nova/+/903611 | |
| 18:18:22 | sean-k-mooney | melwitt: i guess we can reviwe the logs when it finishes | |
| 18:18:35 | melwitt | and it seems like it's often (always?) this volume backed server resize test(s). so I wonder if there's something about it that's different than everything else. I'll see if I can find anything | |
| 18:18:41 | sean-k-mooney | is that in nova-next where we are not using split image or one of the other jobs | |
| 18:19:02 | melwitt | no it's in tempest-integrated-compute-rbac-old-defaultsf | |
| 18:19:08 | sean-k-mooney | well the volume test obviouly has some pci hotplug events | |
| 18:19:36 | sean-k-mooney | so maybe we can mitigate that by adding hw_disk_bus=scisi | |
| 18:19:40 | melwitt | logs are at https://zuul.opendev.org/t/openstack/build/fcc86981ee0e488f9f8b05a5969bfacb | |
| 18:19:53 | melwitt | hm ok | |
| 18:20:10 | sean-k-mooney | i.e. chagne from virtio-blk to virtio-scisi as that will not be a pci hotplu and instead be a scsi attach/detach | |
| 18:20:37 | melwitt | gotcha | |
| 18:21:34 | sean-k-mooney | the sttack trace looks like its in the page fault handeler | |
| 18:21:45 | sean-k-mooney | so this looks like the same failure we saw before | |
| 18:22:02 | sean-k-mooney | i wonder if this is bfv | |
| 18:22:11 | sean-k-mooney | and if we replace the image in that case | |
| 18:23:14 | melwitt | it is bfv | |
| 18:23:58 | sean-k-mooney | ok so we shoudl double check that tempest used the correct cirros image in that case | |
| 18:25:18 | sean-k-mooney | its calling into the share create_server https://github.com/openstack/tempest/blob/ab3686d28d2728001e3bd2fd543575087bf00137/tempest/api/compute/servers/test_server_actions.py#L491 | |
| 18:25:56 | sean-k-mooney | which end up using https://github.com/openstack/tempest/blob/ab3686d28d2728001e3bd2fd543575087bf00137/tempest/common/compute.py#L204 | |
| 18:26:01 | sean-k-mooney | image_id = CONF.compute.image_ref | |
| 18:26:20 | sean-k-mooney | melwitt: have we confirued that to be the split image? | |
| 18:26:50 | sean-k-mooney | its https://zuul.opendev.org/t/openstack/build/fcc86981ee0e488f9f8b05a5969bfacb/log/controller/logs/tempest_conf.txt#24 | |
| 18:27:00 | sean-k-mooney | 86a0308f-80a0-4582-8a64-f2b54e487232 | |
| 18:27:17 | sean-k-mooney | i guess we coudl check the devstack log and conrim what image that is | |