| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-02-21 | |||
| 20:53:44 | sean-k-mooney | it should be powering up the emulator thread core | |
| 20:53:59 | sean-k-mooney | do you see this message https://github.com/openstack/nova/blob/3209f6551652cff7bef0b9d9719ab940dd05a0f8/nova/virt/hardware.py#L776-L777 | |
| 20:54:55 | artom | It just checks cpu_pinning... | |
| 20:54:58 | sean-k-mooney | yep | |
| 20:55:00 | sean-k-mooney | Feb 21 19:15:30.166825 np0036828693 nova-compute[42254]: INFO nova.virt.hardware [None req-ec45061f-e9a4-4b02-9354-0cb390bd28cf tempest-EmulatorThreadTest-1184416592 tempest-EmulatorThreadTest-1184416592-project-member] Computed NUMA topology reserved pCPUs: usable pCPUs: [[], [], [4]], reserved pCPUs: {4} | |
| 20:55:13 | sean-k-mooney | but notice anythying odd | |
| 20:55:36 | sean-k-mooney | we have these two logs back to back | |
| 20:55:39 | sean-k-mooney | Feb 21 19:15:30.166472 np0036828693 nova-compute[42254]: INFO nova.virt.hardware [None req-ec45061f-e9a4-4b02-9354-0cb390bd28cf tempest-EmulatorThreadTest-1184416592 tempest-EmulatorThreadTest-1184416592-project-member] Computed NUMA topology CPU pinning: usable pCPUs: [[6], [5], [4]], vCPUs mapping: [(0, 6), (1, 5)] | |
| 20:55:40 | sean-k-mooney | Feb 21 19:15:30.166825 np0036828693 nova-compute[42254]: INFO nova.virt.hardware [None req-ec45061f-e9a4-4b02-9354-0cb390bd28cf tempest-EmulatorThreadTest-1184416592 tempest-EmulatorThreadTest-1184416592-project-member] Computed NUMA topology reserved pCPUs: usable pCPUs: [[], [], [4]], reserved pCPUs: {4} | |
| 20:56:29 | sean-k-mooney | it calulated both | |
| 20:56:34 | sean-k-mooney | and we shoudl be merging them | |
| 20:57:22 | sean-k-mooney | but then it prints | |
| 20:57:24 | sean-k-mooney | Selected cores for pinning: [(0, 6), (1, 5)], in cell 0 {{(pid=42254) _pack_instance_onto_cores /opt/stack/nova/nova/virt/hardware.py:905}} | |
| 20:58:29 | sean-k-mooney | which is form here https://github.com/openstack/nova/blob/3209f6551652cff7bef0b9d9719ab940dd05a0f8/nova/virt/hardware.py#L905-L906 | |
| 20:59:44 | sean-k-mooney | artom: tldr adding cpuset_reserved to power on and power off would fix it | |
| 20:59:56 | artom | Yeah, lemme try a quick hax of that | |
| 21:00:16 | sean-k-mooney | but but i thin kbased on the code we ere expiectin that not to be required | |
| 21:00:50 | sean-k-mooney | so maybe we need to update the doc string or the propery to include cpuset_reserved | |
| 21:02:00 | sean-k-mooney | i.e. if you made https://github.com/openstack/nova/blob/3209f6551652cff7bef0b9d9719ab940dd05a0f8/nova/objects/instance_numa.py#L291-L296 also include it | |
| 21:02:26 | sean-k-mooney | ok im going to finish there let me know how your hacking goes | |
| 21:02:45 | artom | It's done, just need to push and retry whitebox | |
| 21:03:34 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: online dedicated emulator threads as well https://review.opendev.org/c/openstack/nova/+/909795 | |
| 21:03:57 | artom | Hrmm, I can't Depends-on different Nova changes, can I? | |
| 21:04:49 | sean-k-mooney | you can | |
| 21:05:00 | sean-k-mooney | you can depends on 2 diffent patches in the same repo | |
| 21:05:07 | sean-k-mooney | you normally dont want too | |
| 21:05:09 | artom | Really? And Zuul does like a magic merge of them? | |
| 21:05:14 | sean-k-mooney | yep | |
| 21:05:32 | sean-k-mooney | so as long as they dont have merge conflcit between each other it will work | |
| 21:05:59 | artom | Dayum, I'm impressed | |
| 21:06:06 | artom | OK, https://review.opendev.org/c/openstack/nova/+/909795 can run while I pick up the kids from school | |
| 21:06:15 | sean-k-mooney | we normally done do tha because its way more fragile but it works i limited caes | |
| 21:07:41 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: online dedicated emulator threads as well https://review.opendev.org/c/openstack/nova/+/909795 | |
| 21:08:53 | sean-k-mooney | artom: cell is spelt with a c | |
| 21:08:55 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/909795/1/nova/objects/instance_numa.py#301 | |
| 21:09:26 | artom | Depends what you're celling | |
| 21:09:41 | sean-k-mooney | :) | |
| 21:09:47 | opendevreview | Pavlo Shchelokovskyy proposed openstack/nova master: Fix device type when booting from ISO image https://review.opendev.org/c/openstack/nova/+/909611 | |
| 23:08:13 | opendevreview | Pavlo Shchelokovskyy proposed openstack/nova master: Fix device type when booting from ISO image https://review.opendev.org/c/openstack/nova/+/909611 | |
| #openstack-nova - 2024-02-22 | |||
| 01:02:03 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: power up dedicated cores during pre_live_migration https://review.opendev.org/c/openstack/nova/+/909806 | |
| 01:22:34 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: power up dedicated cores during pre_live_migration https://review.opendev.org/c/openstack/nova/+/909806 | |
| 01:24:17 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: power up dedicated cores during pre_live_migration https://review.opendev.org/c/openstack/nova/+/909806 | |
| 05:18:05 | opendevreview | melanie witt proposed openstack/nova master: Support create with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870932 | |
| 05:18:05 | opendevreview | melanie witt proposed openstack/nova master: Support (resize|cold migration) with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870933 | |
| 05:18:06 | opendevreview | melanie witt proposed openstack/nova master: Support live migration with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/905512 | |
| 05:18:06 | opendevreview | melanie witt proposed openstack/nova master: Support rebuild with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870939 | |
| 05:18:07 | opendevreview | melanie witt proposed openstack/nova master: Support rescue with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/873675 | |
| 05:18:07 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to qemu-img rebase https://review.opendev.org/c/openstack/nova/+/870936 | |
| 05:18:08 | opendevreview | melanie witt proposed openstack/nova master: Support snapshot with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870937 | |
| 05:18:08 | opendevreview | melanie witt proposed openstack/nova master: Add backing_encryption_secret_uuid to BlockDeviceMapping https://review.opendev.org/c/openstack/nova/+/907960 | |
| 05:18:09 | opendevreview | melanie witt proposed openstack/nova master: Support encrypted backing files for qcow2 https://review.opendev.org/c/openstack/nova/+/907961 | |
| 05:18:10 | opendevreview | melanie witt proposed openstack/nova master: Support cross cell resize with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/909595 | |
| 05:18:10 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for raw with LUKS https://review.opendev.org/c/openstack/nova/+/884313 | |
| 05:18:12 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS https://review.opendev.org/c/openstack/nova/+/889912 | |
| 07:11:22 | opendevreview | Amit Uniyal proposed openstack/nova master: enforce remote console shutdown https://review.opendev.org/c/openstack/nova/+/901824 | |
| 08:30:40 | gibi | artom: sean-k-mooney: nice catch on power mgmt. should I revert https://github.com/openstack-k8s-operators/nova-operator/pull/695 ? or we can live with the limitation? | |
| 08:36:53 | opendevreview | melanie witt proposed openstack/nova master: Add backing_encryption_secret_uuid to BlockDeviceMapping https://review.opendev.org/c/openstack/nova/+/907960 | |
| 08:36:54 | opendevreview | melanie witt proposed openstack/nova master: Support encrypted backing files for qcow2 https://review.opendev.org/c/openstack/nova/+/907961 | |
| 08:36:54 | opendevreview | melanie witt proposed openstack/nova master: Support cross cell resize with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/909595 | |
| 08:36:55 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for raw with LUKS https://review.opendev.org/c/openstack/nova/+/884313 | |
| 08:36:55 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS https://review.opendev.org/c/openstack/nova/+/889912 | |
| 09:17:07 | noonedeadpunk | hey folks. I guess I finally realized what is the usecase of root volume detach that was bothering me and which raised discussions here and there. And that is - volume resize. As in order to resize the disk, it must be in detached state. And you can't detach root volume. | |
| 09:38:44 | tkajinam | bauzas, hi. I just noticed I made a mistake in os-vif patch while discussion in another project and submitted a fix for it. I wonder we can merge this before we create os-vif release for caracal... https://review.opendev.org/c/openstack/os-vif/+/909682 | |
| 09:39:04 | bauzas | tkajinam: I'm pretty busy those days, but I can take a look | |
| 09:41:23 | tkajinam | bauzas, thanks and sorry for bothering you. I've left a same comment in a release patch for os-vif... | |
| 09:57:18 | bauzas | tkajinam: that's cool, I'll review it today | |
| 10:05:04 | sean-k-mooney[m] | gibi: i think we can fix the edge cases and backport but we might want to keep a revirt ready incase we need it. my inclinations is it would be ok to have as a known issue in the context of a beta release but if we cant backport the bug fixes in time we might revert for ga | |
| 10:08:00 | sean-k-mooney[m] | tkajinam: i assume you would also like https://review.opendev.org/c/openstack/os-vif/+/909341 | |
| 10:09:33 | sean-k-mooney[m] | bauzas: im +2 on both the os vif changes so let include them in the os-vif release | |
| 10:12:02 | sean-k-mooney[m] | @gibi the isolate issue in particalar is a low probability of impacting people, the general live migration issue however is much more impactful. so we should try and land rene’s fix for that. | |
| 10:17:17 | sean-k-mooney[m] | gibi: looking at the second poc patch it looks like its also affecting live migration with pinned cpus | |
| 10:17:50 | sean-k-mooney[m] | that is very surprising since i tought bauzas had tested that and it was covered | |
| 10:18:39 | bauzas | I'm on a meeting | |
| 10:19:05 | bauzas | and I miss context, what are you guys discussing ? | |
| 10:34:22 | opendevreview | Doug Szumski proposed openstack/nova master: Revert "[libvirt] Live migration fails when config_drive_format=iso9660" https://review.opendev.org/c/openstack/nova/+/909122 | |
| 10:34:37 | gibi | sean-k-mooney[m]: OK i will open a revert with a hold | |
| 10:36:50 | gibi | bauzas: yet another bug in the power mgmt feature | |
| 10:37:24 | bauzas | which is ? | |
| 10:38:06 | opendevreview | Doug Szumski proposed openstack/nova master: Revert "[libvirt] Live migration fails when config_drive_format=iso9660" https://review.opendev.org/c/openstack/nova/+/909122 | |
| 10:39:25 | sean-k-mooney[m] | bauzas: there are two, livemigation is broken and emultor thread policy=isolate | |
| 10:39:52 | bauzas | ack, any bug report to provide me ? | |
| 10:39:58 | gibi | sean-k-mooney[m]: the liv migration one is not specific to power mgmt isnt it? | |
| 10:40:15 | bauzas | sean-k-mooney: fwiw, I haven't tested live-migration | |
| 10:40:29 | sean-k-mooney[m] | i tought you had | |
| 10:40:37 | sean-k-mooney[m] | but that explains why it was missed | |
| 10:41:10 | sean-k-mooney[m] | there is a spereate live migration issue that is unrelated to power management | |
| 10:41:26 | sean-k-mooney[m] | but there is also a power management live migration bug | |
| 10:41:37 | sean-k-mooney[m] | we are not turning on the cores as part of pre live migrate | |
| 10:41:55 | sean-k-mooney[m] | so thats just actully broken | |
| 10:42:22 | sean-k-mooney[m] | the related issue is when cpu_shared_set is used and we dont have a numa topology | |
| 10:42:31 | sean-k-mooney[m] | we dont update the cores of the vm for the desination | |
| 10:42:39 | sean-k-mooney[m] | we have code up for review to fix that | |
| 10:43:03 | sean-k-mooney[m] | that means when combined with mixed shared and dedicated cpus on the same host | |
| 10:43:20 | sean-k-mooney[m] | its possible ot migrate a floating instnace to a core that is offline | |
| 10:43:43 | sean-k-mooney[m] | if the cpu_shared_set and cpu_dedicated_set are not the same on all hosts | |
| 10:44:04 | sean-k-mooney[m] | or rather the source and dest host | |
| 10:44:57 | sean-k-mooney[m] | gibi: the more i think about it the more im leaning towards truning it off by default again and trying to renable it after beta instead | |
| 10:45:00 | bauzas | I think I checked that we were turning the cores on on the right internal method that's called for *any* guest start | |
| 10:45:16 | bauzas | but maybe live-migration uses another path | |
| 10:45:28 | sean-k-mooney[m] | bauzas: the vm on the dest is started by libvirt not nova | |