| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-02-14 | |||
| 16:26:43 | sean-k-mooney | and we copy the data form the source to test | |
| 16:26:45 | sean-k-mooney | *dest | |
| 16:28:13 | ganso | sean-k-mooney: the migration of VM data is, the disk copy no. In the logs that I have I see the dest ssh'ing to the src to scp the disk | |
| 16:29:43 | dougszu | +1, I definitely see the scp initiated from the dest. Eg. for fetching config drive during live migrate. | |
| 16:30:05 | ganso | dougszu: to be honest, if the patch already merged doesn't address the issue, then it becomes even more important that the new one does, and being a non-backportable feature prevents customers from accessing the fix unless they upgrade to caracal, which for some is a long time in the future | |
| 16:30:07 | sean-k-mooney | hum really | |
| 16:30:16 | dougszu | As in the bug report at the top: https://bugs.launchpad.net/nova/+bug/1939869 | |
| 16:30:23 | dougszu | Happy to share more logs | |
| 16:30:23 | sean-k-mooney | i would have to look at that again but for config dirve specificaly | |
| 16:30:31 | sean-k-mooney | we dont actully need to copy it manually | |
| 16:30:36 | sean-k-mooney | libvirt can and will do that now | |
| 16:30:45 | sean-k-mooney | there was a bug in libvirt several year ago | |
| 16:30:54 | sean-k-mooney | that required us to do that but i tought we removed that code | |
| 16:30:57 | dougszu | It's also the glance image (if it's no longer available in glance) | |
| 16:31:32 | sean-k-mooney | so that should be done form the souce to dest i tought but perhaps its on the dest | |
| 16:31:53 | sean-k-mooney | ok lets do this | |
| 16:32:02 | sean-k-mooney | our normall request for any complex bug | |
| 16:32:21 | sean-k-mooney | is to first create a reproducer fucntional test showing the buggy behavior | |
| 16:32:54 | sean-k-mooney | so it woudl be good to try and simulate this with a fuctional test to show that | |
| 16:38:00 | ganso | live_migration_inbound_addr instead | |
| 16:38:00 | ganso | sean-k-mooney: hmmmmm if it was an older libvirt bug that may not longer be valid or that version no longer used since a given version that matches recent nova version we would backport to (for example focal-zed), so maybe instead of backporting the new config option, or even going with the new patch that bumps the object version, we could perhaps shift things around and move that from the source as you say and use the | |
| 16:38:57 | ganso | actually I don't think zed is used on focal, it is from jammy onwards, and in jammy it may be that every libvirt version available does not have that bug | |
| 16:39:12 | sean-k-mooney | so there are a few issue at play here. | |
| 16:39:22 | sean-k-mooney | first you should not be doing live migation by doing "openstack server migrate --live-migration --block-migration`" | |
| 16:39:38 | sean-k-mooney | --block-migation was depercated about 6+ years ago | |
| 16:40:28 | ganso | sean-k-mooney: oh I didn't know that, the customer is using --block-migration because that's the only way they can get the config drive moved. Is there a better way to do this? | |
| 16:42:04 | sean-k-mooney | in mitaka we added auto to do the right thing | |
| 16:42:08 | sean-k-mooney | https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-mitaka | |
| 16:43:26 | ganso | sean-k-mooney: by "auto" you mean, not specifying anything? in that case the code complains that it is not in shared storage | |
| 16:44:00 | sean-k-mooney | so if you do openstack --os-compute-api=2.25 server migrate --live <server> | |
| 16:44:09 | sean-k-mooney | then it will just do the right thing | |
| 16:44:22 | sean-k-mooney | and we will leave libvirt copy the config drive i belive | |
| 16:45:04 | sean-k-mooney | ganso: there was a specific bug with libvirt and live migrating with iso files attached as a cdrom | |
| 16:45:11 | sean-k-mooney | that requried us to copy it first | |
| 16:45:21 | sean-k-mooney | but that was in like ubuntu 16.04 | |
| 16:45:54 | sean-k-mooney | resolved in libvirt v1.2.17. | |
| 16:46:30 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1246201 | |
| 16:46:53 | dougszu | It's this code block running on dest (contains both config drive and glance image copy): https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10892 | |
| 16:47:21 | dougszu | I'm not seeing where it's gated on --block-migration, but could have missed something | |
| 16:47:29 | sean-k-mooney | right that was added by https://review.opendev.org/c/openstack/nova/+/365420/1/nova/virt/libvirt/driver.py | |
| 16:47:51 | sean-k-mooney | that code is not needed anymore and has not been for litrally years | |
| 16:48:27 | dougszu | For config drive yeah, but not the Glance image? | |
| 16:48:31 | sean-k-mooney | ya so that code https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10899-L10911 | |
| 16:48:34 | sean-k-mooney | can be removed | |
| 16:48:53 | sean-k-mooney | for the backing file | |
| 16:49:10 | sean-k-mooney | we cant remvoe it but its not a bug that that does nto use the live | |
| 16:49:24 | sean-k-mooney | *live_migrateion_inboud_adr | |
| 16:50:16 | sean-k-mooney | im not sure if that will now work with the new migration migration_inbound_addr | |
| 16:50:38 | sean-k-mooney | i belive it should but it it does not it would be a bug in that new feature | |
| 16:51:42 | ganso | sean-k-mooney: ok I will ask my customer to re-test that with the --os-compute-api=2.25 --live and without --block-migration and see if that solves the issue | |
| 16:52:00 | sean-k-mooney | ack | |
| 16:52:02 | dougszu | Thanks, I can do some more testing. Good to know the config drive code can be removed. | |
| 16:52:03 | ganso | sean-k-mooney: btw I thought --live was deprecated in favor of --live-migration, --live was the one that specified the dest and --live-migration leaves it to the scheduler | |
| 16:52:22 | sean-k-mooney | oh right | |
| 16:52:42 | sean-k-mooney | ya in either case the impoatnt point it to use a microversion >= 2.25 | |
| 16:52:48 | sean-k-mooney | to get the auto behvior | |
| 16:55:08 | ganso | sean-k-mooney: thanks for the explanation! :) | |
| 16:56:07 | sean-k-mooney | a straignt revert of https://review.opendev.org/c/openstack/nova/+/365420 wont work because of merge conflcits | |
| 16:56:32 | opendevreview | sean mooney proposed openstack/nova stable/mitaka: Revert "[libvirt] Live migration fails when config_drive_format=iso9660" https://review.opendev.org/c/openstack/nova/+/908937 | |
| 16:56:46 | sean-k-mooney | actully never mind apprently it will work | |
| 16:57:09 | opendevreview | sean mooney proposed openstack/nova stable/mitaka: Revert "[libvirt] Live migration fails when config_drive_format=iso9660" https://review.opendev.org/c/openstack/nova/+/908937 | |
| 16:58:51 | sean-k-mooney | actully the striagt revert is wrong for other reason but ill see if i can remove that code properly oh also its agaisnt the wrong branch | |
| 16:58:59 | sean-k-mooney | so ignore ^ | |
| 16:59:52 | sean-k-mooney | dougszu: ganso what i was going to say is if either of ye feel like reverting the config drive bit on master i can review | |
| 17:00:08 | sean-k-mooney | if not ill add this to my todo list and se if we can remove that | |
| 17:00:34 | dougszu | I'll try and have a go tomorrow | |
| 18:13:40 | ganso | sean-k-mooney: I rather have my customer test first to be sure there will be a code path that addresses their config drive migration. At the moment I am not sure as I am not familiar with that part of the code. It should go "auto" as you say but each case could be different | |
| #openstack-nova - 2024-02-15 | |||
| 05:43:19 | opendevreview | Amit Uniyal proposed openstack/nova master: Added context manager for instance lock https://review.opendev.org/c/openstack/nova/+/873648 | |
| 05:43:20 | opendevreview | Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446 | |
| 05:43:20 | opendevreview | Amit Uniyal proposed openstack/nova master: Separate OSError with ValueError https://review.opendev.org/c/openstack/nova/+/908825 | |
| 05:43:21 | opendevreview | Amit Uniyal proposed openstack/nova master: Removed explicit call to delete attachment https://review.opendev.org/c/openstack/nova/+/891289 | |
| 08:47:59 | opendevreview | Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446 | |
| 08:47:59 | opendevreview | Amit Uniyal proposed openstack/nova master: Separate OSError with ValueError https://review.opendev.org/c/openstack/nova/+/908825 | |
| 08:48:00 | opendevreview | Amit Uniyal proposed openstack/nova master: Removed explicit call to delete attachment https://review.opendev.org/c/openstack/nova/+/891289 | |
| 08:49:49 | tkajinam | sean-k-mooney, hi. thanks for your feedback in libvirt related patches ! I've replied to your comment in https://review.opendev.org/c/openstack/nova/+/908546/ so please check it when you have time | |
| 08:50:21 | tkajinam | in short, swtpm_ioctl has been required since swtpm support was initially added to libvirt. | |
| 10:22:43 | opendevreview | Merged openstack/nova master: libvirt: Stop unconditionally enabling evmcs https://review.opendev.org/c/openstack/nova/+/899776 | |
| 11:12:12 | opendevreview | sean mooney proposed openstack/nova stable/2023.2: libvirt: Stop unconditionally enabling evmcs https://review.opendev.org/c/openstack/nova/+/909082 | |
| 11:13:31 | sean-k-mooney[m] | elodilles: gibi ^ can we start backporting that now that it finally merged on master. this need to go to 2023.1 to fix windows on amd | |
| 11:13:53 | sean-k-mooney[m] | there is one other releated patch for live migration but i belive that is still not merged on master | |
| 11:14:30 | sean-k-mooney[m] | it can be backported seperatly | |
| 11:16:59 | gibi | sean-k-mooney[m]: thanks for the backport, +2 | |
| 11:20:11 | artom | sean-k-mooney[m], oh you're doing the enlightenments backports? | |
| 11:52:57 | sean-k-mooney | artom: well it merged os i just did it via the ui | |
| 11:53:07 | sean-k-mooney | ill let you actully do the other one or respin it | |
| 11:54:10 | sean-k-mooney | artom: do you have the linke to the live migration patch by the way | |
| 11:54:30 | artom | sean-k-mooney, https://review.opendev.org/c/openstack/nova/+/904183?usp=email | |
| 11:54:49 | artom | sean-k-mooney, so I'm doing from bobcat to antelope, and the downstream cherry-picl? | |
| 11:55:00 | sean-k-mooney | artom: yes please | |
| 11:55:26 | sean-k-mooney | and can you backport the other one too | |
| 11:57:13 | opendevreview | Takashi Kajinami proposed openstack/os-traits master: Add traits for TPM models https://review.opendev.org/c/openstack/os-traits/+/909107 | |
| 11:57:46 | artom | sean-k-mooney, it didn't merge yet | |
| 11:57:59 | artom | But maybe we don't care, because it has +A | |
| 12:00:00 | sean-k-mooney | yes i know. we do care for the cherry picked form line i just ment when its ready can you do it and ping me | |
| 12:00:19 | sean-k-mooney | if you want to prepare them local youc an but hopefuly it will merge later today | |
| 12:45:30 | elodilles | sean-k-mooney[m]: i'm not super confident about that backport, so left a question in my comment there | |
| 12:46:03 | sean-k-mooney | elodilles: ack, right now you cant boot windows vms on amd host becuase of a reguression intoduced by enableing that feature | |
| 12:46:16 | sean-k-mooney | so we really need to undo that | |
| 12:46:30 | sean-k-mooney | live migration is also broken for a simialr reason | |
| 12:46:56 | elodilles | sean-k-mooney: ACK i understand that, but what about with intel hosts? we might remove a feature that is already used by some operators? | |