| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-10-18 | |||
| 16:33:47 | sean-k-mooney | adn your using images_type=qcow | |
| 16:33:52 | sean-k-mooney | or raw or flat | |
| 16:34:14 | colby__ | we did not set it so its currently default | |
| 16:34:32 | sean-k-mooney | ok that woudl be defcault which for libvirt is qcow | |
| 16:35:09 | sean-k-mooney | imporanly in that config then the expect bevioru is to report local_disk_gb as the free space on the compute node partion where /var/lib/nova is | |
| 16:35:20 | sean-k-mooney | and that was the ame behvior in older release | |
| 16:35:33 | colby__ | ok so set images_type=rbd then? | |
| 16:35:37 | sean-k-mooney | no | |
| 16:35:51 | sean-k-mooney | that means use cpeh for root disk | |
| 16:35:56 | sean-k-mooney | even with out ceinder | |
| 16:36:14 | colby__ | ah sorry I misunderstood. so raw then? | |
| 16:36:21 | sean-k-mooney | no dont change it | |
| 16:36:35 | sean-k-mooney | nova is working as it should form a compute config point of view | |
| 16:36:47 | colby__ | ok | |
| 16:37:01 | sean-k-mooney | i need to check somehting but i alo need to join a 1:1 with my manager | |
| 16:37:12 | colby__ | ok no problem. Thanks for the help | |
| 16:37:42 | sean-k-mooney | did you used to set allcoation raitio by aggreates prehaps? | |
| 16:39:56 | dansmith | ugh, feels like we're back to gate suckage.. some trivial patches are in recheck hell | |
| 16:41:03 | colby__ | sean-k-mooney: we never set anything for disk allocation | |
| 17:03:28 | colby__ | sean-k-mooney: I was able to increase the disk allocation ratio but now seeing this: nova.exception.InvalidSharedStorage: <host> is not on shared storage: Shared storage live-migration requires either shared storage or boot-from-volume with no local disks. This is a boot from volume ceph instance | |
| 17:04:04 | colby__ | is this all due to the disk_gb of the flavors? | |
| 17:08:19 | colby__ | Im migrating from a xena hypervisor to a yoga and seeing this issue | |
| 17:21:38 | sean-k-mooney | colby__: so partly. but you shoudl have had the same behavior on xena and yoga i think | |
| 17:22:01 | sean-k-mooney | colby__: so you should not be passing --shared-srotage wehn live migrating | |
| 17:22:32 | colby__ | I was just using the GUI for the live migration but can switch to the cli | |
| 17:22:36 | sean-k-mooney | you should be using a newer microvstion where that defautl to auto | |
| 17:23:10 | sean-k-mooney | so for boot form volume guest you should not tick shared storage | |
| 17:23:43 | sean-k-mooney | shared storage for that option really means is /var/lib/nova on nfs | |
| 17:23:58 | sean-k-mooney | so are you using images_type=rbd | |
| 17:24:17 | colby__ | no | |
| 17:24:23 | colby__ | just set to default | |
| 17:24:30 | sean-k-mooney | i missed typed | |
| 17:25:11 | sean-k-mooney | what i ment is you only say "shared_stroage=true" if you the vm is booted on a shared file cycle without using cinder volumes | |
| 17:25:21 | sean-k-mooney | so that is not your case | |
| 17:25:35 | colby__ | also there is no check box for shared storage just "block migration" and "disk overcommit" on gui live migration | |
| 17:25:56 | sean-k-mooney | right so your using horizon yes? | |
| 17:26:22 | sean-k-mooney | it likely would be better to do live migration form the openstack client | |
| 17:26:32 | colby__ | yep will try that | |
| 17:28:38 | sean-k-mooney | so in mitaka 2.25 we added auto | |
| 17:28:40 | sean-k-mooney | https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-mitaka | |
| 17:32:14 | melwitt | stephenfin: I wondered if you might be able to shed some light on this bug report regarding an alembic migration https://bugs.launchpad.net/nova/+bug/2008716 they report a failure to drop columns in the build_requests table going from xena -> yoga. yet this doesn't fail in the grenade job for xena -> yoga, so I'm confused if there is really a bug? | |
| 17:40:35 | sean-k-mooney | melwitt: i think gibi ^ looked at that before | |
| 17:40:38 | colby__ | sean-k-mooney: that seemed to work. Im able to migrate from xena to yoga hypervisor. But Im not able to migrate to the original hypervisor that I fully patched (updated qemu/libvirt). On that machine Im seeing the following: qemu-kvm: load of migration failed: Invalid argument: libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: 2023-10-18T17:33:57.637682Z qemu-kvm: Missing section footer for | |
| 17:41:22 | sean-k-mooney | i that sound liek a qemu crash | |
| 17:44:54 | melwitt | sean-k-mooney: yeah, he did and I agree it seems like a bug in the migration. but if it is, why doesn't it fail in grenade? I looked for awhile and couldn't figure it out. I thought stephenfin might have ideas since he worked on alembic stuff | |
| 17:46:21 | sean-k-mooney | i think we get diffent behavior in fressh install vs upgraded case but i dont exactly rememebr why | |
| 17:46:54 | sean-k-mooney | i assume the base scemea we use in once ase either has or does not have the column | |
| 17:48:23 | melwitt | yeah.. if a fresh install of xena doesn't have the columns and the yoga migration would fail to drop them bc they don't exist, I thought it should have happened in grenade since grenade does a fresh install of xena and then upgrades it to yoga | |
| 17:48:34 | sean-k-mooney | https://github.com/openstack/nova/blob/stable/xena/nova/db/api/migrations/versions/d67eeaabee36_initial_version.py#L175-L208 | |
| 17:48:46 | sean-k-mooney | so that is how the build request tabel is created on a clena install | |
| 17:49:21 | sean-k-mooney | https://github.com/openstack/nova/blob/stable/xena/nova/db/api/migrations/versions/d67eeaabee36_initial_version.py#L186 | |
| 17:49:28 | sean-k-mooney | so it has vm_states | |
| 17:49:36 | sean-k-mooney | so a clean install on xena woudl have it | |
| 17:49:54 | sean-k-mooney | but a cloud that was upgraded form wallaby perhaps might not | |
| 17:50:49 | sean-k-mooney | hum it should be there in wallaby https://github.com/openstack/nova/blob/stable/wallaby/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/067_train.py#L178 | |
| 17:51:33 | melwitt | older versions do the initial migration too. I am just not seeing how it's possible to get into a case where you don't have the columns in xena if the yoga migration is the only thing that removes them | |
| 17:51:46 | sean-k-mooney | its also there if you use legacy migrations for green filed in xena https://github.com/openstack/nova/blob/stable/xena/nova/db/api/legacy_migrations/versions/067_train.py#L166 | |
| 17:52:14 | sean-k-mooney | melwitt: perhaps we need to look at the code before we collapsed the migrations | |
| 17:54:26 | sean-k-mooney | if the bug was intoduced anywhere it was in the db compaction series | |
| 17:57:10 | sean-k-mooney | so in the patch before the dbcompaction | |
| 17:57:12 | sean-k-mooney | https://github.com/openstack/nova/blob/2d4eff0ef789957fe5498640015a6cccff43a2c3/nova/db/sqlalchemy/api_models.py#L241 | |
| 17:57:27 | sean-k-mooney | the build request tabel did not have the offending colum in the model | |
| 17:59:11 | sean-k-mooney | ah hwere so even then it was created in migration 6 with the field | |
| 17:59:13 | sean-k-mooney | https://github.com/openstack/nova/blob/2d4eff0ef789957fe5498640015a6cccff43a2c3/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/006_build_request.py#L48 | |
| 18:00:21 | sean-k-mooney | i dont see the removal migation however at this point in the repo | |
| 18:01:31 | melwitt | is that yoga? bc the removal migration was added in yoga I think | |
| 18:01:46 | sean-k-mooney | that is xena | |
| 18:01:58 | melwitt | oh ok | |
| 18:02:00 | sean-k-mooney | so in xena the table was created with the column and i dont see the removal | |
| 18:02:15 | sean-k-mooney | an pre xena the colume also existed | |
| 18:03:19 | sean-k-mooney | do you knwo which commit added the drop | |
| 18:04:17 | sean-k-mooney | https://github.com/openstack/nova/commit/9657297dd6c63e7a1e0c84c3e943b26f1795d388 | |
| 18:06:07 | sean-k-mooney | so honestly i don tsee the bug | |
| 18:06:35 | sean-k-mooney | xena had the colume and both ways of createating a clena db have the column | |
| 18:32:50 | melwitt | yeah, same | |
| 18:42:53 | opendevreview | Alexey Stupnikov proposed openstack/nova master: Preserve cached base images for failed resize ops https://review.opendev.org/c/openstack/nova/+/877410 | |
| 22:02:55 | opendevreview | melanie witt proposed openstack/nova master: docs: Fix unified limits code block text alignment https://review.opendev.org/c/openstack/nova/+/898759 | |
| 23:05:48 | opendevreview | Merged openstack/nova master: Improve logging at '_numa_cells_support_network_metadata' https://review.opendev.org/c/openstack/nova/+/860930 | |
| 23:16:21 | opendevreview | Merged openstack/nova master: Clean up service_get_all() https://review.opendev.org/c/openstack/nova/+/897674 | |
| #openstack-nova - 2023-10-19 | |||
| 11:01:54 | opendevreview | Pavlo Shchelokovskyy proposed openstack/placement master: Use new db session when retrying https://review.opendev.org/c/openstack/placement/+/898807 | |
| 11:02:39 | opendevreview | Pavlo Shchelokovskyy proposed openstack/placement master: Use separate transaction in capacity check https://review.opendev.org/c/openstack/placement/+/898808 | |
| 13:36:07 | opendevreview | Merged openstack/nova master: docs: Fix unified limits code block text alignment https://review.opendev.org/c/openstack/nova/+/898759 | |
| 14:12:12 | dansmith | sean-k-mooney: can you hit this? https://review.opendev.org/c/openstack/nova/+/898741 | |
| 14:17:41 | sean-k-mooney | sorry was doing something else but sure i can look at it now | |
| 14:17:52 | sean-k-mooney | ah the revert sure | |
| 14:18:18 | sean-k-mooney | done | |
| 14:18:47 | dansmith | sean-k-mooney: yeah didn't mean to distract, just figured it was an easy +W | |
| 14:19:20 | sean-k-mooney | no its fine i was just commiting something had litrally just finished | |
| 14:20:11 | sean-k-mooney | so before i started somethign else is the perfect time to ping | |
| 14:54:21 | bauzas | guys, I have a conflicting personal appointment next Friday at 4pm UTC | |
| 14:54:28 | bauzas | next Thur* | |
| 14:54:51 | bauzas | do people prefer we unbook the slot or do we leave it without me ? | |
| 14:56:21 | dansmith | I think we can do things without you, but maybe try to avoid scheduling anything important there and we can use it as open discussion if and as appropriate? | |
| 14:56:35 | bauzas | cool, I just noted it then in the etherpad | |
| 14:58:02 | bauzas | ralonsoh: do you have any nova related topics you'd like to be discussed at the PTG ? | |
| 14:58:46 | ralonsoh | bauzas, hi, I won't be able to attend next week to the PTG. The PTL is haleyb. I'm not aware of any Nova-Neutron topic for this PTG | |
| 14:59:09 | bauzas | ack, good call actually, I missed the PTL change, my bad | |
| 14:59:33 | bauzas | haleyb: cool cool, I was just asking | |
| 14:59:34 | haleyb | bauzas: i'm happy to have a cross-project | |
| 14:59:55 | haleyb | if needed of course | |