Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2026-03-09
17:06:09 nicolairuckel thanks
17:06:13 sean-k-mooney nicolairuckel: https://review.opendev.org/c/zuul/zuul-jobs/+/979136 for context
17:06:44 sean-k-mooney nicolairuckel: while we often have interestign ci issue near feature freeze this cycle has been extra special due to all the upstream packaging changes
17:09:12 nicolairuckel I'm just always confused when jobs that I can't reproduce locally break.
17:09:32 dansmith Uggla: this is not a regression, but probably pretty safe and the case it makes is pretty compelling: https://review.opendev.org/c/openstack/nova/+/958556
17:09:38 dansmith I had not seen this before, unfortunately
17:13:44 Uggla dansmith, I said there are interesting patches in the list. :)
17:15:16 Uggla At least for me it is ok to merge things even if it is safe and useful.
17:16:11 dansmith sean-k-mooney: thoughts? ^
17:16:57 sean-k-mooney Uggla: my question would be what benifit does merging them now vs backporting them in a week by?
17:17:32 sean-k-mooney if its a bug that we feel comfortable merging 2 day before RC1 while we may not consider it valid for an RC2
17:17:43 sean-k-mooney we likely woudl condier backportign it to sable after the release is done
17:17:59 sean-k-mooney if its not a bug i.e. a small feature we are well past FF
17:18:24 sean-k-mooney if there was a small spefic patch you tought was worth merging i am not against lookign at that
17:18:41 sean-k-mooney jsut the tiem to "merge the thign that are close" in my mind was the week before FF
17:19:10 dansmith sean-k-mooney: linked above.. it's sort of a performance bug, 2s query every give minutes in a periodic that goes to 0s with an index
17:19:21 sean-k-mooney dansmith: on where you askign for https://review.opendev.org/c/openstack/nova/+/958556 specificly
17:19:28 dansmith it's also DB so probably not a good one to backport, but yeah maybe should just wait
17:19:29 sean-k-mooney ok ill take a look at that now
17:19:39 sean-k-mooney sorry i tough you were pointing at Uggla message
17:19:56 dansmith nope, sorry
17:20:45 sean-k-mooney so a db index is pretty safe to add in genral although it may have a upgrade on upgrade
17:20:49 Uggla sean-k-mooney benefit, assuming that's something not risky. Having it the release is cool. People will have it day 1 and I could eventually announced if in the highlights.
17:21:03 sean-k-mooney i.e. in terms fo time of the deb has to compute the index
17:21:46 sean-k-mooney dansmith: with alembic we nolonger need ot have the placeholer so it proablme more backporable then it was but let me take a look i guess
17:21:47 dansmith sean-k-mooney: I was also going to ask.. I'm not polished with alembic.. is upgrade the synchronous stage?
17:22:09 dansmith sean-k-mooney: yeah, but still not great to backport unless it fixes a big bug or something
17:22:10 Uggla but again I'm not pushing/forcing to have them by all mean. Relying on your (core) judgement.
17:22:34 sean-k-mooney upgrade in our case i think is part of the expand set
17:22:56 sean-k-mooney but also not an expret on this i would have ot look it up or read our docs
17:22:57 dansmith yeah, so I think that's the right phase for it
17:23:36 dansmith upgrade is what you can run before you deploy new code, so that's right
17:24:00 dansmith anyway, maybe the fact that we're even discussing means it should wait, it's just a juicy performance bug with a very simple solution
17:24:36 sean-k-mooney as a mitigation will alembic complain if you manually added the same index?
17:24:58 dansmith like if the operator has done this manually already?
17:25:05 sean-k-mooney im not sure how picky it is to schema deviation
17:25:09 sean-k-mooney ya
17:25:25 dansmith I also don't know that.. I thought it has to look and see if the uuid of the migration is recorded or not, but I also thought it was supposed to be sort of idempotent, so idk
17:25:53 dansmith I think the fact that we're not positive here means we should wait,
17:25:58 sean-k-mooney basiclly im askign if the down revisiohn has is based on our model defintion or 2903cd72dc14 by looking at the tables in the db
17:26:26 sean-k-mooney fine by me i normally ask stephenfin about ^ or melwitt
17:27:41 sean-k-mooney anyway without diging into those detail the patch looks vaild to me so ill +1 it but beyond that i would have to read up on the detials
17:28:13 dansmith ack Uggla let's punt, but I think this is a good one to track and nag people to look at as soon as H opens
17:29:24 Uggla dansmith 👍
17:32:42 stephenfin sean-k-mooney: dansmith: if the question is about backporting migrations, if you're going back further than the release the last migration was added in, then you'll end up creating a branch https://alembic.sqlalchemy.org/en/latest/branches.html#working-with-branches
17:33:35 dansmith I don't want to backport this
17:33:51 sean-k-mooney stephenfin: it more how safe to we feel https://review.opendev.org/c/openstack/nova/+/958556 is and what its upgrade impact
17:33:54 dansmith I think sean-k-mooney's question was what happens if someone has manually done this or something
17:34:08 sean-k-mooney also that
17:34:40 sean-k-mooney the current patch does not have an upgrade release note and i suspect this will take some time to caluate the inde of there are a large number of rows
17:34:54 dansmith also true
17:34:56 sean-k-mooney i belive this will be appled as part of the db sync when we do the expand migritons
17:35:06 dansmith but if it's in the upgrade phase it should be pre-doable
17:35:11 stephenfin Well they're already out of support, but I believe you'll simply end up with a duplicate index with a different name
17:35:42 stephenfin it would be a 5 minute test if you'd a devstack to hand
17:35:49 sean-k-mooney stephenfin: basicly i was wondering if we fix this in master in 2026.2
17:36:12 sean-k-mooney can operator "fix it themselve" by just creating the index on older rellease without breaking our upgrades
17:36:22 sean-k-mooney or will almebic be unhapply
17:36:39 sean-k-mooney if its just a duplcit index and it wont break future migration that proably ok
17:38:22 sean-k-mooney to be clear 2.264 sec to 0.006 sec for 500k rows is nice but for a perodic that runs every 5 minutes its not unworkable at 2.5seconds
17:38:32 sean-k-mooney defintly worth fixing on master however
17:38:50 dansmith I thought the claim was that this overwhelms conductors when you have a ton of nodes doing it
17:39:07 dansmith "NOTE: missing index for a query that gets regulary executed by every nova-compute node this can overwhelm the nova-conductor service in large scale deployments"
17:39:26 sean-k-mooney oh its every 5 mintes per compute
17:39:26 dansmith (from the etherpad, maybe should be copied into the bug)
17:39:30 dansmith yeah
17:39:44 sean-k-mooney ok that scaling diffently then i initally tought
17:39:59 dansmith and probably scales like n^2 almost because each compute doing it is scanning each other compute record that you ad
17:40:02 dansmith *add
17:40:04 sean-k-mooney although ya it makes sense this is per compute
17:41:04 sean-k-mooney we already have indexs on source_compute and dest_compute and source_node and dest node
17:41:07 stephenfin sean-k-mooney: I can't say, but we've never written our migrations on the assumption that the operator has manually hacked in their own migrations
17:41:20 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/958556/1/nova/db/main/models.py#895
17:41:23 stephenfin so I'm not sure how this would be any different
17:41:52 stephenfin however, if they *had* done that, they can always dummy apply the migration using the stamp command
17:42:04 sean-k-mooney im actully surpised that a quey on teh subset is not coverd by htat
17:42:10 stephenfin https://alembic.sqlalchemy.org/en/latest/api/commands.html#alembic.command.stamp
17:42:45 sean-k-mooney stephenfin: well in the very old days we assumed the dba may create indexes ro set coalation types to optimise for there usecase
17:43:10 sean-k-mooney im not saying that a good thing just that it was a thing
17:43:31 sean-k-mooney the ops guided uses to have soem recomentions for adding indexes but that was eons ago
18:25:46 dansmith Uggla: the other volume bugs there are good things to fix, but I'm not sure in the best way.. I've left some comments
18:42:54 opendevreview Merged openstack/os-traits master: Remove MANIFEST.in https://review.opendev.org/c/openstack/os-traits/+/975992
18:44:20 opendevreview Merged openstack/osc-placement master: Remove MANIFEST.in https://review.opendev.org/c/openstack/osc-placement/+/975984
#openstack-nova - 2026-03-10
07:25:33 opendevreview Max proposed openstack/nova master: fix: leftover volume_attachment on instance delete https://review.opendev.org/c/openstack/nova/+/934984
08:00:17 jkulik gibi, when you find some time, could your provide feedback to https://review.opendev.org/c/openstack/nova-specs/+/975698 ? it's a proposal to fix the race-condition you mentioned in my recent volume-attachment changes
09:40:05 opendevreview Max proposed openstack/nova master: Add regression test for bug #2139135 https://review.opendev.org/c/openstack/nova/+/974831
09:40:06 opendevreview Max proposed openstack/nova master: fix: delete attachments after rescheduling delete https://review.opendev.org/c/openstack/nova/+/974832
09:52:32 opendevreview Max proposed openstack/nova master: db: add indexes for source/dest migrations https://review.opendev.org/c/openstack/nova/+/958556
10:28:08 opendevreview Sylvain Bauza proposed openstack/nova master: Support multiple allocations for vGPUs https://review.opendev.org/c/openstack/nova/+/845757
10:54:51 opendevreview Seyeong Kim proposed openstack/nova master: libvirt: Support boot_index for multiple block devices https://review.opendev.org/c/openstack/nova/+/963665
14:20:09 opendevreview Ashish Gupta proposed openstack/nova master: Run functional test with threading mode https://review.opendev.org/c/openstack/nova/+/979850
15:04:10 opendevreview Merged openstack/nova master: Add regression test for bug #2088066 https://review.opendev.org/c/openstack/nova/+/965927
15:11:39 opendevreview Nicolai Ruckel proposed openstack/nova stable/2025.1: Add functional reproducer for bug 2125030 https://review.opendev.org/c/openstack/nova/+/979864
15:11:40 opendevreview Nicolai Ruckel proposed openstack/nova stable/2025.1: Move cleanup of vTPM secret from driver to compute https://review.opendev.org/c/openstack/nova/+/979865
15:11:41 opendevreview Nicolai Ruckel proposed openstack/nova stable/2025.1: Preserve vTPM state between power off and power on https://review.opendev.org/c/openstack/nova/+/979866
15:25:32 dansmith Uggla: your -1 is still on here from Feb: https://review.opendev.org/c/openstack/nova/+/972045/2
15:25:52 dansmith I guess because you -1'd this patch for the thing on the one below, and since this one didn't change it didn't reset
15:44:06 bauzas gibi: melwitt: just a note that I updated https://review.opendev.org/c/openstack/nova/+/845757 by adding the bug report in the reno file
15:44:18 bauzas (and thanks for your reviews <3 )
15:53:04 MaxLamprecht[m] dansmith: thx for review of the attachment stuff. During debugging I found 2 more things that are really small and broken: https://review.opendev.org/c/openstack/nova/+/973103 and https://review.opendev.org/c/openstack/nova/+/975651

Earlier   Later