| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-03-09 | |||
| 17:01:44 | nicolairuckel | https://review.opendev.org/c/openstack/nova/+/978908/7 | |
| 17:01:50 | opendevreview | Thibaut Démaret proposed openstack/nova-specs master: Spec for default SSD rotation rate on instance disks https://review.opendev.org/c/openstack/nova-specs/+/979696 | |
| 17:02:52 | tkajinam | nicolairuckel, Add Depends-on: https://review.opendev.org/c/openstack/placement/+/979685 to the commit message. Probably right above Signed-Off-By | |
| 17:03:04 | tkajinam | oh, wait | |
| 17:03:05 | tkajinam | not that one | |
| 17:03:07 | sean-k-mooney | https://review.opendev.org/c/openstack/placement/+/979685 | |
| 17:03:13 | sean-k-mooney | is fixing the docs job in placment | |
| 17:03:18 | tkajinam | I guess we need the fix for nova | |
| 17:03:24 | sean-k-mooney | right | |
| 17:03:25 | tkajinam | yeah | |
| 17:03:30 | elodilles | nicolairuckel: i think you don't need to do a depends-on as your patch is in nova repo and the broken job is not placement related | |
| 17:03:53 | sean-k-mooney | there actul failre is the tox-cover job | |
| 17:03:59 | sean-k-mooney | and that was fixed globally right | |
| 17:04:10 | tkajinam | oh yes | |
| 17:04:13 | tkajinam | just recheck | |
| 17:04:13 | elodilles | tkajinam: nicolairuckel: i think we don't need the fix as it was already fixed on Friday | |
| 17:04:16 | nicolairuckel | So just recheck? | |
| 17:04:20 | elodilles | yepp, what sean-k-mooney | |
| 17:04:26 | elodilles | says | |
| 17:04:27 | tkajinam | AttributeError: 'Parsed' object has no attribute 'config_format' | |
| 17:04:41 | tkajinam | that's the root cause and that was fixed by the zuul-job update | |
| 17:04:44 | tkajinam | nicolairuckel, yup | |
| 17:04:47 | sean-k-mooney | ya that was fixed in zuul-jobs | |
| 17:05:48 | elodilles | nicolairuckel: i'll try to look into your patch tomorrow as it's already end of day to me now o:) | |
| 17:06:07 | nicolairuckel | For me too | |
| 17:06:09 | nicolairuckel | thanks | |
| 17:06:13 | sean-k-mooney | nicolairuckel: https://review.opendev.org/c/zuul/zuul-jobs/+/979136 for context | |
| 17:06:44 | sean-k-mooney | nicolairuckel: while we often have interestign ci issue near feature freeze this cycle has been extra special due to all the upstream packaging changes | |
| 17:09:12 | nicolairuckel | I'm just always confused when jobs that I can't reproduce locally break. | |
| 17:09:32 | dansmith | Uggla: this is not a regression, but probably pretty safe and the case it makes is pretty compelling: https://review.opendev.org/c/openstack/nova/+/958556 | |
| 17:09:38 | dansmith | I had not seen this before, unfortunately | |
| 17:13:44 | Uggla | dansmith, I said there are interesting patches in the list. :) | |
| 17:15:16 | Uggla | At least for me it is ok to merge things even if it is safe and useful. | |
| 17:16:11 | dansmith | sean-k-mooney: thoughts? ^ | |
| 17:16:57 | sean-k-mooney | Uggla: my question would be what benifit does merging them now vs backporting them in a week by? | |
| 17:17:32 | sean-k-mooney | if its a bug that we feel comfortable merging 2 day before RC1 while we may not consider it valid for an RC2 | |
| 17:17:43 | sean-k-mooney | we likely woudl condier backportign it to sable after the release is done | |
| 17:17:59 | sean-k-mooney | if its not a bug i.e. a small feature we are well past FF | |
| 17:18:24 | sean-k-mooney | if there was a small spefic patch you tought was worth merging i am not against lookign at that | |
| 17:18:41 | sean-k-mooney | jsut the tiem to "merge the thign that are close" in my mind was the week before FF | |
| 17:19:10 | dansmith | sean-k-mooney: linked above.. it's sort of a performance bug, 2s query every give minutes in a periodic that goes to 0s with an index | |
| 17:19:21 | sean-k-mooney | dansmith: on where you askign for https://review.opendev.org/c/openstack/nova/+/958556 specificly | |
| 17:19:28 | dansmith | it's also DB so probably not a good one to backport, but yeah maybe should just wait | |
| 17:19:29 | sean-k-mooney | ok ill take a look at that now | |
| 17:19:39 | sean-k-mooney | sorry i tough you were pointing at Uggla message | |
| 17:19:56 | dansmith | nope, sorry | |
| 17:20:45 | sean-k-mooney | so a db index is pretty safe to add in genral although it may have a upgrade on upgrade | |
| 17:20:49 | Uggla | sean-k-mooney benefit, assuming that's something not risky. Having it the release is cool. People will have it day 1 and I could eventually announced if in the highlights. | |
| 17:21:03 | sean-k-mooney | i.e. in terms fo time of the deb has to compute the index | |
| 17:21:46 | sean-k-mooney | dansmith: with alembic we nolonger need ot have the placeholer so it proablme more backporable then it was but let me take a look i guess | |
| 17:21:47 | dansmith | sean-k-mooney: I was also going to ask.. I'm not polished with alembic.. is upgrade the synchronous stage? | |
| 17:22:09 | dansmith | sean-k-mooney: yeah, but still not great to backport unless it fixes a big bug or something | |
| 17:22:10 | Uggla | but again I'm not pushing/forcing to have them by all mean. Relying on your (core) judgement. | |
| 17:22:34 | sean-k-mooney | upgrade in our case i think is part of the expand set | |
| 17:22:56 | sean-k-mooney | but also not an expret on this i would have ot look it up or read our docs | |
| 17:22:57 | dansmith | yeah, so I think that's the right phase for it | |
| 17:23:36 | dansmith | upgrade is what you can run before you deploy new code, so that's right | |
| 17:24:00 | dansmith | anyway, maybe the fact that we're even discussing means it should wait, it's just a juicy performance bug with a very simple solution | |
| 17:24:36 | sean-k-mooney | as a mitigation will alembic complain if you manually added the same index? | |
| 17:24:58 | dansmith | like if the operator has done this manually already? | |
| 17:25:05 | sean-k-mooney | im not sure how picky it is to schema deviation | |
| 17:25:09 | sean-k-mooney | ya | |
| 17:25:25 | dansmith | I also don't know that.. I thought it has to look and see if the uuid of the migration is recorded or not, but I also thought it was supposed to be sort of idempotent, so idk | |
| 17:25:53 | dansmith | I think the fact that we're not positive here means we should wait, | |
| 17:25:58 | sean-k-mooney | basiclly im askign if the down revisiohn has is based on our model defintion or 2903cd72dc14 by looking at the tables in the db | |
| 17:26:26 | sean-k-mooney | fine by me i normally ask stephenfin about ^ or melwitt | |
| 17:27:41 | sean-k-mooney | anyway without diging into those detail the patch looks vaild to me so ill +1 it but beyond that i would have to read up on the detials | |
| 17:28:13 | dansmith | ack Uggla let's punt, but I think this is a good one to track and nag people to look at as soon as H opens | |
| 17:29:24 | Uggla | dansmith 👍 | |
| 17:32:42 | stephenfin | sean-k-mooney: dansmith: if the question is about backporting migrations, if you're going back further than the release the last migration was added in, then you'll end up creating a branch https://alembic.sqlalchemy.org/en/latest/branches.html#working-with-branches | |
| 17:33:35 | dansmith | I don't want to backport this | |
| 17:33:51 | sean-k-mooney | stephenfin: it more how safe to we feel https://review.opendev.org/c/openstack/nova/+/958556 is and what its upgrade impact | |
| 17:33:54 | dansmith | I think sean-k-mooney's question was what happens if someone has manually done this or something | |
| 17:34:08 | sean-k-mooney | also that | |
| 17:34:40 | sean-k-mooney | the current patch does not have an upgrade release note and i suspect this will take some time to caluate the inde of there are a large number of rows | |
| 17:34:54 | dansmith | also true | |
| 17:34:56 | sean-k-mooney | i belive this will be appled as part of the db sync when we do the expand migritons | |
| 17:35:06 | dansmith | but if it's in the upgrade phase it should be pre-doable | |
| 17:35:11 | stephenfin | Well they're already out of support, but I believe you'll simply end up with a duplicate index with a different name | |
| 17:35:42 | stephenfin | it would be a 5 minute test if you'd a devstack to hand | |
| 17:35:49 | sean-k-mooney | stephenfin: basicly i was wondering if we fix this in master in 2026.2 | |
| 17:36:12 | sean-k-mooney | can operator "fix it themselve" by just creating the index on older rellease without breaking our upgrades | |
| 17:36:22 | sean-k-mooney | or will almebic be unhapply | |
| 17:36:39 | sean-k-mooney | if its just a duplcit index and it wont break future migration that proably ok | |
| 17:38:22 | sean-k-mooney | to be clear 2.264 sec to 0.006 sec for 500k rows is nice but for a perodic that runs every 5 minutes its not unworkable at 2.5seconds | |
| 17:38:32 | sean-k-mooney | defintly worth fixing on master however | |
| 17:38:50 | dansmith | I thought the claim was that this overwhelms conductors when you have a ton of nodes doing it | |
| 17:39:07 | dansmith | "NOTE: missing index for a query that gets regulary executed by every nova-compute node this can overwhelm the nova-conductor service in large scale deployments" | |
| 17:39:26 | dansmith | (from the etherpad, maybe should be copied into the bug) | |
| 17:39:26 | sean-k-mooney | oh its every 5 mintes per compute | |
| 17:39:30 | dansmith | yeah | |
| 17:39:44 | sean-k-mooney | ok that scaling diffently then i initally tought | |
| 17:39:59 | dansmith | and probably scales like n^2 almost because each compute doing it is scanning each other compute record that you ad | |
| 17:40:02 | dansmith | *add | |
| 17:40:04 | sean-k-mooney | although ya it makes sense this is per compute | |
| 17:41:04 | sean-k-mooney | we already have indexs on source_compute and dest_compute and source_node and dest node | |
| 17:41:07 | stephenfin | sean-k-mooney: I can't say, but we've never written our migrations on the assumption that the operator has manually hacked in their own migrations | |
| 17:41:20 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/958556/1/nova/db/main/models.py#895 | |
| 17:41:23 | stephenfin | so I'm not sure how this would be any different | |
| 17:41:52 | stephenfin | however, if they *had* done that, they can always dummy apply the migration using the stamp command | |