| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-17 | |||
| 20:05:31 | mriedem | so i guess, | |
| 20:05:38 | superdan | was the aggregate move to api in ocata? | |
| 20:06:00 | melwitt | I thought it was earlier | |
| 20:06:08 | melwitt | I mean, it's definitely before ocata | |
| 20:06:09 | superdan | melwitt: if so we can't have deleted aggregates | |
| 20:06:11 | mriedem | newton | |
| 20:06:23 | mriedem | https://github.com/openstack/nova/commit/35748e7a9b3445a60522e088e38bf25e6bfef25e | |
| 20:06:30 | mriedem | i was thinking it's odd to update the migration itself, | |
| 20:06:34 | mriedem | but i guess if you're stuck on 330, | |
| 20:06:40 | mriedem | then you can't get past that anyway | |
| 20:06:41 | superdan | I guess that move happens after the sync though, if they both happened in newton | |
| 20:06:52 | superdan | mriedem: yeah | |
| 20:07:05 | superdan | mriedem: the other option is to convert all the deleted records, but that's kinda silly | |
| 20:07:18 | melwitt | there's a REST API for deleting an aggregate https://developer.openstack.org/api-ref/compute/?expanded=delete-aggregate-detail#delete-aggregate | |
| 20:07:22 | superdan | especially if we're about to delete them in the case of aggregates | |
| 20:07:31 | melwitt | so if that connects to an Aggregate.destroy(), that would do it | |
| 20:07:34 | superdan | melwitt: right, but once they're in the api db they can;t be soft deleted | |
| 20:07:43 | melwitt | ah, that's what you meant. yeah | |
| 20:07:55 | mriedem | so in mitaka i deleted a compute node, | |
| 20:08:14 | mriedem | actually i need to see the online migration code first | |
| 20:08:25 | melwitt | yeah, I have links up still I think | |
| 20:08:49 | melwitt | the online migration only migrates non-soft-deleted things and the blocker doesn't filter on deleted=0 so it just gets everything | |
| 20:09:00 | mriedem | what does https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L595 have to do with compute nodes? | |
| 20:09:36 | mriedem | you guys got some out of tree online migrations? | |
| 20:09:58 | mriedem | or is it just this? https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L597 | |
| 20:10:03 | mriedem | db.aggregate_uuids_online_data_migration, | |
| 20:10:07 | mriedem | that's the only mitaka one in that list | |
| 20:10:29 | mriedem | i should be looking at stable/newton i guess https://github.com/openstack/nova/blob/stable/newton/nova/cmd/manage.py#L786 | |
| 20:10:32 | melwitt | no, it looks like we removed the compute nodes one | |
| 20:10:33 | mriedem | which includes pci devices | |
| 20:10:37 | melwitt | after | |
| 20:10:40 | melwitt | yeah | |
| 20:11:40 | mriedem | oh nvm i see https://github.com/openstack/nova/blob/stable/mitaka/nova/cmd/manage.py#L930 | |
| 20:11:48 | mriedem | in mitaka we had an online migration for compute nodes | |
| 20:11:49 | mriedem | for the uuid | |
| 20:12:27 | mriedem | https://github.com/openstack/nova/blob/stable/mitaka/nova/db/sqlalchemy/api.py#L6531 | |
| 20:12:50 | mriedem | so that online migration can get soft deleted compute nodes with no uuid | |
| 20:12:52 | mriedem | right? | |
| 20:13:31 | melwitt | the online migration does not migrate soft-deleted compute nodes with no uuid | |
| 20:13:56 | melwitt | so I thought our choices are: migrate even soft-deleted things or skip soft-deleted things in the blocker | |
| 20:14:04 | mriedem | because of model_query | |
| 20:14:05 | mriedem | i suppose | |
| 20:14:07 | melwitt | yeah | |
| 20:14:16 | melwitt | model_query defaults to read_deleted='no' | |
| 20:14:18 | mriedem | and the context doesn't have read_deleted='yes' or whatever | |
| 20:14:19 | mriedem | ok | |
| 20:14:20 | mriedem | yup | |
| 20:14:55 | mriedem | ok so in mitaka we have soft deleted compute_nodes, online data migration doesn't get them b/c of model_query, but 330 blocked migration pulls them in b/c it's not using the orm and it explodes | |
| 20:15:01 | mriedem | because we don't filter on deleted==0 | |
| 20:15:07 | melwitt | the only gotcha would be if it were valid for any of those objects: compute nodes, aggregates, pci devices to be read in a deleted state | |
| 20:15:14 | mriedem | which is your fix | |
| 20:15:19 | melwitt | right | |
| 20:15:46 | mriedem | geez melwitt your commit message could have been several more paragraphs of explanation :) | |
| 20:16:11 | melwitt | because this fix would leave the soft-deleted unmigrated records around. the alternate fix would migrate potentially a lot of soft-deleted things for no reason | |
| 20:16:40 | mriedem | we don't read deleted on any of those records as far as i know | |
| 20:16:48 | melwitt | mriedem: yeah, sorry about that. I can add more info to it | |
| 20:16:56 | mriedem | melwitt: i'm only half joking | |
| 20:16:57 | mriedem | :) | |
| 20:16:58 | melwitt | yeah, that's what I thought too | |
| 20:17:08 | mriedem | superdan: do you know if we read deleted any of those? | |
| 20:17:21 | superdan | on compute nodes? | |
| 20:17:32 | melwitt | aggregates and pci devices also | |
| 20:17:38 | mriedem | i have like 1% knowledge of the pci_devices code | |
| 20:17:49 | superdan | definitely not pci_devices | |
| 20:18:32 | superdan | melwitt: if we did read aggregates, we'd generate on load: https://github.com/openstack/nova/blob/stable/newton/nova/objects/aggregate.py#L288 | |
| 20:18:34 | superdan | so we're okay | |
| 20:18:57 | melwitt | ah, nice | |
| 20:20:38 | mriedem | we mostly read deleted instances | |
| 20:20:43 | mriedem | so yeah i think we're ok there | |
| 20:21:18 | mriedem | and flavors | |
| 20:24:23 | mriedem | just testing the test quick | |
| 20:24:28 | mriedem | do we want/need a release note for this? | |
| 20:24:32 | mriedem | since it's going to be backported | |
| 20:24:54 | mriedem | like even a "fixes" note or "upgrade" note? as i said earlier, if you hit this you can't get past 330 anyway | |
| 20:25:02 | mriedem | so this is more about fixing it before people hit it | |
| 20:25:23 | mriedem | last time we backported a db schema migration to stable and released it i got some flak from red hat | |
| 20:27:56 | superdan | this isn't backporting a new one, | |
| 20:28:00 | superdan | it's just tweaking an old one, | |
| 20:28:02 | mriedem | i know | |
| 20:28:03 | superdan | which we did in ocata too | |
| 20:28:06 | mriedem | melwitt: see my comment in the test | |
| 20:28:16 | superdan | but release notes are cool | |
| 20:29:03 | mriedem | so the test doesn't fail on the aggregate b/c you're using an object, which creates the aggregate in the api db | |
| 20:29:11 | mriedem | i don't know why it doesn't fail on the compute node when i remove your fi | |
| 20:29:13 | mriedem | *fix | |
| 20:29:15 | melwitt | oh, hm | |
| 20:29:21 | superdan | mriedem: it's because it has a uuid | |
| 20:29:24 | superdan | mriedem: if you reate it | |
| 20:29:27 | mriedem | ah | |
| 20:29:27 | superdan | I'm thinking | |
| 20:29:29 | mriedem | yup | |
| 20:29:35 | superdan | she'd have to use the db api | |
| 20:29:37 | melwitt | okay, let me fix this to put things in the old db | |
| 20:30:17 | mriedem | yup | |
| 20:30:24 | mriedem | pci device object one is fine | |
| 20:30:31 | mriedem | but use the db api directly for the compute notes and aggregates | |
| 20:30:44 | melwitt | right | |
| 20:30:44 | mriedem | fwiw i've put a hold on https://review.openstack.org/#/c/434928/ until we get this backported and merged | |
| 20:33:46 | diana_clarke | melwitt: Here's an easy review that fixes up the mismatch in metavars you noticed a couple of weeks back: https://review.openstack.org/#/c/431714/ | |
| 20:34:06 | diana_clarke | Have a blast at the PTG, folks! | |