| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-08 | |||
| 20:12:13 | mnaser | I mentioned this earlier but.. this does a full table scan, even with max rows.. https://github.com/openstack/nova/blob/73edcfae7d5ea498fd17fa8e548ae1bd690f408a/nova/db/sqlalchemy/api.py#L5508-L5537 | |
| 20:12:18 | mnaser | I'm not sure what the best way to work around it is.. | |
| 20:17:52 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Instance.hidden field https://review.openstack.org/631123 | |
| 20:17:52 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add TargetDBSetupTask https://review.openstack.org/627892 | |
| 20:17:53 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask https://review.openstack.org/631581 | |
| 20:17:53 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask https://review.openstack.org/633853 | |
| 20:17:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method https://review.openstack.org/621313 | |
| 20:17:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method https://review.openstack.org/633293 | |
| 20:17:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask https://review.openstack.org/627890 | |
| 20:17:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.openstack.org/634832 | |
| 20:17:56 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova.compute.utils.delete_image https://review.openstack.org/637605 | |
| 20:17:56 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.openstack.org/627891 | |
| 20:17:57 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Refactor ComputeManager.remove_volume_connection https://review.openstack.org/642183 | |
| 20:17:57 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add power_on kwarg to ComputeDriver.spawn() method https://review.openstack.org/642590 | |
| 20:17:58 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add finish_snapshot_based_resize_at_dest compute method https://review.openstack.org/635080 | |
| 20:17:58 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask https://review.openstack.org/635646 | |
| 20:17:59 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.openstack.org/614035 | |
| 20:17:59 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.openstack.org/635668 | |
| 20:18:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize() https://review.openstack.org/635684 | |
| 20:18:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Filter duplicates from compute API get_migrations_sorted() https://review.openstack.org/636224 | |
| 20:18:01 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037 | |
| 20:18:01 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize https://review.openstack.org/636253 | |
| 20:18:52 | mriedem | mnaser: are we missing indexes? | |
| 20:20:36 | mriedem | i don't see any indexes on instance_actions_events | |
| 20:20:48 | mriedem | but that does: | |
| 20:20:48 | mriedem | update_statement = table.update().values(deleted=table.c.id).\ where(table.c.action_id.in_(deleted_actions)) | |
| 20:21:45 | mriedem | i don't know if adding an index on instance_actions_events.action_id would help though | |
| 20:22:04 | mriedem | probably need jaypipes_ to tell us what sucks about those queries | |
| 20:26:25 | mriedem | that's also in the mainline code https://github.com/openstack/nova/blob/fb1fee6772bb101eac83845bac9022df77113aaa/nova/db/sqlalchemy/api.py#L5288 | |
| 20:28:30 | mnaser | Yeah I don’t know much. I had to kind orchestrate this outside nova and shortcircuit that code path | |
| 20:31:13 | mriedem | mnaser: do you archive/purge on a cron or just during each upgrade? | |
| 20:31:40 | mriedem | and tbc this just makes that archive/purge slower right? | |
| 20:33:27 | mnaser | mriedem: we’ve kinda not done this for a while so we ended up with huge databases. | |
| 20:33:39 | mnaser | so this may be a case of our fault not doing it as often | |
| 20:35:04 | mriedem | ok, was just trying to figure out if it's worth opening a bug report, i.e. if archive takes like 5 minutes or something, then maybe | |
| 20:43:09 | efried | stephenfin, bauzas, dansmith: Do we need to get those placeholder thingies in before the real release? I.e. modify the release patch? | |
| 20:45:01 | efried | mriedem, melwitt: ^ | |
| 20:49:43 | mriedem | no the placeholder migrations are post-release | |
| 20:49:57 | mriedem | so if we needed to add a schema migration in stable/stein, we have a placeholder for it in train | |
| 20:50:51 | efried | mriedem: Okay, so... is there some reason we couldn't backport placeholder migrations for rocky? | |
| 20:51:12 | mriedem | we don't backport placeholder migrations, | |
| 20:51:18 | mriedem | but apparently forgot to add them for rocky | |
| 20:51:21 | mriedem | *in* stein | |
| 20:51:38 | efried | I'm confused | |
| 20:51:46 | efried | then why is https://review.openstack.org/#/c/650964/ in master and not in stable/stein? | |
| 20:51:58 | mriedem | for example, latest cell db migration in stein is 391 https://github.com/openstack/nova/tree/stable/stein/nova/db/sqlalchemy/migrate_repo/versions so if we needed something in stable/stein, we'd have to add 392 in stable/stein and then copy that to the 392_placeholder in train | |
| 20:52:11 | mriedem | it's essentially a buffer, | |
| 20:52:20 | mriedem | because otherwise the first schema migration in train is 392 | |
| 20:52:30 | mriedem | so if we need something in stein but already used 392 in train, we're stuck | |
| 20:53:26 | efried | oh, so if we hadn't done any migrations in stein, we could set up the buffer for rocky. | |
| 20:53:28 | jaypipes | mriedem: instance_action_events.action_id will automatically have an index on it since it's an FK to instance_actions | |
| 20:53:29 | mriedem | rocky ended at 390 and there was one cell db migration in stein, 391 | |
| 20:53:33 | mriedem | so if we need something on rocky we're f'ed | |
| 20:53:51 | jaypipes | at least for mysql and pgsql it will. | |
| 20:54:06 | mriedem | jaypipes: ah ok, thanks. i seem to remember asking about this same query a year or more ago and came to the same conclusion (or you told me the same at the time) | |
| 20:54:07 | jaypipes | I think :) | |
| 20:54:35 | jaypipes | might want to have mnaser do a quick SHOW TABLE instance_action_events to verify | |
| 20:54:38 | mriedem | else i'm thinking action_id/deleted or something | |
| 20:55:11 | jaypipes | hmm, maybe not since we don't use SQLalchemy create_tables() | |
| 20:55:14 | mnaser | be back in my machine in a sec | |
| 20:56:21 | jaypipes | but if the FOREIGN KEY .. REFERENCES statement/clause is executed at any time (either by SQLAlchemy's declarative_base or using sqlalchemy-migrate) then there is indeed an index created on action_id | |
| 20:56:56 | jaypipes | lemme log into an ocata prod DB and check... | |
| 20:58:40 | jaypipes | mriedem, mnaser: verified. there is indeed an index on (action_id) | |
| 20:59:12 | mriedem | jaypipes: ok cool | |
| 20:59:15 | mriedem | efried: https://review.openstack.org/#/q/project:openstack/nova+status:open+path:%255Enova/db/sqlalchemy/migrate_repo/versions/.* :) | |
| 21:02:51 | openstackgerrit | Dustin Cowles proposed openstack/nova master: WIP/PoC: Use SDK instead of ironicclient for node.get https://review.openstack.org/642899 | |
| 21:03:35 | efried | dustinc: That looks like a sane delta ^ | |
| 21:03:45 | efried | were you planning to push other patches on top of that at the same time? | |
| 21:03:58 | dustinc | not yet, I have them local but was just pushing that one for now | |
| 21:04:06 | mnaser | jaypipes: so the thing is I think it might be the deleted field | |
| 21:05:14 | mnaser | the query looked a something like this: UPDATE instance_actions_events SET deleted=instance_action_events.id WHERE action_id IN (SELECT id FROM instance_actions WHERE instance_uuid IN (SELECT uuid FROM instances WHERE deleted!=0)) | |
| 21:05:35 | mnaser | so you grab a list of all non deleted instances, then you grab all their instance actions, then you grab all the action ids associated to them | |
| 21:05:37 | mriedem | weeee https://github.com/openstack/nova/blob/fb1fee6772bb101eac83845bac9022df77113aaa/nova/db/sqlalchemy/api.py#L174 | |
| 21:06:11 | mriedem | mnaser: other way around - it's soft-deleting actions and events for deleted instances | |
| 21:06:16 | mriedem | deleted!=0 == soft-deleted | |
| 21:06:28 | mriedem | archive is the only thing that actually deletes actions/events | |
| 21:06:33 | mnaser | right, yeah, so it soft deletes them because for some reason we don't soft delete them by default? yeah | |
| 21:06:45 | mnaser | so if you haven't done that in a while, that can be.. a big dataset | |
| 21:06:52 | mriedem | mnaser: corret https://github.com/openstack/nova/blob/fb1fee6772bb101eac83845bac9022df77113aaa/nova/db/sqlalchemy/api.py#L1832 | |
| 21:06:54 | mriedem | *correct | |
| 21:07:15 | mnaser | oh neat | |
| 21:07:37 | mriedem | could be real big if you have lots of pets | |
| 21:08:00 | mnaser | or a cloud with a lot of VMs going up and down | |
| 21:08:23 | mnaser | is there an index on deleted | |
| 21:10:27 | mnaser | instances_host_deleted_cleaned_idx yes there is | |
| 21:23:06 | mriedem | that's my fault | |
| 21:23:32 | mriedem | https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/cross-cell-resize | |
| 21:33:53 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Delete require_instance_exists_using_uuid https://review.openstack.org/650994 | |
| 21:33:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Soft delete virtual_interfaces when instance is destroyed https://review.openstack.org/650995 | |
| 21:36:30 | mriedem | very easy couple of changes ^ | |
| 21:47:44 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop delete_build_requests_with_no_instance_uuid online migration https://review.openstack.org/651001 | |
| 21:48:04 | openstackgerrit | Ghanshyam Mann proposed openstack/nova-specs master: Spec for API inconsistency cleanup https://review.openstack.org/603969 | |
| 21:48:46 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop delete_build_requests_with_no_instance_uuid online migration https://review.openstack.org/651001 | |
| 21:59:36 | mriedem | mordred: i added you to this nova spec https://review.openstack.org/#/c/603969/ about cleaning up old nasty cruft out of the compute API if you want to weigh in on your most hated UX stuff in the compute API | |
| 21:59:59 | mriedem | my personal most hated is the OS-EXT-* prefixes | |
| 22:00:37 | mriedem | i just want do do: server = GET /servers/id; host = server['host'] without looking up the API reference every time | |
| 22:10:26 | openstackgerrit | melanie witt proposed openstack/nova master: Add post-release checklist items to the PTL guide https://review.openstack.org/651009 | |
| 22:12:09 | openstackgerrit | melanie witt proposed openstack/nova master: Add post-release checklist items to the PTL guide https://review.openstack.org/651009 | |
| 22:12:21 | dansmith | efried: what mriedem said | |
| 22:44:37 | efried | lyarwood, jungleboyj, mdbooth: I moved "Swap volume API woes" to the (shiny new) cinder-nova xproj etherpad | |