Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-04-08
20:17:58 openstackgerrit Matt Riedemann proposed openstack/nova master: Add finish_snapshot_based_resize_at_dest compute method https://review.openstack.org/635080
20:17:59 openstackgerrit Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.openstack.org/635668
20:17:59 openstackgerrit Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.openstack.org/614035
20:18:00 openstackgerrit Matt Riedemann proposed openstack/nova master: Filter duplicates from compute API get_migrations_sorted() https://review.openstack.org/636224
20:18:00 openstackgerrit Matt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize() https://review.openstack.org/635684
20:18:01 openstackgerrit Matt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize https://review.openstack.org/636253
20:18:01 openstackgerrit Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037
20:18:52 mriedem mnaser: are we missing indexes?
20:20:36 mriedem i don't see any indexes on instance_actions_events
20:20:48 mriedem update_statement = table.update().values(deleted=table.c.id).\ where(table.c.action_id.in_(deleted_actions))
20:20:48 mriedem but that does:
20:21:45 mriedem i don't know if adding an index on instance_actions_events.action_id would help though
20:22:04 mriedem probably need jaypipes_ to tell us what sucks about those queries
20:26:25 mriedem that's also in the mainline code https://github.com/openstack/nova/blob/fb1fee6772bb101eac83845bac9022df77113aaa/nova/db/sqlalchemy/api.py#L5288
20:28:30 mnaser Yeah I don’t know much. I had to kind orchestrate this outside nova and shortcircuit that code path
20:31:13 mriedem mnaser: do you archive/purge on a cron or just during each upgrade?
20:31:40 mriedem and tbc this just makes that archive/purge slower right?
20:33:27 mnaser mriedem: we’ve kinda not done this for a while so we ended up with huge databases.
20:33:39 mnaser so this may be a case of our fault not doing it as often
20:35:04 mriedem ok, was just trying to figure out if it's worth opening a bug report, i.e. if archive takes like 5 minutes or something, then maybe
20:43:09 efried stephenfin, bauzas, dansmith: Do we need to get those placeholder thingies in before the real release? I.e. modify the release patch?
20:45:01 efried mriedem, melwitt: ^
20:49:43 mriedem no the placeholder migrations are post-release
20:49:57 mriedem so if we needed to add a schema migration in stable/stein, we have a placeholder for it in train
20:50:51 efried mriedem: Okay, so... is there some reason we couldn't backport placeholder migrations for rocky?
20:51:12 mriedem we don't backport placeholder migrations,
20:51:18 mriedem but apparently forgot to add them for rocky
20:51:21 mriedem *in* stein
20:51:38 efried I'm confused
20:51:46 efried then why is https://review.openstack.org/#/c/650964/ in master and not in stable/stein?
20:51:58 mriedem for example, latest cell db migration in stein is 391 https://github.com/openstack/nova/tree/stable/stein/nova/db/sqlalchemy/migrate_repo/versions so if we needed something in stable/stein, we'd have to add 392 in stable/stein and then copy that to the 392_placeholder in train
20:52:11 mriedem it's essentially a buffer,
20:52:20 mriedem because otherwise the first schema migration in train is 392
20:52:30 mriedem so if we need something in stein but already used 392 in train, we're stuck
20:53:26 efried oh, so if we hadn't done any migrations in stein, we could set up the buffer for rocky.
20:53:28 jaypipes mriedem: instance_action_events.action_id will automatically have an index on it since it's an FK to instance_actions
20:53:29 mriedem rocky ended at 390 and there was one cell db migration in stein, 391
20:53:33 mriedem so if we need something on rocky we're f'ed
20:53:51 jaypipes at least for mysql and pgsql it will.
20:54:06 mriedem jaypipes: ah ok, thanks. i seem to remember asking about this same query a year or more ago and came to the same conclusion (or you told me the same at the time)
20:54:07 jaypipes I think :)
20:54:35 jaypipes might want to have mnaser do a quick SHOW TABLE instance_action_events to verify
20:54:38 mriedem else i'm thinking action_id/deleted or something
20:55:11 jaypipes hmm, maybe not since we don't use SQLalchemy create_tables()
20:55:14 mnaser be back in my machine in a sec
20:56:21 jaypipes but if the FOREIGN KEY .. REFERENCES statement/clause is executed at any time (either by SQLAlchemy's declarative_base or using sqlalchemy-migrate) then there is indeed an index created on action_id
20:56:56 jaypipes lemme log into an ocata prod DB and check...
20:58:40 jaypipes mriedem, mnaser: verified. there is indeed an index on (action_id)
20:59:12 mriedem jaypipes: ok cool
20:59:15 mriedem efried: https://review.openstack.org/#/q/project:openstack/nova+status:open+path:%255Enova/db/sqlalchemy/migrate_repo/versions/.* :)
21:02:51 openstackgerrit Dustin Cowles proposed openstack/nova master: WIP/PoC: Use SDK instead of ironicclient for node.get https://review.openstack.org/642899
21:03:35 efried dustinc: That looks like a sane delta ^
21:03:45 efried were you planning to push other patches on top of that at the same time?
21:03:58 dustinc not yet, I have them local but was just pushing that one for now
21:04:06 mnaser jaypipes: so the thing is I think it might be the deleted field
21:05:14 mnaser the query looked a something like this: UPDATE instance_actions_events SET deleted=instance_action_events.id WHERE action_id IN (SELECT id FROM instance_actions WHERE instance_uuid IN (SELECT uuid FROM instances WHERE deleted!=0))
21:05:35 mnaser so you grab a list of all non deleted instances, then you grab all their instance actions, then you grab all the action ids associated to them
21:05:37 mriedem weeee https://github.com/openstack/nova/blob/fb1fee6772bb101eac83845bac9022df77113aaa/nova/db/sqlalchemy/api.py#L174
21:06:11 mriedem mnaser: other way around - it's soft-deleting actions and events for deleted instances
21:06:16 mriedem deleted!=0 == soft-deleted
21:06:28 mriedem archive is the only thing that actually deletes actions/events
21:06:33 mnaser right, yeah, so it soft deletes them because for some reason we don't soft delete them by default? yeah
21:06:45 mnaser so if you haven't done that in a while, that can be.. a big dataset
21:06:52 mriedem mnaser: corret https://github.com/openstack/nova/blob/fb1fee6772bb101eac83845bac9022df77113aaa/nova/db/sqlalchemy/api.py#L1832
21:06:54 mriedem *correct
21:07:15 mnaser oh neat
21:07:37 mriedem could be real big if you have lots of pets
21:08:00 mnaser or a cloud with a lot of VMs going up and down
21:08:23 mnaser is there an index on deleted
21:10:27 mnaser instances_host_deleted_cleaned_idx yes there is
21:23:06 mriedem that's my fault
21:23:32 mriedem https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/cross-cell-resize
21:33:53 openstackgerrit Matt Riedemann proposed openstack/nova master: Delete require_instance_exists_using_uuid https://review.openstack.org/650994
21:33:54 openstackgerrit Matt Riedemann proposed openstack/nova master: Soft delete virtual_interfaces when instance is destroyed https://review.openstack.org/650995
21:36:30 mriedem very easy couple of changes ^
21:47:44 openstackgerrit Matt Riedemann proposed openstack/nova master: Drop delete_build_requests_with_no_instance_uuid online migration https://review.openstack.org/651001
21:48:04 openstackgerrit Ghanshyam Mann proposed openstack/nova-specs master: Spec for API inconsistency cleanup https://review.openstack.org/603969
21:48:46 openstackgerrit Matt Riedemann proposed openstack/nova master: Drop delete_build_requests_with_no_instance_uuid online migration https://review.openstack.org/651001
21:59:36 mriedem mordred: i added you to this nova spec https://review.openstack.org/#/c/603969/ about cleaning up old nasty cruft out of the compute API if you want to weigh in on your most hated UX stuff in the compute API
21:59:59 mriedem my personal most hated is the OS-EXT-* prefixes
22:00:37 mriedem i just want do do: server = GET /servers/id; host = server['host'] without looking up the API reference every time
22:10:26 openstackgerrit melanie witt proposed openstack/nova master: Add post-release checklist items to the PTL guide https://review.openstack.org/651009
22:12:09 openstackgerrit melanie witt proposed openstack/nova master: Add post-release checklist items to the PTL guide https://review.openstack.org/651009
22:12:21 dansmith efried: what mriedem said
22:44:37 efried lyarwood, jungleboyj, mdbooth: I moved "Swap volume API woes" to the (shiny new) cinder-nova xproj etherpad
22:44:40 efried https://etherpad.openstack.org/p/ptg-train-xproj-nova-cinder
22:50:54 openstackgerrit Eric Fried proposed openstack/nova-specs master: DNM: split me https://review.openstack.org/651020
23:11:47 openstackgerrit Eric Fried proposed openstack/nova-specs master: Tools & docs for backlog & abandoned spec process https://review.openstack.org/648800
23:14:29 openstackgerrit Eric Fried proposed openstack/nova-specs master: Abandon flavor-classes backlog spec https://review.openstack.org/649095
23:14:38 openstackgerrit Eric Fried proposed openstack/nova-specs master: Abandon same-instances-scheduling backlog spec https://review.openstack.org/649102
23:14:46 openstackgerrit Eric Fried proposed openstack/nova-specs master: Abandon parallel-scheduler backlog spec https://review.openstack.org/649101
23:14:52 openstackgerrit Eric Fried proposed openstack/nova-specs master: Abandon instance-tasks backlog spec https://review.openstack.org/649100
23:17:56 openstackgerrit Ivens Zambrano proposed openstack/nova-specs master: RMD Plugin: Energy Efficiency using CPU Core P-State control The power state of a core can be setup between a minimum and the maximum frequency on the cores as defined in the factory. Each core on a CPU can be defined individually to perform at specific f https://review.openstack.org/651024
23:18:20 openstackgerrit Eric Fried proposed openstack/nova-specs master: Abandon instance-tasks backlog spec https://review.openstack.org/649100
23:19:36 openstackgerrit Eric Fried proposed openstack/nova-specs master: Abandon flavor-classes backlog spec https://review.openstack.org/649095
23:20:08 openstackgerrit Eric Fried proposed openstack/nova-specs master: Abandon same-instances-scheduling backlog spec https://review.openstack.org/649102
23:20:33 openstackgerrit Eric Fried proposed openstack/nova-specs master: Abandon parallel-scheduler backlog spec https://review.openstack.org/649101
23:30:27 eandersson Does anyone have a example of using the JsonFilter for the Scheduler? :D
23:40:18 eandersson nvm found an example
23:40:47 openstackgerrit Ivens Zambrano proposed openstack/nova-specs master: RMD Plugin: Energy Efficiency using CPU Core P-State control The power state of a core can be setup between a minimum and the maximum frequency on the cores as defined in the factory. Each core on a CPU can be defined individually to perform at specific f https://review.openstack.org/651024

Earlier   Later