| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-06-05 | |||
| 17:46:43 | jaypipes | zzzeek: using a col.alter(...) command. | |
| 17:47:00 | jaypipes | zzzeek: PG hated that and requires me to drop and recreate the columns. | |
| 17:47:36 | zzzeek | jaypipes: it does? usually it just wants you to use a CAST so it can rewrite the data ...oh there'sa syntax for that hold on ... :) | |
| 17:47:48 | jaypipes | zzzeek: so I did that, then MySQL complains about indexes either existing or not existing depending on whether I drop the columns before or after dropping indexes. and when I drop indexes, PG complains there's no such indexes. | |
| 17:48:34 | zzzeek | jaypipes: what is the "PG hated that" error you are getting ? | |
| 17:48:42 | jaypipes | one sec, grabbing. | |
| 17:49:33 | zzzeek | jaypipes: also, your problem might be "migrate_engine.name == 'postgres'", sqlalchemy calls it "postgresql" | |
| 17:50:17 | jaypipes | zzzeek: no, that definitely runs only on postgresql AFAICT... it's the name of the db driver in sqlalchemy-migrate :( | |
| 17:50:29 | zzzeek | jaypipes: weird. | |
| 17:50:54 | zzzeek | jaypipes: so....why not put the whole drop/create thing inside of "if name == 'postgres'" and just use normal ALTER for the others ? | |
| 17:51:32 | jaypipes | zzzeek: I think I've tried that and it failed something about models being out of sync... | |
| 17:51:42 | zzzeek | keeping in mind that PG might just want you to say "ALTER COLUMN...USING", which is pg-specific anyway so you still need the conditional | |
| 17:51:54 | jaypipes | zzzeek: unfortunately, CI doesn't seem to want to allow me to see past test run failures :( | |
| 17:52:26 | jaypipes | zzzeek: how would I do the ALTER COLUMN ... USING thing if using sqlalchemy-migrate's col.alter() thing? | |
| 17:52:35 | zzzeek | jaypipes: that doesn't make any sense. program goes into the migration, and comes out, then it verifies things. if the migration works, then it worked, whether or not it did ALTER or drop/create | |
| 17:52:49 | zzzeek | jaypipes: you'd have to do migrate_engine.execute("ALTER TABLE...") | |
| 17:53:01 | jaypipes | k | |
| 17:53:12 | jaypipes | sdague: is there a way to view old CI test runs? | |
| 17:53:23 | sdague | jaypipes: how old? | |
| 17:53:25 | zzzeek | jaypipes: if that's the issue. not sure if PG has done things w/ the indexes that are more of a big deal to change | |
| 17:53:28 | jaypipes | sdague: a day | |
| 17:53:46 | sdague | yeh, unhide the CI and look through the jenkins comments | |
| 17:53:53 | jaypipes | sdague: oh, duh, thx | |
| 17:53:57 | sdague | the rollup of the last run is just a convenience function | |
| 17:54:36 | jaypipes | zzzeek: ok, here's the failure from the original alter: http://logs.openstack.org/45/470645/1/check/gate-nova-tox-functional-ubuntu-xenial/8180560/testr_results.html.gz | |
| 17:55:26 | jaypipes | zzzeek: that happened when I did the original here: https://review.openstack.org/#/c/470645/1/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py | |
| 17:55:58 | zzzeek | jaypipes: ha! HINT: You might need to specify "USING project_id::integer". me wants another darth vader usb key | |
| 17:56:29 | jaypipes | zzzeek: yeah... so instead of that, I did this: https://review.openstack.org/#/c/470645/2/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py | |
| 17:56:42 | dansmith | mriedem: so this passed the live migration job. Do you still want me to change it or what? https://review.openstack.org/#/c/458537/31/nova/tests/live_migration/hooks/ceph.sh | |
| 17:56:52 | jaypipes | zzzeek: which failed with this: http://logs.openstack.org/45/470645/2/check/gate-nova-tox-functional-ubuntu-xenial/f94cf09/testr_results.html.gz | |
| 17:57:51 | zzzeek | jaypipes: OK, so theres the most ancient issue in SQLA to allow index.drop(checkfirst=True), however, I can get you through that anyway, if you want to go that route. but still, "if engine != 'postgres'", then drop index | |
| 17:58:14 | zzzeek | jaypipes: but still, "if engine == 'postgres': migrate_engine.execute("ALTER.... USING")" | |
| 17:58:31 | jaypipes | zzzeek: not sure, frankly, why https://review.openstack.org/#/c/470645/2/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py didn't work. I don't know why sqlalchemy found indexes in the consumers table to drop but when issuing the SQL statement, PG says no such index exists. :( | |
| 17:58:54 | zzzeek | jaypipes: looks like it's because dropping the columns drops the index | |
| 17:59:04 | zzzeek | jaypipes: then you go to emit DROP INDEX and PG says index does not exist | |
| 17:59:18 | jaypipes | zzzeek: right, which I then tried to fix in the next revision: :) | |
| 17:59:36 | jaypipes | https://review.openstack.org/#/c/470645/3/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py | |
| 17:59:51 | zzzeek | jaypipes: oh, wait, you are assuming consumers.indexes is correct. no. this is not how that works :) | |
| 18:00:00 | zzzeek | jaypipes: throw this into "sqlalchemy-migrate is misleading" | |
| 18:00:02 | jaypipes | zzzeek: which PG then barfed on | |
| 18:00:16 | zzzeek | jaypipes: yeah no, consumers.indexes is not changing itself when you drop those columns | |
| 18:00:19 | mordred | cdent, mriedem: the shade test patch is all green! | |
| 18:00:30 | jaypipes | zzzeek: hmm. ok. | |
| 18:00:42 | zzzeek | jaypipes: so, if you want *that*, do this: "from sqlalchemy import inspect; insp = inspect(migrate_engine); indexes = insp.get_indexes("consumers")" | |
| 18:00:44 | cdent | yayness. I think sdague said he's fast approve the devstack change | |
| 18:00:50 | mordred | although linuxbridge had a sad | |
| 18:01:27 | jaypipes | zzzeek: I think the simplest solution is to go back to revision 1 and do the execute(ALTER TABLE ... USING) thing for Postgresql? | |
| 18:01:38 | zzzeek | jaypipes: that will just give you names of indexes. if it were me I'd then just say "engine.execute("DROP INDEX <name>")", howver you can also say Index("myname").drop() | |
| 18:02:03 | zzzeek | jaypipes: I think it is yes. but all of these different scenarios can work, you're just having very bad luck stepping on every rake in the yard :) | |
| 18:02:23 | jaypipes | indeed :) | |
| 18:02:47 | zzzeek | jaypipes: try the USING with the conditional, i can make that or the other version work for you afte that | |
| 18:02:56 | zzzeek | jaypipes: you don't run mysql/pg functional tests locally ? | |
| 18:03:20 | sdague | that linuxbridge sad was a fail in test_device_tagging | |
| 18:03:34 | jaypipes | zzzeek: only mysql and sqlite, not postgreSQL, no... | |
| 18:03:34 | sdague | mriedem: that one being tracked yet, the error looks like it's going to be a race | |
| 18:03:43 | jaypipes | zzzeek: suppose I should :) | |
| 18:03:58 | zzzeek | jaypipes: it's just a yum install away... :) | |
| 18:04:15 | jaypipes | yeah yeah :) | |
| 18:04:35 | zzzeek | jaypipes: anyway I obv. have everythign running here so i can run through it later if you still cant get it working | |
| 18:04:58 | jaypipes | zzzeek: I'll just install PG :) | |
| 18:05:04 | zzzeek | jaypipes: OK. | |
| 18:10:19 | mriedem | sdague: huh? link? | |
| 18:12:33 | openstackgerrit | Merged openstack/nova master: Be more tolerant of keystone catalog configuration https://review.openstack.org/438049 | |
| 18:12:49 | mriedem | sdague: oh you mean this http://logs.openstack.org/42/471042/2/check/gate-tempest-dsvm-neutron-linuxbridge-ubuntu-xenial/4cc66aa/console.html#_2017-06-05_17_23_22_278409 | |
| 18:12:58 | mriedem | i'm not aware of a bug for that | |
| 18:13:13 | mriedem | dansmith: just leave it | |
| 18:17:54 | openstackgerrit | Doug Hellmann proposed openstack/python-novaclient master: strip the remote prefixes from the release note branch specifiers https://review.openstack.org/471082 | |
| 18:18:16 | sdague | mriedem: yeh - http://logs.openstack.org/42/471042/2/check/gate-tempest-dsvm-neutron-linuxbridge-ubuntu-xenial/4cc66aa/console.html#_2017-06-05_17_23_22_337560 | |
| 18:18:24 | sdague | I'll make a bug here in a minute for it | |
| 18:18:36 | sdague | there are a few hits on it | |
| 18:53:02 | sdague | \o/ for the diagnostics API standardization - https://review.openstack.org/#/c/355540/ | |
| 19:03:12 | openstackgerrit | Chris Dent proposed openstack/nova master: Fix ChanceScheduler returning a wrong node https://review.openstack.org/466725 | |
| 19:03:18 | mriedem | cool. a few wording things in there that could be cleaned up in a follow up change | |
| 19:04:09 | mriedem | now onto the novaclient change | |
| 19:06:10 | mriedem | cfriesen_: this is what you're waiting on btw https://review.openstack.org/#/c/471081/ | |
| 19:06:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Replace messaging.get_transport with get_rpc_transport https://review.openstack.org/469659 | |
| 19:06:32 | cfriesen_ | mriedem: mriedem: cool, thanks | |
| 19:08:26 | cfriesen_ | mriedem: the empty lines are needed for the nested lists to get formatted properly? is that yaml or rst? | |
| 19:09:00 | mriedem | rst | |
| 19:09:19 | cfriesen_ | is the hyphen a formatting element in rst? | |
| 19:09:26 | mriedem | yeah | |
| 19:10:03 | mriedem | http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#bullet-lists | |
| 19:10:14 | cfriesen_ | got it | |
| 19:10:47 | efried | mikal Should https://review.openstack.org/#/c/4379/ have removed the ", by name (not ID!)" part of https://github.com/openstack/nova/blob/master/nova/virt/driver.py#L152 ? | |
| 19:12:16 | efried | or is there some reason we still need to enforce this lookup as being by name? | |
| 19:12:53 | mriedem | i will enjoy whatever answer mikal gives you for a thing done 5 years ago | |
| 19:13:50 | mriedem | oh, | |
| 19:13:59 | efried | Yeah, anyone should feel free to answer this. | |
| 19:14:00 | mriedem | and if/when he replies, make sure it's related to powervm :) | |
| 19:15:09 | efried | mriedem Since you asked, I was working on filling in a gap we missed in the in-tree impl: the only thing we populate in the InstanceInfo is the state. | |
| 19:15:36 | efried | While looking, I found that docstring and recognized that we've never looked that guy up by name (even in the out-of-tree driver). | |
| 19:15:58 | efried | So I went to figure out whether there was any basis for that edict. | |
| 19:16:29 | mriedem | remove ", by name (not ID!)" from the docstring | |
| 19:16:38 | efried | Doing it. Thanks. | |
| 19:16:38 | mriedem | it's always passed an Instance object | |
| 19:19:47 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement project_id, user_id in PUT /allocations https://review.openstack.org/469634 | |
| 19:19:48 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: use separate tables for projects/users https://review.openstack.org/470645 | |
| 19:19:48 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: Add GET /usages to placement API https://review.openstack.org/469635 | |
| 19:20:04 | jaypipes | zzzeek: \o/ ^ | |
| 19:22:05 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Use util.extract_json in allocations handler https://review.openstack.org/471067 | |