Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-06-05
16:34:19 mriedem that's different from the update_instance_info stuff cast to the scheduler
16:34:24 cdent thanks jaypipes, I'm going home, will look from there, where the lack of people ought to allow proper attention
16:34:29 mriedem ^ is all about server group affinity/anti-affinity filters
16:34:30 jaypipes kk
16:35:13 dansmith jaypipes: can I reset colors on that so we can see only deltas from this?
16:35:21 cdent dansmith++
16:35:22 jaypipes dansmith: go for it.
16:35:25 cfriesen_ there are so many races around server group affinity/antiaffinity it's not even funny
16:37:36 cfriesen_ jaypipes: is it possible to have floating point values for resources?
16:37:45 cfriesen_ or is it integer only?
16:37:59 jaypipes cfriesen_: integer only.
16:39:03 cfriesen_ jaypipes: we currently allow a mix of instances with shared/dedicated cpus on the same compute node (with cpusets to keep the affinity straight) and so report usage as floating point. I guess we'll move to fixed point with placement.
16:43:35 edleafe jaypipes: running out for a bit, so I'll go more in depth when I get back.
16:44:27 sdague mriedem: https://review.openstack.org/#/c/467242/ is the glance request_id change ready to go in
16:46:31 jaypipes cfriesen_: what do you mean "report usage as floating point"?
16:51:51 jaypipes dansmith, mriedem: the three of us are having a very Reggae-colored conversation on this etherpad.
16:52:35 mriedem wish i had some ganja
16:53:39 jaypipes mriedem: if you lived in Sarasota...
16:56:12 cfriesen_ jaypipes: we report the CPU usage of a compute node as sum(dedicated vcpus) + sum(shared vcpus)/cpu_allocation_ratio. The total cannot be more than the available CPUs on the compute node.
16:57:27 cfriesen_ jaypipes: with integer usage we'll just have to multiply everything by cpu_allocation_ratio
16:57:36 jaypipes cfriesen_: yup.
17:04:04 cfriesen_ who's responsible for releasenotes issues? My commit https://review.openstack.org/#/c/435141 keeps failing jenkins because it chokes on an older releasenotes file. Something seems to have gotten more strict recently.
17:04:15 mriedem sdague: +2
17:05:22 mriedem cfriesen_: open a novaclient bug
17:06:56 mriedem my guess would be something with sphinx 1.6.1 or 1.6.2
17:07:01 mriedem 1.6.1 is blacklisted but 1.6.2 isn't
17:07:08 cfriesen_ mriedem: okay
17:08:22 mriedem jaypipes: we should have a separate bp and spec for this new placement api thing, the claims in the scheduler bp will depend on it
17:08:42 mriedem but i assume you want to communicate it to the ML so it sinks in a bit first
17:12:13 sdague cfriesen_: it might be worth checking with dhellman or the like
17:12:18 mriedem i know the fix
17:12:20 mriedem just need the bug
17:12:24 mriedem testing locally
17:13:02 mriedem not sure why it passed before though
17:13:54 openstack Launchpad bug 1695939 in python-novaclient "jenkins failing on older releasenote file" [Undecided,New]
17:13:54 cfriesen_ mriedem: https://bugs.launchpad.net/python-novaclient/+bug/1695939
17:14:46 cfriesen_ mriedem: I tried locally modifying test-requirements.txt but it still installed sphinx 1.6.2. odd
17:14:57 mriedem the release note itself is bad
17:15:14 cfriesen_ mriedem: yeah, I tried fixing it but couldn't figure out how to get it to pass
17:15:15 mriedem 1. there is no 'update' section
17:15:21 mriedem 2. it's not formatted for rst
17:15:59 sdague dansmith / melwitt: can either / both of you look at this devstack fix for cells that was proposed - https://review.openstack.org/#/c/470598
17:16:07 jaypipes mriedem: dansmith and I are on a hangout talking about something related... we're paying attention. give us a bit :)
17:16:09 cfriesen_ mriedem: for 2, do all the single dashes become asterisks?
17:16:19 mriedem cfriesen_: i'm fixing
17:25:50 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Use util.extract_json in allocations handler https://review.openstack.org/471067
17:33:12 mriedem wtf i guess this is more complicated than i thought - i've made changes, committed them, rebuilt and it still picks up the old thing - probably because it's processing an old commit
17:33:18 mriedem and i can't re-write history
17:33:28 mriedem but reno should realize i'm changing something
17:39:39 cfriesen_ mriedem: bug in the CI?
17:40:17 mriedem might be a bug in reno, i'm talking to dhellmann about it
17:41:20 jaypipes OMG I want to friggin shoot myself in the head right now.
17:41:29 jaypipes I fucking hate SQL migrations.
17:41:42 dansmith sdague: done
17:41:42 jaypipes zzzeek: you around? could really use your help :(
17:43:58 zzzeek jaypipes: hey
17:44:27 jaypipes zzzeek: hi Mike. got a few minutes to help me with some very frustrating SQL migrations issues, pretty please?
17:44:31 zzzeek yes
17:44:45 sdague dansmith: thanks, it looked weird to me. Good to get the real expert view in there
17:44:52 jaypipes zzzeek: thanks :) it's about https://review.openstack.org/#/c/470645/
17:45:19 jaypipes zzzeek: I've tried a bunch of permutations (see revisions on the 044_placement_add_projects_users.py file
17:45:35 jaypipes zzzeek: and I'll fix PG and break SQLite/MySQL, and fix MySQL and fix PG.
17:45:43 jaypipes zzzeek: going round and round in circles :(
17:45:53 jaypipes zzzeek: http://logs.openstack.org/45/470645/4/check/gate-nova-tox-functional-ubuntu-xenial/0bf172a/testr_results.html.gz
17:46:15 zzzeek *obligatory grr sqlalchemy-migrate is ancient crap* irc message
17:46:23 zzzeek OK so onto the acvtual problem
17:46:29 cfriesen_ dansmith: If I'm reading this right, "nova-manage db online_data_migrations" returns 0 if we're successfully done all the migrations or if there were errors and we bailed out. This seems like a strange design.
17:46:29 jaypipes zzzeek: essentially, I first tried just doing an ALTER TABLE MODIFY COLUMN by changing the project_id and user_id fields in the consumers table from VARCHAR to INT.
17:46:43 jaypipes zzzeek: using a col.alter(...) command.
17:47:00 jaypipes zzzeek: PG hated that and requires me to drop and recreate the columns.
17:47:36 zzzeek jaypipes: it does? usually it just wants you to use a CAST so it can rewrite the data ...oh there'sa syntax for that hold on ... :)
17:47:48 jaypipes zzzeek: so I did that, then MySQL complains about indexes either existing or not existing depending on whether I drop the columns before or after dropping indexes. and when I drop indexes, PG complains there's no such indexes.
17:48:34 zzzeek jaypipes: what is the "PG hated that" error you are getting ?
17:48:42 jaypipes one sec, grabbing.
17:49:33 zzzeek jaypipes: also, your problem might be "migrate_engine.name == 'postgres'", sqlalchemy calls it "postgresql"
17:50:17 jaypipes zzzeek: no, that definitely runs only on postgresql AFAICT... it's the name of the db driver in sqlalchemy-migrate :(
17:50:29 zzzeek jaypipes: weird.
17:50:54 zzzeek jaypipes: so....why not put the whole drop/create thing inside of "if name == 'postgres'" and just use normal ALTER for the others ?
17:51:32 jaypipes zzzeek: I think I've tried that and it failed something about models being out of sync...
17:51:42 zzzeek keeping in mind that PG might just want you to say "ALTER COLUMN...USING", which is pg-specific anyway so you still need the conditional
17:51:54 jaypipes zzzeek: unfortunately, CI doesn't seem to want to allow me to see past test run failures :(
17:52:26 jaypipes zzzeek: how would I do the ALTER COLUMN ... USING thing if using sqlalchemy-migrate's col.alter() thing?
17:52:35 zzzeek jaypipes: that doesn't make any sense. program goes into the migration, and comes out, then it verifies things. if the migration works, then it worked, whether or not it did ALTER or drop/create
17:52:49 zzzeek jaypipes: you'd have to do migrate_engine.execute("ALTER TABLE...")
17:53:01 jaypipes k
17:53:12 jaypipes sdague: is there a way to view old CI test runs?
17:53:23 sdague jaypipes: how old?
17:53:25 zzzeek jaypipes: if that's the issue. not sure if PG has done things w/ the indexes that are more of a big deal to change
17:53:28 jaypipes sdague: a day
17:53:46 sdague yeh, unhide the CI and look through the jenkins comments
17:53:53 jaypipes sdague: oh, duh, thx
17:53:57 sdague the rollup of the last run is just a convenience function
17:54:36 jaypipes zzzeek: ok, here's the failure from the original alter: http://logs.openstack.org/45/470645/1/check/gate-nova-tox-functional-ubuntu-xenial/8180560/testr_results.html.gz
17:55:26 jaypipes zzzeek: that happened when I did the original here: https://review.openstack.org/#/c/470645/1/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py
17:55:58 zzzeek jaypipes: ha! HINT: You might need to specify "USING project_id::integer". me wants another darth vader usb key
17:56:29 jaypipes zzzeek: yeah... so instead of that, I did this: https://review.openstack.org/#/c/470645/2/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/044_placement_add_projects_users.py
17:56:42 dansmith mriedem: so this passed the live migration job. Do you still want me to change it or what? https://review.openstack.org/#/c/458537/31/nova/tests/live_migration/hooks/ceph.sh
17:56:52 jaypipes zzzeek: which failed with this: http://logs.openstack.org/45/470645/2/check/gate-nova-tox-functional-ubuntu-xenial/f94cf09/testr_results.html.gz
17:57:51 zzzeek jaypipes: OK, so theres the most ancient issue in SQLA to allow index.drop(checkfirst=True), however, I can get you through that anyway, if you want to go that route. but still, "if engine != 'postgres'", then drop index
17:58:14 zzzeek jaypipes: but still, "if engine == 'postgres': migrate_engine.execute("ALTER.... USING")"

Earlier   Later