Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-05-01
21:44:04 mriedem message:"UnicodeDecodeError" AND message:"instance_update_at_top" AND tags:"screen-n-cell-region.txt"
21:45:58 edleafe cdent: I was meaning to mark it WIP and forgot. I knew it wasn't fully tested.
21:46:15 melwitt mtreinish, mriedem: so they're set to run serially ... the thing I see is e.g. (u'snapshots_tempest-QosSpecsTestJSON-volume-type-1219921907', -1) not in [(u'gigabytes', 1001), (u'backups', 11), (u'snapshots_lvmdriver-1', -1), (u'volumes_lvmdriver-1', -1), (u'volumes', 11), (u'per_volume_gigabytes', -1), (u'backup_gigabytes', 1001), (u'gigabytes_lvmdriver-1', -1), (u'snapshots', 11)]
21:46:21 cdent edleafe: my bad really, I was being far too optimistic
21:47:29 melwitt mtreinish, mriedem: as though some other test had added a quota default and it gets picked up unexpectedly in another
21:48:06 cdent g'night all
21:50:39 mtreinish melwitt: do you have a link to that failure?
21:51:03 melwitt mtreinish: yeah, sorry I didn't include it http://logs.openstack.org/39/446239/5/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/0b6acd6/logs/testr_results.html.gz
21:57:11 mtreinish melwitt: that's really weird. It looks like tempest sends a PUT with the new default quota entries, cinder returns 200, and the tempest does a GET to list the default quotas and the stuff it just added isn't there
21:58:41 mtreinish melwitt: oh, I think you might be right about a race
21:58:44 mriedem dansmith: while looking at this https://github.com/openstack/nova/blob/master/nova/compute/api.py#L4508 for using uuid rather than id, i realized this could have been a direct lookup if we stored the compute node uuid on the host mapping, but we only store the ComputeNode.host field...
21:59:15 melwitt mtreinish: hm, what I thought is that update_body is returning an additional unexpected item that was added by a different test
22:02:37 mtreinish melwitt: the test does a GET to get all the default quotas, loops over that list increases each quota there by 1 with a big update, then does another list and loops over them to see everything was updated
22:03:26 mtreinish the failure looks like another test class is making a default quota set and cleaning up after itself between the first get and after the update
22:03:48 mtreinish and tempest is failing on the verification that everything it asked to be updated is
22:03:53 mtreinish because a bunch of stuff was deleted
22:04:20 mtreinish at least that's what it looks like
22:04:41 melwitt okay. I'm looking at the code too but highly possible I've not understood it :)
22:05:26 mtreinish I'm having a hard time figuring out where the other default quotas are coming from though. It says volumes_tempest-QosSpecsTestJSON-volume-type-1219921907 which should mean the QosSpecsTestJSON class created it
22:05:34 mtreinish but I don't see where that happens in the code
22:05:39 mtreinish but I could just not be seeing it
22:07:53 melwitt I was focusing on the part where it's asserting (new, old) and I thought it mean there was something unexpected in "new", the return from the PUT
22:08:49 melwitt maybe I got that backward because I'm not familiar with the ContainsAll matcher
22:11:29 mtreinish melwitt: iirc the contains all matcher is basically a for loop that does: for x in arg[0]: self.assertIn(x, arg[1])
22:11:36 mtreinish but I'd have to dig up the code to be sure
22:12:11 melwitt agreed that it should mean the QosSpecsTestJSON created it but all I see is e.g. "cls.create_test_qos_specs(...)" and I dunno what all that entails. if it could possibly include creating a quota default
22:12:33 mtreinish melwitt: but looking at the tempest log: http://logs.openstack.org/39/446239/5/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/0b6acd6/logs/tempest.txt.gz#_2017-04-29_03_26_53_000
22:12:54 mtreinish the teardownclass is deleting volume types around the same time the default quotas tests are doing their thing
22:13:05 mtreinish so I think we want to add an external lock between the classes to protect against that
22:16:15 mtreinish melwitt: can you open a bug for this so we can track it with e-r? I can work on a fix for it. It shouldn't be too hard
22:16:47 melwitt mtreinish: sure
22:17:14 mtreinish unless you wanted to work on the patch :)
22:20:38 melwitt mtreinish: no thanks :)
22:22:51 dansmith mriedem: you mean we could have known which cells to check instead of iterating over all of them?
22:23:16 dansmith mriedem: the id isn't unique, so it only buys us not iterating over all the cells in the case where the compute id happens to be uniquer
22:23:28 dansmith but on average, for similar-sized cells, you'll have nearly 100% overlap
22:25:02 openstackgerrit Ed Leafe proposed openstack/nova master: Remove the can_host column https://review.openstack.org/461494
22:28:57 openstack Launchpad bug 1687488 in tempest "VolumeQuotaClassesTest.test_update_default_quota fails intermittently" [Undecided,New]
22:28:57 melwitt mtreinish: https://bugs.launchpad.net/tempest/+bug/1687488
22:29:09 mtreinish melwitt: thanks
22:31:53 mriedem dansmith: right i meant we could just take the compute node uuid to look up a host mapping to find out the cell it's in, but right now we'll have to iterate the cells (like we do if given the primary key id)
22:32:11 mriedem not a huge deal, just something i thought of when seeing your comment in that host api method
22:32:35 dansmith you mean once we have the uuid from the api, then we can use the uuid for the lookup, right?
22:32:42 mriedem yes
22:32:50 dansmith or do you mean if we had uuid on the hostmapping we could avoid the cell scan?
22:32:54 mriedem that's what i'm adding to HostAPI.compute_node_get now, but we still have to iterate the cells
22:32:58 mriedem correct
22:33:22 mriedem dansmith: https://gist.github.com/mriedem/013d1bcb9fd547a61adcccb99b9f0eae
22:33:29 dansmith okay, since hostmapping is 1:1 to compute node that should be okay, I think.. but we don't want to do the same for service I think
22:33:43 mriedem we don't have a ServiceMapping in the api db
22:33:46 mriedem so we can't
22:34:06 dansmith what I meant was, we can't just tack the service uuid onto hostmapping too
22:34:15 mriedem oh, ok yeah agree
22:34:27 mriedem if we did add uuid to hostmappings, we'd have to have an online data migration
22:34:32 mriedem so it becomes less fun
22:34:36 dansmith yeah ;)
22:35:15 mriedem i bet at some point this will be one of those things we wish we had though, like bdm.uuid
22:35:23 mriedem but if we really need it at some point, we could add it later
22:41:00 openstackgerrit Eric Fried proposed openstack/nova master: nova.utils.get_service_url() https://review.openstack.org/458257
23:22:25 efried mordred jaypipes johnthetubaguy FYI ^^ is failing Jenkins because it Depends-On a keystoneauth patch which apparently isn't get picked up. (Are there only certain projects Depends-On works with?)
23:22:43 openstackgerrit Matt Riedemann proposed openstack/nova master: Handle uuid in HostAPI.compute_node_get https://review.openstack.org/461532
23:23:51 mriedem efried: depends-on doesn't work if the dependent change is in a library or client,
23:23:58 mriedem unless you have a job that uses LIBS_FROM_GIT
23:24:01 mriedem in devstack
23:24:03 efried mriedem Okay, good to know.
23:24:22 efried FWIW, it live-tests okay ;-)
23:27:05 cfriesen_ for testing purposes is there a helper function to compare dicts where one outputs keys/values as strings and the other as unicode?
23:29:24 efried cfriesen_ map()?
23:30:36 efried Or this? http://stackoverflow.com/questions/1254454/fastest-way-to-convert-a-dicts-keys-values-from-unicode-to-str
23:31:57 cfriesen_ efried: I was wondering if nova (or oslo) already had something
23:33:18 openstackgerrit Takashi NATSUME proposed openstack/nova master: Avoid forcing translation on logging calls https://review.openstack.org/413876
23:45:25 viks hi all, seeking attention on Nova libvirt volume driver for Veritas HyperScale at https://review.openstack.org/#/c/443951/ . Pre-req of OS-brick has already been merged(https://review.openstack.org/#/c/442754/ , https://review.openstack.org/#/c/458688/)
23:48:02 mordred efried: sweet. I was travelling all today but will try to connect with keystone folks tomorrow to follow up on that
#openstack-nova - 2017-05-02
00:30:58 openstackgerrit Jay Pipes proposed openstack/nova master: placement: shared resources when finding providers https://review.openstack.org/461574
00:31:23 jaypipes melwitt, dansmith, edleafe, mriedem: ^^ shared providers implemented.
00:31:54 jaypipes alex_xu: ^
00:33:21 jaypipes edleafe: hopefully you now see why I was trying to cut this patch down in size :)
00:35:25 openstackgerrit Jay Pipes proposed openstack/nova master: placement: shared resources when finding providers https://review.openstack.org/461574
00:36:55 openstackgerrit Jay Pipes proposed openstack/nova master: placement: shared resources when finding providers https://review.openstack.org/461574
00:49:19 mriedem oof
00:49:44 mriedem jaypipes: don't we need an os-traits release to use the new misc shared aggregate trait?
00:51:25 mriedem ah i see the todo now
00:51:39 mriedem are we ready to do that release then?
00:55:45 mriedem hmm, live migration job is blowing up
00:56:01 mriedem tdurakov: have you seen this? http://logs.openstack.org/48/461448/1/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/a3f4426/logs/screen-n-cpu.txt.gz?level=TRACE#_May_01_22_15_30_289875
00:56:58 edleafe jaypipes: thanks, will review tomorrow
02:36:19 openstackgerrit jichenjc proposed openstack/nova master: Add description for 410 http handle https://review.openstack.org/461594
03:11:43 openstackgerrit Alex Xu proposed openstack/nova master: Add description to policies in extended_az and extend_ser_attrs https://review.openstack.org/461600
03:11:44 openstackgerrit Alex Xu proposed openstack/nova master: Add description to policies in extended_status and extended_volumes https://review.openstack.org/461601
03:23:34 openstackgerrit Alex Xu proposed openstack/nova master: Add more description to policies in the keypairs.py https://review.openstack.org/461603
03:47:08 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Support tag instances when boot(3/4) https://review.openstack.org/459593
03:54:15 openstackgerrit Jay Pipes proposed openstack/nova master: placement: shared resources when finding providers https://review.openstack.org/461574
04:37:42 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for os-certificates endpoint instead of stevedore https://review.openstack.org/460813
05:15:26 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for hosts endpoint instead of stevedore https://review.openstack.org/459231
05:15:26 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for limits endpoint instead of stevedore https://review.openstack.org/458805
05:15:27 openstackgerrit jichenjc proposed openstack/nova master: use plain routes list for hypervisor endpoint instead of stevedore https://review.openstack.org/459235
05:21:44 openstackgerrit Alex Xu proposed openstack/nova master: Add more description to policies in the keypairs.py https://review.openstack.org/461603
05:21:45 openstackgerrit Alex Xu proposed openstack/nova master: Add description to policies in extended_status and extended_volumes https://review.openstack.org/461601
05:23:12 openstackgerrit Alex Xu proposed openstack/nova master: Use plain routes list for os-interface endpoint instead of stevedore https://review.openstack.org/459979
05:24:12 openstackgerrit Takashi NATSUME proposed openstack/nova master: Specify keymap on server boot (1/2) https://review.openstack.org/458752

Earlier   Later