| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-07-14 | |||
| 16:24:47 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: fix unshelve notification test instability https://review.openstack.org/483986 | |
| 16:25:07 | gibi | mriedem: fix for unshelve test is up ^^ | |
| 16:26:27 | mriedem | thanks | |
| 16:26:55 | gibi | mriedem: I'm leaving now for the weekend. See you on Monday | |
| 17:01:39 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Deprecate 'keymap' options https://review.openstack.org/483994 | |
| 17:13:17 | rbartra | Hello stephenfin, I have a question regarding https://review.openstack.org/#/c/449288/19/nova/api/openstack/compute/flavor_manage.py ... I am wondering how to retrieve the current policy value from the verify_deprecated_policy method? I am trying with oslo_policy.policy, but I keep getting an empty rules list. I am also unable to import 'from nova import policy' which is how the current rule is retrieved in flavor_manage.py . | |
| 17:17:01 | stephenfin | rbartra: How come you can't import nova.policy? | |
| 17:17:57 | stephenfin | Ah, wait - that'd be a circular import | |
| 17:18:28 | stephenfin | rbartra: You could just move that function to nova/policy.py instead? | |
| 17:18:30 | rbartra | stephenfin: correct, that is is the issue I am seeing | |
| 17:18:49 | rbartra | stephenfin: yes, I am cool with that as well | |
| 17:19:00 | stephenfin | As am I | |
| 17:19:12 | rbartra | stephenfin: Great! Thank you! | |
| 17:19:19 | stephenfin | Thanks for sticking with that, btw. I realize I've made you jump through a few hoops for it :) | |
| 17:19:24 | stephenfin | but it'll be worth it in the end | |
| 17:19:41 | rbartra | stephenfin: I appreciate all the reviews and feedback :) | |
| 17:49:08 | ralonsoh | stephenfin: hi, do you have time for https://review.openstack.org/#/c/451433? | |
| 17:49:44 | stephenfin | ralonsoh: Not for a while yet. If you want to take it over, by all means go for it | |
| 17:49:59 | ralonsoh | stephenfin: I was going to say that | |
| 17:50:05 | ralonsoh | stephenfin: thanks! | |
| 17:50:11 | stephenfin | Heck, you can abandon that and start from scratch. There probably isn't that much of value there | |
| 17:50:21 | stephenfin | ralonsoh: Cool. Ping me if/when you want reviews :) | |
| 17:50:25 | ralonsoh | stephenfin: I'll start on Monday | |
| 17:50:30 | ralonsoh | stephenfin: perfect | |
| 17:50:53 | stephenfin | (y) | |
| 18:16:56 | mriedem | we should get this in to classify the failure https://review.openstack.org/#/c/483939/ | |
| 18:17:02 | mriedem | i just had something else fail on that same test | |
| 18:21:04 | openstackgerrit | Chris Dent proposed openstack/nova master: Add functional test for two-cell scheduler behaviors https://review.openstack.org/452006 | |
| 18:21:18 | mriedem | good idea | |
| 18:22:16 | cdent | I think that’s my final act for the day | |
| 18:22:24 | mriedem | smoke bomb exit? | |
| 18:22:43 | cdent | poof | |
| 18:35:50 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Add functional test for resize-confirm plus quota validation https://review.openstack.org/483559 | |
| 18:37:29 | mriedem | oh also so everyone knows, | |
| 18:37:39 | mriedem | there is a bug killing a lot of jobs b/c of some eventlet thread switch in cinder-volume | |
| 18:37:42 | mriedem | spiking since 7/11 | |
| 18:37:52 | mriedem | eventlet seems to be the devil | |
| 18:38:22 | smcginnis | Unfortunately not sure where to even start with that one. If anyone has any expertise in that area, would appreciate the help. | |
| 18:38:50 | smcginnis | Searches show it's usually do to either patching or not patching something and multiple threads. | |
| 18:38:53 | mriedem | i'm interested in anything that went into upper-constraints around 7/10 or 7/11 that cinder depends on and which has something to do with locking | |
| 18:39:06 | mriedem | or threading | |
| 18:39:17 | mriedem | i checked the latest privsep release but that looked like mostly docs changes | |
| 18:41:48 | superdan | mriedem: what does the failure look like? | |
| 18:41:50 | mriedem | oslo.service updates went into upper-constraints then too | |
| 18:42:06 | leakypipes | cdent, figleaf, mriedem, superdan: looks like I fixed the issues with the num_instances > 1 tempest test failures in the placement-claims series. | |
| 18:42:09 | mriedem | https://bugs.launchpad.net/cinder/+bug/1694509 | |
| 18:42:10 | openstack | Launchpad bug 1694509 in Cinder "eventlet errors and cannot switch to a different thread" [High,Confirmed] | |
| 18:42:23 | mriedem | http://logs.openstack.org/49/483149/2/check/gate-novaclient-dsvm-functional-neutron-ubuntu-xenial/d37b26f/logs/screen-c-vol.txt.gz#_Jul_13_20_07_49_514341 | |
| 18:42:50 | mriedem | there is also a GMR in this other one http://logs.openstack.org/00/469800/31/check/gate-tempest-dsvm-cells-ubuntu-xenial/7156eed/logs/screen-c-vol.txt.gz | |
| 18:43:02 | smcginnis | Just looked at this one: http://logs.openstack.org/62/468262/4/gate/gate-tempest-dsvm-neutron-linuxbridge-ubuntu-xenial/e3ad57c/logs/screen-c-vol.txt.gz#_Jul_14_17_05_58_531817 | |
| 18:43:07 | mriedem | http://logs.openstack.org/00/469800/31/check/gate-tempest-dsvm-cells-ubuntu-xenial/7156eed/logs/screen-c-vol.txt.gz#_Jul_14_13_59_21_489470 | |
| 18:43:11 | smcginnis | Slightly different signature. | |
| 18:44:14 | mriedem | so, | |
| 18:44:16 | mriedem | cinder uses taskflow, | |
| 18:44:22 | mriedem | and it looks like taskflow uses futursit | |
| 18:44:25 | mriedem | *futurist | |
| 18:44:32 | mriedem | and futurist had an update in upper-constraints on 7/11 | |
| 18:44:57 | smcginnis | Hmm, suspicious. | |
| 18:45:20 | mriedem | https://review.openstack.org/#/c/482273/ | |
| 18:45:59 | mriedem | clarkb: ^ | |
| 18:46:36 | mriedem | maybe we should try blacklisting that and see if things trend back down | |
| 18:46:57 | superdan | sounds plausible to me | |
| 18:47:23 | mriedem | https://github.com/openstack/futurist/compare/1.1.0...1.2.0 | |
| 18:47:35 | smcginnis | mriedem: It might be worth a shot to at least get more data around this and prove or eliminate possibilities. | |
| 18:47:55 | mriedem | the only thing in the 1.2.0 release for futurist is reqs updates https://github.com/openstack/futurist/compare/1.1.0...1.2.0 | |
| 18:48:02 | mriedem | but eventlet is in there | |
| 18:48:20 | mriedem | although i'm not sure that would make a difference | |
| 18:48:22 | mriedem | since we just use u-c | |
| 18:49:38 | mriedem | this module could also be fishy https://github.com/openstack/cinder/blob/master/cinder/coordination.py | |
| 18:49:47 | mriedem | uses eventlet and tooz | |
| 18:49:53 | mriedem | tooz was also updated in u-c on 7/11 | |
| 18:54:27 | mriedem | https://github.com/openstack/tooz/compare/1.57.3...1.57.4 | |
| 18:54:40 | mriedem | https://github.com/openstack/tooz/commit/c94b2a39b3bc484eff6a9c3c9dc094bbfae070c4 | |
| 18:54:41 | mriedem | yay | |
| 18:56:32 | superdan | mriedem: "replace lock code" should be an auto--1 regex | |
| 18:56:44 | superdan | maybe also .*with etcd.* | |
| 18:57:40 | mriedem | and https://github.com/openstack/tooz/commit/7987f4455a7ff4e75d64c46a42d50017f53bfc9e | |
| 18:57:59 | mriedem | superdan: i'm not following you | |
| 18:58:18 | mriedem | however, i did find that this cinder.coordination code that uses tooz and is in the create volume GMR stacktrace, | |
| 18:58:19 | mriedem | http://logs.openstack.org/49/483149/2/check/gate-novaclient-dsvm-functional-neutron-ubuntu-xenial/d37b26f/logs/screen-c-vol.txt.gz#_Jul_13_19_55_02_134315 | |
| 18:58:22 | superdan | mriedem: I'm making a joke that "replace lock code" is something that probably deserves a lot of scrutiny | |
| 18:58:23 | mriedem | is uses the tooz etc3 backend | |
| 18:58:34 | mriedem | no way due throw it in \...q | |
| 18:58:37 | mriedem | it's gnarly | |
| 18:58:40 | mriedem | *dude | |
| 19:04:52 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Deprecate 'keymap' options https://review.openstack.org/483994 | |
| 19:09:42 | openstackgerrit | Octave Orgeron proposed openstack/nova master: Enables MySQL Cluster Support for Nova https://review.openstack.org/446643 | |
| 19:14:12 | mriedem | https://review.openstack.org/484019 | |
| 19:14:13 | mriedem | clarkb: ^ | |
| 19:14:17 | mriedem | dims: ^ | |
| 19:15:26 | dims | mriedem : clarkb : +2 | |
| 19:15:45 | mriedem | ok i have a good e-r query now | |
| 19:16:06 | PTCbro | Hello! I have a question about instance network cache. In neutronv2/api.py I notice _get_instance_nw_info calls _build_network_info_model which queries neutron for the port info. However it also calls _gather_port_ids_and_networks which asks the cache for the current port_ids. If the cache is empty, then _build_network_info_model will ignore the port info it got from neutron. Thus the empty cache forever remains empty | |
| 19:17:31 | mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22%2Fusr%2Flocal%2Flib%2Fpython2.7%2Fdist-packages%2Ftooz%2Fdrivers%2Fetcd3gw.py%3A113%20in%20acquire%5C%22%20AND%20tags%3A%5C%22screen-c-vol.txt%5C%22&from=7d | |
| 19:17:35 | PTCbro | Is there any way to populate an empty cache? | |
| 20:08:34 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add tags to instance.create Notification https://review.openstack.org/459493 | |
| 20:08:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Support tag instances when boot(4/4) https://review.openstack.org/469800 | |
| 20:22:24 | openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova master: [WIP] Getting rid of crud around castellan options https://review.openstack.org/484036 | |
| 20:28:32 | mriedem | \o/ finally got this resize-confirm + quota check novaclient functional test passing https://review.openstack.org/#/c/483559/ | |
| 20:28:41 | mriedem | going to roll a change on top of that to point at the counting quotas series | |