| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-18 | |||
| 12:58:33 | jaypipes | wznoinsk: heh, remember I have 3 pugs that are *always* bothering me :) | |
| 12:58:53 | sahid | jaypipes: ok thanks jay, sorry to rush you on it but i'm little bit scrary with the ff comming | |
| 12:59:19 | jaypipes | sahid: understood. have you found more cores to look at the bottom two patches in that series? sfinucan maybe? | |
| 12:59:43 | sahid | jaypipes: he is in holidays this week | |
| 13:00:13 | jaypipes | sahid: ah. perhaps you can reach out to another core? maybe mriedem or sdague? | |
| 13:00:17 | jaypipes | or bauzas? | |
| 13:01:17 | sahid | jaypipes: yes sure i see dansmith commenting on one of the patch, then i remember johnthetubaguy interested on this feature and perhaps mriedem | |
| 13:01:28 | alex_xu | nova api meeting is running | |
| 13:01:44 | jaypipes | sahid: dansmith is out for a while (he's adventuring in Eastern Oregon in the mountains) :) | |
| 13:02:21 | sahid | oh i see | |
| 13:03:46 | sahid | johnthetubaguy, mriedem one of you guy have a moment to review the feature emulator threads ? it's basically reserving a pCPU to make running emulator threads | |
| 13:04:39 | johnthetubaguy | still recovering from a nasty bug, but starting to get through my review queue, I had hoped to get to that one | |
| 13:07:42 | openstackgerrit | Zhenyu Zheng proposed openstack/nova: Add release note and docs for filter/sort whitelist https://review.openstack.org/421760 | |
| 13:14:03 | snikitin | johnthetubaguy: Hi John! I heard that you are good in Xen. Could you add this xen related patch into your review queue, please? :) https://review.openstack.org/#/c/399613 | |
| 13:49:13 | EmilienM | owalsh: hey! so I have quick updates about Placement service | |
| 13:49:24 | owalsh | EmilienM: shoot | |
| 13:49:28 | EmilienM | owalsh: the service was working fine in our CI (tripleo & puppet) a few days ago | |
| 13:49:32 | openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova: [py35] Fixes to get more tempest tests working https://review.openstack.org/421911 | |
| 13:49:48 | EmilienM | owalsh: and I've noticed yesterday that nova-compute fails to register the resource again | |
| 13:49:49 | openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova: [WIP] ignore me - trying experimental jobs https://review.openstack.org/421209 | |
| 13:50:02 | EmilienM | it tries in loop though, but it doesn't work | |
| 13:50:15 | EmilienM | I tried to reproduce it locally but it worked, I'm a bit confused why it fails in CI | |
| 13:50:20 | EmilienM | I can provide logs | |
| 13:50:30 | EmilienM | http://logs.openstack.org/66/421366/1/check/gate-puppet-openstack-integration-4-scenario004-tempest-centos-7/f138bcc/logs/nova/nova-compute.txt.gz#_2017-01-17_17_45_52_920 | |
| 13:50:49 | openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova: [WIP] ignore me - trying experimental jobs https://review.openstack.org/421209 | |
| 13:50:50 | EmilienM | cdent, bauzas ^ just fyi | |
| 13:51:59 | owalsh | EmilienM: hrmm, so it would suggest a timing issue, but it tries in a loop... for how long? | |
| 13:52:26 | EmilienM | owalsh: I don't think it's a timing issue this time but let me check | |
| 13:52:44 | EmilienM | owalsh: if you remember, the first time I reported this issue was because we didn't have this loop than dansmith did | |
| 13:52:54 | EmilienM | but now we have the loop and it still fails to register | |
| 13:53:39 | markus_z | mriedem: Did I clear up your doubts on https://review.openstack.org/#/c/347471/ ? The test hook behaves as expected IMO. | |
| 13:54:14 | EmilienM | owalsh: I would like to find in logs where it tries to register is and the trace of failure | |
| 13:54:22 | EmilienM | owalsh: "Unable to refresh my resource provider record" is not enough | |
| 13:56:26 | EmilienM | owalsh: I'm creating a launchpad bug | |
| 14:00:03 | EmilienM | owalsh: do you see something weird in the nova.conf? https://paste.fedoraproject.org/529703/14847479/ | |
| 14:00:49 | gmann | johnthetubaguy: alex_xu Kevin_Zheng +1 on putting sort_key one last on series | |
| 14:01:04 | EmilienM | owalsh: https://bugs.launchpad.net/nova/+bug/1657467 | |
| 14:01:04 | openstack | Launchpad bug 1657467 in OpenStack Compute (nova) "placement: unable to refresh compute resource provider record" [Undecided,New] | |
| 14:01:23 | owalsh | EmilienM: auth_urls is IPv6 loopback... does that work??? | |
| 14:01:47 | EmilienM | owalsh: of course, why not? | |
| 14:02:04 | alex_xu | gmann: johnthetubaguy I guess for now, the policy one have more thing to do. maybe the policy one is last one | |
| 14:02:34 | EmilienM | owalsh: it works for all services like tha. | |
| 14:02:38 | EmilienM | that* | |
| 14:03:00 | gmann | alex_xu: but, sort_key ignore list seems controversial which might take time | |
| 14:03:13 | johnthetubaguy | alex_xu: yeah, what gmann said | |
| 14:03:55 | gmann | alex_xu: johnthetubaguy need to go to bed, ll check logs and patches tomorrow. | |
| 14:04:02 | alex_xu | gmann: johnthetubaguy ok, got it, that works for me | |
| 14:04:11 | alex_xu | gmann: see you tomorrow | |
| 14:04:32 | owalsh | EmilienM: just fishing for something that might differ from running locally | |
| 14:09:06 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 map_instances https://review.openstack.org/421404 | |
| 14:09:06 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 map_cell_and_hosts https://review.openstack.org/421420 | |
| 14:09:07 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 verify_instance https://review.openstack.org/421436 | |
| 14:09:07 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 simple_cell_setup https://review.openstack.org/421333 | |
| 14:09:08 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 map_cell0 https://review.openstack.org/421384 | |
| 14:09:08 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 delete_cell https://review.openstack.org/421459 | |
| 14:09:09 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 discover_hosts https://review.openstack.org/421442 | |
| 14:09:09 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 list_cells https://review.openstack.org/421458 | |
| 14:09:10 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 create_cell https://review.openstack.org/421444 | |
| 14:09:30 | eglynn | cdent: are the placement API logs normally found under /logs/nova ? | |
| 14:10:01 | cdent | eglynn: the logs only show up after it has actually had its first request | |
| 14:10:12 | cdent | so if the resource-tracker can't see the service, it will never make a request | |
| 14:10:16 | cdent | and never make a log entry | |
| 14:10:46 | eglynn | a-ha, /me was wondering why none to be found under http://logs.openstack.org/66/421366/1/check/gate-puppet-openstack-integration-4-scenario004-tempest-centos-7/f138bcc/logs/nova | |
| 14:11:13 | cdent | it's one of the side effects of being hosted under mod-wsgi | |
| 14:11:52 | jaypipes | eglynn: heyo, how are you Eoghan? :) | |
| 14:11:55 | takashin | alex_xu: Could you review https://review.openstack.org/#/c/408955/ again? | |
| 14:12:07 | eglynn | jaypipes: all good :) ... and you? | |
| 14:12:11 | EmilienM | eglynn, cdent: yes logs are in /var/log/nova | |
| 14:12:12 | cdent | eglynn, EmilienM : the presenting error "The placement API endpoint not found." suggests either that the rt can't reach keystone, or that keystone is not yet up to date | |
| 14:12:15 | EmilienM | they always were at least | |
| 14:12:24 | jaypipes | eglynn: busy as always, which is good :) | |
| 14:12:28 | EmilienM | cdent: it worked 2 days ago :( | |
| 14:12:31 | eglynn | indeed | |
| 14:12:35 | EmilienM | cdent: I can show you logs, a sec | |
| 14:13:33 | EmilienM | cdent: here are logs from 5 days ago: | |
| 14:13:35 | EmilienM | http://logs.openstack.org/66/420166/2/check/gate-puppet-openstack-integration-4-scenario004-tempest-centos-7/591bf96/logs/nova/ | |
| 14:13:43 | EmilienM | you can see nova placement api log, and everything works | |
| 14:14:11 | EmilienM | we haven't changed anything on this topic https://github.com/openstack/puppet-nova/commits/master | |
| 14:15:18 | cdent | EmilienM: http://logs.openstack.org/66/420166/2/check/gate-puppet-openstack-integration-4-scenario004-tempest-centos-7/591bf96/logs/nova/nova-compute.txt.gz#_2017-01-13_21_20_29_245 | |
| 14:16:00 | EmilienM | cdent: credentials are created by Puppet, we never changed that since December | |
| 14:16:12 | EmilienM | let me show the logs | |
| 14:16:14 | cdent | I don't dispute that | |
| 14:16:31 | cdent | all I'm saying is that the logs you pointed to me as being successful are saying they have bad credentials | |
| 14:16:35 | EmilienM | http://logs.openstack.org/66/420166/2/check/gate-puppet-openstack-integration-4-scenario004-tempest-centos-7/591bf96/logs/puppet-20170113_212224.txt.gz#_2017-01-13_21_21_14 | |
| 14:16:44 | cdent | I don't know if that means anything or not | |
| 14:16:54 | EmilienM | they are created at 21:21:14 | |
| 14:17:02 | EmilienM | and your log is at 21:20 | |
| 14:17:25 | EmilienM | if you look later, it worked: http://logs.openstack.org/66/420166/2/check/gate-puppet-openstack-integration-4-scenario004-tempest-centos-7/591bf96/logs/nova/nova-compute.txt.gz#_2017-01-13_21_21_32_490 | |
| 14:17:33 | EmilienM | so like I said, it worked on January 13th | |
| 14:17:36 | EmilienM | but not now | |
| 14:17:48 | cdent | I get it, I'm just trying to clarify the pieces | |
| 14:18:08 | EmilienM | something changed in Nova between January 13th and at least 17th (I saw the thing yesterday) | |
| 14:19:31 | EmilienM | cdent: maybe https://github.com/openstack/nova/commit/c17772e1f202c9ffb651adc3299a1990c35766f3 ? | |
| 14:19:59 | openstackgerrit | Zhenyu Zheng proposed openstack/nova: Strict pattern match query parameters https://review.openstack.org/420494 | |
| 14:20:29 | cdent | EmilienM no impact on the presence of the service. I think the problem is most likely on the keystone side, give me a chance to read through the logs more closely | |
| 14:20:58 | EmilienM | cdent: sure, and thanks for helping | |
| 14:21:57 | eglynn | the timing of the credentials creation shouldn't be a show-stopper, right, as the report will be retried since https://review.openstack.org/#/c/418590/4/nova/scheduler/client/report.py landed on Jan 11th | |
| 14:22:48 | Kevin_Zheng | johnthetubaguy: https://review.openstack.org/420494 rebased to the fitler whitelist patch, so it will be the 2nd of the series, please have a check when you got a sec, hope we can merge the 1st and 2nd of the series, I will keep working on the policy one and sort one torrmorow :) | |
| 14:26:39 | pumaranikar | cdent, again checking https://review.openstack.org/#/c/395194/ json formatter for placements. I am wondering if we can use @wsgify.middleware decorator to alter webob response to check error codes. | |
| 14:27:44 | pumaranikar | cdent, not sure about it. Any suggestions on it ? | |