| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-26 | |||
| 18:00:51 | mriedem | the person in the bug report said that nova hard-coding /v3/ in the url prefix is wrong | |
| 18:00:52 | lbragstad | mriedem: i'm digging into some stable maintenance things for keystone at the moment, but i'll look into that next | |
| 18:00:53 | openstackgerrit | Sean Dague proposed openstack/nova master: Migrate to oslo request_id middleware - mv 2.46 https://review.openstack.org/467998 | |
| 18:01:12 | sdague | right, this is the versioned vs. unversioned keystone endpoint issue | |
| 18:01:22 | mriedem | ok - only thing that gives me pause is the v3 api-ref uses /v3/ too https://developer.openstack.org/api-ref/identity/v3/#show-project-details | |
| 18:01:31 | sdague | https://review.openstack.org/#/c/438049/ | |
| 18:01:58 | sdague | mriedem: so that's the result they will get if they made their identity url use the versioned one | |
| 18:02:21 | sdague | it's the same thing tripleo hit | |
| 18:03:18 | sdague | there is other keystoneauth magic we could lean on here | |
| 18:03:30 | sdague | the half baked patch I had died on the vine | |
| 18:04:38 | sdague | actually, I think https://review.openstack.org/#/c/438049/6/nova/api/openstack/identity.py might be the fix for them as well | |
| 18:04:54 | sdague | I didn't like it at the time because of how much magic hiding of issues it does | |
| 18:04:55 | mriedem | hit kolla too https://review.openstack.org/#/c/463500/ | |
| 18:06:28 | sdague | yeh. | |
| 18:06:33 | mriedem | so with that patch of yours, | |
| 18:06:49 | mriedem | they go from a 400 (because of endpoint not found) to a 404 (Because endpoint not found) | |
| 18:06:55 | mriedem | but with a helpful message | |
| 18:07:18 | sdague | actually, they wouldn't get a 404 | |
| 18:07:19 | mriedem | or does ksa actually do magic to find the right v3 endpoint? | |
| 18:07:25 | sdague | yep, it does | |
| 18:07:30 | mriedem | oh well that's neat | |
| 18:07:38 | sdague | that's the magic I wasn't all that comfortable with | |
| 18:07:43 | mriedem | oh | |
| 18:07:47 | mriedem | well | |
| 18:07:53 | sdague | but, honestly, it's probably the right UX for operators | |
| 18:07:59 | mriedem | yeah, plus upgrade impact isn't fun | |
| 18:08:03 | sdague | yep | |
| 18:08:35 | sdague | ok, so how about the following deal. You look at mv 2.46 again, and I'll fix this patch | |
| 18:08:42 | sdague | which we'll probably want to backport | |
| 18:08:48 | openstackgerrit | John Griffith proposed openstack/nova master: cinder: add attachment_create method https://review.openstack.org/459111 | |
| 18:08:49 | openstackgerrit | John Griffith proposed openstack/nova master: cinder: add attachment_update method https://review.openstack.org/459113 | |
| 18:08:49 | openstackgerrit | John Griffith proposed openstack/nova master: WIP/POC Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 18:09:16 | mriedem | sdague: ok | |
| 18:18:44 | mriedem1 | superdan: want to update the link in the channel topic to https://wiki.openstack.org/wiki/Nova/Pike_Release_Schedule ? | |
| 18:18:58 | superdan | oh alright | |
| 18:24:37 | mriedem1 | 2.46 reviewed | |
| 18:25:49 | openstackgerrit | John Griffith proposed openstack/nova master: WIP/POC Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 18:28:55 | lbragstad | mriedem: so you don't need anything from a ksa perspective, just double checking | |
| 18:31:21 | mriedem | lbragstad: sean is on it | |
| 18:31:22 | mriedem | thanks | |
| 18:32:49 | mriedem | leakypipes: figleaf: cdent: is there anything that merged recently which might be causing this? http://logs.openstack.org/32/468232/1/gate/gate-tempest-dsvm-cells-ubuntu-xenial/ad969ba/logs/screen-n-cpu.txt.gz?level=TRACE#_May_26_17_29_52_292260 | |
| 18:32:58 | mriedem | looks like a race to cleanup allocations in the resource tracker when an instance is deleted | |
| 18:33:04 | mriedem | https://bugs.launchpad.net/nova/+bug/1693899 | |
| 18:33:04 | openstack | Launchpad bug 1693899 in OpenStack Compute (nova) "Cleaning up stale resource allocations can fail with InstanceNotFound" [Undecided,New] | |
| 18:33:08 | mriedem | just started in the last 24 hours or so | |
| 18:33:32 | mriedem | https://github.com/openstack/nova/commit/439f67e7bc795f171af8248400c3f3e17a0bbe7e ? | |
| 18:33:43 | mriedem | yup | |
| 18:36:24 | artom | mriedem, the InstanceNotFound test thing - better to squash the two patches into one in the backport, or leave separate? | |
| 18:36:56 | mriedem | artom: gonna be hard to separate them since it's writing a test for something already fixed on master | |
| 18:37:27 | artom | mriedem, it could be fix first then test in the top patch. But yeah, squash makes more sense | |
| 18:37:34 | mriedem | unless you mean the test backport comes after the fix backport on stable too | |
| 18:37:38 | mriedem | ok | |
| 18:37:41 | mriedem | either way | |
| 18:38:34 | mriedem | cdent: figleaf: anyone want to pick that up? seems like just needs to handle that InstanceNotFound in RT, but i didn't review that original patch | |
| 18:38:53 | cdent | mriedem: i can give it a try, that's what I was thinking too | |
| 18:39:07 | artom | I noticed lyarwood proposed the fix for ocata too - if you're still around lyarwood, mind if I jump in in your patch and add the test? | |
| 18:39:36 | mriedem | why is this a warning? https://review.openstack.org/#/c/460177/12/nova/compute/resource_tracker.py@1013 | |
| 18:40:01 | mriedem | because the RT shouldn't be cleaning up allocations when the scheduler/conductor is? | |
| 18:41:31 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Test InstanceNotFound handling in 'nova usage' https://review.openstack.org/468514 | |
| 18:41:50 | cdent | warning seems wrong to me. | |
| 18:42:29 | mriedem | "WARNING I AM DOING THAT THING YOU ASKED ME TO DO" | |
| 18:42:55 | cdent | i'll change that too? | |
| 18:43:02 | mriedem | separate | |
| 18:43:09 | mriedem | but wow http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Deleting%20stale%20allocation%20for%20instance%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d | |
| 18:44:05 | mriedem | cdent: it was latent https://review.openstack.org/#/c/460177/12/nova/scheduler/client/report.py@a929 | |
| 18:44:10 | figleaf | cdent: sorry, was heads down and just saw the above. You got it? | |
| 18:44:10 | mriedem | and if that's in ocata we'll want to backport that | |
| 18:44:59 | cdent | figleaf: yeah, you carry on on your stuff, should be fine | |
| 18:45:01 | mriedem | crap it's all the way back to newton https://github.com/openstack/nova/commit/9374566d194be1e861e7d08dd0e3b2a615ff408c | |
| 18:45:08 | mriedem | cdent: i'll open a separate bug for the warning | |
| 18:45:14 | cdent | ✔ | |
| 18:47:35 | mriedem | https://bugs.launchpad.net/nova/+bug/1693903 | |
| 18:47:35 | openstack | Launchpad bug 1693903 in OpenStack Compute (nova) "Excessive "Deleting stale allocation for instance" warnings in nova-compute logs" [Undecided,New] | |
| 18:47:39 | openstackgerrit | Sean Dague proposed openstack/nova master: Be more tolerant of keystone catalog configuration https://review.openstack.org/438049 | |
| 18:47:58 | sdague | lbragstad / mriedem ^^^ | |
| 18:58:19 | mriedem | sdague: commented. some issues in the release note | |
| 18:58:38 | mriedem | i thought the validation in the quota sets api happened in ocata so i was wrong there, we probably do want the new note under the upgrade section | |
| 18:58:43 | mriedem | but let's do it in a separate new release note please | |
| 19:00:32 | sdague | mriedem: ok, I'm trying to understand what you wanted as release notes | |
| 19:00:34 | sdague | because in the last patch you said make it a dedicated one | |
| 19:01:04 | sdague | oh, that it shows up so much later that we just need a whole other file? | |
| 19:01:27 | sdague | walk me through your ideal world here, I'll make it :) | |
| 19:01:44 | sdague | also, irc client crash, so my back buffer is gone | |
| 19:02:04 | mtreinish | sdague: I want world peace | |
| 19:04:32 | mriedem | sdague: the release note you're updating is already in a tagged release, | |
| 19:04:36 | mriedem | so it should be a separate file | |
| 19:04:43 | mriedem | tox -e venv -- reno new <something> | |
| 19:05:11 | openstackgerrit | John Griffith proposed openstack/nova master: WIP/POC Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 19:05:15 | mriedem | sdague: i was also confused because i thought we had this validation in the quotas api in ocata, | |
| 19:05:18 | mriedem | but we don't, | |
| 19:05:41 | sdague | ok, separate file is fine | |
| 19:05:44 | mriedem | so i think you just need to add a new release note with an upgrade impact that notes the thing about not being able to find keystone v3 will result in a 400 for these apis now | |
| 19:06:04 | sdague | I don't think this is an upgrade impact | |
| 19:06:09 | sdague | I actually think it mitigates one | |
| 19:06:15 | mriedem | if ksa does the magic to find v3 and the only issue is if v3 isn't there, then maybe the upgrade impact is just, 'nova requires keystone v3 now' | |
| 19:06:37 | sdague | the previous patch actually required unversioned keystone endpoint | |
| 19:06:44 | mriedem | it's an upgrade impact if you don't have keystone v3 right? | |
| 19:06:51 | sdague | sure | |
| 19:07:00 | lbragstad | yeah - i was going to say something like that in the comment here - https://review.openstack.org/#/c/438049/7/nova/api/openstack/identity.py,unified | |
| 19:07:00 | mriedem | which i dont think we required before this | |