| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-14 | |||
| 15:53:37 | edmondsw | sdague no, because of what I just added | |
| 15:53:54 | edleafe | edmondsw: right, the general pattern for 401 is to retry once, and if that fails, give up | |
| 15:54:11 | edleafe | (retry auth, not the original call) | |
| 15:54:20 | edmondsw | edleafe right | |
| 15:54:21 | sdague | edleafe: can you point me at 2 examples of that being the pattern in existing software | |
| 15:54:45 | edmondsw | sdague we use to have examples of that all over the place. A lot of them have now moved into keystoneauth | |
| 15:54:51 | edleafe | sdague: in SDK land it is very common | |
| 15:55:02 | sdague | edleafe: ok, 2 references with git urls please | |
| 15:55:34 | sdague | I also think the real fix here is the service role so this doesn't happen | |
| 15:55:36 | bauzas | I'm not an API specialist, but say I get a 400, I know I fscked somewhere | |
| 15:55:59 | bauzas | if I'm getting a 401, I wonder why my creds are wrong | |
| 15:56:09 | sdague | if it is claimed that this is common behavior, that's fine, I'm happy to accept them with references in code showing that it is | |
| 15:56:30 | bauzas | so maybe it's not clear that a 401 could mean something "please retry" so I do understand sdague | |
| 15:57:05 | bauzas | in general, 401 would mean full stop to me | |
| 15:57:15 | bauzas | and me checking if my creds are good | |
| 15:57:25 | sdague | the neutron interaction pattern is confusing because we sometimes use the admin in disk creds, and sometimes the user token | |
| 15:58:35 | edmondsw | sdague one e.g. that should no longer be necessary because the logic has moved into keystoneauth is https://review.openstack.org/#/c/502382/ | |
| 15:59:20 | edmondsw | bauzas yes, but if you were using token x successfully for a while, and then you make another request and get an API, you know it isn't a creds issue... a working token has stopped working, i.e. expired | |
| 15:59:47 | edmondsw | s/API/401/ | |
| 16:00:00 | bauzas | I'm looking at https://tools.ietf.org/html/rfc7235#section-3.1 and I'm puzzled :) | |
| 16:00:03 | sdague | edmondsw: you would also hit that if the user was revoked right? | |
| 16:00:38 | edmondsw | sdague if we really had revocation, yeah... and that's why you'd only retry auth once | |
| 16:01:10 | sdague | bauzas: yeh, the problem auth in HTTP means something very specific, which is not what we do | |
| 16:01:22 | sdague | so our whole thing is fudgy at best | |
| 16:01:24 | edmondsw | if reauth fails, give up... but try reauth once before you give up, because most of the time that will work and it helps so much | |
| 16:01:50 | bauzas | yeah and even microversioning a 400>401 wouldn't help the problem honestly :( | |
| 16:02:00 | sdague | edmondsw: sure, but we could also actually solve the real issue and use the service user right? | |
| 16:02:03 | edmondsw | if we never retried auth on 401, things would break randomly so often that openstack wouldn't be usable | |
| 16:02:12 | sdague | so that the expiration didn't happen | |
| 16:02:32 | edmondsw | this is so fundamental that it's baked into keystoneauth now | |
| 16:02:35 | bauzas | we could | |
| 16:02:57 | sdague | edmondsw: sure, I'm trying to go back to the actual bug and figure out how to not error at all there | |
| 16:03:06 | edleafe | sdague: Here's the one I created. If it's really important I can search other SDKs later: https://github.com/EdLeafe/pyrax/blob/master/pyrax/client.py#L224 | |
| 16:03:49 | sdague | edleafe: it would be good if you could, it's nice to see if this is a pattern within folks that write openstack projects also lives in the larger ecosystem | |
| 16:03:53 | bauzas | is there any reason why we need to pass the user token to neutron and not the service user token ? | |
| 16:04:04 | edmondsw | sdague you might be onto something with service user... I'll have to think through this case again | |
| 16:04:27 | mriedem | johnthetubaguy: L266 https://etherpad.openstack.org/p/cinder-ptg-queens | |
| 16:04:56 | sdague | because if we can not fail at all, but do the right thing, that's better than changing the error code and making all clients have to do this as a retry loop | |
| 16:05:03 | sdague | even if it's a more common retry pattern | |
| 16:05:04 | edmondsw | bauzas sdague I thought we already used the service token for all calls to neutron | |
| 16:05:12 | sdague | edmondsw: we definitely don't | |
| 16:05:15 | edmondsw | ugh | |
| 16:05:24 | mriedem | mikal: hi! | |
| 16:05:25 | sdague | because sometimes you need to operator as the user | |
| 16:05:28 | mikal | mriedem: https://blueprints.launchpad.net/nova/+spec/hurrah-for-privsep | |
| 16:05:28 | edmondsw | that does explain why service token didn't just make this work, then | |
| 16:05:47 | sdague | and I think service token wrapping may not have been added for neutron | |
| 16:05:53 | mikal | And I'm doing the context squashing now, its gonna be a couple of bigish (but very mechanical) patches | |
| 16:10:34 | edleafe | sdague: https://github.com/rackspace/gophercloud/blob/master/provider_client.go#L197-L214 | |
| 16:11:14 | edleafe | sdague: the applications created with SDKs can be long-running, and as a result token expiration is something that needs to be handled smoothly | |
| 16:13:42 | sdague | edleafe: cool, thanks | |
| 16:13:49 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Migration from ``ip`` commands to ``pyroute2`` https://review.openstack.org/484386 | |
| 16:15:35 | sdague | edleafe: we should probably put that in API guidelines then if it's expected, with lots of examples to it in the field | |
| 16:25:01 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Rehome OVO unit tests to tests.unit.test_object.py https://review.openstack.org/489922 | |
| 16:27:40 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add Port Profile info to VIF objects OVS plugin https://review.openstack.org/490819 | |
| 16:28:40 | edmondsw_ | sdague nova using service_auth with neutron: https://github.com/openstack/nova/commit/596e8de5ebd261b2b6610830641d23728b006f53 | |
| 16:29:36 | sdague | oh, so that's even in ocata? | |
| 16:29:45 | edmondsw_ | yeah... | |
| 16:29:46 | sdague | I'm kind of confused now | |
| 16:29:49 | edmondsw_ | me too | |
| 16:29:54 | sdague | maybe they didn't have their config right | |
| 16:30:03 | edmondsw_ | maybe... | |
| 16:30:15 | edmondsw_ | I'll try to dig into that | |
| 16:58:52 | openstackgerrit | Ildiko Vancsa proposed openstack/nova-specs master: Add multiattach support to Nova https://review.openstack.org/499777 | |
| 17:03:20 | openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Return Selection Objects https://review.openstack.org/498830 | |
| 17:05:49 | edmondsw | sdague yeah, I think that bug was hit in an env that was not configured to use service tokens | |
| 17:06:27 | sdague | edmondsw: ok, cool | |
| 17:06:33 | edmondsw | sdague it is optional, and it is a bug for those that aren't using service tokens... so still fix it? | |
| 17:06:41 | edmondsw | it = using service tokens | |
| 17:07:26 | sdague | honestly, I would be more inclined to just make sure there was a better log message | |
| 17:07:42 | sdague | let me think about the fix question | |
| 17:08:24 | edmondsw | that's essentially what the proposed fix does | |
| 17:08:43 | edmondsw | just s/log/error/ | |
| 17:16:39 | openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Return Selection Objects https://review.openstack.org/498830 | |
| 17:21:26 | pooja | bauzas: Thank you for that information! Sorry I got disconnected from IRC earlier | |
| 17:22:53 | pooja | We are still on Newton.. So I believe we can use Caching scheduler rather than multiple scheduler processes at scale? | |
| 17:29:04 | bauzas | pooja: oh, Newton ? what's the exact problem you identified that makes you reasoning about using multiple schedulers ? | |
| 17:30:19 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert "Enable test_iscsi_volume in live migration job" https://review.openstack.org/504143 | |
| 17:30:41 | mriedem | sdague: stephenfin: ^ | |
| 17:30:50 | mriedem | sorry for not WIPing that but it shouldn't have been merged | |
| 17:31:16 | bauzas | mriedem: holy fsck, the problem we discussed yesterday is totally different from the context here https://bugs.launchpad.net/nova/+bug/1497253 | |
| 17:31:17 | openstack | Launchpad bug 1497253 in OpenStack Compute (nova) "different availability zone for nova and cinder when AZ is not explicitly given" [Low,In progress] - Assigned to Roman Podoliaka (rpodolyaka) | |
| 17:31:36 | sdague | mriedem: +A | |
| 17:31:54 | bauzas | mriedem: basically, the problem is the opposite, when you are not asking for a specific AZ but cross_az_attach is False, then you're pissed off | |
| 17:32:03 | mriedem | bauzas: correct | |
| 17:32:07 | mriedem | i read the bug again later | |
| 17:32:09 | sdague | bauzas: right, because nova is sent by default, right? | |
| 17:32:16 | mriedem | sdague: no, | |
| 17:32:19 | mriedem | cinder defaults to nova i think | |
| 17:32:26 | mriedem | and instance.availability_zone is None | |
| 17:32:28 | bauzas | sdague: nova doesn't honor AZs if you don't ask for | |
| 17:32:30 | mriedem | if the user didn't specify an AZ | |
| 17:32:37 | bauzas | but Cinder does, hence thr bug | |
| 17:32:51 | sdague | ah | |
| 17:32:56 | bauzas | at least, the opt is saying "please respect AZs" | |
| 17:33:03 | mriedem | i've always been told that AZs in cinder are half baked | |
| 17:33:08 | bauzas | yup | |
| 17:33:18 | bauzas | I need to think about a possible solution | |
| 17:33:40 | bauzas | I think I already express my opinion about cross_az_attach opt | |
| 17:33:52 | bauzas | I just feel it's promising something we really don't care | |