| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-14 | |||
| 16:01:22 | sdague | so our whole thing is fudgy at best | |
| 16:01:24 | edmondsw | if reauth fails, give up... but try reauth once before you give up, because most of the time that will work and it helps so much | |
| 16:01:50 | bauzas | yeah and even microversioning a 400>401 wouldn't help the problem honestly :( | |
| 16:02:00 | sdague | edmondsw: sure, but we could also actually solve the real issue and use the service user right? | |
| 16:02:03 | edmondsw | if we never retried auth on 401, things would break randomly so often that openstack wouldn't be usable | |
| 16:02:12 | sdague | so that the expiration didn't happen | |
| 16:02:32 | edmondsw | this is so fundamental that it's baked into keystoneauth now | |
| 16:02:35 | bauzas | we could | |
| 16:02:57 | sdague | edmondsw: sure, I'm trying to go back to the actual bug and figure out how to not error at all there | |
| 16:03:06 | edleafe | sdague: Here's the one I created. If it's really important I can search other SDKs later: https://github.com/EdLeafe/pyrax/blob/master/pyrax/client.py#L224 | |
| 16:03:49 | sdague | edleafe: it would be good if you could, it's nice to see if this is a pattern within folks that write openstack projects also lives in the larger ecosystem | |
| 16:03:53 | bauzas | is there any reason why we need to pass the user token to neutron and not the service user token ? | |
| 16:04:04 | edmondsw | sdague you might be onto something with service user... I'll have to think through this case again | |
| 16:04:27 | mriedem | johnthetubaguy: L266 https://etherpad.openstack.org/p/cinder-ptg-queens | |
| 16:04:56 | sdague | because if we can not fail at all, but do the right thing, that's better than changing the error code and making all clients have to do this as a retry loop | |
| 16:05:03 | sdague | even if it's a more common retry pattern | |
| 16:05:04 | edmondsw | bauzas sdague I thought we already used the service token for all calls to neutron | |
| 16:05:12 | sdague | edmondsw: we definitely don't | |
| 16:05:15 | edmondsw | ugh | |
| 16:05:24 | mriedem | mikal: hi! | |
| 16:05:25 | sdague | because sometimes you need to operator as the user | |
| 16:05:28 | edmondsw | that does explain why service token didn't just make this work, then | |
| 16:05:28 | mikal | mriedem: https://blueprints.launchpad.net/nova/+spec/hurrah-for-privsep | |
| 16:05:47 | sdague | and I think service token wrapping may not have been added for neutron | |
| 16:05:53 | mikal | And I'm doing the context squashing now, its gonna be a couple of bigish (but very mechanical) patches | |
| 16:10:34 | edleafe | sdague: https://github.com/rackspace/gophercloud/blob/master/provider_client.go#L197-L214 | |
| 16:11:14 | edleafe | sdague: the applications created with SDKs can be long-running, and as a result token expiration is something that needs to be handled smoothly | |
| 16:13:42 | sdague | edleafe: cool, thanks | |
| 16:13:49 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Migration from ``ip`` commands to ``pyroute2`` https://review.openstack.org/484386 | |
| 16:15:35 | sdague | edleafe: we should probably put that in API guidelines then if it's expected, with lots of examples to it in the field | |
| 16:25:01 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Rehome OVO unit tests to tests.unit.test_object.py https://review.openstack.org/489922 | |
| 16:27:40 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add Port Profile info to VIF objects OVS plugin https://review.openstack.org/490819 | |
| 16:28:40 | edmondsw_ | sdague nova using service_auth with neutron: https://github.com/openstack/nova/commit/596e8de5ebd261b2b6610830641d23728b006f53 | |
| 16:29:36 | sdague | oh, so that's even in ocata? | |
| 16:29:45 | edmondsw_ | yeah... | |
| 16:29:46 | sdague | I'm kind of confused now | |
| 16:29:49 | edmondsw_ | me too | |
| 16:29:54 | sdague | maybe they didn't have their config right | |
| 16:30:03 | edmondsw_ | maybe... | |
| 16:30:15 | edmondsw_ | I'll try to dig into that | |
| 16:58:52 | openstackgerrit | Ildiko Vancsa proposed openstack/nova-specs master: Add multiattach support to Nova https://review.openstack.org/499777 | |
| 17:03:20 | openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Return Selection Objects https://review.openstack.org/498830 | |
| 17:05:49 | edmondsw | sdague yeah, I think that bug was hit in an env that was not configured to use service tokens | |
| 17:06:27 | sdague | edmondsw: ok, cool | |
| 17:06:33 | edmondsw | sdague it is optional, and it is a bug for those that aren't using service tokens... so still fix it? | |
| 17:06:41 | edmondsw | it = using service tokens | |
| 17:07:26 | sdague | honestly, I would be more inclined to just make sure there was a better log message | |
| 17:07:42 | sdague | let me think about the fix question | |
| 17:08:24 | edmondsw | that's essentially what the proposed fix does | |
| 17:08:43 | edmondsw | just s/log/error/ | |
| 17:16:39 | openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Return Selection Objects https://review.openstack.org/498830 | |
| 17:21:26 | pooja | bauzas: Thank you for that information! Sorry I got disconnected from IRC earlier | |
| 17:22:53 | pooja | We are still on Newton.. So I believe we can use Caching scheduler rather than multiple scheduler processes at scale? | |
| 17:29:04 | bauzas | pooja: oh, Newton ? what's the exact problem you identified that makes you reasoning about using multiple schedulers ? | |
| 17:30:19 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert "Enable test_iscsi_volume in live migration job" https://review.openstack.org/504143 | |
| 17:30:41 | mriedem | sdague: stephenfin: ^ | |
| 17:30:50 | mriedem | sorry for not WIPing that but it shouldn't have been merged | |
| 17:31:16 | bauzas | mriedem: holy fsck, the problem we discussed yesterday is totally different from the context here https://bugs.launchpad.net/nova/+bug/1497253 | |
| 17:31:17 | openstack | Launchpad bug 1497253 in OpenStack Compute (nova) "different availability zone for nova and cinder when AZ is not explicitly given" [Low,In progress] - Assigned to Roman Podoliaka (rpodolyaka) | |
| 17:31:36 | sdague | mriedem: +A | |
| 17:31:54 | bauzas | mriedem: basically, the problem is the opposite, when you are not asking for a specific AZ but cross_az_attach is False, then you're pissed off | |
| 17:32:03 | mriedem | bauzas: correct | |
| 17:32:07 | mriedem | i read the bug again later | |
| 17:32:09 | sdague | bauzas: right, because nova is sent by default, right? | |
| 17:32:16 | mriedem | sdague: no, | |
| 17:32:19 | mriedem | cinder defaults to nova i think | |
| 17:32:26 | mriedem | and instance.availability_zone is None | |
| 17:32:28 | bauzas | sdague: nova doesn't honor AZs if you don't ask for | |
| 17:32:30 | mriedem | if the user didn't specify an AZ | |
| 17:32:37 | bauzas | but Cinder does, hence thr bug | |
| 17:32:51 | sdague | ah | |
| 17:32:56 | bauzas | at least, the opt is saying "please respect AZs" | |
| 17:33:03 | mriedem | i've always been told that AZs in cinder are half baked | |
| 17:33:08 | bauzas | yup | |
| 17:33:18 | bauzas | I need to think about a possible solution | |
| 17:33:40 | bauzas | I think I already express my opinion about cross_az_attach opt | |
| 17:33:52 | bauzas | I just feel it's promising something we really don't care | |
| 17:34:20 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: config drive https://review.openstack.org/409404 | |
| 17:36:37 | bauzas | mriedem: sdague: here is one thing: if the user didn't specify an AZ, then we feel we should not honor cross_az_attach and leave the instance be on a separate AZ than the Cinder AZ | |
| 17:36:52 | bauzas | because the instance could be migrated on a totally separate AZ | |
| 17:37:31 | bauzas | so I'd just relax the condition | |
| 17:37:48 | pooja | bauzas: On Newton, when scheduling 60 vms in parallel with heat stack, we see vms in scheduling state for about 30 sec | |
| 17:37:52 | bauzas | and make the conf opt pretty clear that's only enforced if you ask for an AZ | |
| 17:38:08 | pooja | My understanding is its because scheduler is processing them serially | |
| 17:38:18 | bauzas | pooja: that's a correct assumption | |
| 17:39:11 | bauzas | pooja: do you often request for such large requests ? | |
| 17:39:18 | bauzas | s/requests/numbers | |
| 17:40:11 | bauzas | pooja: given we really want to deprecate CachingScheduler, I'd rather be in favor of running multiple schedulers and increasing the conf opt responsible for the number of retries | |
| 17:40:50 | bauzas | pooja: that way, when you migrate to Ocata and eventually Pike, you wouldn't change your architecture and just leave multiple schedulers, since retries would seriously decrease | |
| 17:41:06 | bauzas | eventually, we will stop proposing reschedules | |
| 17:42:45 | openstackgerrit | Ed Leafe proposed openstack/nova master: Add alternate hosts https://review.openstack.org/486215 | |
| 17:42:45 | openstackgerrit | Ed Leafe proposed openstack/nova master: Add Selection objects https://review.openstack.org/499239 | |
| 17:45:09 | pooja | bauzas: Yeah, that makes sense.. We are using heat stacks at scale and the plan is go from 60 to 200 eventually | |
| 17:45:48 | pooja | So with Ocata, multiple schedulers feature is different from the Placement API you referred to earlier? | |
| 17:51:13 | openstackgerrit | Merged openstack/nova master: Remove deprecated keymgr code https://review.openstack.org/439855 | |
| 17:56:34 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Call terminate_connection when shelve_offloading https://review.openstack.org/257275 | |
| 17:58:14 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Call terminate_connection when shelve_offloading https://review.openstack.org/257275 | |
| 18:06:34 | mikal | Is opy35 broken by the testr change? | |
| 18:06:36 | mikal | http://logs.openstack.org/93/503893/1/check/gate-nova-python35/aed6ed0/console.html#_2017-09-14_15_17_45_846189 | |
| 18:07:49 | mikal | Ugh, ignore me | |