Earlier  
Later
New Search
Posted Nick Remark
#openstack-sdks - 2018-01-29
00:35:39 openstackgerrit OpenStack Proposal Bot proposed openstack/python-openstackclient master: Updated from global requirements https://review.openstack.org/538705
01:46:20 openstackgerrit James E. Blair proposed openstack/cliff master: Zuul: Remove project name https://review.openstack.org/538716
02:06:24 openstackgerrit James E. Blair proposed openstack/cliff master: Zuul: Remove project name https://review.openstack.org/538718
03:34:48 openstackgerrit Merged openstack/python-openstackclient master: Updated from global requirements https://review.openstack.org/538705
12:49:45 openstackgerrit James E. Blair proposed openstack/os-client-config master: Zuul: Remove project name https://review.openstack.org/538901
15:05:32 mordred dtroyer: when you're up and in a position to think about things ... grenade does not seem to upgrade openstackclient in the new install
15:06:33 mordred dtroyer: which is causing some ... fun ... with bumping the sdk constraint - but I can't figure out why or where to fix it and could use a hand
15:08:08 dtroyer hmmm, no tit does not. Grenade was designed with only server-side upgrades in mind, that coupled with the clients support all versions, why are we upgrading SDK?
15:09:53 mordred dtroyer: it gets pulled as a transitive depend from one of the python-*clients
15:10:01 mordred dtroyer: lemme find which one real quick
15:10:11 dtroyer ugh
15:10:27 dtroyer there was supposed to be no cross-dependencies between sdk and *client libs
15:10:55 dtroyer so here is where bringing in occ breaks assumptions
15:10:57 mordred dtroyer: it's neutron
15:11:22 dtroyer that makes the entire clinet stack now release-dependant in some manner
15:11:31 mordred dtroyer: http://logs.openstack.org/95/538695/1/check/neutron-grenade/0917140/logs/grenade.sh.txt.gz#_2018-01-29_04_01_38_652
15:12:00 mordred dtroyer: yah - lemme poke some more and see what can be done to unwind that
15:14:48 mordred dtroyer: heh... it's neutron -> python-designateclient -> osc-lib -> openstacksdk
15:15:47 dtroyer so the fix would be to pull the OSC bits out of the lib that is ultimately required by services
15:16:31 dtroyer and all of the "upper layer" client libs that include OSC will have this problem should they be used by another service
15:17:28 mordred dtroyer: why not just upgrade python-openstackclient in grenade? I don't think we're going to unwind python-*client depending on osc-lib since that's what folks are doing for their osc plugins
15:18:04 dtroyer this is grenade finding unknown/unwanted dependencies for us :)
15:18:09 mordred heh
15:18:38 dtroyer we can do an upgrade, and we'll be throwing out the ability to know when now clients are OpenStack release sensitive in ways that we previously said they were not
15:19:50 amotoki individual dependencies looks no problem (neutron -> python-designateclient and python-designateclient -> osc-lib -> openstacksdk).
15:19:54 dtroyer this is where the separation of python-*client and SDK became necessary, and why we can't just use SDK for everything. If we throw that away, then we can use SDK for everything
15:20:00 amotoki do we need to run OSC in a venv like tempest?
15:20:20 mordred dtroyer: I think we should use sdk for everything and delete literally al of the python-*client libraries personally
15:20:32 dtroyer amotoki: yes. Or vendor requirements. Or static link (whatever appears like that in python, py2exe and friends)
15:21:15 dtroyer mordred: I was there at the start of the SDK project. This is what changed my mind. If we do that, everything that uses SDK is tied to OpenStack releases
15:22:20 mordred dtroyer: not once we cut a 1.0 of sdk that has a 100% stable api- the problem here is the massive breaking API sdk change that just happened
15:22:46 mordred dtroyer: but it's possible that what I've just learned is that we need to completely revert all of the changes to sdk and start over from scratch
15:23:14 dtroyer mordred: the 1.0 is not the underlying problem, that's on top
15:23:24 dtroyer and we don't need to start over
15:23:52 dtroyer it is the separation of assumptions that clients and services make that is the problem. Those assumptions intersect but not completely
15:25:06 dtroyer if clients/apps that use SDK could do so in isolation, then the version conflict is a non-issue and both stacks can install the version they need.
15:25:20 dtroyer in a DevStack non-venv world, this just doesn't work
15:25:55 dtroyer this is one of the things that drove the addition (still unfinished IIIRC) of venv support to DevStack
15:26:53 mordred yah - I hear all of that - but I'm not sure how to move forward in a way that will unstick the queens release but not screw everything
15:27:01 dtroyer so yeah, short term just add an upgrade step somewhere to get everything that depends on SDK to grenade
15:27:24 mordred ah - ok - do that short term, unwind things as a longer term step?
15:31:12 dtroyer I suppose…
15:31:51 dtroyer and long-term split all sdk/osc bits out of python-*client libs that are going to be used by another service
15:32:48 dtroyer or we just containerize the control plane upstream and call it a day :)
15:32:52 mordred dtroyer: heh
15:39:46 amotoki this situation happens because both SDK and CLI stuffs are provided by a single deliverable. it makes things complicated
15:40:03 amotoki neutron just needs the python binding of desginateclient, but designateclient OSC plugin depends on osc-lib....
15:40:36 mordred amotoki: yes 100%
15:40:58 amotoki this discussion raises me again a question on where python binding should be implemented.
15:41:22 dtroyer that really doens't matter as long as we do not cross the client/server streams :)
15:42:00 dtroyer really, it was my failure to imagine that client libs with OSc bits in them would be used by servers too that let this happen
15:42:08 mordred amotoki: honestly I truly believe we should delete all of the python-*client bindings - but that's a much bigger engineering effort as we'd have to unwind or rethink a ton of things we don't really have the bandwidth to rethink these days
15:42:15 dtroyer putting the OSc bits into a separate repo solves it
15:42:20 mordred dtroyer: ++
15:42:36 mordred to the osc bits in separare repo
15:42:51 amotoki yeah, but it needs project teams need to maintain more repos..
15:43:36 mordred dtroyer, amotoki: https://review.openstack.org/#/c/538951/ and then https://review.openstack.org/#/c/538695/ depends-on it
15:46:05 amotoki i hope it works
15:46:20 openstackgerrit Stephen Finucane proposed openstack/cliff master: sphinxext: Warn if namespace or command pattern invalid https://review.openstack.org/510913
15:47:30 mordred amotoki: me too
15:48:22 dtroyer I left a comment including the short- and long-term intentions
15:49:33 mordred awesome
15:50:08 mordred dtroyer: it might be worth talking through this whole thing at the PTG, too
15:53:15 dtroyer mordred: yeah, either re-affirm the assumptions or choose to change them...
15:53:24 mordred yup.
15:53:48 mordred dtroyer: btw - I think we're going to need a patch for osc master too - poking at that now
16:00:56 openstackgerrit Monty Taylor proposed openstack/python-openstackclient master: Use find_ip from openstacksdk https://review.openstack.org/538958
16:13:56 mordred dtroyer: I also pushed up this to stable/pike: https://review.openstack.org/538963 Use find_ip from openstacksdk ... if it works and we cut a 3.12.1 release I think we can avoid the grenade change - I'll leave it to your judgement as to which thing you find preferrable
16:15:25 mordred dtroyer: I also think I *might* have an idea of a patch to make to sdk that could deal with the issue as well ... experimenting with that now
16:18:09 dtroyer That cleanup to OSC should happen either way so if it helps we'll push it in
16:19:40 mordred awesome
16:20:04 mordred maybe between the OSC cleanup and the sdk patch I'm working on now we can avoid the short-term grenade hacks
16:48:02 openstackgerrit Monty Taylor proposed openstack/python-openstacksdk master: Provide compatibility for people passing raw sessions https://review.openstack.org/538968
16:48:44 mordred dtroyer, amotoki: ok - I *think* that ^^ will actually fix the current issue without the grenade or osc patches ... the patch adds the neutron-grenade job just to be sure
16:49:07 mordred if that patch comes back green, then we should be able to land it and cut an sdk point release and have all the stuff work
16:50:17 mordred (and if it works, I'll make a followup patch that adds the call in to all of the places it needs to to be called
17:22:33 dtroyer mordred: do we have any idea if anything that isn't OSC does that Session thing?
17:27:06 mordred dtroyer: looking in heat right now to check
17:27:42 mordred dtroyer: it doesn't look like heat does
17:28:01 openstackgerrit Monty Taylor proposed openstack/python-openstacksdk master: Provide compatibility for people passing raw sessions https://review.openstack.org/538968
17:29:43 mordred dtroyer: so we could just call the osc patch a good enough fix
17:29:59 mordred dtroyer: I think I'm mostly just shotgunning all of the possible options I can think of right now :)
17:30:33 dtroyer sure, I'd like to avoid more special cases that will be around forever in the SDK if we can
17:31:10 mordred dtroyer: totally agree
18:07:38 openstackgerrit Monty Taylor proposed openstack/python-openstackclient master: Use find_ip from openstacksdk https://review.openstack.org/538958
19:17:44 Bar_ dtroyer, hey, When is the next osc-lib package is expected to be released?
19:22:21 dtroyer Bar_: We just did what becomes the queens/stable release a week or so ago, nothing is queued up that I'm aware of
19:24:42 Bar_ dtroyer, I have proposed https://review.openstack.org/#/c/531025/ earlier this month, but it didn't make the cut. I have another patch that is dependent on that, though it is not urgent, I wanted to know what time table could i expect.
19:30:44 dtroyer That merged, but yes, not before 1.9.0 was released, gotcha
19:33:12 dtroyer Bar_: is next week soon enough? I'd like to hold off adding tot he release teams work (as small as this is) until the gate issues/backlog is cleared
19:35:58 Bar_ dtroyer, totally fine. Not urgent at all. I suspected it might take months, that's why I asked.
19:36:11 Bar_ am I correct to expect it to be v1.9.1?
19:36:27 dtroyer no, we also merged some global requirements, which force a minor rev
19:36:33 dtroyer it'll be 1.10.0
19:38:12 Bar_ dtroyer, I see. thanks. By the way, it is besides the point, but maybe we should reconsider the reliance on testtools, it hasn't seen meaninful updates for almost a year now.
19:38:54 Bar_ My patch only suppresses exceptions of functionality that is natively supported by unittest
19:39:55 Bar_ testtools breaks that functionality, and there's not much hope there.
19:57:42 mordred Bar_: subTest does not exist in python2 - it's only in python3.4 and above - have you tried testscenarios instead?
19:59:04 mordred http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/tests/functional/cloud/test_devstack.py#n25
19:59:07 Bar_ mordred, sorry, I'm unfamiliar with testscenarios . We don't use it in the relevant repo

Earlier   Later