Earlier  
Later
New Search
Posted Nick Remark
#openstack-sdks - 2018-01-29
15:11:22 dtroyer that makes the entire clinet stack now release-dependant in some manner
15:11:31 mordred dtroyer: http://logs.openstack.org/95/538695/1/check/neutron-grenade/0917140/logs/grenade.sh.txt.gz#_2018-01-29_04_01_38_652
15:12:00 mordred dtroyer: yah - lemme poke some more and see what can be done to unwind that
15:14:48 mordred dtroyer: heh... it's neutron -> python-designateclient -> osc-lib -> openstacksdk
15:15:47 dtroyer so the fix would be to pull the OSC bits out of the lib that is ultimately required by services
15:16:31 dtroyer and all of the "upper layer" client libs that include OSC will have this problem should they be used by another service
15:17:28 mordred dtroyer: why not just upgrade python-openstackclient in grenade? I don't think we're going to unwind python-*client depending on osc-lib since that's what folks are doing for their osc plugins
15:18:04 dtroyer this is grenade finding unknown/unwanted dependencies for us :)
15:18:09 mordred heh
15:18:38 dtroyer we can do an upgrade, and we'll be throwing out the ability to know when now clients are OpenStack release sensitive in ways that we previously said they were not
15:19:50 amotoki individual dependencies looks no problem (neutron -> python-designateclient and python-designateclient -> osc-lib -> openstacksdk).
15:19:54 dtroyer this is where the separation of python-*client and SDK became necessary, and why we can't just use SDK for everything. If we throw that away, then we can use SDK for everything
15:20:00 amotoki do we need to run OSC in a venv like tempest?
15:20:20 mordred dtroyer: I think we should use sdk for everything and delete literally al of the python-*client libraries personally
15:20:32 dtroyer amotoki: yes. Or vendor requirements. Or static link (whatever appears like that in python, py2exe and friends)
15:21:15 dtroyer mordred: I was there at the start of the SDK project. This is what changed my mind. If we do that, everything that uses SDK is tied to OpenStack releases
15:22:20 mordred dtroyer: not once we cut a 1.0 of sdk that has a 100% stable api- the problem here is the massive breaking API sdk change that just happened
15:22:46 mordred dtroyer: but it's possible that what I've just learned is that we need to completely revert all of the changes to sdk and start over from scratch
15:23:14 dtroyer mordred: the 1.0 is not the underlying problem, that's on top
15:23:24 dtroyer and we don't need to start over
15:23:52 dtroyer it is the separation of assumptions that clients and services make that is the problem. Those assumptions intersect but not completely
15:25:06 dtroyer if clients/apps that use SDK could do so in isolation, then the version conflict is a non-issue and both stacks can install the version they need.
15:25:20 dtroyer in a DevStack non-venv world, this just doesn't work
15:25:55 dtroyer this is one of the things that drove the addition (still unfinished IIIRC) of venv support to DevStack
15:26:53 mordred yah - I hear all of that - but I'm not sure how to move forward in a way that will unstick the queens release but not screw everything
15:27:01 dtroyer so yeah, short term just add an upgrade step somewhere to get everything that depends on SDK to grenade
15:27:24 mordred ah - ok - do that short term, unwind things as a longer term step?
15:31:12 dtroyer I suppose…
15:31:51 dtroyer and long-term split all sdk/osc bits out of python-*client libs that are going to be used by another service
15:32:48 dtroyer or we just containerize the control plane upstream and call it a day :)
15:32:52 mordred dtroyer: heh
15:39:46 amotoki this situation happens because both SDK and CLI stuffs are provided by a single deliverable. it makes things complicated
15:40:03 amotoki neutron just needs the python binding of desginateclient, but designateclient OSC plugin depends on osc-lib....
15:40:36 mordred amotoki: yes 100%
15:40:58 amotoki this discussion raises me again a question on where python binding should be implemented.
15:41:22 dtroyer that really doens't matter as long as we do not cross the client/server streams :)
15:42:00 dtroyer really, it was my failure to imagine that client libs with OSc bits in them would be used by servers too that let this happen
15:42:08 mordred amotoki: honestly I truly believe we should delete all of the python-*client bindings - but that's a much bigger engineering effort as we'd have to unwind or rethink a ton of things we don't really have the bandwidth to rethink these days
15:42:15 dtroyer putting the OSc bits into a separate repo solves it
15:42:20 mordred dtroyer: ++
15:42:36 mordred to the osc bits in separare repo
15:42:51 amotoki yeah, but it needs project teams need to maintain more repos..
15:43:36 mordred dtroyer, amotoki: https://review.openstack.org/#/c/538951/ and then https://review.openstack.org/#/c/538695/ depends-on it
15:46:05 amotoki i hope it works
15:46:20 openstackgerrit Stephen Finucane proposed openstack/cliff master: sphinxext: Warn if namespace or command pattern invalid https://review.openstack.org/510913
15:47:30 mordred amotoki: me too
15:48:22 dtroyer I left a comment including the short- and long-term intentions
15:49:33 mordred awesome
15:50:08 mordred dtroyer: it might be worth talking through this whole thing at the PTG, too
15:53:15 dtroyer mordred: yeah, either re-affirm the assumptions or choose to change them...
15:53:24 mordred yup.
15:53:48 mordred dtroyer: btw - I think we're going to need a patch for osc master too - poking at that now
16:00:56 openstackgerrit Monty Taylor proposed openstack/python-openstackclient master: Use find_ip from openstacksdk https://review.openstack.org/538958
16:13:56 mordred dtroyer: I also pushed up this to stable/pike: https://review.openstack.org/538963 Use find_ip from openstacksdk ... if it works and we cut a 3.12.1 release I think we can avoid the grenade change - I'll leave it to your judgement as to which thing you find preferrable
16:15:25 mordred dtroyer: I also think I *might* have an idea of a patch to make to sdk that could deal with the issue as well ... experimenting with that now
16:18:09 dtroyer That cleanup to OSC should happen either way so if it helps we'll push it in
16:19:40 mordred awesome
16:20:04 mordred maybe between the OSC cleanup and the sdk patch I'm working on now we can avoid the short-term grenade hacks
16:48:02 openstackgerrit Monty Taylor proposed openstack/python-openstacksdk master: Provide compatibility for people passing raw sessions https://review.openstack.org/538968
16:48:44 mordred dtroyer, amotoki: ok - I *think* that ^^ will actually fix the current issue without the grenade or osc patches ... the patch adds the neutron-grenade job just to be sure
16:49:07 mordred if that patch comes back green, then we should be able to land it and cut an sdk point release and have all the stuff work
16:50:17 mordred (and if it works, I'll make a followup patch that adds the call in to all of the places it needs to to be called
17:22:33 dtroyer mordred: do we have any idea if anything that isn't OSC does that Session thing?
17:27:06 mordred dtroyer: looking in heat right now to check
17:27:42 mordred dtroyer: it doesn't look like heat does
17:28:01 openstackgerrit Monty Taylor proposed openstack/python-openstacksdk master: Provide compatibility for people passing raw sessions https://review.openstack.org/538968
17:29:43 mordred dtroyer: so we could just call the osc patch a good enough fix
17:29:59 mordred dtroyer: I think I'm mostly just shotgunning all of the possible options I can think of right now :)
17:30:33 dtroyer sure, I'd like to avoid more special cases that will be around forever in the SDK if we can
17:31:10 mordred dtroyer: totally agree
18:07:38 openstackgerrit Monty Taylor proposed openstack/python-openstackclient master: Use find_ip from openstacksdk https://review.openstack.org/538958
19:17:44 Bar_ dtroyer, hey, When is the next osc-lib package is expected to be released?
19:22:21 dtroyer Bar_: We just did what becomes the queens/stable release a week or so ago, nothing is queued up that I'm aware of
19:24:42 Bar_ dtroyer, I have proposed https://review.openstack.org/#/c/531025/ earlier this month, but it didn't make the cut. I have another patch that is dependent on that, though it is not urgent, I wanted to know what time table could i expect.
19:30:44 dtroyer That merged, but yes, not before 1.9.0 was released, gotcha
19:33:12 dtroyer Bar_: is next week soon enough? I'd like to hold off adding tot he release teams work (as small as this is) until the gate issues/backlog is cleared
19:35:58 Bar_ dtroyer, totally fine. Not urgent at all. I suspected it might take months, that's why I asked.
19:36:11 Bar_ am I correct to expect it to be v1.9.1?
19:36:27 dtroyer no, we also merged some global requirements, which force a minor rev
19:36:33 dtroyer it'll be 1.10.0
19:38:12 Bar_ dtroyer, I see. thanks. By the way, it is besides the point, but maybe we should reconsider the reliance on testtools, it hasn't seen meaninful updates for almost a year now.
19:38:54 Bar_ My patch only suppresses exceptions of functionality that is natively supported by unittest
19:39:55 Bar_ testtools breaks that functionality, and there's not much hope there.
19:57:42 mordred Bar_: subTest does not exist in python2 - it's only in python3.4 and above - have you tried testscenarios instead?
19:59:04 mordred http://git.openstack.org/cgit/openstack/python-openstacksdk/tree/openstack/tests/functional/cloud/test_devstack.py#n25
19:59:07 Bar_ mordred, sorry, I'm unfamiliar with testscenarios . We don't use it in the relevant repo
19:59:08 mordred is an example
20:00:13 mordred line 25 is key, then the scenarios attribute on line 32 lists a list of scenarios with a name for the scenario and a dict of attributes to add to the test for each scenario
20:00:33 mordred load_tests then explodes that out during unittest discovery listing to a test named for each scenario
20:00:50 mordred which means they can be scheduled across test runner backends for multi-process/parallel testing
20:02:57 mordred Bar_: if you look at http://codesearch.openstack.org/?q=testscenarios&i=nope&files=&repos= you can see it's used fairly extensively across openstack -so before we were to go trying to solve the same problem using a different tech, I'd personally like to figure out if there is a benefit to using subTest over testscenarios
20:03:02 Bar_ mordred, subtest fails with PY35 when inheriting from testtools.
20:04:28 mordred Bar_: right. but what I'm saying is that rather than trying to convince openstack to move off of testtools, you might have better luck just using testscenarios which works with both python2.7 and python3.5
20:04:49 Bar_ mordred, cool
22:25:26 mordred dtroyer: https://review.openstack.org/#/c/538963/ failed its functional tests - but the error is seems weird
22:25:28 mordred dtroyer: http://logs.openstack.org/63/538963/2/check/osc-functional-devstack/c475bfb/job-output.txt.gz#_2018-01-29_22_22_13_041554
22:25:37 mordred dtroyer: does that make the senses to you?
22:35:19 dtroyer mordred: I've never seen that before…
22:37:41 mordred dtroyer: I just pushed up https://review.openstack.org/539032 Enable cinder services explicitly
22:37:48 mordred dtroyer: just to see if that helps

Earlier   Later