| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-sdks - 2019-10-29 | |||
| 17:13:59 | umbSublime | efried: I'm using keystone v3. As for the microversion I get from the proxy how can i check this? i can only see 'version=2' in vars(conn.compute) | |
| 17:14:58 | umbSublime | v3.11 for ksa | |
| 18:40:01 | mriedem | dtroyer: can we get this simple osc change in https://review.opendev.org/#/c/691039/ to unblock novaclient 16.0.0 from being in u-c https://review.opendev.org/#/c/690097/ | |
| 20:38:01 | efried | mriedem: something changed between then and now, osc isn't building: https://review.opendev.org/#/c/691910/ | |
| 20:38:11 | efried | so your patch won't clear the gate, is my point. | |
| 20:38:47 | mriedem | but not just mine, any you're saying | |
| 20:38:52 | efried | And btw, I have no idea what's going on there. I can't repro "locally" | |
| 20:39:04 | efried | yes, noticed it on one of mine, seemed unrelated, so I put that up to confirm. | |
| 20:39:19 | efried | mebbe you can do your logstash thing and find the culprit? | |
| 20:40:28 | mriedem | my guess is some py2/3 thing was dropped/released | |
| 20:42:00 | mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22TypeError%3A%20Unicode-objects%20must%20be%20encoded%20before%20hashing%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d isn't super useful | |
| 20:43:38 | efried | I guess it wouldn't be, since nobody's been building anything in osc except for me. | |
| 20:43:51 | mriedem | well, could be openstack-requirements for a dependency | |
| 20:43:55 | mriedem | but i'm not seeing anything obvious | |
| 20:44:03 | efried | but it shows that whatever it is only affects python-openstackclient. | |
| 20:44:13 | mriedem | this hashlib library that's blowing up here https://github.com/openstack/python-openstackclient/blob/4bde9af89251431791fc8d69fe09d5e17a8fba8f/openstackclient/tests/functional/compute/v2/test_agent.py#L24 hasn't changed in forever on pyi | |
| 20:44:15 | mriedem | *pypi | |
| 20:44:40 | mriedem | https://zuul.opendev.org/t/openstack/build/49ff6a5645554095b3fb51bce497aff5/log/tox/functional-0.log hmm is that in the standard lib now? | |
| 20:45:01 | mriedem | this job must use py36? | |
| 20:45:04 | efried | did that job recently cut over to py3? | |
| 20:45:14 | mriedem | that's my guess | |
| 20:45:40 | efried | well, its incarnation in python-openstackclient didn't, so unless it has an ancestor that did... | |
| 20:45:46 | mriedem | https://docs.python.org/3.6/library/hashlib.html#hashlib.hash.update | |
| 20:46:01 | mriedem | that's entirely possible, parent projects are moving to py3 in ussuri | |
| 20:46:16 | efried | devstack-tox-functional | |
| 20:47:27 | mriedem | seems ok https://opendev.org/openstack/devstack/src/branch/master/.zuul.yaml#L566 | |
| 20:48:03 | mriedem | parent: multinode | |
| 20:49:46 | efried | ...which is abstract, whatever that means. | |
| 20:49:56 | mriedem | means it just has to be extended i think | |
| 20:50:10 | efried | well, yeah | |
| 20:50:33 | efried | btw, attempting to run simply tox -e functional on my devstack node deadlocks. | |
| 20:50:46 | mriedem | i don't see anything recent in openstack-zuul-jobs that looks related | |
| 20:50:49 | mriedem | or devstack | |
| 20:51:03 | efried | yeah | |
| 20:51:35 | mriedem | the job does have USE_PYTHON3=False in it | |
| 20:51:38 | mriedem | so it should be using py2 | |
| 20:51:52 | efried | but isn't? | |
| 20:52:54 | mriedem | looks like it's using py3 https://zuul.opendev.org/t/openstack/build/49ff6a5645554095b3fb51bce497aff5/log/job-output.txt#30632 | |
| 20:53:10 | mriedem | $ /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3 functional | |
| 20:53:21 | efried | I'll push a fix. | |
| 20:53:35 | efried | ...without for the moment caring that we "should be running py2". Because we really shouldn't. | |
| 20:54:18 | mriedem | i think you just need to put USE_PYTHON3: True in here https://github.com/openstack/python-openstackclient/blob/master/.zuul.yaml#L76 | |
| 20:54:45 | efried | but the problem is that it *is* using py3 and md5().update() needs an encoded string. | |
| 20:55:54 | mriedem | well, a quick fix would be setting basepython = python2 in the tox.ini functional env i think if the job isn't using py3, | |
| 20:56:01 | openstackgerrit | Eric Fried proposed openstack/python-openstackclient master: Fix functional tests for py3 https://review.opendev.org/691980 | |
| 20:56:02 | mriedem | but if the parent is now using py3, yeah idk what's going on there that changed | |
| 20:56:23 | efried | mriedem: That ought to fix at least that error, we'll see if something else crops up. | |
| 20:57:57 | efried | dtroyer: FYI: TL;DR: python-openstackclient builds broke some time between 10/24 and this morning when the functional jobs somehow got cut over to py3. Fixing via https://review.opendev.org/691980 | |
| 20:59:40 | efried | actually, I can narrow it down further. I have a successful build Oct 28 19:31 (that's US Central time) | |
| 20:59:51 | mordred | efried: how did we manage to break builds? | |
| 21:00:02 | efried | mordred: haven't figured that out yet. | |
| 21:00:18 | efried | As far as mriedem and I can tell, "something" cut over to py3 | |
| 21:00:29 | mordred | "awesome" | |
| 21:00:53 | efried | If this build is running py2, that'll be confirmed: https://zuul.opendev.org/t/openstack/build/ad5f0ee04fed4285b9f5429debd9b32d/log/job-output.txt | |
| 21:01:17 | openstackgerrit | Pierre Prinetti proposed openstack/python-openstackclient master: versions: Fix 'versions show' help message https://review.opendev.org/686408 | |
| 21:02:33 | efried | I'm not good at definitively determining whether a job is running under py2 or py3, but afaict that one is py2, and today's broken ones are py3 | |
| 21:02:35 | mordred | efried: that one seems to be py2 - I see a bunch of false for use python3 - and an uninstall of python3 package | |
| 21:02:46 | efried | yup | |
| 21:02:47 | efried | so | |
| 21:02:57 | efried | some zuul ancestor somehow cut over to py3 overnight | |
| 21:03:00 | efried | but mriedem and I couldn't find it. | |
| 21:03:33 | efried | now - the right thing of course is for python-openstackclient to be testing under py3 anyway. | |
| 21:03:37 | mordred | yeah | |
| 21:03:45 | mordred | I mean- luckily that patch from you isn't super big | |
| 21:03:47 | efried | those jobs should have probably been (cloned or) cut over yonks ago. | |
| 21:03:51 | mordred | yup | |
| 21:03:53 | efried | well, not yet :) | |
| 21:03:56 | mordred | haha | |
| 21:04:05 | mordred | I assume that patch is perfect and complete | |
| 21:04:24 | efried | I haven't been able to run the suite locally at all, so just have to wait for zuul to tell us whether that's all that's required to cut over. | |
| 21:05:06 | efried | mordred: actually, since you're here with your godlike powers, perhaps you could set that node up to hold when/if it fails so we can get in there and iterate? | |
| 21:05:38 | openstackgerrit | Pierre Prinetti proposed openstack/python-openstackclient master: versions: Fix 'versions show' help message https://review.opendev.org/686408 | |
| 21:05:47 | efried | still queued at the moment... | |
| 21:08:24 | mordred | efried: yeah - one sec | |
| 21:10:22 | mordred | efried: k. autohold placed | |
| 21:11:32 | efried | cool | |
| 21:26:30 | openstackgerrit | Eric Fried proposed openstack/python-openstackclient master: Fix 'versions show' helps & docs https://review.opendev.org/691989 | |
| 21:29:50 | mriedem | efried: see http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010430.html - just hit the ML, but says that openstackclient needs to support py27 for swift | |
| 21:30:27 | mriedem | osc isn't a library though so i'm not sure about all that | |
| 21:30:47 | mordred | wait - yeah. doesn't make sense that keystonemiddleware would need openstackclient | |
| 21:30:57 | mriedem | i'm replying | |
| 21:31:42 | mordred | ++ | |
| 21:31:51 | mordred | I just checked - osc is not a dep of keystonemiddleware | |
| 21:33:25 | mriedem | i'm guessing it just means devstack + osc for testing swift, but osc doesn't need to be py2 for that | |
| 21:38:31 | mordred | yeah | |
| 21:38:32 | efried | mriedem: FWIW 'openstackclient' and 'python-openstackclient' are different projects. It's likely they meant the latter even though they said the former. And in any case the former isn't a dep. | |
| 21:38:48 | mordred | neither should be a dep | |
| 21:38:54 | efried | just to add further depth to the confusion. | |
| 21:38:55 | mordred | and if either are, we should fix that :) | |
| 21:38:56 | efried | yeah. | |
| 21:39:50 | efried | mordred: http://zuul.openstack.org/stream/d39bf114ffc943c8b28a5c602395b10b?logfile=console.log I could be wrong, but this appears to be running under py2??? | |
| 21:41:07 | efried | is it possible that certain CI nodes are running py3 by default, so if this job doesn't say anything, it gets whatever it happens to land on?? | |
| 21:41:24 | efried | and all four of the runs previously noted just happened to get lucky? | |
| 21:41:30 | mordred | it really shouldn't be - we use the same images across all clouds | |
| 21:42:03 | efried | mm. So maybe I just don't know what I'm looking at. But I saw lots of "pyXX doesn't match your env" messages fly by. | |
| 21:42:30 | mordred | well - this is complicated - because there's the devstack install - and then there's the tox venv inside of which the functional tests run | |
| 21:42:56 | efried | o | |
| 21:43:25 | efried | okay, I'll stfu until it gets a bit further. | |
| 21:44:05 | mordred | looks like the tox file should have been running functional tests themselves in py3 for a while now | |
| 21:44:14 | mordred | so - this will be fun to understand :) | |