| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-12-09 | |||
| 10:27:18 | lyarwood | ah right sorry | |
| 10:27:49 | lyarwood | yeah well I get elod's point that it's just another place to look for deps but if it isn't needed outside of that tox env then I'd be okay with just listing it there in tox.ini | |
| 10:28:11 | stephenfin | In case it helps, the way I was diagnosing those lower-constraints jobs yesterday was to create a new virtualenv, update pip and run the same command as the lower-constraints tox target | |
| 10:28:20 | lyarwood | do you want to push a change on master? | |
| 10:28:35 | stephenfin | I had to do that because on Fedora 33, I get Python 3.9 in my virtualenv which isn't compatible with a few of the dependencies | |
| 10:28:53 | lyarwood | yeah I just hacked the base python version when working on this the other day | |
| 10:29:08 | lyarwood | upgraded pip and reproduced the issue | |
| 10:29:14 | lyarwood | but that's different to this issue again | |
| 10:29:23 | stephenfin | Fair | |
| 10:29:24 | lyarwood | this was just an uncapped dep dropping py2 support | |
| 10:29:53 | lyarwood | tbh I think we might need to do that until py39 is actually supported | |
| 10:30:07 | lyarwood | otherwise some of us on modern distros get stung all the damn time | |
| 10:30:17 | stephenfin | yuuup | |
| 10:30:49 | stephenfin | fwiw, you can also do this | |
| 10:31:02 | lyarwood | oh cool there's a LC failure now on stable/train as well | |
| 10:31:03 | stephenfin | TOX_CONSTRAINTS_FILE=lower-constraints.txt tox -e py36 | |
| 10:31:22 | stephenfin | sub UPPER_ for TOX_ on pre-victoria iirc | |
| 10:31:31 | stephenfin | \o/ | |
| 10:31:41 | elod | wasn't there another discussion in mailing list that the usage of basepython is discouraged? o:) | |
| 10:32:03 | stephenfin | unless this was recently, I fixed that | |
| 10:32:16 | lyarwood | ERROR: No matching distribution found for hacking<1.2.0,>=1.1.0 | |
| 10:32:23 | lyarwood | ^ stephenfin was that the LC issue you were working on? | |
| 10:33:36 | elod | this 'no matching distro' issue seems more like some mirror thing to me. (and again, I might be wrong :X) | |
| 10:33:50 | stephenfin | Not that exact, but it looks familiar. That happens because it can't match the dependencies of that hacking version with those required by other dependencies | |
| 10:33:54 | stephenfin | It's very misleading | |
| 10:34:18 | lyarwood | yeah I thought that the other day but talking to fungi we found https://review.opendev.org/c/openstack/nova/+/765824 | |
| 10:34:42 | lyarwood | stephenfin: yeah indeed it's an awful error message | |
| 10:35:40 | elod | :S | |
| 10:38:48 | lpetrut | hi, I have a quick question about the lower-constraints file: it's supposed to contain only direct dependencies, right? for example, if we need package x, which in turn requires package y, would package y need to be in lower-constraints.txt? | |
| 10:39:12 | stephenfin | lpetrut: it would, yes | |
| 10:39:47 | stephenfin | However, we haven't been very good around managing that since the tooling situation is quite poor | |
| 10:40:09 | stephenfin | So I don't think anyone is going to hold it against you in a review | |
| 10:41:18 | lpetrut | stephenfin: thanks for clearing it out. yep, it's really difficult to maintain, I was hoping to be able to limit lower-constraints to direct dependencies | |
| 10:41:57 | lpetrut | but I guess that would affect its usefulness | |
| 10:43:44 | lyarwood | stephenfin: which version of py36 are you using btw? | |
| 10:44:52 | lyarwood | stephenfin: everything is borked for me with 3.6.12 with setuptools 49.1.3 /o\ | |
| 10:45:21 | lyarwood | and https://github.com/pypa/setuptools/issues/2017 | |
| 10:45:55 | stephenfin | I was using 3.6.12, but that was with OSC, not nova | |
| 10:46:28 | lpetrut | lywarwood: looks like I'm not the only one having a hard time chasing Python dependencies :) | |
| 10:46:30 | lyarwood | yeah it's failing to install MarkupSafe==1.0 | |
| 10:46:58 | lyarwood | lpetrut: yup don't you just love python some days? :) | |
| 10:51:07 | sean-k-mooney | lowerconstriat has more then direct depencies. it was auto generated using pip freeze | |
| 10:51:20 | sean-k-mooney | we have removed some of the indirect deps but not all of them | |
| 10:52:40 | sean-k-mooney | lyarwood: just be glad your not dealing with nodejs | |
| 11:05:53 | stephenfin | lyarwood: you've probably figured this out already but https://github.com/openstack/nova/blob/stable/train/lower-constraints.txt#L37 | |
| 11:06:05 | stephenfin | https://github.com/openstack/nova/blob/stable/train/test-requirements.txt#L5 | |
| 11:06:43 | lyarwood | yeah I've been trying to prove that was the issue but I can't build a venv | |
| 11:06:51 | lyarwood | let me squash this into my other change | |
| 11:07:06 | stephenfin | I'm doing | |
| 11:07:08 | stephenfin | deactivate; rm -rf .venv; virtualenv .venv --python=python3.6; source .venv/bin/activate; pip install --upgrade pip; pip install -vvv -c lower-constraints.txt 'hacking>=1.1.0,<1.2.0' | |
| 11:07:10 | stephenfin | on repeat | |
| 11:07:20 | stephenfin | while playing with lower-constraints.txt | |
| 11:09:55 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: Cap bandit and raise hacking lower-constraint https://review.opendev.org/c/openstack/nova/+/766171 | |
| 11:09:56 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771 | |
| 11:10:16 | lyarwood | oh right because that moves flake etc | |
| 11:10:18 | lyarwood | gah | |
| 11:10:58 | lyarwood | so broken | |
| 11:11:54 | stephenfin | I wonder if we can start using train-era virtualenv and pip? | |
| 11:13:38 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: Cap bandit while also raising hacking and flake lower-constraints https://review.opendev.org/c/openstack/nova/+/766171 | |
| 11:13:39 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771 | |
| 11:14:08 | lyarwood | my issue was more with the version of setuptools being pulled in by default by virtualenv, I don't think that's tied to the version of virtualenv itself right? | |
| 11:14:19 | stephenfin | I think it is | |
| 11:14:22 | lyarwood | ah | |
| 11:14:30 | lyarwood | I thouight that was a python version thing | |
| 11:14:34 | stephenfin | I saw something from fungi...somewhere this morning | |
| 11:14:59 | sean-k-mooney | stephenfin: im not sure about that you can tell virtualenv to download and decompress setuptools | |
| 11:15:08 | sean-k-mooney | i think by default it uses your host copy | |
| 11:15:58 | sean-k-mooney | there is a --setuptools<version> flag | |
| 11:16:46 | stephenfin | if that was the case, why is the gate failing? They (Canonical) are hardly releasing new versions of pip on 18.04 still | |
| 11:16:47 | sean-k-mooney | --download might also be needed | |
| 11:16:53 | sean-k-mooney | but i think that is for latest | |
| 11:17:01 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: [stable-only] Cap bandit while also raising hacking and flake lower-constraints https://review.opendev.org/c/openstack/nova/+/766171 | |
| 11:17:02 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771 | |
| 11:17:11 | sean-k-mooney | stephenfin: is it devstack or tox | |
| 11:17:18 | stephenfin | lyarwood: this works for me http://paste.openstack.org/show/800886/ | |
| 11:17:25 | sean-k-mooney | the job that is failing | |
| 11:17:33 | sean-k-mooney | devstack install pip its self | |
| 11:17:52 | stephenfin | I didn't have to bump bandit, weirdly :-\ | |
| 11:18:11 | lyarwood | huh weird I didn't need the stestr changes | |
| 11:18:15 | lyarwood | \o/ | |
| 11:18:36 | lyarwood | either way lets see what the gate says | |
| 11:18:37 | stephenfin | lyarwood: for me, it complains that oslo.test 2.6.0 needs stestr 2.0.0 | |
| 11:18:49 | lyarwood | that makes sense | |
| 11:20:25 | lyarwood | stephenfin: oh sorry I was using your previous command | |
| 11:20:33 | lyarwood | stephenfin: right so yeah it does, let me update that now | |
| 11:20:46 | stephenfin | ah, whoops, yeah, you need to append '-r requirements.txt .' | |
| 11:21:18 | stephenfin | hahahaha http://paste.openstack.org/show/800887/ | |
| 11:21:34 | stephenfin | that's some dependency tree | |
| 11:21:47 | stephenfin | (from pipdeptree) | |
| 11:22:02 | stephenfin | sean-k-mooney: you were complaining about nodejs? ^ | |
| 11:23:00 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: [stable-only] Cap bandit while also raising hacking, flake and stestr LCs https://review.opendev.org/c/openstack/nova/+/766171 | |
| 11:23:01 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771 | |
| 11:25:23 | stephenfin | I think it's fair to say the idea of maintaining a comprehensive list of lower-constraints will die pretty soon now | |
| 11:26:02 | sean-k-mooney | stephenfin: that has many many duplicates | |
| 11:26:24 | stephenfin | unless that list is updated somewhat regularly, we're going to be playing whack-a-mole as it ages (and it's already pretty well aged) | |
| 11:26:53 | sean-k-mooney | well we are not ment to update it ever for stable brances | |
| 11:27:07 | sean-k-mooney | its only ment to be updated on master | |
| 11:27:11 | stephenfin | you've no choice here. It was wrong | |
| 11:27:20 | sean-k-mooney | it was working previously | |
| 11:27:23 | lyarwood | `working` | |