Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-12-09
10:45:21 lyarwood and https://github.com/pypa/setuptools/issues/2017
10:45:55 stephenfin I was using 3.6.12, but that was with OSC, not nova
10:46:28 lpetrut lywarwood: looks like I'm not the only one having a hard time chasing Python dependencies :)
10:46:30 lyarwood yeah it's failing to install MarkupSafe==1.0
10:46:58 lyarwood lpetrut: yup don't you just love python some days? :)
10:51:07 sean-k-mooney lowerconstriat has more then direct depencies. it was auto generated using pip freeze
10:51:20 sean-k-mooney we have removed some of the indirect deps but not all of them
10:52:40 sean-k-mooney lyarwood: just be glad your not dealing with nodejs
11:05:53 stephenfin lyarwood: you've probably figured this out already but https://github.com/openstack/nova/blob/stable/train/lower-constraints.txt#L37
11:06:05 stephenfin https://github.com/openstack/nova/blob/stable/train/test-requirements.txt#L5
11:06:43 lyarwood yeah I've been trying to prove that was the issue but I can't build a venv
11:06:51 lyarwood let me squash this into my other change
11:07:06 stephenfin I'm doing
11:07:08 stephenfin deactivate; rm -rf .venv; virtualenv .venv --python=python3.6; source .venv/bin/activate; pip install --upgrade pip; pip install -vvv -c lower-constraints.txt 'hacking>=1.1.0,<1.2.0'
11:07:10 stephenfin on repeat
11:07:20 stephenfin while playing with lower-constraints.txt
11:09:55 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: Cap bandit and raise hacking lower-constraint https://review.opendev.org/c/openstack/nova/+/766171
11:09:56 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771
11:10:16 lyarwood oh right because that moves flake etc
11:10:18 lyarwood gah
11:10:58 lyarwood so broken
11:11:54 stephenfin I wonder if we can start using train-era virtualenv and pip?
11:13:38 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: Cap bandit while also raising hacking and flake lower-constraints https://review.opendev.org/c/openstack/nova/+/766171
11:13:39 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771
11:14:08 lyarwood my issue was more with the version of setuptools being pulled in by default by virtualenv, I don't think that's tied to the version of virtualenv itself right?
11:14:19 stephenfin I think it is
11:14:22 lyarwood ah
11:14:30 lyarwood I thouight that was a python version thing
11:14:34 stephenfin I saw something from fungi...somewhere this morning
11:14:59 sean-k-mooney stephenfin: im not sure about that you can tell virtualenv to download and decompress setuptools
11:15:08 sean-k-mooney i think by default it uses your host copy
11:15:58 sean-k-mooney there is a --setuptools<version> flag
11:16:46 stephenfin if that was the case, why is the gate failing? They (Canonical) are hardly releasing new versions of pip on 18.04 still
11:16:47 sean-k-mooney --download might also be needed
11:16:53 sean-k-mooney but i think that is for latest
11:17:01 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: [stable-only] Cap bandit while also raising hacking and flake lower-constraints https://review.opendev.org/c/openstack/nova/+/766171
11:17:02 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771
11:17:11 sean-k-mooney stephenfin: is it devstack or tox
11:17:18 stephenfin lyarwood: this works for me http://paste.openstack.org/show/800886/
11:17:25 sean-k-mooney the job that is failing
11:17:33 sean-k-mooney devstack install pip its self
11:17:52 stephenfin I didn't have to bump bandit, weirdly :-\
11:18:11 lyarwood huh weird I didn't need the stestr changes
11:18:15 lyarwood \o/
11:18:36 lyarwood either way lets see what the gate says
11:18:37 stephenfin lyarwood: for me, it complains that oslo.test 2.6.0 needs stestr 2.0.0
11:18:49 lyarwood that makes sense
11:20:25 lyarwood stephenfin: oh sorry I was using your previous command
11:20:33 lyarwood stephenfin: right so yeah it does, let me update that now
11:20:46 stephenfin ah, whoops, yeah, you need to append '-r requirements.txt .'
11:21:18 stephenfin hahahaha http://paste.openstack.org/show/800887/
11:21:34 stephenfin that's some dependency tree
11:21:47 stephenfin (from pipdeptree)
11:22:02 stephenfin sean-k-mooney: you were complaining about nodejs? ^
11:23:00 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: [stable-only] Cap bandit while also raising hacking, flake and stestr LCs https://review.opendev.org/c/openstack/nova/+/766171
11:23:01 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771
11:25:23 stephenfin I think it's fair to say the idea of maintaining a comprehensive list of lower-constraints will die pretty soon now
11:26:02 sean-k-mooney stephenfin: that has many many duplicates
11:26:24 stephenfin unless that list is updated somewhat regularly, we're going to be playing whack-a-mole as it ages (and it's already pretty well aged)
11:26:53 sean-k-mooney well we are not ment to update it ever for stable brances
11:27:07 sean-k-mooney its only ment to be updated on master
11:27:11 stephenfin you've no choice here. It was wrong
11:27:20 sean-k-mooney it was working previously
11:27:23 lyarwood `working`
11:27:26 lyarwood it wasn't
11:27:29 stephenfin no, it wasn't
11:27:32 lyarwood it's been borked for a while looking at this
11:27:34 sean-k-mooney it was passing ci
11:27:47 lyarwood pip wasn't resolving the deps correctly
11:27:59 stephenfin def test_advanced_feature(self):
11:27:59 stephenfin pass
11:27:59 lyarwood it does now and so it's failing in CI
11:28:00 sean-k-mooney because of the constrits file?
11:28:08 stephenfin advanced feature is working :)
11:28:11 lyarwood the new resolver appears to be doing things correctly
11:28:23 lyarwood the old one didn't
11:28:30 stephenfin previously, the resolver didn't go more than one dependency deep
11:28:38 sean-k-mooney this is a behviaor change
11:28:38 lyarwood haha really
11:28:45 sean-k-mooney we should not be usein gthe new resolve on stable
11:28:56 stephenfin so if you required foo=1.0.0 and bar=2.0.0, but bar required foo=1.2.0, it would work
11:29:04 stephenfin and it no longer will
11:29:14 lyarwood we capped at 20. something but it looks like this has been backported in pip?
11:29:23 stephenfin this is 20.3
11:29:26 lyarwood so unless we lower the cap again
11:29:36 sean-k-mooney stephenfin: the behvioar of the old resovler was if somethign is listed twice we use the first value
11:29:37 stephenfin so we need to cap at less than that
11:29:40 lyarwood yeah I thought 21. broke us
11:29:49 lyarwood with the new resolver
11:29:52 sean-k-mooney that was why the order of deps mattered
11:30:10 stephenfin nah, they clearly don't use semver this isn't a major release
11:30:15 stephenfin *to say this
11:30:39 sean-k-mooney so we need to cap pip right
11:30:49 stephenfin ah, it's calver
11:30:49 sean-k-mooney and not modify the lower constraits
11:31:22 stephenfin we're not changing anything
11:31:29 lyarwood oh it was 20.3
11:31:44 sean-k-mooney stephenfin: you not going to modify lower-constraits on stable. ok
11:31:48 lyarwood https://review.opendev.org/c/openstack/devstack/+/764803 was what I was thinking about in devstack
11:31:59 stephenfin no, we are, but it won't change anything

Earlier   Later