| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-21 | |||
| 16:42:48 | artom | So it should be independent of what's on the "host"? | |
| 16:42:53 | gibi | artom: you have a point | |
| 16:43:13 | gibi | artom: for some reason our ci install tox already in a venv but I lost following that track | |
| 16:43:40 | stephenfin | artom: it installs setuptools in the venv also, right? | |
| 16:44:05 | artom | Don't know... but then it'd be a separate venv for the actual nova install that pulls in decorator... | |
| 16:44:40 | gibi | I agree that something is missing from our understanding about venvs and tox install | |
| 16:45:00 | stephenfin | >>> setuptools.__file__ | |
| 16:45:06 | stephenfin | '/home/stephenfin/Development/openstack/nova/.tox/py36/lib/python3.6/site-packages/setuptools/__init__.py' | |
| 16:45:16 | sean-k-mooney | https://tox.readthedocs.io/en/latest/config.html#conf-requires | |
| 16:45:33 | sean-k-mooney | if we need to pin it we can pin the setuptools and pip version with requires | |
| 16:45:45 | bauzas | I think it's doable | |
| 16:46:03 | gibi | sean-k-mooney: good finding | |
| 16:46:03 | clarkb | tox pulls in virtualenv, virtualenv bundles setuptools for the new virtualenvs it makes | |
| 16:46:09 | sean-k-mooney | it is but fungi and clarkb advised against doing that | |
| 16:46:14 | elodilles | in my abandoned trial in the past i tried to use upper-constraints.txt for tox install, it probably could solve the issue if that works (i don't remember whether it worked) | |
| 16:46:16 | clarkb | virtualenv has an escape hatch for that but I'm not sure how tox exposes that (if at all) | |
| 16:46:24 | bauzas | https://paste.opendev.org/show/809477/ | |
| 16:46:27 | clarkb | you need to pin virtualenv is the tldr if you are going to pin something | |
| 16:46:34 | clarkb | but you have to do it when installing tox | |
| 16:46:46 | bauzas | you can create a venv without setuptools and pip it later | |
| 16:46:58 | clarkb | right but does tox support any of that? | |
| 16:47:04 | clarkb | as it is tox making the virtualenv automatically | |
| 16:47:15 | bauzas | tox uses the venv module, right?N | |
| 16:47:23 | clarkb | not by default I think it can | |
| 16:47:32 | artom | clarkb, gibi, ah, that's the missing understanding, when virtualenv creates a venv, it automatically stuffs the setuptools version it came with in that venv... right? | |
| 16:47:35 | clarkb | basically oyu have to see how much of this is configurable on the tox side to do what you want | |
| 16:47:41 | clarkb | artom: yes exactly | |
| 16:47:44 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Add migration to resolve shadow table discrepancies https://review.opendev.org/c/openstack/nova/+/805738 | |
| 16:47:45 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Walk database migrations in correct order https://review.opendev.org/c/openstack/nova/+/810291 | |
| 16:48:14 | bauzas | clarkb: you're right, I need to dig into tox | |
| 16:48:20 | stephenfin | This looks like good background reading https://tox.readthedocs.io/en/latest/example/package.html | |
| 16:48:21 | gibi | clarkb: thanks | |
| 16:48:25 | artom | Sounds like we have no choice but to pin virtualenv then | |
| 16:48:30 | stephenfin | Sounds like the escape hatches we need might be there | |
| 16:48:42 | artom | Unless there's a way to tell virtualenv which setuptools version to install in the venvs in creates | |
| 16:48:53 | fungi | if you want tox to use venv as the virtualenv backend, these days the trick is to export VIRTUALENV_CREATOR = venv in the inherited setenv in your tox.ini | |
| 16:49:21 | fungi | there used to be a tox-venv plugin, but that was deprecated/abandoned once virtualenv grew the option to call out to the venv module itself | |
| 16:49:30 | bauzas | can we use https://tox.readthedocs.io/en/latest/config.html#conf-requires ? | |
| 16:49:53 | fungi | so it's not tox calling venv directly, but rather tox calling virtualenv and then virtualenv being told to use the venv module to create the env rather than its internal implementation | |
| 16:50:13 | sean-k-mooney | i feel like the complexity of maintianing this pining outways the usefullness of the job | |
| 16:50:29 | bauzas | sean-k-mooney: that's the problem with option 2 | |
| 16:50:45 | bauzas | ideally, I'd like us to maitain setuptools in our reqs | |
| 16:50:52 | bauzas | maintain* even | |
| 16:50:59 | bauzas | but this doesn't sound easy | |
| 16:51:03 | artom | So the value of the job is what - guarantee that will still work with the oldest feasible versions of things? | |
| 16:51:13 | artom | (lower-constraints I mean) | |
| 16:51:25 | stephenfin | sean-k-mooney: We will always have pinning in the form of upper-constraints though | |
| 16:51:26 | sean-k-mooney | yes basically to make sure when we backport we dont raise the requiremetns as a result | |
| 16:51:41 | clarkb | the correct way to maintain setuptools in your requirements is via the pyproject file | |
| 16:51:44 | sean-k-mooney | stephenfin: right but for stable we are not allowed to raise miniums | |
| 16:51:47 | clarkb | not requirements | |
| 16:52:14 | stephenfin | and? we can't raise maximums either | |
| 16:52:17 | clarkb | the new pyproject file allows you to specify build time deps (possibly even tools other than setuptools) | |
| 16:52:28 | clarkb | I don't know what would be involved to make all of that work with pbr though | |
| 16:53:50 | bauzas | clarkb: how other projects consider this issue ? do they want to pin setuptools as well ? | |
| 16:54:01 | bauzas | or do they just fix the decorator issue ? | |
| 16:54:12 | bauzas | (and backport without specific care) | |
| 16:54:22 | bauzas | I guess we're not alone in the dark | |
| 16:54:31 | clarkb | msot of the fixes I've seen have been to fix the dependencies | |
| 16:54:41 | bauzas | and about backports ? | |
| 16:54:44 | clarkb | openstack governance had to switch pydot2 to pydot for example | |
| 16:55:03 | clarkb | zuul-jobs dropped support for old python3.5 which allowed it to use a newer version of a lib | |
| 16:55:27 | clarkb | I don't know how backports or stable issues have been handled | |
| 16:56:34 | bauzas | gibi: I guess you have to write something and bubble it to the community for cross-project thinking | |
| 16:57:08 | gibi | bauzas: I can write up a summary from today on the ML | |
| 16:57:09 | bauzas | that could become an epic saga, but it's worth starting it | |
| 16:57:28 | bauzas | gibi: appreciated, here lemme summarize for the audience | |
| 16:57:46 | bauzas | option 2 seems to be the better, but we struggle finding a good way to write it | |
| 16:58:22 | artom | Stupid question, but could we not just install virtualenv==<version we want> first, and *then* tox? | |
| 16:58:23 | bauzas | option 1 seems to be the alternative, option 3 seems not accepted | |
| 16:58:27 | artom | Or would tox just upgrade? | |
| 16:58:49 | gibi | artom: that could be also something to try | |
| 16:58:56 | bauzas | artom: virtualenv pulls the latest setuptools IIRC | |
| 16:59:04 | bauzas | but that's a flag | |
| 16:59:15 | gibi | bauzas: old virtualenv will pull old setuptools I guess | |
| 16:59:22 | bauzas | again, the problem is how to trigger this flag thru tox | |
| 16:59:42 | bauzas | gibi: I remember having played with it before and you're right | |
| 17:00:01 | bauzas | but you can explicitely specific to pull the latest setuptools | |
| 17:00:07 | bauzas | either way, we're at time | |
| 17:00:24 | bauzas | gibi: thanks for writing the summary and raising it to the community | |
| 17:00:34 | bauzas | folks, wrapping up | |
| 17:00:36 | bauzas | thanks | |
| 17:00:40 | bauzas | #endmeeting | |
| 17:00:41 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2021/nova.2021-09-21-16.00.log.html | |
| 17:00:41 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2021/nova.2021-09-21-16.00.txt | |
| 17:00:41 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2021/nova.2021-09-21-16.00.html | |
| 17:00:41 | opendevmeet | Meeting ended Tue Sep 21 17:00:40 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 17:00:44 | gibi | bauzas: thanks | |
| 17:01:08 | elodilles | thanks bauzas o/ | |
| 17:02:25 | stephenfin | I've already tagged people, but there are some necessary fixes for the DB migration tests here that we should probably merge before I forget about them https://review.opendev.org/c/openstack/nova/+/810291 | |
| 17:02:44 | stephenfin | The alternative is to rediscover this stuff once someone adds a DB migration in Yoga or later | |
| 17:02:57 | stephenfin | (https://review.opendev.org/c/openstack/nova/+/805738 proves that they work) | |
| 17:03:22 | bauzas | stephenfin: I just made use of the review-priority label for your change :) | |
| 17:03:32 | gibi | stephenfin: thanks, I will try to get to them | |
| 17:04:51 | opendevreview | Stephen Finucane proposed openstack/nova master: tools: Ignore bot-generated branch creation patches https://review.opendev.org/c/openstack/nova/+/810285 | |
| 17:05:06 | stephenfin | lyarwood: sorry for the dumb typo in that fixed. Bash is the worst. Fixed ^ | |
| 17:05:33 | opendevreview | Balazs Gibizer proposed openstack/placement stable/xena: [WIP] try to pin setuptools via tox.ini https://review.opendev.org/c/openstack/placement/+/810293 | |
| 17:07:13 | gibi | trying [tox]/requires ^^ | |
| 17:12:33 | bauzas | gibi: me too | |
| 17:12:35 | bauzas | \o | |