| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-21 | |||
| 16:28:06 | bauzas | example https://review.opendev.org/c/openstack/releases/+/808706 | |
| 16:28:44 | sean-k-mooney | i keep an eye on the os-vif ones anyway but someone on the stable team might make more sense | |
| 16:28:52 | bauzas | sean-k-mooney: appreciated | |
| 16:29:19 | bauzas | I'll make the changes in the appropriate repo so you'd get automatically CC'd | |
| 16:29:29 | gibi | sean-k-mooney: thank you | |
| 16:29:53 | sean-k-mooney | no worries | |
| 16:29:58 | elodilles | well, as I also a release core and also propose nova releases lately it would be weird to propose and approve my own patches and then +W o:) | |
| 16:30:36 | bauzas | elodilles: heh, depending whether you're schizophrenic, this could work | |
| 16:30:47 | elodilles | :D | |
| 16:31:17 | bauzas | ok, moving on | |
| 16:31:22 | bauzas | sean-k-mooney: thanks again | |
| 16:31:38 | bauzas | now the big discussion | |
| 16:31:47 | bauzas | pasting the whole section | |
| 16:31:49 | bauzas | (gibi): gathering opinions about the current lower-constraints failure. | |
| 16:31:56 | bauzas | bottom line: on stable branches we are installing tox which installs virtualenv which bundles setuptools unconstrained. This now leads to that we cannot install decorator 3.4.0 on stable any more as it depends on "user_2to3" from setuptools but the recent setuptools 58.0 removed support for that. | |
| 16:32:02 | bauzas | Options to resolve the situation bump decorator major version from 3.4.0 to 4.0.0 on stable branches. Does it against stable policy? pin virtualenv version on stable during tox install disable lower-constraints testing | |
| 16:32:14 | bauzas | shit, I pasted wrong | |
| 16:32:20 | bauzas | gibi: your turn | |
| 16:32:27 | gibi | let me untangle that | |
| 16:32:27 | bauzas | explain the 3 options | |
| 16:32:50 | gibi | so option 1) bump major version of decorator from 3.4 to 4.0 on stable. Is it allowed on stable? | |
| 16:33:01 | gibi | option 2) pin virtualenv during tox install | |
| 16:33:11 | gibi | option 3) disable lower-constraints job | |
| 16:33:28 | gibi | I personally think that using unconstrained setuptools on stable is dangerous | |
| 16:33:36 | gibi | so I would go with 2) long turn | |
| 16:33:39 | gibi | term | |
| 16:33:53 | lyarwood | Agreed, 2 would be my choice | |
| 16:33:53 | artom | Yeah, seems like 2 is the safest... what's the danger with it? Is there a catch? | |
| 16:33:57 | bauzas | gibi: let's explain which stable branches again are impacted | |
| 16:34:13 | bauzas | you already told but this doesn't harm to tell again | |
| 16:34:14 | gibi | so in nova we are impacted stable/ussuri and older | |
| 16:34:16 | sean-k-mooney | gibi: technially i dont think we are allowe to bump miniums on stable on the other hand we already did it a while ago for the inital lower constraitnts fix | |
| 16:34:26 | gibi | on placement we are impacted on master but the fix has been landed | |
| 16:34:40 | gibi | on stable/xena we could release RC2 with the bump | |
| 16:34:47 | elodilles | I also like option 2, and actually found some (abandoned) trial from the past that was similar https://review.opendev.org/q/topic:constrain-tox-install | |
| 16:34:50 | gibi | but on stable/wallaby and back the same quertion applies | |
| 16:35:12 | sean-k-mooney | from a disto point of view 2 is not great | |
| 16:35:22 | sean-k-mooney | since they cannot pin them the same way | |
| 16:35:25 | bauzas | option 1 has a question I can't answer | |
| 16:35:32 | sean-k-mooney | but most distros also dont use lower constratints | |
| 16:35:33 | gibi | does LTS distros use unconstrained setuptools as well? | |
| 16:35:54 | bauzas | can we deliver a .y release by bumping the decorator dependency without breaking the semver rules ? | |
| 16:36:07 | sean-k-mooney | rhel ignored the upper and lower constraitnts | |
| 16:36:14 | sean-k-mooney | debian used to look at lower | |
| 16:36:25 | sean-k-mooney | ubunut used to look at upper | |
| 16:36:46 | sean-k-mooney | we can try 2 but i feel like we are going to end up with 3 | |
| 16:37:00 | gibi | sean-k-mooney: sure we could end up with 3 for different reasons :) | |
| 16:37:12 | artom | So 2 for distros means "maintain this old version of virtualenv in your repos", right? | |
| 16:37:22 | elodilles | bauzas: I think we can, but it's probably not fortunate. semver says that req changes needs MINOR / .y version bump | |
| 16:37:28 | artom | But... don't we already have upper-constraints that does the exact same thing? | |
| 16:37:36 | gibi | artom: yes, as virtualenv bundles setuptools | |
| 16:37:56 | bauzas | elodilles: so option 1 doesn't break stable policy per se | |
| 16:38:00 | gibi | artom: this is tricky as the tox install in our CI does not use constraint file | |
| 16:38:14 | gibi | artom: we install tox, then with tox we install deps | |
| 16:38:25 | gibi | artom: but when we install tox it pulls in setuptools | |
| 16:38:59 | artom | Oh, so this is before any -constraints is applied | |
| 16:39:15 | bauzas | the problem is with the venv, not the dependencies | |
| 16:39:20 | bauzas | artom: right | |
| 16:39:30 | elodilles | bauzas: I also haven't found that written, but I remember that we tried to avoid req changes | |
| 16:39:30 | bauzas | because of the bundle | |
| 16:39:48 | bauzas | but, there are ways to create venvs without bundling setuptools, right? | |
| 16:40:05 | elodilles | bauzas: and the problem is that we will face the same issue whenever something new things comes in | |
| 16:40:12 | artom | Wait, I though the decorator==3.4.0 install happened *inside* the venv, as a dependency? | |
| 16:40:29 | bauzas | elodilles: agreed, option 1 only fixes the decorator issue and doesn't resolve any other issue | |
| 16:40:49 | bauzas | I'm also in favor of option 2 | |
| 16:41:05 | gibi | artom: to have a venv where you can install deps, you have to have the venv package installed | |
| 16:41:08 | bauzas | but I wonder whether we could just tell tox to create venvs without bundling setuptools | |
| 16:41:35 | gibi | bauzas: I think if you have a venv without setuptools then you dont have pip in the venv either | |
| 16:41:35 | stephenfin | just weighing in, 3 isn't an ideal solution either | |
| 16:41:48 | gibi | bauzas: so you cannot install additional things with pip | |
| 16:41:56 | bauzas | gibi: are you sure ? | |
| 16:42:02 | gibi | bauzas: 75% | |
| 16:42:05 | bauzas | I think it does install pip | |
| 16:42:14 | gibi | I think pip depends on setuptools | |
| 16:42:18 | stephenfin | in this case it's the lower-constraint that has broken, however, it's conceivable that the upper constraint for a particularly old release could also become incompatible with setuptools | |
| 16:42:20 | gibi | but I could be wrong | |
| 16:42:32 | gibi | stephenfin: good point | |
| 16:42:35 | artom | gibi, right, so we install tox on the "host", which pulls in unconstrained setuptools, which then break installing decorator==3.4.0, my question is, aren't we installing decorator==3.4.0 inside the venv? | |
| 16:42:47 | gibi | artom: hm, | |
| 16:42:48 | artom | So it should be independent of what's on the "host"? | |
| 16:42:53 | gibi | artom: you have a point | |
| 16:43:13 | gibi | artom: for some reason our ci install tox already in a venv but I lost following that track | |
| 16:43:40 | stephenfin | artom: it installs setuptools in the venv also, right? | |
| 16:44:05 | artom | Don't know... but then it'd be a separate venv for the actual nova install that pulls in decorator... | |
| 16:44:40 | gibi | I agree that something is missing from our understanding about venvs and tox install | |
| 16:45:00 | stephenfin | >>> setuptools.__file__ | |
| 16:45:06 | stephenfin | '/home/stephenfin/Development/openstack/nova/.tox/py36/lib/python3.6/site-packages/setuptools/__init__.py' | |
| 16:45:16 | sean-k-mooney | https://tox.readthedocs.io/en/latest/config.html#conf-requires | |
| 16:45:33 | sean-k-mooney | if we need to pin it we can pin the setuptools and pip version with requires | |
| 16:45:45 | bauzas | I think it's doable | |
| 16:46:03 | gibi | sean-k-mooney: good finding | |
| 16:46:03 | clarkb | tox pulls in virtualenv, virtualenv bundles setuptools for the new virtualenvs it makes | |
| 16:46:09 | sean-k-mooney | it is but fungi and clarkb advised against doing that | |
| 16:46:14 | elodilles | in my abandoned trial in the past i tried to use upper-constraints.txt for tox install, it probably could solve the issue if that works (i don't remember whether it worked) | |
| 16:46:16 | clarkb | virtualenv has an escape hatch for that but I'm not sure how tox exposes that (if at all) | |
| 16:46:24 | bauzas | https://paste.opendev.org/show/809477/ | |
| 16:46:27 | clarkb | you need to pin virtualenv is the tldr if you are going to pin something | |
| 16:46:34 | clarkb | but you have to do it when installing tox | |
| 16:46:46 | bauzas | you can create a venv without setuptools and pip it later | |
| 16:46:58 | clarkb | right but does tox support any of that? | |