| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-04-24 | |||
| 16:11:26 | sean-k-mooney | for uwsgi | |
| 16:11:30 | dansmith | no, I didn't say that | |
| 16:11:31 | bauzas | fwiw, the u-c bump to tooz==4.0.0 is still on hold | |
| 16:11:44 | bauzas | so we may ask for a cap | |
| 16:11:45 | dansmith | I said some of our packages we install from bindep to avoid compiling things like mysqlclient (AFAIK) | |
| 16:11:54 | sean-k-mooney | ah | |
| 16:12:06 | sean-k-mooney | ya sorry you did mention mysql | |
| 16:12:11 | sean-k-mooney | not uwsgi | |
| 16:12:16 | dansmith | I think the uwsgi python module *is* tied directly to the python version though | |
| 16:12:27 | dansmith | so we'd need a python3.9-uwsgi-module-python3 (or whatever the name is) | |
| 16:12:32 | sean-k-mooney | at least in the disto package i think that is correct | |
| 16:13:16 | dansmith | uwsgi-plugin-python3 | |
| 16:13:28 | dansmith | this ^ is compiled directly against the base python3, not 3.9 | |
| 16:14:46 | dansmith | see the last answer here: https://stackoverflow.com/questions/68413988/building-a-uwsgi-plugin-for-python-3-9-failed-for-older-version-it-works-are-th | |
| 16:14:55 | dansmith | might be able to pip install uwsgi with the 3.9 python | |
| 16:14:55 | sean-k-mooney | for the devstack venv path i think you can override that with https://review.opendev.org/c/openstack/devstack/+/558930/26/files/apache-horizon.template | |
| 16:15:08 | sean-k-mooney | WSGIPYTHONHOME | |
| 16:15:14 | dansmith | I' | |
| 16:15:17 | dansmith | I am pretty sure not | |
| 16:15:30 | dansmith | uwsgi runs the python interpreter internally, AFAIK | |
| 16:15:44 | dansmith | that's how it provides its phantom importable modules, etc | |
| 16:15:50 | sean-k-mooney | i know i was doing somethign to get it too work in that seriess | |
| 16:15:55 | sean-k-mooney | but i dotn recall | |
| 16:16:31 | sean-k-mooney | oh it was this https://review.opendev.org/c/openstack/devstack/+/558930/26/functions-common#1605 | |
| 16:16:38 | sean-k-mooney | anywya that wont really help here | |
| 16:16:54 | sean-k-mooney | for the ceph job we whould rever back to 3.8 i guess | |
| 16:17:05 | sean-k-mooney | if we are keeping it on focal | |
| 16:18:28 | sean-k-mooney | mixing the workaround to run with uwsgin for a venv and the ceph job is not helpful i tought there was a simple fix in that but no | |
| 16:19:17 | bauzas | stupid question but I guess we can't block a specific tooz version in our own requirements.txt file ? | |
| 16:19:45 | sean-k-mooney | we can but we are not ment too | |
| 16:19:53 | sean-k-mooney | we can do tooz!=whatever | |
| 16:20:08 | sean-k-mooney | that will cause use to downgrade if its already installed | |
| 16:20:12 | bauzas | and tooz<=version ? | |
| 16:20:22 | sean-k-mooney | we dont want to cap | |
| 16:20:24 | dansmith | right but we'll still install the initial version first, then downgrade it, | |
| 16:20:31 | sean-k-mooney | but we can block know broken ones | |
| 16:20:38 | dansmith | and if someone that runs after us has >=4.0 it will be re-re-installed | |
| 16:20:40 | dansmith | so that's not the best solution, IMHO | |
| 16:21:00 | bauzas | I was just trying to buy us some time :) | |
| 16:21:19 | dansmith | yeah, it might be a quick fix | |
| 16:31:18 | bauzas | fwiw https://review.opendev.org/c/openstack/requirements/+/881329 | |
| 16:31:27 | bauzas | feel free to vote (loudly) | |
| 16:34:24 | sean-k-mooney | dansmith: was the same done for castalan? for glance? | |
| 16:34:48 | sean-k-mooney | its currently castellan===4.1.0 | |
| 16:34:49 | dansmith | we just dropped the 38 jobs | |
| 16:34:53 | sean-k-mooney | ah ok | |
| 16:41:55 | bauzas | so, before I leave, https://review.opendev.org/c/openstack/requirements/+/881329 should unblock the gate | |
| 16:42:28 | bauzas | this will give us time, ideally to investigate the uwsgi 3.9 support | |
| 16:42:52 | bauzas | that being said, I'm OK with dropping 3.8 support for the functional tests | |
| 16:54:55 | sean-k-mooney | i thihnk we should just see if we can deploy devstack with ceph on 22.04 using cephadm | |
| 16:55:19 | sean-k-mooney | if we can we can swap the nova ceph jobs to that | |
| 16:55:37 | sean-k-mooney | i can push a DNM patch to test that but i proably wont have time to actully try it myself | |
| 16:57:00 | dansmith | there's a patch started already IIRC | |
| 16:59:11 | dansmith | ah that was an old DNM I was thinking about | |
| 21:59:11 | opendevreview | Lin Yang proposed openstack/os-traits master: CPU: add traits for new X86 feature "AMX" https://review.opendev.org/c/openstack/os-traits/+/868149 | |
| 22:18:12 | dansmith | um, neutron now requires >=3.9 as well? https://zuul.opendev.org/t/openstack/build/c7a96046411c4e05a669003bb67b299d | |
| 22:18:48 | dansmith | yup: https://review.opendev.org/c/openstack/neutron/+/881333 | |
| 22:19:22 | clarkb | looks like they did it in response to the tooz update | |
| 22:23:02 | opendevreview | Dan Smith proposed openstack/nova master: Remove silent failure to find a node on rebuild https://review.opendev.org/c/openstack/nova/+/880632 | |
| 22:23:02 | opendevreview | Dan Smith proposed openstack/nova master: Stop ignoring missing compute nodes in claims https://review.opendev.org/c/openstack/nova/+/880633 | |
| 22:23:03 | opendevreview | Dan Smith proposed openstack/nova master: Make focal job non-voting https://review.opendev.org/c/openstack/nova/+/881409 | |
| 22:23:10 | dansmith | clarkb: yeah, not sure if they were trying to help with that or not... | |
| 22:23:32 | dansmith | so I imagine that's going to take a while to undo, if at all, so this ^ marks that job as non-voting | |
| 22:24:15 | dansmith | I'm also not sure why we have that focal job, tbh | |
| 22:24:51 | dansmith | ah: https://review.opendev.org/c/openstack/nova/+/861111 | |
| 22:25:01 | dansmith | so I can probably just nuke it instead of making it n-v since it'll fail | |
| 22:27:01 | opendevreview | Dan Smith proposed openstack/nova master: Remove focal job for 2023.2 https://review.opendev.org/c/openstack/nova/+/881409 | |
| 22:27:01 | opendevreview | Dan Smith proposed openstack/nova master: Remove silent failure to find a node on rebuild https://review.opendev.org/c/openstack/nova/+/880632 | |
| 22:27:02 | opendevreview | Dan Smith proposed openstack/nova master: Stop ignoring missing compute nodes in claims https://review.opendev.org/c/openstack/nova/+/880633 | |
| 22:28:13 | dansmith | gmann: ^ | |
| 22:42:56 | dansmith | oh right, the ceph jobs are focal still, which means we're blocked until neutron reverts that or we have to mark ceph jobs as n-v | |
| 22:42:56 | dansmith | fml | |
| 23:07:55 | artom | dansmith, hey, just going to drop this here drive-by style: https://github.com/openstack/devstack-plugin-ceph/blob/master/devstack/plugin.sh#L5-L9 | |
| 23:08:18 | dansmith | artom: what about it? | |
| 23:08:21 | artom | Apparently this cephadm thing is the New Correct Way of installing ceph, essentially bypassing all the Ubuntu Ceph RPMs | |
| 23:08:50 | dansmith | yeah we know :) | |
| 23:09:01 | artom | I'll follow up with an email tomorrow, but that's what I got out of a chat with Giulio | |
| 23:09:36 | dansmith | last I checked (when we discussed at the recent ptg) it either didn't work correctly, or something about what we need for our regular jobs doesn't work with that mode | |
| 23:09:46 | dansmith | I don't know the details, but that's what we need to get worked out | |
| 23:09:58 | artom | Ack - more info (and contact people) to follow tomorrow | |
| 23:10:26 | dansmith | I just rechecked this, which frickler refreshed recently: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315 | |
| 23:10:32 | dansmith | which I'm hoping is supposed to actually work | |
| 23:12:23 | dansmith | you see the jobs there are still deploying without that flag and on focal "until it works" and that was more recent than the cephadm patches | |
| 23:12:39 | dansmith | I mean, before the revert of course | |
| 23:13:34 | dansmith | I want to say it's something like installing with cephadm doesn't leave the ceph config visible which nova needs to resolve the cluster or something like that | |
| 23:40:16 | dansmith | artom: yeah, the cephadm-based job fails pretty early and pretty hard: https://zuul.opendev.org/t/openstack/build/cce90efc7177459e9eebdab966bab75c | |
| 23:40:44 | clarkb | the jammy ceph version is also new enough for nova iirc | |
| 23:41:20 | dansmith | clarkb: right, the reason we didn't do that before was because they didn't yet have jammy packages IIRC, but I think they do now | |
| 23:41:46 | dansmith | either way, it's failing pretty hard | |
| 23:42:09 | clarkb | I think there was actually a misunderstanding. Everyone was looking for cloud archive packages for some reason, but the base distro had packages that were quite new | |
| 23:42:10 | dansmith | I mean the distro-package-based one | |
| 23:42:32 | dansmith | clarkb: no I think people were looking for ceph.org packages, or at least that's what I remembered | |
| 23:42:47 | clarkb | ya that was the first issue, then they looked for UCA packages and didn't find those either | |
| 23:42:49 | dansmith | but perhaps that was because jammy already had them and that's just how we did it before, I dunno | |
| 23:42:53 | dansmith | ack okay | |
| 23:42:57 | clarkb | but the base distro has newish packages that can probabl ybe made to work | |
| 23:43:43 | dansmith | the job running on jammy is about to finish, but it had lots of tempest fail in it | |
| 23:43:52 | dansmith | from this: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315?tab=change-view-tab-header-zuul-results-summary | |
| 23:47:17 | dansmith | qemu: module block-block-rbd not found | |
| 23:47:20 | dansmith | in nova compute's log | |