Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2023-04-24
16:19:53 sean-k-mooney we can do tooz!=whatever
16:20:08 sean-k-mooney that will cause use to downgrade if its already installed
16:20:12 bauzas and tooz<=version ?
16:20:22 sean-k-mooney we dont want to cap
16:20:24 dansmith right but we'll still install the initial version first, then downgrade it,
16:20:31 sean-k-mooney but we can block know broken ones
16:20:38 dansmith and if someone that runs after us has >=4.0 it will be re-re-installed
16:20:40 dansmith so that's not the best solution, IMHO
16:21:00 bauzas I was just trying to buy us some time :)
16:21:19 dansmith yeah, it might be a quick fix
16:31:18 bauzas fwiw https://review.opendev.org/c/openstack/requirements/+/881329
16:31:27 bauzas feel free to vote (loudly)
16:34:24 sean-k-mooney dansmith: was the same done for castalan? for glance?
16:34:48 sean-k-mooney its currently castellan===4.1.0
16:34:49 dansmith we just dropped the 38 jobs
16:34:53 sean-k-mooney ah ok
16:41:55 bauzas so, before I leave, https://review.opendev.org/c/openstack/requirements/+/881329 should unblock the gate
16:42:28 bauzas this will give us time, ideally to investigate the uwsgi 3.9 support
16:42:52 bauzas that being said, I'm OK with dropping 3.8 support for the functional tests
16:54:55 sean-k-mooney i thihnk we should just see if we can deploy devstack with ceph on 22.04 using cephadm
16:55:19 sean-k-mooney if we can we can swap the nova ceph jobs to that
16:55:37 sean-k-mooney i can push a DNM patch to test that but i proably wont have time to actully try it myself
16:57:00 dansmith there's a patch started already IIRC
16:59:11 dansmith ah that was an old DNM I was thinking about
21:59:11 opendevreview Lin Yang proposed openstack/os-traits master: CPU: add traits for new X86 feature "AMX" https://review.opendev.org/c/openstack/os-traits/+/868149
22:18:12 dansmith um, neutron now requires >=3.9 as well? https://zuul.opendev.org/t/openstack/build/c7a96046411c4e05a669003bb67b299d
22:18:48 dansmith yup: https://review.opendev.org/c/openstack/neutron/+/881333
22:19:22 clarkb looks like they did it in response to the tooz update
22:23:02 opendevreview Dan Smith proposed openstack/nova master: Stop ignoring missing compute nodes in claims https://review.opendev.org/c/openstack/nova/+/880633
22:23:02 opendevreview Dan Smith proposed openstack/nova master: Remove silent failure to find a node on rebuild https://review.opendev.org/c/openstack/nova/+/880632
22:23:03 opendevreview Dan Smith proposed openstack/nova master: Make focal job non-voting https://review.opendev.org/c/openstack/nova/+/881409
22:23:10 dansmith clarkb: yeah, not sure if they were trying to help with that or not...
22:23:32 dansmith so I imagine that's going to take a while to undo, if at all, so this ^ marks that job as non-voting
22:24:15 dansmith I'm also not sure why we have that focal job, tbh
22:24:51 dansmith ah: https://review.opendev.org/c/openstack/nova/+/861111
22:25:01 dansmith so I can probably just nuke it instead of making it n-v since it'll fail
22:27:01 opendevreview Dan Smith proposed openstack/nova master: Remove silent failure to find a node on rebuild https://review.opendev.org/c/openstack/nova/+/880632
22:27:01 opendevreview Dan Smith proposed openstack/nova master: Remove focal job for 2023.2 https://review.opendev.org/c/openstack/nova/+/881409
22:27:02 opendevreview Dan Smith proposed openstack/nova master: Stop ignoring missing compute nodes in claims https://review.opendev.org/c/openstack/nova/+/880633
22:28:13 dansmith gmann: ^
22:42:56 dansmith fml
22:42:56 dansmith oh right, the ceph jobs are focal still, which means we're blocked until neutron reverts that or we have to mark ceph jobs as n-v
23:07:55 artom dansmith, hey, just going to drop this here drive-by style: https://github.com/openstack/devstack-plugin-ceph/blob/master/devstack/plugin.sh#L5-L9
23:08:18 dansmith artom: what about it?
23:08:21 artom Apparently this cephadm thing is the New Correct Way of installing ceph, essentially bypassing all the Ubuntu Ceph RPMs
23:08:50 dansmith yeah we know :)
23:09:01 artom I'll follow up with an email tomorrow, but that's what I got out of a chat with Giulio
23:09:36 dansmith last I checked (when we discussed at the recent ptg) it either didn't work correctly, or something about what we need for our regular jobs doesn't work with that mode
23:09:46 dansmith I don't know the details, but that's what we need to get worked out
23:09:58 artom Ack - more info (and contact people) to follow tomorrow
23:10:26 dansmith I just rechecked this, which frickler refreshed recently: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315
23:10:32 dansmith which I'm hoping is supposed to actually work
23:12:23 dansmith you see the jobs there are still deploying without that flag and on focal "until it works" and that was more recent than the cephadm patches
23:12:39 dansmith I mean, before the revert of course
23:13:34 dansmith I want to say it's something like installing with cephadm doesn't leave the ceph config visible which nova needs to resolve the cluster or something like that
23:40:16 dansmith artom: yeah, the cephadm-based job fails pretty early and pretty hard: https://zuul.opendev.org/t/openstack/build/cce90efc7177459e9eebdab966bab75c
23:40:44 clarkb the jammy ceph version is also new enough for nova iirc
23:41:20 dansmith clarkb: right, the reason we didn't do that before was because they didn't yet have jammy packages IIRC, but I think they do now
23:41:46 dansmith either way, it's failing pretty hard
23:42:09 clarkb I think there was actually a misunderstanding. Everyone was looking for cloud archive packages for some reason, but the base distro had packages that were quite new
23:42:10 dansmith I mean the distro-package-based one
23:42:32 dansmith clarkb: no I think people were looking for ceph.org packages, or at least that's what I remembered
23:42:47 clarkb ya that was the first issue, then they looked for UCA packages and didn't find those either
23:42:49 dansmith but perhaps that was because jammy already had them and that's just how we did it before, I dunno
23:42:53 dansmith ack okay
23:42:57 clarkb but the base distro has newish packages that can probabl ybe made to work
23:43:43 dansmith the job running on jammy is about to finish, but it had lots of tempest fail in it
23:43:52 dansmith from this: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315?tab=change-view-tab-header-zuul-results-summary
23:47:17 dansmith qemu: module block-block-rbd not found
23:47:20 dansmith in nova compute's log
23:47:33 sean-k-mooney[m] dansmith: i think you just need to add the ensure podman role to a pre playbook
23:48:20 dansmith sean-k-mooney[m]: could be, but my point is, I don't think that job is like super healthy and well-tested such that switching to it is the obvious choice :)
23:48:48 sean-k-mooney[m] hehe ya...
23:52:23 sean-k-mooney[m] so googleing a little i think there is some issue with cephadm assuming rpm based distors but its runing on ubuntu
23:52:45 sean-k-mooney[m] you can correct this via containers.conf apprently
23:53:15 dansmith yeah idk, but if jammy has new enough ceph that seems like it might be the easiest path to it working
23:53:32 dansmith although we must also be missing that qemu module
23:53:40 sean-k-mooney[m] it is older then whats on ceph.com
23:53:52 sean-k-mooney[m] but its relitivly recent
23:54:05 dansmith but quincy is the release they're trying to install and that's it, AFAICT
23:54:07 sean-k-mooney[m] i think manilla wanted to have a newer verion
23:54:16 dansmith however, I think we're going to have to ask neutron to revert the pin until we figure it out regardless
23:55:11 dansmith it's going to be ugly because they merged it to like all their repos
23:57:21 sean-k-mooney[m] ya i didnt actully get time to look at any of this today
23:57:59 sean-k-mooney[m] i wanted to check if anything had progress before i went to sleep but not much it seams thanks for rechecking that devstack patch
#openstack-nova - 2023-04-25
00:37:45 gmann dansmith: elodilles has patch up to remove all py38 jobs including focal job https://review.opendev.org/c/openstack/nova/+/881339
04:49:59 ykarel dansmith, only merged in neutron repo, others are still unmerged, i pushed the revert
06:55:00 bauzas catching up this night's conversations
07:17:13 opendevreview Balazs Gibizer proposed openstack/nova master: DNM: Revert "Temporary skip some volume detach test in nova-lvm job" https://review.opendev.org/c/openstack/nova/+/881389
08:49:47 gibi bauzas: I need to leave early today so I will miss the nova meeting
09:04:48 bauzas gibi: ack, no worries
10:01:18 opendevreview Amit Uniyal proposed openstack/nova master: WIP: Reproducer for dangling volumes https://review.opendev.org/c/openstack/nova/+/881457
11:27:41 opendevreview Merged openstack/nova stable/yoga: Handle InstanceInvalidState exception https://review.opendev.org/c/openstack/nova/+/872117
13:01:10 dansmith gmann: I don't see that that ever passed
13:02:37 dansmith using the alternate python interpreter on focal seems less good to me than just fixing the ceph job
13:02:51 dansmith and also not necessary if people don't prohibit 3.8
13:03:01 sean-k-mooney dansmith: ya i suggested it because it used to work but it obviously got broken at some point
13:03:10 sean-k-mooney i was hoping it woudl be a quick fix to unblock the gate
13:03:21 sean-k-mooney just makeign the job use 22.04 would be my preference too
13:08:35 bauzas yup, let's try to use Jammy

Earlier   Later