| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-04-24 | |||
| 23:09:36 | dansmith | last I checked (when we discussed at the recent ptg) it either didn't work correctly, or something about what we need for our regular jobs doesn't work with that mode | |
| 23:09:46 | dansmith | I don't know the details, but that's what we need to get worked out | |
| 23:09:58 | artom | Ack - more info (and contact people) to follow tomorrow | |
| 23:10:26 | dansmith | I just rechecked this, which frickler refreshed recently: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315 | |
| 23:10:32 | dansmith | which I'm hoping is supposed to actually work | |
| 23:12:23 | dansmith | you see the jobs there are still deploying without that flag and on focal "until it works" and that was more recent than the cephadm patches | |
| 23:12:39 | dansmith | I mean, before the revert of course | |
| 23:13:34 | dansmith | I want to say it's something like installing with cephadm doesn't leave the ceph config visible which nova needs to resolve the cluster or something like that | |
| 23:40:16 | dansmith | artom: yeah, the cephadm-based job fails pretty early and pretty hard: https://zuul.opendev.org/t/openstack/build/cce90efc7177459e9eebdab966bab75c | |
| 23:40:44 | clarkb | the jammy ceph version is also new enough for nova iirc | |
| 23:41:20 | dansmith | clarkb: right, the reason we didn't do that before was because they didn't yet have jammy packages IIRC, but I think they do now | |
| 23:41:46 | dansmith | either way, it's failing pretty hard | |
| 23:42:09 | clarkb | I think there was actually a misunderstanding. Everyone was looking for cloud archive packages for some reason, but the base distro had packages that were quite new | |
| 23:42:10 | dansmith | I mean the distro-package-based one | |
| 23:42:32 | dansmith | clarkb: no I think people were looking for ceph.org packages, or at least that's what I remembered | |
| 23:42:47 | clarkb | ya that was the first issue, then they looked for UCA packages and didn't find those either | |
| 23:42:49 | dansmith | but perhaps that was because jammy already had them and that's just how we did it before, I dunno | |
| 23:42:53 | dansmith | ack okay | |
| 23:42:57 | clarkb | but the base distro has newish packages that can probabl ybe made to work | |
| 23:43:43 | dansmith | the job running on jammy is about to finish, but it had lots of tempest fail in it | |
| 23:43:52 | dansmith | from this: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315?tab=change-view-tab-header-zuul-results-summary | |
| 23:47:17 | dansmith | qemu: module block-block-rbd not found | |
| 23:47:20 | dansmith | in nova compute's log | |
| 23:47:33 | sean-k-mooney[m] | dansmith: i think you just need to add the ensure podman role to a pre playbook | |
| 23:48:20 | dansmith | sean-k-mooney[m]: could be, but my point is, I don't think that job is like super healthy and well-tested such that switching to it is the obvious choice :) | |
| 23:48:48 | sean-k-mooney[m] | hehe ya... | |
| 23:52:23 | sean-k-mooney[m] | so googleing a little i think there is some issue with cephadm assuming rpm based distors but its runing on ubuntu | |
| 23:52:45 | sean-k-mooney[m] | you can correct this via containers.conf apprently | |
| 23:53:15 | dansmith | yeah idk, but if jammy has new enough ceph that seems like it might be the easiest path to it working | |
| 23:53:32 | dansmith | although we must also be missing that qemu module | |
| 23:53:40 | sean-k-mooney[m] | it is older then whats on ceph.com | |
| 23:53:52 | sean-k-mooney[m] | but its relitivly recent | |
| 23:54:05 | dansmith | but quincy is the release they're trying to install and that's it, AFAICT | |
| 23:54:07 | sean-k-mooney[m] | i think manilla wanted to have a newer verion | |
| 23:54:16 | dansmith | however, I think we're going to have to ask neutron to revert the pin until we figure it out regardless | |
| 23:55:11 | dansmith | it's going to be ugly because they merged it to like all their repos | |
| 23:57:21 | sean-k-mooney[m] | ya i didnt actully get time to look at any of this today | |
| 23:57:59 | sean-k-mooney[m] | i wanted to check if anything had progress before i went to sleep but not much it seams thanks for rechecking that devstack patch | |
| #openstack-nova - 2023-04-25 | |||
| 00:37:45 | gmann | dansmith: elodilles has patch up to remove all py38 jobs including focal job https://review.opendev.org/c/openstack/nova/+/881339 | |
| 04:49:59 | ykarel | dansmith, only merged in neutron repo, others are still unmerged, i pushed the revert | |
| 06:55:00 | bauzas | catching up this night's conversations | |
| 07:17:13 | opendevreview | Balazs Gibizer proposed openstack/nova master: DNM: Revert "Temporary skip some volume detach test in nova-lvm job" https://review.opendev.org/c/openstack/nova/+/881389 | |
| 08:49:47 | gibi | bauzas: I need to leave early today so I will miss the nova meeting | |
| 09:04:48 | bauzas | gibi: ack, no worries | |
| 10:01:18 | opendevreview | Amit Uniyal proposed openstack/nova master: WIP: Reproducer for dangling volumes https://review.opendev.org/c/openstack/nova/+/881457 | |
| 11:27:41 | opendevreview | Merged openstack/nova stable/yoga: Handle InstanceInvalidState exception https://review.opendev.org/c/openstack/nova/+/872117 | |
| 13:01:10 | dansmith | gmann: I don't see that that ever passed | |
| 13:02:37 | dansmith | using the alternate python interpreter on focal seems less good to me than just fixing the ceph job | |
| 13:02:51 | dansmith | and also not necessary if people don't prohibit 3.8 | |
| 13:03:01 | sean-k-mooney | dansmith: ya i suggested it because it used to work but it obviously got broken at some point | |
| 13:03:10 | sean-k-mooney | i was hoping it woudl be a quick fix to unblock the gate | |
| 13:03:21 | sean-k-mooney | just makeign the job use 22.04 would be my preference too | |
| 13:08:35 | bauzas | yup, let's try to use Jammy | |
| 13:11:33 | opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (api) https://review.opendev.org/c/openstack/nova/+/836830 | |
| 13:11:34 | opendevreview | ribaudr proposed openstack/nova master: Check shares support https://review.opendev.org/c/openstack/nova/+/850499 | |
| 13:11:34 | opendevreview | ribaudr proposed openstack/nova master: Add metadata for shares https://review.opendev.org/c/openstack/nova/+/850500 | |
| 13:11:35 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_attach notification https://review.opendev.org/c/openstack/nova/+/850501 | |
| 13:11:35 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_detach notification https://review.opendev.org/c/openstack/nova/+/851028 | |
| 13:11:36 | opendevreview | ribaudr proposed openstack/nova master: Add shares to InstancePayload https://review.opendev.org/c/openstack/nova/+/851029 | |
| 13:11:36 | opendevreview | ribaudr proposed openstack/nova master: Add helper methods to attach/detach shares https://review.opendev.org/c/openstack/nova/+/852085 | |
| 13:11:37 | opendevreview | ribaudr proposed openstack/nova master: Add libvirt test to ensure metadata are working. https://review.opendev.org/c/openstack/nova/+/852086 | |
| 13:11:37 | opendevreview | ribaudr proposed openstack/nova master: Add virt/libvirt error test cases https://review.opendev.org/c/openstack/nova/+/852087 | |
| 13:11:38 | opendevreview | ribaudr proposed openstack/nova master: Add share_info parameter to reboot method for each driver (driver part) https://review.opendev.org/c/openstack/nova/+/854823 | |
| 13:11:38 | opendevreview | ribaudr proposed openstack/nova master: Support rebooting an instance with shares (compute and API part) https://review.opendev.org/c/openstack/nova/+/854824 | |
| 13:11:40 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_attach_error notification https://review.opendev.org/c/openstack/nova/+/860282 | |
| 13:11:40 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_detach_error notification https://review.opendev.org/c/openstack/nova/+/860283 | |
| 13:11:42 | opendevreview | ribaudr proposed openstack/nova master: Add share_info parameter to resume method for each driver (driver part) https://review.opendev.org/c/openstack/nova/+/860284 | |
| 13:11:42 | opendevreview | ribaudr proposed openstack/nova master: Support resuming an instance with shares (compute and API part) https://review.opendev.org/c/openstack/nova/+/860285 | |
| 13:11:44 | opendevreview | ribaudr proposed openstack/nova master: Add helper methods to rescue/unrescue shares https://review.opendev.org/c/openstack/nova/+/860286 | |
| 13:11:44 | opendevreview | ribaudr proposed openstack/nova master: Support rescuing an instance with shares (driver part) https://review.opendev.org/c/openstack/nova/+/860287 | |
| 13:11:46 | opendevreview | ribaudr proposed openstack/nova master: Support rescuing an instance with shares (compute and API part) https://review.opendev.org/c/openstack/nova/+/860288 | |
| 13:11:46 | opendevreview | ribaudr proposed openstack/nova master: Docs about Manila shares API usage https://review.opendev.org/c/openstack/nova/+/871642 | |
| 13:11:48 | opendevreview | ribaudr proposed openstack/nova master: Mounting the shares as part of the initialization process https://review.opendev.org/c/openstack/nova/+/880075 | |
| 13:11:48 | opendevreview | ribaudr proposed openstack/nova master: Deletion of associated share mappings on instance deletion https://review.opendev.org/c/openstack/nova/+/881472 | |
| 13:16:56 | dansmith | bauzas: oh I didn't realize you rechecked my patch and it failed again with *another* uninstallable package | |
| 13:17:01 | dansmith | not sure if it's 3.9 related or not though | |
| 13:17:20 | bauzas | I haven't seen the logs yet | |
| 13:17:31 | dansmith | I rechecked again so we'll see | |
| 13:18:57 | ykarel | fwiw pysaml2 update is py3.9+ only and keystone installation is broken in focal or py38 | |
| 13:19:08 | dansmith | yeah that ^ | |
| 13:19:19 | dansmith | that's what I failed on | |
| 13:19:24 | dansmith | cripes | |
| 13:19:49 | ykarel | revert patch https://review.opendev.org/c/openstack/requirements/+/881466 | |
| 13:21:11 | bauzas | ykarel: last time I looked, this was due to the fact that py39 wanted UUIDs | |
| 13:35:38 | ykarel | bauzas, sorry which issue? /me can't relate to UUIDs | |
| 13:39:13 | dansmith | yeah not sure what the uuid thing is | |
| 13:41:11 | bauzas | it was when elodilles tried to update the py version for ceph-multinode | |
| 13:41:21 | bauzas | but I can be wrong | |
| 13:41:32 | dansmith | oh, unrelated to pysaml I see | |
| 13:41:47 | ykarel | ack | |
| 13:41:57 | bauzas | speaking of https://1742443592df6307c50f-d94079506d616d2cb38d2b70f09b441e.ssl.cf2.rackcdn.com/881339/3/check/nova-ceph-multistore/5f269ed/controller/logs/devstacklog.txt | |
| 13:42:15 | bauzas | 2023-04-24 11:48:22.720 | [01;33mWARNING py.warnings [[01;36mNone req-518808ec-0a14-444d-b6d3-fb6383d03e9c [00;36mNone None[01;33m] [01;35m[01;33m/usr/local/lib/python3.9/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: RegionOne. To ensure interoperability, identifiers should be a valid uuid. | |
| 13:43:34 | dansmith | eharney: are you around to talk about the ceph job stuff? I think you were merging some of those cephadm changes | |
| 13:43:44 | ykarel | ack got it, it's seperate thing | |
| 13:45:12 | eharney | dansmith: yes | |
| 13:45:41 | eharney | dansmith: what's going on there? | |
| 13:45:45 | dansmith | eharney: so, is that how we should be moving the upstream jobs at this point? jammy has quincy ceph packages, as I understand it | |
| 13:45:53 | dansmith | and the cephadm jobs don't work either | |
| 13:47:06 | dansmith | and they're nonvoting so I assumed that meant they weren't "ready" | |
| 13:47:25 | dansmith | eharney: anyway, the context is, we desperately need to get the ceph jobs running on jammy | |