| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-04-25 | |||
| 13:02:37 | dansmith | using the alternate python interpreter on focal seems less good to me than just fixing the ceph job | |
| 13:02:51 | dansmith | and also not necessary if people don't prohibit 3.8 | |
| 13:03:01 | sean-k-mooney | dansmith: ya i suggested it because it used to work but it obviously got broken at some point | |
| 13:03:10 | sean-k-mooney | i was hoping it woudl be a quick fix to unblock the gate | |
| 13:03:21 | sean-k-mooney | just makeign the job use 22.04 would be my preference too | |
| 13:08:35 | bauzas | yup, let's try to use Jammy | |
| 13:11:33 | opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (api) https://review.opendev.org/c/openstack/nova/+/836830 | |
| 13:11:34 | opendevreview | ribaudr proposed openstack/nova master: Check shares support https://review.opendev.org/c/openstack/nova/+/850499 | |
| 13:11:34 | opendevreview | ribaudr proposed openstack/nova master: Add metadata for shares https://review.opendev.org/c/openstack/nova/+/850500 | |
| 13:11:35 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_attach notification https://review.opendev.org/c/openstack/nova/+/850501 | |
| 13:11:35 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_detach notification https://review.opendev.org/c/openstack/nova/+/851028 | |
| 13:11:36 | opendevreview | ribaudr proposed openstack/nova master: Add shares to InstancePayload https://review.opendev.org/c/openstack/nova/+/851029 | |
| 13:11:36 | opendevreview | ribaudr proposed openstack/nova master: Add helper methods to attach/detach shares https://review.opendev.org/c/openstack/nova/+/852085 | |
| 13:11:37 | opendevreview | ribaudr proposed openstack/nova master: Add libvirt test to ensure metadata are working. https://review.opendev.org/c/openstack/nova/+/852086 | |
| 13:11:37 | opendevreview | ribaudr proposed openstack/nova master: Add virt/libvirt error test cases https://review.opendev.org/c/openstack/nova/+/852087 | |
| 13:11:38 | opendevreview | ribaudr proposed openstack/nova master: Add share_info parameter to reboot method for each driver (driver part) https://review.opendev.org/c/openstack/nova/+/854823 | |
| 13:11:38 | opendevreview | ribaudr proposed openstack/nova master: Support rebooting an instance with shares (compute and API part) https://review.opendev.org/c/openstack/nova/+/854824 | |
| 13:11:40 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_attach_error notification https://review.opendev.org/c/openstack/nova/+/860282 | |
| 13:11:40 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_detach_error notification https://review.opendev.org/c/openstack/nova/+/860283 | |
| 13:11:42 | opendevreview | ribaudr proposed openstack/nova master: Add share_info parameter to resume method for each driver (driver part) https://review.opendev.org/c/openstack/nova/+/860284 | |
| 13:11:42 | opendevreview | ribaudr proposed openstack/nova master: Support resuming an instance with shares (compute and API part) https://review.opendev.org/c/openstack/nova/+/860285 | |
| 13:11:44 | opendevreview | ribaudr proposed openstack/nova master: Add helper methods to rescue/unrescue shares https://review.opendev.org/c/openstack/nova/+/860286 | |
| 13:11:44 | opendevreview | ribaudr proposed openstack/nova master: Support rescuing an instance with shares (driver part) https://review.opendev.org/c/openstack/nova/+/860287 | |
| 13:11:46 | opendevreview | ribaudr proposed openstack/nova master: Support rescuing an instance with shares (compute and API part) https://review.opendev.org/c/openstack/nova/+/860288 | |
| 13:11:46 | opendevreview | ribaudr proposed openstack/nova master: Docs about Manila shares API usage https://review.opendev.org/c/openstack/nova/+/871642 | |
| 13:11:48 | opendevreview | ribaudr proposed openstack/nova master: Mounting the shares as part of the initialization process https://review.opendev.org/c/openstack/nova/+/880075 | |
| 13:11:48 | opendevreview | ribaudr proposed openstack/nova master: Deletion of associated share mappings on instance deletion https://review.opendev.org/c/openstack/nova/+/881472 | |
| 13:16:56 | dansmith | bauzas: oh I didn't realize you rechecked my patch and it failed again with *another* uninstallable package | |
| 13:17:01 | dansmith | not sure if it's 3.9 related or not though | |
| 13:17:20 | bauzas | I haven't seen the logs yet | |
| 13:17:31 | dansmith | I rechecked again so we'll see | |
| 13:18:57 | ykarel | fwiw pysaml2 update is py3.9+ only and keystone installation is broken in focal or py38 | |
| 13:19:08 | dansmith | yeah that ^ | |
| 13:19:19 | dansmith | that's what I failed on | |
| 13:19:24 | dansmith | cripes | |
| 13:19:49 | ykarel | revert patch https://review.opendev.org/c/openstack/requirements/+/881466 | |
| 13:21:11 | bauzas | ykarel: last time I looked, this was due to the fact that py39 wanted UUIDs | |
| 13:35:38 | ykarel | bauzas, sorry which issue? /me can't relate to UUIDs | |
| 13:39:13 | dansmith | yeah not sure what the uuid thing is | |
| 13:41:11 | bauzas | it was when elodilles tried to update the py version for ceph-multinode | |
| 13:41:21 | bauzas | but I can be wrong | |
| 13:41:32 | dansmith | oh, unrelated to pysaml I see | |
| 13:41:47 | ykarel | ack | |
| 13:41:57 | bauzas | speaking of https://1742443592df6307c50f-d94079506d616d2cb38d2b70f09b441e.ssl.cf2.rackcdn.com/881339/3/check/nova-ceph-multistore/5f269ed/controller/logs/devstacklog.txt | |
| 13:42:15 | bauzas | 2023-04-24 11:48:22.720 | [01;33mWARNING py.warnings [[01;36mNone req-518808ec-0a14-444d-b6d3-fb6383d03e9c [00;36mNone None[01;33m] [01;35m[01;33m/usr/local/lib/python3.9/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: RegionOne. To ensure interoperability, identifiers should be a valid uuid. | |
| 13:43:34 | dansmith | eharney: are you around to talk about the ceph job stuff? I think you were merging some of those cephadm changes | |
| 13:43:44 | ykarel | ack got it, it's seperate thing | |
| 13:45:12 | eharney | dansmith: yes | |
| 13:45:41 | eharney | dansmith: what's going on there? | |
| 13:45:45 | dansmith | eharney: so, is that how we should be moving the upstream jobs at this point? jammy has quincy ceph packages, as I understand it | |
| 13:45:53 | dansmith | and the cephadm jobs don't work either | |
| 13:47:06 | dansmith | and they're nonvoting so I assumed that meant they weren't "ready" | |
| 13:47:25 | dansmith | eharney: anyway, the context is, we desperately need to get the ceph jobs running on jammy | |
| 13:48:03 | eharney | i was trying to understand the situation with these the other day, i guess they were pinned to focal before because there weren't jammy packages, but maybe now there are? | |
| 13:49:26 | dansmith | eharney: yeah that's the reason for the pinning, but indeed now there seem to be base packages in jammy for quincy | |
| 13:49:40 | dansmith | I rechecked the unpin patch yesterday and it failed with qemu not having the rbd block driver | |
| 13:49:53 | dansmith | eharney: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315?tab=change-view-tab-header-zuul-results-summary | |
| 13:50:47 | eharney | hmm, looking, i'm not really up to speed on this | |
| 13:50:57 | dansmith | eharney: okay is there someone else we should bug? | |
| 13:51:22 | dansmith | eharney: because right now the ceph jobs are all totally wedged because a bunch of packages have just banned anything python<=3.8 | |
| 13:51:40 | eharney | dansmith: not sure who but i think some manila folks have been more active in devstack-plugin-ceph recently | |
| 13:51:41 | dansmith | reverts are in place or in flight for some of them, but we need to get it resolved | |
| 13:51:43 | eharney | ahh | |
| 13:51:53 | dansmith | because focal is not in our PTI right now... | |
| 13:52:06 | dansmith | eharney: so maybe gouthamr ? | |
| 13:57:07 | sean-k-mooney | dansmith: did you notice the ceph nfs job passed? https://zuul.opendev.org/t/openstack/build/3f0ec49fa9e048f6a73e6c03833fecc9/logs | |
| 13:57:31 | dansmith | sean-k-mooney: I assume that's not using the qemu block driver | |
| 13:57:51 | sean-k-mooney | perhaps although that is an odd issue | |
| 13:57:53 | dansmith | sean-k-mooney: the actual setup of ceph seemed to work on jammy, it's just that qemu doesn't have the block driver to load to talk to it (or something) | |
| 13:58:05 | sean-k-mooney | they may have chagned that to a weak dep in the qemu packaging | |
| 13:58:13 | sean-k-mooney | so we might need to install it in devstack explictly | |
| 13:58:19 | dansmith | it seemed like maybe those were provided in the ceph packages instead of qemu or something | |
| 13:58:28 | dansmith | sean-k-mooney: well, I looked and couldn't find any package in jammy for it | |
| 13:58:37 | dansmith | sean-k-mooney: but yes, hopefully it's something simple | |
| 13:58:48 | sean-k-mooney | i assumed it was complied in if im being honest but didnt look | |
| 13:59:01 | dansmith | me too | |
| 13:59:08 | dansmith | until I saw the logs.. let me get you a link | |
| 13:59:20 | dansmith | sean-k-mooney: https://zuul.opendev.org/t/openstack/build/1435ec8b57be4b159c3d373e23673ab5/log/controller/logs/screen-n-cpu.txt#8437 | |
| 14:00:04 | dansmith | it's also weirdly named "block-block-rbd".. perhaps there's some change in jammy with something and someone is prefixing an extra... prefix/ | |
| 14:00:23 | dansmith | although it says "unknown driver rbd" on the next line, which looks right | |
| 14:00:32 | sean-k-mooney | there nova package just depend on qemu-system https://packages.ubuntu.com/jammy/nova-compute-qemu | |
| 14:00:44 | bauzas | dansmith: sean-k-mooney: so, to clarify, once we merge https://review.opendev.org/c/openstack/nova/+/881409 | |
| 14:01:06 | bauzas | we will still have a problem with ceph-multistore due to keystone + cephadm right? | |
| 14:01:12 | dansmith | bauzas: yes | |
| 14:01:16 | sean-k-mooney | hum that is not linked agains librbd | |
| 14:01:19 | sean-k-mooney | https://packages.ubuntu.com/jammy/qemu-system-x86 | |
| 14:01:28 | dansmith | bauzas: not "due to cephadm" | |
| 14:01:34 | sean-k-mooney | ah | |
| 14:01:36 | sean-k-mooney | https://packages.ubuntu.com/jammy/qemu-block-extra | |
| 14:01:36 | dansmith | bauzas: just the ceph job | |
| 14:01:44 | sean-k-mooney | dansmith: we are misisng qemu-block-extra | |
| 14:01:46 | bauzas | because of keystone ? | |
| 14:01:55 | sean-k-mooney | that is what provides the rbd supprot | |
| 14:02:06 | dansmith | sean-k-mooney: ack, nice | |
| 14:02:19 | sean-k-mooney | and just checked its a recommended package | |
| 14:02:26 | eharney | ahh, cool, was just looking for that missing link myself | |
| 14:02:26 | sean-k-mooney | not a dep | |
| 14:02:31 | dansmith | sean-k-mooney: so we likely need a devstack ceph plugin change, let me push that up and try a dep | |
| 14:03:12 | sean-k-mooney | ya so we coudl also do this via bindep in nova as an optional dep . perhaps we shoudl as a follow up | |
| 14:03:26 | sean-k-mooney | i was just going to look at devstack to see how it installes qemu/libvirt | |