| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-17 | |||
| 15:39:46 | mloza | the cinder secrets are the same on both | |
| 15:43:29 | mloza | I can query second ceph cluster using from the compute ceph -c ceph.conf -n client.cinder --keyring=cinder.keyring status | |
| 15:44:18 | mloza | not sure why nova compute is throwing perm denied | |
| 15:44:50 | sean-k-mooney | you likely need to add a scond keyring to the libvirt secrets | |
| 15:46:08 | sean-k-mooney | normally when you add ceph you do something like this https://github.com/openstack/devstack-plugin-ceph/blob/master/devstack/lib/ceph#L244-L259 | |
| 15:46:53 | sean-k-mooney | but unless the same keyrign works for both clusters you will need to add a second one and then set the right keyring infor in the ceph backend in cinder | |
| 15:48:29 | sean-k-mooney | https://ceph.io/geen-categorie/openstack-nova-configure-multiple-ceph-backends-on-one-hypervisor/ | |
| 15:49:04 | sean-k-mooney | mloza: kolla does not actuly support using muliple ceph clusters out of the box today | |
| 15:52:11 | mloza | sean-k-mooney: i have cinder-backup and cinder keyring the same on both ceph cluster. Yes, I modified the playbooks by hand and creating two type of volumes works. | |
| 15:53:30 | mloza | i can only attach first volume type two tho | |
| 15:53:38 | mloza | type tho* | |
| 15:53:56 | sean-k-mooney | you have to do somehting like this i think | |
| 15:53:58 | sean-k-mooney | cat <<EOF>> /etc/cinder/cinder.conf | |
| 15:54:00 | sean-k-mooney | [ceph] | |
| 15:54:02 | sean-k-mooney | volume_driver = cinder.volume.drivers.rbd.RBDDriver | |
| 15:54:04 | sean-k-mooney | volume_backend_name=ceph | |
| 15:54:06 | sean-k-mooney | rbd_pool = volumes | |
| 15:54:08 | sean-k-mooney | rbd_ceph_conf = /etc/ceph/ceph.conf | |
| 15:54:10 | sean-k-mooney | rbd_flatten_volume_from_snapshot = false | |
| 15:54:12 | sean-k-mooney | rbd_max_clone_depth = 5 | |
| 15:54:14 | sean-k-mooney | rbd_store_chunk_size = 4 | |
| 15:54:16 | sean-k-mooney | rados_connect_timeout = -1 | |
| 15:54:18 | sean-k-mooney | glance_api_version = 2 | |
| 15:54:20 | sean-k-mooney | rbd_user =cinder | |
| 15:54:22 | sean-k-mooney | rbd_secret_uuid = `cat /root/uuid` | |
| 15:54:24 | sean-k-mooney | EOF | |
| 15:54:25 | mloza | sean-k-mooney: does nova look up ceph.conf when attaching a volume? | |
| 15:54:26 | sean-k-mooney | 17. Restart the Cinder services. | |
| 15:54:30 | sean-k-mooney | no | |
| 15:54:54 | sean-k-mooney | we only use the info passed form cinder in the attachment connection info | |
| 15:55:02 | sean-k-mooney | which will incudle the rbd_secret_uuid | |
| 15:55:17 | sean-k-mooney | which is used to retirve the keyring form the libvirt secrte | |
| 15:56:31 | sean-k-mooney | mloza: the cinder folks might be able to help more | |
| 15:56:36 | sean-k-mooney | i know this is definetly possible | |
| 15:56:42 | sean-k-mooney | i jsut dont have the docs to hand | |
| 15:56:46 | sean-k-mooney | but other have done this | |
| 15:57:02 | sean-k-mooney | actully https://medium.com/walmartglobaltech/deploying-cinder-with-multiple-ceph-cluster-backends-2cd90d64b10 might be what i was looking for | |
| 15:57:46 | sean-k-mooney | ya that explains the addtion of the second secret and the cinder config | |
| 15:58:25 | sean-k-mooney | """Create the appropriate cinder user on your new Ceph cluster. The username must be different from the existing Ceph cluster cinder username""" | |
| 16:00:07 | mloza | i stumbled upon that blog yesterday | |
| 16:00:23 | mloza | looks like the second secret must be unique | |
| 16:00:41 | mloza | this is how my cinder.conf looks like https://paste.opendev.org/raw/809414/ | |
| 16:01:40 | mloza | these are what inside the libvirt/secrets | |
| 16:01:45 | mloza | (nova-libvirt)[root@compute33 secrets]# pwd | |
| 16:01:47 | mloza | /etc/libvirt/secrets | |
| 16:01:49 | mloza | (nova-libvirt)[root@compute33 secrets]# ls -al | |
| 16:01:51 | mloza | total 16 | |
| 16:01:53 | mloza | drw------- 2 root root 204 Sep 17 15:20 . | |
| 16:01:55 | mloza | drwxr-xr-x 1 root root 75 Sep 17 15:20 .. | |
| 16:01:57 | mloza | -rw------- 1 root root 40 Sep 17 15:20 1d5b8192-6acf-4f06-8c60-1eb7ecbfe911.base64 | |
| 16:01:59 | mloza | -rw------- 1 root root 170 Sep 17 15:20 1d5b8192-6acf-4f06-8c60-1eb7ecbfe911.xml | |
| 16:02:01 | mloza | -rw------- 1 root root 40 Sep 17 15:20 c5730348-ff66-4491-8973-9a31317eaf00.base64 | |
| 16:02:03 | mloza | -rw------- 1 root root 168 Sep 17 15:20 c5730348-ff66-4491-8973-9a31317eaf00.xml | |
| 16:02:05 | mloza | (nova-libvirt)[root@compute33 secrets]# | |
| 16:02:36 | sean-k-mooney | ya so your using the same user for both | |
| 16:02:50 | sean-k-mooney | so i think you need a different user and a diffent secret for each | |
| 16:04:33 | mloza | I guess the same user can stay. I'll try with a different secret | |
| 16:04:39 | mloza | thx for the info | |
| 16:35:02 | lyarwood | ade_lee: https://paste.opendev.org/show/809415/ - looks like libvirtd was super slow plugging the device for some reason | |
| 16:36:02 | lyarwood | assuming that's the request that octavia failed on | |
| 16:41:47 | ade_lee | lyarwood, thanks - lets see if it recurs | |
| 16:42:59 | lyarwood | ade_lee: ack, no idea how to trace the nova requests from octavia fwiw | |
| 16:45:28 | johnsom | lyarwood https://zuul.opendev.org/t/openstack/build/af88c0d00ebb48509a51f6eb941c65c6/log/logs/undercloud/var/log/containers/octavia/worker.log#6865 | |
| 16:45:32 | johnsom | http://192.168.24.3:8774/v2.1/servers/52249295-f031-4f53-b403-86577a6a6e01/os-interface | |
| 16:45:51 | johnsom | {"interfaceAttachment": {"port_id": "47a2114d-ae49-4928-b00e-87948bd73183"}} | |
| 16:45:53 | lyarwood | ah ha | |
| 16:46:53 | lyarwood | odd that it didn't log the request-id | |
| 16:47:21 | johnsom | Yeah, I was puzzled too until I saw it was the tripleo haproxy process that responded with the 504 | |
| 16:47:52 | johnsom | https://zuul.opendev.org/t/openstack/build/af88c0d00ebb48509a51f6eb941c65c6/log/logs/undercloud/var/log/containers/haproxy/haproxy.log#1276 | |
| 17:33:34 | opendevreview | Dan Smith proposed openstack/nova master: DNM: Test glance multistore default https://review.opendev.org/c/openstack/nova/+/809883 | |
| 19:50:25 | mloza | sean-k-mooney: i defined a different ceph user and secret for second cluster however, but i'm getting the same error https://paste.opendev.org/raw/809420/ | |
| 20:02:45 | mloza | is rbd_secret_uuid in [libvirt] for nova.conf supports multiple uuids? | |
| 20:59:29 | opendevreview | melanie witt proposed openstack/nova master: Add legacy limits and usage to placement unified limits https://review.opendev.org/c/openstack/nova/+/713498 | |
| 20:59:29 | opendevreview | melanie witt proposed openstack/nova master: Update quota apis with keystone limits and usage https://review.opendev.org/c/openstack/nova/+/713499 | |
| 20:59:30 | opendevreview | melanie witt proposed openstack/nova master: Add reno for unified limits https://review.opendev.org/c/openstack/nova/+/715271 | |
| 21:08:02 | opendevreview | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | |
| 23:36:55 | opendevreview | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | |
| #openstack-nova - 2021-09-18 | |||
| 01:09:55 | opendevreview | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | |
| 04:04:42 | opendevreview | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | |
| 05:57:06 | opendevreview | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | |
| 07:14:55 | oklhost | sean-k-mooney: what does that mean? is there some documentation around? | |
| 14:01:14 | opendevreview | Takashi Natsume proposed openstack/nova master: Update min supported service version for Yoga. https://review.opendev.org/c/openstack/nova/+/809932 | |
| 15:10:30 | opendevreview | Takashi Kajinami proposed openstack/nova master: Fix the wrong exception used to retry detach API calls https://review.opendev.org/c/openstack/nova/+/809934 | |
| #openstack-nova - 2021-09-19 | |||
| 02:18:09 | opendevreview | Takashi Natsume proposed openstack/nova master: Update contributor guide for Yoga https://review.opendev.org/c/openstack/nova/+/809936 | |
| 06:08:03 | opendevreview | Brin Zhang proposed openstack/nova-specs master: Re-propose Remove tenant_id https://review.opendev.org/c/openstack/nova-specs/+/809939 | |
| 06:42:16 | opendevreview | Wenping Song proposed openstack/placement master: Modify the comment that is confused https://review.opendev.org/c/openstack/placement/+/809948 | |
| 12:11:25 | opendevreview | Merged openstack/os-vif master: Add Python3 yoga unit tests https://review.opendev.org/c/openstack/os-vif/+/808455 | |
| 15:38:09 | opendevreview | Elod Illes proposed openstack/nova stable/ussuri: [stable-only] Set lower-constraints job as non-voting https://review.opendev.org/c/openstack/nova/+/809955 | |
| 21:16:12 | opendevreview | Elod Illes proposed openstack/nova stable/ussuri: [stable-only] Set lower-constraints job as non-voting https://review.opendev.org/c/openstack/nova/+/809955 | |
| 21:25:29 | opendevreview | Elod Illes proposed openstack/nova stable/ussuri: [stable-only] Set lower-constraints job as non-voting https://review.opendev.org/c/openstack/nova/+/809955 | |
| #openstack-nova - 2021-09-20 | |||
| 07:37:06 | bauzas | good morning Nova | |
| 07:37:16 | bauzas | gibi: stephenfin: hola folks | |
| 07:48:16 | lyarwood | \o morning | |
| 07:58:27 | gibi | o. | |
| 07:58:30 | gibi | o/ | |
| 08:56:51 | oklhost | sean-k-mooney: thanks, seems we got less log entries. | |
| 09:14:29 | opendevreview | Thomas Goirand proposed openstack/nova master: Add missing __init__.py in nova/db/api https://review.opendev.org/c/openstack/nova/+/809980 | |
| 09:15:17 | zigo | bauzas: Good morning, there's a ooopsy there, no ? ^ | |
| 09:15:59 | bauzas | zigo: well, good question, I'm not sure we need it | |
| 09:16:16 | bauzas | what kind of issue you have ? | |