Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-09-17
15:30:01 sean-k-mooney https://zuul.opendev.org/t/openstack/build/af88c0d00ebb48509a51f6eb941c65c6/log/logs/undercloud/var/log/containers/nova/nova-api.log#2561
15:32:28 ade_lee sean-k-mooney, hmm - guess I'll do a recheck and see if its something transient -- I see nothing in rabbit logs
15:38:41 mloza hello, I have two ceph clusters and I create cinder volumes with two types however, I can only attach first volume type to the instances, the second volume type throws an error https://paste.opendev.org/show/bq8pYJCpViYfJawKy943/
15:39:46 mloza the cinder secrets are the same on both
15:43:29 mloza I can query second ceph cluster using from the compute ceph -c ceph.conf -n client.cinder --keyring=cinder.keyring status
15:44:18 mloza not sure why nova compute is throwing perm denied
15:44:50 sean-k-mooney you likely need to add a scond keyring to the libvirt secrets
15:46:08 sean-k-mooney normally when you add ceph you do something like this https://github.com/openstack/devstack-plugin-ceph/blob/master/devstack/lib/ceph#L244-L259
15:46:53 sean-k-mooney but unless the same keyrign works for both clusters you will need to add a second one and then set the right keyring infor in the ceph backend in cinder
15:48:29 sean-k-mooney https://ceph.io/geen-categorie/openstack-nova-configure-multiple-ceph-backends-on-one-hypervisor/
15:49:04 sean-k-mooney mloza: kolla does not actuly support using muliple ceph clusters out of the box today
15:52:11 mloza sean-k-mooney: i have cinder-backup and cinder keyring the same on both ceph cluster. Yes, I modified the playbooks by hand and creating two type of volumes works.
15:53:30 mloza i can only attach first volume type two tho
15:53:38 mloza type tho*
15:53:56 sean-k-mooney you have to do somehting like this i think
15:53:58 sean-k-mooney cat <<EOF>> /etc/cinder/cinder.conf
15:54:00 sean-k-mooney [ceph]
15:54:02 sean-k-mooney volume_driver = cinder.volume.drivers.rbd.RBDDriver
15:54:04 sean-k-mooney volume_backend_name=ceph
15:54:06 sean-k-mooney rbd_pool = volumes
15:54:08 sean-k-mooney rbd_ceph_conf = /etc/ceph/ceph.conf
15:54:10 sean-k-mooney rbd_flatten_volume_from_snapshot = false
15:54:12 sean-k-mooney rbd_max_clone_depth = 5
15:54:14 sean-k-mooney rbd_store_chunk_size = 4
15:54:16 sean-k-mooney rados_connect_timeout = -1
15:54:18 sean-k-mooney glance_api_version = 2
15:54:20 sean-k-mooney rbd_user =cinder
15:54:22 sean-k-mooney rbd_secret_uuid = `cat /root/uuid`
15:54:24 sean-k-mooney EOF
15:54:25 mloza sean-k-mooney: does nova look up ceph.conf when attaching a volume?
15:54:26 sean-k-mooney 17. Restart the Cinder services.
15:54:30 sean-k-mooney no
15:54:54 sean-k-mooney we only use the info passed form cinder in the attachment connection info
15:55:02 sean-k-mooney which will incudle the rbd_secret_uuid
15:55:17 sean-k-mooney which is used to retirve the keyring form the libvirt secrte
15:56:31 sean-k-mooney mloza: the cinder folks might be able to help more
15:56:36 sean-k-mooney i know this is definetly possible
15:56:42 sean-k-mooney i jsut dont have the docs to hand
15:56:46 sean-k-mooney but other have done this
15:57:02 sean-k-mooney actully https://medium.com/walmartglobaltech/deploying-cinder-with-multiple-ceph-cluster-backends-2cd90d64b10 might be what i was looking for
15:57:46 sean-k-mooney ya that explains the addtion of the second secret and the cinder config
15:58:25 sean-k-mooney """Create the appropriate cinder user on your new Ceph cluster. The username must be different from the existing Ceph cluster cinder username"""
16:00:07 mloza i stumbled upon that blog yesterday
16:00:23 mloza looks like the second secret must be unique
16:00:41 mloza this is how my cinder.conf looks like https://paste.opendev.org/raw/809414/
16:01:40 mloza these are what inside the libvirt/secrets
16:01:45 mloza (nova-libvirt)[root@compute33 secrets]# pwd
16:01:47 mloza /etc/libvirt/secrets
16:01:49 mloza (nova-libvirt)[root@compute33 secrets]# ls -al
16:01:51 mloza total 16
16:01:53 mloza drw------- 2 root root 204 Sep 17 15:20 .
16:01:55 mloza drwxr-xr-x 1 root root 75 Sep 17 15:20 ..
16:01:57 mloza -rw------- 1 root root 40 Sep 17 15:20 1d5b8192-6acf-4f06-8c60-1eb7ecbfe911.base64
16:01:59 mloza -rw------- 1 root root 170 Sep 17 15:20 1d5b8192-6acf-4f06-8c60-1eb7ecbfe911.xml
16:02:01 mloza -rw------- 1 root root 40 Sep 17 15:20 c5730348-ff66-4491-8973-9a31317eaf00.base64
16:02:03 mloza -rw------- 1 root root 168 Sep 17 15:20 c5730348-ff66-4491-8973-9a31317eaf00.xml
16:02:05 mloza (nova-libvirt)[root@compute33 secrets]#
16:02:36 sean-k-mooney ya so your using the same user for both
16:02:50 sean-k-mooney so i think you need a different user and a diffent secret for each
16:04:33 mloza I guess the same user can stay. I'll try with a different secret
16:04:39 mloza thx for the info
16:35:02 lyarwood ade_lee: https://paste.opendev.org/show/809415/ - looks like libvirtd was super slow plugging the device for some reason
16:36:02 lyarwood assuming that's the request that octavia failed on
16:41:47 ade_lee lyarwood, thanks - lets see if it recurs
16:42:59 lyarwood ade_lee: ack, no idea how to trace the nova requests from octavia fwiw
16:45:28 johnsom lyarwood https://zuul.opendev.org/t/openstack/build/af88c0d00ebb48509a51f6eb941c65c6/log/logs/undercloud/var/log/containers/octavia/worker.log#6865
16:45:32 johnsom http://192.168.24.3:8774/v2.1/servers/52249295-f031-4f53-b403-86577a6a6e01/os-interface
16:45:51 johnsom {"interfaceAttachment": {"port_id": "47a2114d-ae49-4928-b00e-87948bd73183"}}
16:45:53 lyarwood ah ha
16:46:53 lyarwood odd that it didn't log the request-id
16:47:21 johnsom Yeah, I was puzzled too until I saw it was the tripleo haproxy process that responded with the 504
16:47:52 johnsom https://zuul.opendev.org/t/openstack/build/af88c0d00ebb48509a51f6eb941c65c6/log/logs/undercloud/var/log/containers/haproxy/haproxy.log#1276
17:33:34 opendevreview Dan Smith proposed openstack/nova master: DNM: Test glance multistore default https://review.opendev.org/c/openstack/nova/+/809883
19:50:25 mloza sean-k-mooney: i defined a different ceph user and secret for second cluster however, but i'm getting the same error https://paste.opendev.org/raw/809420/
20:02:45 mloza is rbd_secret_uuid in [libvirt] for nova.conf supports multiple uuids?
20:59:29 opendevreview melanie witt proposed openstack/nova master: Update quota apis with keystone limits and usage https://review.opendev.org/c/openstack/nova/+/713499
20:59:29 opendevreview melanie witt proposed openstack/nova master: Add legacy limits and usage to placement unified limits https://review.opendev.org/c/openstack/nova/+/713498
20:59:30 opendevreview melanie witt proposed openstack/nova master: Add reno for unified limits https://review.opendev.org/c/openstack/nova/+/715271
21:08:02 opendevreview melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963
23:36:55 opendevreview melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963
#openstack-nova - 2021-09-18
01:09:55 opendevreview melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963
04:04:42 opendevreview melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963
05:57:06 opendevreview melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963
07:14:55 oklhost sean-k-mooney: what does that mean? is there some documentation around?
14:01:14 opendevreview Takashi Natsume proposed openstack/nova master: Update min supported service version for Yoga. https://review.opendev.org/c/openstack/nova/+/809932
15:10:30 opendevreview Takashi Kajinami proposed openstack/nova master: Fix the wrong exception used to retry detach API calls https://review.opendev.org/c/openstack/nova/+/809934
#openstack-nova - 2021-09-19
02:18:09 opendevreview Takashi Natsume proposed openstack/nova master: Update contributor guide for Yoga https://review.opendev.org/c/openstack/nova/+/809936
06:08:03 opendevreview Brin Zhang proposed openstack/nova-specs master: Re-propose Remove tenant_id https://review.opendev.org/c/openstack/nova-specs/+/809939
06:42:16 opendevreview Wenping Song proposed openstack/placement master: Modify the comment that is confused https://review.opendev.org/c/openstack/placement/+/809948
12:11:25 opendevreview Merged openstack/os-vif master: Add Python3 yoga unit tests https://review.opendev.org/c/openstack/os-vif/+/808455
15:38:09 opendevreview Elod Illes proposed openstack/nova stable/ussuri: [stable-only] Set lower-constraints job as non-voting https://review.opendev.org/c/openstack/nova/+/809955
21:16:12 opendevreview Elod Illes proposed openstack/nova stable/ussuri: [stable-only] Set lower-constraints job as non-voting https://review.opendev.org/c/openstack/nova/+/809955
21:25:29 opendevreview Elod Illes proposed openstack/nova stable/ussuri: [stable-only] Set lower-constraints job as non-voting https://review.opendev.org/c/openstack/nova/+/809955
#openstack-nova - 2021-09-20
07:37:06 bauzas good morning Nova
07:37:16 bauzas gibi: stephenfin: hola folks
07:48:16 lyarwood \o morning
07:58:27 gibi o.
07:58:30 gibi o/
08:56:51 oklhost sean-k-mooney: thanks, seems we got less log entries.
09:14:29 opendevreview Thomas Goirand proposed openstack/nova master: Add missing __init__.py in nova/db/api https://review.opendev.org/c/openstack/nova/+/809980

Earlier   Later