Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-11-20
09:31:56 bauzas we can ask reldel
09:40:40 QianbiaoNG hello anyone knows what IPA means in this project https://opendev.org/x/novajoin
09:41:25 QianbiaoNG And is there an official project for implement a nova vendor metadata service
09:47:21 swp20 bauzas, sean-k-mooney:
09:48:49 swp20 hi guys, i meet a question, when i use 'virsh attach-device gpu.xml domain --persistent --live' to attach device to the running guest,
09:49:58 swp20 and then detach the device with 'virsh detach-device domain gpu.xml --persistent --live' to detach device to the guest,
09:51:41 swp20 even through these actions are successful, and again attach-device to the guest, libvirt raise 'pci device in use by driver qemu'.
09:53:34 swp20 do you meet this problem, and any suggestion?
10:00:38 kashyap QianbiaoNG: I think it might be the https://freeipa.org project ("Identity, Policy, Audit"). I haven't clicked on your URL
10:01:31 QianbiaoNG kashyap ty, seems so.
10:01:47 kashyap QianbiaoNG: Actually, I'm sure it is talking about the FreeIPA server instances.
10:02:09 QianbiaoNG another question: is there an official project provided for implement a nova vendor data metadata service.
10:03:29 kashyap swp20: No answer top off my head, but if you're not using Nova, and directly `virsh`, you can get better help on #virt (OFTC network)
10:03:34 kashyap s/can/might/
10:08:19 swp20 kashyap: thanks, i joined and no person there in virt(OFTC network).
10:09:02 kashyap swp20: I think you're still on Freenode. I see more than 100 people :)
10:10:11 kashyap swp20: You need to use the '/connect' command to join the OFTC server, and then '/join #virt'
11:33:20 admin0 good morning .. i have nova, cinder and glance setup using ceph1 .. works good.. now I added ceph2 for only cinder ( so that cinder has 2 glance backends) one type ssd and one type hdd .. i am able to create the volumes in both hdd and sdd.. i can mount sdd fine .. but when I try to mount hdd, i get: libvirt.libvirtError: internal error: unable to execute QEMU command 'blockdev-add': error connecting: Permission denied" .. the hypervisor has no
11:33:20 admin0 clue of ceph2 .. so it only has ceph details of the first ceph1 cluster only ..
11:33:53 admin0 so question is, what config changes i need to do or what needs to be done so that i can mount the volume from the 2nd ceph to the instance
11:45:56 openstackgerrit Lee Yarwood proposed openstack/nova-specs master: Image and flavor defined ephemeral storage encryption https://review.opendev.org/752284
11:51:51 sean-k-mooney admin0: its likely an issue with your cephx keys and the pool permeions for the nova user on the ceph clusters
11:52:13 openstackgerrit Stephen Finucane proposed openstack/nova-specs master: Update modernize-os-hypervisors-api spec https://review.opendev.org/763043
11:55:54 stephenfin bauzas: I'm happy with https://review.opendev.org/#/c/759828/ now, but I'm holding off on +2 to give you a chance to revisit it if you want
12:10:30 admin0 sean-k-mooney, so hypervisor does not need to have the ceph2 keys or settings ?
12:10:36 admin0 it offloads to cinder ?
12:11:09 admin0 sean-k-mooney, so this means ceph2 also needs a nova user ( which is used to mount )
12:15:23 sean-k-mooney i think it has to have both but lyarwood might be better able to respond
12:15:33 sean-k-mooney its been a while since i deploy ceph
12:23:23 lyarwood admin0: I don't think we support multiple clusters in Nova
12:23:38 sean-k-mooney lyarwood: we dont for rbd
12:23:47 sean-k-mooney lyarwood: bute we should via cinder
12:23:59 sean-k-mooney *rbd images backend
12:24:20 lyarwood sean-k-mooney: the user is still configured locally on the computes for volumes
12:24:43 sean-k-mooney right dont you have to configre the same user on all ceph clusters
12:24:57 sean-k-mooney with the same cephx key
12:24:58 lyarwood sean-k-mooney: with the same key
12:25:04 lyarwood and I'm not sure you can do that can you?
12:25:15 lyarwood I thought the keys were unique to a cluster
12:25:24 lyarwood but that might be a lack of knownledge on my side
12:25:39 sean-k-mooney im not sure i have only done it with one ceph personally
12:26:00 sean-k-mooney but i know people do dpleoy multiple ceph clusters
12:26:11 sean-k-mooney but im not sure if that is 1 per AZ or something like that
12:26:38 sean-k-mooney https://ceph.io/geen-categorie/openstack-nova-configure-multiple-ceph-backends-on-one-hypervisor/
12:26:42 lyarwood yeah, I think the normal way to do with admin0 has done would be to use different pools
12:27:33 sean-k-mooney how the hell does ^ work
12:27:38 lyarwood yeah that's single cluster multiple pools
12:27:53 lyarwood and aggregates to keep them apart
12:28:17 sean-k-mooney no i mesn once nova compute with 2 files witht he same keys and different values
12:28:25 sean-k-mooney nova-compute --config-file /etc/nova/nova-ceph1.conf --config-file /etc/nova/nova-ceph2.conf
12:29:03 lyarwood I think that's a mistake in the write up tbh
12:29:12 lyarwood given what they are trying to do
12:30:08 sean-k-mooney they are trying to run two nova computes on the same hyperviour with different host values but they same hypervior hostname
12:31:03 lyarwood ew
12:31:43 sean-k-mooney but 1 if that comand even works it really should not and 2 that si really going to screw up placemnt
12:32:17 sean-k-mooney admin0: so dont do what in that ceph guide.
12:34:13 openstackgerrit Lucian Petrut proposed openstack/nova master: hyper-v rbd volume support https://review.opendev.org/763550
12:34:17 sean-k-mooney ah found an actully sane example
12:34:20 sean-k-mooney https://medium.com/walmartglobaltech/deploying-cinder-with-multiple-ceph-cluster-backends-2cd90d64b10
12:34:26 sean-k-mooney lyarwood: admin0 ^
12:34:36 sean-k-mooney so the user has to be different on each cluster
12:35:56 sean-k-mooney actully no that is still a hack
12:40:28 lyarwood sean-k-mooney: hmm let me check the connection_info returned by c-vol, I might be missing something here
12:41:08 sean-k-mooney if you used differnt users e.g. nova-<cluster-name>
12:41:17 sean-k-mooney then you could have different cephx keys
12:41:32 sean-k-mooney 1 per user and it should work i think
12:41:46 sean-k-mooney but i did not like there add a libvirt secret thing
12:41:57 lyarwood https://github.com/openstack/cinder/blob/30501b9ecd6d4ed8dd6405be01795d8634013b24/cinder/volume/drivers/rbd.py#L1438-L1453
12:41:57 sean-k-mooney that was the hack in my view
12:42:02 lyarwood https://github.com/openstack/nova/blob/e16800cc0aebf1174c5c0b6c4b043b09622524e9/nova/virt/libvirt/volume/net.py#L37-L76
12:42:09 lyarwood you can have multiple secrets
12:42:24 lyarwood as long as the UUIDs are copied on the computes
12:42:27 admin0 the cinder part is good .. cinder can create volumes in both ceph .. only part is nova works with its default one and has no idea/clue about 2nd one
12:42:53 sean-k-mooney ya so they were manually defineing the second secret on the compute nodes
12:43:20 lyarwood that's fine
12:43:31 lyarwood admin0: apologies so I forgot about https://github.com/openstack/nova/blob/e16800cc0aebf1174c5c0b6c4b043b09622524e9/nova/virt/libvirt/volume/net.py#L37-L76
12:43:56 lyarwood admin0: so c-vol needs to return connection_info with {"auth_enabled": True}
12:45:29 sean-k-mooney is that enough or will you need to maulaly add the persistint key first. i think that would be required yes?
12:45:41 sean-k-mooney that a one time operation on all the compute nodes
12:46:18 lyarwood sean-k-mooney: correct you still need to populate the secret on the computes
12:46:28 lyarwood admin0: and thinking about it that's what you have likely missed here
12:46:47 lyarwood admin0: you need two ceph secrets in libvirt per compute with these two clusters
12:47:36 sean-k-mooney this fells like something that could be imporved with a specless blueprint to allow the config options to have multiple vlaues
12:48:26 lyarwood sean-k-mooney: I'd rather drop the use of the config here and just rely on cinder
12:48:29 lyarwood sean-k-mooney: for volumes
12:49:07 sean-k-mooney well the config options were for the images_backend really
12:49:17 sean-k-mooney but i ment for createing the secret in libvirt
12:49:39 sean-k-mooney i guess we coudl drop that and make it just a deployment step
12:49:42 stephenfin lyarwood: Could you create a bug report for https://review.opendev.org/#/c/761730/ if you're planning to backport it? If you're not, I'll approve now
12:51:22 lyarwood stephenfin: yup can do
12:51:51 lyarwood sean-k-mooney: we can't automate that so it's always going to be a deployment step
12:52:15 lyarwood brb
12:53:45 openstackgerrit Balazs Gibizer proposed openstack/nova master: Prevent nova-compute start with DB credentials https://review.opendev.org/762176
12:57:48 admin0 This uuid must be different than any existing uuid’s used by cinder/libvirt -- so i just generate a new uuid on the fly ?
12:58:19 lyarwood yeah it just needs to be the same across both the computes and host where cinder is running
12:58:45 lyarwood just use uuidgen
13:02:50 openstackgerrit Balazs Gibizer proposed openstack/nova master: Prevent nova-compute start with DB credentials https://review.opendev.org/762176
13:03:03 admin0 lyarwood, this is what i have in cinder.conf -- https://gist.github.com/a1git/e8edfc7c0f148085165d4b8589d3f5a7
13:03:08 admin0 i don't see any uuid in there
13:05:13 sean-k-mooney admin0: i think its in your ceph.conf

Earlier   Later