Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-07-01
14:44:12 sean-k-mooney which is what we want. e.g. keep the file sparse
14:44:58 lyarwood hmmm is that enough?
14:45:11 sean-k-mooney im not sure but lseki could test with dd?
14:45:37 lyarwood I thought you had to at least grow the sparse file so it looks like it's 2G before you can update the qcow2 headers to be 2G
14:46:05 sean-k-mooney not sure but it sounds like something we should test
14:46:22 lyarwood yup
14:46:23 lyarwood brb
14:47:19 sean-k-mooney lseki: can you try that. grow you iamge to like 10G and then use dd to crate a 5G file in the vm
14:47:27 lseki sean-k-mooney: lyarwood oops, I was using a raw file instead of qcow
14:47:36 sean-k-mooney if it had less then 2G before that would fail
14:47:50 sean-k-mooney lseki: oh in that case what the size of the file on disk after that command
14:47:56 sean-k-mooney did it grow it to 2G
14:48:22 lseki yes it did, and `lsblk` inside the vm also is updated to 2G
14:48:54 sean-k-mooney could you convert it over to a qcow boot a vm with the new image and then grow it to say 4G
14:49:01 sean-k-mooney using the same command on the new vm
14:49:12 sean-k-mooney lets see if it will do the right thing in that case
14:49:30 sean-k-mooney if so i think we have a way to adresss this
14:51:52 lseki let me try that...
15:03:37 lyarwood sean-k-mooney: TIL I didn't think blockResize did that, given that we resized the file before I guess that was the case and something has changed?
15:04:14 sean-k-mooney from the name that is what i would expect ti to do
15:04:28 sean-k-mooney maybe not for host devices
15:04:32 sean-k-mooney but for files at least
15:04:34 lyarwood yeah it can't for host devices
15:05:02 lyarwood I might be mixing things up with blockRebase tbh
15:40:56 lseki sean-k-mooney: I downloaded a cirros image and booted a vm from it. I successfully extended the volume to 4G.
15:41:20 lseki https://www.irccloud.com/pastebin/QF9Dk7Wa/
15:42:06 lseki within the vm:
15:42:11 lseki https://www.irccloud.com/pastebin/QJ8flBUZ/
15:43:42 lseki though it didn't work when the disk was attached using the default IDE bus
15:43:53 lseki https://www.irccloud.com/pastebin/pv6e5QOF/
15:44:26 lseki so I had to specify virtio bus while booting the vm `--disk cirros-0.5.1-x86_64-disk.img,bus=virtio`
15:47:12 openstackgerrit Merged openstack/nova master: Correct reported system memory https://review.opendev.org/733827
15:47:30 openstackgerrit Merged openstack/nova master: Extend is_ipv6_supported() to cover more error cases https://review.opendev.org/736167
15:50:23 stephenfin bauzas: Thanks for that review. You'd probably also be interested in gibi's follow-up fix here https://review.opendev.org/#/c/737508/
15:50:50 bauzas stephenfin: ack, thanks for helping me to go off some hardware discussion, my brain being fried ;)
15:50:57 stephenfin :P
15:51:08 stephenfin anytime, buddy
15:59:12 bauzas gibi: stephenfin: holding my +W on https://review.opendev.org/#/c/737508/3 just for saying I honestly feel we should document this
15:59:32 bauzas and not leave operators discovering it by themselves
15:59:39 bauzas but, a FUP for course
16:25:51 stephenfin bauzas: To be honest, I thought that was assumed and I can't find anywhere easy to slot it in :-\
16:30:25 bauzas stephenfin: right, hence my main comment
16:30:33 bauzas anyway, you saw it, +Wing
16:50:02 sean-k-mooney while the libvirt driver could work on non linux hosts. it has been deployed on freebsd years ago that has never really been supported
16:50:56 sean-k-mooney we make assumtions about linux in the code to day so i doubt you could make it work on say window easily even if you wanted too
16:57:15 lseki lyarwood: oh, and the volume size started with 39M, but after doing some dd inside the vm it growed to 2G
16:59:23 sean-k-mooney lseki: so the qcow stayed sparse until we actuly wrote data too it
16:59:30 sean-k-mooney which is what we want right
16:59:35 lseki yes
17:00:56 sean-k-mooney so to summerise 1, we should modify the cinder nfs backend to only extend voluem if its not attached to a vm, 2 when we are using nfs voluem in nova we should skip calling qemu-image and jsut do a block resize via the qemu monitor command
17:01:36 sean-k-mooney lyarwood: ^ does that make sense too you?
17:04:02 sean-k-mooney lseki: for the ide case did you use the correct name?
17:04:24 sean-k-mooney i mean it wont be drive-virtio-disk0 right
17:04:43 sean-k-mooney it would be something like drive-ide-disk0
17:05:06 lseki sean-k-mooney: I tried the one shown by dumpxml, which is simply `disk`
17:05:19 lseki let me try with `drive-ide-disk0`
17:06:30 sean-k-mooney well its just disk for virtio as well
17:06:32 sean-k-mooney <disk type='file' device='disk'>
17:06:46 sean-k-mooney i think you are looking for the alias element
17:06:47 sean-k-mooney <alias name='virtio-disk0'/
17:09:15 lseki oh that's right, so I should use `drive-ide0-0-0` for ide
17:09:23 lseki yeah it works :-)
17:09:34 sean-k-mooney cool
17:09:47 sean-k-mooney so we can lookup the alais from the xml
17:09:56 sean-k-mooney i assume its jsut ide0-0-0
17:10:06 sean-k-mooney and we need to prepend drive-
17:11:11 lseki yes
17:13:17 sean-k-mooney im not going to ask you to test ceph but it might be intersting to see if we could use this patteren for other drivers too and avoid the calll to qemu-img
17:13:26 sean-k-mooney that can be a step 3
17:13:38 sean-k-mooney after 1 and 2
17:13:49 sean-k-mooney whcih will fix your usecase
17:19:15 lseki so currently cinder nfs driver calls qemu-img resize to extend a volume, attached or not
17:19:15 lseki https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/nfs.py#L380
17:19:15 lseki https://github.com/openstack/cinder/blob/master/cinder/image/image_utils.py#L408
17:19:15 lseki the step 1 is to stop calling qemu-img resize if the volume is attached
17:22:04 lseki but regarding the step 2, I'm not familiar with nova code... is there some workflow we can modify to make it call qemu monitor command?
17:22:32 sean-k-mooney this might be in os-brick im not certin
17:22:56 sean-k-mooney lseki: stephenfin or lyarwood have looked at this more recently then i have
17:23:59 sean-k-mooney lseki: is there a bug for this
17:24:10 sean-k-mooney lseki: if so can you summerise this in a comment
17:24:18 lseki https://bugs.launchpad.net/cinder/+bug/1870367
17:24:18 openstack Launchpad bug 1870367 in Cinder "Fail to extend attached volume using generic NFS driver" [High,In progress] - Assigned to Silvan Kaiser (2-silvan)
17:24:31 lseki yeah I'll add a comment there
17:25:42 lyarwood sean-k-mooney / lseki ; we already call blockResize in n-cpu
17:26:15 sean-k-mooney lyarwood: ok so we just need to not call qemu-img in cinder or nova when using nfs
17:26:25 sean-k-mooney and the volume is attached
17:26:57 lyarwood sean-k-mooney: we just need to remove the call in c-vol
17:27:01 lyarwood https://github.com/lyarwood/nova/blob/d4c857dfcb1ccfa5410de55671e69c722bbc990e/nova/virt/libvirt/driver.py#L2044-L2048
17:27:05 lyarwood https://github.com/lyarwood/nova/blob/d4c857dfcb1ccfa5410de55671e69c722bbc990e/nova/virt/libvirt/guest.py#L811-L819
18:03:02 lseki lyarwood / sean-k-mooney: is blockResize called always, no matter what c-vol did before?
18:15:35 lseki testing it in a stable/train devstack here...
18:27:00 lyarwood lseki: yes
18:34:04 lseki oh no
18:34:14 lseki https://www.irccloud.com/pastebin/hmybxXhk/
18:49:35 lseki the driver being used is `LibvirtNFSVolumeDriver`, with parent `LibvirtBaseFileSystemVolumeDriver` and grandpa `LibvirtBaseVolumeDriver`
18:49:54 lseki none of them implement `extend_volume` :-(
18:54:53 lseki so it stops here https://github.com/lyarwood/nova/blob/d4c857dfcb1ccfa5410de55671e69c722bbc990e/nova/virt/libvirt/driver.py#L2100 before reaching `_resize_attached_volume`
19:05:41 lseki I implemented `LibvirtNFSVolumeDriver.extend_volume` that just returns `requested_size`, and then attached extend worked!
19:05:47 lseki 🎉
19:05:47 lseki 🎉

Earlier   Later