| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-07-01 | |||
| 13:37:41 | sean-k-mooney | lseki: which s not normally how a volume resize would work | |
| 13:37:59 | sean-k-mooney | lseki: a volume resize for other driver would have the backend to the resize | |
| 13:38:13 | sean-k-mooney | lseki: i think that only works because qemu is the thing with the lock | |
| 13:38:37 | sean-k-mooney | we could special case but that would fail for iscsi backend and proably ceph | |
| 13:38:37 | lyarwood | well, c-vol resizes the underlying device, n-cpu then resizes the host device on the compute and then we resize the device within QEMU | |
| 13:39:03 | lseki | sean-k-mooney: hmm would it be possible to add a condition to do that only when the backend driver is qemu? | |
| 13:39:26 | lyarwood | the issue here is that we are asking qemu-img to resize the host device, in this case a file. | |
| 13:39:26 | sean-k-mooney | lyarwood: right but presumably the nfs driver cant resize the file because qemu has locked the file right | |
| 13:39:31 | lyarwood | yeah | |
| 13:40:04 | lyarwood | but I guess c-vol is actually calling this | |
| 13:40:06 | sean-k-mooney | lseki: you mean the backedn driver is nfs | |
| 13:40:26 | lseki | sean-k-mooney: oops, yes | |
| 13:40:33 | sean-k-mooney | lyarwood: yes becasue if it was not attached to a vm then it would have to do the resize | |
| 13:40:37 | lyarwood | I think we should be safe to add --force-share to the qemu-img resize call in c-vol tbh | |
| 13:41:09 | sean-k-mooney | lyarwood: will that also resize the filesystem | |
| 13:41:16 | lyarwood | sean-k-mooney: no | |
| 13:41:28 | sean-k-mooney | jsut the qcow/raw file | |
| 13:41:36 | lyarwood | right, virt-resize does both | |
| 13:41:40 | lyarwood | or it's up to the user | |
| 13:41:54 | lyarwood | once n-cpu has called QEMU to resize the device within the domain | |
| 13:43:15 | sean-k-mooney | well i know for a server resize we resize the filesystem automatically | |
| 13:44:34 | lyarwood | that's only for ephemeral disks where we know the format right? | |
| 13:53:17 | sean-k-mooney | yes | |
| 13:53:45 | sean-k-mooney | basically whatever libguestfs supports | |
| 13:54:03 | sean-k-mooney | if it does recognise the filesytem i think it jsut resizes the block device | |
| 14:11:07 | lseki | https://github.com/openstack/cinder/blob/f74c533382ec256ca7dda41e8e451392d81e2944/cinder/image/image_utils.py#L353 | |
| 14:11:07 | lseki | lyarwood: your suggestion is to add `--force-share` to `qemu-img resize` command? | |
| 14:11:14 | lyarwood | lseki: yeah correct | |
| 14:12:30 | lseki | oh | |
| 14:12:37 | lseki | https://www.irccloud.com/pastebin/LBtS5n4l/ | |
| 14:13:43 | lyarwood | lseki: which version of QEMU are you using? | |
| 14:14:08 | lyarwood | lseki: I can't recall which version we require to use that switch as Nova already requires it on the computesd | |
| 14:14:21 | sean-k-mooney | so we can set the trasport we use for cold migration with https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.remote_filesystem_transport | |
| 14:14:29 | lseki | https://www.irccloud.com/pastebin/U58tnfou/ | |
| 14:14:30 | sean-k-mooney | do we have a way to choose the ssh user? | |
| 14:14:45 | sean-k-mooney | like we do for live migration | |
| 14:16:42 | lyarwood | lseki: https://www.irccloud.com/pastebin/U58tnfou/ - hmm 2.11 should have it, we dropped the version check once we hit that | |
| 14:16:44 | lyarwood | or was that 2.12 | |
| 14:16:47 | sean-k-mooney | looks like no... i guess ill just excange ssh keys | |
| 14:17:30 | lyarwood | ah no that's 2.10 | |
| 14:17:37 | lyarwood | lseki: yeah weird you should have that | |
| 14:17:46 | lyarwood | maybe it's missing from resize? | |
| 14:18:41 | lyarwood | yeah it is | |
| 14:18:43 | lyarwood | odd | |
| 14:33:19 | lseki | oh... seems that there's no force-resize param for `img_resize` https://github.com/qemu/qemu/blob/6bb228190ef0b45669d285114cf8a280c55f4b39/qemu-img.c#L3856_L4053 | |
| 14:34:43 | lseki | oops wrong branch | |
| 14:34:49 | lyarwood | yeah I don't really know what we could do in this case | |
| 14:35:02 | lseki | but it's not in the master branch as well https://github.com/qemu/qemu/blob/master/qemu-img.c#L3856_#L4053 | |
| 14:35:28 | lseki | 😢 | |
| 14:35:28 | lseki | 😢 | |
| 14:36:55 | lyarwood | yeah apologies, I assumed it was an available option across the subcommands but that isn't the case | |
| 14:37:20 | lyarwood | we can just truncate RAW files | |
| 14:37:38 | lyarwood | but these are qcow2 | |
| 14:38:05 | sean-k-mooney | we dont allow resize down of attached volumes right | |
| 14:38:10 | sean-k-mooney | that sound bad | |
| 14:38:13 | lyarwood | no I don't think so | |
| 14:38:28 | sean-k-mooney | so you just mean use the truncate command but not to shrink | |
| 14:38:49 | lyarwood | https://docs.openstack.org/api-ref/block-storage/v3/index.html?expanded=extend-a-volume-size-detail#extend-a-volume-size | |
| 14:38:59 | lyarwood | sean-k-mooney: yeah you can use truncate to grow files | |
| 14:39:32 | lyarwood | sean-k-mooney: but that wouldn't work in this case as it wouldn't update the qcow2 headers to use the extra space | |
| 14:39:33 | sean-k-mooney | ya wasnt sure if you ment the command or not | |
| 14:39:45 | sean-k-mooney | yes | |
| 14:40:45 | lyarwood | a hackaround would be to use swap_volume in n-cpu to switch to a larger file | |
| 14:41:12 | sean-k-mooney | lyarwood: so what we likely need to do is if the volume is attach have the backend not resize it and have nova via the qemu monitor command have qemu do the resize | |
| 14:41:24 | lyarwood | tbh using that approach for all backends might be nicer anyway now I think about it | |
| 14:41:43 | sean-k-mooney | so if volume is available then c-vol will do the resize else nova will just invoke qemu to do it | |
| 14:41:59 | sean-k-mooney | and we never call qemu-img for nfs backend | |
| 14:42:16 | lyarwood | block resize by QEMU isn't going to resize the file | |
| 14:42:27 | lyarwood | just as it isn't going to resize host devices for other backends AFAIK | |
| 14:43:08 | sean-k-mooney | so `sudo virsh qemu-monitor-command ubuntu18.04 --hmp "block_resize drive-virtio-disk1 2G"` | |
| 14:43:14 | sean-k-mooney | wont grow the file | |
| 14:43:22 | sean-k-mooney | lseki: didnt you say that worked | |
| 14:43:25 | lyarwood | I don't believe so | |
| 14:43:44 | sean-k-mooney | for qcow it does not need to make the file physically bigger | |
| 14:43:51 | sean-k-mooney | just update the headers | |
| 14:44:02 | sean-k-mooney | so that qemu can allocate it when needed | |
| 14:44:12 | sean-k-mooney | which is what we want. e.g. keep the file sparse | |
| 14:44:58 | lyarwood | hmmm is that enough? | |
| 14:45:11 | sean-k-mooney | im not sure but lseki could test with dd? | |
| 14:45:37 | lyarwood | I thought you had to at least grow the sparse file so it looks like it's 2G before you can update the qcow2 headers to be 2G | |
| 14:46:05 | sean-k-mooney | not sure but it sounds like something we should test | |
| 14:46:22 | lyarwood | yup | |
| 14:46:23 | lyarwood | brb | |
| 14:47:19 | sean-k-mooney | lseki: can you try that. grow you iamge to like 10G and then use dd to crate a 5G file in the vm | |
| 14:47:27 | lseki | sean-k-mooney: lyarwood oops, I was using a raw file instead of qcow | |
| 14:47:36 | sean-k-mooney | if it had less then 2G before that would fail | |
| 14:47:50 | sean-k-mooney | lseki: oh in that case what the size of the file on disk after that command | |
| 14:47:56 | sean-k-mooney | did it grow it to 2G | |
| 14:48:22 | lseki | yes it did, and `lsblk` inside the vm also is updated to 2G | |
| 14:48:54 | sean-k-mooney | could you convert it over to a qcow boot a vm with the new image and then grow it to say 4G | |
| 14:49:01 | sean-k-mooney | using the same command on the new vm | |
| 14:49:12 | sean-k-mooney | lets see if it will do the right thing in that case | |
| 14:49:30 | sean-k-mooney | if so i think we have a way to adresss this | |
| 14:51:52 | lseki | let me try that... | |
| 15:03:37 | lyarwood | sean-k-mooney: TIL I didn't think blockResize did that, given that we resized the file before I guess that was the case and something has changed? | |
| 15:04:14 | sean-k-mooney | from the name that is what i would expect ti to do | |
| 15:04:28 | sean-k-mooney | maybe not for host devices | |
| 15:04:32 | sean-k-mooney | but for files at least | |
| 15:04:34 | lyarwood | yeah it can't for host devices | |
| 15:05:02 | lyarwood | I might be mixing things up with blockRebase tbh | |