| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-07-01 | |||
| 01:49:18 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: DNM: testing nova gate on ubuntu focal https://review.opendev.org/738126 | |
| 02:10:27 | openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: YAML file loading and schema validation https://review.opendev.org/673341 | |
| 02:10:28 | openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: Functions to merge provider configs to provider tree https://review.opendev.org/676522 | |
| 02:10:28 | openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: Enable loading and merging of provider configs https://review.opendev.org/693460 | |
| 02:10:29 | openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: Function to further validate and retrieve configs https://review.opendev.org/676029 | |
| 02:22:38 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: DNM: testing nova gate on ubuntu focal https://review.opendev.org/738126 | |
| 02:27:42 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Fix user creation with GRANT in MySQL 8.0(Ubuntu Focal) https://review.opendev.org/738723 | |
| 02:28:06 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: DNM: testing nova gate on ubuntu focal https://review.opendev.org/738126 | |
| 02:30:14 | gmann | dansmith: melwitt gibi Ubuntu Focal fix for tool/test-setup.sh - https://review.opendev.org/#/c/738723/ Testing it in https://review.opendev.org/#/c/738126/ . | |
| 02:31:11 | gmann | Result should be ready by morning. I will be on holiday tomorrow, so pinged before gate result. | |
| 03:04:29 | openstackgerrit | Wenping Song proposed openstack/nova master: Add instance project_id for cyborg arq https://review.opendev.org/738428 | |
| 06:55:14 | openstackgerrit | Jiri Suchomel proposed openstack/nova master: Move image verification and writing out of download method https://review.opendev.org/738738 | |
| 07:43:55 | huaqiang | a | |
| 07:58:50 | openstackgerrit | Jiri Suchomel proposed openstack/nova master: Move image verification and writing out of download method https://review.opendev.org/738738 | |
| 10:57:46 | stephenfin | bauzas: Can you take a punt at https://review.opendev.org/#/c/733827/ ? | |
| 10:58:05 | bauzas | stephenfin: will look this afternoon | |
| 10:58:34 | stephenfin | Thanks | |
| 13:08:24 | lseki | Hello folks, could you give some suggestions on how to address this bug? | |
| 13:08:35 | lseki | https://bugs.launchpad.net/cinder/+bug/1870367 "Fail to extend attached volume using generic NFS driver" | |
| 13:08:35 | openstack | Launchpad bug 1870367 in Cinder "Fail to extend attached volume using generic NFS driver" [High,In progress] - Assigned to Silvan Kaiser (2-silvan) | |
| 13:08:45 | lseki | in summary, when trying to resize a volume in-use, `qemu-img resize` command fails to get the write lock, because `qemu` process has locked the volume file upon VM start. | |
| 13:08:51 | lseki | This was discussed on Cinder PTG, but we didn't come up with a solution yet | |
| 13:16:23 | lyarwood | lseki: you can use --force-share to workaround the locks but I'm not sure that it would be the best approach here tbh | |
| 13:17:04 | lyarwood | lseki: but then again we were essentially doing that prior to QEMU taking the locks | |
| 13:18:11 | lyarwood | lseki: but without quessing the disk on the compute side first that might be problematic | |
| 13:21:03 | gibi | lyarwood: welcome back! | |
| 13:21:15 | lyarwood | gibi: \o hey thanks, glad to be back finally :) | |
| 13:24:32 | gibi | :) | |
| 13:33:39 | sean-k-mooney | lyarwood: we do quiese the disk for vm snapshots right | |
| 13:34:02 | sean-k-mooney | if we are doing a live snapshot via the qemu-guest agent? | |
| 13:34:20 | sean-k-mooney | we use the monitor to invoke freeze | |
| 13:34:38 | sean-k-mooney | could we do the same for volume snapshots? | |
| 13:36:22 | lyarwood | sean-k-mooney: we do for snapshots, we could for extend as well I guess but I'm not entirely sure if we need to tbh | |
| 13:36:26 | lseki | lyarwood: in my machine (without openstack, just qemu-kvm), I managed to resize an attached disk with the command | |
| 13:36:31 | lseki | `sudo virsh qemu-monitor-command ubuntu18.04 --hmp "block_resize drive-virtio-disk1 2G"` | |
| 13:37:29 | sean-k-mooney | lseki: well that is having qemu do the resize | |
| 13:37:41 | sean-k-mooney | lseki: which s not normally how a volume resize would work | |
| 13:37:59 | sean-k-mooney | lseki: a volume resize for other driver would have the backend to the resize | |
| 13:38:13 | sean-k-mooney | lseki: i think that only works because qemu is the thing with the lock | |
| 13:38:37 | lyarwood | well, c-vol resizes the underlying device, n-cpu then resizes the host device on the compute and then we resize the device within QEMU | |
| 13:38:37 | sean-k-mooney | we could special case but that would fail for iscsi backend and proably ceph | |
| 13:39:03 | lseki | sean-k-mooney: hmm would it be possible to add a condition to do that only when the backend driver is qemu? | |
| 13:39:26 | sean-k-mooney | lyarwood: right but presumably the nfs driver cant resize the file because qemu has locked the file right | |
| 13:39:26 | lyarwood | the issue here is that we are asking qemu-img to resize the host device, in this case a file. | |
| 13:39:31 | lyarwood | yeah | |
| 13:40:04 | lyarwood | but I guess c-vol is actually calling this | |
| 13:40:06 | sean-k-mooney | lseki: you mean the backedn driver is nfs | |
| 13:40:26 | lseki | sean-k-mooney: oops, yes | |
| 13:40:33 | sean-k-mooney | lyarwood: yes becasue if it was not attached to a vm then it would have to do the resize | |
| 13:40:37 | lyarwood | I think we should be safe to add --force-share to the qemu-img resize call in c-vol tbh | |
| 13:41:09 | sean-k-mooney | lyarwood: will that also resize the filesystem | |
| 13:41:16 | lyarwood | sean-k-mooney: no | |
| 13:41:28 | sean-k-mooney | jsut the qcow/raw file | |
| 13:41:36 | lyarwood | right, virt-resize does both | |
| 13:41:40 | lyarwood | or it's up to the user | |
| 13:41:54 | lyarwood | once n-cpu has called QEMU to resize the device within the domain | |
| 13:43:15 | sean-k-mooney | well i know for a server resize we resize the filesystem automatically | |
| 13:44:34 | lyarwood | that's only for ephemeral disks where we know the format right? | |
| 13:53:17 | sean-k-mooney | yes | |
| 13:53:45 | sean-k-mooney | basically whatever libguestfs supports | |
| 13:54:03 | sean-k-mooney | if it does recognise the filesytem i think it jsut resizes the block device | |
| 14:11:07 | lseki | lyarwood: your suggestion is to add `--force-share` to `qemu-img resize` command? | |
| 14:11:07 | lseki | https://github.com/openstack/cinder/blob/f74c533382ec256ca7dda41e8e451392d81e2944/cinder/image/image_utils.py#L353 | |
| 14:11:14 | lyarwood | lseki: yeah correct | |
| 14:12:30 | lseki | oh | |
| 14:12:37 | lseki | https://www.irccloud.com/pastebin/LBtS5n4l/ | |
| 14:13:43 | lyarwood | lseki: which version of QEMU are you using? | |
| 14:14:08 | lyarwood | lseki: I can't recall which version we require to use that switch as Nova already requires it on the computesd | |
| 14:14:21 | sean-k-mooney | so we can set the trasport we use for cold migration with https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.remote_filesystem_transport | |
| 14:14:29 | lseki | https://www.irccloud.com/pastebin/U58tnfou/ | |
| 14:14:30 | sean-k-mooney | do we have a way to choose the ssh user? | |
| 14:14:45 | sean-k-mooney | like we do for live migration | |
| 14:16:42 | lyarwood | lseki: https://www.irccloud.com/pastebin/U58tnfou/ - hmm 2.11 should have it, we dropped the version check once we hit that | |
| 14:16:44 | lyarwood | or was that 2.12 | |
| 14:16:47 | sean-k-mooney | looks like no... i guess ill just excange ssh keys | |
| 14:17:30 | lyarwood | ah no that's 2.10 | |
| 14:17:37 | lyarwood | lseki: yeah weird you should have that | |
| 14:17:46 | lyarwood | maybe it's missing from resize? | |
| 14:18:41 | lyarwood | yeah it is | |
| 14:18:43 | lyarwood | odd | |
| 14:33:19 | lseki | oh... seems that there's no force-resize param for `img_resize` https://github.com/qemu/qemu/blob/6bb228190ef0b45669d285114cf8a280c55f4b39/qemu-img.c#L3856_L4053 | |
| 14:34:43 | lseki | oops wrong branch | |
| 14:34:49 | lyarwood | yeah I don't really know what we could do in this case | |
| 14:35:02 | lseki | but it's not in the master branch as well https://github.com/qemu/qemu/blob/master/qemu-img.c#L3856_#L4053 | |
| 14:35:28 | lseki | 😢 | |
| 14:35:28 | lseki | 😢 | |
| 14:36:55 | lyarwood | yeah apologies, I assumed it was an available option across the subcommands but that isn't the case | |
| 14:37:20 | lyarwood | we can just truncate RAW files | |
| 14:37:38 | lyarwood | but these are qcow2 | |
| 14:38:05 | sean-k-mooney | we dont allow resize down of attached volumes right | |
| 14:38:10 | sean-k-mooney | that sound bad | |
| 14:38:13 | lyarwood | no I don't think so | |
| 14:38:28 | sean-k-mooney | so you just mean use the truncate command but not to shrink | |
| 14:38:49 | lyarwood | https://docs.openstack.org/api-ref/block-storage/v3/index.html?expanded=extend-a-volume-size-detail#extend-a-volume-size | |
| 14:38:59 | lyarwood | sean-k-mooney: yeah you can use truncate to grow files | |
| 14:39:32 | lyarwood | sean-k-mooney: but that wouldn't work in this case as it wouldn't update the qcow2 headers to use the extra space | |
| 14:39:33 | sean-k-mooney | ya wasnt sure if you ment the command or not | |
| 14:39:45 | sean-k-mooney | yes | |
| 14:40:45 | lyarwood | a hackaround would be to use swap_volume in n-cpu to switch to a larger file | |
| 14:41:12 | sean-k-mooney | lyarwood: so what we likely need to do is if the volume is attach have the backend not resize it and have nova via the qemu monitor command have qemu do the resize | |