| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-06-06 | |||
| 17:01:22 | sean-k-mooney | that may have changed recently but it was a limitation in the past | |
| 17:01:37 | opendevreview | sean mooney proposed openstack/nova master: Allow discard with virtio-blk https://review.opendev.org/c/openstack/nova/+/878795 | |
| 17:01:42 | geguileo | sean-k-mooney: but if it does now then maybe we have to give it another go at this whole thing | |
| 17:01:52 | sean-k-mooney | geguileo: without ^ we also dont support discard for all disk buses | |
| 17:02:18 | sean-k-mooney | geguileo: sure but we have to be careful to ensure that the move ops work properly | |
| 17:02:22 | geguileo | sean-k-mooney: that patch should be abandoned... I added a comment to the LP bug | |
| 17:02:33 | sean-k-mooney | that is not your patch | |
| 17:02:38 | sean-k-mooney | its my patch to fix the bug | |
| 17:02:45 | sean-k-mooney | that for some reason was not in launchpad | |
| 17:02:49 | geguileo | sean-k-mooney: how is it different to mine? | |
| 17:03:02 | geguileo | it's literaly the same | |
| 17:03:06 | geguileo | and it doesn't work | |
| 17:03:07 | sean-k-mooney | i make discard work for disk buses | |
| 17:03:20 | geguileo | it just removes a debug log message | |
| 17:03:28 | geguileo | and the log is correct | |
| 17:03:32 | geguileo | discard doesn't work with virtio | |
| 17:03:37 | geguileo | don't know why | |
| 17:03:37 | sean-k-mooney | yes it does | |
| 17:03:49 | sean-k-mooney | it requires a min version fo qemu and libvirt | |
| 17:04:02 | geguileo | I could only make it work with IDE, SCSI, and SATA | |
| 17:04:29 | sean-k-mooney | i can try and find the downstream bz for virtio-blk again one sec | |
| 17:04:38 | geguileo | sean-k-mooney: I have the downstream BZ | |
| 17:04:45 | geguileo | sean-k-mooney: I'm just telling you I can't make it work | |
| 17:04:47 | sean-k-mooney | for qemu supprot of virtio-blk | |
| 17:04:51 | geguileo | (maybe I'm dumb) | |
| 17:04:54 | sean-k-mooney | *trim with virtio--blk | |
| 17:05:06 | geguileo | sean-k-mooney: sure, it says it supports it... I can't make it work without using IDE, SCSI or SATA | |
| 17:05:08 | sean-k-mooney | well it worked in our ci | |
| 17:05:16 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/879077/1 | |
| 17:05:20 | geguileo | sean-k-mooney: did you actually check that the size was reduced? | |
| 17:05:44 | sean-k-mooney | no but if its not that a qemu bug | |
| 17:05:55 | geguileo | sean-k-mooney: but then the log should remain | |
| 17:06:13 | sean-k-mooney | why it would not be correct | |
| 17:06:41 | geguileo | just because fstrim says it has freed space within the guest OS it doesn't mean that it has actually hapened | |
| 17:07:11 | sean-k-mooney | sure there are several layeres at play here | |
| 17:07:24 | geguileo | sean-k-mooney: oh, I'm sure there are many, but they are beyond my expertise | |
| 17:07:39 | geguileo | I'm just reporting as a storage guy saying, don't know why I can't make it work | |
| 17:07:41 | geguileo | lol | |
| 17:07:53 | sean-k-mooney | so you were using local sotrage | |
| 17:07:55 | sean-k-mooney | no cinder | |
| 17:07:57 | sean-k-mooney | booted a vm | |
| 17:08:02 | sean-k-mooney | allcoated space | |
| 17:08:11 | geguileo | I was booting a VM from RBD, iSCSI, NFS | |
| 17:08:12 | sean-k-mooney | and then deleteed it an ddid a trim | |
| 17:08:45 | geguileo | Then I did a live volume migration, which made the disk loose sparseness (became thick) | |
| 17:08:54 | geguileo | then I issued the "fstrim -v --all" | |
| 17:08:59 | sean-k-mooney | well i filed https://bugs.launchpad.net/nova/+bug/2013123 for local storage | |
| 17:09:17 | geguileo | sean-k-mooney: yeah, I replied in that LP bug | |
| 17:09:45 | geguileo | sean-k-mooney: I'm working on a sparseness document, because this is a CF | |
| 17:09:59 | geguileo | (including the cinder side) | |
| 17:10:11 | sean-k-mooney | well discard is not just about sparceness | |
| 17:10:27 | sean-k-mooney | but for what its worth we do not make any statement about spacencie at the api level | |
| 17:11:22 | dansmith | AFAIK, with various file formats you can only expect the used size to decrease if you fully discard a block that covers an extent | |
| 17:11:23 | geguileo | sean-k-mooney: yeah, it's also about SSDs optimization, power consumption, etc | |
| 17:11:36 | dansmith | vmware, TMK, only actually reclaims space when the guest is shutdown | |
| 17:11:51 | geguileo | dansmith: true, but even then I can clearly see the reduction | |
| 17:12:11 | geguileo | I mean, the size goes down from 1GB to 100MB or so... | |
| 17:12:14 | sean-k-mooney | so for example i dont knwo if qcow or raw files will actully reduce space | |
| 17:12:27 | geguileo | sean-k-mooney: qcow2 does | |
| 17:12:32 | sean-k-mooney | for qcow i woudl expect it to be reduced for raw proably not | |
| 17:12:33 | geguileo | it things are set correctly | |
| 17:12:38 | geguileo | (aka all the starts align) | |
| 17:12:51 | geguileo | s/starts/stars | |
| 17:13:12 | geguileo | and I can even make NFS/qcow2 and RBD preserve sparseness on live migration | |
| 17:13:19 | geguileo | changing the nova code | |
| 17:13:33 | sean-k-mooney | how did you chagne the nova code | |
| 17:13:35 | geguileo | to use the detect_zeroes feature | |
| 17:13:52 | geguileo | I created this LP bug for that one https://bugs.launchpad.net/nova/+bug/2023079 | |
| 17:13:56 | sean-k-mooney | ok then we can add that as a new feature if you can explain what it is | |
| 17:14:06 | sean-k-mooney | that is not a bug it woudl be a new feature | |
| 17:14:19 | sean-k-mooney | a small one but its still a feature | |
| 17:14:39 | geguileo | sean-k-mooney: https://paste.openstack.org/show/brFgX6MgBlxjgCrE3rbg/ | |
| 17:14:52 | sean-k-mooney | geguileo: what do you mean by volume in https://bugs.launchpad.net/nova/+bug/2023079 | |
| 17:14:55 | geguileo | sean-k-mooney: but that's not the right code | |
| 17:14:59 | geguileo | it's what I used to test | |
| 17:15:27 | sean-k-mooney | oh "When doing a live volume migration " | |
| 17:15:32 | geguileo | but that has CPU implications when running, so it's best only to change it when the block migration is going to happen | |
| 17:15:44 | sean-k-mooney | geguileo: we have a differnt concept called block-migration in the context of live migrating a vm | |
| 17:15:53 | dansmith | ...yeah | |
| 17:16:03 | geguileo | sean-k-mooney: yes, I mean block live migration, I've updated the bug name, thanks | |
| 17:16:04 | dansmith | I was going to say, we should never be block migrating a volume | |
| 17:16:25 | geguileo | sean-k-mooney: ooooh, then soooooooorry for mixing terms (/me facepalms) | |
| 17:16:27 | sean-k-mooney | geguileo: block live migratin in nova mean live migrate a vm with local raw/qcow storage | |
| 17:16:43 | dansmith | it means we literally move all the data | |
| 17:16:56 | dansmith | (all the disk data) | |
| 17:17:03 | geguileo | my bad, I've updated the LP bug | |
| 17:17:10 | geguileo | dansmith: we are currently moving ALL the data | |
| 17:17:23 | sean-k-mooney | geguileo: ya so i know knwo what your trying to fix | |
| 17:17:27 | geguileo | that's why the detect_zeroes would be good for volume live migration | |
| 17:17:43 | geguileo | sean-k-mooney: thanks for you patience in understanding my ramblings :-) | |
| 17:17:43 | sean-k-mooney | so we expect this to be done by cider using the driver assited migration feature | |
| 17:17:57 | sean-k-mooney | however you want nova ot be intelegent enough | |
| 17:18:07 | sean-k-mooney | so that when we fallback to nova doing the volume migration | |
| 17:18:15 | sean-k-mooney | that we also preseve the sparceness | |
| 17:18:23 | geguileo | sean-k-mooney: problem is that driver assisted migration cannot work between different backends (and afaik it doesn't work for any driver even between volumes of the same array) | |
| 17:18:41 | geguileo | sean-k-mooney: afaik today all online volume migrations are done by nova | |
| 17:18:51 | dansmith | wait what? | |
| 17:18:54 | geguileo | I don't think any cinde driver supports it | |
| 17:19:00 | opendevreview | Amit Uniyal proposed openstack/nova master: Delete dangling bdms https://review.opendev.org/c/openstack/nova/+/882284 | |
| 17:19:00 | opendevreview | Amit Uniyal proposed openstack/nova master: Reproducer for dangling bdms https://review.opendev.org/c/openstack/nova/+/881457 | |