| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-17 | |||
| 11:29:42 | kashyap | mdbooth: Noted | |
| 11:29:54 | kashyap | So, here we go, 8 lines of paste: | |
| 11:29:55 | kashyap | 12:21 < mreitz> Well, if an area is allocated nowhere in the backing chain, block-stream won't allocate it | |
| 11:29:59 | kashyap | 12:21 * kashyap does notice 'detect_zeroes' flag as an option for @BlockDeviceInfo | |
| 11:30:02 | kashyap | 12:22 < mreitz> At least from what I can tell from the code | |
| 11:30:04 | kashyap | 12:22 < mreitz> Yeah, you can open the target BDS with detect-zeroes=on... | |
| 11:30:07 | kashyap | 12:24 < mreitz> But as far as I can tell, copy-on-read actually always uses zero detection and converts requests accordingly | |
| 11:30:10 | kashyap | 12:24 < mreitz> And block-stream performs its writes through the copy-on-read mechanic | |
| 11:30:13 | kashyap | 12:24 < kashyap> mreitz: Ah, so, in that case (when the target block dev is opened with 'detect-zeroes=on'), thin provisioning will be retained? | |
| 11:30:16 | kashyap | 12:24 < mreitz> So I'd think that block-stream should always do zero detection | |
| 11:31:21 | mdbooth | kashyap: Ah, ok. So there's no detect zeroes option because it's always done. | |
| 11:31:45 | mdbooth | So we'd just need to ensure we're doing it on the output? | |
| 11:32:40 | kashyap | mdbooth: I think so. The experiment is: | |
| 11:32:43 | mdbooth | gmann: Anyway, ^^^ is orthogonal to cinderv3. | |
| 11:32:43 | kashyap | Build a backing chain of empty images, stream and see whether something gets allocated. | |
| 11:33:05 | mdbooth | As johnthetubaguy says. | |
| 11:33:24 | mdbooth | And I don't see how we could support swap_volume with multi_attach in any case. | |
| 11:34:21 | mdbooth | kashyap: Never mind the backing chain. Create a sparse flat file with some data allocated in the middle and stream that. | |
| 11:34:39 | mdbooth | Can you get it to create an output file which is also sparse? | |
| 11:34:55 | kashyap | Need to try it out. | |
| 11:35:09 | kashyap | mdbooth: You have a quick way to write zeroes to a qcow2 file? | |
| 11:35:32 | mdbooth | kashyap: To be a good test, I would explicitly not use qcow2 at any point. | |
| 11:35:43 | mdbooth | Should be flat at both source and dest. | |
| 11:35:48 | kashyap | [Maybe dd if=/dev/zero...] | |
| 11:35:59 | mdbooth | However, creating sparse qcow2 is trivial, right? | |
| 11:36:11 | mdbooth | Also, libguestfs if you want to write data into it. | |
| 11:36:12 | kashyap | mdbooth: Not use QCOW2 because, to remove its metadata complications? | |
| 11:36:40 | mdbooth | Yeah, the interesting test is whether sparse detection works *without* qcow2 metadata. | |
| 11:36:56 | kashyap | Oh, yes, 'guestfish'. In the past I did use it, instead of 'dd' | |
| 11:42:08 | kashyap | mdbooth: Sparse qcow2 is the default, no? (When you do `qemu-img create -f qcow2 foo.qcow2 10M`) | |
| 11:44:54 | gmann | kashyap: mdbooth seems interesting to me. but i have to go home now(its late). ll read log and catch you guys later. thanks for help btw | |
| 11:45:37 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/446777 | |
| 12:18:57 | openstackgerrit | Gábor Antal proposed openstack/nova master: Add json style checking for sample notifications https://review.openstack.org/443677 | |
| 12:20:30 | openstackgerrit | Gábor Antal proposed openstack/nova master: Add json style checking for sample notifications https://review.openstack.org/443677 | |
| 12:36:55 | cdent | edleafe: not sure if this will be helpful on your functional test explorations but I was just reminded that I had this that I started before jay's big functional test he did: https://review.openstack.org/#/c/427309/ | |
| 12:55:31 | edleafe | kashyap: are you around for some questions? | |
| 12:55:47 | dims | cdent : could use a "## Need Volunteers/help" section :) thanks for the updates! | |
| 12:56:06 | edleafe | cdent: my problem was the driver mocks. Started on it, but on a client call now | |
| 12:56:06 | openstackgerrit | ChangBo Guo(gcb) proposed openstack/nova master: Remove old oslo.messaging transport aliases https://review.openstack.org/446990 | |
| 12:56:07 | kashyap | edleafe: Hiya, yes. I can try :-) | |
| 12:56:55 | cdent | dims: that's a good idea but I would be half inclined to put that as the top level heading and indent everything else :) | |
| 12:57:06 | dims | :) | |
| 12:58:46 | edleafe | kashyap: it's the same problem that mriedem was talking to you about - volume migration failures | |
| 12:59:27 | edleafe | kashyap: on a call with the client now - give me a few | |
| 13:00:51 | kashyap | edleafe: Sure. Just to note: I wasn't following the entire issue from Nova POV, mdbooth is more in the trenches there. I was mostly following it from libvirt / QEMU perspective | |
| 13:03:43 | edleafe | kashyap: well, there are two different errors: one from libvirt, and one from QEMU :) | |
| 13:04:06 | kashyap | edleafe: One of them is the memory corruption issue? | |
| 13:04:10 | kashyap | (The one from libvirt) | |
| 13:06:03 | kashyap | And do you also have a log for the other one from QEMU? | |
| 13:06:29 | edleafe | kashyap: I can get them | |
| 13:06:33 | mriedem | kashyap: edleafe isn't talking about the thing from yesterday | |
| 13:06:39 | mriedem | he's talking about volume swap | |
| 13:06:46 | edleafe | kashyap: the error is libvirtError: Requested operation is not valid: pivot of disk 'vdb' requires an active copy job | |
| 13:06:57 | kashyap | mriedem: Ah, a different issue | |
| 13:07:16 | edleafe | This is on a newer libvirt, using Mitaka with Matt's patches | |
| 13:07:17 | mdbooth | edleafe: That's an old one | |
| 13:07:35 | edleafe | mdbooth: I saw the bugzilla reports | |
| 13:07:48 | edleafe | mdbooth: their libvirt should have the fixes for that | |
| 13:08:04 | kashyap | mriedem: Meanwhile, for the Gate issue mdbooth was looking yesterday, please see the comment here when you get a moment -- https://bugs.launchpad.net/nova/+bug/1673483/comments/3 | |
| 13:08:04 | openstack | Launchpad bug 1673483 in OpenStack Compute (nova) "libvirt: test_attach_volume_shelved_or_offload_server times out waiting for device detach (which fails)" [Undecided,Confirmed] | |
| 13:08:13 | mdbooth | edleafe: The latest Nova should work with old libvirt on that one | |
| 13:08:22 | kashyap | mriedem: I also notified folks on #ubuntu-packaging IRC on Freenode -- so they can pull in the missing patch | |
| 13:08:45 | edleafe | mriedem: so would your patches make their code "the lates Nova"? | |
| 13:08:54 | edleafe | at least as far as this issue is concerned | |
| 13:09:01 | mdbooth | kashyap: Ubuntu have engineers? I thought they just had marketing. | |
| 13:09:14 | kashyap | mdbooth: :-) C'mon, don't be so mean | |
| 13:09:44 | kashyap | Credit where it's due: Serge Hallyn is a real machine | |
| 13:09:51 | mriedem | edleafe: i had backported to mitaka the fix i thought was related to this pivot issue that lyarwood worked on | |
| 13:10:10 | kashyap | mdbooth: He does most of the work. And once when I met him at a conference, and asked him how on earth is he handling it all | |
| 13:10:13 | edleafe | mriedem: yeah, that's what I thought | |
| 13:10:19 | kashyap | He said: "Yeah, my SO complains..." | |
| 13:10:23 | mriedem | edleafe: this https://review.openstack.org/#/c/433793/ | |
| 13:11:01 | kashyap | But that's just one person. I admittedly don't know any others | |
| 13:11:55 | kashyap | mriedem: Why did you abandon it? | |
| 13:13:05 | mdbooth | mriedem edleafe: Landed in Newton: | |
| 13:13:17 | mdbooth | https://review.openstack.org/#/c/427290/ | |
| 13:15:24 | mriedem | kashyap: mitaka | |
| 13:15:34 | mriedem | kashyap: it's not justified for mitaka at this point | |
| 13:15:37 | kashyap | Ah! | |
| 13:15:48 | kashyap | For a moment, lost track of release status | |
| 13:16:03 | kashyap | Says, it's "Legacy Release" phase https://releases.openstack.org/ | |
| 13:16:26 | kashyap | edleafe: That fix mriedem pointed out is what you're looking for. | |
| 13:17:08 | edleafe | kashyap: just checked their system, and they have those changes applied | |
| 13:17:55 | kashyap | edleafe: And they still see the same pivot failure? | |
| 13:18:35 | SzarkaAlex | @gibi Hi! Are you here? | |
| 13:20:11 | mdbooth | edleafe: Can you do me a favour. There's an edge case in that code that we knowingly glossed over, namely that we can't detect if the job fails. | |
| 13:21:19 | mdbooth | edleafe: Can you correlate the Nova log with the libvirt logs and look for a failure of the rebase job? | |
| 13:21:54 | edleafe | mdbooth: sec... | |
| 13:23:20 | edleafe | mdbooth: like this? http://paste.openstack.org/show/603131/ | |
| 13:23:38 | mdbooth | edleafe: That's the nova log. | |
| 13:23:47 | edleafe | ah | |
| 13:23:49 | mdbooth | Can you look in the libvirt log? | |
| 13:23:59 | mdbooth | /var/log/libvirt/whatever probably | |
| 13:24:36 | edleafe | looking... | |
| 13:24:55 | mdbooth | kashyap: 'requires an active copy job' | |
| 13:25:22 | mdbooth | Is that the same as the pivot-too-early problem? | |
| 13:25:43 | kashyap | mdbooth: Oh, I think vaguely recall, I think that edge case you discussed in one of the reviews... | |
| 13:25:57 | kashyap | mdbooth: Yes, it's the same problem | |
| 13:26:10 | kashyap | Wait, let me think again | |
| 13:26:29 | kashyap | mdbooth: No, I was wrong: | |