Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-03-08
15:27:39 lyarwood kashyap: are you sure that resize only takes Bytes?
15:27:54 mriedem kashyap: i'm not sure what the volume is that it's resizing to, but this is what it's resizing from (i think)
15:28:07 mriedem http://paste.openstack.org/show/601953/
15:28:24 kashyap lyarwood: If you see the pastebin from mriedem, on line-4, the quoted size is the argument to the QMP command 'block_resize'
15:28:40 kashyap mriedem: The QEMU folks are asking about the details of the kind of storage
15:29:18 mriedem this is where the size is calculated https://review.openstack.org/#/c/433793/3/nova/virt/libvirt/driver.py@1396
15:29:21 gibi tojuvone: thanks for the info. I still feel it would be nice to have a non nova impacting alternative mentioned in the spec. but I have no hard feelings is we go forward with the nova based solution
15:29:41 kashyap lyarwood: See line 996 there -- http://git.qemu-project.org/?p=qemu.git;a=blob;f=qapi/block-core.json;h=9bb7f4a17ba9765245616212542725c60ec361a1;hb=HEAD#l996
15:29:54 mriedem ibm a9000 is what they are resizing to
15:30:07 kashyap And, from what?
15:30:58 mriedem HP 3PAR
15:31:02 lyarwood kashyap: thanks
15:31:51 tojuvone gibi, Thanks, I will work on that
15:32:23 kashyap mriedem: I've conveyed this detail to the QEMU folks on IRC, let you know once one of the block layer guys in EST wakes up
15:34:40 mriedem kashyap: thanks
15:34:55 kashyap mriedem: One more
15:35:08 mriedem even if it's 16GB, https://bugzilla.redhat.com/show_bug.cgi?id=1070531 seems to indicate it's still too large
15:35:08 openstack bugzilla.redhat.com bug 1070531 in qemu-kvm "qmp can not give a reasonable hint when block_resize a scsi_debug disk with 10G" [Low,Assigned] - Assigned to armbru
15:35:39 kashyap mriedem: What is the storage backend - is it a plain raw file, a qcow2 file, a SCSI block device, or something else?
15:36:29 openstackgerrit Maciej Szankin proposed openstack/nova master: conf: fix default values reporting infra worker https://review.openstack.org/441227
15:37:00 mriedem kashyap: i'll have to dig for that
15:37:15 mriedem kashyap: fo the thing being swapped to or from?
15:37:25 dansmith jaypipes: did you see my comments here? https://review.openstack.org/#/c/441544/4
15:37:35 dansmith jaypipes: I can fix them up if you don't have time
15:37:36 kashyap mriedem: The context for that question is, from danpb:
15:37:39 kashyap mriedem: To figure out where EINVAL comes from we need to understand what qemu block driver is getting used
15:38:03 dansmith jaypipes: specifically this: https://review.openstack.org/#/c/441544/4/nova/virt/ironic/driver.py
15:38:42 kashyap mriedem: Both.
15:40:51 kashyap mriedem: One more (credit for the guess: Eric Blake), about potentially what's going on:
15:40:52 mriedem maybe i can find some details from the compute logs
15:41:23 jaypipes dansmith: yeah, will get to that after meetings.
15:41:40 kashyap mriedem:
15:41:40 kashyap 16:39 < eblake> I know qcow2 prevents resizes that shrink the volume
15:41:40 kashyap 16:39 < eblake> could it be that the resize failed because of that?
15:41:40 kashyap 16:39 < kashyap> eblake: Could you see that from the pastebin, or are you guessing that there's a shrink going on?
15:41:43 kashyap 16:40 < eblake> I'm just guessing, as that would be one reason for calling the resize requested size invalid
15:41:49 dansmith jaypipes: okay, and then kinda next for this effort is the flavor override patch and the allocation side of this right?
15:42:00 jaypipes dansmith: yup.
15:42:17 dansmith jaypipes: okay cool
15:43:04 mriedem kashyap: we only resize if the new volume that we're resizing to is bigger https://review.openstack.org/#/c/433793/3/nova/compute/manager.py@5006
15:43:29 mriedem looking at the logs, the name on the A9000 volume implies it's an iscsi block device
15:43:39 mriedem however, it also says the size is 10GB,
15:43:49 mriedem but that's not what i'm seeing when we actually run the block_rebase command
15:43:54 kashyap mriedem: On the 16GB size comment, I'm being told it is "any size is valid as long as its below the max possible size; and 16G is trivially in range"
15:43:59 mriedem *block_resize
15:44:26 kashyap And the max possible size is at least a a TB of even EB, apparently
15:44:56 openstackgerrit Dan Smith proposed openstack/nova master: Sort CellMappingList.get_all() for safety https://review.openstack.org/443174
15:44:56 openstackgerrit Dan Smith proposed openstack/nova master: Add functools.wraps to rapper https://review.openstack.org/443175
15:45:57 kashyap mriedem: Apart from implication, can you confirm that it's indeed an iSCSI block dev?
15:47:43 mriedem yeah it's iscsi
15:47:57 efried sean-k-mooney mriedem https://review.openstack.org/#/c/435575/ didn't break our CI; it just makes the jenkins pep8 job fail on our code where it didn't before. Still figuring out how to pull in that plugin so I can validate locally, but without having to fix the whole nova project (which presumably got done by somebody else before this change was merged).
15:48:16 mriedem another thing is they swap from the A9000 to the 3PAR successfully and then they try to swap from the 3PAR to the A9000 and that fails
15:48:22 mriedem and i think that's actually a thing that was recently fixed
15:48:39 openstackgerrit Eric Fried proposed openstack/nova master: WIP: PowerVM Driver: SSP emphemeral disk support https://review.openstack.org/443189
15:49:25 sean-k-mooney mriedem: efried it broke our ci by what i consider to be a bug in devstack
15:49:25 mriedem i wonder if they are hitting https://review.openstack.org/#/c/431540/
15:49:30 mriedem kashyap: lyarwood ^
15:50:22 sean-k-mooney mriedem: when devstack does a pip install of the nova source it installs both the requirements.txt and test-requiremnts.txt gloablly
15:50:26 lyarwood mriedem: yeah that allows you to swap multiple times now
15:51:08 kashyap mriedem: Okay, danpb seems to have nailed it, with his QEMU POV:
15:52:13 kashyap Ok, so QEMU will be using a block device and you can't resize those. More specifically the 'block_resize' command when used against a block device, will just refresh qemu's view of the block device size, and if the requested resize size is larger than the block device size, it'l give EINVAL. IOW, you can use 'block_resize' on a SCSI device *if* you arrange for a resize to be done in the remote
15:52:19 kashyap iSCSI server first, and if you then tell the client host to rescan LUNs, and only then tell QEMU to apply the new size.
15:52:29 sean-k-mooney mriedem: that means that anyting else that uses pep8 is now runnning the addtional ordering tests
15:52:52 sean-k-mooney mriedem: as a result we cant build ovs-dpdk from sources as they dont enforcre the import ordering.
15:53:23 sean-k-mooney mriedem: devstack should not do https://github.com/openstack-dev/devstack/blob/6523d6e09789076d0a430bf7b608511fcb1c406a/inc/python#L336-L348 by default
15:55:43 dansmith johnthetubaguy: see the questions pending on this quotas thing? https://review.openstack.org/#/c/416521
15:55:52 dansmith johnthetubaguy: it doesn't call you out specifically, but it might as well :P
15:56:19 johnthetubaguy dansmith: on the TC call, those should keep me amused while I am listening
15:56:21 mriedem kashyap: so i got the GET volume response info from the compute logs which has the size for each volume, and they both say 10
15:56:27 efried sean-k-mooney Sounds reasonable to me. Launchpad bug? Propose a fix?
15:56:27 mriedem so i don't see why we're even attempting a resize
15:56:29 sean-k-mooney mriedem: the connect to ovs may not be apparent but there makefile enabled flake8 if its installed which it is tanks to devstack and the test defined by flake8-import-order seam to be enabled by default
15:56:39 dansmith johnthetubaguy: even quotas are better than TC stuff :)
15:56:55 kashyap mriedem: Heh, that's a good place to step back and see, if it's even needed.
15:56:58 mriedem kashyap: because resize_to should be 0 since this would be false https://review.openstack.org/#/c/433793/3/nova/compute/manager.py@5006
15:57:01 sean-k-mooney efried: am yes i have asked our ci team to open one and submit a patch to make it optional
15:58:21 mriedem kashyap: multipath is also enabled on these things, which i'm just going to wildly make a guess could ruin everything
15:58:29 sean-k-mooney efried: if we are gong to run the unit tests in the ci we will just use tox and it will install the test-requriments in the virtual env itself so i dont know why devstack is installing them by default for every thing that is installed via pip
15:58:41 mriedem b/c we sure as hell don't test swap volume with multipath backends
15:58:57 openstackgerrit Gábor Antal proposed openstack/nova-specs master: Show persisted scheduler hints in server details https://review.openstack.org/440580
15:59:28 kashyap mriedem: Yeah, 'multipath' -- never in my life I could grok that goddamend thing. Whenever I set out to focus on it to learn, I get distracted.
16:01:00 sean-k-mooney efried: mriedem looks like they have already opened the bug https://bugs.launchpad.net/devstack/+bug/1671144
16:01:00 openstack Launchpad bug 1671144 in devstack "test-requirements are wrongly installed system-wide" [Undecided,New]
16:01:21 openstackgerrit John Garbutt proposed openstack/nova master: POC: add context.check_scope https://review.openstack.org/435485
16:03:05 lyarwood mriedem: we don't appear to test this resize path during our swap_volume tempest tests btw, I'll follow up with that.
16:07:19 mriedem kashyap: ah there are 2 A9K volumes
16:07:22 mriedem 2 swap operations
16:07:34 mriedem 10GB A9K -> 10GB 3PAR,
16:07:42 kashyap And...
16:07:48 mriedem then 10GB 3PAR -> 15GB A9K
16:07:52 mriedem so that's the resize
16:08:03 kashyap Ah, okay, then we're talking about the the second one.
16:12:17 openstackgerrit Maciej Szankin proposed openstack/nova master: conf: fix netconf, my_ip and host are unclear https://review.openstack.org/441210
16:15:04 cburgess I know at least some of you are Lenovo ThinkPad users, what is the goto model these days?
16:16:25 mriedem cburgess: dansmith and melwitt have T460 i think
16:16:31 mriedem cburgess: i'm getting a P50
16:16:39 dansmith mriedem: I have a T460s
16:16:54 dansmith mriedem: but my choice is MBP 13" remember :)
16:16:55 mriedem lyarwood: for https://review.openstack.org/#/c/431540/ is that swap volume from A->B and B->A, or also A->B, B->C?
16:17:06 mriedem dansmith: yeah i know

Earlier   Later