| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-08 | |||
| 15:06:34 | tojuvone | gibi, Craton has also switches, so there is relation. | |
| 15:06:45 | lbragstad | mriedem if you do between now and 10 - just add them here https://etherpad.openstack.org/p/keystone-policy-meeting | |
| 15:07:18 | lbragstad | otherwise - i'll update that topic with a summary if what patrole is after the meeting (because i'm pretty sure that's the big question every has about it) | |
| 15:07:52 | gibi | tojuvone: would Craton push back the TOR switch status to nova to force a change on the status of the server using that TOR? | |
| 15:09:59 | tojuvone | gibi, So I say that you would have the relation known which hosts should go to maintenance if switch goes | |
| 15:10:37 | tojuvone | gibi, so you might put those into maintenance in Craton | |
| 15:10:56 | mriedem | lyarwood: kashyap: remember that block_resize failure thing I was talking about a week ago? got the libvirtd logs and they are super helpful (not really) :) http://paste.openstack.org/show/601952/ | |
| 15:10:56 | tojuvone | gibi, planned maintenance | |
| 15:11:06 | mriedem | "error": {"class": "GenericError", "desc": "Could not resize: Invalid argument"}} len=105 | |
| 15:11:23 | kashyap | mriedem: Hey, BTW, was reading scrollback this morning, heh, it's not Bulgaria | |
| 15:11:30 | kashyap | Ghent, Belgium :-) | |
| 15:12:01 | mriedem | ha | |
| 15:12:02 | dansmith | kashyap: damn | |
| 15:12:09 | mriedem | dansmith was infinitely closer | |
| 15:12:18 | mriedem | i got the B right | |
| 15:12:23 | dansmith | yeah, I win by the horseshoes and hand grenades rule | |
| 15:12:32 | tojuvone | gibi, And one should see that also trough Nova servers API | |
| 15:12:36 | kashyap | Yep; maybe he confused it because I was learning Dutch (and last night was my class, so left early) | |
| 15:12:58 | kashyap | mriedem: Okay, we've got at least the complete command libvirt is sending QEMU | |
| 15:13:07 | kashyap | From line-3, qemuMonitorJSONCommandWithFd:296 | |
| 15:13:10 | tojuvone | gibi, Surely also working the get alarm to user trough Vitrage | |
| 15:13:17 | kashyap | Now to decipher this error message, let me look a bit more | |
| 15:14:11 | gibi | tojuvone: I don't know. Today the I don't get network status or block device backend status through the GET /servers API but I can accept that planned maintenance is different and that might have place in the nova API | |
| 15:14:15 | tojuvone | gibi, as it could receive notification | |
| 15:14:26 | sfinucan | mriedem, johnthetubaguy: Could I ask you to take a look at the PCI weigher spec again? Think I've resolved all your comments https://review.openstack.org/#/c/364468/ | |
| 15:15:24 | gibi | tojuvone: I agree that we need to notify the end user I just wondering if this notification should come from nova our from carlton | |
| 15:15:44 | kashyap | mriedem: Yes, I remember the failure, let me check with QEMU folks on OFTC | |
| 15:16:38 | openstack | bugzilla.redhat.com bug 1070531 in qemu-kvm "qmp can not give a reasonable hint when block_resize a scsi_debug disk with 10G" [Low,Assigned] - Assigned to armbru | |
| 15:16:38 | mriedem | kashyap: i found https://bugzilla.redhat.com/show_bug.cgi?id=1070531 | |
| 15:16:45 | mriedem | i wonder if the size arg is too big? | |
| 15:16:55 | tojuvone | gibi, Craton would be building notifications soon. That is also something if not use Craton, then one might like to have the Nova notif | |
| 15:17:26 | kashyap | mriedem: Potentially; don't want to say something meaningless without digging a bit more :-) | |
| 15:17:54 | mriedem | if i'm looking at this correctly, it's a 15GB block resize | |
| 15:18:45 | gibi | tojuvone: does it mean there will be two alternative implementation of the same feature one in Nova and one in Craton? | |
| 15:18:49 | mriedem | umm, 15360GB? | |
| 15:19:00 | mriedem | self._guest._domain.blockResize(self._disk, size_kb) | |
| 15:19:14 | mriedem | "size":16106127360 | |
| 15:20:19 | kashyap | Oh dear | |
| 15:21:45 | tojuvone | gibi, For the notification? If you would use Craton, yes there would be 2 different then. | |
| 15:22:52 | openstackgerrit | Dan Smith proposed openstack/nova master: Add functools.wraps to rapper https://review.openstack.org/443175 | |
| 15:22:52 | openstackgerrit | Dan Smith proposed openstack/nova master: Sort CellMappingList.get_all() for safety https://review.openstack.org/443174 | |
| 15:23:03 | kashyap | mriedem: Isn't it 16GB? | |
| 15:23:20 | kashyap | It's the size in bytes | |
| 15:23:38 | kashyap | It is indeed 16GB. | |
| 15:24:06 | mriedem | the nova code says kb | |
| 15:24:07 | mriedem | dev.resize(resize_to * units.Gi / units.Ki) | |
| 15:24:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Fix formatting of network options https://review.openstack.org/443086 | |
| 15:24:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Final cleanups in conf/network https://review.openstack.org/431023 | |
| 15:25:24 | kashyap | mriedem: I just checked the QEMU QAPI schema docs; the size there definitely means bytes. | |
| 15:25:49 | kashyap | Note, the 'size' you see there is the QEMU argument to the QMP command 'block_resize' | |
| 15:25:59 | kashyap | mriedem: What type of storage is this, BTW? | |
| 15:26:04 | kashyap | QCOW2? | |
| 15:27:39 | lyarwood | kashyap: are you sure that resize only takes Bytes? | |
| 15:27:54 | mriedem | kashyap: i'm not sure what the volume is that it's resizing to, but this is what it's resizing from (i think) | |
| 15:28:07 | mriedem | http://paste.openstack.org/show/601953/ | |
| 15:28:24 | kashyap | lyarwood: If you see the pastebin from mriedem, on line-4, the quoted size is the argument to the QMP command 'block_resize' | |
| 15:28:40 | kashyap | mriedem: The QEMU folks are asking about the details of the kind of storage | |
| 15:29:18 | mriedem | this is where the size is calculated https://review.openstack.org/#/c/433793/3/nova/virt/libvirt/driver.py@1396 | |
| 15:29:21 | gibi | tojuvone: thanks for the info. I still feel it would be nice to have a non nova impacting alternative mentioned in the spec. but I have no hard feelings is we go forward with the nova based solution | |
| 15:29:41 | kashyap | lyarwood: See line 996 there -- http://git.qemu-project.org/?p=qemu.git;a=blob;f=qapi/block-core.json;h=9bb7f4a17ba9765245616212542725c60ec361a1;hb=HEAD#l996 | |
| 15:29:54 | mriedem | ibm a9000 is what they are resizing to | |
| 15:30:07 | kashyap | And, from what? | |
| 15:30:58 | mriedem | HP 3PAR | |
| 15:31:02 | lyarwood | kashyap: thanks | |
| 15:31:51 | tojuvone | gibi, Thanks, I will work on that | |
| 15:32:23 | kashyap | mriedem: I've conveyed this detail to the QEMU folks on IRC, let you know once one of the block layer guys in EST wakes up | |
| 15:34:40 | mriedem | kashyap: thanks | |
| 15:34:55 | kashyap | mriedem: One more | |
| 15:35:08 | openstack | bugzilla.redhat.com bug 1070531 in qemu-kvm "qmp can not give a reasonable hint when block_resize a scsi_debug disk with 10G" [Low,Assigned] - Assigned to armbru | |
| 15:35:08 | mriedem | even if it's 16GB, https://bugzilla.redhat.com/show_bug.cgi?id=1070531 seems to indicate it's still too large | |
| 15:35:39 | kashyap | mriedem: What is the storage backend - is it a plain raw file, a qcow2 file, a SCSI block device, or something else? | |
| 15:36:29 | openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: fix default values reporting infra worker https://review.openstack.org/441227 | |
| 15:37:00 | mriedem | kashyap: i'll have to dig for that | |
| 15:37:15 | mriedem | kashyap: fo the thing being swapped to or from? | |
| 15:37:25 | dansmith | jaypipes: did you see my comments here? https://review.openstack.org/#/c/441544/4 | |
| 15:37:35 | dansmith | jaypipes: I can fix them up if you don't have time | |
| 15:37:36 | kashyap | mriedem: The context for that question is, from danpb: | |
| 15:37:39 | kashyap | mriedem: To figure out where EINVAL comes from we need to understand what qemu block driver is getting used | |
| 15:38:03 | dansmith | jaypipes: specifically this: https://review.openstack.org/#/c/441544/4/nova/virt/ironic/driver.py | |
| 15:38:42 | kashyap | mriedem: Both. | |
| 15:40:51 | kashyap | mriedem: One more (credit for the guess: Eric Blake), about potentially what's going on: | |
| 15:40:52 | mriedem | maybe i can find some details from the compute logs | |
| 15:41:23 | jaypipes | dansmith: yeah, will get to that after meetings. | |
| 15:41:40 | kashyap | 16:39 < kashyap> eblake: Could you see that from the pastebin, or are you guessing that there's a shrink going on? | |
| 15:41:40 | kashyap | 16:39 < eblake> could it be that the resize failed because of that? | |
| 15:41:40 | kashyap | 16:39 < eblake> I know qcow2 prevents resizes that shrink the volume | |
| 15:41:40 | kashyap | mriedem: | |
| 15:41:43 | kashyap | 16:40 < eblake> I'm just guessing, as that would be one reason for calling the resize requested size invalid | |
| 15:41:49 | dansmith | jaypipes: okay, and then kinda next for this effort is the flavor override patch and the allocation side of this right? | |
| 15:42:00 | jaypipes | dansmith: yup. | |
| 15:42:17 | dansmith | jaypipes: okay cool | |
| 15:43:04 | mriedem | kashyap: we only resize if the new volume that we're resizing to is bigger https://review.openstack.org/#/c/433793/3/nova/compute/manager.py@5006 | |
| 15:43:29 | mriedem | looking at the logs, the name on the A9000 volume implies it's an iscsi block device | |
| 15:43:39 | mriedem | however, it also says the size is 10GB, | |
| 15:43:49 | mriedem | but that's not what i'm seeing when we actually run the block_rebase command | |
| 15:43:54 | kashyap | mriedem: On the 16GB size comment, I'm being told it is "any size is valid as long as its below the max possible size; and 16G is trivially in range" | |
| 15:43:59 | mriedem | *block_resize | |
| 15:44:26 | kashyap | And the max possible size is at least a a TB of even EB, apparently | |
| 15:44:56 | openstackgerrit | Dan Smith proposed openstack/nova master: Add functools.wraps to rapper https://review.openstack.org/443175 | |
| 15:44:56 | openstackgerrit | Dan Smith proposed openstack/nova master: Sort CellMappingList.get_all() for safety https://review.openstack.org/443174 | |