| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-12 | |||
| 14:20:29 | mriedem | yes | |
| 14:20:43 | bauzas | ok | |
| 14:20:43 | mriedem | how do you ensure we test the libvirt driver with the raw image backend? | |
| 14:20:52 | mriedem | since qcow2 is the default | |
| 14:21:07 | bauzas | lemme be political | |
| 14:21:16 | bauzas | that's an excellent question, I'll come to it later | |
| 14:21:18 | mriedem | so the xen image handler defaults to direct_vhd, so that's what was getting tested, but it looks like they have changed the xenserver CI to use vdi_remote_stream https://review.openstack.org/#/c/486446/ | |
| 14:21:38 | mriedem | so now the 3rd party CI is testing the non-default image handler....which seems not great | |
| 14:21:51 | garyk | johnthetubaguy: mriedem: can you please remove the -2 on https://review.openstack.org/#/c/270116/. the spec has been approved | |
| 14:22:02 | garyk | we are still working on some kinks with the CI | |
| 14:22:20 | bauzas | mriedem: I see, thanks for the heads up | |
| 14:23:07 | kashyap | Some of you here might be interested in this 'fun' info: upstream QEMU there is "fierce" and KM-long discussion whether to put some configuration info directly into qcow2 image format (e.g. the min amount of RAM, and importantly the machine type of 'pc' vs. the more advanced 'q35'). | |
| 14:23:24 | openstackgerrit | Merged openstack/nova master: Move db using provider unit tests to functional https://review.openstack.org/572280 | |
| 14:23:24 | kashyap | But no one has seriously committed to it, though. | |
| 14:23:32 | openstackgerrit | Merged openstack/nova master: Adjust db using allocation unit tests https://review.openstack.org/572281 | |
| 14:23:40 | openstackgerrit | Merged openstack/nova master: Remove nova dependencies from test_resource_provider https://review.openstack.org/572282 | |
| 14:23:58 | mriedem | garyk: i can't remove johnthetubaguy's -2 | |
| 14:24:02 | mriedem | i asked him last week to drop it | |
| 14:24:13 | garyk | mriedem: thanks! | |
| 14:24:21 | garyk | johnthetubaguy: ^ | |
| 14:25:16 | kashyap | (It's primarily because, disk images are shipped as qcow2 images. And QEMU folks don't want to break Nova et al, if they switch the default machine type) | |
| 14:25:47 | kashyap | Nova-related discussion: https://bugzilla.redhat.com/show_bug.cgi?id=1581414 | |
| 14:25:48 | openstack | bugzilla.redhat.com bug 1581414 in openstack-nova "OpenStack shouldn't break if the default machine-type in QEMU is "q35"" [Medium,New] - Assigned to kchamart | |
| 14:26:17 | mriedem | simondodsley_: i'm not sure why the cinder documentation for the new qos specs needs to wait for the nova change to merge, | |
| 14:26:29 | mriedem | simondodsley_: as i said in the nova patch, i won't merge the nova change until i at least see the cinder docs change for those new qos specs | |
| 14:26:38 | mriedem | otherwise you have no incentive to actually write the docs | |
| 14:27:42 | mriedem | dansmith: is there anything logged from oslo.messaging when it does a heartbeat while we're running pre_live_migration with the long timeout? | |
| 14:27:54 | mriedem | i was looking for something in the n-cpu logs from this https://review.openstack.org/#/c/574482/ | |
| 14:28:07 | dansmith | mriedem: at debug, I said on your patch | |
| 14:28:23 | dansmith | but that causes a bunch of noise if enabled | |
| 14:28:39 | simondodsley_ | mreidem: I'm actually writing the docs now. If I say I will do something then I will do it... As there is no documentation for these external disk QoS parameters including the originals, that would mean adding a page without the new parameters, and then updating it once the Nova patch merged. | |
| 14:29:04 | mriedem | on *your* patch https://review.openstack.org/#/c/566696/9/nova/compute/rpcapi.py@699 | |
| 14:29:18 | mriedem | but ok, i can make that happen too | |
| 14:31:06 | dansmith | mriedem: oh right I thought it was on one of yours above, but whatever :) | |
| 14:33:20 | mriedem | simondodsley_: replies inline https://review.openstack.org/#/c/558530/3 | |
| 14:34:10 | mriedem | bauzas: yeah, once they remove that stuff it becomes an error - i don't think anyone should probably do that, though, until we have the compute capabilities traits filtering during scheduling | |
| 14:34:25 | mriedem | so at least scheduling of new instances is smarter wrt where trusted certs instances can be built | |
| 14:42:20 | jmccarthy | Anyone have a min to look at this request req-2d031a52-360a-4425-bdf9-1d7630833b43 in nova-api.log https://paste.fedoraproject.org/paste/mUjA9N7upfW0gqmeXjgRMw ? It's a deadlock timeout that seems to be happening fairly often (kolla queens, mysqlcluster database) | |
| 14:49:33 | openstackgerrit | Merged openstack/nova master: Transform instance.exists notification https://review.openstack.org/403660 | |
| 14:54:35 | openstackgerrit | Merged openstack/nova master: Do not use nova.test in placement.test_util https://review.openstack.org/574403 | |
| 14:55:18 | jmccarthy | Seems possibly similar to this bug, https://bugs.launchpad.net/nova/+bug/1642537 although mine is Error: 1205 SQLSTATE: HY000 (ER_LOCK_WAIT_TIMEOUT) not Error: 1213 SQLSTATE: 40001 (ER_LOCK_DEADLOCK) | |
| 14:55:19 | openstack | Launchpad bug 1642537 in OpenStack Compute (nova) "finish_resize fails with DBDeadlock on migrations table" [Undecided,Confirmed] | |
| 14:55:21 | openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: test heartbeat code in pre_live_migration https://review.openstack.org/574482 | |
| 14:57:31 | mriedem | bpoulos: thanks for replying to the ML thread, | |
| 14:57:59 | mriedem | bpoulos: i think my only remaining question on this change then is https://review.openstack.org/#/c/479949/76/nova/image/glance.py@385 | |
| 14:58:22 | mriedem | if trusted_certs is None and enable_certificate_validation is True, why don't we use default_trusted_certificate_ids? | |
| 14:58:43 | mriedem | in download() i mean | |
| 14:59:41 | bpoulos | mriedem: I think the default_trusted_certificate_ids are pulled if both config options are set to True (which would lead trusted_certs to be not None) | |
| 14:59:46 | bpoulos | but I will confirm | |
| 14:59:56 | mriedem | in the api yes i'm aware of that | |
| 15:00:16 | mriedem | i'm just thinking, if i upgrade to rocky and upgrade my computes before the api, | |
| 15:00:20 | mriedem | we could have a gap here | |
| 15:00:54 | mriedem | maybe all that's needed is a release note saying don't set enable_certificate_validation=True in your computes until the API is upgraded to rocky | |
| 15:02:10 | bpoulos | yeah, I can add that release note. It also wouldn't hurt to check for the default ids at that point, since there wouldn't be a risk of overriding the user-provided certs | |
| 15:02:59 | mriedem | we don't need a reno if the download() code condition where trusted_certs is None and enable_certificate_validation=True uses the default certs | |
| 15:03:39 | bpoulos | ok, I'll just make the change then (along with addressing the other comments) | |
| 15:03:55 | mriedem | ok thanks | |
| 15:03:59 | bpoulos | thank you for your feedback! I appreciate you taking a detailed look at the implementation | |
| 15:04:31 | mriedem | np, i want to go through this series in detail again today, and i'll probably address any minor issues myself if i come across them to keep things going, just fyi | |
| 15:05:12 | bpoulos | ok, great, thanks for the heads up | |
| 15:07:33 | edleafe | Question for all you boot-from-volume experts. Can a running VM that is booted from a volume resize that volume? Or must it be taken offline first? | |
| 15:09:41 | openstackgerrit | Merged openstack/nova master: Do not use nova.test in placement.handlers.test_aggregate https://review.openstack.org/574404 | |
| 15:09:47 | openstackgerrit | Merged openstack/nova master: Do not use nova.test in placement.test_requestlog https://review.openstack.org/574405 | |
| 15:09:54 | openstackgerrit | Merged openstack/nova master: Do not use nova.test in placement.test_fault_wrap https://review.openstack.org/574406 | |
| 15:10:01 | openstackgerrit | Merged openstack/nova master: Do not use nova.test in placement.test_handler https://review.openstack.org/574407 | |
| 15:11:19 | lyarwood | edleafe: that depends on the volume and virt drivers being used | |
| 15:11:21 | lyarwood | edleafe: https://developer.openstack.org/api-ref/block-storage/v3/index.html#extend-a-volume-size | |
| 15:12:06 | jmccarthy | Anyone have a min to look at this request req-2d031a52-360a-4425-bdf9-1d7630833b43 in nova-api.log | |
| 15:12:07 | jmccarthy | https://paste.fedoraproject.org/paste/mUjA9N7upfW0gqmeXjgRMw ? | |
| 15:12:07 | jmccarthy | It's a deadlock timeout that seems to be happening fairly often (kolla queens, mysqlcluster database) | |
| 15:12:57 | jmccarthy | Sorry for copy and paste of same question - or if folks have any general suggestions if deadlocks are occuring with nova calls | |
| 15:13:29 | openstackgerrit | Alexandre arents proposed openstack/nova master: Preserve images_type of instance during live migration https://review.openstack.org/570528 | |
| 15:13:33 | jmccarthy | The deletion of instances seems like an area where it happens more often | |
| 15:13:36 | edleafe | lyarwood: thanks. Does it matter if the volume is running as the boot volume? That's the part I couldn't find. | |
| 15:15:55 | cfriesen | is anyone aware of an issue where an RBD-backed swap disk is not restored to the initial size after a resize/revert? We're hitting this on Pike. | |
| 15:16:46 | cfriesen | also, we're seeing a second issue where the guest has to manually re-run "mkswap" after a resize/confirm otherwise it still keeps using the old size of swap | |
| 15:17:57 | lyarwood | edleafe: good question, https://review.openstack.org/#/c/454322/ is the Nova change and AFAICT it does support extending the boot volume | |
| 15:22:12 | edleafe | lyarwood: yeah, looks like we'll need to test this first | |
| 15:23:48 | mriedem | edleafe: https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/nova-support-attached-volume-extend.html#other-end-user-impact | |
| 15:24:01 | mriedem | as far as i know, there are no restrictions in openstack on the root volume | |
| 15:24:09 | mriedem | but either way, the guest has to resize | |
| 15:24:46 | mriedem | garyk: done | |
| 15:24:57 | dansmith | mriedem: you mean the guest has to resize the filesystem | |
| 15:25:05 | mriedem | garyk: but to be clear, i won't +2 that change until the vmware ci is passing with a live migration run | |
| 15:25:10 | mriedem | dansmith: yes | |
| 15:25:16 | mriedem | "The end user will have to perform a partition and/or filesystem resize to fully benefit from the new volume size." | |
| 15:25:52 | dansmith | yup, okay just making sure, because edleafe's original question almost sounded like the guest was triggering the resize of the volume, which isn't a thing | |
| 15:26:06 | garyk | mriedem: thanks! | |
| 15:26:08 | dansmith | "Can a VM...resize that volume" | |
| 15:27:40 | mriedem | efried: +2 on the vdi_remote_stream xenapi change now https://review.openstack.org/#/c/486475/ | |
| 15:27:48 | efried | mriedem: ack | |
| 15:27:48 | mriedem | linked to CI results that use that image handler | |
| 15:28:47 | efried | mriedem: +A | |
| 15:28:52 | mriedem | sweet | |
| 15:29:14 | mriedem | melwitt: we can drop the xenapi image handler bp from the runways slot, the changes are all approved https://review.openstack.org/#/q/topic:bp/xenapi-image-handler-option-improvement+(status:open+OR+status:merged) | |
| 15:29:38 | edleafe | dansmith: yeah, bad wording | |
| 15:30:04 | edleafe | dansmith: the resize would be from an admin API call. | |
| 15:30:49 | dansmith | edleafe: cool, just nitting on the words :) | |
| 15:31:38 | mriedem | efried: so it looks like the 'report cpu features as traits' change finally got life again https://review.openstack.org/#/c/560317/ | |
| 15:31:42 | mriedem | just in time for a runway slot | |