| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-11-16 | |||
| 16:37:09 | kashyap | Ah, okay. | |
| 16:37:13 | cfriesen | kashyap: "unsafe" says that it's possible to keep it in memory and not force it to be written out. I suspect the OS will eventually write it out anyways. | |
| 16:37:37 | openstackgerrit | Merged openstack/python-novaclient master: Fix missing metavar in aggregate-update https://review.openstack.org/512485 | |
| 16:37:43 | kashyap | cfriesen: Yeah, the guest has its own cache indeed. | |
| 16:37:44 | cfriesen | kashyap: but if you hit the system-wide dirty-page limit, it's going to affect writes for all devices on the system | |
| 16:38:27 | cfriesen | kashyap: remember we're talking about "qemu-img convert" here, there's no guest involved | |
| 16:39:11 | kashyap | cfriesen: Yes, sorry for the mixup. I was talking about the other scenario where why people tend to not use caching for guests. | |
| 16:39:30 | openstackgerrit | Merged openstack/nova master: Updated from global requirements https://review.openstack.org/520461 | |
| 16:39:47 | kashyap | But indeed, we need to distinguish the two cases: (1) The cache used for the guest itself; (2) The cache property during 'qemu-img convert' for src and destination images. | |
| 16:42:33 | kashyap | cfriesen: So in your, case suckage is caused by saturating 'sdb' and hitting the 'dirty_bytes' limit | |
| 16:44:06 | cfriesen | kashyap: yep. | |
| 16:45:15 | kashyap | cfriesen: Might want to write a blog post about it, if you can. Useful for OpenStack users to be mindful of this | |
| 16:45:40 | mriedem | andreykurilin: any idea why our is_latest check here is not just checking for the "latest" word? https://github.com/openstack/python-novaclient/blob/master/novaclient/api_versions.py#L96 | |
| 16:45:50 | mriedem | oh nvm https://github.com/openstack/python-novaclient/blob/master/novaclient/api_versions.py#L70 | |
| 16:46:03 | cfriesen | kashyap: not a bad idea....things to be aware of when tuning compute nodes. :) | |
| 16:46:21 | kashyap | Yeah | |
| 16:46:25 | andreykurilin | mriedem: yup, inf simplifies calculation | |
| 16:46:30 | andreykurilin | and comparison | |
| 16:46:52 | cfriesen | kashyap: anyway, thanks for thinking about it even if it doesn't make sense for general use. | |
| 16:46:53 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: adds REST API for nested providers https://review.openstack.org/384807 | |
| 16:46:54 | openstackgerrit | Eric Fried proposed openstack/nova master: Iterator for a provider tree https://review.openstack.org/520243 | |
| 16:46:54 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: update client to set parent provider https://review.openstack.org/385693 | |
| 16:46:55 | openstackgerrit | Eric Fried proposed openstack/nova master: Scheduler set_inventory_for_provider does nested https://review.openstack.org/520643 | |
| 16:47:20 | mriedem | andreykurilin: ok, so it looks like the SecretsHelper code was essentially made useless by https://review.openstack.org/#/c/184281/ | |
| 16:47:21 | kashyap | cfriesen: Np; thanks for opening the can^H^H^H^H bringing up the topic :P | |
| 16:47:28 | cfriesen | :) | |
| 16:47:48 | efried | jaypipes ^ Fixed Tetsuro's comments on that first one; and the next two are the start of ComputeDriver.update_inventory framework. | |
| 16:47:48 | mriedem | andreykurilin: because if we don't have a password or token set, we try to discover the api version here https://github.com/openstack/python-novaclient/blob/master/novaclient/shell.py#L796 | |
| 16:47:51 | mriedem | which eventually requires auth | |
| 16:48:04 | mriedem | and the secrets helper stuff kicks in here https://github.com/openstack/python-novaclient/blob/master/novaclient/shell.py#L852 | |
| 16:48:06 | jaypipes | efried: ack, nice. | |
| 16:48:10 | efried | jaypipes Sorry, the last two. The middle one is just a rebase. | |
| 16:48:16 | jaypipes | ya | |
| 16:48:28 | andreykurilin | mriedem: you are right | |
| 16:48:42 | mriedem | let me see if i can hack that in my devstack to make it prompt for a password | |
| 16:57:58 | openstackgerrit | Ed Leafe proposed openstack/nova master: Return Selection objects from the scheduler driver https://review.openstack.org/495854 | |
| 16:57:58 | openstackgerrit | Ed Leafe proposed openstack/nova master: Refactor the code to check for sufficient hosts https://review.openstack.org/520242 | |
| 16:57:58 | openstackgerrit | Ed Leafe proposed openstack/nova master: Add Selection objects https://review.openstack.org/499239 | |
| 16:57:59 | openstackgerrit | Ed Leafe proposed openstack/nova master: Modify select_destinations() to return objects and alts https://review.openstack.org/510159 | |
| 16:58:00 | openstackgerrit | Ed Leafe proposed openstack/nova master: Make conductor pass and use host_lists https://review.openstack.org/511358 | |
| 16:58:00 | openstackgerrit | Ed Leafe proposed openstack/nova master: Move the claim_resources method to scheduler utils https://review.openstack.org/511357 | |
| 16:58:00 | openstackgerrit | Ed Leafe proposed openstack/nova master: Change RPC for select_destinations() https://review.openstack.org/516707 | |
| 17:02:02 | mriedem | andreykurilin: fixed! | |
| 17:02:50 | mriedem | well, i'm prompted for a password now, but apparently it's not used | |
| 17:06:16 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: instance_mapping/request_spec records exist even after the instances are archived https://review.openstack.org/515034 | |
| 17:06:31 | mriedem | andreykurilin: so given this has been broken since liberty and no one noticed or cared, i'm thinking we could just remove it | |
| 17:06:37 | mriedem | there is no point in even deprecating it | |
| 17:07:50 | openstack | Launchpad bug 1482040 in OpenStack Compute (nova) "rebuild volume-backed instance with image does not replace image in root disk" [Undecided,Confirmed] | |
| 17:07:50 | cfriesen | mriedem: for https://bugs.launchpad.net/nova/+bug/1482040 would it maybe make sense to disallow rebuild of boot-from-volume instance to a different image (but allow it for the same image)? | |
| 17:08:20 | openstack | Launchpad bug 1732744 in python-novaclient "SecretsHelper has not worked since 2015" [Undecided,Triaged] - Assigned to Matt Riedemann (mriedem) | |
| 17:08:20 | mriedem | https://bugs.launchpad.net/python-novaclient/+bug/1732744 | |
| 17:08:33 | cfriesen | mriedem: or should we completely disallow it for boot-from-volume since the image isn't replaced (but other stuff is, so it's still somewhat useful) | |
| 17:08:45 | mriedem | cfriesen: i think you ask me this at least once per week | |
| 17:08:54 | cfriesen | mriedem: last time you said to disallow it | |
| 17:09:08 | cfriesen | mriedem: but I'm wondering about the case where the image matches what was on the volume | |
| 17:10:40 | mriedem | sec | |
| 17:20:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Fail fast if changing image on a volume-backed server rebuild https://review.openstack.org/520660 | |
| 17:20:43 | mriedem | cfriesen: there you go ^ needs tests | |
| 17:23:36 | cfriesen | mriedem: so we want to disallow rebuild even to the same image? (I think it can fix some problems, like rebuilding cinder volume connections) | |
| 17:24:20 | mriedem | cfriesen: look at the patch | |
| 17:24:27 | mriedem | instance.image_ref != image_href | |
| 17:24:58 | cfriesen | mriedem: okay, but then we'll update the image ref in the instance to point to the new image (instead of having nothing like it did at boot) | |
| 17:25:19 | mriedem | cfriesen: huh? | |
| 17:25:31 | mriedem | we don't save the instance.image_ref until later, after we've already failed here | |
| 17:26:07 | cfriesen | mriedem: in the case of a rebuild-to-same image, before the rebuild the instance image ref is empty, after the rebuild it points to that image | |
| 17:26:41 | mriedem | why is the instance.image_ref empty? | |
| 17:27:19 | cfriesen | mriedem: dunno, it's always been that way. novaclient keys off an empty image ref to say "Attempt to boot from volume - no image supplied" | |
| 17:28:05 | mriedem | if the instance.image_ref is empty because it's a boot from volume, then "and compute_utils.is_volume_backed_instance" is True and we raise an exception so you can't rebuild, correct? | |
| 17:28:11 | mriedem | which means we don't get to the code that calls instance.save() | |
| 17:29:09 | cfriesen | mriedem: yeah, it actually means your proposed patch will result disallowing all rebuilds of boot-from-volume. | |
| 17:29:23 | cfriesen | mriedem: I think we need to look at the image specified in the volume metadata | |
| 17:29:50 | mriedem | oh i see what you're saying | |
| 17:30:15 | mriedem | cfriesen: ok comment and -1 the patch | |
| 17:30:24 | mriedem | i'm dealing with novaclient atm | |
| 17:30:59 | cfriesen | mriedem: one other related thing is that there's no easy way to tell using the HTTP API if an instance is boot-from-volume other than that image field. | |
| 17:31:20 | cfriesen | mriedem: no worries, will poke at it. we've got a customer issue around this | |
| 17:31:49 | mriedem | you mean there isn't an easy way for the user to discover if they have a boot from volume instance and shouldn't attempt a rebuild with a different image to avoid the 400? | |
| 17:32:02 | openstackgerrit | Merged openstack/nova stable/newton: Validate new image via scheduler during rebuild https://review.openstack.org/519684 | |
| 17:32:36 | cfriesen | mriedem: we have some orchestration code that does stuff differently for boot-from-volume (not just for rebuild, but that's one case) | |
| 17:32:45 | mriedem | we == nova? | |
| 17:32:50 | mriedem | or we == windriver? | |
| 17:32:55 | cfriesen | mriedem: no, windriver. | |
| 17:33:21 | cfriesen | but I suspect others have similar issues. if you don't know how something was booted, it's hard to tell just by looking at instance details if it's boot-from-volume or not | |
| 17:35:49 | mriedem | https://developer.openstack.org/api-ref/compute/#show-server-details | |
| 17:35:54 | mriedem | "The UUID and links for the image for your server instance. The image object might be an empty string when you boot the server from a volume." | |
| 17:36:11 | cfriesen | mriedem: "might" is not definitive | |
| 17:36:18 | mriedem | well, | |
| 17:36:24 | mriedem | if we know that it's always the case, we can change the docs | |
| 17:36:33 | mriedem | but that would require some investigation | |
| 17:36:55 | mriedem | "The image object is totes empty when you BFV bra!" | |
| 17:36:58 | mriedem | let's change to that | |
| 17:37:11 | mriedem | our docs are boring | |
| 17:37:19 | mriedem | are you totes down? | |
| 17:37:21 | mriedem | bra? | |
| 17:37:40 | cfriesen | I think there's a bra around here somewhere... | |
| 17:38:05 | mriedem | bzzt, wrong | |
| 17:38:11 | mriedem | incorrect usage of the term | |
| 17:38:18 | mriedem | you are now in PC jail, welcome | |
| 17:38:31 | openstackgerrit | Eric Fried proposed openstack/nova master: SchedulerReportClient._get_resource_provider_tree https://review.openstack.org/520663 | |
| 17:39:15 | efried | jaypipes ^ -- and the next one will do a public method that uses that result to populate ProviderTree. | |