| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-08 | |||
| 19:17:23 | dansmith | as opposed to front butt? | |
| 19:18:21 | mriedem | gimme a minute | |
| 19:18:44 | artom | For what it's worth, revert https://review.openstack.org/#/c/595069/ + wait in compute manager is fine | |
| 19:18:54 | artom | Err, *I think that revert blah blah | |
| 19:19:21 | mriedem | https://hangouts.google.com/call/4MUfp6uNnY6ddII9gtXGAEEE | |
| 19:25:59 | mriedem | artom: see _get_neutron_events_for_live_migration for how live migration uses it | |
| 19:26:37 | artom | mriedem, ack | |
| 19:26:54 | mriedem | so i think we just wrap that around the migrate_instance_finish in finish_revert_resize | |
| 19:27:04 | mriedem | i'd probably split that code out to a new helper method for sanity.. | |
| 19:37:42 | artom | Right, I gotta run again, I'll try to have those 2 patches up before the weekend | |
| 20:00:31 | cfriesen | so in devstack I'm running "nova migrate <uuid>" and I'm getting an error saying that it can't migrate to current host. Shouldn't it be disregarding the current host for migration? | |
| 20:03:45 | dansmith | cfriesen: do you only have one host? | |
| 20:03:52 | cfriesen | dansmith: nope, two | |
| 20:04:04 | dansmith | but I think you'll get that if the scheduler doesn't pick the other one as a candidate | |
| 20:04:08 | cfriesen | shed logs see it considering both right up till the weigher | |
| 20:04:23 | cfriesen | but for "migrate" we'd never want the current host | |
| 20:04:27 | cfriesen | only for resize | |
| 20:04:59 | dansmith | they're the same operation, so if allow-to-same is enabled, I think it'll allow it | |
| 20:06:10 | openstackgerrit | Michael Still proposed openstack/nova master: Make privsep.chown mocking for libvirt snapshot tests less magic. https://review.openstack.org/642134 | |
| 20:06:12 | cfriesen | okay, but that makes no sense logically to the user. a "migrate" operation should mean moving it elsewhere. | |
| 20:06:40 | dansmith | cfriesen: yeah, I'm not arguing that it makes sense, I'm just stating what I think is the case | |
| 20:06:57 | dansmith | also, migrate is admin-only by default I think, so it might not get much visibility because of that | |
| 20:07:10 | cfriesen | I suspect you're right. but it means that with "resize on same host" enabled you currently have to explicitly specify a destination. | |
| 20:07:29 | dansmith | no, | |
| 20:07:37 | cfriesen | I'm pretty sure this is a regression, just not sure from when | |
| 20:07:41 | dansmith | it only means that if your current host is the most obvious target right? | |
| 20:08:03 | cfriesen | dansmith: okay, fair enough. it means you can't assume it won't land on the current host | |
| 20:08:30 | dansmith | I dunno how common it is to have allow-to-current enabled in production either | |
| 20:08:41 | dansmith | it was supposed to only be for gate testing | |
| 20:09:00 | cfriesen | It's useful in small clouds. | |
| 20:09:21 | dansmith | again, stating the original intent | |
| 20:09:24 | cfriesen | what was the rationale for defaulting it to "off"? | |
| 20:09:40 | dansmith | because it was only for testing | |
| 20:10:48 | cdent | tautology! | |
| 20:10:49 | cfriesen | I get that, but why was it only for testing? You're trying to resize, why would you force a migration too? | |
| 20:12:11 | dansmith | cfriesen: there were some reasons discussed originally when it was added, but it's been a long time.. probably mostly around how the major users of this stuff ran their clouds at the time | |
| 20:12:27 | dansmith | i.e. packing like flavors together | |
| 20:13:04 | dansmith | again, not saying it makes sense today, just saying why it was done that way | |
| 20:13:23 | cfriesen | currently I'd expect the scheduler to trigger it to move in that case | |
| 20:16:13 | openstack | Launchpad bug 1819216 in OpenStack Compute (nova) "in devstack, "nova migrate <uuid>" will try to migrate to the same host (and then fail)" [Undecided,New] | |
| 20:16:13 | cfriesen | anyways, reported it as a bug: https://bugs.launchpad.net/nova/+bug/1819216 | |
| 20:19:16 | mriedem | did someone say resize to same host problems?! https://review.openstack.org/#/c/638791/ | |
| 20:19:31 | mriedem | if you have servers in an affinity group, you can *only* resize to the same ohst | |
| 20:19:32 | mriedem | *host | |
| 20:19:58 | mriedem | cfriesen: there is also a bug for the thing you hit, semi-related anyway | |
| 20:20:38 | cfriesen | mriedem: right, I knew about the affinity group | |
| 20:20:51 | mriedem | the api says resize to same host, sure, but the compute says 'no way' | |
| 20:22:04 | mriedem | https://bugs.launchpad.net/nova/+bug/1811235 | |
| 20:22:05 | openstack | Launchpad bug 1811235 in OpenStack Compute (nova) "instance's vm-state becomes error when cold-migrate instance to same host failed" [Medium,In progress] - Assigned to Matt Riedemann (mriedem) | |
| 20:23:07 | mriedem | the api check is here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L3626 | |
| 20:23:43 | mriedem | the compute check is here https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L4287 | |
| 20:24:04 | mriedem | which fails for everything except vmware | |
| 20:24:23 | mriedem | since vmware is a cluster and you can migrate within the same "host" | |
| 20:25:11 | mriedem | so allow_resize_to_same_host is true if you're doing a resize, but not a cold migration *unless* you've got vcenter | |
| 20:26:10 | cfriesen | arguably the scheduler should only allow cold migration to same host for vcenter | |
| 20:26:22 | mriedem | the scheduler doesn't know if it's vcenter | |
| 20:26:23 | cfriesen | I guess it's actually the conductor | |
| 20:26:27 | mriedem | or conductor | |
| 20:26:39 | mriedem | the control plane services are not supposed to know anything about the underlying compute driver | |
| 20:26:43 | cfriesen | can't we look at host capabilities or hypervisor type or something? | |
| 20:27:02 | dansmith | cfriesen: and make assumptions? what about vmware 2 which *can* do that thing.. it's a slippery slope | |
| 20:27:07 | dansmith | separation of responsibilities | |
| 20:27:11 | cdent | vmware even has a downstream patch which adds config for "always resize to same "host" " | |
| 20:28:19 | mriedem | cdent: that's kind of where https://review.openstack.org/#/c/638791/ is going | |
| 20:28:28 | cfriesen | how do we handle it for live migration? | |
| 20:28:50 | mriedem | we always ignore the source host | |
| 20:29:01 | mriedem | same for evac | |
| 20:29:18 | cfriesen | seems weird to make live migration and cold migration behave differently | |
| 20:31:47 | mriedem | so, | |
| 20:31:53 | mriedem | now that we report compute driver capabilities as traits, | |
| 20:31:56 | mriedem | and supports_migrate_to_same_host is a driver capability, | |
| 20:32:20 | mriedem | we could technically check the compute node provider traits for the source compute node from the api and determind if it supports cold migrate to the same host and filter the request spec appropriately | |
| 20:32:46 | dansmith | don't you need the opposite of that? | |
| 20:33:10 | dansmith | you need to know if it *needs* the same host for the vmware case, not whether not it *can* yeah? | |
| 20:33:57 | mriedem | yeah maybe, you can't cold migrate across clusters | |
| 20:33:59 | mriedem | right? | |
| 20:34:08 | dansmith | yeah | |
| 20:34:26 | dansmith | so you could exclude the source host unless can-haz-must-be-same-host | |
| 20:35:34 | dansmith | that means looking up traits for the current host and then deciding what to ask the scheduler to do | |
| 20:35:48 | mriedem | yeah that's what i was trying to get to | |
| 20:36:09 | mriedem | from the api, if supports_migrate_to_same_host is false, exclude the source host before calling the scheduler | |
| 20:36:16 | mriedem | if supports_migrate_to_same_host is true, then don't exclude the source host | |
| 20:36:17 | mriedem | right? | |
| 20:36:38 | dansmith | you want libvirt to say it *can't* ? | |
| 20:36:43 | mriedem | it already does | |
| 20:36:50 | mriedem | vmware is the only driver that says supports_migrate_to_same_host=true | |
| 20:37:09 | dansmith | oh okay I'm conflating the migrate and resize to same host cases I guess | |
| 20:37:21 | dansmith | in that case, | |
| 20:37:25 | dansmith | the flag is really backwards, | |
| 20:37:26 | mriedem | yeah we don't have a driver capability for resize to same host | |
| 20:37:39 | dansmith | because you don't want to allow the source if it can, you want to limit to the source if it can, else exclude | |
| 20:37:42 | mriedem | you mean supports_migrate_to_same_host should be "must_migrate_to_same_host"? | |
| 20:37:49 | mriedem | oh | |
| 20:37:50 | dansmith | mriedem: right we have the config flag tho | |
| 20:38:24 | mriedem | yeah again that's kind of where i was going with https://review.openstack.org/#/c/638791/1/nova/conf/workarounds.py | |
| 20:38:29 | mriedem | but it's fugly | |
| 20:39:40 | mriedem | i need to work on an alternative to that which doesn't bypass the scheduler filters | |
| 20:39:47 | mriedem | because that patch is really wild west at the moment | |
| 20:39:55 | mriedem | it's more like --force to the source host | |
| 20:40:02 | mriedem | which is like, -4.5 | |