Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-06-03
08:42:28 gibi OK, I agree this can be a can of worm
08:42:54 gibi suzhengwei: in case of evacuating a suspended VM, is it OK to you to loose the suspended state?
08:44:03 kashyap gibi: I need to think a bit more about it. (libvirt has managedSave() API that does the suspend thingie, which already Nova uses. So we have the primitives...)
08:44:06 suzhengwei If host down, active suspend instance both loose their memory.
08:44:53 kashyap gibi: suzhengwei: What is the main use-case here? The ability to start suspended instances on any compute host from a given pool?
08:44:59 gibi suzhengwei: if you don't want to recover the suspended state that is saved to disk, then I think your proposal is pretty simple and straight forward
08:45:08 suzhengwei Instance Ha, try best to recover the workload as much as possible.
08:45:17 bauzas interestingly, I found some nova admin docs https://docs.openstack.org/nova/latest/admin/node-down.html
08:45:38 gibi kashyap: we looked at it from evacuation perspective. VM is suspended to disk (on shared storage), the host dies, user evacuates VM
08:46:04 kashyap gibi: I see; that makes sense
08:46:51 bauzas gibi: I honestly feel we can just support recreating a new instance
08:47:07 kashyap bauzas: Isn't that what already 'rebuild' is?
08:47:16 kashyap Ah, you said that already above :)
08:47:18 bauzas kashyap: yup, the question was about the memory state
08:47:51 suzhengwei If host down, the suspend instance can be active agian on the origin node. So I think it makes sence to evacuate suspend instances.
08:48:01 suzhengwei can not
08:48:06 bauzas I guess here suzhengwei's concern is that we limitate evacuate on active instances
08:48:14 bauzas right?
08:48:22 bauzas that's the problem we're trying to solve ?
08:48:35 gibi I think so
08:48:39 suzhengwei yes
08:49:09 gibi and I'm totally supportive to extend evac to support paused and suspended instances. It is simple if we allow loosing the running state
08:49:11 bauzas I just remembered we have a --on-shared-storage flag https://docs.openstack.org/nova/latest/admin/evacuate.html#evacuate-a-single-instance
08:49:36 bauzas since evacuate is an admin action, op can use it
08:49:39 bauzas on purpose
08:50:51 bauzas so we already do the check automatically
08:50:52 gibi bauzas: onSharedStorage is deprecated in 2.13
08:51:00 gibi bauzas: today we automatically detect it I guess
08:51:01 bauzas gibi: because we detect this ?
08:51:04 bauzas yeah
08:51:09 gibi "Starting since version 2.14, Nova automatically detects whether the server is on shared storage or not. Therefore this parameter was removed."
08:51:12 gibi yepp
08:51:20 bauzas ok, so I guess we can consider adding suspend
08:51:31 gibi suzhengwei: I suggest to propose a small spec about this. I'm happy to review it
08:51:44 bauzas if the target host is on shared storage, we could just try to boot with the suspended state
08:52:02 bauzas for paused, the implication would be that the evacuated instance would become active
08:52:15 bauzas for suspend, too
08:52:15 gibi bauzas: active, or stopped
08:52:20 gibi bauzas: we can decide
08:52:27 bauzas yup, that's the point
08:52:31 gibi bauzas: but true, it cannot be pasued any more
08:52:36 gibi paused
08:53:05 gibi I don't want to stop the discussion, but we have 8 minutes left. If there any other topic to discuss?
08:53:14 gibi /If/Is/
08:53:24 kashyap bauzas: gibi: One last:
08:53:27 gibi kashyap: go
08:53:30 suzhengwei I think stopped is better. No matter pause or suspend, users can not acess the instance directly.
08:53:43 gibi suzhengwei: I can accept that
08:54:07 kashyap gibi: suzhengwei: On whether it makes sense of moving suspended instances between compute hosts, a thumb-rule can be: "follow the same rules for hardware matching as for a live migration between the hosts"
08:54:19 kashyap (I mean, to uncover any "gotchas")
08:54:47 gibi kashyap: ahh you have a point, this state can be hw dependent
08:54:49 kashyap FWIW, I also just checked the above w/ a QEMU migration developer; and he agrees.
08:54:49 bauzas kashyap: since evacuate is a rebuild, we can't predict this
08:55:25 bauzas gibi: sorry, I wasn't explicit but when I said 'we're gonna try to unsuspend from disk", I was thinking of hardware capabilities
08:55:28 gibi OK, then I propose not to try to recover the suspended state during evac. At list not in the first step
08:55:40 kashyap bauzas: gibi: Hm, so looks like this needs to be fleshed out in a design document
08:55:41 gibi /list/least/
08:56:06 bauzas kashyap: the evacuate workflow is waaaaay different from live-migrate
08:56:16 bauzas you can't just check the source host at first ;)
08:56:27 bauzas and compare both
08:56:45 bauzas the scheduler is just giving you a target and then good luck with it
08:56:57 kashyap bauzas: I see; fair enough
08:57:03 gibi so in summary
08:57:04 bauzas but yeah, we're 4 mins
08:57:07 bauzas left
08:57:07 gibi so in summary
08:57:49 gibi suzhengwei: please propose a spec. I don't see any problem supporting evac for paused and suspended VMs. But they will lose the in memory or suspended state. They will be fresh VMs on the dest host in stopped state
08:58:18 kashyap Yeah; makes sense.
08:58:24 suzhengwei I will.
08:58:26 kashyap (On spec)
08:58:32 gibi suzhengwei: cool, thanks
08:58:48 gibi any last words before we stop the meeting? ;)
08:59:03 suzhengwei nothing from me.
08:59:07 gibi XinxinShen: ?
08:59:37 XinxinShen nothing for me. thanks.
08:59:56 gibi then thanks for joining. please continue discussion if needed
09:00:00 gibi I just stop the meeting log here
09:00:04 gibi #endmeeting
09:00:06 opendevmeet Meeting ended Thu Jun 3 09:00:04 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
09:00:07 opendevmeet Minutes: http://eavesdrop.openstack.org/meetings/nova_extra/2021/nova_extra.2021-06-03-08.00.html
09:00:08 opendevmeet Minutes (text): http://eavesdrop.openstack.org/meetings/nova_extra/2021/nova_extra.2021-06-03-08.00.txt
09:00:09 opendevmeet Log: http://eavesdrop.openstack.org/meetings/nova_extra/2021/nova_extra.2021-06-03-08.00.log.html
09:00:23 bauzas right on time
09:02:59 kashyap gibi: Remind me again, what's the case against trying to "resume" the saved state on the destination, instead of a fresh rebuild (which loses the state)? Because it also involves dealing with the state file transportation?
09:03:31 gibi kashyap: if we need to take care of HW compatibility then we have to blow up the evac process
09:03:46 kashyap gibi: Oh, yikes; yes. How come I forgot so quickly
09:03:59 gibi live migration is special :)
09:04:17 kashyap Tell me all about it.
09:04:23 gibi :D
09:05:04 kashyap gibi: I'm not the person for GIFs, but this captures it: https://kashyapc.fedorapeople.org/Live-Migration.gif
09:05:42 kashyap (Not sure if it's loading correctly for you)
09:06:02 gibi ooh, that is good
09:06:10 gibi I will distribute it :)
09:06:17 kashyap You're welcome ;-)
09:06:28 gibi thanks
09:06:34 kashyap gibi: Does it load smoothly for you? For me not here, strangely
09:06:43 gibi it works for me from firefox
09:06:54 gibi smooth and on auto repeat
09:07:13 kashyap Ah, nod. (It goes in super slow-motion on FF and Chromium.)
09:10:13 bauzas gibi: I think we have a formal contract with evacuate

Earlier   Later