| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-14 | |||
| 14:55:16 | sean-k-mooney | alex_xu: but why do you need to get flaovr info form the xml. for resize we have teh flavor objects and we shoudl have them for cold migrate and live migratie | |
| 14:55:26 | mriedem | if you're doing a resize, you can do it on the same host with libvirt, | |
| 14:55:30 | dansmith | and building an assumption into this that same-host same-flavor migrations can't happen is a bad idea, IMHO | |
| 14:55:32 | mriedem | if you're doing a cold migration, you cannot do it on the same host with libvirt | |
| 14:55:42 | mriedem | and it seems folly to think any other driver is going to implement vpmem support | |
| 14:55:52 | dansmith | sean-k-mooney: they're talking about during host init, with instances in the middle of a migration | |
| 14:55:56 | mriedem | given how complicated this sounds even for libvirt | |
| 14:56:08 | sean-k-mooney | dansmith: ah thanks for the context | |
| 14:56:18 | dansmith | mriedem: we do same-host same-flavor migration on libvirt in the gate with a single worker yeah? | |
| 14:56:33 | mriedem | no | |
| 14:56:33 | sean-k-mooney | dansmith: i dont think so | |
| 14:56:36 | mriedem | resize yes | |
| 14:56:37 | mriedem | not cold migration | |
| 14:56:40 | mriedem | you literally can't | |
| 14:56:45 | dansmith | why not? | |
| 14:56:53 | aspiers | Hrm, sounds like my question might coincidentally be somewhat related to the ongoing conversation, except for image_meta instead of flavors | |
| 14:57:04 | sean-k-mooney | dansmith: the livrt driver does not report supprot for it | |
| 14:57:08 | alex_xu | sean-k-mooney: avoid db query in libvirt init | |
| 14:57:10 | mriedem | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L314 | |
| 14:57:13 | mriedem | because ^ | |
| 14:57:28 | mriedem | remember the other day when we were talking about a workaround option to disallow cold migrating to the same source host as a scheduler option? | |
| 14:57:31 | mriedem | this is why | |
| 14:57:35 | alex_xu | same flavor same host cold migration means....you stop and start the instance again :) | |
| 14:57:36 | sean-k-mooney | alex_xu: why we will be hamming it anyway when the resouce tracker starts up | |
| 14:57:44 | dansmith | mriedem: either way, don't you think that making that assumption in the accounting for these is a bad plan, if that gets enabled in the future for whatever reason? | |
| 14:57:56 | dansmith | alex_xu: no it doesn't | |
| 14:58:14 | dansmith | mriedem: okay that's just because libvirt won't handle the disk copying properly because the names are all the same righht? | |
| 14:58:46 | mriedem | idk what cold migrating on the same host with libvirt would mean | |
| 14:58:58 | alex_xu | yea, same question | |
| 14:59:03 | dansmith | ages ago we had *some* CI system (not vmware) that could do same-host cold migration because it would find bugs in that, but maybe it was xen or smokestack or something | |
| 14:59:31 | dansmith | just seems like a terribly deep assumption to make | |
| 14:59:43 | mriedem | idk, the xenapi driver doesn't support it either https://github.com/openstack/nova/blob/master/nova/virt/xenapi/driver.py#L67 | |
| 14:59:53 | mriedem | only the vmware driver does | |
| 15:00:23 | mriedem | i also don't understand how init_host is connected to a move claim | |
| 15:00:33 | dansmith | alex_xu: explain again why you need to tie flavors to the pmem at all for accounting? | |
| 15:00:51 | dansmith | mriedem: this is for host restart, rebuilding state for what has been given to what | |
| 15:01:09 | dansmith | mriedem: but I don't understand why we need to correlate that with anything other than instance uuid, which is done in the xml, AFAIK | |
| 15:01:30 | mriedem | yeah....i thought the pmem "allocation" type data was stored in instance_extra or something | |
| 15:01:33 | mriedem | keyed by instance uuid | |
| 15:01:42 | alex_xu | for same host resize, we records the claim in memory. but when we confirm and revert the resize, we need unclaim for dst claim or src claim. to distinguish that, only the instance_uuid isn't enough. I use instance_uuid + flavor.id. | |
| 15:01:57 | alex_xu | dansmith: mriedem sean-k-mooney ^ | |
| 15:02:25 | dansmith | alex_xu: isn't the whole point of this pmem stuff that the contents get moved with the instance if it crosses to another host? | |
| 15:02:42 | dansmith | (tbh I don't know what the point of this is, but...) | |
| 15:02:48 | mriedem | if you're adding a new move claim interface to the virt driver, why wouldn't you just pass the information down to the driver that the RT already knows - i.e. the old and new flavor and if it's a same-host resize? | |
| 15:03:40 | mriedem | if we have to piece information together from what the hypervisor says in the domain xml, we screwed up somewhere in the design :) | |
| 15:03:42 | alex_xu | when you confirm resize, you neeed unclaim the assigment for the source host. But for the same host, if only use instance_uuid, when you unclaim by instance_uuid, both the source and dest claim will be unclaim | |
| 15:03:58 | alex_xu | same problem for revert resize | |
| 15:05:01 | dansmith | alex_xu: so change the source allocation to be by migration uuid like we do for allocations? | |
| 15:05:59 | alex_xu | dansmith: the problem is when we revert the migration uuid back to instance uuid. for the resize to different host, those claim on two host's memory, we can't change the migration uuid back to instance uuid when the resize finish | |
| 15:06:20 | dansmith | alex_xu: why not? | |
| 15:06:32 | dansmith | if you can change it from instance to migration, why can't you change it back? | |
| 15:08:10 | alex_xu | dansmith: we claim the vpmem on the dest host(prep_resize), at that point, we need to change the source host claim's instance_uuid to migration_uuid. | |
| 15:08:27 | dansmith | ...right | |
| 15:09:31 | alex_xu | yea...there is no way to change source host claim | |
| 15:11:26 | dansmith | are you saying there's no way to rename a pmem device or whatever you're naming (instance, flavor) ? | |
| 15:12:13 | artom | alex_xu, I feel like we should coordinate on the claim thing | |
| 15:12:41 | artom | Because IIUC your series touches on the same points as NUMA LM | |
| 15:12:59 | alex_xu | dansmith: when nova-compute startup, we populate all the assignment of vpmem and vgpu into memory, store those devices claim in the memory. key by (instance, flavor) | |
| 15:13:15 | alex_xu | artom: yea | |
| 15:13:38 | sean-k-mooney | alex_xu: well why dont we just store them with the instance as teh key | |
| 15:13:51 | dansmith | alex_xu: are the pmem devices actually named something specific in linux? or are they just /dev/pmem/2 and we maintain the mapping between them and the instance? | |
| 15:13:51 | alex_xu | sean-k-mooney: back to the same host resize problem :) | |
| 15:14:10 | artom | alex_xu, what's your series? I'm having trouble searching gerrit for patches you own | |
| 15:14:11 | mriedem | (instance, flavor) is redundant because we have instance.flavor | |
| 15:14:28 | alex_xu | dansmith: they are just /dev/pmem, but I don't get what relationship with the vpmem name at here | |
| 15:14:51 | sean-k-mooney | alex_xu: then use instace+flavor_name which is in the xml or add the uuid to the xml | |
| 15:14:52 | dansmith | alex_xu: okay so I don't understand why you have a problem accounting things with migration uuid vs. instance uuid at any given point | |
| 15:15:01 | mriedem | i think dan is saying, you claim the new_flavor devices on the dest host with the instance, and claim the old_flavor devices on the source host with the migration | |
| 15:15:26 | dansmith | sean-k-mooney: it's not a matter of storing that data, and further, flavor is an object that exists and is keyed in another database, so I think using it as a key here is a bad idea as well | |
| 15:15:33 | dansmith | mriedem: right | |
| 15:15:34 | mriedem | and by "claim the old_flavor devices on the source host with the migration" i mean, swap the mapping for the old_flavor device claim on the source host with the migration for the instance | |
| 15:15:41 | dansmith | mriedem: same pattern as we have elsewhere | |
| 15:16:13 | alex_xu | dansmith: the resize call to the dest host to claim the resource first, then how can we change the in-memory claim info in the source host? by another rpc call, back to the source host? | |
| 15:16:18 | mriedem | when you revert on the source host, you revert that mapping as well | |
| 15:16:38 | dansmith | alex_xu: you get a finish call or a revert call, so you do it then and there right? | |
| 15:16:40 | mriedem | alex_xu: do it when prep_resize casts to resize_instance on the source host? | |
| 15:16:46 | sean-k-mooney | well we could flip it and claim with the migration uuid on the dest | |
| 15:16:52 | dansmith | sean-k-mooney: no | |
| 15:17:02 | sean-k-mooney | ok | |
| 15:17:02 | dansmith | sean-k-mooney: because then you have to change it again if they keep the instance, which is the natural path | |
| 15:17:32 | sean-k-mooney | ya that is true you would have to change it in confrim resize | |
| 15:17:34 | alex_xu | dansmith: we should change to migration uuid before we add new claim for the target | |
| 15:17:42 | mriedem | or like dan said just swap the source host claim during confirm_resize or finish_revert_resize (i'm not sure you'd even need to change the latter) | |
| 15:17:57 | alex_xu | otherwise how can we distinguish the src and dest for the same host resize... | |
| 15:18:14 | mriedem | for same host resize, | |
| 15:18:21 | mriedem | you have some mapping of devices claimed on that host, right? | |
| 15:18:25 | dansmith | alex_xu: are you familiar with how we do this for placement allocations? | |
| 15:18:37 | mriedem | the instance maps to the new flavor device allocations, the migration maps to the old flavor device allocations | |
| 15:18:44 | dansmith | yes ^ | |
| 15:18:46 | alex_xu | yea, I know those thing for placement | |
| 15:19:11 | mriedem | so for same host resize move claim it seems you'd: | |
| 15:19:17 | mriedem | 1. map migration = old flavor | |
| 15:19:27 | mriedem | 2. change instance mapping to point from old to new flavor | |
| 15:19:43 | mriedem | on confirm, you remove the mapping for the migration, | |
| 15:19:47 | mriedem | on revert, you: | |
| 15:19:52 | mriedem | 1. map instance to old flavor, | |
| 15:19:54 | mriedem | 2. drop the migration mapping | |
| 15:19:56 | dansmith | don't say flavor here | |
| 15:19:58 | dansmith | but yes | |
| 15:20:07 | mriedem | flavor device allocation thing 2000 | |