| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-15 | |||
| 15:17:15 | dansmith | right, we need that info stored because we've promised a cluster-wide API for them, and because the filters need to be able to survey the entire landscape efficiently | |
| 15:17:21 | sean-k-mooney | we might not on power off i didnt check | |
| 15:17:41 | sean-k-mooney | e.g. ill take your word for it | |
| 15:20:17 | efried | so there's no reason to store vpmems in the database | |
| 15:20:17 | efried | that means only the virt driver needs to know the connection between an allocation and a specific vpmem namespace, | |
| 15:20:17 | efried | since we're tracking the inventory properly in placement, | |
| 15:20:17 | efried | dansmith: so, for a new resource we're tracking, specifically vpmem for this conversation, | |
| 15:21:09 | dansmith | if that is doable (i.e. we don't have to store them for some other reason) then I think that's ideal, yes | |
| 15:21:13 | efried | since we can recover the vpmem information from the domain xml on operations other than migrations (see below), there's no need to store the vpmem info on the Instance either. | |
| 15:21:13 | efried | and, | |
| 15:21:29 | efried | So the only place we actually need the information is in a migrate context | |
| 15:21:50 | dansmith | do we need it in a migration context? | |
| 15:22:03 | alex_xu | yes | |
| 15:22:06 | efried | not sure | |
| 15:22:07 | dansmith | the two ends of a migration are storing the machine-local context in their own libvirts | |
| 15:22:22 | dansmith | maybe for the revert case? | |
| 15:22:23 | efried | alex_xu: why would we need it in migration context? | |
| 15:22:24 | alex_xu | I need copy the source pmem data to the dest pmem. I need dest pmem device path | |
| 15:22:46 | dansmith | wait | |
| 15:22:58 | dansmith | so for cold migration you're going to migrate the data as well? | |
| 15:23:05 | alex_xu | yes | |
| 15:23:11 | efried | there's a conf option | |
| 15:23:25 | alex_xu | yes, it is configurable by the extra spec | |
| 15:23:31 | dansmith | how is that going to work? write it to disk, consider it a disk image to move and then write it back? | |
| 15:23:41 | sean-k-mooney | dansmith: not by defualt | |
| 15:23:44 | alex_xu | https://review.opendev.org/#/c/634556/12/nova/privsep/libvirt.py | |
| 15:23:54 | dansmith | I *love* this snowflake feature you guys have come up with | |
| 15:23:56 | sean-k-mooney | we are default to not copy the data but we optionally can copy it | |
| 15:23:58 | alex_xu | ^ here is, ssh... | |
| 15:24:17 | dansmith | alex_xu: so you're going to add another scp operation? | |
| 15:24:27 | alex_xu | yes... | |
| 15:24:34 | alex_xu | it is something we hate... | |
| 15:24:34 | dansmith | gross | |
| 15:24:59 | efried | What's the alternative? Block migrations etc. Which is worse? | |
| 15:25:04 | dansmith | so if/when we get to the point of being able to use libvirt to move the images over the TLS tunnel, we're stuck scp'ing this thing? | |
| 15:25:23 | dansmith | efried: not promising that this data is persistent across moves is one option | |
| 15:25:37 | dansmith | efried: isn't this being added as a snowflake "you can never live migrate" feature? | |
| 15:25:54 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: nova-manage: heal port allocations https://review.opendev.org/637955 | |
| 15:25:57 | efried | I thought live migrate was supported | |
| 15:25:58 | sean-k-mooney | dansmith: in the livemigration case qemu/libvirt can copy the data | |
| 15:26:01 | alex_xu | it support lm | |
| 15:26:23 | sean-k-mooney | for cold migrate we have to do it but maybe we can make a libvirt feature request? | |
| 15:26:35 | dansmith | okay, I thought it wasn't going to be able to move that data | |
| 15:26:44 | alex_xu | i'm ok without copy for resize | |
| 15:27:10 | efried | http://specs.openstack.org/openstack/nova-specs/specs/train/approved/virtual-persistent-memory.html#live-migration | |
| 15:27:11 | sean-k-mooney | alex_xu: the issue with scp/rsync for resize is it wont work for cross cell resize | |
| 15:27:22 | efried | apparently libvirt moves it along with RAM | |
| 15:27:33 | sean-k-mooney | alex_xu: e.g. for cross cell resize we cant assume teh compute nodes have network connectivtiy | |
| 15:27:41 | alex_xu | sean-k-mooney: yes, we are going to stop cross cell resize in the initial proposal | |
| 15:27:45 | sean-k-mooney | at least not in the edge deployment case | |
| 15:28:05 | dansmith | live migration is transparent to the user, so that's good, and resize is not, so I think it's reasonable to say that there's no data copy when you resize | |
| 15:28:14 | dansmith | you could resize to a flavor with/without pmem too, | |
| 15:28:24 | dansmith | so not copying would cover that case in both directions | |
| 15:28:38 | efried | per the spec, "reduction" in vpmem would not be allowed | |
| 15:28:50 | dansmith | because of this | |
| 15:29:08 | dansmith | I have to jump on a call now, | |
| 15:29:22 | dansmith | but I'm really unhappy with us adding another "just scp it across" thing | |
| 15:29:35 | efried | alex_xu: Sounds like the spec update https://review.opendev.org/669970 needs to be rethought :) | |
| 15:29:52 | alex_xu | yea, will update again | |
| 15:30:05 | alex_xu | I think the copy for resize can be removed | |
| 15:30:30 | efried | & cold migration | |
| 15:30:36 | efried | (which is the same thing?) | |
| 15:30:38 | dansmith | they are the same thing | |
| 15:31:22 | alex_xu | remove the copy, remove the vpmem field | |
| 15:31:23 | efried | doesn't the VM see the vpmem as persistent storage, though? | |
| 15:31:35 | efried | won't it freak out if it boots and the data is gone? | |
| 15:31:35 | sean-k-mooney | efried: not really | |
| 15:31:45 | efried | okay, then yeah, that vastly simplifies things. | |
| 15:31:56 | sean-k-mooney | it sees it as ram/dimms | |
| 15:32:06 | efried | unfortunate that we already merged https://review.opendev.org/#/c/662697/ -- seems as though we won't be using that? | |
| 15:32:07 | sean-k-mooney | the data is ment to be persitent | |
| 15:32:11 | alex_xu | and the most of usecase is for cache, so it is ok | |
| 15:32:37 | sean-k-mooney | but you should really store your data soewhere else and keep the working set in teh pmem | |
| 15:33:47 | sean-k-mooney | ya most workload use it as high capasity scratch space for operating on a subset of the data but the long term storage of the data should be in a cinder volume | |
| 15:36:38 | efried | I've put a hold on https://review.opendev.org/#/c/634548/ for now | |
| 15:51:01 | gibi | efried: thanks for catching bug in https://review.opendev.org/#/c/637955/ I've fixed it. | |
| 15:51:21 | efried | gibi: cool | |
| 15:51:33 | efried | got a nice backlog today, but hopefully I can get back around to it. | |
| 15:51:55 | efried | gibi: since you're around, would you mind pushing https://review.opendev.org/#/c/657464/ ? | |
| 15:52:11 | gibi | efried: on it | |
| 15:52:40 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Update supported transports for iscsi connector https://review.opendev.org/524443 | |
| 15:52:51 | efried | thanks | |
| 16:00:41 | efried | sean-k-mooney: is rebuild admin-only? | |
| 16:00:46 | sean-k-mooney | no | |
| 16:00:57 | sean-k-mooney | rebuild and resize can both be done by tenants | |
| 16:01:34 | efried | sean-k-mooney: okay, so back to that thing we were looking at earlier, it looks like setup_networks_on_host is caled in rebuild and resize flows as well as migrate | |
| 16:03:17 | efried | hm, unless teardown=True only happens on migrations... | |
| 16:03:36 | openstackgerrit | Merged openstack/os-resource-classes master: Propose FPGA and PGPU resource classes https://review.opendev.org/657464 | |
| 16:03:42 | efried | does _confirm_resize only happen on migration-y resizes? | |
| 16:04:53 | sean-k-mooney | efried: it happesn on resize or cold migration | |
| 16:05:22 | sean-k-mooney | in both cases we go into resize_verify and you chave to do reseize --confim | |
| 16:05:25 | efried | rats, nother meeting, will come back to this... | |
| 16:06:01 | sean-k-mooney | efried: stephenfin is adding an openstack server migrate --confirm as synatactic sugar but its the same call underneath | |
| 16:07:14 | stephenfin | that threw me too, fwiw | |
| 16:07:32 | stephenfin | (whether cold migrations needed to be confirmed or not) | |
| 16:07:51 | sean-k-mooney | stephenfin: well that depends on your config settings | |
| 16:07:58 | sean-k-mooney | but ya | |
| 16:08:40 | sean-k-mooney | i hit that oddity back in hevana so at this point i dont even think about them differently anymore. | |
| 16:08:48 | sean-k-mooney | (cold migraton vs resize) | |
| 17:17:36 | bbobrov | sean-k-mooney: hey | |