Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-07-15
15:16:14 sean-k-mooney dansmith: well if we delete teh domain on power off we need to reserve the cores so that we dont violate the requiremetn by booting a new vm
15:16:15 efried operations like scheduling, determining available resources...
15:16:23 sean-k-mooney if we dont undefine the domain on power off
15:16:31 dansmith sean-k-mooney: again, we do not delete the domain on poweroff
15:16:32 sean-k-mooney then its possible we dont need to store tehm for that
15:16:40 sean-k-mooney but we would still want to have it for the filter
15:17:09 sean-k-mooney we do on power on because it calls hard reboot
15:17:14 efried So in the utopian future where all of that is done properly in placement, the scheduler wouldn't need that info in the db and we could theoretically get rid of it.
15:17:15 dansmith right, we need that info stored because we've promised a cluster-wide API for them, and because the filters need to be able to survey the entire landscape efficiently
15:17:21 sean-k-mooney we might not on power off i didnt check
15:17:41 sean-k-mooney e.g. ill take your word for it
15:20:17 efried so there's no reason to store vpmems in the database
15:20:17 efried that means only the virt driver needs to know the connection between an allocation and a specific vpmem namespace,
15:20:17 efried since we're tracking the inventory properly in placement,
15:20:17 efried dansmith: so, for a new resource we're tracking, specifically vpmem for this conversation,
15:21:09 dansmith if that is doable (i.e. we don't have to store them for some other reason) then I think that's ideal, yes
15:21:13 efried since we can recover the vpmem information from the domain xml on operations other than migrations (see below), there's no need to store the vpmem info on the Instance either.
15:21:13 efried and,
15:21:29 efried So the only place we actually need the information is in a migrate context
15:21:50 dansmith do we need it in a migration context?
15:22:03 alex_xu yes
15:22:06 efried not sure
15:22:07 dansmith the two ends of a migration are storing the machine-local context in their own libvirts
15:22:22 dansmith maybe for the revert case?
15:22:23 efried alex_xu: why would we need it in migration context?
15:22:24 alex_xu I need copy the source pmem data to the dest pmem. I need dest pmem device path
15:22:46 dansmith wait
15:22:58 dansmith so for cold migration you're going to migrate the data as well?
15:23:05 alex_xu yes
15:23:11 efried there's a conf option
15:23:25 alex_xu yes, it is configurable by the extra spec
15:23:31 dansmith how is that going to work? write it to disk, consider it a disk image to move and then write it back?
15:23:41 sean-k-mooney dansmith: not by defualt
15:23:44 alex_xu https://review.opendev.org/#/c/634556/12/nova/privsep/libvirt.py
15:23:54 dansmith I *love* this snowflake feature you guys have come up with
15:23:56 sean-k-mooney we are default to not copy the data but we optionally can copy it
15:23:58 alex_xu ^ here is, ssh...
15:24:17 dansmith alex_xu: so you're going to add another scp operation?
15:24:27 alex_xu yes...
15:24:34 alex_xu it is something we hate...
15:24:34 dansmith gross
15:24:59 efried What's the alternative? Block migrations etc. Which is worse?
15:25:04 dansmith so if/when we get to the point of being able to use libvirt to move the images over the TLS tunnel, we're stuck scp'ing this thing?
15:25:23 dansmith efried: not promising that this data is persistent across moves is one option
15:25:37 dansmith efried: isn't this being added as a snowflake "you can never live migrate" feature?
15:25:54 openstackgerrit Balazs Gibizer proposed openstack/nova master: nova-manage: heal port allocations https://review.opendev.org/637955
15:25:57 efried I thought live migrate was supported
15:25:58 sean-k-mooney dansmith: in the livemigration case qemu/libvirt can copy the data
15:26:01 alex_xu it support lm
15:26:23 sean-k-mooney for cold migrate we have to do it but maybe we can make a libvirt feature request?
15:26:35 dansmith okay, I thought it wasn't going to be able to move that data
15:26:44 alex_xu i'm ok without copy for resize
15:27:10 efried http://specs.openstack.org/openstack/nova-specs/specs/train/approved/virtual-persistent-memory.html#live-migration
15:27:11 sean-k-mooney alex_xu: the issue with scp/rsync for resize is it wont work for cross cell resize
15:27:22 efried apparently libvirt moves it along with RAM
15:27:33 sean-k-mooney alex_xu: e.g. for cross cell resize we cant assume teh compute nodes have network connectivtiy
15:27:41 alex_xu sean-k-mooney: yes, we are going to stop cross cell resize in the initial proposal
15:27:45 sean-k-mooney at least not in the edge deployment case
15:28:05 dansmith live migration is transparent to the user, so that's good, and resize is not, so I think it's reasonable to say that there's no data copy when you resize
15:28:14 dansmith you could resize to a flavor with/without pmem too,
15:28:24 dansmith so not copying would cover that case in both directions
15:28:38 efried per the spec, "reduction" in vpmem would not be allowed
15:28:50 dansmith because of this
15:29:08 dansmith I have to jump on a call now,
15:29:22 dansmith but I'm really unhappy with us adding another "just scp it across" thing
15:29:35 efried alex_xu: Sounds like the spec update https://review.opendev.org/669970 needs to be rethought :)
15:29:52 alex_xu yea, will update again
15:30:05 alex_xu I think the copy for resize can be removed
15:30:30 efried & cold migration
15:30:36 efried (which is the same thing?)
15:30:38 dansmith they are the same thing
15:31:22 alex_xu remove the copy, remove the vpmem field
15:31:23 efried doesn't the VM see the vpmem as persistent storage, though?
15:31:35 efried won't it freak out if it boots and the data is gone?
15:31:35 sean-k-mooney efried: not really
15:31:45 efried okay, then yeah, that vastly simplifies things.
15:31:56 sean-k-mooney it sees it as ram/dimms
15:32:06 efried unfortunate that we already merged https://review.opendev.org/#/c/662697/ -- seems as though we won't be using that?
15:32:07 sean-k-mooney the data is ment to be persitent
15:32:11 alex_xu and the most of usecase is for cache, so it is ok
15:32:37 sean-k-mooney but you should really store your data soewhere else and keep the working set in teh pmem
15:33:47 sean-k-mooney ya most workload use it as high capasity scratch space for operating on a subset of the data but the long term storage of the data should be in a cinder volume
15:36:38 efried I've put a hold on https://review.opendev.org/#/c/634548/ for now
15:51:01 gibi efried: thanks for catching bug in https://review.opendev.org/#/c/637955/ I've fixed it.
15:51:21 efried gibi: cool
15:51:33 efried got a nice backlog today, but hopefully I can get back around to it.
15:51:55 efried gibi: since you're around, would you mind pushing https://review.opendev.org/#/c/657464/ ?
15:52:11 gibi efried: on it
15:52:40 openstackgerrit Stephen Finucane proposed openstack/nova master: Update supported transports for iscsi connector https://review.opendev.org/524443
15:52:51 efried thanks
16:00:41 efried sean-k-mooney: is rebuild admin-only?
16:00:46 sean-k-mooney no
16:00:57 sean-k-mooney rebuild and resize can both be done by tenants
16:01:34 efried sean-k-mooney: okay, so back to that thing we were looking at earlier, it looks like setup_networks_on_host is caled in rebuild and resize flows as well as migrate
16:03:17 efried hm, unless teardown=True only happens on migrations...
16:03:36 openstackgerrit Merged openstack/os-resource-classes master: Propose FPGA and PGPU resource classes https://review.opendev.org/657464
16:03:42 efried does _confirm_resize only happen on migration-y resizes?
16:04:53 sean-k-mooney efried: it happesn on resize or cold migration
16:05:22 sean-k-mooney in both cases we go into resize_verify and you chave to do reseize --confim
16:05:25 efried rats, nother meeting, will come back to this...

Earlier   Later