Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-11-20
22:04:08 dansmith the flow chart of "how your instance behaves if you have a vtpm" is starting to get pretty scary
22:04:10 efried today it makes up the directory based on the VM ID
22:04:20 efried dansmith: well, it's always the corner cases that suck.
22:04:25 efried 80/20
22:04:37 dansmith efried: well, snapshot is not a corner case IMHO :)
22:04:50 dansmith and ceph definitely isn't
22:04:53 efried no, but second-restore-of-snapshot-because-host-went-down-twice is
22:05:04 dansmith I wouldn't say so
22:05:05 sean-k-mooney so unless you take steps to put the vtpm data store on shared storeate i think we can assume it would be lost on evacuate in general
22:05:17 efried sean-k-mooney: definitely that
22:05:23 dansmith sean-k-mooney: which means unbootable instance if you put your FDE keys there
22:05:50 dansmith so in the flow chart, it means evacuate has a giant asterisk next to it :)
22:05:53 sean-k-mooney yes maybe although resuce could save you?
22:06:10 dansmith "we will reconstruct the smoking hull of an instance somewhere else where you know your data is in there, but it's unreadable" :)
22:06:16 sean-k-mooney e.g. allwo you to reinject the key if you still have it
22:06:27 dansmith sean-k-mooney: the whole point of the tpm is to not have to do that right?
22:06:37 efried not to be *able* to do that
22:06:43 dansmith secure enclave doesn't have the same meaning if you also have it on a post-it under your monitor
22:06:51 sean-k-mooney it has other services it can provide to the os but key storage is one of them yes
22:07:34 efried I guess I'll just write up the big asterisk in the spec & docs.
22:07:57 dansmith I'd like to see a list of all these special cases yeah
22:07:59 efried You would kinda have to make a new backup of the vtpm every time you write to it.
22:08:12 dansmith because if the list is too large, it starts to become not so useful
22:09:22 sean-k-mooney efried: you could make the same argument for the root disk i think that is out of scope of nova
22:09:40 efried sean-k-mooney: right, but the answer to "evacuate doesn't help you with ephemeral root disk" is "use volume/ceph".
22:10:09 sean-k-mooney and here its use barbican
22:10:10 efried with a vTPM the answer to "evacuate doesn't help you with ephemeral root disk" is "you better have a very recent snapshot of your vTPM"
22:10:15 efried no
22:10:24 efried barbican only stores the key used to unlock the file we've been discussing.
22:10:31 efried said file will be stored in swift
22:10:35 efried but only during snapshot
22:10:42 sean-k-mooney sure but as a user you can also store keys your self in barbican
22:10:49 efried how does that help?
22:10:57 efried you mean if you want to not use a vTPM at all?
22:11:14 sean-k-mooney use the tpm as a local secure cache fo the key
22:11:24 sean-k-mooney if you evacuate retive the backup form barbican
22:11:25 efried if barbican is secure enough for you, you would just use barbican.
22:11:43 efried though arguably you're counting on that ultimately here anyway.
22:11:45 sean-k-mooney you dont want to transfer it over the network everythime you boot
22:11:56 efried we're doing that anyway.
22:12:02 sean-k-mooney well ya
22:12:19 sean-k-mooney i was just going to say you have to do that to unlock the vtpm
22:12:27 efried Also I believe "VM talks to barbican" is a thing we're avoiding
22:12:36 efried limiting to "host talks to barbican"
22:13:10 sean-k-mooney barbican is a top level openstack sericve so worklaods can use it if they want too
22:13:16 efried if they want to, yes.
22:13:30 efried but it's the openstack user's credentials as opposed to $random_vm_user
22:13:46 sean-k-mooney oh yes that is what i ment by vm
22:13:48 efried and also I imagine there are reasons for the VM to be blocked from the keymgr service network-wise.
22:13:49 sean-k-mooney sorry
22:13:58 sean-k-mooney i ment application in the vm
22:15:28 sean-k-mooney anyway i think a resonable alternitive is if you need the keys to sruvice beyond the life time of the vm or be shared in a "sercure" way store them in barbican
22:15:57 sean-k-mooney if you you can tollerate losing the key in a evacuate case then your can store it soly in the tpm
22:16:44 sean-k-mooney dansmith: you primary concern is we loose the key used to encyrpt the os drive and loose data right if we loos the vtpm
22:16:52 sean-k-mooney wyoud ^ not solve that issue
22:17:10 sean-k-mooney *would
22:18:37 efried resize does a destroy and then a spawn?
22:19:02 sean-k-mooney it will redefine the domain but you dont loose data
22:19:11 sean-k-mooney and it may change host
22:20:03 sean-k-mooney e.g. if you resize form a flavor with 10G root disk to one with 20 it will grow the disk file
22:20:17 sean-k-mooney and if its libvirt the partion table and filesystem if it can
22:20:52 sean-k-mooney so we would have to copy the vtpm file if we move host
22:21:06 efried yeah, I'm exploring corner cases with the vTPM. If your resize adds/removes a vTPM where it was previously absent/present, that's fine. If both old & new flavor have the same version/model, we should carry the data. If they have a different version/model...?
22:21:07 sean-k-mooney if its to the same host we shoudl not need to do anything
22:21:39 sean-k-mooney if they are different models i would expect you to loose the contents of the tpm
22:21:52 sean-k-mooney unless we block that intentionally
22:22:08 efried yes to the first, the second is the question.
22:22:25 efried there's no way to persist the data, period. The versions/models are incompatible.
22:23:04 sean-k-mooney ya that is proably true or at least involved to do so i would not try in v1 of this
22:23:31 sean-k-mooney there might be a way to convert but im not aware of one off the top of my head
22:24:12 efried sean-k-mooney: not without introspecting, which ain't my business.
22:24:30 sean-k-mooney swtpm might have a fucntion to do it
22:24:49 sean-k-mooney but ya unless libvirt support it i think its out of scope
22:24:55 efried so I'm gonna say don't block it, cause who am I to say you didn't mean to do it
22:25:29 sean-k-mooney that would be my feeling too
22:25:51 sean-k-mooney we shoudl documetn it i guess
22:26:02 efried doing that now
22:26:21 sean-k-mooney but i dont think we shoudl guard in code against all pebcak errors
22:26:29 efried agreed
22:27:29 efried hmph, the thing about saving the swift ID in instance sysmeta has a problem code flow-wise.
22:27:48 efried because the driver is the thing that knows the vdev is in this particular file path
22:27:59 efried but the only thing we've asked the driver to do is snapshot.
22:28:19 efried and (I think) the driver can't tell if it's being asked to snapshot as part of a shelve or backup or...
22:28:47 sean-k-mooney am im not 100% sure about that last part
22:29:04 efried so I guess it could edit the sysmeta itself... but if it's not a shelve and the compute mgr is about to blow away the instance, the compute mgr needs a way to get that information so it can stuff it into the image meta.
22:29:13 efried I guess it can pull it from the instance
22:29:43 efried so the contract with the virt driver's snapshot method has to spell out where & how this info is stored in the sysmeta.
22:30:29 sean-k-mooney we create snapshot for shelve but i dont think the compute magener actully calls snapshot as part of that flow i think its donw by the driver
22:30:47 efried eh?
22:30:49 sean-k-mooney that said i could be way off base there so im checking
22:30:56 efried compute manager's shelve calls driver's snapshot
22:31:24 sean-k-mooney oh ok
22:31:31 efried https://opendev.org/openstack/nova/src/branch/master/nova/compute/manager.py#L5911
22:32:32 sean-k-mooney ya your right i though we had a shelve fucntion on the driver interface that we called but this is handeled in the compute manager instead
22:33:30 sean-k-mooney so snapshot is passed the instace so it can get the flavor and image metadata
22:33:58 sean-k-mooney so the snapshot function can check if the isntace has a vtpm and directly update the system metadata as you said
22:34:14 sean-k-mooney oh right but it would not know if its a shleve or not
22:36:00 sean-k-mooney oh it can jsut check the instance.task_state right?
22:36:38 efried could, but compute mgr has to be able to peel it out anyway, so it's simpler if snapshot just always behaves the same and compute mgr pulls the info out of the sysmeta if it needs it.
22:37:14 sean-k-mooney ok

Earlier   Later