| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-20 | |||
| 21:56:47 | dansmith | efried: I'm not saying that it should necessarily, I'm saying it's going to eff up someone one way or the other | |
| 21:57:09 | dansmith | people use snapshot to roll back from a bad OS upgrade, and if you don't keep their keys, they're going to get screwed | |
| 21:57:12 | sean-k-mooney | should it be configurable? | |
| 21:57:19 | dansmith | but if you do, and they spawn that image in a bunch of places, ... | |
| 21:57:37 | efried | dansmith: well... we could also delete the swift obj (and mod the image meta) the first time we restore it. | |
| 21:57:50 | efried | that works poorly for crashy hosts tho | |
| 21:58:09 | dansmith | efried: that's fine if you store it in the image record, despite a failed host | |
| 21:58:26 | dansmith | efried: but I think that's probably going to be fragile and maybe confusing why the first instance got it and none of the others did | |
| 21:58:30 | dansmith | but perhaps that's the best option | |
| 21:58:35 | dansmith | just saying, it's going to be icky | |
| 21:59:13 | efried | I'm saying: if my workflow is to back up stuff in case my host goes down, and I do that backup, and then my host goes down, and I restore it, and then my host goes down again, then when I restore it the second time, my vTPM will be gone. | |
| 21:59:31 | dansmith | oh, yes, I see what you mean | |
| 21:59:35 | sean-k-mooney | is this something we want the user to express a policy on e.g. hw_vtpm_snapshot=true|false | |
| 21:59:53 | dansmith | efried: this is also bad for the shared storage case in general actually, | |
| 22:00:02 | efried | sean-k-mooney: we could configure it to death, really. | |
| 22:00:07 | dansmith | efried: one thing about ceph backed instances is you can recover from a failed host with an evac | |
| 22:00:40 | dansmith | efried: but in this case you will fail to boot after evac because we don't have the keys anywhere | |
| 22:00:40 | sean-k-mooney | dansmith: you would loose the tpm data in that case | |
| 22:00:41 | dansmith | so that's another case this breaks pretty badly | |
| 22:00:46 | dansmith | volume-backed or ceph/nfs backed ephemeral | |
| 22:00:51 | dansmith | sean-k-mooney: right, that's what I just said | |
| 22:01:46 | sean-k-mooney | nfs might be ok if you also put the tpm datastore on nfs but ya. the impact of that will depend on what you used the tpm for i guess | |
| 22:02:09 | dansmith | sean-k-mooney: oh right, but not ceph | |
| 22:02:31 | efried | because ceph can only handle the disk itself? | |
| 22:02:36 | efried | similar to volume-backed | |
| 22:02:44 | dansmith | we don't actually mount the storage on the host with ceph | |
| 22:02:54 | dansmith | qemu talks to ceph itself, so we don't like have a dir mounted or anything | |
| 22:02:55 | sean-k-mooney | the tpm data store is not a volume so we wont create it as such in ceph | |
| 22:03:03 | efried | wow, neat. | |
| 22:03:22 | efried | dansmith: you know, in that case it's juuust possible qemu will store the vTPM file on ceph too | |
| 22:03:26 | sean-k-mooney | we kind of which qemu could do the same for the other backedn too | |
| 22:03:37 | dansmith | efried: I don't think so | |
| 22:03:38 | efried | I doubt it though. | |
| 22:03:39 | efried | yeah. | |
| 22:03:45 | dansmith | efried: we'd have to give it a volume id for that or something | |
| 22:04:08 | dansmith | the flow chart of "how your instance behaves if you have a vtpm" is starting to get pretty scary | |
| 22:04:10 | efried | today it makes up the directory based on the VM ID | |
| 22:04:20 | efried | dansmith: well, it's always the corner cases that suck. | |
| 22:04:25 | efried | 80/20 | |
| 22:04:37 | dansmith | efried: well, snapshot is not a corner case IMHO :) | |
| 22:04:50 | dansmith | and ceph definitely isn't | |
| 22:04:53 | efried | no, but second-restore-of-snapshot-because-host-went-down-twice is | |
| 22:05:04 | dansmith | I wouldn't say so | |
| 22:05:05 | sean-k-mooney | so unless you take steps to put the vtpm data store on shared storeate i think we can assume it would be lost on evacuate in general | |
| 22:05:17 | efried | sean-k-mooney: definitely that | |
| 22:05:23 | dansmith | sean-k-mooney: which means unbootable instance if you put your FDE keys there | |
| 22:05:50 | dansmith | so in the flow chart, it means evacuate has a giant asterisk next to it :) | |
| 22:05:53 | sean-k-mooney | yes maybe although resuce could save you? | |
| 22:06:10 | dansmith | "we will reconstruct the smoking hull of an instance somewhere else where you know your data is in there, but it's unreadable" :) | |
| 22:06:16 | sean-k-mooney | e.g. allwo you to reinject the key if you still have it | |
| 22:06:27 | dansmith | sean-k-mooney: the whole point of the tpm is to not have to do that right? | |
| 22:06:37 | efried | not to be *able* to do that | |
| 22:06:43 | dansmith | secure enclave doesn't have the same meaning if you also have it on a post-it under your monitor | |
| 22:06:51 | sean-k-mooney | it has other services it can provide to the os but key storage is one of them yes | |
| 22:07:34 | efried | I guess I'll just write up the big asterisk in the spec & docs. | |
| 22:07:57 | dansmith | I'd like to see a list of all these special cases yeah | |
| 22:07:59 | efried | You would kinda have to make a new backup of the vtpm every time you write to it. | |
| 22:08:12 | dansmith | because if the list is too large, it starts to become not so useful | |
| 22:09:22 | sean-k-mooney | efried: you could make the same argument for the root disk i think that is out of scope of nova | |
| 22:09:40 | efried | sean-k-mooney: right, but the answer to "evacuate doesn't help you with ephemeral root disk" is "use volume/ceph". | |
| 22:10:09 | sean-k-mooney | and here its use barbican | |
| 22:10:10 | efried | with a vTPM the answer to "evacuate doesn't help you with ephemeral root disk" is "you better have a very recent snapshot of your vTPM" | |
| 22:10:15 | efried | no | |
| 22:10:24 | efried | barbican only stores the key used to unlock the file we've been discussing. | |
| 22:10:31 | efried | said file will be stored in swift | |
| 22:10:35 | efried | but only during snapshot | |
| 22:10:42 | sean-k-mooney | sure but as a user you can also store keys your self in barbican | |
| 22:10:49 | efried | how does that help? | |
| 22:10:57 | efried | you mean if you want to not use a vTPM at all? | |
| 22:11:14 | sean-k-mooney | use the tpm as a local secure cache fo the key | |
| 22:11:24 | sean-k-mooney | if you evacuate retive the backup form barbican | |
| 22:11:25 | efried | if barbican is secure enough for you, you would just use barbican. | |
| 22:11:43 | efried | though arguably you're counting on that ultimately here anyway. | |
| 22:11:45 | sean-k-mooney | you dont want to transfer it over the network everythime you boot | |
| 22:11:56 | efried | we're doing that anyway. | |
| 22:12:02 | sean-k-mooney | well ya | |
| 22:12:19 | sean-k-mooney | i was just going to say you have to do that to unlock the vtpm | |
| 22:12:27 | efried | Also I believe "VM talks to barbican" is a thing we're avoiding | |
| 22:12:36 | efried | limiting to "host talks to barbican" | |
| 22:13:10 | sean-k-mooney | barbican is a top level openstack sericve so worklaods can use it if they want too | |
| 22:13:16 | efried | if they want to, yes. | |
| 22:13:30 | efried | but it's the openstack user's credentials as opposed to $random_vm_user | |
| 22:13:46 | sean-k-mooney | oh yes that is what i ment by vm | |
| 22:13:48 | efried | and also I imagine there are reasons for the VM to be blocked from the keymgr service network-wise. | |
| 22:13:49 | sean-k-mooney | sorry | |
| 22:13:58 | sean-k-mooney | i ment application in the vm | |
| 22:15:28 | sean-k-mooney | anyway i think a resonable alternitive is if you need the keys to sruvice beyond the life time of the vm or be shared in a "sercure" way store them in barbican | |
| 22:15:57 | sean-k-mooney | if you you can tollerate losing the key in a evacuate case then your can store it soly in the tpm | |
| 22:16:44 | sean-k-mooney | dansmith: you primary concern is we loose the key used to encyrpt the os drive and loose data right if we loos the vtpm | |
| 22:16:52 | sean-k-mooney | wyoud ^ not solve that issue | |
| 22:17:10 | sean-k-mooney | *would | |
| 22:18:37 | efried | resize does a destroy and then a spawn? | |
| 22:19:02 | sean-k-mooney | it will redefine the domain but you dont loose data | |
| 22:19:11 | sean-k-mooney | and it may change host | |
| 22:20:03 | sean-k-mooney | e.g. if you resize form a flavor with 10G root disk to one with 20 it will grow the disk file | |
| 22:20:17 | sean-k-mooney | and if its libvirt the partion table and filesystem if it can | |
| 22:20:52 | sean-k-mooney | so we would have to copy the vtpm file if we move host | |
| 22:21:06 | efried | yeah, I'm exploring corner cases with the vTPM. If your resize adds/removes a vTPM where it was previously absent/present, that's fine. If both old & new flavor have the same version/model, we should carry the data. If they have a different version/model...? | |
| 22:21:07 | sean-k-mooney | if its to the same host we shoudl not need to do anything | |
| 22:21:39 | sean-k-mooney | if they are different models i would expect you to loose the contents of the tpm | |