| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-20 | |||
| 22:03:38 | efried | I doubt it though. | |
| 22:03:39 | efried | yeah. | |
| 22:03:45 | dansmith | efried: we'd have to give it a volume id for that or something | |
| 22:04:08 | dansmith | the flow chart of "how your instance behaves if you have a vtpm" is starting to get pretty scary | |
| 22:04:10 | efried | today it makes up the directory based on the VM ID | |
| 22:04:20 | efried | dansmith: well, it's always the corner cases that suck. | |
| 22:04:25 | efried | 80/20 | |
| 22:04:37 | dansmith | efried: well, snapshot is not a corner case IMHO :) | |
| 22:04:50 | dansmith | and ceph definitely isn't | |
| 22:04:53 | efried | no, but second-restore-of-snapshot-because-host-went-down-twice is | |
| 22:05:04 | dansmith | I wouldn't say so | |
| 22:05:05 | sean-k-mooney | so unless you take steps to put the vtpm data store on shared storeate i think we can assume it would be lost on evacuate in general | |
| 22:05:17 | efried | sean-k-mooney: definitely that | |
| 22:05:23 | dansmith | sean-k-mooney: which means unbootable instance if you put your FDE keys there | |
| 22:05:50 | dansmith | so in the flow chart, it means evacuate has a giant asterisk next to it :) | |
| 22:05:53 | sean-k-mooney | yes maybe although resuce could save you? | |
| 22:06:10 | dansmith | "we will reconstruct the smoking hull of an instance somewhere else where you know your data is in there, but it's unreadable" :) | |
| 22:06:16 | sean-k-mooney | e.g. allwo you to reinject the key if you still have it | |
| 22:06:27 | dansmith | sean-k-mooney: the whole point of the tpm is to not have to do that right? | |
| 22:06:37 | efried | not to be *able* to do that | |
| 22:06:43 | dansmith | secure enclave doesn't have the same meaning if you also have it on a post-it under your monitor | |
| 22:06:51 | sean-k-mooney | it has other services it can provide to the os but key storage is one of them yes | |
| 22:07:34 | efried | I guess I'll just write up the big asterisk in the spec & docs. | |
| 22:07:57 | dansmith | I'd like to see a list of all these special cases yeah | |
| 22:07:59 | efried | You would kinda have to make a new backup of the vtpm every time you write to it. | |
| 22:08:12 | dansmith | because if the list is too large, it starts to become not so useful | |
| 22:09:22 | sean-k-mooney | efried: you could make the same argument for the root disk i think that is out of scope of nova | |
| 22:09:40 | efried | sean-k-mooney: right, but the answer to "evacuate doesn't help you with ephemeral root disk" is "use volume/ceph". | |
| 22:10:09 | sean-k-mooney | and here its use barbican | |
| 22:10:10 | efried | with a vTPM the answer to "evacuate doesn't help you with ephemeral root disk" is "you better have a very recent snapshot of your vTPM" | |
| 22:10:15 | efried | no | |
| 22:10:24 | efried | barbican only stores the key used to unlock the file we've been discussing. | |
| 22:10:31 | efried | said file will be stored in swift | |
| 22:10:35 | efried | but only during snapshot | |
| 22:10:42 | sean-k-mooney | sure but as a user you can also store keys your self in barbican | |
| 22:10:49 | efried | how does that help? | |
| 22:10:57 | efried | you mean if you want to not use a vTPM at all? | |
| 22:11:14 | sean-k-mooney | use the tpm as a local secure cache fo the key | |
| 22:11:24 | sean-k-mooney | if you evacuate retive the backup form barbican | |
| 22:11:25 | efried | if barbican is secure enough for you, you would just use barbican. | |
| 22:11:43 | efried | though arguably you're counting on that ultimately here anyway. | |
| 22:11:45 | sean-k-mooney | you dont want to transfer it over the network everythime you boot | |
| 22:11:56 | efried | we're doing that anyway. | |
| 22:12:02 | sean-k-mooney | well ya | |
| 22:12:19 | sean-k-mooney | i was just going to say you have to do that to unlock the vtpm | |
| 22:12:27 | efried | Also I believe "VM talks to barbican" is a thing we're avoiding | |
| 22:12:36 | efried | limiting to "host talks to barbican" | |
| 22:13:10 | sean-k-mooney | barbican is a top level openstack sericve so worklaods can use it if they want too | |
| 22:13:16 | efried | if they want to, yes. | |
| 22:13:30 | efried | but it's the openstack user's credentials as opposed to $random_vm_user | |
| 22:13:46 | sean-k-mooney | oh yes that is what i ment by vm | |
| 22:13:48 | efried | and also I imagine there are reasons for the VM to be blocked from the keymgr service network-wise. | |
| 22:13:49 | sean-k-mooney | sorry | |
| 22:13:58 | sean-k-mooney | i ment application in the vm | |
| 22:15:28 | sean-k-mooney | anyway i think a resonable alternitive is if you need the keys to sruvice beyond the life time of the vm or be shared in a "sercure" way store them in barbican | |
| 22:15:57 | sean-k-mooney | if you you can tollerate losing the key in a evacuate case then your can store it soly in the tpm | |
| 22:16:44 | sean-k-mooney | dansmith: you primary concern is we loose the key used to encyrpt the os drive and loose data right if we loos the vtpm | |
| 22:16:52 | sean-k-mooney | wyoud ^ not solve that issue | |
| 22:17:10 | sean-k-mooney | *would | |
| 22:18:37 | efried | resize does a destroy and then a spawn? | |
| 22:19:02 | sean-k-mooney | it will redefine the domain but you dont loose data | |
| 22:19:11 | sean-k-mooney | and it may change host | |
| 22:20:03 | sean-k-mooney | e.g. if you resize form a flavor with 10G root disk to one with 20 it will grow the disk file | |
| 22:20:17 | sean-k-mooney | and if its libvirt the partion table and filesystem if it can | |
| 22:20:52 | sean-k-mooney | so we would have to copy the vtpm file if we move host | |
| 22:21:06 | efried | yeah, I'm exploring corner cases with the vTPM. If your resize adds/removes a vTPM where it was previously absent/present, that's fine. If both old & new flavor have the same version/model, we should carry the data. If they have a different version/model...? | |
| 22:21:07 | sean-k-mooney | if its to the same host we shoudl not need to do anything | |
| 22:21:39 | sean-k-mooney | if they are different models i would expect you to loose the contents of the tpm | |
| 22:21:52 | sean-k-mooney | unless we block that intentionally | |
| 22:22:08 | efried | yes to the first, the second is the question. | |
| 22:22:25 | efried | there's no way to persist the data, period. The versions/models are incompatible. | |
| 22:23:04 | sean-k-mooney | ya that is proably true or at least involved to do so i would not try in v1 of this | |
| 22:23:31 | sean-k-mooney | there might be a way to convert but im not aware of one off the top of my head | |
| 22:24:12 | efried | sean-k-mooney: not without introspecting, which ain't my business. | |
| 22:24:30 | sean-k-mooney | swtpm might have a fucntion to do it | |
| 22:24:49 | sean-k-mooney | but ya unless libvirt support it i think its out of scope | |
| 22:24:55 | efried | so I'm gonna say don't block it, cause who am I to say you didn't mean to do it | |
| 22:25:29 | sean-k-mooney | that would be my feeling too | |
| 22:25:51 | sean-k-mooney | we shoudl documetn it i guess | |
| 22:26:02 | efried | doing that now | |
| 22:26:21 | sean-k-mooney | but i dont think we shoudl guard in code against all pebcak errors | |
| 22:26:29 | efried | agreed | |
| 22:27:29 | efried | hmph, the thing about saving the swift ID in instance sysmeta has a problem code flow-wise. | |
| 22:27:48 | efried | because the driver is the thing that knows the vdev is in this particular file path | |
| 22:27:59 | efried | but the only thing we've asked the driver to do is snapshot. | |
| 22:28:19 | efried | and (I think) the driver can't tell if it's being asked to snapshot as part of a shelve or backup or... | |
| 22:28:47 | sean-k-mooney | am im not 100% sure about that last part | |
| 22:29:04 | efried | so I guess it could edit the sysmeta itself... but if it's not a shelve and the compute mgr is about to blow away the instance, the compute mgr needs a way to get that information so it can stuff it into the image meta. | |
| 22:29:13 | efried | I guess it can pull it from the instance | |
| 22:29:43 | efried | so the contract with the virt driver's snapshot method has to spell out where & how this info is stored in the sysmeta. | |
| 22:30:29 | sean-k-mooney | we create snapshot for shelve but i dont think the compute magener actully calls snapshot as part of that flow i think its donw by the driver | |
| 22:30:47 | efried | eh? | |
| 22:30:49 | sean-k-mooney | that said i could be way off base there so im checking | |
| 22:30:56 | efried | compute manager's shelve calls driver's snapshot | |
| 22:31:24 | sean-k-mooney | oh ok | |
| 22:31:31 | efried | https://opendev.org/openstack/nova/src/branch/master/nova/compute/manager.py#L5911 | |
| 22:32:32 | sean-k-mooney | ya your right i though we had a shelve fucntion on the driver interface that we called but this is handeled in the compute manager instead | |
| 22:33:30 | sean-k-mooney | so snapshot is passed the instace so it can get the flavor and image metadata | |
| 22:33:58 | sean-k-mooney | so the snapshot function can check if the isntace has a vtpm and directly update the system metadata as you said | |
| 22:34:14 | sean-k-mooney | oh right but it would not know if its a shleve or not | |