| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-20 | |||
| 22:04:10 | efried | today it makes up the directory based on the VM ID | |
| 22:04:20 | efried | dansmith: well, it's always the corner cases that suck. | |
| 22:04:25 | efried | 80/20 | |
| 22:04:37 | dansmith | efried: well, snapshot is not a corner case IMHO :) | |
| 22:04:50 | dansmith | and ceph definitely isn't | |
| 22:04:53 | efried | no, but second-restore-of-snapshot-because-host-went-down-twice is | |
| 22:05:04 | dansmith | I wouldn't say so | |
| 22:05:05 | sean-k-mooney | so unless you take steps to put the vtpm data store on shared storeate i think we can assume it would be lost on evacuate in general | |
| 22:05:17 | efried | sean-k-mooney: definitely that | |
| 22:05:23 | dansmith | sean-k-mooney: which means unbootable instance if you put your FDE keys there | |
| 22:05:50 | dansmith | so in the flow chart, it means evacuate has a giant asterisk next to it :) | |
| 22:05:53 | sean-k-mooney | yes maybe although resuce could save you? | |
| 22:06:10 | dansmith | "we will reconstruct the smoking hull of an instance somewhere else where you know your data is in there, but it's unreadable" :) | |
| 22:06:16 | sean-k-mooney | e.g. allwo you to reinject the key if you still have it | |
| 22:06:27 | dansmith | sean-k-mooney: the whole point of the tpm is to not have to do that right? | |
| 22:06:37 | efried | not to be *able* to do that | |
| 22:06:43 | dansmith | secure enclave doesn't have the same meaning if you also have it on a post-it under your monitor | |
| 22:06:51 | sean-k-mooney | it has other services it can provide to the os but key storage is one of them yes | |
| 22:07:34 | efried | I guess I'll just write up the big asterisk in the spec & docs. | |
| 22:07:57 | dansmith | I'd like to see a list of all these special cases yeah | |
| 22:07:59 | efried | You would kinda have to make a new backup of the vtpm every time you write to it. | |
| 22:08:12 | dansmith | because if the list is too large, it starts to become not so useful | |
| 22:09:22 | sean-k-mooney | efried: you could make the same argument for the root disk i think that is out of scope of nova | |
| 22:09:40 | efried | sean-k-mooney: right, but the answer to "evacuate doesn't help you with ephemeral root disk" is "use volume/ceph". | |
| 22:10:09 | sean-k-mooney | and here its use barbican | |
| 22:10:10 | efried | with a vTPM the answer to "evacuate doesn't help you with ephemeral root disk" is "you better have a very recent snapshot of your vTPM" | |
| 22:10:15 | efried | no | |
| 22:10:24 | efried | barbican only stores the key used to unlock the file we've been discussing. | |
| 22:10:31 | efried | said file will be stored in swift | |
| 22:10:35 | efried | but only during snapshot | |
| 22:10:42 | sean-k-mooney | sure but as a user you can also store keys your self in barbican | |
| 22:10:49 | efried | how does that help? | |
| 22:10:57 | efried | you mean if you want to not use a vTPM at all? | |
| 22:11:14 | sean-k-mooney | use the tpm as a local secure cache fo the key | |
| 22:11:24 | sean-k-mooney | if you evacuate retive the backup form barbican | |
| 22:11:25 | efried | if barbican is secure enough for you, you would just use barbican. | |
| 22:11:43 | efried | though arguably you're counting on that ultimately here anyway. | |
| 22:11:45 | sean-k-mooney | you dont want to transfer it over the network everythime you boot | |
| 22:11:56 | efried | we're doing that anyway. | |
| 22:12:02 | sean-k-mooney | well ya | |
| 22:12:19 | sean-k-mooney | i was just going to say you have to do that to unlock the vtpm | |
| 22:12:27 | efried | Also I believe "VM talks to barbican" is a thing we're avoiding | |
| 22:12:36 | efried | limiting to "host talks to barbican" | |
| 22:13:10 | sean-k-mooney | barbican is a top level openstack sericve so worklaods can use it if they want too | |
| 22:13:16 | efried | if they want to, yes. | |
| 22:13:30 | efried | but it's the openstack user's credentials as opposed to $random_vm_user | |
| 22:13:46 | sean-k-mooney | oh yes that is what i ment by vm | |
| 22:13:48 | efried | and also I imagine there are reasons for the VM to be blocked from the keymgr service network-wise. | |
| 22:13:49 | sean-k-mooney | sorry | |
| 22:13:58 | sean-k-mooney | i ment application in the vm | |
| 22:15:28 | sean-k-mooney | anyway i think a resonable alternitive is if you need the keys to sruvice beyond the life time of the vm or be shared in a "sercure" way store them in barbican | |
| 22:15:57 | sean-k-mooney | if you you can tollerate losing the key in a evacuate case then your can store it soly in the tpm | |
| 22:16:44 | sean-k-mooney | dansmith: you primary concern is we loose the key used to encyrpt the os drive and loose data right if we loos the vtpm | |
| 22:16:52 | sean-k-mooney | wyoud ^ not solve that issue | |
| 22:17:10 | sean-k-mooney | *would | |
| 22:18:37 | efried | resize does a destroy and then a spawn? | |
| 22:19:02 | sean-k-mooney | it will redefine the domain but you dont loose data | |
| 22:19:11 | sean-k-mooney | and it may change host | |
| 22:20:03 | sean-k-mooney | e.g. if you resize form a flavor with 10G root disk to one with 20 it will grow the disk file | |
| 22:20:17 | sean-k-mooney | and if its libvirt the partion table and filesystem if it can | |
| 22:20:52 | sean-k-mooney | so we would have to copy the vtpm file if we move host | |
| 22:21:06 | efried | yeah, I'm exploring corner cases with the vTPM. If your resize adds/removes a vTPM where it was previously absent/present, that's fine. If both old & new flavor have the same version/model, we should carry the data. If they have a different version/model...? | |
| 22:21:07 | sean-k-mooney | if its to the same host we shoudl not need to do anything | |
| 22:21:39 | sean-k-mooney | if they are different models i would expect you to loose the contents of the tpm | |
| 22:21:52 | sean-k-mooney | unless we block that intentionally | |
| 22:22:08 | efried | yes to the first, the second is the question. | |
| 22:22:25 | efried | there's no way to persist the data, period. The versions/models are incompatible. | |
| 22:23:04 | sean-k-mooney | ya that is proably true or at least involved to do so i would not try in v1 of this | |
| 22:23:31 | sean-k-mooney | there might be a way to convert but im not aware of one off the top of my head | |
| 22:24:12 | efried | sean-k-mooney: not without introspecting, which ain't my business. | |
| 22:24:30 | sean-k-mooney | swtpm might have a fucntion to do it | |
| 22:24:49 | sean-k-mooney | but ya unless libvirt support it i think its out of scope | |
| 22:24:55 | efried | so I'm gonna say don't block it, cause who am I to say you didn't mean to do it | |
| 22:25:29 | sean-k-mooney | that would be my feeling too | |
| 22:25:51 | sean-k-mooney | we shoudl documetn it i guess | |
| 22:26:02 | efried | doing that now | |
| 22:26:21 | sean-k-mooney | but i dont think we shoudl guard in code against all pebcak errors | |
| 22:26:29 | efried | agreed | |
| 22:27:29 | efried | hmph, the thing about saving the swift ID in instance sysmeta has a problem code flow-wise. | |
| 22:27:48 | efried | because the driver is the thing that knows the vdev is in this particular file path | |
| 22:27:59 | efried | but the only thing we've asked the driver to do is snapshot. | |
| 22:28:19 | efried | and (I think) the driver can't tell if it's being asked to snapshot as part of a shelve or backup or... | |
| 22:28:47 | sean-k-mooney | am im not 100% sure about that last part | |
| 22:29:04 | efried | so I guess it could edit the sysmeta itself... but if it's not a shelve and the compute mgr is about to blow away the instance, the compute mgr needs a way to get that information so it can stuff it into the image meta. | |
| 22:29:13 | efried | I guess it can pull it from the instance | |
| 22:29:43 | efried | so the contract with the virt driver's snapshot method has to spell out where & how this info is stored in the sysmeta. | |
| 22:30:29 | sean-k-mooney | we create snapshot for shelve but i dont think the compute magener actully calls snapshot as part of that flow i think its donw by the driver | |
| 22:30:47 | efried | eh? | |
| 22:30:49 | sean-k-mooney | that said i could be way off base there so im checking | |
| 22:30:56 | efried | compute manager's shelve calls driver's snapshot | |
| 22:31:24 | sean-k-mooney | oh ok | |
| 22:31:31 | efried | https://opendev.org/openstack/nova/src/branch/master/nova/compute/manager.py#L5911 | |
| 22:32:32 | sean-k-mooney | ya your right i though we had a shelve fucntion on the driver interface that we called but this is handeled in the compute manager instead | |
| 22:33:30 | sean-k-mooney | so snapshot is passed the instace so it can get the flavor and image metadata | |
| 22:33:58 | sean-k-mooney | so the snapshot function can check if the isntace has a vtpm and directly update the system metadata as you said | |
| 22:34:14 | sean-k-mooney | oh right but it would not know if its a shleve or not | |
| 22:36:00 | sean-k-mooney | oh it can jsut check the instance.task_state right? | |
| 22:36:38 | efried | could, but compute mgr has to be able to peel it out anyway, so it's simpler if snapshot just always behaves the same and compute mgr pulls the info out of the sysmeta if it needs it. | |
| 22:37:14 | sean-k-mooney | ok | |
| 22:37:53 | sean-k-mooney | anyway night all o/ | |