| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-01 | |||
| 20:24:35 | jroll | right | |
| 20:24:43 | jroll | so take the fact that's it's a tpm out of your head | |
| 20:24:51 | efried | hah, that's easy enough | |
| 20:24:56 | efried | I can't even spell tpm yet. | |
| 20:24:58 | jroll | basically, there's a file for the instance on the hypervisor's disk, which needs to be encrypted | |
| 20:25:05 | jroll | to encrypt a file, you need a secret | |
| 20:25:30 | efried | to your understanding, is the value of `secret` the actual secret, or is it an identifier for a secret that needs to be looked up? | |
| 20:25:35 | jroll | the 6dd3e4a5-1d76-44ce-961f-f119f5aad935 in your paste is a reference to a secret object stored in libvirt (i.e. virsh secret-set 6dd3e4a5-1d76-44ce-961f-f119f5aad935 myawesomesecret) | |
| 20:25:46 | efried | "a reference" <= ✔ | |
| 20:26:05 | jroll | see https://libvirt.org/formatsecret.html#vTPMUsageType | |
| 20:26:11 | efried | "stored in libvirt" -- from the host or from within the VM | |
| 20:26:13 | efried | ...okay... | |
| 20:26:20 | jroll | from the host, yeah | |
| 20:26:26 | jroll | the VM knows nothing about this file | |
| 20:26:27 | efried | oh, look, a vTPM usage man page, that's convenient. | |
| 20:26:30 | jroll | :D | |
| 20:26:50 | jroll | I found it from a libvirt page you linked in that spec :P | |
| 20:26:56 | efried | butbutbut... | |
| 20:27:10 | efried | I thought the whole point of this was that someone who walks away with the disk can't get at the TPM | |
| 20:27:31 | jroll | correct, because the file representing the TPM is encrypted | |
| 20:27:45 | efried | with a key that's available on the host tho? | |
| 20:27:47 | jroll | I *assume*, but have not verified, that this secret is stored in memory, not on disk | |
| 20:28:18 | jroll | which is why I mentioned we may want to call virSecretSetValue on every instance start() | |
| 20:28:42 | jroll | as if I'm correct about it being in-memory, a host reboot or instance migration would mean we need to set that secret again | |
| 20:28:54 | efried | I thought I read somewhere that it's in memory | |
| 20:28:57 | efried | https://github.com/qemu/qemu/blob/master/docs/specs/tpm.txt | |
| 20:29:08 | efried | L14 | |
| 20:29:14 | efried | and 31 | |
| 20:29:52 | jroll | The TIS interface makes a memory mapped IO region in the area 0xfed40000 - | |
| 20:29:54 | jroll | 0xfed44fff available to the guest operating system. | |
| 20:30:17 | jroll | I read that as it maps the decrypted version of the file on disk into memory, but I'm not good with terms at this level | |
| 20:31:12 | jroll | I thought I read somewhere that swtpm keeps the state on disk, but I don't see it now | |
| 20:31:20 | efried | "...reboot or ... migration ... set that secret again" | |
| 20:31:45 | jroll | so if swtpm keeps it in memory, and encrypted, that's even better | |
| 20:32:07 | efried | If the host has possession of the master secret, everything sounds easy. But it also sounds like it doesn't satisfy your use case, which is "must not be compromised if someone walks off with the disk". | |
| 20:32:36 | jroll | if the host keeps the master secret in memory only, that means it is not compromised if someone walks off with the disk | |
| 20:32:58 | efried | okay, how does the master secret get into memory? | |
| 20:33:07 | jroll | via virSecretSetValue | |
| 20:33:13 | jroll | per https://libvirt.org/formatsecret.html#vTPMUsageType | |
| 20:33:25 | efried | ...with a value or file that comes from where? | |
| 20:33:37 | jroll | that's the TODO in this spec that I don't have an answer to | |
| 20:33:41 | jroll | :) | |
| 20:34:28 | efried | - Host boots | |
| 20:34:28 | efried | Because I don't expect a flow like | |
| 20:34:29 | efried | to be acceptable in general | |
| 20:34:29 | efried | - now your VMs can start | |
| 20:34:29 | efried | - Operator intervention is required to type in a password or load up a file | |
| 20:34:47 | jroll | I agree | |
| 20:35:12 | jroll | so we'd have to have something in the instance database record, which is either a secret, or a reference to a secret | |
| 20:35:25 | jroll | ideally the secret would be in barbican or equivalent | |
| 20:35:55 | jroll | and that secret could be fetched by nova-compute as needed | |
| 20:36:26 | efried | and you're allowed to fetch it by virtue of... being an authed user? | |
| 20:37:17 | jroll | I would assume by being the nova user | |
| 20:37:35 | jroll | or having access to the rabbit bus, if we had nova-compute ask nova-conductor for it | |
| 20:38:07 | efried | <red flag> upcall </red flag> | |
| 20:38:20 | jroll | heh | |
| 20:38:40 | efried | I was looking over some stuff about trusted certs for image signatures | |
| 20:38:57 | jroll | do computes not ever ask the conductor for info? what about all the database reads? | |
| 20:39:03 | efried | https://docs.openstack.org/nova/latest/user/certificate-validation.html | |
| 20:39:24 | efried | jroll: at least in theory, traffic only ever flows one way, from the conductor down to the compute | |
| 20:39:52 | efried | There may be one upcall left over, I can't remember, but we've been working hard to purge them and not allow any new ones. | |
| 20:40:12 | jroll | erm, how does a compute read from the database? | |
| 20:40:28 | efried | I think there are two databases | |
| 20:40:30 | jroll | it isn't an RPC upcall, I get that, but it's still a call to the conductor, just buried in o.vo | |
| 20:40:56 | efried | I'm not real well versed on this stuff I'm afraid | |
| 20:41:04 | umbSublime | I'm not sure that would apply, but say the secret were stored in barbican couldn't the compute host query barbican by means of the service_token ? | |
| 20:41:04 | jroll | ok | |
| 20:41:09 | jroll | just sayin': https://github.com/openstack/nova/blob/master/nova/cmd/compute.py#L53 | |
| 20:41:24 | jroll | umbSublime: probably, yeah | |
| 20:42:33 | efried | jroll: I don't know how relevant the certificate validation thing is; I was just looking at how it tries to bootstrap secrets. | |
| 20:43:41 | efried | unfortunately it seems to have added new CLI/API fields -- that would be the "specify a secret to nova boot" route. I would hate to have to do that because it blows up the feature scope significantly. | |
| 20:43:55 | jroll | right | |
| 20:44:10 | efried | without that, we like add a new extra spec and parlay it into xml and we're done. | |
| 20:44:27 | efried | with it, we've got microversions and client/CLI enhancements and on and on. | |
| 20:44:31 | jroll | it looks similar to how TPM secret storage could work, in that nova-compute has to fetch a secret from barbican to boot the instance | |
| 20:44:55 | jroll | in the image validation case, the user provides the secret reference, because the user signs the image | |
| 20:45:13 | jroll | in the vTPM case, only nova needs to know the secret, not the user, so it can handle everything internally | |
| 20:45:34 | efried | But how does nova know the secret? | |
| 20:45:42 | efried | In a way that doesn't allow you to get at it if you steal the disk? | |
| 20:45:47 | jroll | nova could generate the secret for a TPM at instance create time, put it in barbican, and then pull it later as needed (migration, etc) | |
| 20:46:59 | efried | Okay. I guess I should go figure out how barbican works. Hopefully VMG's homegrown key manager equivalent responds the same way as barbican/vault does? | |
| 20:47:14 | jroll | close enough | |
| 20:47:18 | efried | because penick said something about using that in place of barbican | |
| 20:47:27 | jroll | if we do this all in castellan, we can make a castellan backend for our thing | |
| 20:47:32 | efried | ...yeah | |
| 20:47:35 | jroll | seems like the right thing to do anyway | |
| 20:47:57 | efried | right, iiuc nova requires whatever key manager to be brokered/brokerable through castellan. | |
| 20:48:07 | jroll | mhm | |
| 20:50:07 | jroll | efried: anything else I can help with? | |
| 20:50:07 | efried | Now does the tpm show up to the VM user as virginal? | |
| 20:50:07 | efried | - nova boots instance with <encryption secret='$secret_uuid'> | |
| 20:50:07 | efried | - nova uses secret to virSecretSetValue to get $secret_uuid | |
| 20:50:07 | efried | - nova creates a new secret in $backend via castellan | |
| 20:50:07 | efried | okay, so | |
| 20:50:22 | jroll | I assume just like a new TPM | |
| 20:50:24 | jroll | or hope? | |
| 20:50:34 | efried | And the VM user bootstraps the tpm with his own secret, which only he knows | |
| 20:50:40 | jroll | that is a question for a qemu or swtpm person, I guess | |
| 20:50:42 | jroll | right | |
| 20:51:07 | efried | so nova ("the nova user") has to be the root of trust | |