| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-01 | |||
| 20:32:58 | efried | okay, how does the master secret get into memory? | |
| 20:33:07 | jroll | via virSecretSetValue | |
| 20:33:13 | jroll | per https://libvirt.org/formatsecret.html#vTPMUsageType | |
| 20:33:25 | efried | ...with a value or file that comes from where? | |
| 20:33:37 | jroll | that's the TODO in this spec that I don't have an answer to | |
| 20:33:41 | jroll | :) | |
| 20:34:28 | efried | - Host boots | |
| 20:34:28 | efried | Because I don't expect a flow like | |
| 20:34:29 | efried | to be acceptable in general | |
| 20:34:29 | efried | - now your VMs can start | |
| 20:34:29 | efried | - Operator intervention is required to type in a password or load up a file | |
| 20:34:47 | jroll | I agree | |
| 20:35:12 | jroll | so we'd have to have something in the instance database record, which is either a secret, or a reference to a secret | |
| 20:35:25 | jroll | ideally the secret would be in barbican or equivalent | |
| 20:35:55 | jroll | and that secret could be fetched by nova-compute as needed | |
| 20:36:26 | efried | and you're allowed to fetch it by virtue of... being an authed user? | |
| 20:37:17 | jroll | I would assume by being the nova user | |
| 20:37:35 | jroll | or having access to the rabbit bus, if we had nova-compute ask nova-conductor for it | |
| 20:38:07 | efried | <red flag> upcall </red flag> | |
| 20:38:20 | jroll | heh | |
| 20:38:40 | efried | I was looking over some stuff about trusted certs for image signatures | |
| 20:38:57 | jroll | do computes not ever ask the conductor for info? what about all the database reads? | |
| 20:39:03 | efried | https://docs.openstack.org/nova/latest/user/certificate-validation.html | |
| 20:39:24 | efried | jroll: at least in theory, traffic only ever flows one way, from the conductor down to the compute | |
| 20:39:52 | efried | There may be one upcall left over, I can't remember, but we've been working hard to purge them and not allow any new ones. | |
| 20:40:12 | jroll | erm, how does a compute read from the database? | |
| 20:40:28 | efried | I think there are two databases | |
| 20:40:30 | jroll | it isn't an RPC upcall, I get that, but it's still a call to the conductor, just buried in o.vo | |
| 20:40:56 | efried | I'm not real well versed on this stuff I'm afraid | |
| 20:41:04 | umbSublime | I'm not sure that would apply, but say the secret were stored in barbican couldn't the compute host query barbican by means of the service_token ? | |
| 20:41:04 | jroll | ok | |
| 20:41:09 | jroll | just sayin': https://github.com/openstack/nova/blob/master/nova/cmd/compute.py#L53 | |
| 20:41:24 | jroll | umbSublime: probably, yeah | |
| 20:42:33 | efried | jroll: I don't know how relevant the certificate validation thing is; I was just looking at how it tries to bootstrap secrets. | |
| 20:43:41 | efried | unfortunately it seems to have added new CLI/API fields -- that would be the "specify a secret to nova boot" route. I would hate to have to do that because it blows up the feature scope significantly. | |
| 20:43:55 | jroll | right | |
| 20:44:10 | efried | without that, we like add a new extra spec and parlay it into xml and we're done. | |
| 20:44:27 | efried | with it, we've got microversions and client/CLI enhancements and on and on. | |
| 20:44:31 | jroll | it looks similar to how TPM secret storage could work, in that nova-compute has to fetch a secret from barbican to boot the instance | |
| 20:44:55 | jroll | in the image validation case, the user provides the secret reference, because the user signs the image | |
| 20:45:13 | jroll | in the vTPM case, only nova needs to know the secret, not the user, so it can handle everything internally | |
| 20:45:34 | efried | But how does nova know the secret? | |
| 20:45:42 | efried | In a way that doesn't allow you to get at it if you steal the disk? | |
| 20:45:47 | jroll | nova could generate the secret for a TPM at instance create time, put it in barbican, and then pull it later as needed (migration, etc) | |
| 20:46:59 | efried | Okay. I guess I should go figure out how barbican works. Hopefully VMG's homegrown key manager equivalent responds the same way as barbican/vault does? | |
| 20:47:14 | jroll | close enough | |
| 20:47:18 | efried | because penick said something about using that in place of barbican | |
| 20:47:27 | jroll | if we do this all in castellan, we can make a castellan backend for our thing | |
| 20:47:32 | efried | ...yeah | |
| 20:47:35 | jroll | seems like the right thing to do anyway | |
| 20:47:57 | efried | right, iiuc nova requires whatever key manager to be brokered/brokerable through castellan. | |
| 20:48:07 | jroll | mhm | |
| 20:50:07 | jroll | efried: anything else I can help with? | |
| 20:50:07 | efried | Now does the tpm show up to the VM user as virginal? | |
| 20:50:07 | efried | - nova boots instance with <encryption secret='$secret_uuid'> | |
| 20:50:07 | efried | - nova uses secret to virSecretSetValue to get $secret_uuid | |
| 20:50:07 | efried | - nova creates a new secret in $backend via castellan | |
| 20:50:07 | efried | okay, so | |
| 20:50:22 | jroll | I assume just like a new TPM | |
| 20:50:24 | jroll | or hope? | |
| 20:50:34 | efried | And the VM user bootstraps the tpm with his own secret, which only he knows | |
| 20:50:40 | jroll | that is a question for a qemu or swtpm person, I guess | |
| 20:50:42 | jroll | right | |
| 20:51:07 | efried | so nova ("the nova user") has to be the root of trust | |
| 20:51:16 | efried | in this scenario | |
| 20:52:08 | jroll | I guess by some definition, yes | |
| 20:52:15 | jroll | it always comes down to the "hardware" supplier :) | |
| 20:52:18 | efried | because at any time "the nova user" could go grab that secret, decrypt the vtpm, and get at the VM user's secret | |
| 20:52:37 | jroll | right | |
| 20:52:47 | efried | okay, so "walk away with disk" is a requirement, but "compromised root" is not. | |
| 20:52:49 | jroll | assuming the nova user has the encrypted vtpm state | |
| 20:52:59 | efried | yeah, ability to read memory or whatever. | |
| 20:53:17 | jroll | yeah, I think that's something we just have to live with, in an emulated world | |
| 20:53:30 | jroll | if you've root on the hypervisor you can just go ahead and read the VM's memory anyway, right | |
| 20:53:42 | efried | I guess? | |
| 20:54:04 | efried | Okay, thanks for the talk jroll. If our assumptions are correct, I think I can work with this. | |
| 20:54:17 | jroll | efried: you're welcome, I hope they are correct :) | |
| 20:54:37 | efried | penick said he was going to get "security people" to vet things, but it would be a while before they were available. | |
| 20:54:58 | jroll | ¯\_(ツ)_/¯ | |
| 20:55:00 | efried | I'll at least write up spec content under these assumptions, so that they can have something to point at and say it's wrong, rather than just a void. | |
| 20:55:07 | jroll | I forgot about this thing for the last couple months, until today | |
| 20:56:51 | openstackgerrit | Merged openstack/nova master: Switch to devstack-plugin-ceph-tempest-py3 for ceph https://review.opendev.org/691765 | |
| 21:34:48 | openstackgerrit | Merged openstack/nova master: Avoid error 500 on shelve task_state race https://review.opendev.org/692206 | |
| 21:38:33 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Convert legacy nova-live-migration and nova-multinode-grenade to py3 https://review.opendev.org/692374 | |
| 22:50:16 | openstack | bug 1824435 in OpenStack Compute (nova) train "fill_virtual_interface_list migration fails on second attempt" [Medium,In progress] https://launchpad.net/bugs/1824435 - Assigned to melanie witt (melwitt) | |
| 22:50:16 | openstackgerrit | Merged openstack/nova stable/train: Add regression test for bug 1824435 https://review.opendev.org/691402 | |
| 22:50:20 | openstackgerrit | Merged openstack/nova stable/queens: libvirt: Ignore volume exceptions during post_live_migration https://review.opendev.org/691284 | |
| 23:50:47 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/train: Avoid error 500 on shelve task_state race https://review.opendev.org/692628 | |
| #openstack-nova - 2019-11-02 | |||
| 00:09:38 | openstackgerrit | Merged openstack/nova master: Add new policy rule for viewing host status UNKNOWN https://review.opendev.org/679181 | |
| 00:16:19 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/stein: Avoid error 500 on shelve task_state race https://review.opendev.org/692630 | |
| 00:24:48 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: Avoid error 500 on shelve task_state race https://review.opendev.org/692631 | |
| 00:30:29 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/queens: Avoid error 500 on shelve task_state race https://review.opendev.org/692632 | |
| 01:19:44 | openstackgerrit | Merged openstack/nova master: Add Aggregate image caching progress notifications https://review.opendev.org/691390 | |
| 02:34:33 | henriqueof | Can someone please help | |
| 02:35:05 | henriqueof | Just got various randon instance at paused energy state and can't get them back uop. | |
| 02:35:49 | henriqueof | Whenever I try to force reboot nova-comput logs just says : Instance is paused unexpectedly. Ignore. | |
| 02:51:37 | henriqueof | Joust found the reason, my storage went on emergency mode and locked any IO, looks like it is fixed now. | |
| 05:58:57 | openstackgerrit | Merged openstack/nova master: Nova compute: add in log exception to help debug failures https://review.opendev.org/669545 | |
| 12:23:38 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/train: Nova compute: add in log exception to help debug failures https://review.opendev.org/692662 | |
| 17:10:54 | openstackgerrit | Merged openstack/nova stable/train: Remove redundant call to get/create default security group https://review.opendev.org/691403 | |