| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-12-04 | |||
| 12:47:12 | gibi | efried: opened the spec. Let's see how ugly is the snapshot part | |
| 12:55:57 | efried | gibi: I can tldr it for you: | |
| 12:55:57 | efried | The vTPM belongs to the instance, not the image. So you can't do things like createImage to clone the VM and expect the vTPM to be cloned. (IMO this is sensible. Don't copy my security stuff around, please.) | |
| 12:57:03 | gibi | yeah that seems reasonable. Do not leak secrete data between clones | |
| 12:57:38 | efried | Similarly, if you do a rebuild, you keep your vTPM; you don't get whatever came from the new image. | |
| 12:58:14 | efried | Whatever's in the image might be expecting a given vTPM state that won't exist, but that's up to you to get right. | |
| 12:58:25 | efried | In particular, we *do* save the vTPM state when you do createBackup | |
| 12:59:48 | efried | So if you rebuild from that backup, we'll restore the vTPM as it existed at the point when you did the backup, which ought to maximize the chance that it's in the state the VM is expecting. | |
| 13:01:15 | gibi | Can backups be used to clone a VM ? | |
| 13:01:56 | efried | theoretically, yes, they do almost the same thing as createImage. | |
| 13:02:22 | efried | Documentation would be the key here: If you're backing up, use createBackup; if you're cloning, use createImage. | |
| 13:02:35 | efried | Which is what ops should be doing anyway, but I'm sure those get abused frequently today. | |
| 13:03:44 | efried | I think createBackup tries to maintain one image at a time, deleting the previous one any time you create a new one. Otherwise it's not different than createImage. | |
| 13:03:55 | gibi | I see | |
| 13:05:33 | efried | So the other two main quirks of concern are: | |
| 13:05:33 | efried | - Evacuate doesn't restore your vTPM, sorry. | |
| 13:05:33 | efried | - You have to do stuff as the instance owner. (IMO this is also good: my vTPM is secure stuff, I don't even want the admin to have access to it.) | |
| 13:06:35 | gibi | doesn't shared disk + evacuate could keep the vTMP file? or the vTPM file is somewhere far from the instance disk? | |
| 13:06:43 | efried | the latter :( | |
| 13:07:03 | efried | the vTPM is maintained by libvirt | |
| 13:07:10 | efried | not in the instance | |
| 13:07:13 | gibi | OK so it is some obscure location that the deployer won't share between compute hosts | |
| 13:07:17 | efried | well | |
| 13:07:36 | gibi | anyhow this could be a future improvement in libvirt + nova | |
| 13:07:39 | efried | the location is well known (though I'm not sure there's a contract that it won't change at some point) | |
| 13:07:51 | efried | theoretically it may be possible for that directory to be shared | |
| 13:07:55 | gibi | ack | |
| 13:08:13 | efried | but that would be up to the deployer to figure out, and I'm not going to do anything in nova to try to support (or prevent) it. | |
| 13:08:20 | gibi | sure | |
| 13:08:23 | gibi | make sense | |
| 13:08:53 | gibi | when you say "You have to do stuff as the instance owner" you mean nova needs to act in behalf of the instance owner to manage the vTPM or what is the stuff there? | |
| 13:09:59 | efried | correct | |
| 13:10:39 | efried | Nova needs to unlock the vTPM by defining a secret to libvirt when the guest is created | |
| 13:10:57 | efried | and, because security, the secret is owned by the instance owner; even the admin can't get at it. | |
| 13:11:45 | gibi | does it mean that the admin cannot delet the VM either? | |
| 13:11:51 | efried | Probably the most awkward of these areas is if you reboot the hypervisor, your VMs won't boot until the owner of the VM goes to boot them. | |
| 13:12:01 | efried | Hm, interesting question. | |
| 13:12:21 | efried | The admin won't be able to delete the secret in barbican, or the backup in swift | |
| 13:12:30 | efried | but we don't have to make those error conditions. | |
| 13:12:50 | efried | So the instance could still be destroyed, but those artifacts would be leaked in that case and the owner would have to go clean them up manually. | |
| 13:12:52 | gibi | but then it means we leave junk around | |
| 13:13:25 | efried | I'm not too sure how barbican/swift work in that regard. Maybe policy allows admins to delete things even if they don't own them. I guess that would make sense. | |
| 13:13:57 | gibi | can we have some way out of this. I accep that the admin cannot read or write the vTPM data but cannot even delete when the user is potentially long gone? | |
| 13:14:12 | efried | yeah, I don't know that answer. | |
| 13:14:37 | gibi | ok, noting it down in the spec as question | |
| 13:14:38 | efried | Note that the folks asking for this (VMG) aren't actually using barbican, they have their own home-grown key manager. | |
| 13:15:27 | efried | So I imagine they would be able to control that aspect of things. | |
| 13:16:19 | efried | I can find out whether swift has a similar policy capability. Thinking about it, I would imagine they do. It would only make sense that the admin, who could theoretically shut the whole thing down, should have the ability to delete individual bits, as long as they can't "see" them while they're doing it. | |
| 13:17:23 | gibi | ack, that could be a way out | |
| 13:18:17 | efried | Okay, so given all of that, do you think it is reasonable to support this thing, with proper documentation of these quirks? | |
| 13:18:51 | gibi | the next step that the admin does is draining a compute to do some maintenance. I see that live migration works. that is good. It would be nice to have a way to cold migrate a VM even if it loses the vTPM data in the process | |
| 13:19:18 | gibi | efried: I'm hesitant, as this is raw at the edges | |
| 13:19:36 | gibi | let me go through the spec. | |
| 13:19:41 | efried | cold migrate works | |
| 13:19:45 | gibi | ohh | |
| 13:19:47 | gibi | coolio | |
| 13:20:04 | efried | and shelve/unshelve (which uses snapshots) so cross-cell resize also works. | |
| 13:21:34 | gibi | OK. let me read the spec, but it seems workable at least. And an admin can only disable the feature by not configuring it in the nova conf | |
| 13:21:44 | gibi | s/only/always/ | |
| 13:22:58 | efried | yup | |
| 13:23:04 | efried | thanks gibi | |
| 13:23:19 | gibi | efried: have fun! | |
| 13:23:23 | efried | enjoy your doc :) | |
| 13:23:33 | gibi | ... a long queue ... | |
| 13:24:06 | gibi | it is the 4th hour I'm queueing... | |
| 13:49:23 | efried | egads | |
| 13:49:39 | efried | Does Hungary have a national health service like the UK? | |
| 13:50:26 | gibi | yes, this is the type of service everyone gets automatically in Hungary | |
| 13:50:43 | efried | I remember the UK NHS being notoriously slow like that. | |
| 13:51:00 | sean-k-mooney | slow but free | |
| 13:51:14 | sean-k-mooney | and good once you are seen | |
| 13:52:02 | gibi | I have a private issurance but it does not cover this type of follow up I have to do after my surgery I had the beginning of this year | |
| 13:52:06 | gibi | so I sit and wait | |
| 13:53:16 | mriedem | efried: i've invalidated https://bugs.launchpad.net/nova/+bug/1855015 since the jobs i'm looking at aren't failures for the most part | |
| 13:53:16 | openstack | Launchpad bug 1855015 in OpenStack Compute (nova) "Intermittent fails in nova-next job with "Multiple possible networks found, use a Network ID to be more specific."" [Undecided,Invalid] | |
| 13:53:16 | sean-k-mooney | efried: even without deleting the barbican key they can still do a vm delete right | |
| 13:53:34 | mriedem | efried: there is a tempest test which is triggering the error in the api logs but the test handles the 400 response | |
| 13:53:44 | sean-k-mooney | and ya most service allow admins to delete stuff even if they dont own it so i suspect barbican and swift can do the same | |
| 13:54:25 | gibi | efried, mriedem there is a fix for the tempest multiple network conflict https://review.opendev.org/#/c/682964 | |
| 13:54:33 | efried | good lord son, you posted two more comments and changed status while I was writing my response. | |
| 13:55:08 | mriedem | gotta be quick | |
| 13:55:10 | efried | gibi: thanks, they told me about that in -neutron | |
| 13:55:19 | efried | mriedem: okay if I reopen and mark as dup? | |
| 13:56:01 | openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove use of legacy 'FloatingIP' object https://review.opendev.org/697153 | |
| 13:56:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove use of legacy 'Network' object https://review.opendev.org/697154 | |
| 13:56:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove use of legacy 'SecurityGroup' object https://review.opendev.org/697155 | |
| 13:56:03 | openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove unused nova-network objects https://review.opendev.org/697156 | |
| 13:56:03 | openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Follow-up for I12a96ea659ed402cc4d1bd52a50e2e16042b6372 https://review.opendev.org/697252 | |
| 13:56:14 | efried | mriedem: unclear why it would be invalid. It caused at least three patches to fail that I was looking directly at. The rest I didn't check explicitly, but no reason to believe it would be different. | |
| 13:57:21 | mriedem | dupe against what? | |
| 13:57:35 | mriedem | https://bugs.launchpad.net/tempest/+bug/1844568 ? | |
| 13:57:35 | openstack | Launchpad bug 1844568 in tempest "[compute] "create_test_server" if networks is undefined and more than one network is present" [Undecided,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | |
| 13:57:44 | efried | yes | |
| 13:57:51 | mriedem | sure go ahead | |
| 13:58:51 | sean-k-mooney | stephenfin: has your changes to _wait_for_state_change already merged | |
| 13:58:53 | gibi | efried: comments in https://review.opendev.org/#/c/686804 | |
| 13:59:08 | stephenfin | sean-k-mooney: yes | |
| 13:59:30 | sean-k-mooney | ok so i can drop the new class in that case but it means i have to do the rewite in the backport now | |
| 13:59:40 | efried | stephenfin: did you see vTPM conversation from earlier? | |
| 13:59:46 | stephenfin | I did not | |