| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-20 | |||
| 15:31:01 | dansmith | that opens up an attack vector | |
| 15:31:17 | efried | on unshelve, I need to make sure that the instance I'm unshelving is $instance_uuid, and I also need to suck in the vtpm image with that $instance_Uuid | |
| 15:31:43 | dansmith | if I can create an image, then I could create a tpm image for your instance and when you go to unshelve, you use my tpm instead of nothing beause you didn't have one before, so I can inject keys and get you to use them potentially | |
| 15:31:48 | sean-k-mooney | dansmith: because the uuid can be updated | |
| 15:32:07 | sean-k-mooney | we likely need to sotre this nova side to be safe | |
| 15:32:12 | dansmith | if you're going to store the image in a glance image, storing the id of that in sysmeta would be much stronger a link I think | |
| 15:32:23 | sean-k-mooney | ya that is what i was thinking | |
| 15:32:36 | dansmith | and maybe the sha256 of it too | |
| 15:32:51 | efried | dansmith: Seems like that attack vector is open anyway, just narrower... oh, yeah, a signature ought to close it. | |
| 15:33:19 | dansmith | efried: it means you have to have the ability to modify the internals of nova and create a malicious image.. it's much narrower | |
| 15:33:43 | dansmith | creating images is done via the api.. sysmeta is only at the db layer so requires a much larger exploit | |
| 15:34:17 | sean-k-mooney | ya also if the tpm is encypted | |
| 15:34:21 | efried | dansmith: also keep in mind the file is useless without... yeah. | |
| 15:34:29 | dansmith | note that I don't *know* that abusing glance in this way is legit | |
| 15:34:32 | sean-k-mooney | you not only need to replace the iamge it has to be encypted with teh right key | |
| 15:34:47 | dansmith | efried: it's not useless, it's still a dictionary attack vector, but I understand | |
| 15:35:17 | dansmith | sean-k-mooney: right, but the key is get-able from barbican via the api | |
| 15:35:21 | efried | dansmith: so the instance record sticks around when offloaded? | |
| 15:35:29 | dansmith | efried: that's the whole point of shelve | |
| 15:35:35 | efried | okay | |
| 15:35:56 | sean-k-mooney | true | |
| 15:35:58 | efried | keep the instance record, blow away the allocations etc. | |
| 15:36:13 | efried | and I assume we can unshelve onto any host, doesn't have to be the original. | |
| 15:36:22 | sean-k-mooney | yes | |
| 15:36:28 | efried | (within the bounds of scheduler restrictions of course) | |
| 15:36:30 | sean-k-mooney | we go to the schduler to select the hosts | |
| 15:36:39 | dansmith | likely not the original | |
| 15:36:56 | dansmith | shelve was intended as "cold storage" so some significant amount of time would have passed before you unshelve, so very unlikely you land on the same host | |
| 15:37:20 | dansmith | I will not always be used that way, but.. can't make any assumptions | |
| 15:37:24 | efried | okay, cool, I'm going to write this up and let the stakeholders know we've doubled the complexity again. | |
| 15:37:52 | efried | And maybe go ask -glance for recommendations on how to store the thingy. | |
| 15:38:03 | dansmith | efried: we might ask jroll and the Verizon Media (tm) people what they think about storing the tpm in glance | |
| 15:38:06 | dansmith | security-wise | |
| 15:38:25 | efried | Right. | |
| 15:38:47 | efried | Thanks for the talk dansmith sean-k-mooney | |
| 15:39:01 | sean-k-mooney | the other usecase for shelve is public cloud to avoid billing but ya unell you have a server affinity group or somethin its very unlikly to land on the same host | |
| 15:39:08 | dansmith | efried: already been asking some questions in -glance | |
| 15:39:18 | sean-k-mooney | no worres i need to review your spec and code too | |
| 15:39:25 | sean-k-mooney | ill try to do that this week | |
| 15:40:06 | efried | cool | |
| 15:46:01 | bauzas | mriedem: want me to care on backporting https://review.opendev.org/#/c/695145/ to stable branches ? | |
| 15:46:13 | bauzas | or you rather and me reviewing ? | |
| 15:46:29 | mriedem | can you do both? | |
| 15:46:51 | bauzas | both backporting and reviewing ? well, is that a thing ? | |
| 15:47:03 | bauzas | we have elod and lyarwood we can bug | |
| 15:47:51 | mriedem | i have things in stable/train you could review | |
| 15:48:12 | bauzas | mriedem: sure thing | |
| 15:48:21 | mriedem | i guess i don't know if you're just talking about that specific change or reviewing code in general | |
| 15:48:29 | mriedem | because both would be good | |
| 15:48:38 | bauzas | mriedem: no, sorry, was talking of proposing backports of this change | |
| 15:49:01 | bauzas | mriedem: cool, I'm committed to reviewing for the last 2 days | |
| 15:49:28 | bauzas | anyway, let's just backport this stuff... | |
| 15:57:09 | dansmith | jroll: you around? | |
| 15:58:54 | efried | I suspect in this case it would be possible to use the keymgr backend to store this blob. But that's really not a good general model. | |
| 15:59:17 | efried | this is really a job for swift. And then the responsibility of the admin to make sure their swift is appropriately hardened. | |
| 16:00:18 | efried | do we have a precedent for nova talking to swift?? | |
| 16:00:24 | efried | I... don't see one. | |
| 16:00:40 | dansmith | so, if you have no swift, we what.. lose your tpm on snapshot/shelve or refuse to do it? | |
| 16:00:50 | efried | yeah, I would think so. | |
| 16:00:55 | dansmith | which? | |
| 16:01:04 | efried | refuse | |
| 16:01:22 | dansmith | how about instead of that, | |
| 16:01:32 | efried | I mean, there seems to have been plenty of talk on recent features about "not supported" (you can try it but it won't work right) versus actually blocked | |
| 16:01:36 | dansmith | we refuse to ever boot an instance with a tpm if there's no swift endpoint in the catalog? | |
| 16:01:46 | dansmith | efried: yeah and they all suck | |
| 16:02:02 | efried | dansmith: I can live with that; same as the requirement for the keymgr to exist. | |
| 16:02:04 | dansmith | efried: because the user doesn't know any of this, other than that "every openstack cloud they use seems to be different and incompatible" | |
| 16:02:20 | fgonzales3246 | hello all, I'm configuring SSL connections into my MySQL database for the components and when I activate this into the dabase and in each connection my controller starts to slow down and the requests fail due to timeout. | |
| 16:02:23 | dansmith | efried: ack, I'd much rather do that, or at least do that in addition to the other | |
| 16:02:52 | efried | dansmith: IMO it's better to block operations you know are broken than to simply state they're not supported and let them sh*t all over themselves if tried. | |
| 16:02:59 | fgonzales3246 | I tried to isolate one component like, cinder, but it occurs the same... seems that the requests doesn't go to the compute node | |
| 16:03:06 | efried | but I think the counterargument has been that we want to be able to enable the operation without needing a new microversion. | |
| 16:03:32 | fgonzales3246 | but the services are all up and running, do you know what could I be missing? Something related to nova-conductor or something else? | |
| 16:03:38 | dansmith | efried: of course, I'm saying having "the plan" be a thousand places where seemingly fundamental operations can fail with the appropriate combination of obscure backend configs sucks from the user's perspective | |
| 16:03:40 | efried | anyway, in this case we're not talking API changes, and we're talking blocking at boot rather than (well, in addition to) shelve, so I'm good. | |
| 16:03:51 | efried | dansmith: totally agree. | |
| 16:03:56 | efried | block it if you know it's broken. | |
| 16:04:09 | dansmith | ...but try not to rely on that in design | |
| 16:04:27 | efried | now I just hope I can detect a swift | |
| 16:04:46 | efried | oh, | |
| 16:05:03 | efried | cause I wanted to add a check for "key manager configured" to the "do we enable vTPM in the first place" logic | |
| 16:05:11 | efried | but couldn't find a reliable way to inspect CONF for that | |
| 16:05:20 | dansmith | you need the catalog, no? | |
| 16:05:24 | efried | but yeah, | |
| 16:05:36 | efried | I can at least look in the catalog to make sure there's a keymgr endpoint. | |
| 16:05:40 | efried | um | |
| 16:05:59 | efried | except I'm doing this on init_host and don't have a context? | |
| 16:06:05 | dansmith | no, | |
| 16:06:10 | dansmith | you do this at boot time for the instance | |
| 16:06:17 | dansmith | non-tpm instances don't care about this | |
| 16:06:37 | dansmith | what you want to avoid is agreeing to create an instance for the user which in six weeks will be unsnapshottable | |
| 16:06:44 | efried | Sorry, I'm talking about something slightly different: the part where the libvirt driver decides whether to set the "can I vTPM?" flag. | |
| 16:06:53 | dansmith | I'm not talking about that | |
| 16:06:55 | efried | ...causing it to expose the traits | |
| 16:06:57 | efried | I know, I am. | |
| 16:07:02 | efried | I understand and agree with what you're talking about. | |
| 16:07:16 | dansmith | that too may be good, but expecting keystone is up and reachable when compute starts is not okay I think | |
| 16:07:33 | efried | orly? ... okay. | |
| 16:07:34 | dansmith | computes on the other side of a network partition shouldn't just fail to start I think | |
| 16:07:42 | efried | fair enough | |