Posted	Nick	Remark
#openstack-nova - 2019-11-20
15:25:20	dansmith	is that for things like glance multi-store, you have to know that those things are linked together when you move/copy between stores
15:25:28	dansmith	and,
15:25:39	dansmith	if you use that snapshot to spawn to a ceph-using compute node,
15:25:45	dansmith	what do you do with that tpm image?
15:25:50	dansmith	copy to everyone? ignore?
15:26:00	sean-k-mooney	efried: right we would need to snapshot it and store it somewhere
15:26:09	dansmith	"copy to everyone" meaning.. any instance that CoWs from that should get a copy of the tpm
15:26:47	efried	good question, I hope if you're using a snapshot to create multiple VMs from, you would want them to start with a clean TPM, but mebbe not.
15:26:58	dansmith	efried: but how do you know?
15:27:08	dansmith	efried: if I snapshot my instance and delete it, then come back a year later for black friday,
15:27:10	sean-k-mooney	dansmith: by the way this topic in general sound more like the goal of glare to solve. e.g. a generic artifact store
15:27:15	dansmith	you don't know if I'm going to create one or a hundred of those
15:27:18	sean-k-mooney	but i dont think glare is a thing anymore
15:27:34	efried	dansmith: your point is that we can't differentiate between shelve-offload-unshelve and snapshot-restore?
15:27:38	dansmith	sean-k-mooney: maybe, not sure glare was really for keeping this kind of thing, vs things like template configs.. but maybe
15:28:12	dansmith	efried: no we definitely can, I'm saying snapshot-restore has a similar problem if you store the thing in glance or otherwise link it via the snapshot itself, vs. the image record in nova
15:28:26	sean-k-mooney	so i guess there are different usecase but i would not assuem that we want to share tpm snapshot betwen multipel instnaces
15:29:01	sean-k-mooney	that said i can see a use case for that where i store secure boot keys or something in the tpm snapshot
15:29:04	dansmith	sean-k-mooney: agree
15:29:11	efried	sean-k-mooney: I think dansmith is saying that, even if we put that stake in the ground, we can't necessarily know when we go to use a snapshot
15:29:21	dansmith	right,
15:29:27	sean-k-mooney	ya
15:29:36	efried	but dansmith I would think we would only ever restore the vTPM if the operation is unshelve. If it's just "boot with this image" we don't.
15:29:43	dansmith	so it would be better if we have a solution where we the linkage is between the instance and the tpm, not the snapshot image and the tpm
15:30:15	efried	yeah, that works. I was actually thinking to use the instance UUID as the link
15:30:27	dansmith	eh?
15:30:34	openstackgerrit	Matt Riedemann proposed openstack/nova master: PoC for using COMPUTE_SAME_HOST_COLD_MIGRATE https://review.opendev.org/695220
15:30:45	efried	in the metadata of both images, store vtpm_original_instance_uuid=$instance_uuid
15:30:54	dansmith	that's a bad idea, IMHO
15:31:01	dansmith	that opens up an attack vector
15:31:17	efried	on unshelve, I need to make sure that the instance I'm unshelving is $instance_uuid, and I also need to suck in the vtpm image with that $instance_Uuid
15:31:43	dansmith	if I can create an image, then I could create a tpm image for your instance and when you go to unshelve, you use my tpm instead of nothing beause you didn't have one before, so I can inject keys and get you to use them potentially
15:31:48	sean-k-mooney	dansmith: because the uuid can be updated
15:32:07	sean-k-mooney	we likely need to sotre this nova side to be safe
15:32:12	dansmith	if you're going to store the image in a glance image, storing the id of that in sysmeta would be much stronger a link I think
15:32:23	sean-k-mooney	ya that is what i was thinking
15:32:36	dansmith	and maybe the sha256 of it too
15:32:51	efried	dansmith: Seems like that attack vector is open anyway, just narrower... oh, yeah, a signature ought to close it.
15:33:19	dansmith	efried: it means you have to have the ability to modify the internals of nova and create a malicious image.. it's much narrower
15:33:43	dansmith	creating images is done via the api.. sysmeta is only at the db layer so requires a much larger exploit
15:34:17	sean-k-mooney	ya also if the tpm is encypted
15:34:21	efried	dansmith: also keep in mind the file is useless without... yeah.
15:34:29	dansmith	note that I don't *know* that abusing glance in this way is legit
15:34:32	sean-k-mooney	you not only need to replace the iamge it has to be encypted with teh right key
15:34:47	dansmith	efried: it's not useless, it's still a dictionary attack vector, but I understand
15:35:17	dansmith	sean-k-mooney: right, but the key is get-able from barbican via the api
15:35:21	efried	dansmith: so the instance record sticks around when offloaded?
15:35:29	dansmith	efried: that's the whole point of shelve
15:35:35	efried	okay
15:35:56	sean-k-mooney	true
15:35:58	efried	keep the instance record, blow away the allocations etc.
15:36:13	efried	and I assume we can unshelve onto any host, doesn't have to be the original.
15:36:22	sean-k-mooney	yes
15:36:28	efried	(within the bounds of scheduler restrictions of course)
15:36:30	sean-k-mooney	we go to the schduler to select the hosts
15:36:39	dansmith	likely not the original
15:36:56	dansmith	shelve was intended as "cold storage" so some significant amount of time would have passed before you unshelve, so very unlikely you land on the same host
15:37:20	dansmith	I will not always be used that way, but.. can't make any assumptions
15:37:24	efried	okay, cool, I'm going to write this up and let the stakeholders know we've doubled the complexity again.
15:37:52	efried	And maybe go ask -glance for recommendations on how to store the thingy.
15:38:03	dansmith	efried: we might ask jroll and the Verizon Media (tm) people what they think about storing the tpm in glance
15:38:06	dansmith	security-wise
15:38:25	efried	Right.
15:38:47	efried	Thanks for the talk dansmith sean-k-mooney
15:39:01	sean-k-mooney	the other usecase for shelve is public cloud to avoid billing but ya unell you have a server affinity group or somethin its very unlikly to land on the same host
15:39:08	dansmith	efried: already been asking some questions in -glance
15:39:18	sean-k-mooney	no worres i need to review your spec and code too
15:39:25	sean-k-mooney	ill try to do that this week
15:40:06	efried	cool
15:46:01	bauzas	mriedem: want me to care on backporting https://review.opendev.org/#/c/695145/ to stable branches ?
15:46:13	bauzas	or you rather and me reviewing ?
15:46:29	mriedem	can you do both?
15:46:51	bauzas	both backporting and reviewing ? well, is that a thing ?
15:47:03	bauzas	we have elod and lyarwood we can bug
15:47:51	mriedem	i have things in stable/train you could review
15:48:12	bauzas	mriedem: sure thing
15:48:21	mriedem	i guess i don't know if you're just talking about that specific change or reviewing code in general
15:48:29	mriedem	because both would be good
15:48:38	bauzas	mriedem: no, sorry, was talking of proposing backports of this change
15:49:01	bauzas	mriedem: cool, I'm committed to reviewing for the last 2 days
15:49:28	bauzas	anyway, let's just backport this stuff...
15:57:09	dansmith	jroll: you around?
15:58:54	efried	I suspect in this case it would be possible to use the keymgr backend to store this blob. But that's really not a good general model.
15:59:17	efried	this is really a job for swift. And then the responsibility of the admin to make sure their swift is appropriately hardened.
16:00:18	efried	do we have a precedent for nova talking to swift??
16:00:24	efried	I... don't see one.
16:00:40	dansmith	so, if you have no swift, we what.. lose your tpm on snapshot/shelve or refuse to do it?
16:00:50	efried	yeah, I would think so.
16:00:55	dansmith	which?
16:01:04	efried	refuse
16:01:22	dansmith	how about instead of that,
16:01:32	efried	I mean, there seems to have been plenty of talk on recent features about "not supported" (you can try it but it won't work right) versus actually blocked
16:01:36	dansmith	we refuse to ever boot an instance with a tpm if there's no swift endpoint in the catalog?
16:01:46	dansmith	efried: yeah and they all suck
16:02:02	efried	dansmith: I can live with that; same as the requirement for the keymgr to exist.
16:02:04	dansmith	efried: because the user doesn't know any of this, other than that "every openstack cloud they use seems to be different and incompatible"
16:02:20	fgonzales3246	hello all, I'm configuring SSL connections into my MySQL database for the components and when I activate this into the dabase and in each connection my controller starts to slow down and the requests fail due to timeout.
16:02:23	dansmith	efried: ack, I'd much rather do that, or at least do that in addition to the other
16:02:52	efried	dansmith: IMO it's better to block operations you know are broken than to simply state they're not supported and let them sh*t all over themselves if tried.
16:02:59	fgonzales3246	I tried to isolate one component like, cinder, but it occurs the same... seems that the requests doesn't go to the compute node

Earlier   Later