| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-20 | |||
| 15:09:19 | bauzas | efried: some other projets create their own RPs and we need to know them | |
| 15:09:20 | efried | and dustinc has mostly already coded it up | |
| 15:09:26 | efried | so it's really just a question of test surface | |
| 15:09:34 | mnaser | memory channels onto the Infinity Fabric to progress through the SoC. To achieve the lowest latency, you can set the memory frequency to be equal to the Infinity Fabric speed. Lowering the memory clock speed also results in power savings in the memory controller, thus allowing the rest of the SoC to consume more power potentially resulting in a performance boost elsewhere, depending on the workload." | |
| 15:09:34 | mnaser | sean-k-mooney: "By default, the BIOS for EPYC 7002 Series processors will run at the maximum allowable clock frequency by the platform. This configuration results in the maximum memory bandwidth for the processor, but in some cases, it may not be the lowest latency. The Infinity Fabric will have a maximum speed of 1467 MHz (lower in some platforms), resulting in a single clock penalty to transfer data from the | |
| 15:09:50 | bauzas | efried: for the moment, we infer their existences based on naming convention but I'm all up for other ideas :-) | |
| 15:10:26 | gibi | bauzas: a counter example is shared disk. there we said use the uuid of the placement aggregate instead of the name of the sharing RP in the nova conf. | |
| 15:10:34 | sean-k-mooney | mnaser: ya so in gen 1 they were tied of the same clock | |
| 15:10:48 | gibi | bauzas: so we are getting better at this | |
| 15:10:49 | bauzas | gibi: aaaaaand we discussed 30 mins of any potential issues with it :) | |
| 15:10:56 | dansmith | efried: reminder: I'm fine with logging the conflict late when the admin configured it wrong | |
| 15:11:02 | sean-k-mooney | so your memroy bandwith was limited by the speed at which the infinity frabric could run | |
| 15:11:04 | bauzas | the chicken-and-egg problem | |
| 15:11:10 | dansmith | maybe nobody else is :) | |
| 15:11:13 | gibi | bauzas: that is what PTG for. isn't it? ;) | |
| 15:11:21 | sean-k-mooney | mnaser: they split them to allwo you to tune them seperatly | |
| 15:11:34 | bauzas | gibi: that's surely some justification for my expense report, for sure :p | |
| 15:11:38 | sean-k-mooney | mnaser: so you can optimise for latency or througput | |
| 15:12:05 | efried | dansmith: That wfm too. The only question is, in addition to logging the error, we have to *pick one* to use. | |
| 15:12:10 | bauzas | "look ! this isn't a very expensive dinner, this was just a brainstorming session in a crowdy area !" | |
| 15:12:13 | efried | or, I suppose, refuse to use either. | |
| 15:12:19 | dansmith | efried: refuse to use either | |
| 15:12:33 | bauzas | "and we ordered snacks meanwhile" | |
| 15:12:39 | dansmith | IMHO | |
| 15:12:46 | efried | dansmith: ack. I agree. We shall make it so. | |
| 15:12:51 | efried | dansmith: Since you're being so nice already, and I've obv got you swapped out of whatever else you would be doing, can we talk about shelve-offloading an instance with a vTPM? | |
| 15:13:11 | dansmith | efried: yeah | |
| 15:13:37 | efried | Do you have a better idea than "create a glance (or maybe swift?) object for the vdev file"? | |
| 15:13:42 | gibi | bauzas: there wasn't too much snack provided during the PTG so _we had to_ order some :) | |
| 15:14:11 | efried | dansmith: seems like this is a natural for swift, but that adds yet another required component. | |
| 15:14:27 | bauzas | "good luck, expenses team, for understanding my bills !" | |
| 15:14:41 | dansmith | efried: that would be one place I guess, but yeah, can't really depend on swift being there | |
| 15:15:04 | mnaser | sean-k-mooney: interestingly enough looking at the bios the 4-link xGMI max speed is set to 10.667Gbps | |
| 15:15:28 | dansmith | efried: I dunno how safe that would really be, not that it's less secure than the alternatives, but.. | |
| 15:15:50 | mnaser | "Setting this to a lower speed can save uncore power that can be used to increase core frequency or reduce overall power. It will also decrease cross socket bandwidth and increase cross socket latency." | |
| 15:16:00 | kashyap | coreycb: One more on that thing you asked earlier: QEMU upstream has this notion of "versioned CPU models", and we'll be getting new named models with the affected flags turned off. | |
| 15:16:08 | mnaser | i wonder why SMC would ship a box with with it set to the minimum speed for a server where i dont really care about cpu performance, ill bump that to 18Gbps.. | |
| 15:16:25 | efried | dansmith: Well, we're relying on the encrypted-ness of the vdev file already. A main use case is "someone walks away with the disk and we're still okay". | |
| 15:16:28 | kashyap | coreycb: I'll update the Ubuntu bug you pointed. | |
| 15:16:43 | coreycb | kashyap: that sounds nice. great, thanks | |
| 15:17:06 | efried | dansmith: though since you bring it up, I wonder if it would be possible to (ab)use barbican... | |
| 15:17:36 | dansmith | efried: sure, but access to that enables an offline attack, and not everyone may have the same security policy for their swift cluster as they do for their hypervisor nodes | |
| 15:17:59 | dansmith | efried: switft may be a best effort uuber cheap storage bucket for them, whereas hypervisor and glance storage is high security at a premium | |
| 15:18:21 | dansmith | efried: do you have a pointer to what glance calls the feature of linking multiple image payloads together? | |
| 15:18:35 | efried | oh, is that already a thing? | |
| 15:19:16 | dansmith | I thought it was, and I thought you made mention of it | |
| 15:19:19 | efried | I was just going to, like, make up a metadata key that would store the a unique identifier and stuff the same value on both images to link them. | |
| 15:19:41 | dansmith | oh okay, I think there was actually a use case for this feature for something else at some point, but I might be wrong | |
| 15:19:43 | sean-k-mooney | dansmith: do you mean the ablity to have multple image location or something else | |
| 15:20:02 | efried | sean-k-mooney: talking about being able to "link" two images together in some logical fashion. | |
| 15:20:07 | dansmith | sean-k-mooney: that's not what we're talking about | |
| 15:20:24 | sean-k-mooney | oh ok like we propose to do for cyborg | |
| 15:20:47 | sean-k-mooney | e.g. where the os image has a metadata key that specify the bitstream image for the fpga | |
| 15:21:06 | efried | yeah, that's a close enough use case | |
| 15:21:21 | efried | in this case the two images are completely 1:1 | |
| 15:21:37 | efried | but otherwise it's a pretty similar model. | |
| 15:21:55 | sean-k-mooney | im not sure there is an existing freature for that | |
| 15:22:04 | sean-k-mooney | if there was we would have suggested using it for cyborg | |
| 15:22:16 | dansmith | it's similar, but I would expect that glance would gain a new container format for that, | |
| 15:22:24 | dansmith | whereas this is more "please store this binary blob for me" | |
| 15:22:54 | sean-k-mooney | well that is what the bitstream kindo of is but sure | |
| 15:22:59 | efried | see https://review.opendev.org/#/c/686804/6/specs/ussuri/approved/add-emulated-virtual-tpm.rst@197 | |
| 15:22:59 | efried | sean-k-mooney: we're talking about the emulated TPM device, which libvirt stores outside of the instance dir. When we shelve-offload, we need to carry that vdev along somehow. We're brainstorming ways to make that happen. | |
| 15:23:19 | sean-k-mooney | ah ok | |
| 15:23:31 | dansmith | sean-k-mooney: it's not if there's some agreed-upon format for containing those bitstreams, even though the bitstream themselves aren't standard | |
| 15:24:07 | sean-k-mooney | dansmith: well for the bitstream i think the plan was intilaly to use raw | |
| 15:24:16 | sean-k-mooney | but there isnt a standard format | |
| 15:24:18 | dansmith | sean-k-mooney: you mean "bare"? | |
| 15:24:29 | dansmith | or you mean no additional wrapper around the bitstream? | |
| 15:24:33 | sean-k-mooney | i mean image type raw container type bare | |
| 15:24:46 | dansmith | yeah, so that doesn't seem like a good idea to me | |
| 15:24:55 | dansmith | because nova thinks it can boot those kinds of things | |
| 15:24:57 | efried | sean-k-mooney: the spec had originally been proposing to copy the vdev into the instance dir so it would be carried along for live migration, and then unpack it on the other side. But the instance dir is blown away for offload, so that model doesn't carry. | |
| 15:25:00 | dansmith | also, the other thing to keep in mind here, | |
| 15:25:05 | sean-k-mooney | well the bitstream format is vendeor depenent | |
| 15:25:13 | sean-k-mooney | it would be nice to have a sperate format for it | |
| 15:25:20 | dansmith | is that for things like glance multi-store, you have to know that those things are linked together when you move/copy between stores | |
| 15:25:28 | dansmith | and, | |
| 15:25:39 | dansmith | if you use that snapshot to spawn to a ceph-using compute node, | |
| 15:25:45 | dansmith | what do you do with that tpm image? | |
| 15:25:50 | dansmith | copy to everyone? ignore? | |
| 15:26:00 | sean-k-mooney | efried: right we would need to snapshot it and store it somewhere | |
| 15:26:09 | dansmith | "copy to everyone" meaning.. any instance that CoWs from that should get a copy of the tpm | |
| 15:26:47 | efried | good question, I hope if you're using a snapshot to create multiple VMs from, you would want them to start with a clean TPM, but mebbe not. | |
| 15:26:58 | dansmith | efried: but how do you know? | |
| 15:27:08 | dansmith | efried: if I snapshot my instance and delete it, then come back a year later for black friday, | |
| 15:27:10 | sean-k-mooney | dansmith: by the way this topic in general sound more like the goal of glare to solve. e.g. a generic artifact store | |
| 15:27:15 | dansmith | you don't know if I'm going to create one or a hundred of those | |
| 15:27:18 | sean-k-mooney | but i dont think glare is a thing anymore | |
| 15:27:34 | efried | dansmith: your point is that we can't differentiate between shelve-offload-unshelve and snapshot-restore? | |
| 15:27:38 | dansmith | sean-k-mooney: maybe, not sure glare was really for keeping this kind of thing, vs things like template configs.. but maybe | |
| 15:28:12 | dansmith | efried: no we definitely can, I'm saying snapshot-restore has a similar problem if you store the thing in glance or otherwise link it via the snapshot itself, vs. the image record in nova | |
| 15:28:26 | sean-k-mooney | so i guess there are different usecase but i would not assuem that we want to share tpm snapshot betwen multipel instnaces | |
| 15:29:01 | sean-k-mooney | that said i can see a use case for that where i store secure boot keys or something in the tpm snapshot | |
| 15:29:04 | dansmith | sean-k-mooney: agree | |
| 15:29:11 | efried | sean-k-mooney: I think dansmith is saying that, even if we put that stake in the ground, we can't necessarily know when we go to use a snapshot | |
| 15:29:21 | dansmith | right, | |
| 15:29:27 | sean-k-mooney | ya | |
| 15:29:36 | efried | but dansmith I would think we would only ever restore the vTPM if the operation is unshelve. If it's just "boot with this image" we don't. | |
| 15:29:43 | dansmith | so it would be better if we have a solution where we the linkage is between the instance and the tpm, not the snapshot image and the tpm | |