Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-11-20
15:06:32 gibi efried: we have predicatable named nested RPs | 1110cf59-cabf-526c-bacc-08baabbac692 | aio:Open vSwitch agent:br-test
15:06:55 mnaser sean-k-mooney: sorry, was in a meeting too then, but no that wasnt enabled, but they are much older and also running ddr3 memory for comparision
15:07:02 mnaser much lower latency but lower bandwidth
15:07:12 gibi efried: but as I said I have no direct use case so I don't want to push for the naming support in provider config
15:07:14 sean-k-mooney so that was cross socket latency
15:07:16 bauzas efried: FWIW, we stecked a few things at the PTG between gibi, stephenfin and me that were tied to naming conventions :)
15:07:23 bauzas we sketched*
15:07:24 efried I know, what I'm saying is that we shouldn't have architected anything to rely on naming conventions, and we shouldn't perpetuate such things.
15:07:35 efried but that ship has probably sailed.
15:07:37 sean-k-mooney mnaser: i was trying to figure out if the intel number were between socket or two numa nodes in the same socket
15:07:37 mnaser sean-k-mooney: yep!
15:07:43 bauzas efried: indeed
15:07:49 sean-k-mooney ya those latency numbers for amd are bad
15:07:54 mnaser nope, between two sockets, which is interesting in just how much faster cross socket latency is low
15:08:21 efried In any case, if gibi doesn't have any real, current bw use cases, and bauzas doesn't have any real, current vgpu use cases, I'm fine making the decision to remove identify-by-name to limit scope/complexity.
15:08:23 gibi efried: there are things already architectured that way. like the hostname of the compute node is an id neutron use. and nova user the name of the device RP from neutron
15:08:24 mnaser sean-k-mooney: im looking at the bios for the speed between the two sockets, i learned something about the amd architecture that doesnt seem ideal
15:08:41 gibi efried: agree with the limited scope
15:08:41 efried we've removed a lot of bells and whistles for that reason, just to make this go.
15:08:52 gibi efried: go for it
15:08:56 sean-k-mooney so that lend more crednece to enableing numa per ccx and usign muiti numa guests
15:08:56 bauzas efried: it's not really a vgpu thingy
15:08:59 gibi efried: we can always add that later
15:09:05 bauzas efried: it's more what gibi said
15:09:08 efried otoh, where I started this morning actually handles things including name pretty well IMO
15:09:19 sean-k-mooney mnaser: the fact the infity fabric speed used to be tied to the meory speed
15:09:19 bauzas efried: some other projets create their own RPs and we need to know them
15:09:20 efried and dustinc has mostly already coded it up
15:09:26 efried so it's really just a question of test surface
15:09:34 mnaser memory channels onto the Infinity Fabric to progress through the SoC. To achieve the lowest latency, you can set the memory frequency to be equal to the Infinity Fabric speed. Lowering the memory clock speed also results in power savings in the memory controller, thus allowing the rest of the SoC to consume more power potentially resulting in a performance boost elsewhere, depending on the workload."
15:09:34 mnaser sean-k-mooney: "By default, the BIOS for EPYC 7002 Series processors will run at the maximum allowable clock frequency by the platform. This configuration results in the maximum memory bandwidth for the processor, but in some cases, it may not be the lowest latency. The Infinity Fabric will have a maximum speed of 1467 MHz (lower in some platforms), resulting in a single clock penalty to transfer data from the
15:09:50 bauzas efried: for the moment, we infer their existences based on naming convention but I'm all up for other ideas :-)
15:10:26 gibi bauzas: a counter example is shared disk. there we said use the uuid of the placement aggregate instead of the name of the sharing RP in the nova conf.
15:10:34 sean-k-mooney mnaser: ya so in gen 1 they were tied of the same clock
15:10:48 gibi bauzas: so we are getting better at this
15:10:49 bauzas gibi: aaaaaand we discussed 30 mins of any potential issues with it :)
15:10:56 dansmith efried: reminder: I'm fine with logging the conflict late when the admin configured it wrong
15:11:02 sean-k-mooney so your memroy bandwith was limited by the speed at which the infinity frabric could run
15:11:04 bauzas the chicken-and-egg problem
15:11:10 dansmith maybe nobody else is :)
15:11:13 gibi bauzas: that is what PTG for. isn't it? ;)
15:11:21 sean-k-mooney mnaser: they split them to allwo you to tune them seperatly
15:11:34 bauzas gibi: that's surely some justification for my expense report, for sure :p
15:11:38 sean-k-mooney mnaser: so you can optimise for latency or througput
15:12:05 efried dansmith: That wfm too. The only question is, in addition to logging the error, we have to *pick one* to use.
15:12:10 bauzas "look ! this isn't a very expensive dinner, this was just a brainstorming session in a crowdy area !"
15:12:13 efried or, I suppose, refuse to use either.
15:12:19 dansmith efried: refuse to use either
15:12:33 bauzas "and we ordered snacks meanwhile"
15:12:39 dansmith IMHO
15:12:46 efried dansmith: ack. I agree. We shall make it so.
15:12:51 efried dansmith: Since you're being so nice already, and I've obv got you swapped out of whatever else you would be doing, can we talk about shelve-offloading an instance with a vTPM?
15:13:11 dansmith efried: yeah
15:13:37 efried Do you have a better idea than "create a glance (or maybe swift?) object for the vdev file"?
15:13:42 gibi bauzas: there wasn't too much snack provided during the PTG so _we had to_ order some :)
15:14:11 efried dansmith: seems like this is a natural for swift, but that adds yet another required component.
15:14:27 bauzas "good luck, expenses team, for understanding my bills !"
15:14:41 dansmith efried: that would be one place I guess, but yeah, can't really depend on swift being there
15:15:04 mnaser sean-k-mooney: interestingly enough looking at the bios the 4-link xGMI max speed is set to 10.667Gbps
15:15:28 dansmith efried: I dunno how safe that would really be, not that it's less secure than the alternatives, but..
15:15:50 mnaser "Setting this to a lower speed can save uncore power that can be used to increase core frequency or reduce overall power. It will also decrease cross socket bandwidth and increase cross socket latency."
15:16:00 kashyap coreycb: One more on that thing you asked earlier: QEMU upstream has this notion of "versioned CPU models", and we'll be getting new named models with the affected flags turned off.
15:16:08 mnaser i wonder why SMC would ship a box with with it set to the minimum speed for a server where i dont really care about cpu performance, ill bump that to 18Gbps..
15:16:25 efried dansmith: Well, we're relying on the encrypted-ness of the vdev file already. A main use case is "someone walks away with the disk and we're still okay".
15:16:28 kashyap coreycb: I'll update the Ubuntu bug you pointed.
15:16:43 coreycb kashyap: that sounds nice. great, thanks
15:17:06 efried dansmith: though since you bring it up, I wonder if it would be possible to (ab)use barbican...
15:17:36 dansmith efried: sure, but access to that enables an offline attack, and not everyone may have the same security policy for their swift cluster as they do for their hypervisor nodes
15:17:59 dansmith efried: switft may be a best effort uuber cheap storage bucket for them, whereas hypervisor and glance storage is high security at a premium
15:18:21 dansmith efried: do you have a pointer to what glance calls the feature of linking multiple image payloads together?
15:18:35 efried oh, is that already a thing?
15:19:16 dansmith I thought it was, and I thought you made mention of it
15:19:19 efried I was just going to, like, make up a metadata key that would store the a unique identifier and stuff the same value on both images to link them.
15:19:41 dansmith oh okay, I think there was actually a use case for this feature for something else at some point, but I might be wrong
15:19:43 sean-k-mooney dansmith: do you mean the ablity to have multple image location or something else
15:20:02 efried sean-k-mooney: talking about being able to "link" two images together in some logical fashion.
15:20:07 dansmith sean-k-mooney: that's not what we're talking about
15:20:24 sean-k-mooney oh ok like we propose to do for cyborg
15:20:47 sean-k-mooney e.g. where the os image has a metadata key that specify the bitstream image for the fpga
15:21:06 efried yeah, that's a close enough use case
15:21:21 efried in this case the two images are completely 1:1
15:21:37 efried but otherwise it's a pretty similar model.
15:21:55 sean-k-mooney im not sure there is an existing freature for that
15:22:04 sean-k-mooney if there was we would have suggested using it for cyborg
15:22:16 dansmith it's similar, but I would expect that glance would gain a new container format for that,
15:22:24 dansmith whereas this is more "please store this binary blob for me"
15:22:54 sean-k-mooney well that is what the bitstream kindo of is but sure
15:22:59 efried see https://review.opendev.org/#/c/686804/6/specs/ussuri/approved/add-emulated-virtual-tpm.rst@197
15:22:59 efried sean-k-mooney: we're talking about the emulated TPM device, which libvirt stores outside of the instance dir. When we shelve-offload, we need to carry that vdev along somehow. We're brainstorming ways to make that happen.
15:23:19 sean-k-mooney ah ok
15:23:31 dansmith sean-k-mooney: it's not if there's some agreed-upon format for containing those bitstreams, even though the bitstream themselves aren't standard
15:24:07 sean-k-mooney dansmith: well for the bitstream i think the plan was intilaly to use raw
15:24:16 sean-k-mooney but there isnt a standard format
15:24:18 dansmith sean-k-mooney: you mean "bare"?
15:24:29 dansmith or you mean no additional wrapper around the bitstream?
15:24:33 sean-k-mooney i mean image type raw container type bare
15:24:46 dansmith yeah, so that doesn't seem like a good idea to me
15:24:55 dansmith because nova thinks it can boot those kinds of things
15:24:57 efried sean-k-mooney: the spec had originally been proposing to copy the vdev into the instance dir so it would be carried along for live migration, and then unpack it on the other side. But the instance dir is blown away for offload, so that model doesn't carry.
15:25:00 dansmith also, the other thing to keep in mind here,

Earlier   Later