| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-03 | |||
| 18:17:02 | efried | "not terrible to continue with both" is why I haven't downvoted | |
| 18:17:25 | dansmith | okay | |
| 18:17:33 | efried | but "want the pure-placement approach" is why I haven't upvoted. | |
| 18:17:43 | efried | Who's going to work the code for bauzas's spec. | |
| 18:17:44 | efried | ? | |
| 18:17:52 | sean-k-mooney | bauzas: | |
| 18:17:55 | dansmith | um, bauzas I hope :D | |
| 18:18:15 | sean-k-mooney | assumign its approved if not he will bw working on other things | |
| 18:18:23 | dansmith | mriedem: any thoughts on this? if you and efried are both -0.9 then it's probably not worth expecting it to happen | |
| 18:18:26 | efried | If that person were dedicated instead to working on the topo reshape, that would get us closer | |
| 18:18:35 | efried | faster | |
| 18:18:42 | efried | even if it doesn't all come together in train. | |
| 18:19:04 | efried | whereas I'm not sure what we gain by having the separate weigher. | |
| 18:19:44 | sean-k-mooney | we get numa support for gpus in trian which will be the basis of our next longer support release donwstream | |
| 18:19:45 | dansmith | the weigher gets us a lot closer to the correct behavior with a minimal amount of code and pretty predictable amount of work | |
| 18:19:50 | dansmith | it doesn't get us to perfection, obviously | |
| 18:20:43 | efried | closer to the correct behavior... for just vgpu, right? | |
| 18:21:05 | dansmith | well, the bulk of this is reusable for other child devices, | |
| 18:21:07 | sean-k-mooney | we already do the right thing for pci pasthough | |
| 18:21:16 | sean-k-mooney | and for hugepages and cpu pinning | |
| 18:21:16 | dansmith | although I expect the weigher would probably be specific to gpus | |
| 18:21:24 | efried | other child devices - that we don't model in placement yet | |
| 18:21:31 | efried | so n/a? | |
| 18:22:08 | efried | and please let's not start modeling devices in non-NUMA-aware placement and adding weigher code to do the NUMA stuff for that. | |
| 18:22:16 | dansmith | efried: other things we hang off of the compute RP that aren't specific in placement, but yeah, nothing pending, just saying this isn't adding a bunch of stuff to do gpu-specific things | |
| 18:22:18 | sean-k-mooney | we dont track them in plamcneet and can provide better placmente and affintiy because of that | |
| 18:22:53 | efried | until we track them *properly* in placement, right. | |
| 18:23:07 | sean-k-mooney | which has been promsied for 2 years at this point | |
| 18:23:25 | efried | good, so let's get it the f done instead of keep kicking it down the road | |
| 18:23:32 | dansmith | efried: just FYI I hate myself for even arguing about this, because pure-placement is obviously the right way, I'm just hedging because of the pain of things not actually being in train | |
| 18:23:59 | dansmith | because seriously... years. | |
| 18:24:39 | efried | I've said before (or maybe just thought it) vgpu may have been nice as a foot in the door proving device modeling in placement, but premature. And now we're trying to retrofit to make it work instead of shelving it until we can make the infrastructure support it properly. | |
| 18:25:03 | efried | but | |
| 18:25:19 | efried | I still don't feel strongly enough to block | |
| 18:25:28 | sean-k-mooney | efried: for context the CAT stuff im working on here https://review.opendev.org/#/c/662264/ which you would also like me to track only with numa awer placment was requested for pike | |
| 18:25:38 | dansmith | if we didn't have basic gpu stuff that we have today at this point, I think we'd probably be having a much different conversation.. and not a fun one. | |
| 18:26:17 | efried | Tell ya, though, if sam_subtree lands pretty soon, as appears likely to do, that may be enough to make me -1 the vgpu affinity spec. | |
| 18:26:22 | dansmith | I thought you were the "don't wait forever for perfect" guy? :) | |
| 18:26:57 | efried | Totally. And I have no illusions that numa-in-placement will be perfect right away | |
| 18:27:19 | dansmith | well, if you think that's going to happen, you should just preemtively -1 it, or | |
| 18:27:21 | efried | which btw is why we should not be arguing about hugepages and splitting CPUs finer than per-numa-node. | |
| 18:27:28 | dansmith | or -1 it contingent on that merging when you think it will | |
| 18:27:44 | dansmith | because really, messaging the likelihood of this being a thing is pretty important to us | |
| 18:28:24 | efried | and why we punted on can_split | |
| 18:28:50 | efried | "messaging the likelihood of this being a thing" is actually the best argument I've heard so far. | |
| 18:29:52 | dansmith | it's the most important I think | |
| 18:29:58 | sean-k-mooney | well if we cant have memory fine then a numa node we can implemenat CAT | |
| 18:30:17 | sean-k-mooney | the cpus inventories and cache inventores shoudl be in the same RP | |
| 18:30:26 | sean-k-mooney | and cache regins are finer then numa node | |
| 18:30:40 | sean-k-mooney | we can have mulplie l3 cache regins in the same numa node | |
| 18:30:49 | efried | understood. | |
| 18:30:56 | efried | So | |
| 18:31:23 | sean-k-mooney | it would basicaly be a 3 level tree | |
| 18:31:25 | efried | when we implement CAT *after* having implemented NUMA topo as its own thing, we can reshape the CPUs and L3s down into grandchild providers. | |
| 18:31:45 | sean-k-mooney | yes we could | |
| 18:32:00 | efried | but we should not be trying to solve CAT preemptively. | |
| 18:32:02 | sean-k-mooney | or we could do that first and reshap them under the numa node when they are added | |
| 18:32:28 | efried | I owe you a response in the pqos spec, but that doesn't buy us anything at this point. | |
| 18:32:48 | sean-k-mooney | we talked about it a little today in our squad call | |
| 18:33:02 | sean-k-mooney | im going to simplfy it and proably add a depency on some of the placmenet stuff | |
| 18:33:24 | sean-k-mooney | i will likely make memory bandwith depend on the numa in placment spec | |
| 18:33:38 | sean-k-mooney | i would like to keep cache not depenent on tha tbut may on same tree | |
| 18:34:34 | sean-k-mooney | well it depends i have to see is there a way to us more of the plamcent stuff | |
| 18:35:05 | sean-k-mooney | it significantly incresase the risk we wont have CAT in train however | |
| 18:35:53 | efried | I'm addressing similar concerns in the providers.yaml spec | |
| 18:36:47 | sean-k-mooney | ill admit i have nto really been following that one. i kindof got burt out on that topic last cycle | |
| 18:37:05 | sean-k-mooney | is it approching agreement or still in flux | |
| 18:37:36 | efried | Dakshina has me convinced that there's a reason to do cache in child providers. But only to ensure that one request group gets all its cacheways from one package -- *not* because we can do anything about affinity. | |
| 18:37:40 | efried | however | |
| 18:37:44 | efried | in tree | |
| 18:37:59 | efried | we can do affinity after GET /a_c via the NUMATopologyFilter | |
| 18:39:54 | efried | was gonna say that couples the filter to libvirt, but that's already dismally true anyway. | |
| 18:40:26 | efried | sean-k-mooney: I haven't posted those comments yet, but will ping you once I have. | |
| 18:41:06 | sean-k-mooney | ok. im goign to leave the rebasing it so that we can continue the disucssion in one place | |
| 18:41:19 | sean-k-mooney | are you off for the 4th of july weekend? | |
| 18:41:29 | sean-k-mooney | i assume yes | |
| 18:41:49 | efried | and dansmith, I'll review tetsuro's same_subtree patch. I have the broad strokes from an earlier iteration, but it's now been squashed and reworked a little bit, so I need to refresh and see how far off it still is. And if it's as close as I think, I'll take a stand on the vgpu affinity spec. | |
| 18:42:05 | dansmith | thanks | |
| 18:42:06 | efried | sean-k-mooney: yes, I'll be off Th & F | |
| 18:42:31 | efried | Friday is my 22nd anniversary :) | |
| 18:43:19 | dustinc | efired: congrats! | |
| 18:43:20 | sean-k-mooney | 22nd, surly not wedding? unless you were married quite young | |
| 18:43:24 | dustinc | efried.. | |
| 18:44:03 | sean-k-mooney | as in before its legal. | |
| 18:44:12 | efried | hehe. I was married at 20 | |
| 18:44:30 | efried | so it was legal to marry, just not legal to buy alcohol :P | |
| 18:44:45 | efried | thanks dustinc | |
| 18:45:26 | sean-k-mooney | well congrats and i always get confused about how the us legislates for mimium ages for things | |
| 18:45:39 | efried | you're not the only one | |
| 18:46:20 | dustinc | can active duty still buy beer/wine at 18 on base? | |
| 18:46:23 | dustinc | gets even more confusing | |
| 18:46:25 | efried | 21 for alcohol is country-wide at this point. Louisiana was (I think) the last one to capitulate in the late '90s (from 18) when the fed cut off their road funding. | |
| 18:47:13 | sean-k-mooney | you can drive from 16 and get a gun licence at 16 or 18 right and marry at like 16-18 depending on the state | |
| 18:47:17 | efried | sean-k-mooney: I grew up near London, where the official age was 18 but not enforced at all. I was pubbing from 14. | |
| 18:47:21 | sean-k-mooney | and vote at 18 too right | |
| 18:48:06 | dustinc | the problem with voting here is you can vote at 18 but nobody does until 60 sigh... | |
| 18:48:08 | efried | vote at 18, yes, that was a big deal ~20y ago. Driving depends on the state, but it's 16 in most places. You can get an exception for "hardship" or farming (which are the same thing to me :P ) | |
| 18:48:16 | efried | that's not a problem | |
| 18:48:34 | efried | the problem with voting is that you are allowed to do so despite being completely ignorant | |
| 18:48:49 | dustinc | that doesn't change with age | |