| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-03 | |||
| 18:06:36 | efried | and then obviously we have to do the things in nova to 1) reshape and 2) request accordingly. | |
| 18:06:59 | dansmith | yeah the reshape is what I'm concerned about.. that delaying us another cycle | |
| 18:07:01 | efried | I feel it is very well understood what 1 and 2 will look like. | |
| 18:07:29 | sean-k-mooney | 1 is https://review.opendev.org/#/q/topic:story/2006068+(status:open+OR+status:merged) right | |
| 18:07:50 | dansmith | those are all placement, | |
| 18:07:53 | dansmith | the reshape is in nova | |
| 18:08:13 | sean-k-mooney | oh ya but the reshap cant happen until the placment stuff is done | |
| 18:08:37 | dansmith | sure, just being clear | |
| 18:08:52 | dansmith | the reshape is more complicated (IMHO, and IME), so unless that's "close" I'd be more concerned | |
| 18:08:58 | sean-k-mooney | im not sure we fully underdand what the reshap will looklike | |
| 18:09:00 | efried | yeah, those are a minor bugfix for rg-to-rp mappings. Really only cdent's is needed, the rest are what-ifs. | |
| 18:09:29 | efried | The nested magic spec has several examples of what the topo will look like | |
| 18:09:48 | dansmith | I think knowing what it will look like is not so much the problem | |
| 18:10:38 | efried | deciding which resources need to go where? | |
| 18:10:50 | sean-k-mooney | yes | |
| 18:11:09 | dansmith | the actual mechanics of doing the reshape.. I know it's doable and we know what the end result will look like | |
| 18:11:25 | sean-k-mooney | for exampel i think cpus are going to need to be moded in a nested resouce provider of the numa node not as an inventory of the numa node | |
| 18:12:11 | sean-k-mooney | hugepages should proably be on the numa node but as a new resouce class per pagesize or as somthign else? | |
| 18:12:30 | dansmith | efried: case in point ^ :) | |
| 18:12:31 | sean-k-mooney | dose memroy_mb stay on the compute resouce proveiider? | |
| 18:12:41 | dansmith | still plenty of discussion to derail the reshape | |
| 18:15:48 | dansmith | efried: anyway, I think you oughta make a clear decision on this point, and -2 bauzas' thing if so, so that the messaging is clear what to expect and focus on | |
| 18:16:19 | dansmith | I'm not saying that's a terrible thing, I just think it's not terrible to continue with both | |
| 18:16:35 | dansmith | obviously I want the pure-placement approach for all the right reasons | |
| 18:17:02 | efried | "not terrible to continue with both" is why I haven't downvoted | |
| 18:17:25 | dansmith | okay | |
| 18:17:33 | efried | but "want the pure-placement approach" is why I haven't upvoted. | |
| 18:17:43 | efried | Who's going to work the code for bauzas's spec. | |
| 18:17:44 | efried | ? | |
| 18:17:52 | sean-k-mooney | bauzas: | |
| 18:17:55 | dansmith | um, bauzas I hope :D | |
| 18:18:15 | sean-k-mooney | assumign its approved if not he will bw working on other things | |
| 18:18:23 | dansmith | mriedem: any thoughts on this? if you and efried are both -0.9 then it's probably not worth expecting it to happen | |
| 18:18:26 | efried | If that person were dedicated instead to working on the topo reshape, that would get us closer | |
| 18:18:35 | efried | faster | |
| 18:18:42 | efried | even if it doesn't all come together in train. | |
| 18:19:04 | efried | whereas I'm not sure what we gain by having the separate weigher. | |
| 18:19:44 | sean-k-mooney | we get numa support for gpus in trian which will be the basis of our next longer support release donwstream | |
| 18:19:45 | dansmith | the weigher gets us a lot closer to the correct behavior with a minimal amount of code and pretty predictable amount of work | |
| 18:19:50 | dansmith | it doesn't get us to perfection, obviously | |
| 18:20:43 | efried | closer to the correct behavior... for just vgpu, right? | |
| 18:21:05 | dansmith | well, the bulk of this is reusable for other child devices, | |
| 18:21:07 | sean-k-mooney | we already do the right thing for pci pasthough | |
| 18:21:16 | dansmith | although I expect the weigher would probably be specific to gpus | |
| 18:21:16 | sean-k-mooney | and for hugepages and cpu pinning | |
| 18:21:24 | efried | other child devices - that we don't model in placement yet | |
| 18:21:31 | efried | so n/a? | |
| 18:22:08 | efried | and please let's not start modeling devices in non-NUMA-aware placement and adding weigher code to do the NUMA stuff for that. | |
| 18:22:16 | dansmith | efried: other things we hang off of the compute RP that aren't specific in placement, but yeah, nothing pending, just saying this isn't adding a bunch of stuff to do gpu-specific things | |
| 18:22:18 | sean-k-mooney | we dont track them in plamcneet and can provide better placmente and affintiy because of that | |
| 18:22:53 | efried | until we track them *properly* in placement, right. | |
| 18:23:07 | sean-k-mooney | which has been promsied for 2 years at this point | |
| 18:23:25 | efried | good, so let's get it the f done instead of keep kicking it down the road | |
| 18:23:32 | dansmith | efried: just FYI I hate myself for even arguing about this, because pure-placement is obviously the right way, I'm just hedging because of the pain of things not actually being in train | |
| 18:23:59 | dansmith | because seriously... years. | |
| 18:24:39 | efried | I've said before (or maybe just thought it) vgpu may have been nice as a foot in the door proving device modeling in placement, but premature. And now we're trying to retrofit to make it work instead of shelving it until we can make the infrastructure support it properly. | |
| 18:25:03 | efried | but | |
| 18:25:19 | efried | I still don't feel strongly enough to block | |
| 18:25:28 | sean-k-mooney | efried: for context the CAT stuff im working on here https://review.opendev.org/#/c/662264/ which you would also like me to track only with numa awer placment was requested for pike | |
| 18:25:38 | dansmith | if we didn't have basic gpu stuff that we have today at this point, I think we'd probably be having a much different conversation.. and not a fun one. | |
| 18:26:17 | efried | Tell ya, though, if sam_subtree lands pretty soon, as appears likely to do, that may be enough to make me -1 the vgpu affinity spec. | |
| 18:26:22 | dansmith | I thought you were the "don't wait forever for perfect" guy? :) | |
| 18:26:57 | efried | Totally. And I have no illusions that numa-in-placement will be perfect right away | |
| 18:27:19 | dansmith | well, if you think that's going to happen, you should just preemtively -1 it, or | |
| 18:27:21 | efried | which btw is why we should not be arguing about hugepages and splitting CPUs finer than per-numa-node. | |
| 18:27:28 | dansmith | or -1 it contingent on that merging when you think it will | |
| 18:27:44 | dansmith | because really, messaging the likelihood of this being a thing is pretty important to us | |
| 18:28:24 | efried | and why we punted on can_split | |
| 18:28:50 | efried | "messaging the likelihood of this being a thing" is actually the best argument I've heard so far. | |
| 18:29:52 | dansmith | it's the most important I think | |
| 18:29:58 | sean-k-mooney | well if we cant have memory fine then a numa node we can implemenat CAT | |
| 18:30:17 | sean-k-mooney | the cpus inventories and cache inventores shoudl be in the same RP | |
| 18:30:26 | sean-k-mooney | and cache regins are finer then numa node | |
| 18:30:40 | sean-k-mooney | we can have mulplie l3 cache regins in the same numa node | |
| 18:30:49 | efried | understood. | |
| 18:30:56 | efried | So | |
| 18:31:23 | sean-k-mooney | it would basicaly be a 3 level tree | |
| 18:31:25 | efried | when we implement CAT *after* having implemented NUMA topo as its own thing, we can reshape the CPUs and L3s down into grandchild providers. | |
| 18:31:45 | sean-k-mooney | yes we could | |
| 18:32:00 | efried | but we should not be trying to solve CAT preemptively. | |
| 18:32:02 | sean-k-mooney | or we could do that first and reshap them under the numa node when they are added | |
| 18:32:28 | efried | I owe you a response in the pqos spec, but that doesn't buy us anything at this point. | |
| 18:32:48 | sean-k-mooney | we talked about it a little today in our squad call | |
| 18:33:02 | sean-k-mooney | im going to simplfy it and proably add a depency on some of the placmenet stuff | |
| 18:33:24 | sean-k-mooney | i will likely make memory bandwith depend on the numa in placment spec | |
| 18:33:38 | sean-k-mooney | i would like to keep cache not depenent on tha tbut may on same tree | |
| 18:34:34 | sean-k-mooney | well it depends i have to see is there a way to us more of the plamcent stuff | |
| 18:35:05 | sean-k-mooney | it significantly incresase the risk we wont have CAT in train however | |
| 18:35:53 | efried | I'm addressing similar concerns in the providers.yaml spec | |
| 18:36:47 | sean-k-mooney | ill admit i have nto really been following that one. i kindof got burt out on that topic last cycle | |
| 18:37:05 | sean-k-mooney | is it approching agreement or still in flux | |
| 18:37:36 | efried | Dakshina has me convinced that there's a reason to do cache in child providers. But only to ensure that one request group gets all its cacheways from one package -- *not* because we can do anything about affinity. | |
| 18:37:40 | efried | however | |
| 18:37:44 | efried | in tree | |
| 18:37:59 | efried | we can do affinity after GET /a_c via the NUMATopologyFilter | |
| 18:39:54 | efried | was gonna say that couples the filter to libvirt, but that's already dismally true anyway. | |
| 18:40:26 | efried | sean-k-mooney: I haven't posted those comments yet, but will ping you once I have. | |
| 18:41:06 | sean-k-mooney | ok. im goign to leave the rebasing it so that we can continue the disucssion in one place | |
| 18:41:19 | sean-k-mooney | are you off for the 4th of july weekend? | |
| 18:41:29 | sean-k-mooney | i assume yes | |