| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-29 | |||
| 17:42:04 | efried | that is, are there discrete values that are considered small on some systems and large on others? | |
| 17:42:10 | stephenfin | ALL the pagesizes | |
| 17:42:22 | sean-k-mooney | efried: in practice not really but technically there can be | |
| 17:42:30 | stephenfin | sean-k-mooney: 4k is a small page _everywhere_, right? | |
| 17:42:32 | sean-k-mooney | small is almost always 4k | |
| 17:42:38 | sean-k-mooney | no | |
| 17:42:40 | efried | those are different statements | |
| 17:42:54 | sean-k-mooney | small is the native pagesize of the host | |
| 17:43:01 | sean-k-mooney | and the smallest in the available set | |
| 17:43:25 | sean-k-mooney | it is almost alwasy 4k and raely 16k or 64k | |
| 17:43:47 | stephenfin | sean-k-mooney: The internet tells me 4k is hardcoded as the default page size in Linux | |
| 17:43:57 | stephenfin | https://unix.stackexchange.com/a/128218 | |
| 17:44:18 | sean-k-mooney | yes on x86, aarch64 and power9 | |
| 17:44:41 | stephenfin | okay, we don't need to care about anything else, realistically | |
| 17:44:56 | sean-k-mooney | that why i said its almost alwasy 4k | |
| 17:45:01 | stephenfin | we can stick a giant TODO in somewhere in case someone wants to run openstack on obscure architecture | |
| 17:45:03 | sean-k-mooney | we can proably treat it as such | |
| 17:45:07 | stephenfin | agreed | |
| 17:45:15 | stephenfin | s/TODO/NOTE/ | |
| 17:45:36 | stephenfin | efried: what are you referring to? | |
| 17:45:44 | sean-k-mooney | large is defiend as any pagesize that is not the same as small on the plathform | |
| 17:45:49 | efried | but accept that it's not going to be perfect, and that we have to do the rest in the NTF. | |
| 17:45:49 | efried | Use ^ where possible to make the placement pass a bit better | |
| 17:45:49 | efried | Supply traits saying things like I_HAVE_$SIZE_PAGES_HERE, where $SIZE can include both discrete and abstract sizes. | |
| 17:45:49 | efried | Represent all the memory as MEMORY_MB with a step_size that's the least common denominator of all the page sizes. | |
| 17:45:49 | efried | okay, so I'm afraid the compromise we need to make is this: | |
| 17:46:12 | efried | ...which may entail bouncing a host late. | |
| 17:46:53 | stephenfin | I don't understand how step_size would work | |
| 17:46:54 | sean-k-mooney | efried: if we do that we need to keep the hugepage code in the resouce tracker and numa toplogy filter for ever | |
| 17:47:24 | efried | I'm afraid that may be unavoidable, unless we make big changes in placement, or people agree to stop needing that shit. | |
| 17:47:29 | sean-k-mooney | efried: if we had a placement exteion weree we could pass a step size in the querry that would help | |
| 17:47:39 | stephenfin | I have a host that have 16 1GB hugepages and the rest are normal small pages | |
| 17:47:45 | efried | stephenfin: example, if we have 4K, 1M, and 2G pages, the step_size has to be 4K | |
| 17:47:59 | stephenfin | you will always use 4k so | |
| 17:48:06 | stephenfin | every host is going to report small pages | |
| 17:48:13 | efried | okay, so be it. | |
| 17:48:15 | sean-k-mooney | on that host | |
| 17:48:15 | sean-k-mooney | efried: not all the moroy can be allocated in 4k | |
| 17:48:23 | sean-k-mooney | hugepages are preallcoted | |
| 17:48:31 | sean-k-mooney | with a given page size | |
| 17:48:32 | efried | yeah, I understand that. We have to do that part via the NTF. I don't see a way around it. | |
| 17:48:32 | stephenfin | I don't get how this solves anything | |
| 17:48:40 | efried | "solves" | |
| 17:48:47 | efried | it gives us a way to move forward | |
| 17:49:03 | efried | and support all the variants we need | |
| 17:49:14 | sean-k-mooney | well the thing is if we had different resocue classes hugepage was one of the things we had determin could be fully done by placment | |
| 17:49:19 | stephenfin | this is essentially not modelling different page sizes in placement | |
| 17:49:21 | sean-k-mooney | if we modeld the different pages sizes in placment | |
| 17:49:25 | efried | stephenfin: correct. | |
| 17:49:46 | stephenfin | NUMA is all about memory | |
| 17:49:47 | sean-k-mooney | so thats a lot of work for very littel benifit | |
| 17:49:50 | sean-k-mooney | yep | |
| 17:49:53 | efried | we cannot do that and also support requests for MEMORY_MB=$X that don't care about pages. | |
| 17:50:03 | stephenfin | what's the point in modelling NUMA in placement if we don't do the memory modelling? | |
| 17:50:08 | efried | affinity | |
| 17:50:17 | sean-k-mooney | efried: we can if MEMORY_MB is always for non numa memory | |
| 17:50:19 | stephenfin | we have the NUMA topology filter for that | |
| 17:50:28 | stephenfin | good enough | |
| 17:50:41 | efried | stephenfin: this allows us to move a big swatch of the affinity filtering to the placement query. | |
| 17:50:48 | efried | just not all of it. | |
| 17:50:50 | sean-k-mooney | efried: no it does not | |
| 17:50:54 | efried | and not as much as we would have liked | |
| 17:50:56 | efried | wha? | |
| 17:50:58 | stephenfin | very little of it | |
| 17:51:00 | efried | wha?? | |
| 17:51:01 | sean-k-mooney | we will have to do it all again in the ntf | |
| 17:51:08 | stephenfin | yup | |
| 17:51:14 | efried | no | |
| 17:51:14 | efried | um | |
| 17:51:16 | efried | look | |
| 17:51:20 | sean-k-mooney | we may have less host to check | |
| 17:51:26 | stephenfin | what's the objection to modelling different page sizes as different resource types? | |
| 17:51:31 | stephenfin | they're different things | |
| 17:51:36 | sean-k-mooney | but we still need to check all aspecs again | |
| 17:51:38 | stephenfin | something that ask for one can't consume the other | |
| 17:51:54 | efried | Because it makes it impossible to request a certain amount of memory without asking for specific pages. | |
| 17:51:58 | sean-k-mooney | there is another way to do it | |
| 17:52:06 | sean-k-mooney | we can do it in 1 resouce class | |
| 17:52:10 | sean-k-mooney | MEMORY_MB | |
| 17:52:11 | stephenfin | but it doesn't | |
| 17:52:18 | efried | how would you do it stephenfin? | |
| 17:52:19 | sean-k-mooney | but we need 1 RP per page size | |
| 17:52:31 | stephenfin | we report MEMORY_MB for normal 4k pages | |
| 17:52:33 | sean-k-mooney | and each RP would be under a numa node | |
| 17:52:49 | stephenfin | and then PAGES_4M, PAGES_1G, etc. for the various hugepage sizes | |
| 17:53:00 | sean-k-mooney | that wont work | |
| 17:53:10 | stephenfin | if instances aren't requesting page sizes, they get MEMORY_MB | |
| 17:53:11 | sean-k-mooney | because it breaks large | |
| 17:53:22 | stephenfin | it does, and we'll need to figure out a migration plan for that | |
| 17:53:27 | sean-k-mooney | mem_page_size=large is the most common case | |
| 17:53:32 | efried | Okay, so if you don't ask for large pages, you only get small pages, and if the only way to satisfy your memory requirement would be by using the large pages, you just bounce. | |
| 17:53:41 | stephenfin | you can't use the large pages | |
| 17:53:46 | stephenfin | unless you *explicitly* ask for them | |
| 17:53:50 | efried | is that what happens today? | |
| 17:53:52 | stephenfin | yup | |
| 17:54:01 | efried | okay cool. | |
| 17:54:09 | sean-k-mooney | yes | |
| 17:54:16 | stephenfin | given a host with 8GB of RAM, of which 4GB is allocated to 1GB huge pages | |
| 17:54:21 | sean-k-mooney | which is why you have to partion the host today | |
| 17:54:37 | sean-k-mooney | we have 2 memory tracker in nova today | |