| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-03 | |||
| 10:26:19 | sudipto | yeah | |
| 10:26:32 | johnthetubaguy | did that go in the spec, i remember it being nasty | |
| 10:26:44 | bauzas | but like I said, I just think each cpu pin is a RP with an inventory of 1 | |
| 10:26:51 | sudipto | johnthetubaguy, didn't read that in the spec. I may have missed if it was there. | |
| 10:27:00 | bauzas | a NUMA cell would be a parent RP | |
| 10:27:05 | bauzas | of that CPU pin | |
| 10:27:17 | johnthetubaguy | The NUMA node child providers will have inventory records populated for the NUMA_CORE, NUMA_THREAD and NUMA_MEMORY_MB resource classes. | |
| 10:27:19 | bauzas | and a compute RP would be the parent of all NUMA cell RPs | |
| 10:27:24 | johnthetubaguy | http://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/nested-resource-providers.html#proposed-change | |
| 10:27:35 | bauzas | johnthetubaguy: right, but that doesn't mention the CPU pins | |
| 10:27:40 | sudipto | yeah | |
| 10:27:52 | johnthetubaguy | ah, gotcha | |
| 10:28:09 | johnthetubaguy | thats a qualitative problem, which I guess is what bauzas said earlier | |
| 10:28:17 | bauzas | johnthetubaguy: not really | |
| 10:28:24 | bauzas | johnthetubaguy: you need to *consume* a pin | |
| 10:28:40 | sudipto | yeah consume the pin quantitatively. | |
| 10:28:51 | bauzas | johnthetubaguy: hence my thinking of having a RP being the CPU pin itself | |
| 10:29:04 | bauzas | with an inventory of VCPU with amount of 1 | |
| 10:29:18 | johnthetubaguy | so there are a few sorts of pins right now, which one is this? | |
| 10:29:33 | bauzas | and pinning the CPU would mean writing an allocation against that inventory | |
| 10:29:41 | johnthetubaguy | really you care about getting a dedicated pCPU on the "correct" numa node, I thought? | |
| 10:29:58 | johnthetubaguy | or is this something else | |
| 10:30:04 | bauzas | johnthetubaguy: I'm considering the CPU-pinning feature we implemented in Nova | |
| 10:30:18 | bauzas | there are many ways to schedule the pinning | |
| 10:30:23 | bauzas | but AFAICT, we fully pin | |
| 10:30:47 | sudipto | we could choose to pin it to a NUMA node or choose to just pin a cpuset (older nova behaviour) | |
| 10:31:18 | bauzas | whatever | |
| 10:31:21 | sudipto | My problem is with the representation of the inventory data for cpusets itself with the current inventory field called 'total' | |
| 10:31:51 | bauzas | and ? | |
| 10:31:55 | johnthetubaguy | so the current inventroy doesn't work right, hence the need for nested, and the new resources of NUMA_CORE, etc | |
| 10:32:07 | bauzas | if you consider a single pCPU being a RP | |
| 10:32:25 | sudipto | so we start considering every single pCPU as a RP and then do what bauzas suggested. | |
| 10:32:27 | bauzas | with an VCPU inventory attached to it | |
| 10:32:28 | johnthetubaguy | well, no, I think a NUMA_CORE is a colleciton of pCPU, and you consume one of them | |
| 10:32:47 | johnthetubaguy | but I may have this upside down | |
| 10:32:51 | bauzas | johnthetubaguy: you mean a NUMA cell | |
| 10:33:04 | johnthetubaguy | well, no | |
| 10:33:16 | bauzas | I'm confused then | |
| 10:33:21 | johnthetubaguy | a NUMA cell, reports as a provider of NUMA_CORE | |
| 10:33:27 | johnthetubaguy | contains lots of pCPUs you can claim | |
| 10:33:28 | johnthetubaguy | I think | |
| 10:33:31 | bauzas | righrt | |
| 10:33:42 | bauzas | that's just another indirection | |
| 10:33:56 | johnthetubaguy | the pCPUs are just the NUMA_CORE resources | |
| 10:33:57 | johnthetubaguy | I think | |
| 10:33:58 | bauzas | I just feel for CPU pinning that we need to have an inventory of 1 | |
| 10:34:12 | sudipto | ok | |
| 10:34:17 | bauzas | so you "take" that pCPU | |
| 10:34:21 | johnthetubaguy | you don't care about the particular pCPU though | |
| 10:34:31 | johnthetubaguy | you only care about the NUMA node its on relative to the others you have | |
| 10:34:32 | bauzas | that's a scheduling decision | |
| 10:34:40 | bauzas | the fact is | |
| 10:34:43 | johnthetubaguy | doesn't seem to need the ironic 1:1 treatment | |
| 10:34:52 | bauzas | the current scheduler doesn't choose the ping | |
| 10:34:53 | bauzas | pin | |
| 10:35:00 | bauzas | it leaves the compute picking one pin | |
| 10:35:13 | johnthetubaguy | sure, the new one can pick the NUMA node, doesn't have to pick the pCPU | |
| 10:35:21 | bauzas | it just verifies that the the cell has at least one pin left | |
| 10:35:23 | bauzas | but | |
| 10:35:34 | bauzas | we have a shit ton of corner cases in the current model | |
| 10:35:43 | bauzas | and libvirt is pretty laxist on that | |
| 10:36:12 | bauzas | I just discovered that libvirt is totally okay having two VMs asking the same full excluse access to a pCPU | |
| 10:36:22 | bauzas | exclusive* | |
| 10:36:24 | johnthetubaguy | yep, the key thing is what do you care about, you don't care its pin 7, you care its in the same NUMA_NODE as your other resources | |
| 10:36:39 | bauzas | yeah, but there are situations where we fail | |
| 10:36:57 | bauzas | I'm not a NUMA specialist but I swear having seen problems with that | |
| 10:37:06 | johnthetubaguy | we do today, because we don't lock on the NUMA cell, only on the compute node | |
| 10:37:11 | bauzas | mostly because we don't claim that in the scheduler | |
| 10:37:12 | johnthetubaguy | at least thats my understanding of the issue | |
| 10:37:24 | bauzas | right | |
| 10:37:31 | johnthetubaguy | hence the change above | |
| 10:37:37 | lyarwood | johnthetubaguy / bauzas ; apologies if it's too late to ask but would https://review.openstack.org/#/c/420146/ be something we can squeeze in ahead of the RC1 tag? ./me lost track of it this week with everything else flying around. | |
| 10:37:55 | bauzas | anyway, I think sorting that problem is not trival and could be discussed at PTH | |
| 10:37:57 | bauzas | PTG even | |
| 10:38:36 | johnthetubaguy | the buckets are NUMA_NODEs rather than the computes | |
| 10:38:46 | bauzas | lyarwood: I need to pick my daughter | |
| 10:38:58 | johnthetubaguy | lyarwood: lets take a look, there is always RC2 | |
| 10:39:00 | bauzas | oh I thought RC1 sailed already | |
| 10:39:09 | bauzas | but it's not AFAICT | |
| 10:39:21 | bauzas | lemme verify the releases repo | |
| 10:39:27 | lyarwood | I thought it was pushed to Monday, maybe I'm mistaken, but it can wait | |
| 10:39:27 | johnthetubaguy | yeah, I don't see the tag right now | |
| 10:39:50 | bauzas | yeah https://review.openstack.org/#/c/428531/ | |
| 10:40:02 | johnthetubaguy | oh, this is based on the we don't do releases on fridays because it breaks weekends | |
| 10:40:24 | johnthetubaguy | although this one shouldn't impact others | |
| 10:41:14 | bauzas | all the Depends-On landed | |
| 10:41:29 | bauzas | so nothing prevents https://review.openstack.org/#/c/428531/ to be merged except the -W of course | |
| 10:41:58 | johnthetubaguy | lyarwood: I kinda like this use of os.path.ismount, seems sensible, shame out findmnt | |
| 10:42:07 | bauzas | johnthetubaguy: the only problem with tagging late is that we delay the requirements unfreeze | |
| 10:42:32 | bauzas | honestly, I don't think tagging RC1 requires to wait over monday | |
| 10:42:40 | lyarwood | johnthetubaguy: yeah the regression in findmnt is being addressed but IMHO we just don't need to use it | |
| 10:42:40 | bauzas | but I leave mriedem decide | |
| 10:42:53 | bauzas | oh that reminds me, let me drop a courtesy highlight | |
| 10:43:05 | johnthetubaguy | bauzas: we have tagged on friday before, using the timezone of thursday on some pacific island | |
| 10:43:11 | lyarwood | lol | |
| 10:43:14 | bauzas | \o/ | |
| 10:43:17 | bauzas | problem solved. | |
| 10:43:22 | johnthetubaguy | indeed :) | |
| 10:43:26 | bauzas | anyway, I need to bail out, school time | |
| 10:43:55 | bauzas | I could pretend the ISS is still on Thursday given they cross the world every 1.5h | |