| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-23 | |||
| 13:45:46 | fried_rice | giblet: That's correct, but it also shouldn't need to. | |
| 13:45:51 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: RP delete inventories (v1.5) https://review.openstack.org/514642 | |
| 13:45:51 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: RP list: member_of and resources parameters (v1.3, v1.4) https://review.openstack.org/511183 | |
| 13:45:52 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: CLI for traits (v1.6) https://review.openstack.org/514643 | |
| 13:45:53 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: Resource class set (v1.7) https://review.openstack.org/514644 | |
| 13:45:54 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: CLI allocation candidates (v1.10) https://review.openstack.org/514647 | |
| 13:45:54 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: Usages per project and user (v1.8, v1.9) https://review.openstack.org/514646 | |
| 13:45:55 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: New dict format of allocations (v1.11, v1.12) https://review.openstack.org/542819 | |
| 13:45:55 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: Transactionally update allocations (v1.13) https://review.openstack.org/546674 | |
| 13:45:56 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: Add nested resource providers (v1.14) https://review.openstack.org/546675 | |
| 13:45:57 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: Limit allocation candidates (v1.15, v1.16) https://review.openstack.org/548043 | |
| 13:45:57 | openstackgerrit | Bence Romsics proposed openstack/osc-placement master: Allocation candidates parameter: required (v1.17) https://review.openstack.org/548326 | |
| 13:46:39 | fried_rice | giblet: Because if you have two request groups that otherwise look the same, then it shouldn't matter which of those two pieces of the response you "use" for which. | |
| 13:46:45 | kashyap | superdan: BTW, thanks for the validation question, that prompted me to file the bug that I needed to, for the new libvirt API: https://review.openstack.org/#/c/534384/10/nova/conf/libvirt.py | |
| 13:46:52 | giblet | leakypipes, fried_rice: port1 and port2 both needs BW resource, an allocation candidate will have RP - BW pairs for both port1 and port2 but there is no way to tell which pair maps to which port | |
| 13:46:56 | kashyap | superdan: And the libvirt dev already assigned it to himself and working on it :-) | |
| 13:47:07 | fried_rice | But if the request groups were differentiated by, say, a custom trait indicating a physical network, then those traits will be in the provider summary, and you can use that to distinguish. | |
| 13:47:07 | superdan | kashyap: cool | |
| 13:47:32 | kashyap | superdan: Appreciate the eagle eyes. | |
| 13:47:40 | giblet | fried_rice: true, the response is not ambigous. It is just haaard to make a generic code that does the mapping | |
| 13:47:46 | leakypipes | giblet: the provider_summaries section of the allocation candidates response contains all the trait and usage information that callers would need to decide that mapping, though. | |
| 13:47:49 | fried_rice | giblet: Well, you'd have to do the math, yeah. | |
| 13:48:05 | fried_rice | which I agree is awkward | |
| 13:48:14 | giblet | leakypipes, fried_rice: thanks, I understand now | |
| 13:48:20 | giblet | a bit better... | |
| 13:49:02 | fried_rice | bauwser: As we've currently got things conceived, if you want to do a don't-care-about-NUMA-affinity resource request on a system that's set up for NUMA, you should split your CPU resources into separate request groups. | |
| 13:49:40 | fried_rice | bauwser: It's awkward, but without further invention, it's the only way to guarantee that you'd be able to get a result spread across multiple nodes if you need. | |
| 13:49:43 | bauwser | fried_rice: not sure I understand the latter | |
| 13:50:09 | bauwser | WDYM by splitting your resources into separate groups ? | |
| 13:50:31 | kashyap | superdan: Do you absolutely want me to remove the white space change? https://review.openstack.org/#/c/534384/10/nova/virt/libvirt/driver.py | |
| 13:50:32 | fried_rice | bauwser: Instead of resources=VCPU:2, which would always give you both VCPUs from the same provider, you would have to say resources1=VCPU:1&resources2=VCPU:1 | |
| 13:50:41 | superdan | kashyap: I absolutely do | |
| 13:50:56 | kashyap | superdan: :-) Did it to keep the conditionals in line. But where is the next chance to "fix it"? | |
| 13:50:59 | bauwser | fried_rice: that looks a terrible upgrade impact for operators IMHO | |
| 13:51:09 | kashyap | A separate change just to do that is overkill | |
| 13:51:14 | superdan | kashyap: it's not worth fixing, IMHO | |
| 13:51:28 | kashyap | Okay, I'll curb my OCD there | |
| 13:51:33 | bauwser | as an operator, I just want to provide flavors, either for specific NUMA calls, or just for asking resources | |
| 13:51:34 | leakypipes | bauwser, giblet, fried_rice: I'm almost done reworking the cpu-resources spec with all the feedback from yesterday's 6-hour sage IRC conversation. Gimme about an hour and the new spec should answer a number of your questions. | |
| 13:51:34 | fried_rice | bauwser: Again, without further invention, that's the way it's got to be. Think about the implications if we supported splitting a single VCPU:2 across multiple RPs. | |
| 13:52:01 | bauwser | fried_rice: we haven't modeled NUMA yet, hence my questions | |
| 13:52:11 | sean-k-mooney[m] | Fried_rice thats concering as if i have a gust that can't fit on a single numa node i cant boot that anymore if i dont request a multi numa topology | |
| 13:52:27 | bauwser | fried_rice: if we go sharding the resources between NUMA nodes, then that's a terrible operator impact IMHO | |
| 13:52:31 | fried_rice | bauwser: The example I gave was DISK_GB. If I say resources=DISK_GB:1024, but my compute host has multiple storage RPs (including shared), I definitely don't want GET /a_c to return me candidates where those DISK_GB are split across multiple. | |
| 13:52:35 | bauwser | that needs at least to be properly documented | |
| 13:53:08 | bauwser | fried_rice: again, I understand nested RPs | |
| 13:53:15 | bauwser | fried_rice: so, yeah I agree | |
| 13:53:22 | fried_rice | bauwser: That goes for nested and sharing alike. | |
| 13:53:35 | bauwser | fried_rice: if you shard your resources between separate nested children, then it's understandable | |
| 13:53:45 | bauwser | fried_rice: but here I'm talking of NUMA | |
| 13:53:50 | sahid | leakypipes: mriedem, thanks for your efforts on reviewing the trusted vfs specs | |
| 13:53:53 | fried_rice | bauwser: I don't disagree that this is awkward. But we would have to come up with some new semantic to allow spreading. resources=VCPU:2&allow_spread=resources | |
| 13:53:57 | fried_rice | (ew) | |
| 13:54:16 | bauwser | that looks to me the default behaviour to have IMHO | |
| 13:54:26 | bauwser | by default, spread between NUMA nodes | |
| 13:54:26 | fried_rice | bauwser: You're talking about a user wanting to get a non-NUMA-affined instance on a NUMA-modeled compute host. | |
| 13:54:46 | fried_rice | bauwser: Unless we start special-casing by resource class, no can do. Because of the DISK_GB example. | |
| 13:54:53 | bauwser | fried_rice: I'm talking of a user using a flavor that doesn't ask for NUMA resources, just vCPUs, yeah | |
| 13:55:07 | fried_rice | ...which would be an issue *today* (whereas NUMA is a tomorrow thing) | |
| 13:55:08 | sean-k-mooney[m] | bauwser: fried_rice this will be a behaviour change form today. Today we only numa affites if you ask for it | |
| 13:55:19 | bauwser | what sean-k-mooney[m] wrote | |
| 13:55:28 | bauwser | that would be a terrible upgrade impact | |
| 13:55:30 | mriedem | sahid: np, i left some comments in the 2nd patch in the code series | |
| 13:55:56 | bauwser | hence why I originally modeled my spec with vCPU resources being on the root RP | |
| 13:56:09 | bauwser | and only NUMA specific resources being on the children | |
| 13:56:16 | bauwser | to keep existing behaviour compatible | |
| 13:56:33 | fried_rice | bauwser: Okay, but then we would need some way to decrement the NUMA resources when the top-level resources are requested/consumed. | |
| 13:56:37 | sahid | mriedem: ok i will address them, i also have noticed your comment on the first patch | |
| 13:56:59 | bauwser | fried_rice: that's a separate NUMA resource class, see my spec | |
| 13:57:05 | bauwser | the terms are maybe not good | |
| 13:57:24 | leakypipes | sahid: np. thanks for your patience. | |
| 13:57:35 | leakypipes | sahid: I should have the cpu-resources next revision up shortly for your review. | |
| 13:57:50 | bauwser | but if I'm asking for resources:VCPU=2,NUMA_VCPU=2 then it'll decrement both | |
| 13:58:06 | sahid | leakypipes: ok, sure i will | |
| 13:58:13 | sahid | review it | |
| 13:58:17 | sean-k-mooney[m] | Well i think we need to have vcpus under numa but i think we will need something likek a grouping level attribute that defaults to host. E. G. Host, numa, rp grouping | |
| 13:58:21 | fried_rice | bauwser: yup. But how is that better than asking for resources1:VCPU:1&resources2=VCPU:1 | |
| 13:59:00 | bauwser | fried_rice: because existing flavors will still work for non-NUMA queries | |
| 13:59:03 | bauwser | that's all the difference | |
| 13:59:25 | sean-k-mooney[m] | Fried_rice how do you know how to create the groups | |
| 13:59:29 | bauwser | here, you're asking to change all the operator's flavor to explicitly tell whether they care about NUMA sharding or not | |
| 13:59:31 | fried_rice | bauwser: Existing flavors have resources:VCPU=2,NUMA_VCPU=2 ? | |
| 13:59:46 | bauwser | existing flavors just have VCPU resources | |
| 13:59:50 | fried_rice | right | |
| 14:00:11 | figleaf | fried_rice: how could you possibly know which NUMA nodes to decrement resources from? | |
| 14:00:15 | bauwser | like I wrote, we can have Nova to ask for both resource classes if it sees that | |
| 14:00:23 | fried_rice | So look, there are ways to handle this with and without changing the placement API | |
| 14:00:33 | fried_rice | yeah, that'd be one way. | |
| 14:00:47 | sean-k-mooney[m] | Existing flavours without numa_nodes set dont have numa affinity for cpus | |
| 14:00:53 | fried_rice | The other way is to have nova see that the flavor doesn't care about NUMA affinity and *nova* splits up the resource requests | |
| 14:00:56 | bauwser | I mean, if a flavor tells only about vCPUs, then query Placement only for that, but make allocations for both VCPU and NUMA_SOMETHING_VCPU | |
| 14:01:11 | bauwser | fried_rice: that's what I wrote in the spec | |
| 14:01:16 | bauwser | nova could make the translation | |
| 14:01:38 | bauwser | leakypipes: ^ I hope you follow | |
| 14:01:46 | fried_rice | bauwser: Cool; either way, we're having nova massage the request. But I prefer the path where we're actually representing true inventory numbers. | |
| 14:01:48 | sean-k-mooney[m] | Fried_rice how does nova know how to split it up | |
| 14:02:12 | fried_rice | sean-k-mooney[m]: For VCPU, it's one per request group, plain and simple. | |
| 14:02:13 | bauwser | sean-k-mooney[m]: there are two different things, the placement query and the allocation | |
| 14:02:39 | bauwser | fried_rice: also, note I'd like to not depend on request groups for NUMA | |
| 14:02:48 | fried_rice | bauwser: Whyzat? | |