| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-10 | |||
| 16:17:52 | dansmith | mriedem: he's saying the config doesn't currently include the intended behavior | |
| 16:18:10 | dansmith | which is fine, the operator may have to tell the tool what they're using their pinning for, | |
| 16:18:15 | stephenfin | mriedem: To paraphrase a kid with a spoon, there is no trait | |
| 16:18:21 | dansmith | but the actual conversion of the formats does not need to be hand-edited everywhere | |
| 16:18:37 | mriedem | stephenfin: i don't know what that means (the spoon kid thing) but sure there is never a trait unless we add one | |
| 16:18:50 | dansmith | matrix | |
| 16:18:52 | dansmith | can't believe you didn't get a 90s movie reference dude | |
| 16:18:55 | mriedem | my point was, if the control plane needs to know things about how the compute is configured/supported, then we use traits for that now | |
| 16:19:01 | mriedem | i'm not a matrix fanboy | |
| 16:19:27 | stephenfin | mriedem: We don't need a trait though - we have resources | |
| 16:19:34 | dansmith | right, that's not really the problem | |
| 16:20:32 | stephenfin | the problem is that we're going from a world where two different types of resource have been munged together, and we're trying to unmunge them as cleanly as possible | |
| 16:20:32 | dansmith | the controller side of this seems easy to make flexible enough to handle the rolling config of computes to me | |
| 16:21:06 | stephenfin | using service versions? | |
| 16:21:13 | dansmith | I would say that the scheduler should ask for the new format by default. If placement returns some options, then we filter and schedule to those if possible. Basically, prefer the upgraded machines | |
| 16:21:21 | dansmith | if we get back no candidates or filter them all out, | |
| 16:21:25 | sean-k-mooney | we are going form a world where teh VCPU reouse was the number of virtual cpus avialabel to it meaning the number of shared cpus | |
| 16:21:44 | dansmith | we check the service version to determine if there are old computes in the deployment. If so, we query again for the older format to see if there's any room that way | |
| 16:21:49 | artom | So... maybe we need entirely new resource names then? | |
| 16:22:00 | dansmith | potentially cache that determination for ten minutes or something, but... | |
| 16:22:15 | mriedem | artom: moving away from VCPU is a non starter to me | |
| 16:22:20 | dansmith | then you can upgrade and convert computes in one step, which is what OSA and other tools are going to want to do.. | |
| 16:22:22 | mriedem | it's baked into *everything* | |
| 16:22:29 | dansmith | they want to upgrade and fix the config as one step generally | |
| 16:23:11 | artom | mriedem, I'm not saying remove it, I'm saying leave it "legacy CPU resource thing", and come up with new resources to mean shared CPU and dedicated CPU | |
| 16:23:19 | dansmith | artom: -3 | |
| 16:23:58 | artom | Anyways, I have 0 context and func tests to fix | |
| 16:24:29 | sean-k-mooney | artom: we talked about SCPU and PCPU in the past but we said no we want to keep VCPU resouces | |
| 16:24:37 | artom | dansmith, hey man, dinner first | |
| 16:24:42 | sean-k-mooney | so way to late to go back that route | |
| 16:24:42 | stephenfin | dansmith: so tl;dr: kill the static scheduler-only config option and instead do "give me PCPU, but if you can't give me PCPU then search for VCPU" instead | |
| 16:25:14 | dansmith | stephenfin: yes, but only the last part of there are old computes around. once you do that one time and find everything is upgraded, stop even doing that check | |
| 16:25:27 | dansmith | stephenfin: remove that compat step in U, no deprecation cycle needed for that | |
| 16:25:46 | stephenfin | What do you mean by old computes? | |
| 16:25:51 | sean-k-mooney | dansmith: if we did that we would need a trait to make the scond query safe | |
| 16:26:21 | mriedem | stephenfin: older than train computes | |
| 16:26:25 | mriedem | based on the nova-compute service rpc api version | |
| 16:26:26 | dansmith | stephenfin: service version will tell you if all computes have been upgraded.. so actually, maybe just always do that in T if no candidates, because they could do the upgrade and the config tweak separately | |
| 16:26:26 | sean-k-mooney | we would need a support_PCPU capablity trait and need to add it as a forbiden trait for the second query | |
| 16:26:43 | stephenfin | that won't work though | |
| 16:26:49 | dansmith | stephenfin: so yeah, what you said | |
| 16:26:51 | stephenfin | because by default we don't report PCPU on Train | |
| 16:27:08 | stephenfin | doing so would force people to set new config options as soon as they upgrade | |
| 16:27:14 | stephenfin | which we can't do | |
| 16:27:29 | sean-k-mooney | stephenfin: if you look at artom migratio ncode we ignore the "can live migrate with numa" config if everythin is upgraded | |
| 16:27:40 | dansmith | stephenfin: that's why I backed away from the version check there | |
| 16:28:30 | sean-k-mooney | that would still require everything to be restated however. | |
| 16:28:33 | stephenfin | sean-k-mooney: right, but that's because the "can live migrate with numa" check only needs to know that code is new enough. it doesn't need the operator to tweak some config first | |
| 16:28:38 | stephenfin | which this does | |
| 16:28:58 | stephenfin | dansmith: Ah, yeah, I missed the "so actually" | |
| 16:29:35 | dansmith | stephenfin: the reason I was heading in that direction, is because: | |
| 16:29:36 | sean-k-mooney | i think if we were to do the double query as i said we need the compute capableity trait to protect against lading on a new host when old hsot are available and we ask for vcpus | |
| 16:29:41 | mriedem | you'd only do that fallback for VCPU if the flavor in the request spec (in the scheduler) has resources:PCPU=x right? | |
| 16:30:12 | stephenfin | sean-k-mooney: not gonna happen - the NUMATopologyFilter or libvirt driver protect us | |
| 16:30:15 | dansmith | stephenfin: I was going to suggest we also always expose the inventory, even if we have to synthesize it from the older config, but I'm guessing you're going to say we'd potentially expose as shared or dedicated and be wrong for the intention of the operator right? | |
| 16:30:17 | sean-k-mooney | mriedem: no we are translating hw:cpu_policy=dedicated into PCPU requests | |
| 16:30:38 | dansmith | stephenfin: require them to convert their configs before U, but expose the new inventory right away | |
| 16:30:44 | mriedem | ok but my point is, just make sure we're not unconditionally doing that fallback re-query | |
| 16:30:50 | stephenfin | sean-k-mooney: because if you land on a Train compute node, that'll have explicitly set 'NUMATopology.pcpuset' to None | |
| 16:31:11 | dansmith | mriedem: yes, only do the fallback query for PCPU things | |
| 16:31:19 | sean-k-mooney | ok we could have issue with the limit paramater on placement | |
| 16:31:26 | stephenfin | and if that's None, we've got nothing to pin to | |
| 16:31:33 | stephenfin | so placement will pass but the filter will fail | |
| 16:31:40 | sean-k-mooney | but if we do the second query with out a limit then ya the numa toplogy filter would prevent that | |
| 16:31:49 | sean-k-mooney | so we dont need the trait | |
| 16:32:11 | stephenfin | dansmith: Yeah, exactly | |
| 16:32:17 | dansmith | I don't think the limit is a problem | |
| 16:32:31 | dansmith | stephenfin: so they express their intent right now how? | |
| 16:32:33 | sean-k-mooney | dansmith: doesnt cern set it to like 15 | |
| 16:32:43 | stephenfin | dansmith: My previous solution had been to expose CPU inventory on hosts without the new configuration as both PCPU and VCPU | |
| 16:33:09 | dansmith | sean-k-mooney: we're asking placement for a query that will return things with PCPU resources.. if placement returns nothing then there's nothing that will fit | |
| 16:33:10 | sean-k-mooney | the default limit is 1000 so that should not be a proablem but if its really low then i expect it could be | |
| 16:33:13 | dansmith | sean-k-mooney: their problem is different | |
| 16:33:31 | sean-k-mooney | dansmith: im not talking baout the first query for pcpus | |
| 16:33:32 | dansmith | stephenfin: yeah, I don't think that's a good plan | |
| 16:33:42 | sean-k-mooney | dansmith: i ment the second query for VCPUS | |
| 16:33:58 | stephenfin | Yeah, it's really not. I detailed why in that ML post | |
| 16:34:00 | dansmith | sean-k-mooney: I don't see what the problem is | |
| 16:34:09 | sean-k-mooney | the new host can have invetories of both | |
| 16:34:19 | sean-k-mooney | we could get 15 new hosts and no old hosts | |
| 16:34:26 | dansmith | stephenfin: I'm asking about today.. they use this ambiguous config thing.. how do they control which type land where? | |
| 16:34:28 | sean-k-mooney | then the numa toplogy filter would elinate all hosts | |
| 16:35:06 | stephenfin | the scheduler option and the NUMATopologyFilter | |
| 16:35:44 | dansmith | stephenfin: meaning static config to determine how to treat pinned cpu requests? | |
| 16:36:04 | stephenfin | If the scheduler option is set to False (so it's not translating 'hw:cpu_policy=dedicated' to 'resources:PCPU') then we'll keep requesting VCPU from placement | |
| 16:36:09 | dansmith | no, | |
| 16:36:12 | dansmith | I'm asking about TODAY | |
| 16:36:19 | stephenfin | oh, today | |
| 16:36:21 | dansmith | not the mythical future where your patches are landed | |
| 16:36:23 | stephenfin | gotcha | |
| 16:36:25 | stephenfin | host aggregates | |
| 16:36:28 | dansmith | right, okay | |
| 16:36:31 | stephenfin | and metadata | |
| 16:36:36 | dansmith | that's what I thought | |
| 16:36:46 | stephenfin | that's what you're _supposed_ to use, of course. We never enforced it | |
| 16:37:01 | dansmith | so the problem is that computes literally don't have access to the information they need to know what kind of thing to expose, because they have config, but no access to the aggregate info | |
| 16:37:08 | sean-k-mooney | and windrieve developed a host agent to allow you to mix | |
| 16:37:18 | sean-k-mooney | so they exploited that we did nto enforece it | |
| 16:37:19 | stephenfin | correct | |
| 16:37:29 | dansmith | stephenfin: so, if we defaulted to the looser interpretation of the inventory, | |