Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-09-28
21:45:05 cdent but even if inventory is not consume at the top we must have the resource provider uuid of the compute node so we know what the destination is
21:45:37 edleafe we are always returning root RPs, right?
21:45:47 cdent so even if the allocation part of the allocation candiate doesn’t consumer, there will be a record in the other half of the tuple which has the roots
21:45:59 edleafe IOW, we'd never return a NIC by itself
21:46:04 cdent edleafe: i think so, yes, at least in the resource providers half of the tuple
21:46:06 efried More recently we've talked about modeling NUMA by making the NUMA nodes child RPs and having them provide the CPU/MEM(/possibly-other-things-like-PFs) resources.
21:46:17 cdent efried: right, but they are still a child
21:46:32 efried Well, okay, but if the tree is three deep, the root RP may still not have the traits the leaf is inheriting.
21:46:32 cdent so you could alloc against them, but still _place_ on the parent
21:46:52 cdent I think inheritance is a very bad way of thinking of this
21:46:59 cdent and that’s what my comment was inspired by
21:47:04 cdent (on the review)
21:48:07 cdent which spec has allocation candidates?
21:48:38 edleafe cdent: yeah, this is more like composition than inheritance
21:49:00 cdent ah, it’s got the wrong name: http://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/placement-allocation-requests.html
21:49:25 dansmith gah
21:49:31 dansmith does anyone else get stopped in devstack doing this?
21:49:31 dansmith 403 Forbidden: You are not authorized to complete publicize_image action. (HTTP 403)
21:49:50 dansmith must be residue leftover from my previous install, but I don't know what it is
21:50:35 cdent efried: in there, provider_summaries is the second half of the tuple, contains the rp info. in there will be represented the info to construct a nested provider hierarchy, I don’t know if that’s defined yet
21:50:56 cdent efried: but in that a trait will be on the thing to which the trait was associated, not its children
21:51:30 efried cdent You could totally return *just* the RPs that are being claimed against, and the scheduler would have to use the root RP UUID to pull the whole tree from placement at that point.
21:52:10 efried Or you could return the whole tree, and the scheduler would not have to do that, but there would be empty "allocations".
21:52:11 cdent you could, yes, but I don’t think that was the plan. I suggested at one point we should just return uuids and require the client to go back for more info if it wanted
21:52:26 efried Or you could return just the claimed RPs in the allocations, but return the whole RP trees in the provider summaries.
21:52:26 cdent why would there be empty allocations?
21:52:51 cdent the structure in provider_summaries is not directly mapped to what is in allocations
21:52:54 mriedem dansmith: making an image public?
21:53:06 mriedem but devstack uploads the image...
21:53:07 dansmith mriedem: that's just during stack
21:53:43 efried dansmith Could be perms in your configured temp dir, which glance uses to store a copy of the image while it's uploading it.
21:54:02 dansmith efried: I dunno what would have changed from one stack run to another
21:54:07 cdent efried: so I’m still confused about how/why you want inheritance to be a thing
21:54:33 efried cdent Lemme find that conversation. It's in the same spec, same area, earlier rev...
21:54:34 dansmith mriedem: efried: this is a centos box and my best guess is that clean.sh doesn't clean something for redhat systems like it does for ubuntu or something
21:54:41 dansmith guess what I'm doing right now?
21:55:05 mriedem umm
21:55:23 efried cdent PS6
21:55:37 efried cdent There's an example there
21:56:30 efried cdent And more background in PS4
21:56:40 efried ...which is where jaypipes actually provided the example
21:57:55 cdent that’s no inherited, the trait still matched on the compute rp, we _also_ got numa node 1 because we required magic cache thingy
21:58:31 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove old compat code from servers ViewBuilder._get_metadata https://review.openstack.org/508326
21:58:52 cdent the compute rp is a member of the results
21:59:02 efried cdent But with no inventory.
21:59:11 efried no inventory consumed* that is
21:59:35 cdent if it were not a member of the results, then that would be inheritance
22:00:05 efried But we need to be crisp about this because when we start doing the "grouping" thing, it's going to be important to be able to say that you *don't* get results that *don't* have your traits for a given group.
22:00:08 efried I said that horribly.
22:01:24 efried In this example, the NUMA node and the compute node are *different* RPs.
22:02:01 cdent I agree that we need to be crisp, which is why I’m disputing the use of the term “inherited” it’s misleading
22:02:03 efried If I ask for inventory on a resource class, I should only get a RP that has inventory in that resource class. If I ask for a required trait, I should only get a RP that has that trait.
22:03:10 efried In this case, I'm asking for e.g. CPU inventory, which belongs to the NUMA node (not the compute host). And I'm asking for trait HW_CPU_MAGIC_CACHE_THINGY, which belongs to the NUMA node (not the compute host). So far so good.
22:03:25 efried But then I'm *also* asking for trait HW_CPU_X86_SSE2.
22:03:38 efried That trait belongs to the compute RP but *not* the NUMA node RP.
22:04:10 efried So by the rules, with no "inheritance" (or whatever you wanna call it), that request would kick out the NUMA node RP because the NUMA node RP doesn't have the HW_CPU_X86_SSE2 trait.
22:04:59 efried If we want to do it that way, it's possible, but it means the implementor who models the RPs would have to *duplicate* the HW_CPU_X86_SSE2 trait (and all those other CPUey traits) onto each NUMA node RP.
22:05:24 efried jaypipes put a stake in the ground and said we could inherit downward, which is why we needed to make this statement in the spec.
22:06:09 efried Based on your confusion, though, it seems like we would have done well to include the example, as well as listing the no-inheritance-but-duplication thing in Alternatives.
22:07:23 cdent I guess maybe I’m too embedded but the statement is a) obvious, b) not inheritance. It is simply that you “got” the compute node, because you asked for that trait
22:07:41 cdent it’s not that the numa node has that trait by inheritance
22:07:44 efried But I don't want the compute nod.
22:07:46 efried node
22:07:48 cdent you do
22:07:52 cdent you _have_ to have the compute node
22:07:53 efried I'm not claiming any inventory off of it.
22:07:55 cdent it is your destination
22:08:25 cdent it is the place where your inventory is
22:08:27 efried Meh, that's kind of an artifact of the root RP happening to be a compute node.
22:08:42 efried cdent Ah, so wait, that's where we're disagreeing I think.
22:08:51 efried The compute node RP is *not* where your inventory is.
22:08:58 efried The NUMA node RP is where your inventory is.
22:09:31 efried Unless you're saying the compute node inherits (accumulates? composes?) the inventory from its descendants.
22:09:33 cdent which is “within” the copute node, by representation (even if it happens to be only logically so)
22:10:14 cdent nested providers are nested
22:10:38 cdent and they are generally created when a compute node does get_inventory
22:10:56 efried Right. So inventory is implicitly crossing RP boundaries (you don't like the word "inheritance"; fine; pick another word). The disputed statement is saying traits do the same.
22:11:19 efried Also trying to think not totally compute-node-is-the-root-RP-centric
22:11:27 cdent I’m saying it’s not relevant when requesting allocation_candidates to know that
22:11:40 cdent (that being “traits compose”)
22:13:08 efried I'm coming from, "it is relevant, both from the perspective of the requestor and the perspective of the implementor of the RP models being looked at by the allocation_candidates API." Let me think through it again...
22:14:28 cdent If we’re trying to be crispy, then the crispiness I’m trying to go for in the example is that we shouldn’t say “the numa node is HW_CPU_X86_SSE2” when it is the parent that has that trait
22:15:30 efried Then we would have to duplicate the trait on each NUMA node.
22:15:59 cdent that’s what I’m not understanding. How are you reaching that conclusion?
22:16:33 efried In this case I'm assuming the resources themselves are owned by the NUMA node RPs. Not by the compute node RP.
22:16:43 efried GET /allocation_candidates?resources=VCPU:1,required=HW_CPU_X86_SSE2,HW_CPU_MAGIC_CACHE_THINGY
22:17:51 efried I haven't gotten the impression that we're supposed to return allocations that satisfy any permutation of that request. I've gotten the impression we're supposed to return allocations that satisfy *all* of those criteria.
22:18:13 cdent well, not to draw us off into the weeds too far, but if you’re going to put the VCPU inventory on a numa node, then presumably you put the hw traits of cpus on the numa node too?
22:18:26 cdent ignore that
22:18:30 efried Ah, that's the point of contention. jaypipes said (in PS4)... okay.
22:19:00 cdent I agree that we should only satisfy all those criteria
22:19:44 efried Okay. Then as modeled, without "inheritance", we would get nada back from that request.
22:20:00 efried because there isn't a single resource provider that satisfies all of it.
22:20:21 cdent so perhaps you are using the term “inheritance” to mean something like “which direction in the resource provider tree should would look to see if a trait is available”?
22:20:28 cdent up, down, both
22:20:31 jaypipes efried: I also said in PS4 that I disagreed with you calling it inheritance. :)
22:20:31 cdent and you’re saying only up
22:20:53 cdent if “only up” then cool, but don’t use the term inheritiance :)
22:20:56 efried bah, let's come up with a word for it that isn't "inheritance" so we can stop getting distracted by that. What do we call it.
22:21:08 jaypipes propagation.

Earlier   Later