| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-01-10 | |||
| 14:08:06 | efried | sean-k-mooney What do you call that root node? | |
| 14:08:09 | efried | Is it a host? | |
| 14:08:21 | efried | a "cloud"? | |
| 14:08:50 | efried | some nebulous as-yet-unnamed entity that exists solely as an anchor point for this multi-owner provider tree? | |
| 14:09:06 | cdent | sean-k-mooney: That's kind of related to why I disputed that root provider be a thing in the data struture or representation. I think we should be able to access a tree of providers anywhere in whatever trees people like, and declare subtrees to be whole if that's what suits them. That is, if we're gonna have trees, let's have _trees_. | |
| 14:09:06 | efried | ...but doesn't correspond to anything in the real world? | |
| 14:09:15 | cdent | (I'd rather just not have trees, but that ship sailed) | |
| 14:11:16 | sean-k-mooney | efried: in the case of a nova created tree i was assuming a compute node. e.g. a host yes | |
| 14:12:10 | efried | sean-k-mooney Right, I was referring to what you said earlier about "i dont think nova should own the root node of the tree" and "the root node [is] something each service can create subtree from" | |
| 14:12:45 | sean-k-mooney | yep i think we are violently agreeing on that point :) | |
| 14:12:46 | mgoddard | efried: so with your change, will a generation change in the provider tree cause inventory, traits, and aggregates to be updated, and checked that all those responses contain the same generation? | |
| 14:12:49 | efried | In that picture, who owns the root node, and what does it represent? | |
| 14:14:01 | sean-k-mooney | efried: everyone owns it. i think we just need to agree on how the root node is create for example by saying that the node is idenfied by the host_id which default to the hostname | |
| 14:14:29 | efried | sean-k-mooney But if the root node represents a compute host, doesn't it make sense for compute to own/create it? | |
| 14:14:46 | sean-k-mooney | efried: not in a converged deployment | |
| 14:15:25 | sean-k-mooney | for example if you have cinder running on the same physical server then its also a storage node not just a compute node | |
| 14:16:42 | sean-k-mooney | if cinder and nova agree on how to create/select that root node then can boot create subtrees for that phyical server | |
| 14:16:49 | efried | mgoddard At the report client level, the plan for set_traits_for_provider is: If PUT /rp/{uuid}/traits succeeds (200), I'll update_traits on the ProviderTree, setting the generation based on what's in the PUT response. (The generation is an attribute of the provider, so updating it via update_traits updates it for anything else associated with that provider.) | |
| 14:17:33 | efried | mgoddard If the PUT fails 409, set_traits_for_provider will raise (a subclass of) this new conflict exception. | |
| 14:17:34 | sean-k-mooney | s/ then can boot/ then both can/ | |
| 14:18:39 | efried | mgoddard The report client consumer (i.e. resource tracker) should then redrive the overarching operation, which should entail first asking report client to refresh its cached representation of that provider in the ProviderTree. | |
| 14:19:11 | efried | mgoddard Which means reGETting the provider and all its associated bits (traits, aggs, etc.) and calling the appropriate ProviderTree methods to update them all. | |
| 14:19:53 | efried | mgoddard Then the consumer (resource tracker) would make whatever change again (in this example setting the traits). Rinse, repeat. | |
| 14:20:14 | efried | mgoddard Not sure if I'm explaining this particularly well. | |
| 14:20:35 | sean-k-mooney | efried: cdent anyway that when a little off topic but i think one design would be for neutorn to create its resouces under the numa nodes create by nova and add the resouces it creates to an aggreate using the neutron agent uuid as the aggreate uuid. | |
| 14:21:00 | efried | sean-k-mooney Why do you need the aggregate in that picture? | |
| 14:21:11 | efried | sean-k-mooney Are the neutron resources common to multiple hosts? | |
| 14:21:41 | efried | sean-k-mooney If so, they should *not* be created as providers in the tree under the numa nodes. | |
| 14:22:00 | sean-k-mooney | efried: no they are specific to that compute but i want a simple way to look up all the resocurce providers created by the neutron agent on that host | |
| 14:22:48 | efried | sean-k-mooney Ahhh, that makes sense. | |
| 14:23:16 | efried | sean-k-mooney So the aggregate is really just saying "neutron owns these". It's not associating the providers with any shared providers. | |
| 14:23:42 | efried | sean-k-mooney Which is a crucial distinction, because you wouldn't want an allocation candidate request to bleed across compute trees. | |
| 14:24:08 | efried | sean-k-mooney So in this scenario, the providers are associated with an aggregate, but nobody in that aggregate has the MISC_SHARES_VIA_AGGREGATE trait. | |
| 14:25:06 | sean-k-mooney | efried: yes exactly | |
| 14:26:10 | efried | mgoddard But note that report client is the pinch point for everything. You can't directly change the report client's cached ProviderTree from outside the report client. Conversely and more importantly, you can't effect a change to placement by changing that cached ProviderTree. You can only do that by calling report client methods like set_*_on_provider etc. | |
| 14:27:13 | efried | mgoddard Where it gets confusing is that ComputeDriver.update_provider_tree *sounds* like it's doing just that (effecting a change to placement by changing the cached ProviderTree). | |
| 14:27:22 | efried | mgoddard Which is almost but not quite the case. | |
| 14:28:26 | efried | mgoddard RT will ask RC for the provider tree; RC will return a *copy*. RT will then ask ComputeDriver to update_provider_tree. RT will then ask RC to diff and flush any changes back to placement, which will update RC's cached ProviderTree in the process. | |
| 14:30:01 | efried | sean-k-mooney Is there any reason that agg ID has to be the same across all the computes for which that neutron instance is handling the network providers? | |
| 14:30:29 | efried | sean-k-mooney I guess I can see the benefits either way. | |
| 14:30:58 | sean-k-mooney | efried: in this case i was assuming the aggregate would not span multiple computes | |
| 14:31:16 | openstackgerrit | Mark Goddard proposed openstack/nova master: Add support to scheduler client for setting traits https://review.openstack.org/532539 | |
| 14:31:32 | sean-k-mooney | i am considering if aggregates can be used to group resouce providres in a singel compute that are releated in some way | |
| 14:32:42 | sean-k-mooney | efried: do you rememebr https://review.openstack.org/#/c/502306/14/specs/queens/approved/bandwidth-resource-provider.rst effectivly im wondering how best to remove the agent resouce providers so that we can also model numa affinity of networking resources | |
| 14:32:54 | mgoddard | efried: Ah, I think I see now. We can only get a 200 back if no other changes have occurred other than those we just PUT, therefore our provider tree must be up to date. Thanks for explaining | |
| 14:33:05 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Collocate encryptor and volume driver calls https://review.openstack.org/460243 | |
| 14:33:08 | lyarwood | mdbooth / stephenfin ; ^ one final respin with the comment and nits taken care of | |
| 14:36:00 | efried | mgoddard Just so. Given that report client is the pinch point for placement in nova, and there's only one report client, and it's running on one compute node, under only one thread, we shouldn't actually see these conflicts if nova is the only thingy managing all the providers. But as you can see from the conversation above, there are plans to potentially have multiple sources of ownership for a given provider. | |
| 14:36:56 | sean-k-mooney | efried: https://review.openstack.org/#/c/502306/14/specs/queens/approved/bandwidth-resource-provider.rst was assumign numa would either be a trait or we would have multiple parents. i like haveing numa nodes be resouce provider but that means neutron agent cant be if tehre sub resouce have numa affinity. | |
| 14:37:23 | efried | mgoddard BTW, I put a * above: today aggregates aren't included in the generation thing. cdent is working on fixing that. It's unclear whether that's going to get deferred to Rocky. IMO sean-k-mooney just described a non-sharing-RP use case where we need it. | |
| 14:38:00 | efried | sean-k-mooney Multiple parents, eh? How so? | |
| 14:38:22 | efried | sean-k-mooney I *think* the best way to model NUMA affinity is going to be using a numbered group with granular syntax. | |
| 14:38:52 | mriedem | yikun: if you wanted to start looking at something, this part of the spec hasn't been started https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/neutron-new-port-binding-api.html | |
| 14:38:57 | mriedem | yikun: "Prior to the RPC call from live_migration on the source host to pre_live_migration on the dest host, start a wait thread for the vif-plugged event from Neutron, similar to during initial spawn." | |
| 14:39:43 | sean-k-mooney | efried: if resouce are allowed to have 1+ parents tehn one parent is the numa node and the other the neutron agent but that makes the trees a graph which we dont whant in general | |
| 14:41:44 | ameeda | anyone can help me to write functional regression test for this review.openstack.org/#/c/526900/ ? | |
| 14:42:34 | efried | sean-k-mooney Agree. The tree should be a tree (to quote cdent), the neutron agent can control the providers (at least the inventories thereof, if not the RP records themselves), and we can use aggregates to "tag" those providers as being controlled thusly. | |
| 14:42:46 | efried | sean-k-mooney An alternative approach for such tagging would be to use a trait. | |
| 14:43:16 | sean-k-mooney | efried: ya that was the other thing i was going to explore | |
| 14:43:31 | efried | sean-k-mooney Which might engender less confusion wrt sharing providers (which these ain't) as well as remove the requirement for the agg generation thing cdent is working. | |
| 14:43:36 | sean-k-mooney | e.g. CUSTOM_AGENT_<uuid goes here> | |
| 14:43:42 | cdent | efried: you're welcome to quote me efried, but you're taking me out of context. the implication i was trying to make there is that the trees should be big and anything can be a root | |
| 14:44:07 | efried | cdent Sorry, I take it back. Just didn't want to plagiarize :) | |
| 14:44:49 | efried | sean-k-mooney Or just CUSTOM_MANAGED_BY_NEUTRON. Why would you need a UUID in the trait? | |
| 14:44:51 | cdent | my utterances here are cca | |
| 14:45:16 | efried | I'll quote you on that. | |
| 14:45:23 | efried | (but not give you credit) | |
| 14:45:30 | sean-k-mooney | efried: again so i have a singel api call i can make to retrive all resouce provders that are managed by that agent | |
| 14:46:12 | sean-k-mooney | both aggreates via memberof and traits with the uuid give me that within the existing model | |
| 14:46:23 | efried | sean-k-mooney Isn't the scope of one neutron in this scenario the same as the scope of one placement service? | |
| 14:47:21 | sean-k-mooney | one placement service? is someone suggesting there would be more then one placement service? | |
| 14:47:42 | cdent | efried: sean's assertion was neutron agent, not neutron | |
| 14:48:01 | sean-k-mooney | oh yes what cdent said | |
| 14:48:03 | cdent | sean-k-mooney: in the dark corners of the universe there has been talk of nested placement services | |
| 14:48:18 | efried | (cdent that's not what I was talking about) | |
| 14:48:32 | efried | cdent sean-k-mooney In that case, there's one neutron agent per compute host, yah? | |
| 14:48:39 | sean-k-mooney | cdent: beyond one per cell that sound like i need something stronger then the coffee im drinking | |
| 14:49:00 | cdent | sean-k-mooney: yeah, it's the sort of thing where you should keep _me_ in drinks in dublin to not raise again | |
| 14:49:33 | sean-k-mooney | efried: actully you could have several per compute node. e.g. sriov + ovs on the same node though in practice yes | |
| 14:50:06 | efried | sean-k-mooney But not one agent for multiple computes. | |
| 14:50:40 | efried | sean-k-mooney So my point is, you can ask for providers (having CUSTOM_MANAGED_BY_NEUTRON) && (in tree <compute RP UUID>) | |
| 14:50:50 | sean-k-mooney | efried: :( well for agent based neutron no. but this is odl.... | |
| 14:50:59 | sean-k-mooney | * there is | |
| 14:51:29 | efried | cdent I thought GET /resource_providers had a queryparam for "having traits". I don't see it at a glance. | |
| 14:52:38 | cdent | i'm not sure that got merged (yet) | |
| 14:52:46 | cdent | I do think code to do it somewhere though | |
| 14:52:55 | efried | cdent Oh, okay, in flight | |
| 14:53:26 | efried | It would be somewhere in that fabulous placement update email summary... | |
| 14:54:08 | mriedem | alex_xu: i'm still +2 on https://review.openstack.org/#/c/330406/ - i think the migration_links thing is correct; it's consistent with other APIs that support paging. you have a good point about the changes-since before 2.59 though, but that could be addressed in a follow up. | |
| 14:54:18 | mriedem | since it assumes people would actually do that | |
| 14:54:34 | sean-k-mooney | efried: worst comes worse neuron can jsut get teh whole tree and walk it to see if the inventries it creates exist or not. | |
| 14:54:51 | cdent | efried: I think it is something that alex_xu was working on but a lot of his trait related stuff got abandoned | |
| 14:55:31 | efried | cdent Yah, I can't find any such thing in open state (assuming it would have 'trait' somewhere in the title/description) | |
| 14:56:30 | efried | mgoddard So at a glance, it looks like what you've done here is modeled after the inventory updating stuff. | |
| 14:56:55 | mgoddard | efried: Correct | |
| 14:57:00 | cdent | efriend I suspect that quite a few query style things, on /resource_providers, got dropped when /allocation_candidates took the focus, especially if the use cases on /resource_providers werent yet fully formed | |
| 14:57:42 | efried | mgoddard I haven't fully synthesized this stance yet, but I *think* I'm going to come to the conclusion that that's unnecessarily complicated (even for inventory) - and even incorrect in that it does retries at this low level rather than at the consumer level. | |
| 14:59:31 | efried | cdent GET /resource_providers?having_all=T1,T2 and/or ?having_any=T3,T4 seems like a fairly natural thing to expect, but of course there needs to be a real use case for it. What sean-k-mooney described could count as such. | |
| 14:59:52 | sean-k-mooney | efried: one of the other issue is that nova uses the nova compute node uuid of the host_id(hostname by default) which is sotre in the name filed of the compute node RP i think so we cant uses in_tree in this case and need to use name | |
| 15:00:23 | efried | I believe in_tree accepts name or UUID, doesn't it? | |