| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-01-10 | |||
| 13:51:48 | cdent | sean-k-mooney: there's some stuff in the very early rp specs about such things | |
| 13:51:49 | efried | sean-k-mooney There shouldn't be any such thing as an "unused" aggregate UUID. Unless you mean "used only for one provider", which is pointless. | |
| 13:52:14 | cdent | but if I remember right, the thinking was that the client would already have some identifier in min | |
| 13:52:15 | cdent | d | |
| 13:52:30 | efried | ...because when you remove the last association, placement ought to get rid of that agg ID. | |
| 13:53:02 | efried | cdent Yeah, that's a weird one. In the PowerVM SSP case, the only thing that makes sense is to give the agg the same UUID as the shared storage pool | |
| 13:53:22 | efried | Though I *suppose* I could give the agg the UUID of the cluster instead. It's effectively the same thing in powervm land. | |
| 13:53:43 | sean-k-mooney | efried: well what i mean is if i am createing a set of recouse providers and i want to make them part of a aggregate to group them i need to choose a uuid that is not useed by anyone else. | |
| 13:54:30 | cdent | placement does not clean up the unused agg uuids. that was an early design decision, sort of resulting from how the tables were being normalized to not use uuid keys (and a few others things, it's so long ago) | |
| 13:54:32 | efried | sean-k-mooney Yes, which one would normally do by generating one randomly, BUT that breaks down quickly if you have more than one point of control for that agg. | |
| 13:54:45 | efried | cdent Whoah, so how do they get cleaned up? | |
| 13:54:58 | cdent | they don't | |
| 13:55:02 | cdent | at least not last I checked | |
| 13:55:03 | efried | They just... leak? | |
| 13:55:04 | mgoddard | efried: I have some concerns with the storage of the generation in the provider tree | |
| 13:55:06 | mgoddard | efried: there is a single generation per-provider that covers inventory, traits, and aggregates | |
| 13:55:10 | mgoddard | efried: but if we call e.g. pt.update_inventory() with a new generation, the traits or aggregates for that provider may not necessarily correspond to the generation that gets set | |
| 13:55:12 | mgoddard | efried: I can't see anything that's using the tree's stored generations currently, but presumably they're in there for some future purpose? | |
| 13:55:32 | efried | mgoddard That's precisely what I'm working on right now. | |
| 13:56:14 | cdent | sean-k-mooney: selecting an unused uuid is kind of the easy part of why use uuids? | |
| 13:56:28 | cdent | but you can always check it, with member_of | |
| 13:56:53 | efried | cdent Except for the multi-source sync issue, aforementioned. | |
| 13:57:18 | cdent | link? | |
| 13:57:22 | efried | mgoddard The client code should never be setting/incrementing the generation directly. It should be blindly passing the generation from the API response into those ProviderTree methods. | |
| 13:57:52 | sean-k-mooney | cdent: oh yes i know i can just gen a new one. the reson i brought this up is i knind of have a usecase where really i would like to have a resouce provider have two parents but alternively i could use an aggregate to model the second relationship | |
| 13:57:58 | efried | cdent Word salad on that review we discussed in the sched mtg, stand by... | |
| 13:58:11 | mdbooth | kashyap: Ah, looks like I removed qemu-img from that task. | |
| 13:58:24 | cdent | efried: I think in that context were assuming that we're "choosing" a know uuid from some authority | |
| 13:58:33 | cdent | known | |
| 13:58:46 | sean-k-mooney | cdent: yes | |
| 13:59:09 | efried | cdent Which would be fine, if that's what we're doing, AND the multiple control points understand the semantics of the provider having multiple aggs associated or not. | |
| 13:59:36 | cdent | sean-k-mooney: I will keep you in beverages of your choice for the duration of dublin if you promise me to never bring up multi-parents again :) | |
| 14:00:02 | sean-k-mooney | cdent: efried so in the case of vhost-vfio interfaces i was considering them to be owned by neutron not nova yes i want to model numa relationship too | |
| 14:00:09 | kashyap | mdbooth: I see, stopped the audit for a bit; will get back. | |
| 14:00:09 | efried | mgoddard Which is what report client should be doing, but isn't yet, but I'm working on it. | |
| 14:00:51 | kashyap | mdbooth: https://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg00845.html -- [PATCH 0/2] qemu-img: Let "info" warn and go ahead without -U | |
| 14:00:59 | sean-k-mooney | cdent:hehe yes i know we dont want to have multiple parent so im exploing other options. | |
| 14:01:17 | kashyap | (The "-U" == '--force-share') | |
| 14:01:27 | efried | mgoddard It also sends the generation - whatever was the last value it got when it retrieved the RP record - whenever it updates the RP or associations (traits, aggs*, etc.). And then the placement API bounces with a 409 if that's not the same as the generation it thinks that RP should have. | |
| 14:01:58 | efried | mgoddard The client can/should respond to that by re-GETting the provider/associations, redriving the change, and re-PUTting with the new generation. | |
| 14:02:07 | sean-k-mooney | cdent: basically i want to model the resouce is a sub resouce of a numa node(eventully) and have an easy way to associate them with the neutron agent that manages them | |
| 14:02:56 | sean-k-mooney | cdent: without the numa aspect i had originally considerd modeling the neutron agent as a resouce providers and making it the parent of the sub resouce | |
| 14:02:59 | cdent | that sounds a bit like the model of shared provider where the neutron agent is a resource provider | |
| 14:03:10 | efried | sean-k-mooney Placement won't stop you from doing that, but it would be highly irregular for neutron to own (create/modify/control) a provider in the middle of a tree that's otherwise owned by compute. | |
| 14:03:52 | cdent | I think we've already talked in the past about considering being able to parent neutron rps into a compute generated tree, as leafs | |
| 14:03:59 | sean-k-mooney | efried: well the resoce for this design is i dont think nova long term should be respocible for tracking networking resouces in placement | |
| 14:04:01 | efried | sean-k-mooney It would be preferable if you found a way to model it such that the provider(s) owned by neutron were sharing providers associated by aggregate with ones in the compute's tree. | |
| 14:04:10 | cdent | there was some of that in denver. not that we _will_ do it, but might be able to do it | |
| 14:05:49 | cdent | I have some vague concerns that the provider tree model in compute is going to make it harder to manage rps from multiple places, which was an original goal | |
| 14:06:09 | efried | I mean, I guess, as long as compute and the neutron agent both agree on how that dance is done. It worries me, though, because different vendors' drivers have to talk to the same neutron agent sometimes. | |
| 14:06:17 | cdent | I need to unvague those concerns, but at the moment I can't even get devstack to do the right things for my simple experiments, so... | |
| 14:07:00 | sean-k-mooney | cdent: well the issue is i dont think nova should own the root node of the tree. i think we should considerd the root node to be something earch service can create subtree from | |
| 14:08:06 | efried | sean-k-mooney What do you call that root node? | |
| 14:08:09 | efried | Is it a host? | |
| 14:08:21 | efried | a "cloud"? | |
| 14:08:50 | efried | some nebulous as-yet-unnamed entity that exists solely as an anchor point for this multi-owner provider tree? | |
| 14:09:06 | cdent | sean-k-mooney: That's kind of related to why I disputed that root provider be a thing in the data struture or representation. I think we should be able to access a tree of providers anywhere in whatever trees people like, and declare subtrees to be whole if that's what suits them. That is, if we're gonna have trees, let's have _trees_. | |
| 14:09:06 | efried | ...but doesn't correspond to anything in the real world? | |
| 14:09:15 | cdent | (I'd rather just not have trees, but that ship sailed) | |
| 14:11:16 | sean-k-mooney | efried: in the case of a nova created tree i was assuming a compute node. e.g. a host yes | |
| 14:12:10 | efried | sean-k-mooney Right, I was referring to what you said earlier about "i dont think nova should own the root node of the tree" and "the root node [is] something each service can create subtree from" | |
| 14:12:45 | sean-k-mooney | yep i think we are violently agreeing on that point :) | |
| 14:12:46 | mgoddard | efried: so with your change, will a generation change in the provider tree cause inventory, traits, and aggregates to be updated, and checked that all those responses contain the same generation? | |
| 14:12:49 | efried | In that picture, who owns the root node, and what does it represent? | |
| 14:14:01 | sean-k-mooney | efried: everyone owns it. i think we just need to agree on how the root node is create for example by saying that the node is idenfied by the host_id which default to the hostname | |
| 14:14:29 | efried | sean-k-mooney But if the root node represents a compute host, doesn't it make sense for compute to own/create it? | |
| 14:14:46 | sean-k-mooney | efried: not in a converged deployment | |
| 14:15:25 | sean-k-mooney | for example if you have cinder running on the same physical server then its also a storage node not just a compute node | |
| 14:16:42 | sean-k-mooney | if cinder and nova agree on how to create/select that root node then can boot create subtrees for that phyical server | |
| 14:16:49 | efried | mgoddard At the report client level, the plan for set_traits_for_provider is: If PUT /rp/{uuid}/traits succeeds (200), I'll update_traits on the ProviderTree, setting the generation based on what's in the PUT response. (The generation is an attribute of the provider, so updating it via update_traits updates it for anything else associated with that provider.) | |
| 14:17:33 | efried | mgoddard If the PUT fails 409, set_traits_for_provider will raise (a subclass of) this new conflict exception. | |
| 14:17:34 | sean-k-mooney | s/ then can boot/ then both can/ | |
| 14:18:39 | efried | mgoddard The report client consumer (i.e. resource tracker) should then redrive the overarching operation, which should entail first asking report client to refresh its cached representation of that provider in the ProviderTree. | |
| 14:19:11 | efried | mgoddard Which means reGETting the provider and all its associated bits (traits, aggs, etc.) and calling the appropriate ProviderTree methods to update them all. | |
| 14:19:53 | efried | mgoddard Then the consumer (resource tracker) would make whatever change again (in this example setting the traits). Rinse, repeat. | |
| 14:20:14 | efried | mgoddard Not sure if I'm explaining this particularly well. | |
| 14:20:35 | sean-k-mooney | efried: cdent anyway that when a little off topic but i think one design would be for neutorn to create its resouces under the numa nodes create by nova and add the resouces it creates to an aggreate using the neutron agent uuid as the aggreate uuid. | |
| 14:21:00 | efried | sean-k-mooney Why do you need the aggregate in that picture? | |
| 14:21:11 | efried | sean-k-mooney Are the neutron resources common to multiple hosts? | |
| 14:21:41 | efried | sean-k-mooney If so, they should *not* be created as providers in the tree under the numa nodes. | |
| 14:22:00 | sean-k-mooney | efried: no they are specific to that compute but i want a simple way to look up all the resocurce providers created by the neutron agent on that host | |
| 14:22:48 | efried | sean-k-mooney Ahhh, that makes sense. | |
| 14:23:16 | efried | sean-k-mooney So the aggregate is really just saying "neutron owns these". It's not associating the providers with any shared providers. | |
| 14:23:42 | efried | sean-k-mooney Which is a crucial distinction, because you wouldn't want an allocation candidate request to bleed across compute trees. | |
| 14:24:08 | efried | sean-k-mooney So in this scenario, the providers are associated with an aggregate, but nobody in that aggregate has the MISC_SHARES_VIA_AGGREGATE trait. | |
| 14:25:06 | sean-k-mooney | efried: yes exactly | |
| 14:26:10 | efried | mgoddard But note that report client is the pinch point for everything. You can't directly change the report client's cached ProviderTree from outside the report client. Conversely and more importantly, you can't effect a change to placement by changing that cached ProviderTree. You can only do that by calling report client methods like set_*_on_provider etc. | |
| 14:27:13 | efried | mgoddard Where it gets confusing is that ComputeDriver.update_provider_tree *sounds* like it's doing just that (effecting a change to placement by changing the cached ProviderTree). | |
| 14:27:22 | efried | mgoddard Which is almost but not quite the case. | |
| 14:28:26 | efried | mgoddard RT will ask RC for the provider tree; RC will return a *copy*. RT will then ask ComputeDriver to update_provider_tree. RT will then ask RC to diff and flush any changes back to placement, which will update RC's cached ProviderTree in the process. | |
| 14:30:01 | efried | sean-k-mooney Is there any reason that agg ID has to be the same across all the computes for which that neutron instance is handling the network providers? | |
| 14:30:29 | efried | sean-k-mooney I guess I can see the benefits either way. | |
| 14:30:58 | sean-k-mooney | efried: in this case i was assuming the aggregate would not span multiple computes | |
| 14:31:16 | openstackgerrit | Mark Goddard proposed openstack/nova master: Add support to scheduler client for setting traits https://review.openstack.org/532539 | |
| 14:31:32 | sean-k-mooney | i am considering if aggregates can be used to group resouce providres in a singel compute that are releated in some way | |
| 14:32:42 | sean-k-mooney | efried: do you rememebr https://review.openstack.org/#/c/502306/14/specs/queens/approved/bandwidth-resource-provider.rst effectivly im wondering how best to remove the agent resouce providers so that we can also model numa affinity of networking resources | |
| 14:32:54 | mgoddard | efried: Ah, I think I see now. We can only get a 200 back if no other changes have occurred other than those we just PUT, therefore our provider tree must be up to date. Thanks for explaining | |
| 14:33:05 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Collocate encryptor and volume driver calls https://review.openstack.org/460243 | |
| 14:33:08 | lyarwood | mdbooth / stephenfin ; ^ one final respin with the comment and nits taken care of | |
| 14:36:00 | efried | mgoddard Just so. Given that report client is the pinch point for placement in nova, and there's only one report client, and it's running on one compute node, under only one thread, we shouldn't actually see these conflicts if nova is the only thingy managing all the providers. But as you can see from the conversation above, there are plans to potentially have multiple sources of ownership for a given provider. | |