Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-01-10
13:47:07 efried Ahh, that's it, thanks cdent
13:47:17 cdent https://developer.openstack.org/api-ref/placement/#list-resource-providers
13:47:57 cdent It's not clear how you're supposed to discovery an aggregate, though, other than by looking at https://developer.openstack.org/api-ref/placement/#list-resource-provider-aggregates
13:48:06 cdent or knowing the uuid prior
13:48:12 mdbooth kashyap: I'm not convinced cache() would be called on a running instance. As always with that code, though, it's far from obvious without checking carefully.
13:48:28 mdbooth kashyap: cache() mostly means 'create'
13:48:29 kashyap mdbooth: Yeah, I should've been careful in pointing that out
13:48:37 efried cdent Yeah. That said, is there a use case for that?
13:48:50 sean-k-mooney cdent: ah yes cool. that still leaves me with one question. how do i create the aggreate in the first place with out a top level aggregates api?
13:49:18 cdent efried: not that I'm aware, but just as I was thinking of it I stumbled on "How do I know the aggregates"
13:49:27 cdent sean-k-mooney: it gets created when you use it
13:49:40 kashyap mdbooth: I didn't do a thorough audit, though. Taking notes as I find instances of it & then see where they're called on a running guest
13:49:47 kashyap For live snapshot we do for sure
13:50:00 cdent so if you PUT to /resource_provider/{uuid}/aggregates with a new uuid there ya go
13:50:01 efried sean-k-mooney You "create" it by assigning it to a provider via PUT /rp/{uuid}/aggs
13:50:02 sean-k-mooney cdent: so the first time i add a resouce provider to an agregate it creates the uuid
13:50:04 efried yeah, what he said.
13:50:12 cdent yes
13:50:12 mdbooth kashyap: Did you look in the imagecache periodic task?
13:50:43 kashyap mdbooth: Not yet; so far I'm just noting down instances where it's called. E.g:
13:50:46 kashyap nova/virt/libvirt/imagebackend.py
13:50:49 kashyap - cache() --> fetch_func_sync()
13:50:51 kashyap --> get_disk_size() [from nova/virt/libvirt/driver.py]
13:50:54 kashyap --> qemu_img_info() [from nova/virt/images.py]
13:50:56 kashyap - verify_base_size() --> get_disk_size()
13:50:59 kashyap - class LVM() --> create_image() --> create_lvm_image() --> get_disk_size() --> qemu_img_info()
13:51:00 sean-k-mooney cdent: so the problem of determining an un used uuid for the aggregate is left to the client
13:51:09 cdent yes
13:51:12 efried sean-k-mooney I knew I wrote that code: https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L431
13:51:12 kashyap (And then see the calls for the live guest.)
13:51:44 sean-k-mooney cdent: ok cool. more reading for me to do :)
13:51:48 cdent sean-k-mooney: there's some stuff in the very early rp specs about such things
13:51:49 efried sean-k-mooney There shouldn't be any such thing as an "unused" aggregate UUID. Unless you mean "used only for one provider", which is pointless.
13:52:14 cdent but if I remember right, the thinking was that the client would already have some identifier in min
13:52:15 cdent d
13:52:30 efried ...because when you remove the last association, placement ought to get rid of that agg ID.
13:53:02 efried cdent Yeah, that's a weird one. In the PowerVM SSP case, the only thing that makes sense is to give the agg the same UUID as the shared storage pool
13:53:22 efried Though I *suppose* I could give the agg the UUID of the cluster instead. It's effectively the same thing in powervm land.
13:53:43 sean-k-mooney efried: well what i mean is if i am createing a set of recouse providers and i want to make them part of a aggregate to group them i need to choose a uuid that is not useed by anyone else.
13:54:30 cdent placement does not clean up the unused agg uuids. that was an early design decision, sort of resulting from how the tables were being normalized to not use uuid keys (and a few others things, it's so long ago)
13:54:32 efried sean-k-mooney Yes, which one would normally do by generating one randomly, BUT that breaks down quickly if you have more than one point of control for that agg.
13:54:45 efried cdent Whoah, so how do they get cleaned up?
13:54:58 cdent they don't
13:55:02 cdent at least not last I checked
13:55:03 efried They just... leak?
13:55:04 mgoddard efried: I have some concerns with the storage of the generation in the provider tree
13:55:06 mgoddard efried: there is a single generation per-provider that covers inventory, traits, and aggregates
13:55:10 mgoddard efried: but if we call e.g. pt.update_inventory() with a new generation, the traits or aggregates for that provider may not necessarily correspond to the generation that gets set
13:55:12 mgoddard efried: I can't see anything that's using the tree's stored generations currently, but presumably they're in there for some future purpose?
13:55:32 efried mgoddard That's precisely what I'm working on right now.
13:56:14 cdent sean-k-mooney: selecting an unused uuid is kind of the easy part of why use uuids?
13:56:28 cdent but you can always check it, with member_of
13:56:53 efried cdent Except for the multi-source sync issue, aforementioned.
13:57:18 cdent link?
13:57:22 efried mgoddard The client code should never be setting/incrementing the generation directly. It should be blindly passing the generation from the API response into those ProviderTree methods.
13:57:52 sean-k-mooney cdent: oh yes i know i can just gen a new one. the reson i brought this up is i knind of have a usecase where really i would like to have a resouce provider have two parents but alternively i could use an aggregate to model the second relationship
13:57:58 efried cdent Word salad on that review we discussed in the sched mtg, stand by...
13:58:11 mdbooth kashyap: Ah, looks like I removed qemu-img from that task.
13:58:24 cdent efried: I think in that context were assuming that we're "choosing" a know uuid from some authority
13:58:33 cdent known
13:58:46 sean-k-mooney cdent: yes
13:59:09 efried cdent Which would be fine, if that's what we're doing, AND the multiple control points understand the semantics of the provider having multiple aggs associated or not.
13:59:36 cdent sean-k-mooney: I will keep you in beverages of your choice for the duration of dublin if you promise me to never bring up multi-parents again :)
14:00:02 sean-k-mooney cdent: efried so in the case of vhost-vfio interfaces i was considering them to be owned by neutron not nova yes i want to model numa relationship too
14:00:09 efried mgoddard Which is what report client should be doing, but isn't yet, but I'm working on it.
14:00:09 kashyap mdbooth: I see, stopped the audit for a bit; will get back.
14:00:51 kashyap mdbooth: https://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg00845.html -- [PATCH 0/2] qemu-img: Let "info" warn and go ahead without -U
14:00:59 sean-k-mooney cdent:hehe yes i know we dont want to have multiple parent so im exploing other options.
14:01:17 kashyap (The "-U" == '--force-share')
14:01:27 efried mgoddard It also sends the generation - whatever was the last value it got when it retrieved the RP record - whenever it updates the RP or associations (traits, aggs*, etc.). And then the placement API bounces with a 409 if that's not the same as the generation it thinks that RP should have.
14:01:58 efried mgoddard The client can/should respond to that by re-GETting the provider/associations, redriving the change, and re-PUTting with the new generation.
14:02:07 sean-k-mooney cdent: basically i want to model the resouce is a sub resouce of a numa node(eventully) and have an easy way to associate them with the neutron agent that manages them
14:02:56 sean-k-mooney cdent: without the numa aspect i had originally considerd modeling the neutron agent as a resouce providers and making it the parent of the sub resouce
14:02:59 cdent that sounds a bit like the model of shared provider where the neutron agent is a resource provider
14:03:10 efried sean-k-mooney Placement won't stop you from doing that, but it would be highly irregular for neutron to own (create/modify/control) a provider in the middle of a tree that's otherwise owned by compute.
14:03:52 cdent I think we've already talked in the past about considering being able to parent neutron rps into a compute generated tree, as leafs
14:03:59 sean-k-mooney efried: well the resoce for this design is i dont think nova long term should be respocible for tracking networking resouces in placement
14:04:01 efried sean-k-mooney It would be preferable if you found a way to model it such that the provider(s) owned by neutron were sharing providers associated by aggregate with ones in the compute's tree.
14:04:10 cdent there was some of that in denver. not that we _will_ do it, but might be able to do it
14:05:49 cdent I have some vague concerns that the provider tree model in compute is going to make it harder to manage rps from multiple places, which was an original goal
14:06:09 efried I mean, I guess, as long as compute and the neutron agent both agree on how that dance is done. It worries me, though, because different vendors' drivers have to talk to the same neutron agent sometimes.
14:06:17 cdent I need to unvague those concerns, but at the moment I can't even get devstack to do the right things for my simple experiments, so...
14:07:00 sean-k-mooney cdent: well the issue is i dont think nova should own the root node of the tree. i think we should considerd the root node to be something earch service can create subtree from
14:08:06 efried sean-k-mooney What do you call that root node?
14:08:09 efried Is it a host?
14:08:21 efried a "cloud"?
14:08:50 efried some nebulous as-yet-unnamed entity that exists solely as an anchor point for this multi-owner provider tree?
14:09:06 efried ...but doesn't correspond to anything in the real world?
14:09:06 cdent sean-k-mooney: That's kind of related to why I disputed that root provider be a thing in the data struture or representation. I think we should be able to access a tree of providers anywhere in whatever trees people like, and declare subtrees to be whole if that's what suits them. That is, if we're gonna have trees, let's have _trees_.
14:09:15 cdent (I'd rather just not have trees, but that ship sailed)
14:11:16 sean-k-mooney efried: in the case of a nova created tree i was assuming a compute node. e.g. a host yes
14:12:10 efried sean-k-mooney Right, I was referring to what you said earlier about "i dont think nova should own the root node of the tree" and "the root node [is] something each service can create subtree from"
14:12:45 sean-k-mooney yep i think we are violently agreeing on that point :)
14:12:46 mgoddard efried: so with your change, will a generation change in the provider tree cause inventory, traits, and aggregates to be updated, and checked that all those responses contain the same generation?
14:12:49 efried In that picture, who owns the root node, and what does it represent?
14:14:01 sean-k-mooney efried: everyone owns it. i think we just need to agree on how the root node is create for example by saying that the node is idenfied by the host_id which default to the hostname
14:14:29 efried sean-k-mooney But if the root node represents a compute host, doesn't it make sense for compute to own/create it?
14:14:46 sean-k-mooney efried: not in a converged deployment
14:15:25 sean-k-mooney for example if you have cinder running on the same physical server then its also a storage node not just a compute node
14:16:42 sean-k-mooney if cinder and nova agree on how to create/select that root node then can boot create subtrees for that phyical server

Earlier   Later