| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-01-10 | |||
| 13:34:52 | efried | mgoddard It's in the middle of a rather messy restack, won't be ready to show for a little while yet. | |
| 13:35:22 | efried | mgoddard But now that I know you're also wanting it for the ironic traits bp, seems it needs to be peeled out of that series. | |
| 13:36:21 | efried | which is probably not super hard. Though at the moment it's based on a change that raises a conflict exception for RP creation, which I think *is* tied pretty heavily into that series. | |
| 13:44:24 | kashyap | mdbooth: Just noting for the record, looked for the past few minutes: | |
| 13:44:26 | kashyap | _rebase_with_qemu_img() , _live_snapshot() and _get_instance_disk_info_from_config() [from nova/virt/libvirt/driver.py] | |
| 13:44:42 | kashyap | nova/virt/libvirt/imagebackend.py: | |
| 13:44:59 | kashyap | - cache() --> fetch_func_sync() --> get_disk_size() --> qemu_img_info() | |
| 13:45:04 | sean-k-mooney | efried: QQ is there a top level api for placement aggregates? e.g. can i list all aggregates or list all resocue providers in an aggregate given the aggregate uuid? | |
| 13:45:21 | efried | sean-k-mooney sec... | |
| 13:46:04 | efried | sean-k-mooney Hum, I thought there was, cause I'm gonna need it. Still looking... | |
| 13:46:46 | sean-k-mooney | efried: no rush. you can get teh aggregates a resouce provider is part of but at least looking at the master docs the recprical api does not appear to exist | |
| 13:46:56 | cdent | you can use member_of to get all rps in a given aggregate | |
| 13:47:07 | efried | Ahh, that's it, thanks cdent | |
| 13:47:17 | cdent | https://developer.openstack.org/api-ref/placement/#list-resource-providers | |
| 13:47:57 | cdent | It's not clear how you're supposed to discovery an aggregate, though, other than by looking at https://developer.openstack.org/api-ref/placement/#list-resource-provider-aggregates | |
| 13:48:06 | cdent | or knowing the uuid prior | |
| 13:48:12 | mdbooth | kashyap: I'm not convinced cache() would be called on a running instance. As always with that code, though, it's far from obvious without checking carefully. | |
| 13:48:28 | mdbooth | kashyap: cache() mostly means 'create' | |
| 13:48:29 | kashyap | mdbooth: Yeah, I should've been careful in pointing that out | |
| 13:48:37 | efried | cdent Yeah. That said, is there a use case for that? | |
| 13:48:50 | sean-k-mooney | cdent: ah yes cool. that still leaves me with one question. how do i create the aggreate in the first place with out a top level aggregates api? | |
| 13:49:18 | cdent | efried: not that I'm aware, but just as I was thinking of it I stumbled on "How do I know the aggregates" | |
| 13:49:27 | cdent | sean-k-mooney: it gets created when you use it | |
| 13:49:40 | kashyap | mdbooth: I didn't do a thorough audit, though. Taking notes as I find instances of it & then see where they're called on a running guest | |
| 13:49:47 | kashyap | For live snapshot we do for sure | |
| 13:50:00 | cdent | so if you PUT to /resource_provider/{uuid}/aggregates with a new uuid there ya go | |
| 13:50:01 | efried | sean-k-mooney You "create" it by assigning it to a provider via PUT /rp/{uuid}/aggs | |
| 13:50:02 | sean-k-mooney | cdent: so the first time i add a resouce provider to an agregate it creates the uuid | |
| 13:50:04 | efried | yeah, what he said. | |
| 13:50:12 | cdent | yes | |
| 13:50:12 | mdbooth | kashyap: Did you look in the imagecache periodic task? | |
| 13:50:43 | kashyap | mdbooth: Not yet; so far I'm just noting down instances where it's called. E.g: | |
| 13:50:46 | kashyap | nova/virt/libvirt/imagebackend.py | |
| 13:50:49 | kashyap | - cache() --> fetch_func_sync() | |
| 13:50:51 | kashyap | --> get_disk_size() [from nova/virt/libvirt/driver.py] | |
| 13:50:54 | kashyap | --> qemu_img_info() [from nova/virt/images.py] | |
| 13:50:56 | kashyap | - verify_base_size() --> get_disk_size() | |
| 13:50:59 | kashyap | - class LVM() --> create_image() --> create_lvm_image() --> get_disk_size() --> qemu_img_info() | |
| 13:51:00 | sean-k-mooney | cdent: so the problem of determining an un used uuid for the aggregate is left to the client | |
| 13:51:09 | cdent | yes | |
| 13:51:12 | efried | sean-k-mooney I knew I wrote that code: https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L431 | |
| 13:51:12 | kashyap | (And then see the calls for the live guest.) | |
| 13:51:44 | sean-k-mooney | cdent: ok cool. more reading for me to do :) | |
| 13:51:48 | cdent | sean-k-mooney: there's some stuff in the very early rp specs about such things | |
| 13:51:49 | efried | sean-k-mooney There shouldn't be any such thing as an "unused" aggregate UUID. Unless you mean "used only for one provider", which is pointless. | |
| 13:52:14 | cdent | but if I remember right, the thinking was that the client would already have some identifier in min | |
| 13:52:15 | cdent | d | |
| 13:52:30 | efried | ...because when you remove the last association, placement ought to get rid of that agg ID. | |
| 13:53:02 | efried | cdent Yeah, that's a weird one. In the PowerVM SSP case, the only thing that makes sense is to give the agg the same UUID as the shared storage pool | |
| 13:53:22 | efried | Though I *suppose* I could give the agg the UUID of the cluster instead. It's effectively the same thing in powervm land. | |
| 13:53:43 | sean-k-mooney | efried: well what i mean is if i am createing a set of recouse providers and i want to make them part of a aggregate to group them i need to choose a uuid that is not useed by anyone else. | |
| 13:54:30 | cdent | placement does not clean up the unused agg uuids. that was an early design decision, sort of resulting from how the tables were being normalized to not use uuid keys (and a few others things, it's so long ago) | |
| 13:54:32 | efried | sean-k-mooney Yes, which one would normally do by generating one randomly, BUT that breaks down quickly if you have more than one point of control for that agg. | |
| 13:54:45 | efried | cdent Whoah, so how do they get cleaned up? | |
| 13:54:58 | cdent | they don't | |
| 13:55:02 | cdent | at least not last I checked | |
| 13:55:03 | efried | They just... leak? | |
| 13:55:04 | mgoddard | efried: I have some concerns with the storage of the generation in the provider tree | |
| 13:55:06 | mgoddard | efried: there is a single generation per-provider that covers inventory, traits, and aggregates | |
| 13:55:10 | mgoddard | efried: but if we call e.g. pt.update_inventory() with a new generation, the traits or aggregates for that provider may not necessarily correspond to the generation that gets set | |
| 13:55:12 | mgoddard | efried: I can't see anything that's using the tree's stored generations currently, but presumably they're in there for some future purpose? | |
| 13:55:32 | efried | mgoddard That's precisely what I'm working on right now. | |
| 13:56:14 | cdent | sean-k-mooney: selecting an unused uuid is kind of the easy part of why use uuids? | |
| 13:56:28 | cdent | but you can always check it, with member_of | |
| 13:56:53 | efried | cdent Except for the multi-source sync issue, aforementioned. | |
| 13:57:18 | cdent | link? | |
| 13:57:22 | efried | mgoddard The client code should never be setting/incrementing the generation directly. It should be blindly passing the generation from the API response into those ProviderTree methods. | |
| 13:57:52 | sean-k-mooney | cdent: oh yes i know i can just gen a new one. the reson i brought this up is i knind of have a usecase where really i would like to have a resouce provider have two parents but alternively i could use an aggregate to model the second relationship | |
| 13:57:58 | efried | cdent Word salad on that review we discussed in the sched mtg, stand by... | |
| 13:58:11 | mdbooth | kashyap: Ah, looks like I removed qemu-img from that task. | |
| 13:58:24 | cdent | efried: I think in that context were assuming that we're "choosing" a know uuid from some authority | |
| 13:58:33 | cdent | known | |
| 13:58:46 | sean-k-mooney | cdent: yes | |
| 13:59:09 | efried | cdent Which would be fine, if that's what we're doing, AND the multiple control points understand the semantics of the provider having multiple aggs associated or not. | |
| 13:59:36 | cdent | sean-k-mooney: I will keep you in beverages of your choice for the duration of dublin if you promise me to never bring up multi-parents again :) | |
| 14:00:02 | sean-k-mooney | cdent: efried so in the case of vhost-vfio interfaces i was considering them to be owned by neutron not nova yes i want to model numa relationship too | |
| 14:00:09 | efried | mgoddard Which is what report client should be doing, but isn't yet, but I'm working on it. | |
| 14:00:09 | kashyap | mdbooth: I see, stopped the audit for a bit; will get back. | |
| 14:00:51 | kashyap | mdbooth: https://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg00845.html -- [PATCH 0/2] qemu-img: Let "info" warn and go ahead without -U | |
| 14:00:59 | sean-k-mooney | cdent:hehe yes i know we dont want to have multiple parent so im exploing other options. | |
| 14:01:17 | kashyap | (The "-U" == '--force-share') | |
| 14:01:27 | efried | mgoddard It also sends the generation - whatever was the last value it got when it retrieved the RP record - whenever it updates the RP or associations (traits, aggs*, etc.). And then the placement API bounces with a 409 if that's not the same as the generation it thinks that RP should have. | |
| 14:01:58 | efried | mgoddard The client can/should respond to that by re-GETting the provider/associations, redriving the change, and re-PUTting with the new generation. | |
| 14:02:07 | sean-k-mooney | cdent: basically i want to model the resouce is a sub resouce of a numa node(eventully) and have an easy way to associate them with the neutron agent that manages them | |
| 14:02:56 | sean-k-mooney | cdent: without the numa aspect i had originally considerd modeling the neutron agent as a resouce providers and making it the parent of the sub resouce | |
| 14:02:59 | cdent | that sounds a bit like the model of shared provider where the neutron agent is a resource provider | |
| 14:03:10 | efried | sean-k-mooney Placement won't stop you from doing that, but it would be highly irregular for neutron to own (create/modify/control) a provider in the middle of a tree that's otherwise owned by compute. | |
| 14:03:52 | cdent | I think we've already talked in the past about considering being able to parent neutron rps into a compute generated tree, as leafs | |
| 14:03:59 | sean-k-mooney | efried: well the resoce for this design is i dont think nova long term should be respocible for tracking networking resouces in placement | |
| 14:04:01 | efried | sean-k-mooney It would be preferable if you found a way to model it such that the provider(s) owned by neutron were sharing providers associated by aggregate with ones in the compute's tree. | |
| 14:04:10 | cdent | there was some of that in denver. not that we _will_ do it, but might be able to do it | |
| 14:05:49 | cdent | I have some vague concerns that the provider tree model in compute is going to make it harder to manage rps from multiple places, which was an original goal | |
| 14:06:09 | efried | I mean, I guess, as long as compute and the neutron agent both agree on how that dance is done. It worries me, though, because different vendors' drivers have to talk to the same neutron agent sometimes. | |
| 14:06:17 | cdent | I need to unvague those concerns, but at the moment I can't even get devstack to do the right things for my simple experiments, so... | |
| 14:07:00 | sean-k-mooney | cdent: well the issue is i dont think nova should own the root node of the tree. i think we should considerd the root node to be something earch service can create subtree from | |
| 14:08:06 | efried | sean-k-mooney What do you call that root node? | |
| 14:08:09 | efried | Is it a host? | |
| 14:08:21 | efried | a "cloud"? | |
| 14:08:50 | efried | some nebulous as-yet-unnamed entity that exists solely as an anchor point for this multi-owner provider tree? | |
| 14:09:06 | efried | ...but doesn't correspond to anything in the real world? | |