| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-01-10 | |||
| 14:20:35 | sean-k-mooney | efried: cdent anyway that when a little off topic but i think one design would be for neutorn to create its resouces under the numa nodes create by nova and add the resouces it creates to an aggreate using the neutron agent uuid as the aggreate uuid. | |
| 14:21:00 | efried | sean-k-mooney Why do you need the aggregate in that picture? | |
| 14:21:11 | efried | sean-k-mooney Are the neutron resources common to multiple hosts? | |
| 14:21:41 | efried | sean-k-mooney If so, they should *not* be created as providers in the tree under the numa nodes. | |
| 14:22:00 | sean-k-mooney | efried: no they are specific to that compute but i want a simple way to look up all the resocurce providers created by the neutron agent on that host | |
| 14:22:48 | efried | sean-k-mooney Ahhh, that makes sense. | |
| 14:23:16 | efried | sean-k-mooney So the aggregate is really just saying "neutron owns these". It's not associating the providers with any shared providers. | |
| 14:23:42 | efried | sean-k-mooney Which is a crucial distinction, because you wouldn't want an allocation candidate request to bleed across compute trees. | |
| 14:24:08 | efried | sean-k-mooney So in this scenario, the providers are associated with an aggregate, but nobody in that aggregate has the MISC_SHARES_VIA_AGGREGATE trait. | |
| 14:25:06 | sean-k-mooney | efried: yes exactly | |
| 14:26:10 | efried | mgoddard But note that report client is the pinch point for everything. You can't directly change the report client's cached ProviderTree from outside the report client. Conversely and more importantly, you can't effect a change to placement by changing that cached ProviderTree. You can only do that by calling report client methods like set_*_on_provider etc. | |
| 14:27:13 | efried | mgoddard Where it gets confusing is that ComputeDriver.update_provider_tree *sounds* like it's doing just that (effecting a change to placement by changing the cached ProviderTree). | |
| 14:27:22 | efried | mgoddard Which is almost but not quite the case. | |
| 14:28:26 | efried | mgoddard RT will ask RC for the provider tree; RC will return a *copy*. RT will then ask ComputeDriver to update_provider_tree. RT will then ask RC to diff and flush any changes back to placement, which will update RC's cached ProviderTree in the process. | |
| 14:30:01 | efried | sean-k-mooney Is there any reason that agg ID has to be the same across all the computes for which that neutron instance is handling the network providers? | |
| 14:30:29 | efried | sean-k-mooney I guess I can see the benefits either way. | |
| 14:30:58 | sean-k-mooney | efried: in this case i was assuming the aggregate would not span multiple computes | |
| 14:31:16 | openstackgerrit | Mark Goddard proposed openstack/nova master: Add support to scheduler client for setting traits https://review.openstack.org/532539 | |
| 14:31:32 | sean-k-mooney | i am considering if aggregates can be used to group resouce providres in a singel compute that are releated in some way | |
| 14:32:42 | sean-k-mooney | efried: do you rememebr https://review.openstack.org/#/c/502306/14/specs/queens/approved/bandwidth-resource-provider.rst effectivly im wondering how best to remove the agent resouce providers so that we can also model numa affinity of networking resources | |
| 14:32:54 | mgoddard | efried: Ah, I think I see now. We can only get a 200 back if no other changes have occurred other than those we just PUT, therefore our provider tree must be up to date. Thanks for explaining | |
| 14:33:05 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Collocate encryptor and volume driver calls https://review.openstack.org/460243 | |
| 14:33:08 | lyarwood | mdbooth / stephenfin ; ^ one final respin with the comment and nits taken care of | |
| 14:36:00 | efried | mgoddard Just so. Given that report client is the pinch point for placement in nova, and there's only one report client, and it's running on one compute node, under only one thread, we shouldn't actually see these conflicts if nova is the only thingy managing all the providers. But as you can see from the conversation above, there are plans to potentially have multiple sources of ownership for a given provider. | |
| 14:36:56 | sean-k-mooney | efried: https://review.openstack.org/#/c/502306/14/specs/queens/approved/bandwidth-resource-provider.rst was assumign numa would either be a trait or we would have multiple parents. i like haveing numa nodes be resouce provider but that means neutron agent cant be if tehre sub resouce have numa affinity. | |
| 14:37:23 | efried | mgoddard BTW, I put a * above: today aggregates aren't included in the generation thing. cdent is working on fixing that. It's unclear whether that's going to get deferred to Rocky. IMO sean-k-mooney just described a non-sharing-RP use case where we need it. | |
| 14:38:00 | efried | sean-k-mooney Multiple parents, eh? How so? | |
| 14:38:22 | efried | sean-k-mooney I *think* the best way to model NUMA affinity is going to be using a numbered group with granular syntax. | |
| 14:38:52 | mriedem | yikun: if you wanted to start looking at something, this part of the spec hasn't been started https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/neutron-new-port-binding-api.html | |
| 14:38:57 | mriedem | yikun: "Prior to the RPC call from live_migration on the source host to pre_live_migration on the dest host, start a wait thread for the vif-plugged event from Neutron, similar to during initial spawn." | |
| 14:39:43 | sean-k-mooney | efried: if resouce are allowed to have 1+ parents tehn one parent is the numa node and the other the neutron agent but that makes the trees a graph which we dont whant in general | |
| 14:41:44 | ameeda | anyone can help me to write functional regression test for this review.openstack.org/#/c/526900/ ? | |
| 14:42:34 | efried | sean-k-mooney Agree. The tree should be a tree (to quote cdent), the neutron agent can control the providers (at least the inventories thereof, if not the RP records themselves), and we can use aggregates to "tag" those providers as being controlled thusly. | |
| 14:42:46 | efried | sean-k-mooney An alternative approach for such tagging would be to use a trait. | |
| 14:43:16 | sean-k-mooney | efried: ya that was the other thing i was going to explore | |
| 14:43:31 | efried | sean-k-mooney Which might engender less confusion wrt sharing providers (which these ain't) as well as remove the requirement for the agg generation thing cdent is working. | |
| 14:43:36 | sean-k-mooney | e.g. CUSTOM_AGENT_<uuid goes here> | |
| 14:43:42 | cdent | efried: you're welcome to quote me efried, but you're taking me out of context. the implication i was trying to make there is that the trees should be big and anything can be a root | |
| 14:44:07 | efried | cdent Sorry, I take it back. Just didn't want to plagiarize :) | |
| 14:44:49 | efried | sean-k-mooney Or just CUSTOM_MANAGED_BY_NEUTRON. Why would you need a UUID in the trait? | |
| 14:44:51 | cdent | my utterances here are cca | |
| 14:45:16 | efried | I'll quote you on that. | |
| 14:45:23 | efried | (but not give you credit) | |
| 14:45:30 | sean-k-mooney | efried: again so i have a singel api call i can make to retrive all resouce provders that are managed by that agent | |
| 14:46:12 | sean-k-mooney | both aggreates via memberof and traits with the uuid give me that within the existing model | |
| 14:46:23 | efried | sean-k-mooney Isn't the scope of one neutron in this scenario the same as the scope of one placement service? | |
| 14:47:21 | sean-k-mooney | one placement service? is someone suggesting there would be more then one placement service? | |
| 14:47:42 | cdent | efried: sean's assertion was neutron agent, not neutron | |
| 14:48:01 | sean-k-mooney | oh yes what cdent said | |
| 14:48:03 | cdent | sean-k-mooney: in the dark corners of the universe there has been talk of nested placement services | |
| 14:48:18 | efried | (cdent that's not what I was talking about) | |
| 14:48:32 | efried | cdent sean-k-mooney In that case, there's one neutron agent per compute host, yah? | |
| 14:48:39 | sean-k-mooney | cdent: beyond one per cell that sound like i need something stronger then the coffee im drinking | |
| 14:49:00 | cdent | sean-k-mooney: yeah, it's the sort of thing where you should keep _me_ in drinks in dublin to not raise again | |
| 14:49:33 | sean-k-mooney | efried: actully you could have several per compute node. e.g. sriov + ovs on the same node though in practice yes | |
| 14:50:06 | efried | sean-k-mooney But not one agent for multiple computes. | |
| 14:50:40 | efried | sean-k-mooney So my point is, you can ask for providers (having CUSTOM_MANAGED_BY_NEUTRON) && (in tree <compute RP UUID>) | |
| 14:50:50 | sean-k-mooney | efried: :( well for agent based neutron no. but this is odl.... | |
| 14:50:59 | sean-k-mooney | * there is | |
| 14:51:29 | efried | cdent I thought GET /resource_providers had a queryparam for "having traits". I don't see it at a glance. | |
| 14:52:38 | cdent | i'm not sure that got merged (yet) | |
| 14:52:46 | cdent | I do think code to do it somewhere though | |
| 14:52:55 | efried | cdent Oh, okay, in flight | |
| 14:53:26 | efried | It would be somewhere in that fabulous placement update email summary... | |
| 14:54:08 | mriedem | alex_xu: i'm still +2 on https://review.openstack.org/#/c/330406/ - i think the migration_links thing is correct; it's consistent with other APIs that support paging. you have a good point about the changes-since before 2.59 though, but that could be addressed in a follow up. | |
| 14:54:18 | mriedem | since it assumes people would actually do that | |
| 14:54:34 | sean-k-mooney | efried: worst comes worse neuron can jsut get teh whole tree and walk it to see if the inventries it creates exist or not. | |
| 14:54:51 | cdent | efried: I think it is something that alex_xu was working on but a lot of his trait related stuff got abandoned | |
| 14:55:31 | efried | cdent Yah, I can't find any such thing in open state (assuming it would have 'trait' somewhere in the title/description) | |
| 14:56:30 | efried | mgoddard So at a glance, it looks like what you've done here is modeled after the inventory updating stuff. | |
| 14:56:55 | mgoddard | efried: Correct | |
| 14:57:00 | cdent | efriend I suspect that quite a few query style things, on /resource_providers, got dropped when /allocation_candidates took the focus, especially if the use cases on /resource_providers werent yet fully formed | |
| 14:57:42 | efried | mgoddard I haven't fully synthesized this stance yet, but I *think* I'm going to come to the conclusion that that's unnecessarily complicated (even for inventory) - and even incorrect in that it does retries at this low level rather than at the consumer level. | |
| 14:59:31 | efried | cdent GET /resource_providers?having_all=T1,T2 and/or ?having_any=T3,T4 seems like a fairly natural thing to expect, but of course there needs to be a real use case for it. What sean-k-mooney described could count as such. | |
| 14:59:52 | sean-k-mooney | efried: one of the other issue is that nova uses the nova compute node uuid of the host_id(hostname by default) which is sotre in the name filed of the compute node RP i think so we cant uses in_tree in this case and need to use name | |
| 15:00:23 | efried | I believe in_tree accepts name or UUID, doesn't it? | |
| 15:00:37 | efried | no, never mind. | |
| 15:01:03 | sean-k-mooney | got to run to a meeting but efried did you not have a systax for this discribed also in your generic device management proposal. | |
| 15:01:30 | sean-k-mooney | be back in 30 mins | |
| 15:02:00 | efried | sean-k-mooney Syntax for what? And I doubt it, I don't recall getting to a 'syntax' level of detail in the generic device management discussions. | |
| 15:03:04 | mriedem | alex_xu: makes me wonder if we've added other query strings in higher microversions to apis that allowed additionalProperties before :) | |
| 15:04:16 | efried | mgoddard I think we got away with retries at the report client level for inventory because at the time inventory was the only thing that could affect generation, AND we were guaranteed to be the only thing messing with that provider, AND there were no trees or sharing providers. dansmith cdent and Jay should check me on this, but I think we're going to want to pull those retries outta there (at least for 409s) and subsume t | |
| 15:04:17 | efried | hem in the wholesale retries from the resource tracker level. | |
| 15:04:48 | cdent | that's probably right and aligns with what was said monday | |
| 15:05:20 | kashyap | mriedem: A heads-up: Given your Nova commit 8075797, https://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg00845.html -- [PATCH 0/2] qemu-img: Let "info" warn and go ahead without -U ['--force-share'] | |
| 15:05:52 | kashyap | I (& DanPB too) pointed out that Nova already added support to it | |
| 15:06:14 | kashyap | Where the QEMU folks were asking if Nova / other management tools use it -- https://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg01816.html | |
| 15:09:27 | mriedem | kashyap: so they are talking about deprecating and removing the locking thing because everyone is just bypassing it to get their code working again? | |
| 15:10:15 | alex_xu | efried: cdent anything I can help on trait? | |
| 15:10:18 | mriedem | i think nova hits qemu-info from a lot of places | |
| 15:10:30 | kashyap | mriedem: The discussion is still in flux. I don't think they're going to _remove_ it. | |
| 15:10:37 | mriedem | so auditing when we can just ignore it and bypass the lock would be difficult | |
| 15:10:47 | kashyap | The aim of the locking change is to not let users shoot themselves in the foot | |
| 15:10:55 | mriedem | yeah i realize | |
| 15:10:56 | kashyap | But that WILl cause some inconvenience, in terms of usage behaviour | |
| 15:11:07 | kashyap | Trying to get a sense of what is the behaviour across versions | |
| 15:11:07 | cdent | alex_xu: I don't think we need to do anything immediately but we were discussing needing to be able to get a list of resource providers that have a particular trait | |
| 15:11:08 | alex_xu | mriedem: so...after that patch merge, we have a window the order version API is broken | |
| 15:11:19 | mriedem | kashyap: so for the shareable disk thing in libvirt 3.10, does that just bypass the lock in qemu 2.10? | |
| 15:11:29 | kashyap | mriedem: Also, I was just adding a TODO item is that, we should investigate using the run-time command 'query-block' | |