Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-02-19
16:32:37 edleafe ^^ that
16:32:59 cdent because all we really need to know (for now) is that they are there
16:33:01 efried mriedem: Up to this point, virt.PowerVM has full control over the Shared Storage Pool. We want to keep it that way in Placement-land.
16:33:06 jaypipes mriedem: I'm not saying that it's my *preferred* architecture for a virt driver. but it is what it is..
16:33:36 sean-k-mooney efried: at least for the bandwidth case the provider would not be a shared provider but it was going to be associated to the neutron agent by an aggregate
16:34:03 efried sean-k-mooney: Okay, maybe it was neutron owning the NICs/PFs?
16:34:12 mriedem efried: are those SSP's restricted to a single nova-compute service?
16:34:17 sean-k-mooney efried: in that case also not shared
16:34:21 mriedem or can multiple virt drivers be monkeying with the same SSP?
16:34:35 efried mriedem: The latter. We recognize and accept the caveats of co-management.
16:34:46 sean-k-mooney neutron would be creating RPs in the compute node tree in both cases however
16:34:54 efried ^
16:35:37 sean-k-mooney mriedem: SSP?
16:35:48 mriedem shared storage pool
16:35:48 efried sean-k-mooney: Shared Storage Pool, PowerVM style in this case.
16:35:58 sean-k-mooney oh ya ok
16:36:27 mriedem ok so the spec says the virt driver can see these other things but shouldn't modify them, but then you're saying you totally do want to modify them and that's the intent
16:36:38 efried mriedem: No, those are different things.
16:36:42 mriedem in other words, update_provider_tree is a blank check to the virt driver
16:37:11 mriedem as we've seen with the RT managing allocations,
16:37:17 mriedem this is probably goign to end badly
16:37:34 efried The networky examples in sean-k-mooney's camp are going to be providers in the tree that's rooted at the compute node RP. Those are the ones virt should avoid mucking with.
16:37:36 sean-k-mooney efried: oh speaking of PowerVM as an aside it support PCI devices corret but does not expose pci address am i rembering that correctly. i mentioned it on stephenfin's numa aware vswitch spec but was not sure if i remembered correctly
16:38:05 efried sean-k-mooney: I haven't made it all the way through that spec yet, but will look out for that, thanks.
16:38:47 mriedem so non-compute child resource providers - don't touch; non-compute shared aggregate providers, go nuts
16:38:55 efried mriedem: Blank check, sort of. It's going to be important for the virt driver to understand what it's doing with its providers.
16:39:31 mriedem do you have an example of what a virt driver would need to do with a shared provider aggregate?
16:39:31 efried mriedem: The caveats are just caveats. If a virt driver figures out some way it's going to co-manage the providers neutron creates... that sounds weird but doable.
16:39:33 sean-k-mooney mriedem: well it depends for the neutron created resouce providers i had envisioned the nova conductor makeing the claims against them
16:40:04 sean-k-mooney mriedem: that said nova could delegate that to neutron as part of the port bind also
16:40:24 mriedem sean-k-mooney: are you talking about the bw based scheduling spec?
16:40:58 mriedem we want any kind of 'claim' happening before we hit a compute, which is where port binding happens
16:41:03 sean-k-mooney mriedem: ya. that was one of the open questions should nova claim the bandwith or tell neutron to do it as part of port bining which we want to move to condutor
16:41:17 mriedem port binding was never moving to conductor
16:41:17 efried mriedem: Sure: we're polling the SSP to get available capacity for inventory reporting. If that changes (disk is added, disk goes offline, cluster node tanks, whatever), we want to modify the SSP's inventory accordingly. In this example, whichever virt gets there first, wins.
16:41:20 mriedem port creation was moving to conductor
16:41:30 sean-k-mooney mriedem: it was as part of the multi port binding spec
16:41:37 jaypipes sean-k-mooney: port binding != resource allocation. I'd rather have the conductor do the resource allocation steps than the nova-compute during port-binding
16:42:21 mriedem jaypipes: agree, otherwise we have a late claim on the wrong host and we fail
16:42:27 sean-k-mooney jaypipes: today yes port-binding happens on compute but intent is for it to move to conductor before we call the compute node
16:42:34 jaypipes mriedem: and we're back to square one again.
16:42:44 mriedem sean-k-mooney: i don't remember ever talking about moving port binding to conductor
16:42:50 sean-k-mooney let me see if i can get the relevent part of the spec
16:42:59 mriedem in the long ago, johnthetubaguy was working on moving port *creation* to conductor
16:43:07 efried To cdent's point, would y'all be on board with imposing a restriction in the code that MISC_SHARES_VIA_AGGREGATE and tree-ness are mutually exclusive?
16:43:15 jaypipes mriedem: and that is resource allocation, not port binding...
16:43:34 johnthetubaguy s/nodes/nods/
16:43:38 efried Freudian slip ^ :)
16:43:45 johnthetubaguy :)
16:43:47 jaypipes efried: I don't really see the need, frankly...
16:43:50 cdent efried: I think it's probably okay to not enforce it. just don't got parent or child hunting in u_p_t
16:44:00 mriedem jaypipes: port creation is resource allocation? in what way?
16:44:03 mriedem besides quota
16:44:08 efried jaypipes: It's not a *need* per se; would just allow us to kick this can down the road.
16:44:14 jaypipes efried: just don't ever search deeper than a single node level for any sharing providers
16:44:21 johnthetubaguy it was nice for case where you pick the correct IP segment for routed networks
16:44:40 johnthetubaguy where placement deals with the IP resources, that may be limited in the case of public ones
16:44:41 jaypipes mriedem: what johnthetubaguy said.
16:45:04 jaypipes mriedem: but it's not important right now.
16:45:28 jaypipes mriedem: I would be fine just mandating that sharing provider information gathering never exceeds the single level of the sharing provider.
16:45:34 sean-k-mooney mriedem: jaypipes https://git.openstack.org/cgit/openstack/nova-specs/tree/specs/queens/approved/neutron-new-port-binding-api.rst#n139
16:45:36 efried jaypipes: Okay, but do we include the sharing provider's whole tree or not?
16:45:44 mriedem efried: i'd say no
16:46:01 cdent no
16:46:02 efried I'm worried about the future impact of spoofing a child as a root
16:46:05 mriedem efried: i personally don't think the virt driver should be managing all of this stuff
16:46:18 jaypipes sean-k-mooney: oh, that's for live migration... yeah.
16:46:24 cdent efried: just because you don't include the info, doesn't mean it isn't out there
16:46:37 mriedem i mean, we've talked about not wanting nova to orchestrate everything in the cloud, and this is a giant step (it seems to me at least) in that direction of orchestratoin
16:46:44 jaypipes cdent: the truth is out there, Scully.
16:46:54 sean-k-mooney jaypipes: ya but the intent was to do the port binding always in the conductor form that point on. both binding and creation actully
16:46:54 cdent it's all lies jaypipes
16:46:55 efried cdent: Well, that's kind of the issue, because right now, we have no way in ProviderTree of creating a root with a parent_uuid
16:47:06 mriedem sean-k-mooney: https://git.openstack.org/cgit/openstack/nova-specs/tree/specs/queens/approved/neutron-new-port-binding-api.rst#n139 is not port creation
16:47:11 jaypipes efried: no, we should not. the "whole tree" is clearly more than "the single level of the node".
16:47:27 sean-k-mooney mriedem: right it happens after port creation
16:47:52 mriedem sean-k-mooney: that spec has no reliance on johnthetubaguy's work to move port creation to the super-conductor
16:47:56 mriedem as far as i remember anyway
16:48:07 mriedem it's for live migration, where we already have the port created on a live vm
16:48:15 cdent efried: all we want to do is represent a provider, in tree. you're saying that if a provider has a parent we can't _not_ include the parent in the provider tree?
16:48:22 mriedem john's thing was for picking the correct host for routed networks
16:48:36 efried cdent: No, I'm saying if we do include it, we have to lie about its parentage.
16:48:48 cdent no we don't. we just don't have it
16:48:51 sean-k-mooney mriedem: that spec was written with live migration but for bandwitdh based sceduling we needed to move port/creation to the conductor too so we can read the bandwith request form the neutron port before hitting placement
16:49:03 efried I suppose that's one way of looking at it.
16:49:07 mriedem sean-k-mooney: sure, the bw based scheduling spec is a whole other clusterfuck
16:49:15 mriedem sean-k-mooney: the live migration one for port binding is pretty clear
16:49:19 cdent The ProviderTree is a representation of what we care about, now. It is not the whole truth?
16:49:46 cdent I would hope that it is just whatever is useful.
16:49:46 sean-k-mooney mriedem: maybe we did not explictly say it in either i guess i was jsut under the impressions we had discussed makeing that change for all code paths
16:50:07 mriedem sean-k-mooney: the bw based scheduling spec has at least like 3 major dependencies
16:50:20 efried cdent: But please let's not make the argument that we shouldn't include the non-owned descendants of the compute host.
16:50:20 mriedem and at this rate will get done in maybe the next openstack A release
16:51:08 mriedem sean-k-mooney: the dep tree within just nova is already in LP https://blueprints.launchpad.net/nova/+spec/bandwidth-resource-provider
16:51:14 efried cdent: We have literally no way of knowing which ones those would be.
16:51:18 mriedem that doesn't include changes needed in neutron
16:51:39 sean-k-mooney mriedem: well we could strip it donw a little but ya i know its currently state it need many things like nova neutron negociation via os-vif object which is very nice but strictly required
16:51:56 mriedem you mean but *not* strictly required?
16:52:03 sean-k-mooney sorry yes
16:52:30 cdent efried: my feeling is that the core meaning of "ProviderTree" is "this compute node and its children". That it includes auxilliary things like shared providers is bonus.

Earlier   Later