Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-01-29
15:03:57 cdent let me explain
15:04:10 cdent given two providers with equal inventory, one has trait X
15:04:21 jaypipes efried: do you want me to respond to the ML part and not the private email?
15:04:55 cdent we make a query for allocation candidates that both poviders can satisfy
15:05:06 cdent how do we say "don't use the provider that has trait X"
15:05:07 openstackgerrit Stephen Finucane proposed openstack/nova stable/pike: tests: Use correct response type in tests https://review.openstack.org/538938
15:05:19 efried Perhaps it would be best if cdent replied to the ML one with whatever's still relevant from the private thread; then you can respond to that. Keep the thread linear-ish.
15:05:21 openstackgerrit Stephen Finucane proposed openstack/nova stable/pike: Handle images with no data https://review.openstack.org/538939
15:05:24 gibi ameeda: sure, you can get sysmeta from the db, but I thought, based on the comments, that you are trying to find a way to get sysmeta via the REST API
15:05:25 jaypipes cdent: you don't. you just don't include the trait in required_traits= param.
15:05:36 jaypipes cdent: it's a sorting/weighing/preference problem, not a filtering problem.
15:05:54 cdent jaypipes: right, that's basically what I say at the end of my message
15:05:58 openstackgerrit Stephen Finucane proposed openstack/nova stable/pike: Handle images with no data https://review.openstack.org/538939
15:05:59 jaypipes gotcha
15:06:11 efried cdent But I don't see how that's relevant to the discussion
15:06:34 efried I totally didn't pick up on any part of rado's original note that implied we needed/wanted to do that.
15:07:11 mriedem stephenfin: question in that backport
15:07:26 cdent efried: rado was expressing concern about allocations being made to the wrong pool when not expressing the trait
15:07:27 jaypipes efried: the fundamental problem with the vmware conversation is that resource pools are being represented with nested providers instead of aggregates. aggregates are the thing that provides the "grouping" concept that the resource pools represent. nested providers represent a different relationship: that of dependent trees of providers.
15:07:42 cdent "is there a way to make the scheduler allocate only from one specific RP if no custom trait is specified in the request"
15:08:09 efried Heh. Oh. I took a much simpler approach to that question. It went like this: "No."
15:08:58 cdent The complicating factor here is that I think nested is _exactly_ the model for how resource pool are supposed to work
15:08:59 efried jaypipes ++ and in fact trying to use NRP for pools will break horribly
15:09:21 efried oh, wow, okay, clearly starting on opposite ends of the rope.
15:09:37 efried cdent Can you describe that hierarchy please?
15:09:41 cdent because they represent a hierarcy of containment
15:10:21 cdent so, nrp came about in part because of numa and pci relationships, right?
15:10:30 efried cdent Have you seen the response I made to the ML yet? I want to baseline and make sure I've got my idea of the hierarchy down before we go any further.
15:10:47 cdent efried: yes, I've read it twice and still trying to make sure i understand it
15:10:54 efried k
15:11:34 stephenfin mriedem: Reply left. tl;dr: I think we're fine for stable/pike
15:11:55 stephenfin assuming I'm reading g-r correctly
15:12:05 mriedem we just need 2.8.0?
15:12:13 cdent the thing about your model which is not necessarily desirable is the way in which esxi hosts are managed as a concrete resources. vcenter doesn't really want to do that (although it is possible and would be a reasonable fallback)
15:12:18 stephenfin mriedem: 2.6.0+
15:13:02 efried jaypipes The aggregate model won't allow you to request a deploy to a specific resource pool. But maybe I'm misunderstanding the original intent, which I thought was to do just that.
15:13:42 cdent the intent is to deploy to a specific resource pool
15:14:12 cdent if you have an aggregate for each specific resource pool, that might be workabe
15:14:30 efried But you still can't deploy to a specific resource pool with just that.
15:14:42 cdent brb
15:14:44 efried You would have to have some other mechanism in place.
15:14:45 mriedem can you add a custom filter and custom scheduler hint which is just 'make it go directly here'
15:15:10 jaypipes efried: uhm, yeah you could request a specific aggregate...
15:15:19 efried jaypipes How?
15:15:28 efried With some mechanism yet to be designed?
15:15:46 jaypipes efried: we already have the member_of param. we can work that in to alloc candidates very easily.
15:16:08 efried okay, agree, but not yet designed or started.
15:16:23 jaypipes efried: of course, that would require the users knowing about the aggregate UUID/names...
15:16:31 jaypipes efried: which generally we have not encouraged.
15:16:41 mriedem stephenfin: got it
15:16:42 mriedem thanks
15:17:14 mriedem there is no aggregate name https://developer.openstack.org/api-ref/placement/#list-resource-provider-aggregates
15:17:33 jaypipes mriedem: sorry, UUID...
15:17:58 jaypipes efried: still, what is the root of the problem that is trying to be solved?
15:18:32 cdent It remains unclear to me why (eventually) nested isn't the ideal solution for this stuff? A compute-node is an rp, it cotains resource pool rps and maybe even some exsi rp, a hybrd representing the stuff managed by the conpute node
15:18:39 jaypipes efried: I mean, are we doing scheduling here or are we just doing resource tracking and vcenter basically takes over the scheduling/placement decisions (that's what I *thought* the situation was currently...)
15:18:59 mriedem sounds like the latter
15:19:13 stephenfin mriedem: Is this something that should wait for a few weeks? https://review.openstack.org/#/c/534371/
15:19:38 mriedem stephenfin: is that just cleanup? if so, yes
15:19:45 efried sorry guys, pulled into a quick meeting, back in a bit.
15:19:52 mriedem stephenfin: we're in bug fix only mode at this point
15:19:53 stephenfin mriedem: (y)
15:19:55 mriedem or tests, docs, etc
15:22:04 cdent jaypipes: the underlying feature that rado wants to expose here are resource pools and he wants to do it in a way that allows placement to be aware of them
15:22:52 jaypipes cdent: will the *end user* be capable of a) understanding what these resource pools are and b) asking for a particular resource pool to be used by an instance?
15:23:23 cdent jaypipes: yes. "tenant virtual data center"
15:23:43 stephenfin lyarwood (or another stable-core): fancy hitting these? https://review.openstack.org/#/c/538939/ https://review.openstack.org/#/c/538938/
15:24:02 jaypipes cdent: why not have multiple nova-compute services, each managing a single "resource pool"?
15:26:44 cdent because the clusters behind a nova-compute service are intentionally dynamic: it's desirable to be able to create destroy and change resource pools in the cluster without messing with the openstack control plane much
15:27:08 efried I'm back.
15:27:56 efried cdent Do these clusters / resource pools (are those different things) contain single compute-ish nodes to which instances get deployed, OR do they contain some general pool of resources from which an instance could be deployed using some resource from one member and some from another?
15:28:02 jaypipes cdent: that's problematic for many reasons, as you know.
15:29:44 cdent jaypipes: yes, but problematic is how we push the ball forward, no? efried cluster and resource pools are different, yes. logically an instance is deployed to a resource pool and not a specific host (although physically they are, of course)
15:30:01 cdent a cluster contains 1 or more resource pools
15:30:29 cdent a resource pool has access to some (dynamic) subsection of a clusters inventory
15:31:24 efried Are physical hosts the underlying members of a resource pool?
15:32:25 cdent efried: that's not really the way it is modeled, no, but I'm not sure what you're actually asking
15:32:42 efried cdent I'm just not understanding where the physical resources are here.
15:32:44 jaypipes cdent: what does the vmware virt driver *currently* do? does it report the entire amount of VCPU, memory, etc for all resource pools and all clusters that vCenter knows about?
15:33:27 jaypipes cdent: and the scheduler always picks the single nova-compute service daemon and basically all scheduling/placement is simply deferred to the vCenter API?
15:33:35 cdent right now: the vcenter represents one cluster, and reports all vcpu, memory, etc that it knows abouts
15:34:32 cdent but there are a multiple nova-copute services, each for a given cluster
15:34:50 jaypipes cdent: I'm not seeing the benefit of modeling things using placement's resource providers data modeling if deployments using vCenter won't actually end up using any of that information when doing placement/scheduling decisions anyway.
15:37:40 cdent jaypipes: two reasons a) if you have multiple clusters, and thus multiple nova-compues, and those clusters differet in various ways (for example by having different tenant virtual data centers) you need to scheduler differently, b) despite our troubles, we've made a very good resource/provider/inventory/etc modelling system with placement and it is a good place to track and select where to place things
15:38:35 cdent "this cluster over there is the one with all the GPUs"
15:38:43 efried cdent It'll help me a lot to understand what is underneath a resource pool. If a resource pool reports a total of 8 VCPUs available, does that mean I am guaranteed to be able to satisfy a deploy requesting 8 VCPUs? (All other things being available)
15:39:27 efried Cause I got the impression that those 8 VCPUs might be spread across say four physical hosts, and an instance could only go to one of those, so the biggest one you could handle might only be 2 VCPUs or whatever.
15:40:20 jaypipes don't see the point.
15:40:20 jaypipes cdent: for a) if the end user is just saying "I'm in tenant X, so launch an instance on virtual data center X", that is a stupid-simple scheduling decision that needs nothing other than a single trait (CUSTOM_TENANT_X decorates a single resource provider representing the nova-compute service that manages that tenant virtual data center). For b) again, if vCenter isn't going to use that information in its intra-cluster scheduling decisions, I
15:42:02 jaypipes cdent: another question for you..
15:42:22 cdent jaypipes: can I pause you for a minute while I answer efried, then your current question?
15:43:11 jaypipes cdent: even if placement ended up getting proper inventory information for these vCenter clusters, would VMWare OpenStack users actually be able looking at that inventory information? Or would they be using VMWare GUIs to look at inventory information for their specific tenant virtual datacenter?
15:43:19 jaypipes cdent: of course, no worries
15:44:42 cdent efried: a while back there were some adjustment made to the vmware driver to make sure that the max_unit was no greater than what any individual host could provide. so your second statement about "the biggest one" is an issue in some cases, but it's squishy because of the DRS (dynamic resource scheduler thing which can do unattended live migrations and what not, magic stuff I haven't dug into)
15:46:24 efried So it is in fact the case that there are some number of physical hosts behind a resource pool, and an instance can only be deployed to one, not spread out across more than one. True statement?
15:46:26 cdent efried: so, yes, there is a fair bit of complexity which is _why_ being able to manipulate the entire provider tree from within the virt driver has been of such interest to rado (and me).
15:46:49 jaypipes maciejjozefczyk: you around? wanted to chat with you about https://bugs.launchpad.net/nova/+bug/1742747
15:46:50 openstack Launchpad bug 1742747 in OpenStack Compute (nova) "RT overrides default allocation_ratios for ram cpu and disk" [Undecided,In progress] - Assigned to Maciej Jozefczyk (maciej.jozefczyk)
15:46:56 cdent efried: it can move and you can never know that it did
15:47:52 efried "it" the physical host? "can move" to another resource pool? and "you" the virt driver or you the operator?

Earlier   Later