Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-03-09
15:58:03 leakypipes superdan: and how does that signal to other agents running in other containers what the value should be that they look up?
15:58:36 superdan the same way as if this override yaml file is present and has a uuid set I guess?
15:59:26 superdan for the container file namespace issue, you can put that one option in its own conf file in a common location
15:59:39 superdan *filesystem namespace I mean
15:59:43 leakypipes superdan: how so? the agent may a) be managing multiple compute nodes and need information on multiple providers and b) needs some way of indicating what those compute node identifiers are.
16:00:10 leakypipes superdan: "one option in its own conf file" vs a standardized descriptor file format for resource providers?
16:00:25 superdan I'm confused.
16:00:31 leakypipes superdan: I would have thought you'd be supportive of not adding yet more configuration options.
16:00:51 mriedem leakypipes: this spec doesn't talk about non-nova agents managing multiple compute nodes
16:01:08 openstackgerrit Balazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider https://review.openstack.org/502306
16:01:23 mriedem i'm reading this all as 1:1 with a single compute node
16:02:21 leakypipes mriedem: I can add text about 1:M because that's what neutron agents do... many of them run on controller nodes and manage resources for hundreds of compute nodes.
16:03:46 mriedem sorry for my lack of neutron agent know-how, but that's not the case for OVS and LB is it?
16:03:55 mriedem otherwise how would os-vif plugins work for those?
16:04:03 mriedem since we don't use rpc or rest apis in os-vif
16:04:44 mriedem i'm not trying to be snarky, i just don't know the details on how neutron agents can all be deployed, and for which types of neutron backends
16:06:38 leakypipes in a container on a controller...
16:06:38 leakypipes mriedem: imagine a neutron agent that is managing network bandwidth resources for a rack of compute nodes. it needs to know the UUIDs (or compute node names) of those compute node resource providers so that it can add child providers to each compute node that represent the PFs that have a limited supply of ingress/egress bandwidth for physical networks. That agent could run on a single compute node or it could run on the TOR, or it could run
16:08:20 mriedem is that a thing today with the traditional OVS agent? or is this a neutron agent of the future?
16:08:32 giblet Kevin_Zheng, alex_xu_: I've updated the bandwidth spec based on the comments so far https://review.openstack.org/#/c/502306
16:08:33 mriedem like, a new agent service for managing placement type resource information
16:10:55 leakypipes mriedem: I'm not entirely sure where you're going with these questions... I can remove the multi-provider stuff from the spec, but frankly, I don't see why we'd want to limit ourselves to agents that only work with a single resource provider.
16:11:49 mriedem leakypipes: i'm just trying to understand. i don't know if i'm just retarded and don't know how neutron agents work *today*, or if you're talking about something that could be built off of this spec for a new neutron agent in the future. but if it's a problem then i'll just stop asking questions.
16:11:58 leakypipes superdan: to answer your question above about "if we just let them explicitly set the name on the provider, then can't we avoid providing a generic override mechanism so that other services can use the name to sync up?", we will need to provide trait override information in the (near) future -- think about the whole forbidden traits and cpu_model stuff. Do you propose adding more CONF options for trait overrides as well?
16:12:34 leakypipes mriedem: I'm feeling ganged up on.
16:12:37 mriedem operators, or external services, can set traits
16:12:50 mriedem nova just needs to not overwrite them
16:13:07 mriedem i don't think we need config options for that
16:13:12 leakypipes mriedem: and how do we signal to nova not to override *some* traits but not others?
16:13:13 mriedem it's why we have the rest api
16:13:21 mriedem CUSTOM_?
16:13:48 mriedem i thought there was some discussion about that at the ptg about overriding standard vs custom traits
16:13:50 mriedem would have to look
16:13:58 leakypipes mriedem: that's problematic because the specific traits we need to prevent overriding are things like CPU features when CONF.cpu_model is set.
16:15:04 mriedem L494 https://etherpad.openstack.org/p/nova-ptg-rocky
16:15:44 superdan leakypipes: I'm not ganging up on you, I promise
16:16:13 superdan leakypipes: I think we can avoid overwriting traits easily without any config by having things like the virt driver assert which ones it is authoritative over and leaving others alone
16:16:36 mriedem wrt merging traits i'd be fine with just not touching any traits that aren't in the set that we (nova-compute) says should be on the provider
16:16:39 mriedem the compute node provider
16:16:46 mriedem standard or custom
16:17:15 mriedem but if n-cpu says trait X should be on the provider, and someone removed it, then we're going to end up putting it back on
16:17:27 leakypipes superdan: how are you proposing that the virt driver indicate which traits it is authoritative over?
16:17:38 superdan leakypipes: like I said in the room,
16:18:05 superdan some sort of contract between compute and virt, where virt returns all the traits and a "assert", "de-assert", "dont-care" sort of thing
16:18:19 openstackgerrit Balazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider https://review.openstack.org/502306
16:18:25 superdan compute looks at the traits currently on a provider, and doesn't touch any that are not mentioned or listed as dont-care
16:18:31 superdan same for any compute-manager-owned traits
16:18:34 leakypipes superdan: and for non-virt things?
16:18:38 superdan same
16:19:00 mriedem i'm not yet sure why we need that level of granularity
16:19:11 leakypipes superdan: this "contract" of which you speak... that is my proposed provider config file.
16:19:25 superdan leakypipes: no, it's not
16:19:29 leakypipes yes, it is :)
16:20:10 mriedem the difference is the file is defined out of band, while the virt driver is in control of what it says should be on the provider
16:20:36 cdent presumably the contract has to live _somewhere_, or is just just "known" somehow?
16:20:43 mriedem the file is a proxy iow, so that nova can report stuff that the operator setup, w/o the operator having to set the traits on the provider directly in the rest api
16:22:00 mriedem cdent: i'm not yet sure why we need a contract, why we don't just merge in the common sense of what merge means
16:22:27 mriedem RP has traits x,y,z, nova-compute says it should also have 1,2,3, so we merge those and the RP has 1,2,3,x,y,z
16:22:53 mriedem nova-compute gets its traits from the virt driver
16:23:07 leakypipes mriedem: the file is saying "for this resource provider, ignore these auto-discovered traits or always use these traits even if you didn't auto-discover them.
16:23:38 mriedem leakypipes: so the file would have the forbidden traits logic built in?
16:23:59 leakypipes mriedem: yes, that's what the always and ignore sections are.
16:24:00 mriedem like pci whitelist, but traits blacklist
16:24:47 leakypipes mriedem: though the forbidden traits thing is more about *requesting* a specific traits *not* be present on a provider.
16:24:56 mriedem idk, i can't say i like the proxy aspect of this when we have a rest api for placement
16:25:19 leakypipes mriedem: which is different from the provider config YAML file which is saying what always should be set or always should be ignored for a provider.
16:25:23 mriedem it would be one thing if the traits were only set in the compute_nodes table and we didn't have a REST API to manipulate that data externally
16:26:13 mriedem the ignore case is the CPU features thing when host_model is set?
16:26:29 mriedem the 'always set these traits' thing can be done externally from nova
16:26:41 leakypipes mriedem: we absolutely *do* have a REST API for manipulating things. the provider config YAML stuff is a proposal for how to solve two primary problems: 1) notifying external agents about the identifier for a provider it needs to know about and 2) resolving the "admin set this thing and agent overwrote it" problem.
16:27:14 mriedem leakypipes: i'm talking about different rest apis
16:27:19 openstackgerrit Merged openstack/nova master: Fix indentation in doc/source/cli/* https://review.openstack.org/549166
16:27:36 mriedem my point is that since we do have a placement rest api to set traits on a provider, we do'nt need nova to proxy that information via a yaml file
16:28:05 mriedem i get (1)
16:28:13 leakypipes mriedem: and what happens with the compute node auto-discover traits stuff?
16:28:21 mriedem for (2) if we simply merge the traits set externally with the traits that nova-compute reports, then we should be fine
16:28:53 leakypipes mriedem: somebody is going to overwrite somebody else. the provider config file is a solution to instruct things not to overwrite (or to always overwrite) certain attributes.
16:29:24 mriedem why does somebody need to overwrite somebody else?
16:29:39 mriedem that's what we do today, yes, but we can change that to merge
16:29:48 leakypipes mriedem: let's say the virt driver reports AVX2 trait and somebody removed that trait. the virt driver will just continue to report that trait if we "merge traits".
16:29:59 mriedem right
16:30:09 openstackgerrit Lee Yarwood proposed openstack/nova master: trivial: Rename reboot kwarg to hard_reboot https://review.openstack.org/551289
16:30:15 leakypipes mriedem: so we need a way of instructing the virt driver not to report that trait.
16:30:25 mriedem why would someone remove it, is what i'm trying to get at
16:31:10 leakypipes mriedem: ok, let's use allocation ratio as a different example. say the virt driver uses an allocation ratio, but an admin has externally set the allocation ratio. who wins?
16:31:56 leakypipes mriedem: superdan's proposal was to add 3 more default_xxx_allocation_ratio CONF options to control this behaviour. My proposal is put all these things into a single provider config file format.
16:32:20 mriedem i'm not sure if we'd be able to tell if the allocation ratio was changed externally
16:32:34 mriedem we would see that it's different,
16:32:44 mriedem but it could be different because of a config change and restart of nova-compute
16:32:51 leakypipes mriedem: the absence of an allocation_ratio attribute on an inventory record int he provider config file format means "don't change this".
16:33:04 kaisers_ Hi! Can somebody pls. point me to where the QEMU_VERSION var in nova/virt/images.py is set from? I've an env running qemu-img 2.5.0 but nova (master) keeps on identifying this as 2010001 (which seemingly is 2.10.something) at that point which breaks things... :)
16:33:23 cdent (I think the trait removal is likely going to be pretty common (If I'm understanding it right): you buy some hardware that exposes cpus traits that you don't want to allow to limit or drive scheduling, so you need to say "on this machine turn off cpu trait barney")
16:33:23 mriedem kaisers_: nova.virt.libvirt.driver
16:33:27 mriedem somewhere in init_host or something
16:33:36 kaisers_ mriedem: thanks
16:34:35 mriedem cdent: wouldn't the operator just not configure the flavor to require trait barney?
16:35:00 cdent there are other flavors that do want barney
16:35:13 cdent but these machines are not for that
16:35:18 mriedem aggregates then
16:35:19 mriedem ?

Earlier   Later