Posted	Nick	Remark
#openstack-nova - 2018-09-24
14:28:14	dansmith	where does it end?
14:29:13	mriedem	that's what i said above
14:29:19	dansmith	I know
15:02:05	efried	jaypipes: "efried: I specifically left out the "identification of the provider before you need it" because the clients of such a descriptor file would undoubtedly have different ideas of how to map local identifiers to RP identifiers."
15:02:44	efried	jaypipes: If we're talking about RPs representing devices, yeah, which is what those last two specs are trying to define.
15:03:09	efried	jaypipes: And those specs are attempting to account for the differences in hypervisors etc.
15:03:25	efried	jaypipes: But what about e.g. NUMA node RPs?
15:03:41	jaypipes	efried: yes. that is why I didn't put multiple providers in a single provider descriptor file and left it up to the caller to determine whether they use something like a directory with descriptor files named for the provider UUID or the provider's "local name" (NUMA0, compute_node, some PCI address, whatever...)
15:04:38	jaypipes	efried: each hypervisor (or thing like Cyborg) is going to have its own way of identifying local devices.
15:04:54	efried	I guess the same thing applies: as long as we've specified/documented how those RPs are going to be named, presumably the consumer can figure out what the names are going to be beforehand.
15:04:57	jaypipes	efried: and therefore each hypervisor needs to "own" the mapping of its local device name to the resource provider UUID
15:05:13	jaypipes	efried: yes, exactly my point.
15:05:13	efried	well, yeah, but not everything is a device.
15:05:37	bauzas	mmmm
15:05:49	jaypipes	efried: if libvirt wants to call its root compute node provider "compute_node_{hostname}" cool. but Xen might call it, e.g. "dom0_{hostname}"
15:06:01	jaypipes	efried: my point being we don't want to hard-code the names of things.
15:06:01	bauzas	so the problem is to know which is which, right?
15:06:25	efried	cdent's concern was that we don't want to require the consumer to go get a report from placement in order to figure out what's named what and then populate the file. His point was that that would be a PITA for deployment tools like ansible.
15:06:54	efried	jaypipes: Oh, certainly don't want to hardcode the name of anything - couldn't if we wanted to.
15:07:40	efried	but I guess we *do* need to make sure that the names are generated in a deterministic fashion that an operator can reproduce.
15:07:49	jaypipes	efried: well, that's essentially what cdent's gripe involves: hard-coding the name of the compute node resource provider so tools like ansible can have a stable way of calling things like `openstack provider-inventory $HOSTNAME`
15:08:13	efried	We've got to have a starting point.
15:08:26	efried	and we've already established that part, I thought.
15:08:34	jaypipes	efried: what part?
15:08:56	efried	The root compute RP's UUID (the host UUID) and name (the node UUID) - right?
15:09:13	efried	in non-ironic those are the same
15:09:15	jaypipes	efried: yes, that ship is sailed.
15:09:22	efried	okay, and I don't think that's a bad thing.
15:09:29	jaypipes	efried: even for ironic.
15:09:36	jaypipes	it's just the compute_nodes.uuid value that is used.
15:09:59	jaypipes	efried: hold up, that's not true.
15:09:59	efried	um, I thought for ironic the node UUID was different because multiple nodes managed by one host.
15:10:02	efried	yeah
15:10:06	efried	anyway, beside the point
15:10:11	jaypipes	efried: we use compute_nodes.hypervisor_hostname
15:10:13	efried	point is that we've already established a rule
15:10:13	jaypipes	as the RP name.
15:10:59	jaypipes	https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L1417
15:11:52	jaypipes	efried: ^
15:12:19	efried	yeah, but didn't we decide hypervisor_hostname was actually the node UUID in the ironic case?
15:12:24	jaypipes	efried: so we already use hypervisor_hostname as RP name.
15:12:32	mriedem	efried: yes it is
15:12:35	jaypipes	efried: oh, I see what you mean.
15:12:39	jaypipes	efried: yes, that is true.
15:12:47	jaypipes	it's the Ironic node UUID.
15:13:22	efried	right, okay, so I was wrong before that the RP UUID and name were the same for non-ironic; the name is actually the hypervisor hostname which is like text.
15:13:39	efried	anyway, point is, we've got an established rule for both the name and the UUID of the root RP.
15:13:56	efried	And in Denver we talked about using the parent RP's UUID as part of the child RP's name, to ensure name uniqueness
15:14:09	efried	So that's one part of the naming rules
15:14:59	efried	For each type of child RP as long as we make the remainder of the naming convention deterministic *and document it*, that should allow consumers to figure out how to identify them for purposes of their inventory file.
15:15:45	efried	It may be kind of brittle in practice, but I can't think of a better solution.
15:16:31	jaypipes	efried: I guess I'd prefer to use the root provider's *name*, not UUID, in the child provider's name.
15:16:58	jaypipes	efried: and yes, I realize for Ironic, any child providers would have a UUID-based name since the root provider name is a UUID.
15:17:08	efried	if we want the convention to be extensible, we should use something about the parent provider, not the root provider
15:17:17	efried	which only gets different when we have >2 levels of tree
15:17:38	jaypipes	efried: yes, the parent.
15:17:57	efried	but if we use the name, then we'll have N-1 UUIDs in the name for a provider at the Nth level down, which is :(
15:18:21	jaypipes	efried: so for a SRIOV NIC under the first NUMA node on a compute node might be cn1_numa0_sriovpf0 or something like that?
15:18:55	jaypipes	efried: I don't understand your last comment. could you elaborate?
15:19:08	efried	Sorry, I wasn't being as clever as you.
15:19:32	efried	I was thinking dumbly $PARENT_NAME - $MY_NAME
15:19:52	efried	Which means your example would have ended up being cn1_cn1_numa0_sriovpf0
15:20:23	efried	um
15:20:25	jaypipes	ah. no, just use the parent name, not the concat of the ancestry tree
15:20:26	efried	that's not even right.
15:20:29	efried	yeah
15:20:56	jaypipes	so, the "formula" would be just: {$PARENT_NAME}_{$CHILD_NAME}
15:21:08	efried	well, since names have to be unique, that should work just fine.
15:21:09	jaypipes	or whatever separator you wanted to use instead of _
15:21:37	efried	So we have to establish the naming convention for $CHILD_NAME, which will be different for any given type of provider x hypervisor/virt
15:21:44	efried	and document it
15:21:55	efried	and then operators ought to be able to figure it out from there
15:21:56	efried	if they need to
15:21:59	jaypipes	efried: do we really though?
15:22:13	efried	well, if we don't, then they have to do the two-step
15:22:28	efried	ask placement, figure it out, use that to populate the file, which then gets used to update placement.
15:22:33	openstackgerrit	Aditya Vaja proposed openstack/nova stable/queens: fix typo in IVS related privsep method https://review.openstack.org/604817
15:22:34	jaypipes	efried: I mean if the "use case" is just for deployment tooling to be able to list the inventory for a compute node and its children, we already have the one heuristic that would be needed.
15:22:54	jaypipes	efried: i.e. ansbile would always call `openstack provider inventory list $HYPERVISOR_HOSTNAME`
15:23:11	efried	isn't there a chicken/egg though?
15:23:15	jaypipes	efried: and to find the child providers it would be:
15:23:26	jaypipes	`openstack provider list --in-tree $HYPERVISOR_HOSTNAME`
15:24:03	jaypipes	efried: well, of course, there's not going to be any inventory or provider records until the nova-compute runs, but I don't think that's any different from today's landscape for ansible/deploy tools
15:24:23	efried	it is
15:24:39	efried	because in today's tooling, you can set cpu_allocation_ratio beforehand
15:25:08	jaypipes	and?
15:25:26	efried	Well, I don't know if we really care enough to feel it's worth the hoops of fire it would take to make this new world similarly configurable-before-deployment.
15:25:36	efried	But if I'm understanding cdent's concern correctly, that was it.
15:25:36	jaypipes	what regarding the provider descriptor file format would prevent ansible from writing out an allocation ratio override for the compute node?
15:26:01	efried	oh, that's exactly the point. The allocation ratio for CPU may not *live* in the compute node provider.
15:26:07	efried	It may live in the (multiple) NUMA node providers
15:26:29	efried	which ansible would need a way to figure out how to identify before deployment, if we're wanting to keep this paradigm of pre-deployment configurability.
15:26:57	jaypipes	efried: right, but we don't currently set any allocation ratio for NUMA node providers anywhere (certainly not in any config file)
15:27:10	efried	NUMA is just an example.
15:27:16	efried	and
15:27:27	efried	we don't today, but might tomorrow
15:27:29	jaypipes	efried: this is why I think setting data for inventory records using configuration files is silly.
15:27:48	efried	you think it should be done by invoking placement after all the auto setup is done?
15:28:01	jaypipes	as opposed to setting them en-masse after the records have been created.
15:28:07	jaypipes	yes.

Earlier   Later