Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-09-24
14:25:23 openstackgerrit Sylvain Bauza proposed openstack/nova master: WIP: libvirt: implement reshaper for vgpu https://review.openstack.org/599208
14:26:01 mriedem https://review.openstack.org/#/c/213433/
14:26:09 mriedem ^ ftersin's old patch to name the volumes that nova creates
14:26:27 dansmith yeah, you said that, but... what about the volume/
14:26:38 dansmith meaning, lots of people ask for volume_type, but was that the only one ask for name
14:26:39 dansmith ?
14:27:09 mriedem idk
14:27:23 mriedem i wouldn't be surprised if others would come out of the woodwork asking for proxying the name later
14:28:14 dansmith where does it end?
14:29:13 mriedem that's what i said above
14:29:19 dansmith I know
15:02:05 efried jaypipes: "efried: I specifically left out the "identification of the provider before you need it" because the clients of such a descriptor file would undoubtedly have different ideas of how to map local identifiers to RP identifiers."
15:02:44 efried jaypipes: If we're talking about RPs representing devices, yeah, which is what those last two specs are trying to define.
15:03:09 efried jaypipes: And those specs are attempting to account for the differences in hypervisors etc.
15:03:25 efried jaypipes: But what about e.g. NUMA node RPs?
15:03:41 jaypipes efried: yes. that is why I didn't put multiple providers in a single provider descriptor file and left it up to the caller to determine whether they use something like a directory with descriptor files named for the provider UUID or the provider's "local name" (NUMA0, compute_node, some PCI address, whatever...)
15:04:38 jaypipes efried: each hypervisor (or thing like Cyborg) is going to have its own way of identifying local devices.
15:04:54 efried I guess the same thing applies: as long as we've specified/documented how those RPs are going to be named, presumably the consumer can figure out what the names are going to be beforehand.
15:04:57 jaypipes efried: and therefore each hypervisor needs to "own" the mapping of its local device name to the resource provider UUID
15:05:13 jaypipes efried: yes, exactly my point.
15:05:13 efried well, yeah, but not everything is a device.
15:05:37 bauzas mmmm
15:05:49 jaypipes efried: if libvirt wants to call its root compute node provider "compute_node_{hostname}" cool. but Xen might call it, e.g. "dom0_{hostname}"
15:06:01 jaypipes efried: my point being we don't want to hard-code the names of things.
15:06:01 bauzas so the problem is to know which is which, right?
15:06:25 efried cdent's concern was that we don't want to require the consumer to go get a report from placement in order to figure out what's named what and then populate the file. His point was that that would be a PITA for deployment tools like ansible.
15:06:54 efried jaypipes: Oh, certainly don't want to hardcode the name of anything - couldn't if we wanted to.
15:07:40 efried but I guess we *do* need to make sure that the names are generated in a deterministic fashion that an operator can reproduce.
15:07:49 jaypipes efried: well, that's essentially what cdent's gripe involves: hard-coding the name of the compute node resource provider so tools like ansible can have a stable way of calling things like `openstack provider-inventory $HOSTNAME`
15:08:13 efried We've got to have a starting point.
15:08:26 efried and we've already established that part, I thought.
15:08:34 jaypipes efried: what part?
15:08:56 efried The root compute RP's UUID (the host UUID) and name (the node UUID) - right?
15:09:13 efried in non-ironic those are the same
15:09:15 jaypipes efried: yes, that ship is sailed.
15:09:22 efried okay, and I don't think that's a bad thing.
15:09:29 jaypipes efried: even for ironic.
15:09:36 jaypipes it's just the compute_nodes.uuid value that is used.
15:09:59 jaypipes efried: hold up, that's not true.
15:09:59 efried um, I thought for ironic the node UUID was different because multiple nodes managed by one host.
15:10:02 efried yeah
15:10:06 efried anyway, beside the point
15:10:11 jaypipes efried: we use compute_nodes.hypervisor_hostname
15:10:13 efried point is that we've already established a rule
15:10:13 jaypipes as the RP name.
15:10:59 jaypipes https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L1417
15:11:52 jaypipes efried: ^
15:12:19 efried yeah, but didn't we decide hypervisor_hostname was actually the node UUID in the ironic case?
15:12:24 jaypipes efried: so we already use hypervisor_hostname as RP name.
15:12:32 mriedem efried: yes it is
15:12:35 jaypipes efried: oh, I see what you mean.
15:12:39 jaypipes efried: yes, that is true.
15:12:47 jaypipes it's the Ironic node UUID.
15:13:22 efried right, okay, so I was wrong before that the RP UUID and name were the same for non-ironic; the name is actually the hypervisor hostname which is like text.
15:13:39 efried anyway, point is, we've got an established rule for both the name and the UUID of the root RP.
15:13:56 efried And in Denver we talked about using the parent RP's UUID as part of the child RP's name, to ensure name uniqueness
15:14:09 efried So that's one part of the naming rules
15:14:59 efried For each type of child RP as long as we make the remainder of the naming convention deterministic *and document it*, that should allow consumers to figure out how to identify them for purposes of their inventory file.
15:15:45 efried It may be kind of brittle in practice, but I can't think of a better solution.
15:16:31 jaypipes efried: I guess I'd prefer to use the root provider's *name*, not UUID, in the child provider's name.
15:16:58 jaypipes efried: and yes, I realize for Ironic, any child providers would have a UUID-based name since the root provider name is a UUID.
15:17:08 efried if we want the convention to be extensible, we should use something about the parent provider, not the root provider
15:17:17 efried which only gets different when we have >2 levels of tree
15:17:38 jaypipes efried: yes, the parent.
15:17:57 efried but if we use the name, then we'll have N-1 UUIDs in the name for a provider at the Nth level down, which is :(
15:18:21 jaypipes efried: so for a SRIOV NIC under the first NUMA node on a compute node might be cn1_numa0_sriovpf0 or something like that?
15:18:55 jaypipes efried: I don't understand your last comment. could you elaborate?
15:19:08 efried Sorry, I wasn't being as clever as you.
15:19:32 efried I was thinking dumbly $PARENT_NAME - $MY_NAME
15:19:52 efried Which means your example would have ended up being cn1_cn1_numa0_sriovpf0
15:20:23 efried um
15:20:25 jaypipes ah. no, just use the parent name, not the concat of the ancestry tree
15:20:26 efried that's not even right.
15:20:29 efried yeah
15:20:56 jaypipes so, the "formula" would be just: {$PARENT_NAME}_{$CHILD_NAME}
15:21:08 efried well, since names have to be unique, that should work just fine.
15:21:09 jaypipes or whatever separator you wanted to use instead of _
15:21:37 efried So we have to establish the naming convention for $CHILD_NAME, which will be different for any given type of provider x hypervisor/virt
15:21:44 efried and document it
15:21:55 efried and then operators ought to be able to figure it out from there
15:21:56 efried if they need to
15:21:59 jaypipes efried: do we really though?
15:22:13 efried well, if we don't, then they have to do the two-step
15:22:28 efried ask placement, figure it out, use that to populate the file, which then gets used to update placement.
15:22:33 openstackgerrit Aditya Vaja proposed openstack/nova stable/queens: fix typo in IVS related privsep method https://review.openstack.org/604817
15:22:34 jaypipes efried: I mean if the "use case" is just for deployment tooling to be able to list the inventory for a compute node and its children, we already have the one heuristic that would be needed.
15:22:54 jaypipes efried: i.e. ansbile would always call `openstack provider inventory list $HYPERVISOR_HOSTNAME`
15:23:11 efried isn't there a chicken/egg though?
15:23:15 jaypipes efried: and to find the child providers it would be:
15:23:26 jaypipes `openstack provider list --in-tree $HYPERVISOR_HOSTNAME`
15:24:03 jaypipes efried: well, of course, there's not going to be any inventory or provider records until the nova-compute runs, but I don't think that's any different from today's landscape for ansible/deploy tools
15:24:23 efried it is
15:24:39 efried because in today's tooling, you can set cpu_allocation_ratio beforehand
15:25:08 jaypipes and?
15:25:26 efried Well, I don't know if we really care enough to feel it's worth the hoops of fire it would take to make this new world similarly configurable-before-deployment.
15:25:36 efried But if I'm understanding cdent's concern correctly, that was it.
15:25:36 jaypipes what regarding the provider descriptor file format would prevent ansible from writing out an allocation ratio override for the compute node?
15:26:01 efried oh, that's exactly the point. The allocation ratio for CPU may not *live* in the compute node provider.
15:26:07 efried It may live in the (multiple) NUMA node providers
15:26:29 efried which ansible would need a way to figure out how to identify before deployment, if we're wanting to keep this paradigm of pre-deployment configurability.

Earlier   Later