| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-30 | |||
| 15:09:55 | bauzas | ack, thanks | |
| 15:09:58 | jaypipes | np | |
| 15:10:06 | mriedem | cfriesen: also, poutine, wtf?! | |
| 15:10:07 | bauzas | because I'm just litterally working on the implementation | |
| 15:10:20 | cfriesen | mriedem: good poutine is awesome. crappy poutine sucks | |
| 15:10:26 | stephenfin | jaypipes: I'm guessing you can't ignore NUMA completely though, if you want to support use case 4 and the likes? https://review.openstack.org/#/c/555081/10/specs/rocky/approved/cpu-resources.rst@425 | |
| 15:10:27 | bauzas | jaypipes: and I'm hold until we get consensus on the right approach | |
| 15:10:42 | bauzas | jaypipes: and FWIW, naichuan's implementation for Xen is impacted too | |
| 15:11:40 | jaypipes | stephenfin: no, I specifically *don't* want to include the realtime extra specs or any of the numa_nodes/numa_cells goo. | |
| 15:12:02 | jaypipes | stephenfin: because the purpose of that spec is to handle requests for dedicated and shared CPU resources. nothing to do with NUMA really. | |
| 15:12:17 | sahid | jangutter: i imagine what you are going to do it's add in nova a new condition for vif detail vrouter, which will enable the multiqueue=True | |
| 15:12:27 | mriedem | stephenfin: wasn't trying to bug you for reviews on my spec, just saying i'm in the same boat | |
| 15:12:30 | sahid | that under the section vhostuser | |
| 15:12:32 | jaypipes | stephenfin: and I'm deliberately trying to not bring in the awful coupling of NUMATopology and InstanceNUMATopology and the horrible extra specs that serve to construct that mess. | |
| 15:12:34 | mriedem | stephenfin: we can happily drown together | |
| 15:12:37 | mriedem | you can be leo | |
| 15:12:39 | mriedem | i'll be kate | |
| 15:12:49 | mriedem | / who didn't drown / | |
| 15:12:51 | jaypipes | mriedem: wow, how chivalrous of you :P | |
| 15:13:12 | jangutter | sahid: actually, the idea is to migrate the 'vrouter' vif which is not vhostuser... | |
| 15:13:14 | openstackgerrit | Merged openstack/nova-specs master: Libvirt file backed memory https://review.openstack.org/563704 | |
| 15:13:37 | stephenfin | jaypipes: I've missed something so. I thought requesting 'resources1:PCPU=8 resources2:PCPU=8 group_policy=isolate' means host NUMA topology would need to be accounted for | |
| 15:13:43 | jangutter | sahid: I did some checking, and it looks like multiqueue is an instance-wide setting, not a vif-specific one. | |
| 15:14:30 | jaypipes | stephenfin: no. it has nothing explicitly to do with NUMA. that just means "I need 2 groups of 8 dedicated CPU resources. they need to be provided by two distinct resource providers" | |
| 15:14:36 | jangutter | sahid: ref: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L708 | |
| 15:15:21 | jaypipes | stephenfin: the fact that the compute node resource provider might be a parent to two other child providers that *represent* different sockets or NUMA cells is really not relevant to the request that two different providers of dedicated CPU supply the requested resource groups. | |
| 15:16:07 | sahid | jangutter: i'm not sure i understand what you want do? | |
| 15:16:20 | jaypipes | stephenfin: in other words, if some non-NUMA devices *happened* to expose PCPU inventory under a provider tree, they would meet the requirements... | |
| 15:16:25 | bauzas | jaypipes: stephenfin: just a point, we could have the same upgrade impact with VCPU resources being in child RPs than the one I described with http://lists.openstack.org/pipermail/openstack-dev/2018-May/130783.html | |
| 15:16:32 | stephenfin | jaypipes: Practically speaking so, requesting 'resources1:PCPU=8 resources2:PCPU=8 group_policy=isolate' will always fail until NUMA is modelled in placement? | |
| 15:16:40 | stephenfin | Ah, gotcha | |
| 15:16:41 | jaypipes | stephenfin: again, we are trying desperately to free ourselves of the NUMA coupling that has plagued all of this for years. | |
| 15:17:13 | jangutter | sahid: vrouter kernel mode plugging needs to create a tap with multiqueue enabled, if libvirt starts the VM up looking for it. | |
| 15:17:30 | stephenfin | jaypipes: But we're still coupled, no? I mean, we're going to need NUMA for the above request to work | |
| 15:17:42 | bhagyashris | efried: any suggestion about second approach | |
| 15:17:43 | jaypipes | stephenfin: I'm trying my best to treat NUMA as just the model of distances to various memory (and memory page) caches to a processor/bus, instead of coupling all that mess with CPU pinning. | |
| 15:18:07 | jangutter | sahid: the VIFGeneric portion can set the libvirt config correctly for multiqueue, but, it looks like it's going to be difficult to pass "multiqueue" on a per-vif basis without modding large portions of the os-vif plugging code. | |
| 15:18:09 | sahid | jangutter: ok so you plugin needs to know whether multiqueue is enable, which is via the os-vif object nova is filling then passing the the os-vif plugin | |
| 15:18:21 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add osprofiler config options to generated reference https://review.openstack.org/571225 | |
| 15:18:51 | jangutter | sahid: yep, so, either we can pass "multiqueue=True" via the vif os-vif object, or via the instance os-vif object. | |
| 15:19:59 | bauzas | jaypipes: unfortunately from what I understood from operators, pinning is there because NUMA affinity misses for PCI devices | |
| 15:20:09 | bauzas | jaypipes: so both are somehow related | |
| 15:20:16 | stephenfin | jaypipes: To be clear though, use case 4 in that spec cannot be achieved with modelling NUMA in placement, right? | |
| 15:20:19 | jangutter | sahid: the trick is, passing it via the VIF looks very tricky. A lot of code has to be added there, since, paradoxically, the choice whether to enable multiqueue depends on the instance and flavor settings, not VIF settings. | |
| 15:20:37 | stephenfin | jaypipes: Use case 4 = https://review.openstack.org/#/c/555081/10/specs/rocky/approved/cpu-resources.rst@425 | |
| 15:20:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add osprofiler config options to generated reference https://review.openstack.org/571225 | |
| 15:21:03 | bauzas | jaypipes: in terms of priority, I'd consider NUMA placement and NUMA affinity be prioritary to CPU pinning, because folks care more about proximity and throughput than real placement decisions | |
| 15:21:13 | openstackgerrit | Merged openstack/nova master: doc: Start using openstackdoctheme's extlink extension https://review.openstack.org/540888 | |
| 15:21:14 | sahid | jangutter: ok i see, I could see it live in instance actually | |
| 15:21:19 | bauzas | CPU pinning is just a hack | |
| 15:21:33 | stephenfin | Hurrah! The extlink patch merged at last | |
| 15:21:44 | jangutter | sahid: for example, you can spawn an instance asking for multiqueue, but only 1 cpu, and multiqueue will magically disable :-p | |
| 15:21:50 | jaypipes | bauzas: I'm referring to the poor software design of coupling the topology objects together with memory pages, usage of instance virtual NUMA topology on a host NUMA topology, and all that. | |
| 15:22:00 | bauzas | jaypipes: sure, I understood that point | |
| 15:22:03 | stephenfin | bauzas: I think jaypipes is suggesting we can do CPU pinning without _needing_ to think about NUMA affinity. They don't have to be tied | |
| 15:22:29 | bauzas | stephenfin: my own take is that CPU pinning isn't required if we go with NUMA affinity | |
| 15:22:30 | sahid | jangutter: yes i that is not what you want? | |
| 15:22:38 | stephenfin | But anyone that is using this will want to couple them, given the performance implications of not doing so | |
| 15:22:54 | stephenfin | bauzas: It isn't currently, no | |
| 15:22:57 | jangutter | sahid: That's perfectly acceptable, as long as both sides agree. | |
| 15:23:06 | bauzas | stephenfin: I don't disagree with your point :) | |
| 15:23:17 | jaypipes | stephenfin: use case 4 can be implemented, sure, but until sockets are modeled as child providers or a compute node parent provider, the scheduler would always return NoValidHosts for a request like use case 4. that clear? | |
| 15:23:26 | bauzas | stephenfin: I mean, CPU pinning is tied to NUMA resources because people care of throughput | |
| 15:23:45 | stephenfin | jaypipes: Sure, if s/sockets/NUMA nodes/ | |
| 15:23:48 | jangutter | sahid: another reason for keeping it as a parameter of the instance... | |
| 15:23:52 | stephenfin | because of Cluster-on-Die etc. | |
| 15:23:58 | jaypipes | stephenfin: sigh... | |
| 15:24:02 | stephenfin | Sorry :) | |
| 15:24:40 | jangutter | sahid: thanks, I'll respin. The nova bit is much easier to do with it being part of the instance. | |
| 15:24:56 | jaypipes | stephenfin: quick RST doc question for you.. | |
| 15:25:07 | stephenfin | Oooh, yay | |
| 15:25:12 | stephenfin | jaypipes: shoot | |
| 15:25:21 | jaypipes | stephenfin: how can I make the text of an anchor different between two calling points to that anchor? | |
| 15:25:43 | stephenfin | :ref:`text you want to display <text-of-anchor>` | |
| 15:25:45 | jaypipes | stephenfin: you rightfully pointed out that the granular request spec is linked in two places. | |
| 15:25:54 | jaypipes | stephenfin: ah.. | |
| 15:25:55 | sahid | ok i don't really see why it's reflecting better the state of the instance using multiqueue or not, but i'm do not disagree with you to have that attribute with instance | |
| 15:25:59 | jaypipes | stephenfin: ty sir | |
| 15:26:29 | sahid | jangutter: please ping me when you send a new version | |
| 15:27:12 | jaypipes | stephenfin: and no need for the trailing _ char when using :ref? | |
| 15:27:35 | melwitt | dansmith, mriedem: are we cool to approve this follow up specless blueprint for https://blueprints.launchpad.net/nova/+spec/memory-backend-file-discard that depends on the libvirt-file-backed-memory blueprint approved today? | |
| 15:28:06 | stephenfin | Nope. :ref:`x` is a Sphinx thing that lets you cross-reference between docs etc., if you need to. `x`_ is plain old rST (docutils) | |
| 15:28:29 | jaypipes | stephenfin: ack. and the < > brackets are required, yes? | |
| 15:28:30 | dansmith | melwitt: um, not sure why we need that extra blueprint, tbh | |
| 15:28:39 | dansmith | we're just waiting for libvirt 4.4.0 and then could tag the tweak to nova against the original blueprint, IMHO | |
| 15:29:02 | stephenfin | jaypipes: If you wish to use a different display text, yes | |
| 15:29:22 | mriedem | melwitt: had never seen it, but seems it could be part of the original bp | |
| 15:29:44 | dansmith | mriedem: melwitt yeah, it will just be a small tweak to the xml we generate if libvirt is new enough | |
| 15:29:46 | melwitt | okay, I saw the link in the review comments on the spec for the original | |
| 15:30:22 | mriedem | we could amend the spec to add it as a separate work item | |
| 15:30:49 | jaypipes | stephenfin: hmm, getting a doc error... | |
| 15:30:52 | mriedem | i've been meaning to go over the spec anyway | |
| 15:31:34 | dansmith | mriedem: it was commented on the spec, just left out because it's not critical and isn't in released libvirt yet | |
| 15:31:36 | melwitt | discussed here https://review.openstack.org/#/c/563704/8/specs/rocky/approved/libvirt-file-backed-memory.rst@170 | |
| 15:31:58 | jaypipes | stephenfin: any ideas? http://paste.openstack.org/show/722356/ | |
| 15:32:04 | efried | bhagyashris: Second approach? | |
| 15:32:10 | dansmith | a tiny spec revision when it becomes available would be plenty, and/or just an extra patch tagged against the bp | |
| 15:32:42 | mriedem | when is libvirt 4.4.0 going to get released? | |
| 15:32:50 | mriedem | within the next month or so? | |
| 15:33:25 | stephenfin | jaypipes: Oops. I sent you the wrong way. That's for headers :( | |