| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-30 | |||
| 15:13:12 | jangutter | sahid: actually, the idea is to migrate the 'vrouter' vif which is not vhostuser... | |
| 15:13:14 | openstackgerrit | Merged openstack/nova-specs master: Libvirt file backed memory https://review.openstack.org/563704 | |
| 15:13:37 | stephenfin | jaypipes: I've missed something so. I thought requesting 'resources1:PCPU=8 resources2:PCPU=8 group_policy=isolate' means host NUMA topology would need to be accounted for | |
| 15:13:43 | jangutter | sahid: I did some checking, and it looks like multiqueue is an instance-wide setting, not a vif-specific one. | |
| 15:14:30 | jaypipes | stephenfin: no. it has nothing explicitly to do with NUMA. that just means "I need 2 groups of 8 dedicated CPU resources. they need to be provided by two distinct resource providers" | |
| 15:14:36 | jangutter | sahid: ref: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L708 | |
| 15:15:21 | jaypipes | stephenfin: the fact that the compute node resource provider might be a parent to two other child providers that *represent* different sockets or NUMA cells is really not relevant to the request that two different providers of dedicated CPU supply the requested resource groups. | |
| 15:16:07 | sahid | jangutter: i'm not sure i understand what you want do? | |
| 15:16:20 | jaypipes | stephenfin: in other words, if some non-NUMA devices *happened* to expose PCPU inventory under a provider tree, they would meet the requirements... | |
| 15:16:25 | bauzas | jaypipes: stephenfin: just a point, we could have the same upgrade impact with VCPU resources being in child RPs than the one I described with http://lists.openstack.org/pipermail/openstack-dev/2018-May/130783.html | |
| 15:16:32 | stephenfin | jaypipes: Practically speaking so, requesting 'resources1:PCPU=8 resources2:PCPU=8 group_policy=isolate' will always fail until NUMA is modelled in placement? | |
| 15:16:40 | stephenfin | Ah, gotcha | |
| 15:16:41 | jaypipes | stephenfin: again, we are trying desperately to free ourselves of the NUMA coupling that has plagued all of this for years. | |
| 15:17:13 | jangutter | sahid: vrouter kernel mode plugging needs to create a tap with multiqueue enabled, if libvirt starts the VM up looking for it. | |
| 15:17:30 | stephenfin | jaypipes: But we're still coupled, no? I mean, we're going to need NUMA for the above request to work | |
| 15:17:42 | bhagyashris | efried: any suggestion about second approach | |
| 15:17:43 | jaypipes | stephenfin: I'm trying my best to treat NUMA as just the model of distances to various memory (and memory page) caches to a processor/bus, instead of coupling all that mess with CPU pinning. | |
| 15:18:07 | jangutter | sahid: the VIFGeneric portion can set the libvirt config correctly for multiqueue, but, it looks like it's going to be difficult to pass "multiqueue" on a per-vif basis without modding large portions of the os-vif plugging code. | |
| 15:18:09 | sahid | jangutter: ok so you plugin needs to know whether multiqueue is enable, which is via the os-vif object nova is filling then passing the the os-vif plugin | |
| 15:18:21 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add osprofiler config options to generated reference https://review.openstack.org/571225 | |
| 15:18:51 | jangutter | sahid: yep, so, either we can pass "multiqueue=True" via the vif os-vif object, or via the instance os-vif object. | |
| 15:19:59 | bauzas | jaypipes: unfortunately from what I understood from operators, pinning is there because NUMA affinity misses for PCI devices | |
| 15:20:09 | bauzas | jaypipes: so both are somehow related | |
| 15:20:16 | stephenfin | jaypipes: To be clear though, use case 4 in that spec cannot be achieved with modelling NUMA in placement, right? | |
| 15:20:19 | jangutter | sahid: the trick is, passing it via the VIF looks very tricky. A lot of code has to be added there, since, paradoxically, the choice whether to enable multiqueue depends on the instance and flavor settings, not VIF settings. | |
| 15:20:37 | stephenfin | jaypipes: Use case 4 = https://review.openstack.org/#/c/555081/10/specs/rocky/approved/cpu-resources.rst@425 | |
| 15:20:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add osprofiler config options to generated reference https://review.openstack.org/571225 | |
| 15:21:03 | bauzas | jaypipes: in terms of priority, I'd consider NUMA placement and NUMA affinity be prioritary to CPU pinning, because folks care more about proximity and throughput than real placement decisions | |
| 15:21:13 | openstackgerrit | Merged openstack/nova master: doc: Start using openstackdoctheme's extlink extension https://review.openstack.org/540888 | |
| 15:21:14 | sahid | jangutter: ok i see, I could see it live in instance actually | |
| 15:21:19 | bauzas | CPU pinning is just a hack | |
| 15:21:33 | stephenfin | Hurrah! The extlink patch merged at last | |
| 15:21:44 | jangutter | sahid: for example, you can spawn an instance asking for multiqueue, but only 1 cpu, and multiqueue will magically disable :-p | |
| 15:21:50 | jaypipes | bauzas: I'm referring to the poor software design of coupling the topology objects together with memory pages, usage of instance virtual NUMA topology on a host NUMA topology, and all that. | |
| 15:22:00 | bauzas | jaypipes: sure, I understood that point | |
| 15:22:03 | stephenfin | bauzas: I think jaypipes is suggesting we can do CPU pinning without _needing_ to think about NUMA affinity. They don't have to be tied | |
| 15:22:29 | bauzas | stephenfin: my own take is that CPU pinning isn't required if we go with NUMA affinity | |
| 15:22:30 | sahid | jangutter: yes i that is not what you want? | |
| 15:22:38 | stephenfin | But anyone that is using this will want to couple them, given the performance implications of not doing so | |
| 15:22:54 | stephenfin | bauzas: It isn't currently, no | |
| 15:22:57 | jangutter | sahid: That's perfectly acceptable, as long as both sides agree. | |
| 15:23:06 | bauzas | stephenfin: I don't disagree with your point :) | |
| 15:23:17 | jaypipes | stephenfin: use case 4 can be implemented, sure, but until sockets are modeled as child providers or a compute node parent provider, the scheduler would always return NoValidHosts for a request like use case 4. that clear? | |
| 15:23:26 | bauzas | stephenfin: I mean, CPU pinning is tied to NUMA resources because people care of throughput | |
| 15:23:45 | stephenfin | jaypipes: Sure, if s/sockets/NUMA nodes/ | |
| 15:23:48 | jangutter | sahid: another reason for keeping it as a parameter of the instance... | |
| 15:23:52 | stephenfin | because of Cluster-on-Die etc. | |
| 15:23:58 | jaypipes | stephenfin: sigh... | |
| 15:24:02 | stephenfin | Sorry :) | |
| 15:24:40 | jangutter | sahid: thanks, I'll respin. The nova bit is much easier to do with it being part of the instance. | |
| 15:24:56 | jaypipes | stephenfin: quick RST doc question for you.. | |
| 15:25:07 | stephenfin | Oooh, yay | |
| 15:25:12 | stephenfin | jaypipes: shoot | |
| 15:25:21 | jaypipes | stephenfin: how can I make the text of an anchor different between two calling points to that anchor? | |
| 15:25:43 | stephenfin | :ref:`text you want to display <text-of-anchor>` | |
| 15:25:45 | jaypipes | stephenfin: you rightfully pointed out that the granular request spec is linked in two places. | |
| 15:25:54 | jaypipes | stephenfin: ah.. | |
| 15:25:55 | sahid | ok i don't really see why it's reflecting better the state of the instance using multiqueue or not, but i'm do not disagree with you to have that attribute with instance | |
| 15:25:59 | jaypipes | stephenfin: ty sir | |
| 15:26:29 | sahid | jangutter: please ping me when you send a new version | |
| 15:27:12 | jaypipes | stephenfin: and no need for the trailing _ char when using :ref? | |
| 15:27:35 | melwitt | dansmith, mriedem: are we cool to approve this follow up specless blueprint for https://blueprints.launchpad.net/nova/+spec/memory-backend-file-discard that depends on the libvirt-file-backed-memory blueprint approved today? | |
| 15:28:06 | stephenfin | Nope. :ref:`x` is a Sphinx thing that lets you cross-reference between docs etc., if you need to. `x`_ is plain old rST (docutils) | |
| 15:28:29 | jaypipes | stephenfin: ack. and the < > brackets are required, yes? | |
| 15:28:30 | dansmith | melwitt: um, not sure why we need that extra blueprint, tbh | |
| 15:28:39 | dansmith | we're just waiting for libvirt 4.4.0 and then could tag the tweak to nova against the original blueprint, IMHO | |
| 15:29:02 | stephenfin | jaypipes: If you wish to use a different display text, yes | |
| 15:29:22 | mriedem | melwitt: had never seen it, but seems it could be part of the original bp | |
| 15:29:44 | dansmith | mriedem: melwitt yeah, it will just be a small tweak to the xml we generate if libvirt is new enough | |
| 15:29:46 | melwitt | okay, I saw the link in the review comments on the spec for the original | |
| 15:30:22 | mriedem | we could amend the spec to add it as a separate work item | |
| 15:30:49 | jaypipes | stephenfin: hmm, getting a doc error... | |
| 15:30:52 | mriedem | i've been meaning to go over the spec anyway | |
| 15:31:34 | dansmith | mriedem: it was commented on the spec, just left out because it's not critical and isn't in released libvirt yet | |
| 15:31:36 | melwitt | discussed here https://review.openstack.org/#/c/563704/8/specs/rocky/approved/libvirt-file-backed-memory.rst@170 | |
| 15:31:58 | jaypipes | stephenfin: any ideas? http://paste.openstack.org/show/722356/ | |
| 15:32:04 | efried | bhagyashris: Second approach? | |
| 15:32:10 | dansmith | a tiny spec revision when it becomes available would be plenty, and/or just an extra patch tagged against the bp | |
| 15:32:42 | mriedem | when is libvirt 4.4.0 going to get released? | |
| 15:32:50 | mriedem | within the next month or so? | |
| 15:33:25 | stephenfin | jaypipes: Oops. I sent you the wrong way. That's for headers :( | |
| 15:33:32 | melwitt | I was thinking maybe we should just approve the follow on bp since it's already been created? | |
| 15:33:33 | stephenfin | jaypipes: Try `foo <bar>`_ | |
| 15:34:20 | mriedem | it's just as easy to obsolete the blueprint | |
| 15:34:42 | mriedem | if it doesn't happen in rocky, then yeah i'd make it a separate blueprint in stein | |
| 15:36:11 | dansmith | mriedem: I dunno, 4.4.0 is out, so the version might not be right | |
| 15:36:19 | stephenfin | jaypipes: Hmm, that doesn't work either. Maybe ignore that one for now. I was sure :ref: would do it | |
| 15:36:22 | dansmith | mriedem: but they do monthly releases I think, so should be soon if not now | |
| 15:37:04 | mriedem | just say that kubevirt really needs this like yesterday | |
| 15:37:27 | stephenfin | jaypipes: Got it. `foo <bar_>`_ (the underscore after 'bar' is the important bit) | |
| 15:37:51 | jaypipes | stephenfin: bingo, that got it. thx for your help! :) | |
| 15:39:46 | melwitt | this page says libvirt v4.4.0 is unreleased https://libvirt.org/news.html | |
| 15:40:30 | dansmith | I found a package for it, hang on | |
| 15:40:31 | dansmith | https://launchpad.net/ubuntu/+source/linux/4.4.0-121.145 | |
| 15:40:36 | dansmith | maybe that's built from master or something | |
| 15:40:43 | dansmith | oh danh | |
| 15:40:54 | dansmith | that's the kernel, google turned it up instead | |
| 15:40:58 | dansmith | so yeah nevermind :D | |
| 15:41:05 | melwitt | ah okay. last version was released v4.3.0 (2018-05-02) so if monthly then it will be any day now | |
| 15:41:24 | dansmith | https://imgur.com/a/effQ3Gn | |