| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-30 | |||
| 15:16:41 | jaypipes | stephenfin: again, we are trying desperately to free ourselves of the NUMA coupling that has plagued all of this for years. | |
| 15:17:13 | jangutter | sahid: vrouter kernel mode plugging needs to create a tap with multiqueue enabled, if libvirt starts the VM up looking for it. | |
| 15:17:30 | stephenfin | jaypipes: But we're still coupled, no? I mean, we're going to need NUMA for the above request to work | |
| 15:17:42 | bhagyashris | efried: any suggestion about second approach | |
| 15:17:43 | jaypipes | stephenfin: I'm trying my best to treat NUMA as just the model of distances to various memory (and memory page) caches to a processor/bus, instead of coupling all that mess with CPU pinning. | |
| 15:18:07 | jangutter | sahid: the VIFGeneric portion can set the libvirt config correctly for multiqueue, but, it looks like it's going to be difficult to pass "multiqueue" on a per-vif basis without modding large portions of the os-vif plugging code. | |
| 15:18:09 | sahid | jangutter: ok so you plugin needs to know whether multiqueue is enable, which is via the os-vif object nova is filling then passing the the os-vif plugin | |
| 15:18:21 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add osprofiler config options to generated reference https://review.openstack.org/571225 | |
| 15:18:51 | jangutter | sahid: yep, so, either we can pass "multiqueue=True" via the vif os-vif object, or via the instance os-vif object. | |
| 15:19:59 | bauzas | jaypipes: unfortunately from what I understood from operators, pinning is there because NUMA affinity misses for PCI devices | |
| 15:20:09 | bauzas | jaypipes: so both are somehow related | |
| 15:20:16 | stephenfin | jaypipes: To be clear though, use case 4 in that spec cannot be achieved with modelling NUMA in placement, right? | |
| 15:20:19 | jangutter | sahid: the trick is, passing it via the VIF looks very tricky. A lot of code has to be added there, since, paradoxically, the choice whether to enable multiqueue depends on the instance and flavor settings, not VIF settings. | |
| 15:20:37 | stephenfin | jaypipes: Use case 4 = https://review.openstack.org/#/c/555081/10/specs/rocky/approved/cpu-resources.rst@425 | |
| 15:20:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add osprofiler config options to generated reference https://review.openstack.org/571225 | |
| 15:21:03 | bauzas | jaypipes: in terms of priority, I'd consider NUMA placement and NUMA affinity be prioritary to CPU pinning, because folks care more about proximity and throughput than real placement decisions | |
| 15:21:13 | openstackgerrit | Merged openstack/nova master: doc: Start using openstackdoctheme's extlink extension https://review.openstack.org/540888 | |
| 15:21:14 | sahid | jangutter: ok i see, I could see it live in instance actually | |
| 15:21:19 | bauzas | CPU pinning is just a hack | |
| 15:21:33 | stephenfin | Hurrah! The extlink patch merged at last | |
| 15:21:44 | jangutter | sahid: for example, you can spawn an instance asking for multiqueue, but only 1 cpu, and multiqueue will magically disable :-p | |
| 15:21:50 | jaypipes | bauzas: I'm referring to the poor software design of coupling the topology objects together with memory pages, usage of instance virtual NUMA topology on a host NUMA topology, and all that. | |
| 15:22:00 | bauzas | jaypipes: sure, I understood that point | |
| 15:22:03 | stephenfin | bauzas: I think jaypipes is suggesting we can do CPU pinning without _needing_ to think about NUMA affinity. They don't have to be tied | |
| 15:22:29 | bauzas | stephenfin: my own take is that CPU pinning isn't required if we go with NUMA affinity | |
| 15:22:30 | sahid | jangutter: yes i that is not what you want? | |
| 15:22:38 | stephenfin | But anyone that is using this will want to couple them, given the performance implications of not doing so | |
| 15:22:54 | stephenfin | bauzas: It isn't currently, no | |
| 15:22:57 | jangutter | sahid: That's perfectly acceptable, as long as both sides agree. | |
| 15:23:06 | bauzas | stephenfin: I don't disagree with your point :) | |
| 15:23:17 | jaypipes | stephenfin: use case 4 can be implemented, sure, but until sockets are modeled as child providers or a compute node parent provider, the scheduler would always return NoValidHosts for a request like use case 4. that clear? | |
| 15:23:26 | bauzas | stephenfin: I mean, CPU pinning is tied to NUMA resources because people care of throughput | |
| 15:23:45 | stephenfin | jaypipes: Sure, if s/sockets/NUMA nodes/ | |
| 15:23:48 | jangutter | sahid: another reason for keeping it as a parameter of the instance... | |
| 15:23:52 | stephenfin | because of Cluster-on-Die etc. | |
| 15:23:58 | jaypipes | stephenfin: sigh... | |
| 15:24:02 | stephenfin | Sorry :) | |
| 15:24:40 | jangutter | sahid: thanks, I'll respin. The nova bit is much easier to do with it being part of the instance. | |
| 15:24:56 | jaypipes | stephenfin: quick RST doc question for you.. | |
| 15:25:07 | stephenfin | Oooh, yay | |
| 15:25:12 | stephenfin | jaypipes: shoot | |
| 15:25:21 | jaypipes | stephenfin: how can I make the text of an anchor different between two calling points to that anchor? | |
| 15:25:43 | stephenfin | :ref:`text you want to display <text-of-anchor>` | |
| 15:25:45 | jaypipes | stephenfin: you rightfully pointed out that the granular request spec is linked in two places. | |
| 15:25:54 | jaypipes | stephenfin: ah.. | |
| 15:25:55 | sahid | ok i don't really see why it's reflecting better the state of the instance using multiqueue or not, but i'm do not disagree with you to have that attribute with instance | |
| 15:25:59 | jaypipes | stephenfin: ty sir | |
| 15:26:29 | sahid | jangutter: please ping me when you send a new version | |
| 15:27:12 | jaypipes | stephenfin: and no need for the trailing _ char when using :ref? | |
| 15:27:35 | melwitt | dansmith, mriedem: are we cool to approve this follow up specless blueprint for https://blueprints.launchpad.net/nova/+spec/memory-backend-file-discard that depends on the libvirt-file-backed-memory blueprint approved today? | |
| 15:28:06 | stephenfin | Nope. :ref:`x` is a Sphinx thing that lets you cross-reference between docs etc., if you need to. `x`_ is plain old rST (docutils) | |
| 15:28:29 | jaypipes | stephenfin: ack. and the < > brackets are required, yes? | |
| 15:28:30 | dansmith | melwitt: um, not sure why we need that extra blueprint, tbh | |
| 15:28:39 | dansmith | we're just waiting for libvirt 4.4.0 and then could tag the tweak to nova against the original blueprint, IMHO | |
| 15:29:02 | stephenfin | jaypipes: If you wish to use a different display text, yes | |
| 15:29:22 | mriedem | melwitt: had never seen it, but seems it could be part of the original bp | |
| 15:29:44 | dansmith | mriedem: melwitt yeah, it will just be a small tweak to the xml we generate if libvirt is new enough | |
| 15:29:46 | melwitt | okay, I saw the link in the review comments on the spec for the original | |
| 15:30:22 | mriedem | we could amend the spec to add it as a separate work item | |
| 15:30:49 | jaypipes | stephenfin: hmm, getting a doc error... | |
| 15:30:52 | mriedem | i've been meaning to go over the spec anyway | |
| 15:31:34 | dansmith | mriedem: it was commented on the spec, just left out because it's not critical and isn't in released libvirt yet | |
| 15:31:36 | melwitt | discussed here https://review.openstack.org/#/c/563704/8/specs/rocky/approved/libvirt-file-backed-memory.rst@170 | |
| 15:31:58 | jaypipes | stephenfin: any ideas? http://paste.openstack.org/show/722356/ | |
| 15:32:04 | efried | bhagyashris: Second approach? | |
| 15:32:10 | dansmith | a tiny spec revision when it becomes available would be plenty, and/or just an extra patch tagged against the bp | |
| 15:32:42 | mriedem | when is libvirt 4.4.0 going to get released? | |
| 15:32:50 | mriedem | within the next month or so? | |
| 15:33:25 | stephenfin | jaypipes: Oops. I sent you the wrong way. That's for headers :( | |
| 15:33:32 | melwitt | I was thinking maybe we should just approve the follow on bp since it's already been created? | |
| 15:33:33 | stephenfin | jaypipes: Try `foo <bar>`_ | |
| 15:34:20 | mriedem | it's just as easy to obsolete the blueprint | |
| 15:34:42 | mriedem | if it doesn't happen in rocky, then yeah i'd make it a separate blueprint in stein | |
| 15:36:11 | dansmith | mriedem: I dunno, 4.4.0 is out, so the version might not be right | |
| 15:36:19 | stephenfin | jaypipes: Hmm, that doesn't work either. Maybe ignore that one for now. I was sure :ref: would do it | |
| 15:36:22 | dansmith | mriedem: but they do monthly releases I think, so should be soon if not now | |
| 15:37:04 | mriedem | just say that kubevirt really needs this like yesterday | |
| 15:37:27 | stephenfin | jaypipes: Got it. `foo <bar_>`_ (the underscore after 'bar' is the important bit) | |
| 15:37:51 | jaypipes | stephenfin: bingo, that got it. thx for your help! :) | |
| 15:39:46 | melwitt | this page says libvirt v4.4.0 is unreleased https://libvirt.org/news.html | |
| 15:40:30 | dansmith | I found a package for it, hang on | |
| 15:40:31 | dansmith | https://launchpad.net/ubuntu/+source/linux/4.4.0-121.145 | |
| 15:40:36 | dansmith | maybe that's built from master or something | |
| 15:40:43 | dansmith | oh danh | |
| 15:40:54 | dansmith | that's the kernel, google turned it up instead | |
| 15:40:58 | dansmith | so yeah nevermind :D | |
| 15:41:05 | melwitt | ah okay. last version was released v4.3.0 (2018-05-02) so if monthly then it will be any day now | |
| 15:41:24 | dansmith | https://imgur.com/a/effQ3Gn | |
| 15:41:27 | dansmith | thanks a lot google | |
| 15:41:39 | melwitt | you got googled | |
| 15:41:47 | dansmith | apparently | |
| 15:42:38 | melwitt | okay, I'll mark the bp as Obsolete with a note that libvirt 4.4.0 is soon and that the addition of the discard feature can be rolled into the original bp | |
| 15:42:48 | dansmith | cool | |
| 15:43:22 | openstackgerrit | Chris Friesen proposed openstack/nova master: add parameter docstring for 'params' to libvirt.guest.Guest.migrate() https://review.openstack.org/571236 | |
| 15:43:37 | cfriesen | kashyap: ^ | |
| 15:43:59 | zcorneli | Works for me. I'll look into what it'll take to add that to the code (particularly the migration segments) | |
| 15:44:08 | kashyap | cfriesen: Thanks; will review tomm; heading to my Dutch class :-) | |
| 15:45:18 | cfriesen | kashyap: no rush | |
| 15:50:42 | melwitt | zcorneli: cool, thanks. to be clear, it's also fine for sahid to work on it if you want, authors can work together on one blueprint, just link the blueprint in the commit message of the new patches when libvirt 4.4.0 is out | |
| 15:51:10 | melwitt | we're just organizing the patches on one blueprint as one feature altogether | |