| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-11 | |||
| 13:09:05 | sean-k-mooney | the proable with that is it mean that you cant have geust with more cores then fit on one host numa node | |
| 13:09:11 | sean-k-mooney | without have a multi numa guest | |
| 13:09:21 | sean-k-mooney | and that is want people dont want to have to care about | |
| 13:09:34 | stephenfin | that's no different to huge pages and PCI devices | |
| 13:09:48 | sean-k-mooney | pci device does not creat a numa toplogy | |
| 13:09:55 | sean-k-mooney | hugepage does | |
| 13:09:56 | stephenfin | sorry, not PCI devices | |
| 13:09:59 | stephenfin | yeah | |
| 13:10:05 | sean-k-mooney | hugepages and cpu pinning | |
| 13:10:25 | sean-k-mooney | i would prefer not to special case pmem | |
| 13:10:49 | sean-k-mooney | if we were to special case pmem i would prefer to relax hugepages and cpu pinning to be the same | |
| 13:10:52 | stephenfin | So would I. We have way too many special corner cases as-is. We should be reducing that, not increasing it | |
| 13:10:58 | stephenfin | Yeah, that | |
| 13:11:08 | sean-k-mooney | e.g i have always wanted to decouple pinning/hugepages form numa | |
| 13:12:17 | stephenfin | oh, I'm well aware :) | |
| 13:13:01 | sean-k-mooney | so ya that is where i stand. im not going to block it but i consider it tech debth | |
| 13:14:09 | sean-k-mooney | stephenfin: at least im consitent on the topic :) | |
| 13:15:38 | sean-k-mooney | stephenfin: alex_xu luyao https://libvirt.org/formatdomain.html#elementsMemory | |
| 13:15:45 | sean-k-mooney | The node subelement configures the guest NUMA node to attach the memory to. The element shall be used only if the guest has NUMA nodes configured. | |
| 13:16:02 | sean-k-mooney | so it looks like not is optional in the target field | |
| 13:16:14 | sean-k-mooney | so lest just not set it | |
| 13:16:38 | sean-k-mooney | and get rid fo the virtual numa node. | |
| 13:16:58 | stephenfin | I see this above in the opening paragraph though: "Some hypervisors may require NUMA configured for the guest." | |
| 13:17:12 | stephenfin | If kvm/qemu one such hypervisor? | |
| 13:17:16 | sean-k-mooney | if the guest has a numa toplogy we could set it to 0 but i think that would be the default | |
| 13:17:51 | sean-k-mooney | no | |
| 13:17:56 | sean-k-mooney | https://github.com/qemu/qemu/blob/master/docs/nvdimm.txt | |
| 13:18:05 | sean-k-mooney | -machine pc,nvdimm | |
| 13:18:07 | sean-k-mooney | -m $RAM_SIZE,slots=$N,maxmem=$MAX_SIZE | |
| 13:18:09 | sean-k-mooney | -object memory-backend-file,id=mem1,share=on,mem-path=$PATH,size=$NVDIMM_SIZE | |
| 13:18:11 | sean-k-mooney | -device nvdimm,id=nvdimm1,memdev=mem1 | |
| 13:18:22 | sean-k-mooney | basic usage create an nvdim device with no numa node | |
| 13:18:39 | sean-k-mooney | efried_afk: alex_xu luyao ^ | |
| 13:19:11 | stephenfin | okay, if that's true I can definitely live with it | |
| 13:19:23 | stephenfin | remind me to go and convert the QEMU docs to rST sometime | |
| 13:19:30 | stephenfin | or Markdown or something | |
| 13:19:35 | luyao | sean-k-mooney: the vm can't run if no numa node | |
| 13:19:40 | sean-k-mooney | plain text is fine | |
| 13:19:48 | sean-k-mooney | luyao: have you tested it | |
| 13:19:54 | luyao | sean-k-mooney: yes | |
| 13:20:03 | sean-k-mooney | well qemu said it can | |
| 13:20:14 | sean-k-mooney | what error did you get | |
| 13:20:18 | stephenfin | But rendered HTML is better (for me, anyway) ;-) | |
| 13:20:36 | sean-k-mooney | stephenfin: and much worse for me :) | |
| 13:20:43 | stephenfin | Is dansmith around yet? | |
| 13:22:06 | luyao | sean-k-mooney: I don't remember it clearly, something like you must specify an numa node | |
| 13:22:25 | sean-k-mooney | can you test it again | |
| 13:22:31 | luyao | sean-k-mooney: actually, the libvirt part you mentioned is my updating before | |
| 13:22:39 | sean-k-mooney | ill try a nested env to test it | |
| 13:22:59 | sean-k-mooney | which libvirt part | |
| 13:23:11 | luyao | sean-k-mooney: About the nvdimm config | |
| 13:23:55 | sean-k-mooney | right the libvirt doc say the node is optional in the target | |
| 13:24:11 | sean-k-mooney | so libvirt hsould not require a numa toplogy | |
| 13:24:33 | sean-k-mooney | and the qemu examle does not create a numa toplogy | |
| 13:24:50 | openstackgerrit | Chris Dent proposed openstack/os-traits master: Build pdf docs https://review.opendev.org/681465 | |
| 13:25:46 | openstackgerrit | Chris Dent proposed openstack/os-resource-classes master: Build pdf docs https://review.opendev.org/681463 | |
| 13:26:06 | luyao | sean-k-mooney: that node description is for all memory devices, for nvdimm , we do need a numa | |
| 13:26:19 | luyao | sean-k-mooney: I'll try to do a test | |
| 13:27:06 | sean-k-mooney | i think we may only need the node to be set if there is a numa node in the guest | |
| 13:27:15 | sean-k-mooney | if its not present i think we wont need it | |
| 13:27:19 | openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: report VPMEM resources by provider tree https://review.opendev.org/678454 | |
| 13:27:19 | openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver discovering PMEM namespaces https://review.opendev.org/678453 | |
| 13:27:20 | openstackgerrit | Luyao Zhong proposed openstack/nova master: Parse vpmem related flavor extra spec https://review.opendev.org/678456 | |
| 13:27:20 | openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Support VM creation with vpmems and vpmems cleanup https://review.opendev.org/678455 | |
| 13:27:21 | openstackgerrit | Luyao Zhong proposed openstack/nova master: Add functional tests for virtual persistent memory https://review.opendev.org/678470 | |
| 13:27:21 | openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | |
| 13:27:22 | openstackgerrit | Luyao Zhong proposed openstack/nova master: objects: use all_things_equal from objects.base https://review.opendev.org/681397 | |
| 13:27:43 | sean-k-mooney | so we could hard code it ot 0 if there is a numa toplogy and not set it if there is not | |
| 13:30:01 | mriedem | why is it that Intel_Zuul comments can't be hidden with the 3rd party toggle CI button? | |
| 13:30:14 | sean-k-mooney | it does not have CI in the name | |
| 13:30:22 | mriedem | blarg | |
| 13:30:23 | sean-k-mooney | which it should have | |
| 13:30:26 | mriedem | alex_xu: ^ can we get that fixed? | |
| 13:32:37 | openstackgerrit | sean mooney proposed openstack/nova master: [DNM] pmem pull request test. https://review.opendev.org/681474 | |
| 13:33:07 | sean-k-mooney | efried: alex_xu luyao in theory ^ should cause the intel ci to run with my pull request that removes the hardcoding | |
| 13:33:52 | sean-k-mooney | it also might make the first party ci unhappy but i have turned off all the check test anyway | |
| 13:34:46 | mriedem | artom: you still have a busted test in https://review.opendev.org/#/c/634606/84 but if you want to tack on a follow up patch to the end of the series i wouldn't be opposed to just fixing it that way | |
| 13:35:06 | artom | mriedem, what's the risk in pushing a new PS? | |
| 13:35:23 | mriedem | is dan doing reviews on the other patches later up the stack? | |
| 13:35:38 | mriedem | just wait for me to hit https://review.opendev.org/#/c/640021/50 again quick | |
| 13:36:55 | artom | Ack | |
| 13:37:40 | openstackgerrit | sean mooney proposed openstack/nova master: [DNM] pmem pull request test. https://review.opendev.org/681474 | |
| 13:37:40 | efried | stephenfin, sean-k-mooney, luyao: here's spec discussion about numa stuff https://review.opendev.org/#/c/601596/16/specs/train/approved/virtual-persistent-memory.rst@270 | |
| 13:38:50 | efried | and the one below it | |
| 13:39:10 | sean-k-mooney | efried: yes they have implemented what the spec said but its still problemenatic | |
| 13:39:34 | efried | but you agreed it was okay to be consistent with the other problematic things that do the same? | |
| 13:39:44 | sean-k-mooney | it is not consitent4 | |
| 13:39:50 | stephenfin | yeah, it's not consistent | |
| 13:39:54 | stephenfin | that's the bit I don't like | |
| 13:39:59 | stephenfin | at all at all | |
| 13:40:00 | sean-k-mooney | the other thing actully pin the cores and memoyr to a host numa node | |
| 13:40:23 | efried | oh, so it's bad in a different way than the other things are bad | |
| 13:40:31 | efried | and we'd rather have it be consistently bad. | |
| 13:40:35 | sean-k-mooney | the other things imporve performance | |
| 13:40:51 | efried | but we're agreed it's too hard to have it be good (and inconsistent) until "later". | |
| 13:40:52 | sean-k-mooney | this does not and it make the other code more complex | |
| 13:41:26 | sean-k-mooney | efried: it hink we can get the same beahvior if we jsut dont set the node elemnt in the xml | |
| 13:41:34 | stephenfin | efried: I'm not sure what you mean | |
| 13:41:36 | sean-k-mooney | and dont create a numa toplogy at all | |
| 13:41:54 | stephenfin | we're saying it should either not have a NUMA topology, if that's possible (which sean-k-mooney thinks it is) | |