| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-11 | |||
| 12:56:43 | mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22DestinationDiskExists%3A%20The%20supplied%20disk%20path%20(path)%20already%20exists%2C%20it%20is%20expected%20not%20to%20exist.%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d | |
| 13:02:40 | stephenfin | sean-k-mooney: What do you think of https://review.opendev.org/#/c/678455/28/nova/virt/libvirt/driver.py@5468 ? | |
| 13:04:08 | sean-k-mooney | i dont like it but its what we said we would do a the PTG | |
| 13:04:27 | sean-k-mooney | i would prefer to actully give the guest a numa toplogy like we do with hugepages | |
| 13:04:38 | sean-k-mooney | but just not requrie the pmem namespace to be numa affined | |
| 13:04:45 | stephenfin | Me too | |
| 13:04:56 | sean-k-mooney | but i know some people want to supprot non numa guest with pmem | |
| 13:05:12 | stephenfin | But the spec says this "Persistent memory is by nature NUMA sensitive." | |
| 13:05:18 | sean-k-mooney | yes it is | |
| 13:05:23 | stephenfin | and also "Libvirt domain specification requires each virtual persistent memory to be associated with one guest NUMA node." | |
| 13:05:48 | sean-k-mooney | its litrally memory dimms that happen to not loose ther content on host reboot | |
| 13:06:23 | sean-k-mooney | yes so we are working around a qemu limitation by giving the guest a virtual numa toplogy of 1 numa node | |
| 13:06:38 | stephenfin | hmm, yeah, this should definitely have NUMA affinity enforced | |
| 13:06:39 | sean-k-mooney | then not affinig it to a host numa node to make it act like a floating vm | |
| 13:06:59 | stephenfin | via the NUMATopologyFilter, of course, until we have NUMA in placement | |
| 13:07:01 | sean-k-mooney | thats what i argued but again we chose not too | |
| 13:07:43 | sean-k-mooney | without reopentin the spec i think we have to just live with this in train | |
| 13:08:04 | sean-k-mooney | and kill it with fire in U | |
| 13:08:15 | stephenfin | idk. I'd like to see how hard it would be to do the NUMA affinity thing | |
| 13:08:17 | stephenfin | It seems easy | |
| 13:08:28 | stephenfin | just another conditional in numa_get_constraints | |
| 13:08:34 | sean-k-mooney | it would be | |
| 13:08:39 | sean-k-mooney | yep | |
| 13:09:05 | sean-k-mooney | the proable with that is it mean that you cant have geust with more cores then fit on one host numa node | |
| 13:09:11 | sean-k-mooney | without have a multi numa guest | |
| 13:09:21 | sean-k-mooney | and that is want people dont want to have to care about | |
| 13:09:34 | stephenfin | that's no different to huge pages and PCI devices | |
| 13:09:48 | sean-k-mooney | pci device does not creat a numa toplogy | |
| 13:09:55 | sean-k-mooney | hugepage does | |
| 13:09:56 | stephenfin | sorry, not PCI devices | |
| 13:09:59 | stephenfin | yeah | |
| 13:10:05 | sean-k-mooney | hugepages and cpu pinning | |
| 13:10:25 | sean-k-mooney | i would prefer not to special case pmem | |
| 13:10:49 | sean-k-mooney | if we were to special case pmem i would prefer to relax hugepages and cpu pinning to be the same | |
| 13:10:52 | stephenfin | So would I. We have way too many special corner cases as-is. We should be reducing that, not increasing it | |
| 13:10:58 | stephenfin | Yeah, that | |
| 13:11:08 | sean-k-mooney | e.g i have always wanted to decouple pinning/hugepages form numa | |
| 13:12:17 | stephenfin | oh, I'm well aware :) | |
| 13:13:01 | sean-k-mooney | so ya that is where i stand. im not going to block it but i consider it tech debth | |
| 13:14:09 | sean-k-mooney | stephenfin: at least im consitent on the topic :) | |
| 13:15:38 | sean-k-mooney | stephenfin: alex_xu luyao https://libvirt.org/formatdomain.html#elementsMemory | |
| 13:15:45 | sean-k-mooney | The node subelement configures the guest NUMA node to attach the memory to. The element shall be used only if the guest has NUMA nodes configured. | |
| 13:16:02 | sean-k-mooney | so it looks like not is optional in the target field | |
| 13:16:14 | sean-k-mooney | so lest just not set it | |
| 13:16:38 | sean-k-mooney | and get rid fo the virtual numa node. | |
| 13:16:58 | stephenfin | I see this above in the opening paragraph though: "Some hypervisors may require NUMA configured for the guest." | |
| 13:17:12 | stephenfin | If kvm/qemu one such hypervisor? | |
| 13:17:16 | sean-k-mooney | if the guest has a numa toplogy we could set it to 0 but i think that would be the default | |
| 13:17:51 | sean-k-mooney | no | |
| 13:17:56 | sean-k-mooney | https://github.com/qemu/qemu/blob/master/docs/nvdimm.txt | |
| 13:18:05 | sean-k-mooney | -machine pc,nvdimm | |
| 13:18:07 | sean-k-mooney | -m $RAM_SIZE,slots=$N,maxmem=$MAX_SIZE | |
| 13:18:09 | sean-k-mooney | -object memory-backend-file,id=mem1,share=on,mem-path=$PATH,size=$NVDIMM_SIZE | |
| 13:18:11 | sean-k-mooney | -device nvdimm,id=nvdimm1,memdev=mem1 | |
| 13:18:22 | sean-k-mooney | basic usage create an nvdim device with no numa node | |
| 13:18:39 | sean-k-mooney | efried_afk: alex_xu luyao ^ | |
| 13:19:11 | stephenfin | okay, if that's true I can definitely live with it | |
| 13:19:23 | stephenfin | remind me to go and convert the QEMU docs to rST sometime | |
| 13:19:30 | stephenfin | or Markdown or something | |
| 13:19:35 | luyao | sean-k-mooney: the vm can't run if no numa node | |
| 13:19:40 | sean-k-mooney | plain text is fine | |
| 13:19:48 | sean-k-mooney | luyao: have you tested it | |
| 13:19:54 | luyao | sean-k-mooney: yes | |
| 13:20:03 | sean-k-mooney | well qemu said it can | |
| 13:20:14 | sean-k-mooney | what error did you get | |
| 13:20:18 | stephenfin | But rendered HTML is better (for me, anyway) ;-) | |
| 13:20:36 | sean-k-mooney | stephenfin: and much worse for me :) | |
| 13:20:43 | stephenfin | Is dansmith around yet? | |
| 13:22:06 | luyao | sean-k-mooney: I don't remember it clearly, something like you must specify an numa node | |
| 13:22:25 | sean-k-mooney | can you test it again | |
| 13:22:31 | luyao | sean-k-mooney: actually, the libvirt part you mentioned is my updating before | |
| 13:22:39 | sean-k-mooney | ill try a nested env to test it | |
| 13:22:59 | sean-k-mooney | which libvirt part | |
| 13:23:11 | luyao | sean-k-mooney: About the nvdimm config | |
| 13:23:55 | sean-k-mooney | right the libvirt doc say the node is optional in the target | |
| 13:24:11 | sean-k-mooney | so libvirt hsould not require a numa toplogy | |
| 13:24:33 | sean-k-mooney | and the qemu examle does not create a numa toplogy | |
| 13:24:50 | openstackgerrit | Chris Dent proposed openstack/os-traits master: Build pdf docs https://review.opendev.org/681465 | |
| 13:25:46 | openstackgerrit | Chris Dent proposed openstack/os-resource-classes master: Build pdf docs https://review.opendev.org/681463 | |
| 13:26:06 | luyao | sean-k-mooney: that node description is for all memory devices, for nvdimm , we do need a numa | |
| 13:26:19 | luyao | sean-k-mooney: I'll try to do a test | |
| 13:27:06 | sean-k-mooney | i think we may only need the node to be set if there is a numa node in the guest | |
| 13:27:15 | sean-k-mooney | if its not present i think we wont need it | |
| 13:27:19 | openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver discovering PMEM namespaces https://review.opendev.org/678453 | |
| 13:27:19 | openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: report VPMEM resources by provider tree https://review.opendev.org/678454 | |
| 13:27:20 | openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Support VM creation with vpmems and vpmems cleanup https://review.opendev.org/678455 | |
| 13:27:20 | openstackgerrit | Luyao Zhong proposed openstack/nova master: Parse vpmem related flavor extra spec https://review.opendev.org/678456 | |
| 13:27:21 | openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | |
| 13:27:21 | openstackgerrit | Luyao Zhong proposed openstack/nova master: Add functional tests for virtual persistent memory https://review.opendev.org/678470 | |
| 13:27:22 | openstackgerrit | Luyao Zhong proposed openstack/nova master: objects: use all_things_equal from objects.base https://review.opendev.org/681397 | |
| 13:27:43 | sean-k-mooney | so we could hard code it ot 0 if there is a numa toplogy and not set it if there is not | |
| 13:30:01 | mriedem | why is it that Intel_Zuul comments can't be hidden with the 3rd party toggle CI button? | |
| 13:30:14 | sean-k-mooney | it does not have CI in the name | |
| 13:30:22 | mriedem | blarg | |
| 13:30:23 | sean-k-mooney | which it should have | |
| 13:30:26 | mriedem | alex_xu: ^ can we get that fixed? | |
| 13:32:37 | openstackgerrit | sean mooney proposed openstack/nova master: [DNM] pmem pull request test. https://review.opendev.org/681474 | |
| 13:33:07 | sean-k-mooney | efried: alex_xu luyao in theory ^ should cause the intel ci to run with my pull request that removes the hardcoding | |
| 13:33:52 | sean-k-mooney | it also might make the first party ci unhappy but i have turned off all the check test anyway | |
| 13:34:46 | mriedem | artom: you still have a busted test in https://review.opendev.org/#/c/634606/84 but if you want to tack on a follow up patch to the end of the series i wouldn't be opposed to just fixing it that way | |