| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-10-09 | |||
| 12:57:19 | sean-k-mooney | i saw your latest commnt in email however | |
| 13:09:10 | lyarwood | elod: huh weird, I was sure this landed in stable/victoria | |
| 13:09:16 | lyarwood | elod: thanks for catching that | |
| 13:10:04 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/victoria: Use absolute path during qemu img rebase https://review.opendev.org/757098 | |
| 13:10:22 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/ussuri: Use absolute path during qemu img rebase https://review.opendev.org/757083 | |
| 13:10:34 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: Use absolute path during qemu img rebase https://review.opendev.org/757084 | |
| 13:10:49 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: Use absolute path during qemu img rebase https://review.opendev.org/757085 | |
| 13:16:28 | elod | lyarwood: np :) | |
| 13:28:16 | CeeMac | afternoon | |
| 13:28:52 | CeeMac | quick question, is there a way to set the libvirt cpu_mode using metadata or some such on a flavour/image/instance at all? | |
| 13:28:58 | sean-k-mooney | stephenfin: how would you feel about default all instance with a numa toplogy to hw:mem_page_size=small unless otherwise set. this is needed for numa in placment anyway but would you be open to doing this anyway even without numa in placment | |
| 13:29:49 | sean-k-mooney | we might want to restrict it to new instance only but we do have customer hit this edgecase a lot | |
| 13:30:44 | sean-k-mooney | the set hw:cpu_policy=dediecated and dont realise that unless they also set hw:mem_page_size=small then there vm will eventually get killed by the OOM reaper | |
| 13:31:11 | sean-k-mooney | since the vms memory is pined to a numa node but the did nto enable per numa memory tracking | |
| 13:33:07 | sean-k-mooney | stephenfin:by the way for your image properties patch should we send a mail to the list to let the glance folks know we are going to move documenation of image properties back into nova | |
| 13:43:07 | stephenfin | sean-k-mooney: I don't think an email is necessary. I'll just propose a patch to glance to remove the existing doc and point back to nova. If they want to keep their copy, that's fine | |
| 13:44:26 | stephenfin | sean-k-mooney: As for mem pages, I'm not sure. You can't overcommit with 'hw:mem_page_size' set, right? Even if you could, your overcommit would be against memory from that cell, rather than the entire host. Not necessarily a bad thing but both things are changes in behavior | |
| 13:47:01 | sean-k-mooney | right now it broken to not set it | |
| 13:47:29 | sean-k-mooney | we will use the global memory when determining if the vm wil fit then pin it to one numa node | |
| 13:47:49 | sean-k-mooney | so over commit is broken in any case | |
| 13:48:33 | sean-k-mooney | once we track numa in placment its not negoicable it has to do numa aware memroy tracking which is what setting that would do | |
| 13:48:50 | sean-k-mooney | we have had 2-3 customer hit this in the last year or so | |
| 13:51:47 | sean-k-mooney | for numa in placment this is adress by partioning the cloud into numa nost and non numa hosts. | |
| 13:52:32 | sean-k-mooney | we currently dont enforce that you se hw:mem_page_size if you have a numa toplogy but its always a bug/user errror to not set it | |
| 13:55:14 | stephenfin | if we can figure out a way to do overcommit on a per-node basis, then I'd be okay with that | |
| 13:55:21 | stephenfin | otherwise, it's a big change | |
| 13:56:30 | sean-k-mooney | well as is said overcomit does not work properly today | |
| 13:56:49 | sean-k-mooney | the OOM reaper will kill the vm if the numa node runs out of memory | |
| 13:56:56 | sean-k-mooney | even if the othe rnuma node is fully empty | |
| 13:57:21 | stephenfin | you could say the same thing about overcommit without NUMA | |
| 13:57:32 | sean-k-mooney | its not the same | |
| 13:57:45 | stephenfin | no, but it's similar | |
| 13:57:49 | sean-k-mooney | if you set over commit to 1.0 e.g. no over commit | |
| 13:58:04 | sean-k-mooney | with the current state the vms can be killed becuase the numa node is exausted | |
| 13:58:15 | sean-k-mooney | unless you set hw:mem_page_size=small | |
| 13:59:02 | stephenfin | yes, I understand that | |
| 13:59:03 | sean-k-mooney | its requried for numa in plamcent for numa hosts | |
| 13:59:18 | sean-k-mooney | for non numa hosts i was not going to be an issue since numa instance could not land there | |
| 13:59:33 | stephenfin | but if you had work and instances evenly distributed between NUMA nodes, then it would behave the same as the non-NUMA case | |
| 13:59:46 | sean-k-mooney | so we defered this conversation the last 2 cycles because numa in palcment will prevent it form happening | |
| 14:00:20 | sean-k-mooney | stephenfin: without my numa loadbancing feature it wont evenly disturbute | |
| 14:00:31 | sean-k-mooney | unless you consume all the cpus on the first node | |
| 14:00:39 | sean-k-mooney | even then it won evenly disturbute | |
| 14:02:25 | openstackgerrit | Dan Smith proposed openstack/nova master: Allow excluding image type capabilities https://review.opendev.org/756534 | |
| 14:03:19 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Refactor _claim_pci_device_for_interface_attach to prepare for qos https://review.opendev.org/756895 | |
| 14:03:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Make remove allocation symmetric with add allocation https://review.opendev.org/757110 | |
| 14:03:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Allow extending an existing instance allocation https://review.opendev.org/757109 | |
| 14:03:21 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Factor out port resource deallocation https://review.opendev.org/757111 | |
| 14:07:29 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support interface attach with qos ports https://review.opendev.org/756530 | |
| 14:40:40 | CeeMac | any thoughts on if its possible to set the libvirt cpu_mode value against an image, flavour or instance? | |
| 15:03:50 | sean-k-mooney | CeeMac: its not possible and it was reject in both the image and flaovr in the past | |
| 15:04:16 | sean-k-mooney | CeeMac: we would prefer you to use traits to sate what instuctions you need instead | |
| 15:04:38 | sean-k-mooney | we will then schdule you to a host that has a cpu model that you can use | |
| 15:05:08 | sean-k-mooney | cpu_mode specificly is defined by the admin in the config and not overrideable because it has live migration implciations | |
| 15:06:13 | CeeMac | sean-k-mooney: thanks, I'm trying to expose hardware virtualization to a specific instance and didn't want to go down the line of making a global cpu_mode change | |
| 15:06:30 | CeeMac | sean-k-mooney: do you know which / if any specific traits would be avaialable to do that? | |
| 15:06:56 | sean-k-mooney | CeeMac: we intentionally dont expose this via a user or public api as its a interoperatblity issue and only applys to one virt dirver | |
| 15:07:27 | sean-k-mooney | CeeMac: do you mean nested virt e.g. expose vmx | |
| 15:07:35 | CeeMac | yes | |
| 15:07:39 | CeeMac | kvm is enabled for nested | |
| 15:07:41 | sean-k-mooney | or are you asking to chdule based on qemu vs kvm | |
| 15:07:46 | sean-k-mooney | ah | |
| 15:07:47 | CeeMac | but the guest isn't seeing the flag | |
| 15:07:56 | sean-k-mooney | then you can use vmx i bleive | |
| 15:08:22 | sean-k-mooney | https://github.com/openstack/os-traits/blob/master/os_traits/hw/cpu/x86/intel.py#L26 | |
| 15:08:40 | sean-k-mooney | so HW_CPU_X86_VMX | |
| 15:08:53 | sean-k-mooney | sorry | |
| 15:09:02 | sean-k-mooney | HW_CPU_X86_INTEL_VMX | |
| 15:09:26 | sean-k-mooney | for amd systems you would use HW_CPU_X86_SVM | |
| 15:09:45 | sean-k-mooney | we dont currently have a way to say X or Y unfutetly | |
| 15:10:00 | CeeMac | thanks, is that something I can specify directly on an instance? or is it added at the image or flavour? | |
| 15:10:20 | sean-k-mooney | no you can only set the required trait in the flaovr/image | |
| 15:10:23 | sean-k-mooney | not per instnace | |
| 15:10:37 | CeeMac | right | |
| 15:10:39 | sean-k-mooney | doign things per instance does not really work with nova | |
| 15:10:41 | CeeMac | so its flavour or image? | |
| 15:10:48 | sean-k-mooney | yes both work | |
| 15:10:49 | CeeMac | yeah i noticed that :) | |
| 15:10:51 | CeeMac | cool | |
| 15:10:54 | CeeMac | great thanks a lot | |
| 15:11:01 | CeeMac | i'll try creatign a custom flavour | |
| 15:12:53 | sean-k-mooney | CeeMac: that will just ensure you land on a host that has nested vert enabled | |
| 15:12:59 | sean-k-mooney | but all other vms on that host will also get it | |
| 15:13:20 | sean-k-mooney | configureing nested virt is a system wide thing | |
| 15:13:28 | CeeMac | right | |
| 15:13:42 | sean-k-mooney | the other way that people do this without a trait | |
| 15:13:46 | CeeMac | so the vmx flag should automatically be exposed to the guest if KVM is configured for nested? | |
| 15:14:05 | sean-k-mooney | is to use the instance extra and host aggrates | |
| 15:14:20 | sean-k-mooney | am yes if you use host model or host-passthough as the mode | |
| 15:15:07 | melwitt | dansmith: I found a small bug in the cherry-pick hash check script, if you wouldn't mind reviewing https://review.opendev.org/756639 | |
| 15:16:00 | dansmith | melwitt: cool | |
| 15:16:23 | melwitt | thanks! | |
| 15:16:34 | CeeMac | i don't have cpu_mode set in nova.conf by the look of it | |
| 15:17:26 | CeeMac | and looking at the virsh dumpxml for an instance the cpu mode is 'custom' with a fallback model Skylake-Client-IBRS | |
| 15:18:50 | sean-k-mooney | CeeMac if you dont set it it default to None which in the libvirt driver is converted to host-model | |
| 15:18:55 | melwitt | lyarwood: easy fix for the cherry-pick hash check script if you get a moment https://review.opendev.org/756639 | |
| 15:19:33 | CeeMac | sean-k-mooney: thanks, i think I recall reading something like that | |
| 15:19:45 | CeeMac | i can't seem to find where to set the trait in metadata | |
| 15:19:51 | CeeMac | could be veing blind though | |
| 15:20:52 | sean-k-mooney | openstack flavor set --property trait:HW_CPU_X86_VMX=retuired <flavor> | |