| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-10-09 | |||
| 13:52:32 | sean-k-mooney | we currently dont enforce that you se hw:mem_page_size if you have a numa toplogy but its always a bug/user errror to not set it | |
| 13:55:14 | stephenfin | if we can figure out a way to do overcommit on a per-node basis, then I'd be okay with that | |
| 13:55:21 | stephenfin | otherwise, it's a big change | |
| 13:56:30 | sean-k-mooney | well as is said overcomit does not work properly today | |
| 13:56:49 | sean-k-mooney | the OOM reaper will kill the vm if the numa node runs out of memory | |
| 13:56:56 | sean-k-mooney | even if the othe rnuma node is fully empty | |
| 13:57:21 | stephenfin | you could say the same thing about overcommit without NUMA | |
| 13:57:32 | sean-k-mooney | its not the same | |
| 13:57:45 | stephenfin | no, but it's similar | |
| 13:57:49 | sean-k-mooney | if you set over commit to 1.0 e.g. no over commit | |
| 13:58:04 | sean-k-mooney | with the current state the vms can be killed becuase the numa node is exausted | |
| 13:58:15 | sean-k-mooney | unless you set hw:mem_page_size=small | |
| 13:59:02 | stephenfin | yes, I understand that | |
| 13:59:03 | sean-k-mooney | its requried for numa in plamcent for numa hosts | |
| 13:59:18 | sean-k-mooney | for non numa hosts i was not going to be an issue since numa instance could not land there | |
| 13:59:33 | stephenfin | but if you had work and instances evenly distributed between NUMA nodes, then it would behave the same as the non-NUMA case | |
| 13:59:46 | sean-k-mooney | so we defered this conversation the last 2 cycles because numa in palcment will prevent it form happening | |
| 14:00:20 | sean-k-mooney | stephenfin: without my numa loadbancing feature it wont evenly disturbute | |
| 14:00:31 | sean-k-mooney | unless you consume all the cpus on the first node | |
| 14:00:39 | sean-k-mooney | even then it won evenly disturbute | |
| 14:02:25 | openstackgerrit | Dan Smith proposed openstack/nova master: Allow excluding image type capabilities https://review.opendev.org/756534 | |
| 14:03:19 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Refactor _claim_pci_device_for_interface_attach to prepare for qos https://review.opendev.org/756895 | |
| 14:03:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Make remove allocation symmetric with add allocation https://review.opendev.org/757110 | |
| 14:03:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Allow extending an existing instance allocation https://review.opendev.org/757109 | |
| 14:03:21 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Factor out port resource deallocation https://review.opendev.org/757111 | |
| 14:07:29 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support interface attach with qos ports https://review.opendev.org/756530 | |
| 14:40:40 | CeeMac | any thoughts on if its possible to set the libvirt cpu_mode value against an image, flavour or instance? | |
| 15:03:50 | sean-k-mooney | CeeMac: its not possible and it was reject in both the image and flaovr in the past | |
| 15:04:16 | sean-k-mooney | CeeMac: we would prefer you to use traits to sate what instuctions you need instead | |
| 15:04:38 | sean-k-mooney | we will then schdule you to a host that has a cpu model that you can use | |
| 15:05:08 | sean-k-mooney | cpu_mode specificly is defined by the admin in the config and not overrideable because it has live migration implciations | |
| 15:06:13 | CeeMac | sean-k-mooney: thanks, I'm trying to expose hardware virtualization to a specific instance and didn't want to go down the line of making a global cpu_mode change | |
| 15:06:30 | CeeMac | sean-k-mooney: do you know which / if any specific traits would be avaialable to do that? | |
| 15:06:56 | sean-k-mooney | CeeMac: we intentionally dont expose this via a user or public api as its a interoperatblity issue and only applys to one virt dirver | |
| 15:07:27 | sean-k-mooney | CeeMac: do you mean nested virt e.g. expose vmx | |
| 15:07:35 | CeeMac | yes | |
| 15:07:39 | CeeMac | kvm is enabled for nested | |
| 15:07:41 | sean-k-mooney | or are you asking to chdule based on qemu vs kvm | |
| 15:07:46 | sean-k-mooney | ah | |
| 15:07:47 | CeeMac | but the guest isn't seeing the flag | |
| 15:07:56 | sean-k-mooney | then you can use vmx i bleive | |
| 15:08:22 | sean-k-mooney | https://github.com/openstack/os-traits/blob/master/os_traits/hw/cpu/x86/intel.py#L26 | |
| 15:08:40 | sean-k-mooney | so HW_CPU_X86_VMX | |
| 15:08:53 | sean-k-mooney | sorry | |
| 15:09:02 | sean-k-mooney | HW_CPU_X86_INTEL_VMX | |
| 15:09:26 | sean-k-mooney | for amd systems you would use HW_CPU_X86_SVM | |
| 15:09:45 | sean-k-mooney | we dont currently have a way to say X or Y unfutetly | |
| 15:10:00 | CeeMac | thanks, is that something I can specify directly on an instance? or is it added at the image or flavour? | |
| 15:10:20 | sean-k-mooney | no you can only set the required trait in the flaovr/image | |
| 15:10:23 | sean-k-mooney | not per instnace | |
| 15:10:37 | CeeMac | right | |
| 15:10:39 | sean-k-mooney | doign things per instance does not really work with nova | |
| 15:10:41 | CeeMac | so its flavour or image? | |
| 15:10:48 | sean-k-mooney | yes both work | |
| 15:10:49 | CeeMac | yeah i noticed that :) | |
| 15:10:51 | CeeMac | cool | |
| 15:10:54 | CeeMac | great thanks a lot | |
| 15:11:01 | CeeMac | i'll try creatign a custom flavour | |
| 15:12:53 | sean-k-mooney | CeeMac: that will just ensure you land on a host that has nested vert enabled | |
| 15:12:59 | sean-k-mooney | but all other vms on that host will also get it | |
| 15:13:20 | sean-k-mooney | configureing nested virt is a system wide thing | |
| 15:13:28 | CeeMac | right | |
| 15:13:42 | sean-k-mooney | the other way that people do this without a trait | |
| 15:13:46 | CeeMac | so the vmx flag should automatically be exposed to the guest if KVM is configured for nested? | |
| 15:14:05 | sean-k-mooney | is to use the instance extra and host aggrates | |
| 15:14:20 | sean-k-mooney | am yes if you use host model or host-passthough as the mode | |
| 15:15:07 | melwitt | dansmith: I found a small bug in the cherry-pick hash check script, if you wouldn't mind reviewing https://review.opendev.org/756639 | |
| 15:16:00 | dansmith | melwitt: cool | |
| 15:16:23 | melwitt | thanks! | |
| 15:16:34 | CeeMac | i don't have cpu_mode set in nova.conf by the look of it | |
| 15:17:26 | CeeMac | and looking at the virsh dumpxml for an instance the cpu mode is 'custom' with a fallback model Skylake-Client-IBRS | |
| 15:18:50 | sean-k-mooney | CeeMac if you dont set it it default to None which in the libvirt driver is converted to host-model | |
| 15:18:55 | melwitt | lyarwood: easy fix for the cherry-pick hash check script if you get a moment https://review.opendev.org/756639 | |
| 15:19:33 | CeeMac | sean-k-mooney: thanks, i think I recall reading something like that | |
| 15:19:45 | CeeMac | i can't seem to find where to set the trait in metadata | |
| 15:19:51 | CeeMac | could be veing blind though | |
| 15:20:52 | sean-k-mooney | openstack flavor set --property trait:HW_CPU_X86_VMX=retuired <flavor> | |
| 15:21:00 | sean-k-mooney | i think that is the correct command | |
| 15:21:04 | sean-k-mooney | stephenfin: ^ | |
| 15:21:33 | sean-k-mooney | stephenfin: do you know if you have to quote the trait? | |
| 15:21:39 | CeeMac | ah yes, the old cli :) | |
| 15:21:49 | sean-k-mooney | ... retuired should be required | |
| 15:21:54 | CeeMac | got it | |
| 15:22:00 | sean-k-mooney | CeeMac: never trust my spelling | |
| 15:22:18 | stephenfin | sean-k-mooney: only if there are spaces or things the shell could confuse | |
| 15:22:29 | stephenfin | I usually quote to be safe | |
| 15:22:50 | sean-k-mooney | ya i often do but i was not sure if it would mess up the parsing of the = | |
| 15:24:22 | CeeMac | stephenfin: so "trait:.....required" <flavour> | |
| 15:25:14 | stephenfin | CeeMac: Yup. We list the supported versions here https://docs.openstack.org/nova/latest/configuration/extra-specs.html#trait | |
| 15:25:19 | stephenfin | *supported traits | |
| 15:25:41 | CeeMac | perfect, thanks | |
| 15:26:04 | sean-k-mooney | CeeMac: just be aware that we and them together | |
| 15:26:19 | sean-k-mooney | so if you use the amd trait and the intel trait then it wont find a host | |
| 15:26:45 | CeeMac | right, good to know :) | |
| 15:26:53 | sean-k-mooney | if you have a mix you should use a custom extra spec instead of a trait and use host aggreates | |
| 15:27:31 | CeeMac | i'm pretty sure at this point I don't want to do it at all, but i'll try it in my test env to be sure | |
| 15:27:41 | CeeMac | stephenfin: from that link it looks like I need trait{group}:HW_CPU_X86_VMX | |
| 15:27:55 | CeeMac | is the {group} specific to something? | |
| 15:28:02 | sean-k-mooney | there was a proposal to support "in" so you coudl do somehtin like trait:in(intel...,amd...)=required | |
| 15:28:13 | sean-k-mooney | CeeMac: you shoudl avoid groups | |