| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-12-02 | |||
| 22:50:27 | rm_work | ah well | |
| 22:50:27 | sean-k-mooney | vms always need a AZ | |
| 22:50:54 | sean-k-mooney | rm_work: well you can propose it upstream or do it with api middelware | |
| 22:51:17 | sean-k-mooney | e.g. you coudl write middleware to reject all server creates that dont have an az | |
| 22:51:50 | rm_work | yeah that might be an interesting approach | |
| 22:51:56 | sean-k-mooney | this is config driven api bevhiaor and will break interoprablity between your cloud and others however | |
| 22:52:21 | sean-k-mooney | e.g. by default you should be able to boot without specifying an az | |
| 22:52:39 | sean-k-mooney | so it will be a qurik of your cloud that its always required | |
| 22:52:50 | rm_work | yeah, unfortunate but documentable | |
| 22:52:58 | rm_work | and internal only cloud so not hugely worried | |
| 23:09:32 | openstackgerrit | DatLQ proposed openstack/nova stable/victoria: Fix unplugging VIF when migrate/resize VM https://review.opendev.org/c/openstack/nova/+/764504 | |
| 23:57:28 | dcapone2004 | hi, is there any using openstack flavor traits/image metadata to pass a very specific CPU configuration to qemu-kvm in openstack nova when a virtual machine is started? Essentially, we are trying to find a way to implement this solution in an openstack environment | |
| 23:57:28 | dcapone2004 | https://www.mess.org/2020/06/22/Hyper-V-in-KVM/#:~:text=Hyper%2DV%20on%20Windows%20requires,it%20is%... | |
| #openstack-nova - 2020-12-03 | |||
| 00:03:22 | sean-k-mooney | dcapone2004: what exactly are you trying to enable | |
| 00:03:53 | sean-k-mooney | dcapone2004:in general the answer is no by design that is not allowed but we may already have a way to do what you want in a more abstract way | |
| 00:04:18 | sean-k-mooney | <feature policy="disable" name="hypervisor"/> | |
| 00:04:20 | sean-k-mooney | <feature policy="require" name="vmx"/> | |
| 00:04:45 | sean-k-mooney | so do you jsut want vmx or do you also want to disabel the hyperviror feature flag | |
| 00:04:49 | sean-k-mooney | you can do both already | |
| 00:05:26 | sean-k-mooney | vmx will be present if you use host-model or host-passthough for the cpu_mode in the libvirt section of the nova.conf | |
| 00:06:14 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.cpu_mode | |
| 00:06:52 | sean-k-mooney | if you use cpu_mode=custom then you need to use cpu_model_extra_flags to add vmx https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.cpu_model_extra_flags | |
| 00:07:01 | sean-k-mooney | for <feature policy="disable" name="hypervisor"/> | |
| 00:08:27 | sean-k-mooney | you need to add either hw:hide_hypervisor_id=true in the flavor extra specs or img_hide_hypervisor_id in the image properties | |
| 00:08:36 | sean-k-mooney | dcapone2004:^ | |
| 00:09:11 | sean-k-mooney | dcapone2004: that will allow you to set those two flags in the libvirt xml | |
| 00:10:42 | sean-k-mooney | dcapone2004: note that adding vmx will only work if you actully have nested virt enabled in the host kernel kvm module parmaters | |
| 00:10:51 | sean-k-mooney | in this case the intel_kvm module | |
| 00:11:32 | sean-k-mooney | it also requires you to use virt-type kvm rather then qemu but you should be using kvm by default anyway just tought i would mention it just in case | |
| 00:11:55 | sean-k-mooney | dcapone2004: hopefully that helps | |
| 00:20:58 | dcapone2004 | sean I already tried all you suggested | |
| 00:21:00 | dcapone2004 | it doesn't work | |
| 00:21:21 | dcapone2004 | the hide_hyperivsor_id=true property passes a kvm=no to qemu | |
| 00:21:51 | dcapone2004 | instead of hypervisor=no, I have nested virtualization enabled and it works when my nested hypervisor is Linux/KVM | |
| 00:23:38 | dcapone2004 | but just as described in several google search articles I found, when trying it with hyper-v, once hyper-v is enabled in a VM, the VM locks up and no longer boots... the solution appears to be based on those settings in that article I found | |
| 00:24:11 | dcapone2004 | I am passing the vmx flag to the VM with nova.conf and the cpu_model_extra_flags configuration | |
| 00:31:30 | sean-k-mooney | what version of openstack are you using and what version of the linux kernel are you using | |
| 00:32:34 | sean-k-mooney | dcapone2004: hide_hyperivsor_id=true should not disable kvm | |
| 00:34:01 | sean-k-mooney | oh i see what its doing | |
| 00:35:19 | sean-k-mooney | it will produce something like this by default http://paste.openstack.org/show/800674/ | |
| 00:35:47 | sean-k-mooney | vmx=on,pdcm=on,pcid=on,hypervisor=on, with kvm=off | |
| 00:36:04 | dcapone2004 | ussuri, kernel 4.10, centos 8 standard kernel | |
| 00:36:27 | sean-k-mooney | but i think that changes if you set the image type to windows | |
| 00:36:55 | sean-k-mooney | ussuri has the hyperv change i was wondering about | |
| 00:37:20 | dcapone2004 | so, I don't have as many flags as you enabled | |
| 00:37:32 | dcapone2004 | what happens on mine is that NO hypervisor option is passed | |
| 00:37:40 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5362-L5387 | |
| 00:37:46 | dcapone2004 | and I think I need it to explicitly pass no based on that "fix" article I read | |
| 00:37:49 | sean-k-mooney | this is the addtional logic for windows guest | |
| 00:38:09 | sean-k-mooney | dcapone2004: im using cpu_mode=host-passthough | |
| 00:38:22 | sean-k-mooney | so its adding all of my virutalisable cpu flags by default | |
| 00:38:57 | sean-k-mooney | dcapone2004: can you check if you have os_type=windows in the image metadata | |
| 00:39:12 | sean-k-mooney | dcapone2004: if not that might fix it | |
| 00:39:27 | dcapone2004 | yeah, I'm using custom as we have some different CPUs in our clusters so i use custom to use the lowest common CPU model among them all | |
| 00:39:55 | dcapone2004 | I then use flavor traits to expose higher CPU models to VMs when needed based on the flavor | |
| 00:40:28 | dcapone2004 | aka, AVX2 to expose Haswell and have the scheduler route the VM to a node that has Haswell or better CPUs | |
| 00:40:30 | sean-k-mooney | traits wont make the cpu feature avaiable just so you are aware | |
| 00:40:53 | sean-k-mooney | i assume you are using the recently added support to list multiple cpu models | |
| 00:40:59 | dcapone2004 | correct | |
| 00:41:09 | sean-k-mooney | ya that should work | |
| 00:41:46 | dcapone2004 | I am assuming that it at least passes through the features that are included in cpu_map.xml definitions of each cpu, just none of the "extra traits" unless explicitly defined | |
| 00:42:31 | sean-k-mooney | custom will enable everytin in cpu_map yes | |
| 00:42:36 | dcapone2004 | the VMs recognize the CPU as Haswell vs IvyBridge when booted when I use the flavor when I define AVX2 as required | |
| 00:42:50 | dcapone2004 | ok, that is the behavior I thought I was receiving | |
| 00:43:08 | sean-k-mooney | yes we have logic to select the cpu model form the list that includes the feature you requested | |
| 00:43:20 | sean-k-mooney | we test them in order i belive and use the first that has all requested features | |
| 00:43:37 | dcapone2004 | correct that is the behavior I see and expected | |
| 00:43:57 | sean-k-mooney | well that was the behavior we intended but that does not mean we dont have a bug :) | |
| 00:44:04 | sean-k-mooney | nice to know it works | |
| 00:44:09 | sean-k-mooney | so in your case | |
| 00:44:22 | sean-k-mooney | i think we need to extend this https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5390-L5392 | |
| 00:44:24 | dcapone2004 | only this hypervisor option I can't seem to figure out if it possible and I've been combing through the docs for 2 days now...I usually prefer to solve my issues myself before asking because every time I read docs for one thing I'm trying to fix, I end up learning 4 more things :-) | |
| 00:44:32 | sean-k-mooney | to add hypervior=off | |
| 00:45:14 | sean-k-mooney | dcapone2004: i may have missed it but did you check if you have os-type=windows in the image | |
| 00:45:41 | sean-k-mooney | so that https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5384-L5386 runs | |
| 00:45:48 | dcapone2004 | let me double check, pretty sure our windows images have a bunch of options set according to the docs recommendations but let me verify | |
| 00:46:59 | dcapone2004 | oddly enough i don't....we have os_distro set to win2k19 which is what i think was in the docs | |
| 00:47:19 | dcapone2004 | let me try adding that additional property and see what happens | |
| 00:49:25 | dcapone2004 | so in Horizon, os_type shows up under the XenAPI Driver Options....I think that is why we did not set it | |
| 00:49:38 | dcapone2004 | as I/we thought it was a Xen specific property | |
| 00:49:48 | sean-k-mooney | that code was added to trick nvidia dirvers into tinking it its runing on a physical host | |
| 00:50:00 | sean-k-mooney | hopefully it will help | |
| 00:50:08 | sean-k-mooney | on the horizon front | |
| 00:50:18 | sean-k-mooney | that is actully managed via glance metadefs | |
| 00:50:22 | dcapone2004 | am I looking at the correct property .... yes in my search for documentation, I read the "bug" report and the full history on that implementation :-) | |
| 00:50:27 | sean-k-mooney | so those should likely be updated | |
| 00:51:19 | dcapone2004 | but it is the correct property that I am looking to enable even though it appears to be documented a bit poorly in horizon? | |
| 00:54:02 | dcapone2004 | if (virt_type in ("qemu", "kvm") and | |
| 00:54:26 | sean-k-mooney | no its this one https://github.com/openstack/glance/blob/45749c30c1c02375a85eb17be0ccd983c695953f/etc/metadefs/compute-libvirt-image.json#L97-L105 | |
| 00:54:28 | dcapone2004 | That code calls a different hyperV specific function based on this property so I am hoping it is a winner :-)... will let you know shortly | |
| 00:55:05 | dcapone2004 | yeah that option in horizon is listed under XenApi in Horizon for some reason | |
| 00:55:16 | sean-k-mooney | weird | |
| 00:55:44 | sean-k-mooney | ok well if it does not work feel free to file that as a bug and link to the code i pointed too and the article. | |
| 00:55:58 | sean-k-mooney | we shoudl be able to add hypervior=no if libvirt support it | |
| 00:56:06 | sean-k-mooney | which it shoudl | |
| 00:56:23 | sean-k-mooney | but i would hope its not required with the existing code | |
| 00:56:52 | dcapone2004 | ok...I feel confident it is going to work with os_type=windows....if it does it is more of a documentation bug in glance/horizon for that property | |
| 00:57:33 | dcapone2004 | this is what horizon says about the property...OS Type (os_type) | |
| 00:57:34 | dcapone2004 | and it limits the injected host name to less than 16 characters. | |
| 00:58:21 | sean-k-mooney | what i have a ussuri deployment too let me check that wrong | |