Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-12-02
22:51:17 sean-k-mooney e.g. you coudl write middleware to reject all server creates that dont have an az
22:51:50 rm_work yeah that might be an interesting approach
22:51:56 sean-k-mooney this is config driven api bevhiaor and will break interoprablity between your cloud and others however
22:52:21 sean-k-mooney e.g. by default you should be able to boot without specifying an az
22:52:39 sean-k-mooney so it will be a qurik of your cloud that its always required
22:52:50 rm_work yeah, unfortunate but documentable
22:52:58 rm_work and internal only cloud so not hugely worried
23:09:32 openstackgerrit DatLQ proposed openstack/nova stable/victoria: Fix unplugging VIF when migrate/resize VM https://review.opendev.org/c/openstack/nova/+/764504
23:57:28 dcapone2004 hi, is there any using openstack flavor traits/image metadata to pass a very specific CPU configuration to qemu-kvm in openstack nova when a virtual machine is started? Essentially, we are trying to find a way to implement this solution in an openstack environment
23:57:28 dcapone2004 https://www.mess.org/2020/06/22/Hyper-V-in-KVM/#:~:text=Hyper%2DV%20on%20Windows%20requires,it%20is%...
#openstack-nova - 2020-12-03
00:03:22 sean-k-mooney dcapone2004: what exactly are you trying to enable
00:03:53 sean-k-mooney dcapone2004:in general the answer is no by design that is not allowed but we may already have a way to do what you want in a more abstract way
00:04:18 sean-k-mooney <feature policy="disable" name="hypervisor"/>
00:04:20 sean-k-mooney <feature policy="require" name="vmx"/>
00:04:45 sean-k-mooney so do you jsut want vmx or do you also want to disabel the hyperviror feature flag
00:04:49 sean-k-mooney you can do both already
00:05:26 sean-k-mooney vmx will be present if you use host-model or host-passthough for the cpu_mode in the libvirt section of the nova.conf
00:06:14 sean-k-mooney https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.cpu_mode
00:06:52 sean-k-mooney if you use cpu_mode=custom then you need to use cpu_model_extra_flags to add vmx https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.cpu_model_extra_flags
00:07:01 sean-k-mooney for <feature policy="disable" name="hypervisor"/>
00:08:27 sean-k-mooney you need to add either hw:hide_hypervisor_id=true in the flavor extra specs or img_hide_hypervisor_id in the image properties
00:08:36 sean-k-mooney dcapone2004:^
00:09:11 sean-k-mooney dcapone2004: that will allow you to set those two flags in the libvirt xml
00:10:42 sean-k-mooney dcapone2004: note that adding vmx will only work if you actully have nested virt enabled in the host kernel kvm module parmaters
00:10:51 sean-k-mooney in this case the intel_kvm module
00:11:32 sean-k-mooney it also requires you to use virt-type kvm rather then qemu but you should be using kvm by default anyway just tought i would mention it just in case
00:11:55 sean-k-mooney dcapone2004: hopefully that helps
00:20:58 dcapone2004 sean I already tried all you suggested
00:21:00 dcapone2004 it doesn't work
00:21:21 dcapone2004 the hide_hyperivsor_id=true property passes a kvm=no to qemu
00:21:51 dcapone2004 instead of hypervisor=no, I have nested virtualization enabled and it works when my nested hypervisor is Linux/KVM
00:23:38 dcapone2004 but just as described in several google search articles I found, when trying it with hyper-v, once hyper-v is enabled in a VM, the VM locks up and no longer boots... the solution appears to be based on those settings in that article I found
00:24:11 dcapone2004 I am passing the vmx flag to the VM with nova.conf and the cpu_model_extra_flags configuration
00:31:30 sean-k-mooney what version of openstack are you using and what version of the linux kernel are you using
00:32:34 sean-k-mooney dcapone2004: hide_hyperivsor_id=true should not disable kvm
00:34:01 sean-k-mooney oh i see what its doing
00:35:19 sean-k-mooney it will produce something like this by default http://paste.openstack.org/show/800674/
00:35:47 sean-k-mooney vmx=on,pdcm=on,pcid=on,hypervisor=on, with kvm=off
00:36:04 dcapone2004 ussuri, kernel 4.10, centos 8 standard kernel
00:36:27 sean-k-mooney but i think that changes if you set the image type to windows
00:36:55 sean-k-mooney ussuri has the hyperv change i was wondering about
00:37:20 dcapone2004 so, I don't have as many flags as you enabled
00:37:32 dcapone2004 what happens on mine is that NO hypervisor option is passed
00:37:40 sean-k-mooney https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5362-L5387
00:37:46 dcapone2004 and I think I need it to explicitly pass no based on that "fix" article I read
00:37:49 sean-k-mooney this is the addtional logic for windows guest
00:38:09 sean-k-mooney dcapone2004: im using cpu_mode=host-passthough
00:38:22 sean-k-mooney so its adding all of my virutalisable cpu flags by default
00:38:57 sean-k-mooney dcapone2004: can you check if you have os_type=windows in the image metadata
00:39:12 sean-k-mooney dcapone2004: if not that might fix it
00:39:27 dcapone2004 yeah, I'm using custom as we have some different CPUs in our clusters so i use custom to use the lowest common CPU model among them all
00:39:55 dcapone2004 I then use flavor traits to expose higher CPU models to VMs when needed based on the flavor
00:40:28 dcapone2004 aka, AVX2 to expose Haswell and have the scheduler route the VM to a node that has Haswell or better CPUs
00:40:30 sean-k-mooney traits wont make the cpu feature avaiable just so you are aware
00:40:53 sean-k-mooney i assume you are using the recently added support to list multiple cpu models
00:40:59 dcapone2004 correct
00:41:09 sean-k-mooney ya that should work
00:41:46 dcapone2004 I am assuming that it at least passes through the features that are included in cpu_map.xml definitions of each cpu, just none of the "extra traits" unless explicitly defined
00:42:31 sean-k-mooney custom will enable everytin in cpu_map yes
00:42:36 dcapone2004 the VMs recognize the CPU as Haswell vs IvyBridge when booted when I use the flavor when I define AVX2 as required
00:42:50 dcapone2004 ok, that is the behavior I thought I was receiving
00:43:08 sean-k-mooney yes we have logic to select the cpu model form the list that includes the feature you requested
00:43:20 sean-k-mooney we test them in order i belive and use the first that has all requested features
00:43:37 dcapone2004 correct that is the behavior I see and expected
00:43:57 sean-k-mooney well that was the behavior we intended but that does not mean we dont have a bug :)
00:44:04 sean-k-mooney nice to know it works
00:44:09 sean-k-mooney so in your case
00:44:22 sean-k-mooney i think we need to extend this https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5390-L5392
00:44:24 dcapone2004 only this hypervisor option I can't seem to figure out if it possible and I've been combing through the docs for 2 days now...I usually prefer to solve my issues myself before asking because every time I read docs for one thing I'm trying to fix, I end up learning 4 more things :-)
00:44:32 sean-k-mooney to add hypervior=off
00:45:14 sean-k-mooney dcapone2004: i may have missed it but did you check if you have os-type=windows in the image
00:45:41 sean-k-mooney so that https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5384-L5386 runs
00:45:48 dcapone2004 let me double check, pretty sure our windows images have a bunch of options set according to the docs recommendations but let me verify
00:46:59 dcapone2004 oddly enough i don't....we have os_distro set to win2k19 which is what i think was in the docs
00:47:19 dcapone2004 let me try adding that additional property and see what happens
00:49:25 dcapone2004 so in Horizon, os_type shows up under the XenAPI Driver Options....I think that is why we did not set it
00:49:38 dcapone2004 as I/we thought it was a Xen specific property
00:49:48 sean-k-mooney that code was added to trick nvidia dirvers into tinking it its runing on a physical host
00:50:00 sean-k-mooney hopefully it will help
00:50:08 sean-k-mooney on the horizon front
00:50:18 sean-k-mooney that is actully managed via glance metadefs
00:50:22 dcapone2004 am I looking at the correct property .... yes in my search for documentation, I read the "bug" report and the full history on that implementation :-)
00:50:27 sean-k-mooney so those should likely be updated
00:51:19 dcapone2004 but it is the correct property that I am looking to enable even though it appears to be documented a bit poorly in horizon?
00:54:02 dcapone2004 if (virt_type in ("qemu", "kvm") and
00:54:26 sean-k-mooney no its this one https://github.com/openstack/glance/blob/45749c30c1c02375a85eb17be0ccd983c695953f/etc/metadefs/compute-libvirt-image.json#L97-L105
00:54:28 dcapone2004 That code calls a different hyperV specific function based on this property so I am hoping it is a winner :-)... will let you know shortly
00:55:05 dcapone2004 yeah that option in horizon is listed under XenApi in Horizon for some reason
00:55:16 sean-k-mooney weird
00:55:44 sean-k-mooney ok well if it does not work feel free to file that as a bug and link to the code i pointed too and the article.
00:55:58 sean-k-mooney we shoudl be able to add hypervior=no if libvirt support it
00:56:06 sean-k-mooney which it shoudl
00:56:23 sean-k-mooney but i would hope its not required with the existing code
00:56:52 dcapone2004 ok...I feel confident it is going to work with os_type=windows....if it does it is more of a documentation bug in glance/horizon for that property
00:57:33 dcapone2004 this is what horizon says about the property...OS Type (os_type)
00:57:34 dcapone2004 and it limits the injected host name to less than 16 characters.
00:58:21 sean-k-mooney what i have a ussuri deployment too let me check that wrong
00:58:33 dcapone2004 makes it seem as though the property is very Xen specific....your code clearly shows it is used in libvirt as well
00:59:59 sean-k-mooney ya so its not under the libvirt driver options for image section and is in zen with differet text
01:00:44 sean-k-mooney oh https://github.com/openstack/glance/commit/e1fe3024bb2482853130fac0be87e6a6a44e610e

Earlier   Later