| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-12-03 | |||
| 09:37:23 | openstackgerrit | Maciej Jozefczyk proposed openstack/nova master: Force refresh instance info_cache during heal https://review.openstack.org/591607 | |
| 09:40:22 | openstackgerrit | Michael Still proposed openstack/nova master: Fix sloppy initialization of the new disk ops semaphore. https://review.openstack.org/621346 | |
| 09:40:23 | openstackgerrit | Michael Still proposed openstack/nova master: Remove utils.execute() from libvirt remotefs calls. https://review.openstack.org/619701 | |
| 09:40:23 | openstackgerrit | Michael Still proposed openstack/nova master: Remove utils.execute() calls from xenapi. https://review.openstack.org/619700 | |
| 09:40:24 | openstackgerrit | Michael Still proposed openstack/nova master: Move nova.libvirt.utils away from using nova.utils.execute(). https://review.openstack.org/619703 | |
| 09:40:24 | openstackgerrit | Michael Still proposed openstack/nova master: Remove utils.execute() from quobyte libvirt storage driver. https://review.openstack.org/619702 | |
| 09:40:25 | openstackgerrit | Michael Still proposed openstack/nova master: Remove final users of utils.execute() in libvirt. https://review.openstack.org/619705 | |
| 09:40:25 | openstackgerrit | Michael Still proposed openstack/nova master: Imagebackend should call processutils.execute directly. https://review.openstack.org/619704 | |
| 09:40:26 | openstackgerrit | Michael Still proposed openstack/nova master: Remove utils.execute() from the hyperv driver. https://review.openstack.org/620008 | |
| 09:40:26 | openstackgerrit | Michael Still proposed openstack/nova master: Remove the final user of utils.execute() from virt.images https://review.openstack.org/620007 | |
| 09:40:27 | openstackgerrit | Michael Still proposed openstack/nova master: Move a generic bridge helper to a linux_net privsep file. https://review.openstack.org/620010 | |
| 09:40:27 | openstackgerrit | Michael Still proposed openstack/nova master: Remove utils.execute() from virt.disk.api. https://review.openstack.org/620009 | |
| 09:40:28 | openstackgerrit | Michael Still proposed openstack/nova master: Move some linux network helpers to use privsep. https://review.openstack.org/621398 | |
| 09:40:28 | openstackgerrit | Michael Still proposed openstack/nova master: Move bridge creation to privsep. https://review.openstack.org/620180 | |
| 09:40:29 | openstackgerrit | Michael Still proposed openstack/nova master: Move interface enabling to privsep. https://review.openstack.org/621528 | |
| 09:40:29 | openstackgerrit | Michael Still proposed openstack/nova master: Move simple execute call to processutils. https://review.openstack.org/621527 | |
| 09:40:30 | openstackgerrit | Michael Still proposed openstack/nova master: Move setting mac addresses for network devices to privsep. https://review.openstack.org/621529 | |
| 09:45:49 | izza_ | hi..can anyone help me here about error that we've encountered upon testing vgpu in devstack using tensorflow | |
| 09:46:16 | izza_ | encountering below error: The TensorFlow library was compiled to use SSE4.1 instructions, but these aren't available on your machine. | |
| 10:05:13 | bauzas | izza_: good morning | |
| 10:05:34 | bauzas | izza_: when you say "testing vgpus", you mean using https://docs.openstack.org/nova/latest/admin/virtual-gpu.html ? | |
| 10:06:05 | bauzas | izza_: I guess you use the libvirt driver with KVM ? | |
| 10:08:03 | izza_ | able to detect the vgpu...the thing is when we tried to deploy tensorflow in devstack, we're encountering such error.. yes, it is already set as KVM | |
| 10:08:26 | bauzas | izza_: I guess you probably don't have the right CPU model | |
| 10:11:02 | bauzas | izza_: which CPU mode do you use for all your computes ? | |
| 10:12:04 | bauzas | context is https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.cpu_mode | |
| 10:13:00 | bauzas | AFAIR, using "None" for your CPU mode in Nova makes you get a very old CPU model | |
| 10:13:08 | bauzas | for migration reasons | |
| 10:23:04 | bauzas | izza_: if you wanna get more on CPU models, see https://www.berrange.com/posts/2018/06/29/cpu-model-configuration-for-qemu-kvm-on-x86-hosts/ | |
| 10:32:31 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: DNM: troubleshoot cells v1 tests on gate https://review.openstack.org/621538 | |
| 10:38:23 | sean-k-mooney | bauzas: using none for cpu mode should result in host model which will be the most recent cpu model that matches your host | |
| 10:40:30 | izza_ | so it should be none? | |
| 10:41:34 | sean-k-mooney | izza_: you should always set it explctly to a supported value or not at all. | |
| 10:41:51 | sean-k-mooney | none will indrectly result in host-model | |
| 10:43:44 | sean-k-mooney | oh actully maybe bauzas is correct https://github.com/openstack/nova/blob/master/nova/conf/libvirt.py#L490-L493 | |
| 10:46:15 | sean-k-mooney | it looks like we treat the string "none" differently form that value None | |
| 10:46:18 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3796-L3815 | |
| 10:47:48 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Add workaround to remove instance dir during cleanup with rbd https://review.openstack.org/618478 | |
| 10:48:07 | sean-k-mooney | izza_: try setting it to host-model explcitly | |
| 10:49:05 | sean-k-mooney | izza_: bauzas may already have covered this but you might also need to use a different vGPU type to enable tensorflow to work with gpu accleration. | |
| 10:49:17 | sean-k-mooney | only some models support opencl/cuda | |
| 10:50:48 | bauzas | sean-k-mooney: izza_: the conf option help says that cpu_mode defaults to host-model if KVM | |
| 10:51:07 | bauzas | so you should get the SSE4 instructions if your CPU is recent | |
| 10:51:25 | bauzas | but worth checking the guest XML | |
| 10:51:49 | izza_ | im checking it now | |
| 10:52:23 | bauzas | izza_: please tell us the config options values for [libvirt]/cpu_mode and [libvirt]/cpu_model | |
| 10:52:32 | bauzas | izza_: besides the virt type | |
| 10:52:36 | sean-k-mooney | sse4 is pretty old. i would be surprised if the host did not supported it | |
| 10:52:41 | bauzas | I agree | |
| 10:53:06 | bauzas | the only thing I could think of would be cpu_mode explicitely set to None | |
| 10:53:48 | izza_ | [libvirt] | |
| 10:53:50 | bauzas | which would make the guest using a very old CPU model that doesn't support SSE1 | |
| 10:53:53 | bauzas | SSE4* | |
| 10:54:04 | izza_ | live_migration_uri = qemu+ssh://stack@%s/system | |
| 10:54:10 | izza_ | cpu_mode = host-passthrough | |
| 10:54:16 | izza_ | cpu_model_extra_flags = sse4_1 | |
| 10:54:22 | izza_ | virt_type = kvm | |
| 10:54:47 | sean-k-mooney | izza_: host-passthrough will give you the same feature flags as the host. you dod not need the extra flags | |
| 10:55:14 | izza_ | noted on this..we will remove it.. | |
| 10:55:23 | sean-k-mooney | izza_: you have sse in the hosts /proc/cpuinfo right? | |
| 10:55:39 | izza_ | but we tried to remove it and set is as passthrough without flags | |
| 10:55:48 | izza_ | but still unable to get the other flags | |
| 10:56:07 | sean-k-mooney | every xeon since nehelem has had sse 4.2+ | |
| 10:56:26 | izza_ | yes, sse4.1 is available in /proc/cpuinfo | |
| 10:56:52 | sean-k-mooney | izza_: and your setting cpu_mode = host-passthrough in the /etc/nova/nova-cpu.conf | |
| 10:58:01 | izza_ | it is set as "none" in /etc/nova/nova-cpu.conf | |
| 10:58:12 | sean-k-mooney | izza_: that the issue | |
| 10:58:30 | izza_ | what should i do? | |
| 10:58:36 | sean-k-mooney | /etc/nova/nova-cpu.conf is used by the compute agent /etc/nova/nova.conf is used by the schduer/conductor | |
| 10:59:07 | izza_ | ok will try to change it | |
| 10:59:57 | izza_ | we will advise you for the result | |
| 11:00:08 | sean-k-mooney | [[post-config|$NOVA_CPU_CONF]] | |
| 11:00:11 | sean-k-mooney | [libvirt] | |
| 11:00:13 | sean-k-mooney | cpu_mode = host-passthrough | |
| 11:00:15 | sean-k-mooney | virt_type = kvm | |
| 11:00:19 | sean-k-mooney | that is how to set it in your local.conf by the way | |
| 11:00:51 | izza_ | ok noted on that | |
| 11:00:54 | izza_ | we'll try | |
| 11:01:00 | sean-k-mooney | you can change it locally and then restart the nova services with sudo systemctl restart devstack@n-* | |
| 11:01:20 | bauzas | what sean-k-mooney said | |
| 11:01:30 | sean-k-mooney | * by locally i mean in /etc/nova/nova-cpu.conf | |
| 11:01:34 | bauzas | izza_: your n-cpu service uses the wrong file | |
| 11:01:36 | izza_ | thanks guys | |
| 11:04:40 | sean-k-mooney | bauzas: well a different file we use /etc/nova/nova-cpu.conf instead of /etc/nova/nova.conf intentionally | |
| 11:04:49 | izza_ | got it...already capture the cpu details after changing the said file | |
| 11:05:11 | bauzas | 'wrong' is maybe not the right word | |
| 11:05:28 | bauzas | what I mean is that n-cpu uses another conf file by default | |
| 11:05:31 | bauzas | but you can change this | |
| 11:05:45 | sean-k-mooney | yep | |
| 11:05:56 | bauzas | just don't use systemctl in this case | |
| 11:06:11 | sean-k-mooney | ya not? | |
| 11:06:25 | sean-k-mooney | *why not? | |
| 11:06:50 | bauzas | just becauses it precludes the config file name | |
| 11:07:08 | bauzas | sometimes I prefer stopping the service and starting it by hand | |
| 11:07:21 | bauzas | maybe because I'm an old guy | |
| 11:07:40 | sean-k-mooney | systemctl is the correct way to do that in devstack for like 2-3 releases | |
| 11:08:12 | sean-k-mooney | infact that devstack docs use n-cpu as the example https://docs.openstack.org/devstack/latest/systemd.html#manipulating-units | |
| 11:14:37 | jangutter | bauzas: I'm still sometimes using 'ifconfig', where I should be using 'ip', and really, really, should be using 'nmcli'. | |
| 11:14:50 | bauzas | sean-k-mooney: you misunderstood me | |
| 11:15:19 | bauzas | sean-k-mooney: I'm not saying to not use systemctl (although there is still a devstack option for it :p ) | |