| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-18 | |||
| 09:19:46 | kashyap | s/is/is still/ | |
| 09:21:06 | openstackgerrit | Yongfeng Du proposed openstack/nova master: Update the file for IPv4-only or IPv6-only network https://review.openstack.org/465891 | |
| 09:29:46 | openstackgerrit | Eli Qiao proposed openstack/nova master: Correct force host checking for LM and evacuation. https://review.openstack.org/465895 | |
| 09:32:17 | openstackgerrit | Eli Qiao proposed openstack/nova master: Correct force host checking for LM and evacuation https://review.openstack.org/465895 | |
| 09:33:44 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix uuid replacement in aggregate notification test https://review.openstack.org/465896 | |
| 09:41:32 | openstackgerrit | Jianghua Wang proposed openstack/nova master: Skip policy rules on attach_network for none network allocation https://review.openstack.org/465903 | |
| 09:43:58 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Delete existing allocations when retrying https://review.openstack.org/460433 | |
| 09:43:58 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Scheduler returning alternatives per destination https://review.openstack.org/465175 | |
| 09:43:59 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Allocate instance by the conductor https://review.openstack.org/460455 | |
| 09:43:59 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Don't delete allocation if instance being scheduled https://review.openstack.org/460177 | |
| 09:46:55 | kashyap | mdbooth: Noticed the other day, wow, this is merged: https://review.openstack.org/#/c/383859/ ("Fix races with nfs volume mount/umount") | |
| 09:47:07 | mdbooth | kashyap: Yeah :) | |
| 10:13:32 | pjm6 | good morning | |
| 10:13:35 | pjm6 | i have a problem with live migration, basically I put a cpu_mode: custom and cpu_model: SandyBridge | |
| 10:13:38 | pjm6 | when i migrate from compute01 to compute02 it works good | |
| 10:13:40 | pjm6 | but the other way arround no, because tthe compute02 has more flags | |
| 10:13:43 | pjm6 | but by forcing the cpu_model SandyBridge this should be working, no? | |
| 10:15:08 | kashyap | pjm6: Do you have the same cpu_{mode,model} config on all Compute nodes | |
| 10:16:09 | pjm6 | kashyap, yes I have | |
| 10:16:40 | kashyap | BTW, what release are you trying this with? Can't say more without details | |
| 10:16:51 | pjm6 | of course, sorry | |
| 10:16:56 | pjm6 | i'm in Ocata version | |
| 10:17:09 | pjm6 | using nova 15.1.2 | |
| 10:20:13 | pjm6 | kashyap, this is old but https://blog.codecentric.de/en/2015/03/true-kvm-live-migration-openstack-icehouse-ceph-based-vm-storage/ it seems that I have the problem #2 | |
| 10:23:26 | pjm6 | the error says: Unacceptable CPU info: CPU doesn't have compatibility | |
| 10:24:59 | kashyap | pjm6: That was in IceHouse | |
| 10:25:26 | pjm6 | kashyap, yes I said that was an older version, but the problem seems similar | |
| 10:25:28 | pjm6 | for isntance | |
| 10:26:20 | pjm6 | the error log: https://pastebin.com/zd35WqqJ | |
| 10:27:22 | pjm6 | then I use the virsh cpu-baseline and I get this: https://pastebin.com/AxUx0kYf | |
| 10:28:06 | kashyap | pjm6: BTW, just a gentle note: this channel is for devel purposes, maybe you're already aware | |
| 10:30:21 | kashyap | pjm6: I'm just curious, any reason you don't use 'host-model'? | |
| 10:30:39 | kashyap | Which is actually the default, which use the closest available named-CPU model | |
| 10:31:14 | pjm6 | kashyap, i didn't use host-model because I though that if i wanted to use SandyBridge the cpu_mode should be custom | |
| 10:31:36 | pjm6 | or i can use cpu_mode: host-model and cpu_model: SandyBridge? | |
| 10:31:41 | kashyap | pjm6: Yes, that is true _if_ you specifically want the SandyBridge | |
| 10:31:55 | kashyap | pjm6: No, you can't do that | |
| 10:32:15 | pjm6 | kashyap, the problem is that i have two different cpus, one is Broadwell the other is SandyBridge | |
| 10:32:36 | pjm6 | i was trying to "downgrade" the instruction set in order to allow live migration | |
| 10:33:08 | pjm6 | but it seems that when i migrate for the newer to the old, he's comparing physical cpu specs from one server to the other, and then fails | |
| 10:33:14 | pjm6 | 2 line of the first logs | |
| 10:34:05 | kashyap | pjm6: Ah-ha, okay, your problem is clearer. If using the lowest-common denominator CPU model between the two hosts, it should 'just work' | |
| 10:34:49 | pjm6 | yes, that's what gived to me the virsh cpu-baseline (which had the two CPU configs) | |
| 10:38:15 | kashyap | pjm6: That should clearly work. Do you have a libvirt debug log with log filters enabled? And please file a Nova bug with details | |
| 10:38:48 | pjm6 | kashyap, for now i have debug disabled and don't have any filter log | |
| 10:38:56 | pjm6 | but i can activate the debug yes | |
| 10:39:15 | pjm6 | I also see that the dumpxml are using the correct cpu model | |
| 10:40:15 | kashyap | pjm6: BTW, to enable libvirt log filters, this is the way to go: https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt | |
| 10:40:58 | kashyap | pjm6: What are the libvirt / QEMU versions? Just in case if I happened to have an env. I can test it | |
| 10:41:11 | kashyap | (To see if could reproduce the behaviour.) | |
| 10:41:42 | pjm6 | kashyap, will see that | |
| 10:42:13 | pjm6 | libvirt 2.5.0 | qemu 1:2.8 | |
| 10:43:23 | kashyap | pjm6: Do you have a libvirt guest XML for a VM started on both hosts? | |
| 10:43:36 | kashyap | (Can you post them somewhere, please) | |
| 10:44:38 | pjm6 | kashyap, you want a dumpxml of a VM that was created in compute01 and compute02 right/ | |
| 10:44:53 | kashyap | pjm6: Yep | |
| 10:44:59 | pjm6 | 1 sec please | |
| 10:52:48 | kashyap | pjm6: Also, one more: please get `virsh capabilities` from both hosts | |
| 10:54:55 | pjm6 | dumpxml: https://pastebin.com/xbuA2BFS and https://pastebin.com/NPRAEjbV | |
| 10:57:45 | pjm6 | virsh capabilities: https://pastebin.com/upCBBTu6 and https://pastebin.com/aLDeQipG | |
| 10:58:10 | kashyap | Thanks; will look in a few | |
| 10:58:22 | pjm6 | well kashyap | |
| 10:58:24 | pjm6 | i found a problem! | |
| 10:58:37 | kashyap | pjm6: ? | |
| 10:58:37 | pjm6 | compute node02 for some reason is using broadwell | |
| 10:58:40 | kashyap | Hehe | |
| 10:59:02 | pjm6 | well this is strange | |
| 10:59:06 | pjm6 | first i create in compute01 | |
| 10:59:11 | pjm6 | and then migrate to compute02 | |
| 10:59:20 | pjm6 | the was fine | |
| 10:59:25 | pjm6 | and the dumpxml was using sandybridge | |
| 11:03:21 | kashyap | pjm6: Can you get the real error from /var/log/libvirt/libvirtd.log, too? | |
| 11:04:42 | pjm6 | I don't have that file yet | |
| 11:07:34 | sfinucan | (My) life would be so much easier if every instance had a NUMA topology associated with it | |
| 11:10:02 | pjm6 | kashyap, i already activate as you have | |
| 11:10:20 | pjm6 | in the doc | |
| 11:11:33 | kashyap | pjm6: For the actual libvirt error, maybe it's redirected to systemd journal, do you see anything in: | |
| 11:11:37 | kashyap | journalctl -u libvirtd -l --since=yesterday -p err | |
| 11:11:51 | kashyap | s/yesterday/today/ | |
| 11:12:10 | pjm6 | yeah i see error | |
| 11:12:13 | pjm6 | but idk if is this | |
| 11:12:26 | kashyap | pjm6: Can you paste what it is | |
| 11:12:29 | kashyap | sfinucan: What is causing grief? | |
| 11:12:48 | pjm6 | sure | |
| 11:12:49 | kashyap | Oh, you tell it in the following comment | |
| 11:13:33 | pjm6 | internal error: unable to execute QEMU command 'query-cpu-definitions': The command query-cpu-definitions has not been found | |
| 11:14:14 | sfinucan | sfinucan: People changing things in patches to fix something, but neglecting to mention that they fixed said thing | |
| 11:14:22 | sfinucan | Whoops, kashyap ^ | |
| 11:14:44 | kashyap | sfinucan: Yeah, I hear you | |
| 11:22:26 | kashyap | pjm6: I think that error comes as part of a series of commands libvirt invocates during some high-level operation like migration. | |
| 11:22:49 | kashyap | But anyway, ensuring your destination Compute node really is also using the custom CPU model should get you going. As you've noticed the problem. | |
| 11:23:19 | pjm6 | yeah for some reason idk why when creating a VM he don't update the XML | |
| 11:29:26 | pjm6 | kashyap, | |
| 11:29:31 | pjm6 | it seems it's working now | |
| 11:29:40 | pjm6 | but the virsh dumpxml don't show anything related to cpu | |
| 11:29:41 | pjm6 | this is normal/ | |
| 11:30:46 | pjm6 | forget it appears, with less info | |
| 11:31:18 | pjm6 | https://pastebin.com/gbC4inS | |
| 11:31:25 | pjm6 | i just restared the service | |
| 11:40:07 | openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/nova master: WIP: Get disk size of instance in separate process https://review.openstack.org/465653 | |