| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-17 | |||
| 11:47:31 | admin0 | well, hav been trying to do kolla many times .. but still on osa | |
| 11:47:40 | yoctozepto | no problem | |
| 11:48:08 | sean-k-mooney | ok so you libvirt is running in an lxc container or on the host? | |
| 11:48:24 | admin0 | thing is if i ask some hard questions on kolla related to config or troubleshooting, there is no help on kolla but on osa its more in depth and instant .. which probably is because of rackspace | |
| 11:48:29 | admin0 | its on metal | |
| 11:48:56 | sean-k-mooney | ok can you try virsh --connect qemu:///system list | |
| 11:48:58 | yoctozepto | admin0: :O no help on kolla? how is that? | |
| 11:49:30 | sean-k-mooney | well i guess with --all | |
| 11:50:24 | sean-k-mooney | the xmls should be in /etc/libvirt/qemu | |
| 11:50:27 | yoctozepto | admin0: finally try: virsh define /etc/libvirt/qemu/that_one_file_you_missed_so_much.xml - maybe there was an error in there | |
| 11:51:09 | admin0 | error: invalid argument: could not find capabilities for arch=x86_64 -- wow .same server .. | |
| 11:51:44 | yoctozepto | see, now you got help from kolla, your argument is no longer valid (pun intended) | |
| 11:51:55 | yoctozepto | ;D | |
| 11:52:13 | yoctozepto | kidding aside, this really looks lolz | |
| 11:52:45 | sean-k-mooney | it looks like libvirt cannot find the x86_64 emulator | |
| 11:52:57 | admin0 | i will try kolla again .. won't doubt that .. but this is #nova channel :D | |
| 11:53:01 | yoctozepto | mhm, at least its config/definitions | |
| 11:53:22 | yoctozepto | admin0: yeah, I'm just teasing | |
| 11:53:36 | yoctozepto | though reboot does not get rid of those | |
| 11:53:38 | sean-k-mooney | well its complaining that virsh capabilities is not working | |
| 11:54:11 | sean-k-mooney | no a reboot would not cause this and nither would nuking the contolers | |
| 11:54:26 | sean-k-mooney | this host has been broken by something else that is unrelated | |
| 11:54:57 | yoctozepto | +1, I know the drill | |
| 11:54:59 | admin0 | in virsh capabilities i do see <arch>x86_64</arch> reported with the flags | |
| 11:55:05 | yoctozepto | some issues popping up only after reboot | |
| 11:55:20 | yoctozepto | as in "cleanup gone wrong and discovered weeks later" | |
| 11:55:40 | yoctozepto | odd | |
| 11:56:00 | yoctozepto | uhm | |
| 11:56:10 | yoctozepto | is kvm loaded by any chance? | |
| 11:56:18 | yoctozepto | both main and cpu-specific? | |
| 11:56:22 | yoctozepto | lsmod | grep kvm | |
| 11:56:53 | yoctozepto | admin0: ^ | |
| 11:56:55 | admin0 | but this is a simulated "oops something bad happened" .. i rebooted the box , rebooted the vms multiple times before nuking the controllers and restart to make sure no skeletons in the closet pop up | |
| 11:57:09 | admin0 | i see kvm_intel, and kvm and irqbypass | |
| 11:57:15 | yoctozepto | hmm, odd | |
| 11:57:27 | yoctozepto | and what about | |
| 11:57:29 | yoctozepto | dmesg | grep kvm | |
| 11:57:37 | admin0 | null on that | |
| 11:57:54 | yoctozepto | then fine | |
| 11:58:05 | sean-k-mooney | admin0: this behavior is not related to the host reboot or the contoler unless you are useing shard stroageve form the contolers on the compute nodes | |
| 11:58:05 | admin0 | modprobe kvm && dmesg | grep kvm = also nothing | |
| 11:58:06 | yoctozepto | everything's all right but it does not work | |
| 11:58:19 | yoctozepto | nah, that should be nothing | |
| 11:58:20 | sean-k-mooney | e.g. you put your instnace data dir on an nfs share | |
| 11:58:25 | admin0 | this has no ceph and no cinder .. its local storage | |
| 11:58:34 | admin0 | i have the disks file locally | |
| 11:58:37 | yoctozepto | yeah, the error is that x86_64 missing | |
| 11:58:38 | yoctozepto | somehow | |
| 11:58:51 | yoctozepto | what distro is this anyways? | |
| 11:59:02 | yoctozepto | (right time to ask that heh) | |
| 11:59:04 | admin0 | ubuntu 16.04 | |
| 11:59:06 | sean-k-mooney | admin0: i think you need to set up your experiment again clenly | |
| 11:59:11 | yoctozepto | quite an oldie there | |
| 11:59:46 | admin0 | ok.. then for the purpose of this, how do do virsh capabilities and edit hte nova file manually so that it just boots up, gets the IP and rsync, recovery is possible | |
| 11:59:47 | sean-k-mooney | the behavior you are seeing is not consitent with how nova operates | |
| 12:00:02 | admin0 | i undestand it may not boot in a very performant or optimal state, but the scope is recovery here | |
| 12:00:20 | sean-k-mooney | normally you would just do virsh start on the domains | |
| 12:00:25 | sean-k-mooney | no need to edit them | |
| 12:00:33 | admin0 | same would be if the virsh/os crashed and i manually had to copy the disk and xml to another (non-similar) compute node | |
| 12:00:41 | yoctozepto | admin0: is qemu-kvm installed? | |
| 12:01:01 | admin0 | qemu-kvm is already the newest version (1:2.10+dfsg-0ubuntu3.8~cloud1). | |
| 12:01:33 | admin0 | <type arch='x86_64' machine='pc-i440fx-artful'>hvm</type> | |
| 12:01:37 | yoctozepto | then I'm out of cards with this issue, really what comes to mind is some bug in ubuntu's libvirt that you have discovered | |
| 12:01:41 | admin0 | i guess that pc-* is the issue ? | |
| 12:01:56 | yoctozepto | well, you could try modifying that | |
| 12:02:31 | yoctozepto | pc-i440fx-xenial should be there | |
| 12:02:42 | yoctozepto | (at least that's what common sense suggests) | |
| 12:02:56 | admin0 | i see https://gist.github.com/a1git/f3696c126a732cda88757c09263bc2de in virsh capabilites related to x86_64 | |
| 12:03:16 | yoctozepto | nothing suspicious | |
| 12:03:37 | sean-k-mooney | the machine type is normal | |
| 12:03:45 | gibi | efried: I got pulled downstream this morning so my review on the provider config progressing slowly. sorry | |
| 12:03:53 | sean-k-mooney | it will now show up in the cpu feature list | |
| 12:04:00 | sean-k-mooney | *not | |
| 12:04:15 | sean-k-mooney | its basicaly the "motherboard" for lack of a better name the vm uses | |
| 12:09:34 | admin0 | i edited pc-i440fx-artful to pc-i440fx-xenial .. same error as invalid argument: could not find capabilities for arch=x86_64 . i have the virsh capabilities in full , lsmod and the vm definition here => https://gist.github.com/a1git/4528c53090f8934b9218586817c51911 | |
| 12:10:42 | sean-k-mooney | admin0: its not related to the model | |
| 12:11:10 | admin0 | ok .. but based on the output, there is no way this instance can boot up ? | |
| 12:11:12 | sean-k-mooney | admin0: you could set it to "pc" which is an aliase for the latest model supproted that is no q35 | |
| 12:11:55 | sean-k-mooney | admin0: the machine types are not listed in that output | |
| 12:12:15 | admin0 | how do i list them ? | |
| 12:13:08 | sean-k-mooney | its likely because the x68 emulator is also not listed | |
| 12:16:15 | sean-k-mooney | you shoudl have a section like this | |
| 12:16:19 | sean-k-mooney | <guest> | |
| 12:16:20 | sean-k-mooney | <os_type>hvm</os_type> | |
| 12:16:22 | sean-k-mooney | <arch name='x86_64'> | |
| 12:16:24 | sean-k-mooney | <wordsize>64</wordsize> | |
| 12:16:26 | sean-k-mooney | <emulator>/usr/local/bin/qemu-system-x86_64</emulator> | |
| 12:16:28 | sean-k-mooney | <machine maxCpus='255'>pc-i440fx-4.1</machine> | |
| 12:16:30 | sean-k-mooney | <machine canonical='pc-i440fx-4.1' maxCpus='255'>pc</machine> | |
| 12:16:34 | sean-k-mooney | in my case it extends quite far | |
| 12:17:09 | admin0 | i will try to remove the libvirt-bin package and reinstall it | |
| 12:17:19 | sean-k-mooney | its not libvirt that is the issue | |
| 12:17:31 | sean-k-mooney | it cant find the x86 qemu bindary | |
| 12:17:36 | stephenfin | sean-k-mooney: Any decisions on what we want to do about https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L299-L301 ? | |
| 12:17:42 | stephenfin | That function doesn't exist | |
| 12:17:44 | sean-k-mooney | so you need to reinstall qemu | |
| 12:18:28 | sean-k-mooney | set_vif_host_backedn_brdige_config? | |
| 12:18:34 | sean-k-mooney | stephenfin: what is the context? | |
| 12:18:43 | stephenfin | yes | |
| 12:18:53 | stephenfin | it hasn't existed since commit 92323586b5d03b31c18657ad64646a8ce3b8a742 | |