| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-17 | |||
| 11:40:24 | yoctozepto | could boot any vm in there | |
| 11:40:28 | yoctozepto | on that compute host | |
| 11:40:36 | yoctozepto | if you really want to show it is there for some reason | |
| 11:40:39 | sean-k-mooney | so when the contler are gone you can still use virsh to start the instaces | |
| 11:40:52 | admin0 | the only way i see is install virt-manager on my machne, add the compute node, and then create a new instance pointing to this disk | |
| 11:40:53 | sean-k-mooney | by the way you proably need to use sudo virsh list --all | |
| 11:41:09 | yoctozepto | ^ and this | |
| 11:41:12 | sean-k-mooney | if you dont use sudo you wont see the system slice | |
| 11:41:33 | yoctozepto | sean-k-mooney: admin0 mentioned downing the compute | |
| 11:41:36 | admin0 | i am logged in as root, did sudo virsh list --all .. see nothing | |
| 11:41:43 | yoctozepto | so it's likely it's reallly not there | |
| 11:41:46 | admin0 | yes .. the compute node was also rebooted | |
| 11:41:50 | yoctozepto | yeah | |
| 11:41:59 | yoctozepto | then either restore controllers | |
| 11:42:07 | sean-k-mooney | ok in that case have you confirmed the libvirtd systemd service is running | |
| 11:42:19 | admin0 | so test case is "oops something bad happened" . controllers are nuked, db is gone and compute nodes are rebooted | |
| 11:42:30 | sean-k-mooney | no the domains will be there | |
| 11:42:38 | sean-k-mooney | it wont be affect by the reboot | |
| 11:42:39 | admin0 | which folder | |
| 11:43:06 | sean-k-mooney | ill have to check but first you should make sure the libvirt is running | |
| 11:43:22 | sean-k-mooney | do sudo systemctl status libvirtd | |
| 11:43:47 | admin0 | its active and running | |
| 11:44:08 | sean-k-mooney | and you have no domain definitions? | |
| 11:44:24 | sean-k-mooney | did you put your instnace dir on nfs? | |
| 11:44:43 | yoctozepto | sean-k-mooney is right, I just verified instance files are really persistent | |
| 11:44:47 | yoctozepto | sorry, I got confused | |
| 11:44:56 | admin0 | nope -- https://gist.github.com/a1git/d5d6c0cbb1e89ce6f7a7c5476b43f472 | |
| 11:45:04 | yoctozepto | my path is: /etc/libvirt/qemu/ | |
| 11:45:05 | admin0 | its local disk | |
| 11:45:08 | admin0 | oh | |
| 11:45:13 | admin0 | i was lookign into the wrong place | |
| 11:45:26 | admin0 | i see the instance definitions | |
| 11:45:36 | admin0 | but i cannot see them using sudo virsh list --all then ? | |
| 11:45:43 | yoctozepto | hmm, strange | |
| 11:45:58 | sean-k-mooney | how have you deployed openstack by the way | |
| 11:46:14 | sean-k-mooney | devstack? ooo? kolla? | |
| 11:47:00 | yoctozepto | (I have kolla and sean-k-mooney was right regarding behavior in there) | |
| 11:47:04 | admin0 | osa ( openstack ansible ) | |
| 11:47:12 | yoctozepto | ah, the third option | |
| 11:47:18 | admin0 | :D | |
| 11:47:20 | yoctozepto | or the fourth actually ;D | |
| 11:47:25 | yoctozepto | (counting devstack) | |
| 11:47:31 | admin0 | well, hav been trying to do kolla many times .. but still on osa | |
| 11:47:40 | yoctozepto | no problem | |
| 11:48:08 | sean-k-mooney | ok so you libvirt is running in an lxc container or on the host? | |
| 11:48:24 | admin0 | thing is if i ask some hard questions on kolla related to config or troubleshooting, there is no help on kolla but on osa its more in depth and instant .. which probably is because of rackspace | |
| 11:48:29 | admin0 | its on metal | |
| 11:48:56 | sean-k-mooney | ok can you try virsh --connect qemu:///system list | |
| 11:48:58 | yoctozepto | admin0: :O no help on kolla? how is that? | |
| 11:49:30 | sean-k-mooney | well i guess with --all | |
| 11:50:24 | sean-k-mooney | the xmls should be in /etc/libvirt/qemu | |
| 11:50:27 | yoctozepto | admin0: finally try: virsh define /etc/libvirt/qemu/that_one_file_you_missed_so_much.xml - maybe there was an error in there | |
| 11:51:09 | admin0 | error: invalid argument: could not find capabilities for arch=x86_64 -- wow .same server .. | |
| 11:51:44 | yoctozepto | see, now you got help from kolla, your argument is no longer valid (pun intended) | |
| 11:51:55 | yoctozepto | ;D | |
| 11:52:13 | yoctozepto | kidding aside, this really looks lolz | |
| 11:52:45 | sean-k-mooney | it looks like libvirt cannot find the x86_64 emulator | |
| 11:52:57 | admin0 | i will try kolla again .. won't doubt that .. but this is #nova channel :D | |
| 11:53:01 | yoctozepto | mhm, at least its config/definitions | |
| 11:53:22 | yoctozepto | admin0: yeah, I'm just teasing | |
| 11:53:36 | yoctozepto | though reboot does not get rid of those | |
| 11:53:38 | sean-k-mooney | well its complaining that virsh capabilities is not working | |
| 11:54:11 | sean-k-mooney | no a reboot would not cause this and nither would nuking the contolers | |
| 11:54:26 | sean-k-mooney | this host has been broken by something else that is unrelated | |
| 11:54:57 | yoctozepto | +1, I know the drill | |
| 11:54:59 | admin0 | in virsh capabilities i do see <arch>x86_64</arch> reported with the flags | |
| 11:55:05 | yoctozepto | some issues popping up only after reboot | |
| 11:55:20 | yoctozepto | as in "cleanup gone wrong and discovered weeks later" | |
| 11:55:40 | yoctozepto | odd | |
| 11:56:00 | yoctozepto | uhm | |
| 11:56:10 | yoctozepto | is kvm loaded by any chance? | |
| 11:56:18 | yoctozepto | both main and cpu-specific? | |
| 11:56:22 | yoctozepto | lsmod | grep kvm | |
| 11:56:53 | yoctozepto | admin0: ^ | |
| 11:56:55 | admin0 | but this is a simulated "oops something bad happened" .. i rebooted the box , rebooted the vms multiple times before nuking the controllers and restart to make sure no skeletons in the closet pop up | |
| 11:57:09 | admin0 | i see kvm_intel, and kvm and irqbypass | |
| 11:57:15 | yoctozepto | hmm, odd | |
| 11:57:27 | yoctozepto | and what about | |
| 11:57:29 | yoctozepto | dmesg | grep kvm | |
| 11:57:37 | admin0 | null on that | |
| 11:57:54 | yoctozepto | then fine | |
| 11:58:05 | admin0 | modprobe kvm && dmesg | grep kvm = also nothing | |
| 11:58:05 | sean-k-mooney | admin0: this behavior is not related to the host reboot or the contoler unless you are useing shard stroageve form the contolers on the compute nodes | |
| 11:58:06 | yoctozepto | everything's all right but it does not work | |
| 11:58:19 | yoctozepto | nah, that should be nothing | |
| 11:58:20 | sean-k-mooney | e.g. you put your instnace data dir on an nfs share | |
| 11:58:25 | admin0 | this has no ceph and no cinder .. its local storage | |
| 11:58:34 | admin0 | i have the disks file locally | |
| 11:58:37 | yoctozepto | yeah, the error is that x86_64 missing | |
| 11:58:38 | yoctozepto | somehow | |
| 11:58:51 | yoctozepto | what distro is this anyways? | |
| 11:59:02 | yoctozepto | (right time to ask that heh) | |
| 11:59:04 | admin0 | ubuntu 16.04 | |
| 11:59:06 | sean-k-mooney | admin0: i think you need to set up your experiment again clenly | |
| 11:59:11 | yoctozepto | quite an oldie there | |
| 11:59:46 | admin0 | ok.. then for the purpose of this, how do do virsh capabilities and edit hte nova file manually so that it just boots up, gets the IP and rsync, recovery is possible | |
| 11:59:47 | sean-k-mooney | the behavior you are seeing is not consitent with how nova operates | |
| 12:00:02 | admin0 | i undestand it may not boot in a very performant or optimal state, but the scope is recovery here | |
| 12:00:20 | sean-k-mooney | normally you would just do virsh start on the domains | |
| 12:00:25 | sean-k-mooney | no need to edit them | |