| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-17 | |||
| 11:54:57 | yoctozepto | +1, I know the drill | |
| 11:54:59 | admin0 | in virsh capabilities i do see <arch>x86_64</arch> reported with the flags | |
| 11:55:05 | yoctozepto | some issues popping up only after reboot | |
| 11:55:20 | yoctozepto | as in "cleanup gone wrong and discovered weeks later" | |
| 11:55:40 | yoctozepto | odd | |
| 11:56:00 | yoctozepto | uhm | |
| 11:56:10 | yoctozepto | is kvm loaded by any chance? | |
| 11:56:18 | yoctozepto | both main and cpu-specific? | |
| 11:56:22 | yoctozepto | lsmod | grep kvm | |
| 11:56:53 | yoctozepto | admin0: ^ | |
| 11:56:55 | admin0 | but this is a simulated "oops something bad happened" .. i rebooted the box , rebooted the vms multiple times before nuking the controllers and restart to make sure no skeletons in the closet pop up | |
| 11:57:09 | admin0 | i see kvm_intel, and kvm and irqbypass | |
| 11:57:15 | yoctozepto | hmm, odd | |
| 11:57:27 | yoctozepto | and what about | |
| 11:57:29 | yoctozepto | dmesg | grep kvm | |
| 11:57:37 | admin0 | null on that | |
| 11:57:54 | yoctozepto | then fine | |
| 11:58:05 | sean-k-mooney | admin0: this behavior is not related to the host reboot or the contoler unless you are useing shard stroageve form the contolers on the compute nodes | |
| 11:58:05 | admin0 | modprobe kvm && dmesg | grep kvm = also nothing | |
| 11:58:06 | yoctozepto | everything's all right but it does not work | |
| 11:58:19 | yoctozepto | nah, that should be nothing | |
| 11:58:20 | sean-k-mooney | e.g. you put your instnace data dir on an nfs share | |
| 11:58:25 | admin0 | this has no ceph and no cinder .. its local storage | |
| 11:58:34 | admin0 | i have the disks file locally | |
| 11:58:37 | yoctozepto | yeah, the error is that x86_64 missing | |
| 11:58:38 | yoctozepto | somehow | |
| 11:58:51 | yoctozepto | what distro is this anyways? | |
| 11:59:02 | yoctozepto | (right time to ask that heh) | |
| 11:59:04 | admin0 | ubuntu 16.04 | |
| 11:59:06 | sean-k-mooney | admin0: i think you need to set up your experiment again clenly | |
| 11:59:11 | yoctozepto | quite an oldie there | |
| 11:59:46 | admin0 | ok.. then for the purpose of this, how do do virsh capabilities and edit hte nova file manually so that it just boots up, gets the IP and rsync, recovery is possible | |
| 11:59:47 | sean-k-mooney | the behavior you are seeing is not consitent with how nova operates | |
| 12:00:02 | admin0 | i undestand it may not boot in a very performant or optimal state, but the scope is recovery here | |
| 12:00:20 | sean-k-mooney | normally you would just do virsh start on the domains | |
| 12:00:25 | sean-k-mooney | no need to edit them | |
| 12:00:33 | admin0 | same would be if the virsh/os crashed and i manually had to copy the disk and xml to another (non-similar) compute node | |
| 12:00:41 | yoctozepto | admin0: is qemu-kvm installed? | |
| 12:01:01 | admin0 | qemu-kvm is already the newest version (1:2.10+dfsg-0ubuntu3.8~cloud1). | |
| 12:01:33 | admin0 | <type arch='x86_64' machine='pc-i440fx-artful'>hvm</type> | |
| 12:01:37 | yoctozepto | then I'm out of cards with this issue, really what comes to mind is some bug in ubuntu's libvirt that you have discovered | |
| 12:01:41 | admin0 | i guess that pc-* is the issue ? | |
| 12:01:56 | yoctozepto | well, you could try modifying that | |
| 12:02:31 | yoctozepto | pc-i440fx-xenial should be there | |
| 12:02:42 | yoctozepto | (at least that's what common sense suggests) | |
| 12:02:56 | admin0 | i see https://gist.github.com/a1git/f3696c126a732cda88757c09263bc2de in virsh capabilites related to x86_64 | |
| 12:03:16 | yoctozepto | nothing suspicious | |
| 12:03:37 | sean-k-mooney | the machine type is normal | |
| 12:03:45 | gibi | efried: I got pulled downstream this morning so my review on the provider config progressing slowly. sorry | |
| 12:03:53 | sean-k-mooney | it will now show up in the cpu feature list | |
| 12:04:00 | sean-k-mooney | *not | |
| 12:04:15 | sean-k-mooney | its basicaly the "motherboard" for lack of a better name the vm uses | |
| 12:09:34 | admin0 | i edited pc-i440fx-artful to pc-i440fx-xenial .. same error as invalid argument: could not find capabilities for arch=x86_64 . i have the virsh capabilities in full , lsmod and the vm definition here => https://gist.github.com/a1git/4528c53090f8934b9218586817c51911 | |
| 12:10:42 | sean-k-mooney | admin0: its not related to the model | |
| 12:11:10 | admin0 | ok .. but based on the output, there is no way this instance can boot up ? | |
| 12:11:12 | sean-k-mooney | admin0: you could set it to "pc" which is an aliase for the latest model supproted that is no q35 | |
| 12:11:55 | sean-k-mooney | admin0: the machine types are not listed in that output | |
| 12:12:15 | admin0 | how do i list them ? | |
| 12:13:08 | sean-k-mooney | its likely because the x68 emulator is also not listed | |
| 12:16:15 | sean-k-mooney | you shoudl have a section like this | |
| 12:16:19 | sean-k-mooney | <guest> | |
| 12:16:20 | sean-k-mooney | <os_type>hvm</os_type> | |
| 12:16:22 | sean-k-mooney | <arch name='x86_64'> | |
| 12:16:24 | sean-k-mooney | <wordsize>64</wordsize> | |
| 12:16:26 | sean-k-mooney | <emulator>/usr/local/bin/qemu-system-x86_64</emulator> | |
| 12:16:28 | sean-k-mooney | <machine maxCpus='255'>pc-i440fx-4.1</machine> | |
| 12:16:30 | sean-k-mooney | <machine canonical='pc-i440fx-4.1' maxCpus='255'>pc</machine> | |
| 12:16:34 | sean-k-mooney | in my case it extends quite far | |
| 12:17:09 | admin0 | i will try to remove the libvirt-bin package and reinstall it | |
| 12:17:19 | sean-k-mooney | its not libvirt that is the issue | |
| 12:17:31 | sean-k-mooney | it cant find the x86 qemu bindary | |
| 12:17:36 | stephenfin | sean-k-mooney: Any decisions on what we want to do about https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L299-L301 ? | |
| 12:17:42 | stephenfin | That function doesn't exist | |
| 12:17:44 | sean-k-mooney | so you need to reinstall qemu | |
| 12:18:28 | sean-k-mooney | set_vif_host_backedn_brdige_config? | |
| 12:18:34 | sean-k-mooney | stephenfin: what is the context? | |
| 12:18:43 | stephenfin | yes | |
| 12:18:53 | stephenfin | it hasn't existed since commit 92323586b5d03b31c18657ad64646a8ce3b8a742 | |
| 12:19:10 | stephenfin | ||
| 12:19:11 | stephenfin | <stephenfin> sean-k-mooney: but that function has existed since it was removed in commit 92323586b5d03b31c18657ad64646a8ce3b8a742 | |
| 12:19:11 | stephenfin | <stephenfin> sean-k-mooney: so that means that code is not being called by anything yet. In either case, want to work on the fix or will I? | |
| 12:19:11 | stephenfin | <stephenfin> sean-k-mooney: fwiw, I'm running pytype over the code base to see what it reveals. That's one of two actual issues it has found so far | |
| 12:19:45 | sean-k-mooney | oh you pinged me earlier i did not see it | |
| 12:20:20 | jawad_axd | Hi all! Question is: I want to move couple of vms from one openstack evn(with local storage) to another openstack env(with ceph backend). Both vms have 40G root disk and 100G ephemeral disk.I can not take snapshot of instances since they are using ephemeral disks. How can I take backup of these vms with ephemeral disks and boot those two instance in new environment(with ceph backend)? | |
| 12:20:53 | stephenfin | The easy fix would be to re-add the function, but if no one has hit this yet (it's been there since 18.0.0) perhaps we can remove the call | |
| 12:20:58 | sean-k-mooney | stephenfin: the only place that is called is here https://github.com/openstack/nova/blob/b44b540fc70504f3869ef23022642095de0ea99e/nova/virt/libvirt/vif.py#L611-L612 | |
| 12:20:59 | stephenfin | I call look into it if you don't have time | |
| 12:21:10 | sean-k-mooney | so it the non os-vif code path for linux bridge | |
| 12:21:10 | stephenfin | yup | |
| 12:21:22 | sean-k-mooney | i think that was there just for nova-net? | |
| 12:21:26 | admin0 | sean-k-mooney, i redid libvirt-bin , restarted libvirtd and now i see it | |
| 12:21:27 | admin0 | under hvm | |
| 12:21:36 | admin0 | trying to define the instance now | |
| 12:22:02 | admin0 | defined :) | |
| 12:22:28 | sean-k-mooney | stephenfin: i think we can remvoe that branch | |
| 12:22:32 | stephenfin | sean-k-mooney: aha, yes https://github.com/openstack/nova/blob/b44b540fc70504f3869ef23022642095de0ea99e/nova/network/os_vif_util.py#L527-L528 | |
| 12:22:35 | stephenfin | we sure can | |
| 12:22:39 | stephenfin | I'll go kill it | |
| 12:22:43 | stephenfin | thanks, sean-k-mooney | |
| 12:22:48 | sean-k-mooney | stephenfin: anything that would be taking it would be broken at the moment anyway | |