| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-20 | |||
| 09:23:10 | sean-k-mooney | adding the mock to the class was a way of sutbing out the field without needed to add a patcher and cleanup function in __init__ | |
| 09:23:39 | aspiers | I found a way to apply a mock to all tests in a class, but efried hated it (and he was right, it's fugly) | |
| 09:23:55 | aspiers | https://review.opendev.org/#/c/666616/23/nova/tests/unit/virt/libvirt/test_driver.py@23555 | |
| 09:24:07 | aspiers | but it does work at least | |
| 09:24:36 | aspiers | you call __enter__() during setUp() and __exit__() during tearDown() | |
| 09:24:54 | aspiers | brb | |
| 09:25:36 | sean-k-mooney | aspiers: there are cleaner ways | |
| 09:26:48 | sean-k-mooney | ill test it locally with my change and then ill comment on yours | |
| 09:41:40 | sean-k-mooney | well that is annoying my original code passes locally so i guess i need top use the gate to test this. | |
| 09:43:15 | openstackgerrit | sean mooney proposed openstack/nova master: Libvirt: report storage bus traits https://review.opendev.org/666914 | |
| 10:16:55 | yaawang | stephenfin: Hi, can you take a look at the patch: https://review.opendev.org/#/c/670298, it's a part of spec vCPU model selection. :) | |
| 10:30:24 | cervigni | hello guys, I am trying to have the VGPU feature working on rhel7 but I always hit this issue | |
| 10:30:30 | cervigni | 2019-08-20 18:28:10.527 17202 ERROR nova.virt.libvirt.driver [req-031bc6f7-be61-406b-aefd-020a24e76f5e d7ef7ad4b0424f78a5010503b095ae84 35d0e97603b04dadaee6dfafbd19e5f9 - default default] [instance: 9300b931-12e5-421a-a8ae-ef1854d4cb38] Failed to start libvirt guest: libvirtError: internal error: qemu unexpectedly closed the monitor: 2019-08-20T10:28:10.301572Z qemu-kvm: -device vfio-pci,id=hostdev0,sysfsdev=/sys/bus/mdev/devices/a0543cde-3934- | |
| 10:30:30 | cervigni | 460f-8994-6be8c80fe417,display=off,bus=pci.0,addr=0x6: vfio error: a0543cde-3934-460f-8994-6be8c80fe417: error getting device from group 81: Input/output error | |
| 10:31:11 | cervigni | what can this be related to? I intalled the vGPU drivers, flavor is correct, vgpu is set to nvidia-106 | |
| 10:31:15 | sean-k-mooney | have you enabled the iommu on the host | |
| 10:31:42 | cervigni | GRUB_CMDLINE_LINUX="crashkernel=auto spectre_v2=retpoline rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet intel_iommu=on iommu=pt modprobe.blacklist=nouveau" | |
| 10:31:43 | cervigni | yes | |
| 10:32:18 | cervigni | is there anything else I should enable/disable? | |
| 10:32:26 | sean-k-mooney | is there a traceback in the libvirt instacne log | |
| 10:32:52 | cervigni | yes | |
| 10:32:53 | cervigni | 2019-08-20T10:28:10.301572Z qemu-kvm: -device vfio-pci,id=hostdev0,sysfsdev=/sys/bus/mdev/devices/a0543cde-3934-460f-8994-6be8c80fe417,display=off,bus=pci.0,addr=0x6: vfio error: a0543cde-3934-460f-8994-6be8c80fe417: error getting device from group 81: Input/output error | |
| 10:32:53 | cervigni | Verify all devices in group 81 are bound to vfio-<bus> or pci-stub and not already in use | |
| 10:32:53 | cervigni | 2019-08-20 10:28:10.318+0000: shutting down, reason=failed | |
| 10:33:10 | sean-k-mooney | ah i know what happening then | |
| 10:33:32 | cervigni | really? | |
| 10:33:42 | sean-k-mooney | for sriov or vgpus to work all devices in the same iommu group need to use the same driver | |
| 10:33:58 | cervigni | I had exactly the same issue with ubuntu. I moved to rhel7 because I thought that was the issue | |
| 10:34:14 | sean-k-mooney | so you need to due a few things first you need to check what device are part of iommu group 81 | |
| 10:34:30 | sean-k-mooney | and ensure that tey are all bound to vfio-pci | |
| 10:34:52 | sean-k-mooney | for example but the gpu and the gpus onboard audio contoller | |
| 10:35:11 | sean-k-mooney | they are two different pcie endpoint that share the same iommu group in most cases | |
| 10:35:35 | aspiers | stephenfin: why does test-requirements.txt have psycopg2 at all? is postgres still supported? | |
| 10:36:02 | cervigni | IOMMU Group 81: | |
| 10:36:02 | cervigni | lspci: -s: Invalid slot number | |
| 10:36:02 | sean-k-mooney | if you intalled the card in a pci slot connected to the chipset via the dmi bridge rahter then then directly to the cpu that could also cause the same issue were you are sharing an iommu groupe between multiple devices | |
| 10:36:13 | cervigni | uhm that is definitely a wrong group | |
| 10:36:15 | sean-k-mooney | that is not how you check | |
| 10:36:16 | stephenfin | aspiers: I recall cfriesen mentioning something about them caring about postgres in the distant pass, but I honestly can't say any more than that | |
| 10:36:20 | stephenfin | yaawang: Sure thing | |
| 10:36:40 | aspiers | stephenfin: about who caring? | |
| 10:36:51 | stephenfin | Windriver | |
| 10:36:54 | aspiers | oh | |
| 10:37:04 | aspiers | SUSE used to use pg for OpenStack, but ditched it quite a while ago | |
| 10:37:06 | stephenfin | *Wind River? | |
| 10:37:22 | stephenfin | Yeah, I've no idea if it even works nowadays | |
| 10:37:24 | sean-k-mooney | cervigni: give me a sec and ill see if i can get the command | |
| 10:37:26 | aspiers | Yeah X-D Windriver makes me think of DLLs *shudder* | |
| 10:38:00 | aspiers | stephenfin: seems dumb that I have to install pg_config to run unit tests | |
| 10:38:13 | stephenfin | Or build docs | |
| 10:38:23 | stephenfin | or do anything, really | |
| 10:38:35 | cervigni | thanks | |
| 10:38:37 | stephenfin | tbh, we shouldn't even need the MySQL dependencies for that | |
| 10:39:09 | aspiers | true | |
| 10:39:36 | sean-k-mooney | cervigni: this is not the exact command but more or less does what you want | |
| 10:40:59 | sean-k-mooney | http://paste.openstack.org/show/760302/ | |
| 10:41:18 | sean-k-mooney | you will want to pipe that to less and check iommu group 81 | |
| 10:42:05 | cervigni | that is the same of what I pasted earlier, for IOMMU group 81 (where the problem comes from) | |
| 10:42:10 | sean-k-mooney | you can also just do ls /sys/kernel/iommu_groups/81/devices/ | |
| 10:42:21 | cervigni | it reports only a lspci: -s: Invalid slot number | |
| 10:42:38 | sean-k-mooney | yes -s take a pci address not iommu group number | |
| 10:42:45 | sean-k-mooney | they are totally different things | |
| 10:42:51 | cervigni | ok | |
| 10:43:16 | sean-k-mooney | what do you get for "ls /sys/kernel/iommu_groups/81/devices/" | |
| 10:43:35 | cervigni | [root@compute-r6-h28 ~]# ls /sys/kernel/iommu_groups/81/devices/a0543cde-3934-460f-8994-6be8c80fe417/ | |
| 10:43:35 | cervigni | driver iommu_group mdev_type nvidia power remove subsystem uevent | |
| 10:56:40 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Note use of 'nova-manage db sync --config-file' https://review.opendev.org/671298 | |
| 10:56:40 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Document global options for nova-manage https://review.opendev.org/677443 | |
| 10:56:41 | openstackgerrit | Stephen Finucane proposed openstack/nova master: config: Explicitly register 'remote_debug' CLI opts https://review.opendev.org/677444 | |
| 11:34:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move calls to ovs-vsctl to privsep. https://review.opendev.org/639282 | |
| 11:34:51 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move setting of device trust to privsep. https://review.opendev.org/639283 | |
| 11:34:51 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move final bridge commands to privsep. https://review.opendev.org/639580 | |
| 11:34:52 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Cleanup the _execute shim in nova/network. https://review.opendev.org/639581 | |
| 11:34:52 | openstackgerrit | Stephen Finucane proposed openstack/nova master: We no longer need rootwrap. https://review.opendev.org/554438 | |
| 11:34:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Cleanup no longer required filters and add a release note. https://review.opendev.org/639826 | |
| 11:34:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Address nits from privsep series https://review.opendev.org/677210 | |
| 11:34:54 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP! Stop passing 'root_helper' argument to os-brick https://review.opendev.org/677451 | |
| 12:43:26 | openstackgerrit | Adam Spiers proposed openstack/nova master: Track libvirt host/domain capabilities for multiple machine types https://review.opendev.org/673151 | |
| 12:46:36 | openstackgerrit | Adam Spiers proposed openstack/nova master: Track libvirt host/domain capabilities for multiple machine types https://review.opendev.org/673151 | |
| 12:50:41 | openstackgerrit | Adam Spiers proposed openstack/nova master: Extract new base class for provider usage functional tests https://review.opendev.org/676964 | |
| 12:50:41 | openstackgerrit | Adam Spiers proposed openstack/nova master: Provide HW_CPU_X86_AMD_SEV trait when SEV is supported https://review.opendev.org/638680 | |
| 12:51:33 | openstackgerrit | Alexandra Settle proposed openstack/nova stable/rocky: Fixing broken links https://review.opendev.org/677460 | |
| 12:53:31 | alex_xu | aspiers: the line 697-701 of https://review.opendev.org/#/c/673151/11/nova/virt/libvirt/host.py is about the why we need that set of machine type? | |
| 12:54:10 | aspiers | alex_xu: We need to cover the main variants | |
| 12:54:39 | alex_xu | aspiers: ok, so the pc and q35 can cover all the cases, right? | |
| 12:54:43 | aspiers | yes | |
| 12:55:03 | aspiers | this is mentioned in the commit message | |
| 12:55:06 | alex_xu | aspiers: thanks, i see now | |
| 12:55:15 | alex_xu | aspiers: is it too arch specific? | |
| 12:55:31 | alex_xu | although I didn't see other arch need this method | |
| 12:55:32 | aspiers | pc / q35 are not hardcoded | |
| 12:56:18 | aspiers | alex_xu: they come from utils.get_default_machine_type() and libvirt's getCapabilities | |
| 12:56:28 | alex_xu | I mean in other arch, it is also about default and canonical covering all the cases | |
| 12:56:30 | aspiers | so other architectures will get a different set | |
| 12:56:36 | aspiers | yes, correct | |
| 12:57:03 | alex_xu | ok, I'm just going to trust the libvirt and platform experts | |
| 12:57:34 | aspiers | that's why I had to remove 'bamboo' from the fixture | |
| 12:58:15 | sean-k-mooney | bamboo as a fake arch or machine type :) | |
| 12:58:27 | alex_xu | hah | |
| 12:58:29 | aspiers | because the static fixtures for getDomainCapabilities on ppc did not return correct values for bamboo | |