| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-01-08 | |||
| 12:08:05 | gibi | stephenfin: I saw that and I have the same problem, I cannot start fixing those tests as I have no way to reproduce the problem | |
| 12:08:19 | gibi | (no I did not tried spinning up a ppc VM with qemu) | |
| 12:08:47 | stephenfin | Yeah, I've suggested that zigo simply mark the package as arch-specific. If it's not tested, it's broken | |
| 12:08:49 | gibi | stephenfin: did we declare support for ppc somewhere in nova doc? | |
| 12:08:55 | stephenfin | I don't think so | |
| 12:09:04 | stephenfin | We don't block it but https://github.com/openstack/nova/blob/46899968619e4ea0ff2ab380977619bb29578d43/nova/virt/libvirt/driver.py#L572-L581 | |
| 12:10:00 | lyarwood | odd why don't we list aarch64 there? | |
| 12:10:05 | lyarwood | I thought we had jobs now | |
| 12:10:14 | lyarwood | ah third party ignore me | |
| 12:10:23 | stephenfin | someone's working on it but it's not merged yet, no | |
| 12:10:26 | gibi | stephenfin: good point | |
| 12:10:43 | gibi | stephenfin: if we even log a warning then that should be enough to close the bug | |
| 12:12:23 | gibi | let's see what zigo's oppinion about this | |
| 12:19:37 | sean-k-mooney | xinranwang: i should be able to join too | |
| 12:20:05 | zigo | stephenfin: gibi: My opinion is: if you can't test with PPC, and that the unit test is reported as broken on non-x86_64, then skip it on non-x86_64 rather than just closing the bug ... | |
| 12:20:09 | sean-k-mooney | stephenfin: yes i can take a look at that shortly | |
| 12:20:53 | stephenfin | zigo: what about when someone adds the next test that doesn't work on x86_64 | |
| 12:21:13 | sean-k-mooney | gibi: the vf are deleted by the kernel when the PF is passhtough to the vm | |
| 12:21:16 | zigo | stephenfin: These issues are *very* rare. | |
| 12:21:39 | zigo | So we can act on a case-by-case ... | |
| 12:21:57 | gibi | sean-k-mooney: thanks | |
| 12:22:48 | stephenfin | I disagree. You skip tests because they're not applicable. These tests are applicable. They simply don't work on PPC | |
| 12:22:53 | sean-k-mooney | the managed=yes part of <hostdev mode='subsystem' type='pci' managed='yes'> i blelive tells libvirt to manage bining the pf to vfio-pci form its current nic driver and to restore it when the vm is done | |
| 12:22:54 | stephenfin | who knows what else doesn't work | |
| 12:23:13 | sean-k-mooney | by which i mean its stop and the pf returns to the host | |
| 12:23:48 | sean-k-mooney | technically its udev coupled with the kernel config that decide what driver it goes back too | |
| 12:23:49 | zigo | stephenfin: As much as I understand, they would work if s/ide/scsi/ ... | |
| 12:24:42 | stephenfin | Perhaps. I genuinely don't know | |
| 12:24:43 | sean-k-mooney | if you used udev rules to allocate the vfs then that will recreate them | |
| 12:24:46 | stephenfin | which is kind of the point | |
| 12:25:20 | sean-k-mooney | on the ppc front | |
| 12:25:35 | sean-k-mooney | we have support power kvm for years | |
| 12:25:46 | sean-k-mooney | which is libvirt/kvm on ppc | |
| 12:25:54 | sean-k-mooney | ibm has a ci for it | |
| 12:26:06 | sean-k-mooney | that runs against nova | |
| 12:26:10 | stephenfin | sean-k-mooney: clearly they don't, or these tests would be failing there | |
| 12:26:25 | stephenfin | or their CI job matrix isn't complete | |
| 12:26:39 | sean-k-mooney | it only runs integration tests ie tempest | |
| 12:27:28 | sean-k-mooney | https://oplab9.parqtec.unicamp.br/pub/ppc64el/openstack/nova/68/767368/1/check/tempest-dsvm-full-focal-py3/ef10362/ | |
| 12:27:39 | sean-k-mooney | that a run from the IBM PowerKVM CI | |
| 12:27:48 | sean-k-mooney | ppc64el | |
| 12:28:21 | sean-k-mooney | stephenfin: gibi zigo we have no unit or func test coverage | |
| 12:29:01 | sean-k-mooney | but we cant mark the package as not supported on ppc or it will break the ablity to install nova on ppc which has been supported for 4+ years | |
| 12:30:45 | sean-k-mooney | powervm support started in ocata and was finished in queens. powerkvm i think predates powervm which is there own hypervior | |
| 12:34:12 | sean-k-mooney | stephenfin: i think we should reopen https://bugs.launchpad.net/nova/+bug/1909972 | |
| 12:34:13 | openstack | Launchpad bug 1909972 in OpenStack Compute (nova) "a number of tests fail under ppc64el arch" [Low,Won't fix] | |
| 12:35:15 | sean-k-mooney | stephenfin: we cant just declare ppc64el as unsupported when it has been supported for years | |
| 12:35:40 | sean-k-mooney | stephenfin: we even have a mulit arch dfg downstream that is workign to get support for power into osp | |
| 12:38:17 | sean-k-mooney | stephenfin: we event ship ppc version of osp in osp 16 https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html/release_notes/chap-introduction | |
| 12:38:59 | sean-k-mooney | ppc64le to be specific | |
| 12:46:11 | zigo | At least, LibvirtBlockInfoTest.test_get_disk_mapping_stable_rescue_ide_cdrom can be fixed... | |
| 12:46:24 | zigo | LibvirtBlockInfoTest.test_get_disk_mapping_rescue_with_config too. | |
| 12:46:58 | zigo | Test result is {'bus': 'scsi', 'dev': 'sda', 'type': 'cdrom'} while it's expecting {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'} | |
| 12:47:25 | sean-k-mooney | yes | |
| 12:47:47 | lyarwood | yeah that's my bad, smells like I didn't mock out the arch check somewhere | |
| 12:47:49 | gibi | sean-k-mooney: could we ask IBM to include unit and functional test in their CI? then we could have coverage and properly fix the test | |
| 12:48:14 | sean-k-mooney | gibi: we can always ask but not sure if it will happen :) | |
| 12:48:17 | zigo | This is also a problem on arm, FYI. | |
| 12:48:27 | zigo | (the above 2 unit tests failing...) | |
| 12:48:34 | sean-k-mooney | i might reach out to our multi arch folks downstream and see if they can help | |
| 12:49:04 | sean-k-mooney | we do package nova for ppc64el downstream and we have limited downstream ci for it | |
| 12:49:54 | sean-k-mooney | presumably that shoudl included unit and functional test but they may still be priamlary working with 16 which is train or relying on x86 runs | |
| 12:50:00 | sean-k-mooney | for unit and functional | |
| 12:50:50 | sean-k-mooney | zigo: for arm we can test that in the gate | |
| 12:51:07 | sean-k-mooney | zigo: we have limited ablity to run thigns in the arm check pipeline | |
| 12:51:24 | sean-k-mooney | well we can run anything we want in that pipeline just limited capsity | |
| 12:51:34 | sean-k-mooney | running unit and funcitonal there i think would be ok | |
| 12:51:55 | sean-k-mooney | ill ask infra and can submit a patch to do that | |
| 12:52:25 | sean-k-mooney | actully openstack-qa woudl be better | |
| 12:55:51 | zigo | Great ! | |
| 12:56:24 | lyarwood | what was the bug for that again? | |
| 12:57:04 | lyarwood | nvm | |
| 12:57:52 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Mock get_arch during some stable rescue unit tests https://review.opendev.org/c/openstack/nova/+/769916 | |
| 13:00:19 | gibi | sean-k-mooney, stephenfin, zigo: I've reached out to the IBM CI contact person and asked about running unit and functional tests in there system on ppc http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019726.html | |
| 13:00:52 | zigo | Nice ! Thanks. | |
| 13:07:27 | xinranwang | sean-k-mooney: thanks! | |
| 13:09:11 | sean-k-mooney | gibi: ack. ill be afk for 30 mins or so, if they dont respond or cant add them in a week or so ill follow up with our multi arch team and see if there is anything we can do. we normally have issue ruinng upstream ci on our hardware due to it being conencted to our internal network but i could ask if they have any disconnected pool that could be used for public unit/func test runs. im not | |
| 13:09:13 | sean-k-mooney | aware of one but there might be | |
| 13:09:43 | sean-k-mooney | gibi: they might be providing testign for rdo for example on power | |
| 13:10:01 | gibi | sean-k-mooney: ack, thanks | |
| 13:16:50 | openstackgerrit | Lee Yarwood proposed openstack/nova master: tests: Posion nova.objects.fields.Architecture.from_host https://review.opendev.org/c/openstack/nova/+/769920 | |
| 13:17:02 | lyarwood | gibi: ^ nice idea, forgot we could do this. I'm going to let this fail in the gate and then clean things up afterwards. | |
| 13:17:15 | gibi | lyarwood: cool, thanks | |
| 13:36:07 | openstackgerrit | Merged openstack/placement master: Bump oslo.log version to 4.3.0 https://review.opendev.org/c/openstack/placement/+/760229 | |
| 13:52:00 | lyarwood | Failed: 249 with the Architecture.from_host poison in place /o\ | |
| 13:52:13 | lyarwood | I'll clean that up next week | |
| 13:55:48 | gibi | lyarwood: ack | |
| 14:05:42 | gibi | sean-k-mooney: if you are around, smartnic discussion happening on openstack-meeting (neutron ddrivers) | |
| 14:09:25 | stephenfin | artom: When you're about, can you say if you're happy with my response at https://review.opendev.org/c/openstack/nova/+/758155 | |
| 14:10:05 | stephenfin | Also, if you've bandwidth, reviews will be appreciated. I'd love dansmith to look at them but I don't know if he'll have the time/inclination so I need to find people != gibi :) | |
| 14:17:11 | jawad_axd | something with nvidia driver or windows or openstack..Anybody had same problem or any suggestions? Thanks | |
| 14:17:11 | jawad_axd | Hi nova folks, I have done passthrough nvidia k20c with window 10 instance. Installed latest drivers from nvidia site. I can see " This device is working properly" in device manager.Apparently, gpu is not being used for any application. I can see only developer task in nvidia control panel.No 3D settings where I can set to use GPU for windows. Why 3D settings are missing in nvidia control panel? Is it | |
| 14:24:50 | dansmith | stephenfin: oof, yeah that probably needs several sets of eyes. I won't get to it today, but maybe next week | |
| 14:25:45 | stephenfin | Agreed. Any time in the near future is good | |
| 14:36:14 | artom | stephenfin, I suppose if every reviewer runs your tool manually it's as good as CI | |
| 14:36:39 | artom | We'd also need to make sure your tool has no bugs, but then that's the same as with a CI job | |
| 14:53:12 | bauzas | jawad_axd: is the PCI device provided in the instance ? | |
| 14:53:43 | jawad_axd | @bauzas Yes | |
| 14:54:00 | bauzas | so, it's either a windows or a nivdia issue, sorry | |
| 14:54:06 | sean-k-mooney | jawad_axd: i assume if you look in the device manager the device is there with the nvida drivae but error 43 | |