| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-02 | |||
| 16:08:40 | sean-k-mooney | that is possibly true yes in this specifc instance othe Runtime error that was raise i think its safe but you are concerend that there coudl be other case where it would not be | |
| 16:09:21 | dansmith | I'm saying they used to fail if anything was written to stderr and now they won't | |
| 16:09:26 | sean-k-mooney | i woudl hope openssl would not exit with code 0 for anything other then sucess but i dont know that for certine even if i belive it very likely | |
| 16:09:33 | dansmith | and depending on what is going on here, that could be, you know, a big deal | |
| 16:09:35 | sean-k-mooney | yes | |
| 16:09:42 | dansmith | depends on the command and what is going on | |
| 16:10:07 | sean-k-mooney | well we can see the command it fixed | |
| 16:10:30 | dansmith | I'm saying someone needs to go make that determination, IMHO and not just blindly approve this | |
| 16:10:38 | sean-k-mooney | we are encrypting input text with a shared key using ase-123-cbc | |
| 16:11:12 | sean-k-mooney | fair im still wondering why we are doing this via the shell in the first place | |
| 16:22:22 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Fix functional tests for USE_NEUTRON https://review.openstack.org/649385 | |
| 16:22:22 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add functional regression test for bug 1669054 https://review.openstack.org/649386 | |
| 16:22:23 | openstack | bug 1669054 in OpenStack Compute (nova) stein "RequestSpec.ignore_hosts from resize is reused in subsequent evacuate" [Medium,In progress] https://launchpad.net/bugs/1669054 - Assigned to Matt Riedemann (mriedem) | |
| 16:22:23 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Do not persist RequestSpec.ignore_hosts https://review.openstack.org/649387 | |
| 16:23:41 | openstackgerrit | Merged openstack/nova master: Correct lower-constraints.txt and the related tox job https://review.openstack.org/622972 | |
| 16:23:50 | openstackgerrit | Merged openstack/nova master: Adding tests to demonstrate bug #1821824 https://review.openstack.org/647957 | |
| 16:23:51 | openstack | bug 1821824 in OpenStack Compute (nova) "Forbidden traits in flavor properties don't work" [Undecided,In progress] https://launchpad.net/bugs/1821824 - Assigned to Magnus Bergman (magnusbe) | |
| 16:24:02 | dansmith | looks like dimitri did a fairly decent analysis of the openssl code, which is good, but I also wonder if we couldn't just ignore lines that start with "WARNING" or something from the stderr and retain some of the original behavor | |
| 16:24:11 | dansmith | anyway, I don't really have time to dig into it super deep | |
| 16:25:42 | openstackgerrit | Merged openstack/nova master: Add placement as required project to functional py36 and 37 https://review.openstack.org/649068 | |
| 16:34:22 | openstackgerrit | Merged openstack/nova master: libvirt: Use 'writeback' QEMU cache mode when 'none' is not viable https://review.openstack.org/641981 | |
| 16:35:19 | cfriesen | sean-k-mooney: apparently nova-compute chokes on startup with this particular QAT hardware even with no whitelist/alias entries in nova.conf. | |
| 16:41:20 | sean-k-mooney | cfriesen: ok then this is likely related to gibi_off's change to auto lookup the netdev name for bandwith based scheduling | |
| 16:42:42 | cfriesen | sean-k-mooney: yeah, confirmed that we're dying in the code gibi_off added in Dec 2018. | |
| 16:44:34 | cfriesen | sean-k-mooney: we're hitting this with the standard embedded Intel QAT, so it's going to cause grief with standard hardware | |
| 16:45:13 | sean-k-mooney | its proably this bit https://github.com/openstack/nova/commit/c02e213d507c830427a86d6a4bb4f7a2f5158590#diff-f4019782d93a196a0d026479e6aa61b1R5938 | |
| 16:45:58 | cfriesen | sean-k-mooney: the issue is that there is no "net" in the device path (i.e. /sys/bus/pci/devices/<pci_addr>/net) | |
| 16:46:23 | sean-k-mooney | ya | |
| 16:46:26 | sean-k-mooney | so https://github.com/openstack/nova/blob/c02e213d507c830427a86d6a4bb4f7a2f5158590/nova/virt/libvirt/driver.py#L5938-L5940 | |
| 16:46:41 | sean-k-mooney | should only be executed for VF that are network devices | |
| 16:47:22 | sean-k-mooney | that shoudl be a simple fix | |
| 16:47:57 | sean-k-mooney | but we will need to land it in RC2 or backport to stien before thurday to include it in the GA release | |
| 16:49:39 | sean-k-mooney | cfriesen: we are expecting qat to hit the final return however | |
| 16:49:46 | sean-k-mooney | so there is something else going on | |
| 16:50:17 | cfriesen | sean-k-mooney: there are VFs for this device, so I was assuming we're enumerating the VFs | |
| 16:50:46 | sean-k-mooney | yay be we are only ment to report the it as type SRIOV_VF if its a nic | |
| 16:51:03 | sean-k-mooney | all non nic VF are ment to be TYPE_PCI | |
| 16:51:38 | cfriesen | sean-k-mooney: where is that code? | |
| 16:53:32 | sean-k-mooney | im looking for it now but its the only thing that prevented you geting a qat device instead fo a nic VF when you ahave a neturon prot of vnic_type direct in the past | |
| 17:01:09 | cfriesen | sean-k-mooney: it kind of looks like _get_pcidev_info() is calling self._host.device_lookup_by_name() to get the XML for the device. Is it possible libvirt is doing something different? | |
| 17:03:27 | openstackgerrit | Artem Vasilyev proposed openstack/nova master: systemd detection result caching nit fixes https://review.openstack.org/649229 | |
| 17:03:48 | sean-k-mooney | is that where you are having the failure? | |
| 17:04:19 | sean-k-mooney | cfriesen: can you post a copy fo the error to paste.openstack.org | |
| 17:04:59 | cfriesen | sean-k-mooney: yeah, nova-compute startup. here's the starlingx bug, the nova stuff is partway down: https://bugs.launchpad.net/starlingx/+bug/1821938 | |
| 17:05:00 | openstack | Launchpad bug 1821938 in StarlingX "No nova hypervisor can be enabled on workers with QAT devices" [High,Triaged] | |
| 17:05:19 | sean-k-mooney | cfriesen: thanks | |
| 17:05:39 | cfriesen | sean-k-mooney: extra info: http://paste.openstack.org/show/748734/ | |
| 17:06:24 | sean-k-mooney | ya so this is not failing because of https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6067 | |
| 17:06:24 | cfriesen | we don't have the resources to fix this in the near future, got other stuff to deal with | |
| 17:06:43 | sean-k-mooney | its failing because of https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6047 | |
| 17:06:50 | sean-k-mooney | because it does not have a netdev | |
| 17:07:14 | cfriesen | sean-k-mooney: correct, but you were wondering why we were going down the VF path for things that werent nics | |
| 17:07:16 | sean-k-mooney | which is because of gibis change | |
| 17:07:34 | sean-k-mooney | ya but the other filtering im thinking of could be else where | |
| 17:07:48 | sean-k-mooney | i think we just need to put a guard around that call | |
| 17:08:00 | cfriesen | sean-k-mooney: looks like _get_device_capabilities() also assumes that SRIOV_VF is a NIC | |
| 17:09:12 | sean-k-mooney | yes it does | |
| 17:09:26 | sean-k-mooney | although it is reading form libvirt | |
| 17:09:34 | sean-k-mooney | instead of sysfs | |
| 17:09:38 | sean-k-mooney | so it proably fine | |
| 17:10:21 | cfriesen | sean-k-mooney: _get_pcinet_info calls get_net_name_by_vf_pci_address() | |
| 17:10:26 | cfriesen | so I think it'll choke | |
| 17:10:44 | sean-k-mooney | ill quickly hack something up one sec | |
| 17:12:18 | sean-k-mooney | cfriesen: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6008-L6010 i think will guard it for that case | |
| 17:12:58 | sean-k-mooney | actully no it wont | |
| 17:14:23 | sean-k-mooney | actully it should be fine | |
| 17:14:24 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/pci/utils.py#L205 | |
| 17:14:31 | sean-k-mooney | the exception is caught internally | |
| 17:14:48 | cfriesen | ah, yes | |
| 17:15:13 | sean-k-mooney | am is that the case in the starlingx code? | |
| 17:15:46 | cfriesen | should be, but we were choking earlier in _get_device_type() | |
| 17:16:19 | sean-k-mooney | oh i see the issue | |
| 17:16:30 | sean-k-mooney | we are doing | |
| 17:16:32 | sean-k-mooney | 'parent_ifname': | |
| 17:16:34 | sean-k-mooney | pci_utils.get_ifname_by_pci_address( | |
| 17:16:36 | sean-k-mooney | pci_address, pf_interface=True), | |
| 17:17:01 | sean-k-mooney | so we are calling pci_utils.get_ifname_by_pci_address directly so we dont catch the excpetion | |
| 17:17:02 | cfriesen | yep | |
| 17:17:13 | sean-k-mooney | where as the other code calls get_net_name_by_vf_pci_address which does | |
| 17:17:59 | sean-k-mooney | ok this is a simile fix ill do it now and make it as closing the bug | |
| 17:18:22 | sean-k-mooney | efried: im going to try and fix https://bugs.launchpad.net/starlingx/+bug/1821938 can we land it in stien? | |
| 17:18:23 | openstack | Launchpad bug 1821938 in StarlingX "No nova hypervisor can be enabled on workers with QAT devices" [High,Triaged] | |
| 17:18:23 | cfriesen | I think one of my coworkers is going to open a nova bug | |
| 17:18:46 | sean-k-mooney | ok ill add the starlingx bug as a related bug so | |
| 17:18:54 | sean-k-mooney | or do ye want to fix it | |
| 17:19:11 | cfriesen | go for it | |
| 17:20:53 | cfriesen | I think my guy is having lunch. :) I'll send you the nova bug when I get the number. | |
| 17:24:31 | stephenfin | mriedem: https://review.openstack.org/626949 | |
| 17:26:05 | openstackgerrit | Jared Winborne proposed openstack/nova master: Leave the brackets on Ceph Monitor IPv6 addresses for libguestfs https://review.openstack.org/649405 | |
| 17:31:19 | KH-Jared | I fully expect my change didn't follow some proper practice on the change I just submitted, guess I just have to wait and find out what that is at this point | |
| 17:37:09 | openstackgerrit | sean mooney proposed openstack/nova master: gracefuly handel none nic VFs https://review.openstack.org/649409 | |
| 17:37:34 | sean-k-mooney | cfriesen: ^ | |
| 17:37:56 | mriedem | KH-Jared: commented | |
| 17:38:00 | sean-k-mooney | i need to run the unit test and see if any fail and alsoadd a new one | |
| 17:39:12 | KH-Jared | ty mriedem | |
| 17:39:17 | sean-k-mooney | mriedem: not sure if you were following but ^ is a fix for https://bugs.launchpad.net/starlingx/+bug/1821938 | |
| 17:39:18 | openstack | Launchpad bug 1821938 in StarlingX "No nova hypervisor can be enabled on workers with QAT devices" [High,Triaged] | |
| 17:39:32 | mriedem | sean-k-mooney: i wasn't | |
| 17:39:45 | sean-k-mooney | think we could land it in stien if i get it ready soon | |
| 17:39:52 | mriedem | idk wtf a qat device is | |