| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-22 | |||
| 19:32:46 | sean-k-mooney | other vms worked fine so im wondering if this is related to load/memory pressure. | |
| 19:41:39 | mriedem | d15ab1ed-0000-0000-0000-00000000000 :) | |
| 19:43:32 | efried | sean-k-mooney: speaking of CI... | |
| 19:44:08 | sean-k-mooney | what did i break now :) | |
| 19:44:15 | efried | I got an extremely vague question from within the blue walls today about what kinds of tests the PCI job ought to be running. | |
| 19:44:25 | efried | I hadn't a clue, but thought you might. | |
| 19:46:07 | sean-k-mooney | ideally pci passthough via alaise in the flavor + neutron sriov interface, then it would be nice to alos test cold migration/resize and possibly live migration in the neutron case | |
| 19:46:18 | efried | someone said something like the PCI job was just setting up devstack and not running any actual tests. I'm not sure how accurate that could be, or if they were looking at the wrong thing, or if at some point we disabled all the actual tests while we were working on the infra, or what. | |
| 19:46:32 | sean-k-mooney | i would just run the standared tempest test with tweeked flavors and different vnic_types | |
| 19:46:58 | sean-k-mooney | efried: it runs test but it boots like 1 vm or very few tests | |
| 19:48:02 | efried | was the PCI job just hitting SR-IOV? | |
| 19:48:17 | efried | Cause there's an SRIOV job that runs against neutron | |
| 19:48:22 | efried | not sure if those are related | |
| 19:49:17 | sean-k-mooney | the pci job was ment for testing QAT integration with flavor bassed passthough | |
| 19:49:29 | sean-k-mooney | there was a seperate one for neutron sriov testing | |
| 19:49:53 | sean-k-mooney | looking at http://52.27.155.124/pci/482200/7/ | |
| 19:50:07 | sean-k-mooney | there are not test in the test repofrt | |
| 19:50:35 | sean-k-mooney | but im not sure that means threare are not test | |
| 19:51:18 | sean-k-mooney | it looks like some test are run http://52.27.155.124/pci/482200/7/console_status.log.gz | |
| 19:52:48 | efried | I can't tell if they pass | |
| 19:54:21 | sean-k-mooney | nor can i | |
| 19:54:46 | sean-k-mooney | looking at the devstack output and the tempest config i cant see tehm creating any custom flavor either | |
| 19:55:55 | efried | how did you know where to find these logs? | |
| 19:56:11 | efried | the CI is running, just not reporting? | |
| 19:56:41 | sean-k-mooney | i clicked on an old log got a 404 and went up one directory | |
| 19:57:13 | efried | how did you know this patch had any results? Did you search for reviewedby:Intel PCI? | |
| 19:57:52 | sean-k-mooney | i looked at one of my patches that i knew it had commented on in the past | |
| 19:58:08 | sean-k-mooney | http://52.27.155.124/ you can see the rest of the logs are here for the other intel cis | |
| 19:58:28 | efried | IC | |
| 19:59:00 | sean-k-mooney | this is a ovh log server that i used ot own/payfor via intel for the ci logs | |
| 19:59:49 | openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: debug config opts infinite recursion https://review.opendev.org/654468 | |
| 20:00:11 | sean-k-mooney | or maybe it moved | |
| 20:00:16 | sean-k-mooney | it used to be http://intel-openstack-ci-logs.ovh | |
| 20:00:37 | efried | I think the 52. is aws | |
| 20:01:18 | sean-k-mooney | ya the pci ci used an aws log server. they moved to using our ovh one for a while i guess it moved back to aws at some point | |
| 20:01:20 | efried | yeah, that whole class A is amazon's. Not sure if that means aws, but probably | |
| 20:01:54 | sean-k-mooney | basicaly it just had to be a server that was not on the intel network | |
| 20:02:06 | efried | right, so the outside world could see it. | |
| 20:02:45 | sean-k-mooney | actully no os that you did not have to go through a 3 level security review to host an external facing service on the coperate network | |
| 20:03:10 | efried | yeah, that too | |
| 20:03:33 | sean-k-mooney | if its not on the copreate network it was way eaiser aslo the pci ci was originally in china which casued issues | |
| 20:04:07 | efried | okay, I'm going to dive over the firewall and throw some of this intel back at intel. Thanks for the help. | |
| 20:04:52 | sean-k-mooney | cool anyway my adivice would be to do something like im doing in the nfv ci test job | |
| 20:05:19 | sean-k-mooney | e.g. run standared tempest test but use a custom flavor or tweek teh vnic_type used in the tempest config | |
| 20:05:31 | sean-k-mooney | that will test 90% of the edgecases | |
| 20:05:50 | sean-k-mooney | at that point if the ci is stable it can be extended to test more neich things | |
| 20:07:13 | mriedem | dansmith: despite my typo in https://review.opendev.org/#/c/640197/ i'd like to get that in so we can move the grenade live migration job in-tree (this is under the queens backport) - because of devstack changes in queens + grenade i have to disable the ceph stuff in the grenade case, but it's either that or just drop the job altogether from queens which i'd like to avoid | |
| 20:07:48 | mriedem | the commit message attempts to explain the mess and reasoning for just punting and disabling the ceph case | |
| 20:09:26 | dansmith | okay | |
| 20:14:54 | sean-k-mooney | mriedem: speaking of ceph we have a downstream request to backport two of your patches to newton wich is eol upstrema. https://review.opendev.org/#/q/topic:bug/1635008+(status:open+OR+status:merged) i know its been quite a while but are you aware of any reaon of the top of your head why that woudl be a bad idea. | |
| 20:15:28 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional recreate test for regression bug 1825537 https://review.opendev.org/654066 | |
| 20:15:30 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop source node allocations if finish_resize fails https://review.opendev.org/654067 | |
| 20:15:30 | openstack | bug 1825537 in OpenStack Compute (nova) "finish_resize failures incorrectly revert allocations" [Medium,In progress] https://launchpad.net/bugs/1825537 - Assigned to Matt Riedemann (mriedem) | |
| 20:15:34 | sean-k-mooney | it looked resonable to me at first glance but just said i woudl ask | |
| 20:15:58 | mriedem | sean-k-mooney: yes | |
| 20:16:20 | mriedem | there was a regression introduced | |
| 20:16:39 | sean-k-mooney | oh im glad i asked :) | |
| 20:16:50 | mriedem | for $5 i'll tell you the secret | |
| 20:17:54 | mriedem | https://review.opendev.org/#/q/I6fc7108817fcd9df4a342c9dabbf14ab7911d06a | |
| 20:18:00 | sean-k-mooney | https://bugs.launchpad.net/charm-nova-compute/+bug/1671422? | |
| 20:18:01 | openstack | Launchpad bug 1671422 in OpenStack nova-compute charm "charms: nova/cinder/ceph rbd integration broken on Ocata" [Critical,Fix released] - Assigned to James Page (james-page) | |
| 20:18:08 | mriedem | that might be the other one | |
| 20:18:37 | mriedem | https://review.opendev.org/#/q/Ieba216275c07ab16414065ee47e66915e9e9477d | |
| 20:19:01 | sean-k-mooney | ok so it did not handel olde connection info. so if we are to backprot it we need to backport the fixes for the regression too | |
| 20:19:35 | mriedem | sure | |
| 20:19:48 | mriedem | or just tell the bank to upgrade | |
| 20:20:16 | sean-k-mooney | i mean if i could i would but... | |
| 20:20:44 | sean-k-mooney | that said depending on how invasive the other fixes are that might be answer | |
| 20:23:28 | sean-k-mooney | huh the last one was only fixed recently in the grand scheme fo things | |
| 20:41:44 | efried | sean-k-mooney: I can see whitelist/alias stuff in the nova.confZ | |
| 20:42:04 | efried | pci_passthrough_whitelist = [{"vendor_id":"8086","product_id":"1520"}] | |
| 20:42:05 | efried | pci_alias = {"name":"pci_network_card","vendor_id":"8086","product_id":"1520"} | |
| 20:42:23 | efried | Presumably that does no good without a flavor requesting something at that alias? | |
| 20:42:51 | efried | well, I mean, I guess it at least proves that the parts of the pci subsystem that parse those and generate the list of legal devices don't explode. | |
| 20:43:58 | efried | Do you have any idea where a guy could find the script that's running the show here? | |
| 20:47:00 | sean-k-mooney | correct | |
| 20:47:21 | sean-k-mooney | unless the flavor request the alaise it does not do anything | |
| 20:47:41 | sean-k-mooney | for the device to be useable for neutron the physnet would have to be set in the whitelist | |
| 20:48:21 | sean-k-mooney | they could be using a local.conf like i am. my guess would be it burried in a jenkins job somewhere | |
| 20:48:33 | efried | that's what I figured | |
| 20:48:47 | efried | I also really, really don't want to get too deeply involved here. | |
| 20:48:59 | sean-k-mooney | ha to late :P | |
| 20:49:04 | efried | I recognize it's a Learning Opportunity | |
| 20:49:25 | sean-k-mooney | well its being converted to zull v3 right | |
| 20:49:26 | efried | but if I go start debugging the CI, my life is over. | |
| 20:49:29 | efried | yes, afaik | |
| 20:49:51 | sean-k-mooney | so i woudl ignore how its currently running | |
| 20:50:02 | sean-k-mooney | and plan for how it will run | |
| 20:50:07 | efried | I sort of get the impression that there was someone competent doing all the work here, and then whoever that is left the company. | |
| 20:50:58 | efried | And those left behind are scrambling to figure out wtf is going on. | |
| 20:51:17 | sean-k-mooney | well yes but that happened liek 2 years ago :P | |
| 20:51:23 | efried | oh | |
| 20:51:34 | sean-k-mooney | actully the pci ci was always different | |
| 20:51:46 | efried | well, I know it would take *me* at least two years to figure out how this stuff works... | |
| 20:52:01 | sean-k-mooney | but the nfv ci was run by me an waldek but he left to joing redhat 2 years ago | |
| 20:52:48 | sean-k-mooney | the pci ci was run by a team of 1 person out of the PRC site but they had other respociblityes | |
| 20:53:05 | efried | different topic: "completing numa affinity polices for neutron sriov interfaces." <== on the PTG agenda. What's the status of this? Bullet says you were going to write a spec or something? | |
| 20:53:26 | sean-k-mooney | ya basically what i want to get out of that topic it the direct | |
| 20:53:36 | sean-k-mooney | i could just repopose the old spec | |
| 20:53:51 | sean-k-mooney | which say use a flavor/image property to specify the policy | |