| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-12-05 | |||
| 11:19:03 | sean-k-mooney | zbr_: no | |
| 11:19:14 | sean-k-mooney | that would be still a violation of the testing interface | |
| 11:19:33 | sean-k-mooney | if you want to change it then you shoudl propose a governace motion via a patch | |
| 11:20:05 | zbr_ | i know, but before working on such a change i wanted to get some feedback. | |
| 11:20:43 | sean-k-mooney | min would be we should keep pep8 and have an optional seperate linters target | |
| 11:20:49 | sean-k-mooney | *mine | |
| 11:21:07 | zbr_ | we already have this, probably is more than an year old | |
| 11:21:11 | sean-k-mooney | exitsting project that run addtionall linters in pep8 shoudl move them | |
| 11:21:16 | sean-k-mooney | to the linter target | |
| 11:53:13 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Unify '_wait_until_deleted' implementations https://review.opendev.org/689181 | |
| 11:53:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Unify '_build_minimal_create_server_request' implementations https://review.opendev.org/695024 | |
| 11:53:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Make '_IntegratedTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182 | |
| 11:58:48 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Remove 'get_invalid_image' https://review.opendev.org/697454 | |
| 12:00:04 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Unify '_wait_until_deleted' implementations https://review.opendev.org/689181 | |
| 12:00:04 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Unify '_build_minimal_create_server_request' implementations https://review.opendev.org/695024 | |
| 12:00:05 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Make '_IntegratedTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182 | |
| 12:00:05 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Remove 'get_invalid_image' https://review.opendev.org/697454 | |
| 12:26:59 | openstackgerrit | Merged openstack/nova master: Integrate 'pre-commit' https://review.opendev.org/665518 | |
| 14:02:55 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/train: Cache security group driver https://review.opendev.org/697475 | |
| 14:09:05 | shilpasd | mriedem: requesting to review 'https://review.opendev.org/#/c/612626/ | |
| 14:09:48 | openstackgerrit | sean mooney proposed openstack/nova master: support pci numa affinity policies in flavor and image https://review.opendev.org/674072 | |
| 14:10:03 | shilpasd | bauzas: requesting you to review 'https://review.opendev.org/#/c/650188/8 Allow compute nodes to use DISK_GB from shared storage RP' as per your priorities | |
| 14:10:13 | bauzas | shilpasd: ack | |
| 14:10:26 | shilpasd | bauzas: thank you | |
| 14:11:09 | shilpasd | @nova core: please review 'https://review.opendev.org/#/c/650188/8 Allow compute nodes to use DISK_GB from shared storage RP' | |
| 14:17:22 | stephenfin | Is anyone else seeing this message when running 'tox -e py36 |
|
| 14:19:46 | mnaser | eandersson: im just about ready to give up on rabbit clusters | |
| 14:24:29 | mriedem | shilpasd: comments on https://review.opendev.org/#/c/694462/ | |
| 14:25:15 | shilpasd | mriedem: thank you for review, will work on them | |
| 14:25:36 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/stein: Cache security group driver https://review.opendev.org/697480 | |
| 14:26:37 | mriedem | stephenfin: trying it | |
| 14:29:24 | mriedem | efried: this is why that multiple possible networks found thing is intermittent https://bugs.launchpad.net/nova/+bug/1831048 | |
| 14:29:24 | openstack | Launchpad bug 1844568 in tempest "duplicate for #1831048 [compute] "create_test_server" if networks is undefined and more than one network is present" [Medium,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | |
| 14:29:36 | mriedem | neutron tests create a new network at the same time we're creating a server and kaboom | |
| 14:30:01 | mriedem | though we shouldn't be running tempest.api.network tests in nova-next, but it could fail during scenario runs or something i guess? | |
| 14:30:08 | mriedem | stephenfin: i didn't get that problem | |
| 14:30:12 | mriedem | i'm on ubuntu bionic | |
| 14:30:20 | stephenfin | okay, it's a Fedora issue | |
| 14:30:22 | stephenfin | mriedem++ thanks | |
| 14:40:04 | mriedem | i wonder if the nova-next job hits that more often because of how we do concurrency differently in that job from others? | |
| 14:40:27 | mriedem | i think tempest-full runs api tests concurrently and scenario tests serially, and i think nova-next runs them all concurretly | |
| 14:43:01 | mriedem | yeah so i bet a network scenario test creates a network while the compute api tests are running and it blows up | |
| 14:44:04 | sean-k-mooney | mriedem: for the senario test do we not run those with concurancy 1 | |
| 14:45:40 | sean-k-mooney | but yes i can see how it could fial in if a neutron test created a network in paralle with a compute test although ideally it would do that in a different tenant so that it could not affect other tests | |
| 14:46:03 | mriedem | there is probably a scenario test that creates a public network | |
| 14:46:16 | mriedem | and yes we run all tests, api and scenario, concurrently in nova-next | |
| 14:47:17 | sean-k-mooney | ah ok so we dont do the two pahse execution we do in tempst full wehre it runs all test excpet the senario tests first with concurrency and then runs the senario tests serially | |
| 14:47:35 | sean-k-mooney | ya that would proably make it more likely to fail in nova-next | |
| 14:49:57 | mriedem | all things i just said above | |
| 14:50:56 | mriedem | efried: gibi: so i think it was actually enabling tempest.scenario.test_minbw_allocation_placement.MinBwAllocationPlacementTest.test_qos_min_bw_allocation_basic in nova-next again that is making this worse | |
| 14:51:07 | mriedem | that test creates a public network | |
| 14:51:22 | mriedem | and everything that failed in this job was right around the time that test was running https://zuul.opendev.org/t/openstack/build/b7872ac3886e4cc7a060c1376a2a3395/log/job-output.txt#79124 | |
| 14:51:37 | mriedem | which is also our spike in failures http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Details%3A%20%7B'code'%3A%20409%2C%20'message'%3A%20'Multiple%20possible%20networks%20found%2C%20use%20a%20Network%20ID%20to%20be%20more%20specific.'%7D%5C%22%20AND%20tags%3A%5C%22console%5C%22%20AND%20voting%3A1&from=7d | |
| 14:57:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Skip test_minbw_allocation_placement in nova-next job https://review.opendev.org/697491 | |
| 14:57:49 | mriedem | gibi: efried: stephenfin: dansmith: if you wanna help the gate and love patriotism ^ | |
| 14:58:20 | stephenfin | I hate patriotism, but I do have the feels for the gate | |
| 14:59:00 | stephenfin | yuuup, +2 | |
| 15:01:02 | stephenfin | lyarwood: What was the reason for the request for configurables here? https://review.opendev.org/#/c/669674/3/nova/compute/manager.py@2613 | |
| 15:01:49 | stephenfin | I want to -1 that patch _because_ of the seemingly extraneous knob but I could missing something | |
| 15:07:34 | lyarwood | stephenfin: looking | |
| 15:10:54 | gibi | mriedem: you are right. at the same time of the skip the test we need to keep an eye on the real fix https://review.opendev.org/#/c/682964 | |
| 15:11:31 | mriedem | lyarwood: stephenfin: i left several comments in there too, | |
| 15:12:08 | mriedem | blindly retrying on *any* exception when doing something like deleting an attachment makes me nervous because if cinder returns a 404 because the original request completed and then we keep retrying, we're just going to keep retrying and getting 404s until we exhaust our retries | |
| 15:12:21 | stephenfin | good point | |
| 15:12:25 | lyarwood | stephenfin: no specific reason other than a personal dislike of hardcoded retry values tbh | |
| 15:12:27 | mriedem | iow, can we scope those retries to just 500 errors? | |
| 15:12:58 | stephenfin | and I have a personal dislike of knobs that no one ever touches | |
| 15:13:05 | stephenfin | Unstoppable force, meet immovable object :) | |
| 15:13:31 | lyarwood | Meets the guy with +2 over the guy with +1 so you win | |
| 15:13:53 | lyarwood | mriedem: yeah that's fair, I think a previous PS did previously and I've likely missed that cinder_exception.ClientException covers both? | |
| 15:14:48 | lyarwood | yeah it's the base, gah my bad. | |
| 15:15:26 | mriedem | right so https://github.com/openstack/python-cinderclient/blob/5.0.0/cinderclient/exceptions.py#L242 for 404 will create a NotFound(ClientException) and we'd retry on that | |
| 15:15:49 | mriedem | so we need to scope the retries to ClientExceptions with code==500 only imo | |
| 15:16:36 | mriedem | would also be nice to eventually add long_rpc_timeout to cinder for apis like this that have been known to take awhile | |
| 15:16:43 | mriedem | i know we've seen attachment delete timeout in the gate | |
| 15:16:57 | mriedem | surely red hat has some people working on cinder yeah? | |
| 15:17:37 | lyarwood | yup we do, I can follow up with them about that | |
| 15:17:46 | mriedem | https://bugs.launchpad.net/cinder/+bug/1763712 is where i suggested using long_rpc_timeout in cinder | |
| 15:17:47 | openstack | Launchpad bug 1763712 in Cinder "Unable to update the attachment.: MessagingTimeout" [Medium,Confirmed] | |
| 15:17:59 | mriedem | it shouldn't be that hard to add since the cinder rpc stuff is likely very similar to nova's | |
| 15:32:22 | mriedem | gibi: on https://review.opendev.org/#/c/693248/ can't we just hard-code the server created to use the net0 network - that's the one that has the port with the qos allocations | |
| 15:34:48 | mriedem | oh well i guess shared != external | |
| 15:35:15 | mriedem | so we got the shared public external network and nova puked because by default non-admins can't create servers on external networks | |
| 15:39:38 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Cache security group driver https://review.opendev.org/697505 | |
| 15:44:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Regression test for bug 1849657 https://review.opendev.org/693248 | |
| 15:44:16 | openstack | bug 1849657 in OpenStack Compute (nova) train " allocation key is missing from the binding:profile of the neutron qos port when the server is created by a non-admin user" [Medium,Fix committed] https://launchpad.net/bugs/1849657 - Assigned to Balazs Gibizer (balazs-gibizer) | |
| 15:49:01 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: log when loading security group driver https://review.opendev.org/652783 | |
| 15:53:54 | efried | mriedem: ack, thanks for that. | |
| 15:54:02 | efried | my cold has decided to kill me, so I may be sparse today. | |
| 15:54:46 | mriedem | RIP | |
| 15:55:37 | mriedem | efried: fyi i've got a thing in the nova meeting agenda for today but likely not going to be able to attend, it's just a question about if a spec is needed for that filter i'm working on | |
| 15:56:05 | efried | ack | |
| 15:56:17 | mriedem | also, the price is right is on in 4 minutes | |
| 15:56:31 | mriedem | good time to get on the couch with a wubby | |
| 16:00:45 | efried | stephenfin: https://review.opendev.org/#/c/696507/ if you want to update that commit message quick I'll fast approve, no need to rebase anything. | |
| 16:01:32 | efried | do we still say things like $show is on at $time? Who watches anything on a broadcast schedule? | |
| 16:02:17 | stephenfin | okidok | |
| 16:03:36 | openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove SG tests that don't apply to neutron https://review.opendev.org/696507 | |
| 16:03:44 | stephenfin | efried: ^ | |
| 16:04:18 | efried | stephenfin: +A, thanks for accommodating my pickiness. | |