| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-25 | |||
| 13:57:49 | alex_xu | mriedem: thanks | |
| 14:02:45 | mriedem | sdague: i see our functional tests fail with this requests stuff a lot lately, is there some retry magic we should be building in? http://logs.openstack.org/20/459420/1/check/gate-nova-tox-functional-ubuntu-xenial/c0bdfa6/console.html#_2017-04-25_13_55_17_705409 | |
| 14:02:49 | mriedem | to the functional test api client? | |
| 14:07:25 | sdague | mriedem: hmmm... that's weird | |
| 14:07:45 | sdague | mriedem: so that whole BadStatusLine remains a mystery to me | |
| 14:08:08 | sdague | I wonder if it's further eventlet fallout? | |
| 14:08:14 | sdague | mriedem: how frequent is it? | |
| 14:08:21 | cdent | sdague: I seem to recall that having shown up in the past as an artifact of weird eventlet collisions | |
| 14:08:23 | mriedem | 11 hits in 7 days on that job | |
| 14:08:28 | cdent | but I may be remembering something else | |
| 14:08:55 | sdague | I remember looking at it in the gate, and assuming it was load / proxy related | |
| 14:12:26 | openstackgerrit | Alex Xu proposed openstack/nova master: Deprecate Multinic, floatingip action and os-virtual-interface API https://review.openstack.org/457181 | |
| 14:13:24 | alex_xu | mriedem: sdague ^ done | |
| 14:13:35 | mriedem | alex_xu: thanks | |
| 14:13:42 | alex_xu | np | |
| 14:13:46 | jaypipes | alex_xu: pls see my request on https://review.openstack.org/#/c/445864. If you can push that change quickly, I can re-review and +2/W it with johnthetubaguy. | |
| 14:14:01 | mriedem | bauzas: you might be interested in the oldest stale in progress bug (which isn't actually in progress): https://bugs.launchpad.net/nova/+bug/1528743 | |
| 14:14:01 | openstack | Launchpad bug 1528743 in OpenStack Compute (nova) "HostState in Scheduler can be incorrect" [Undecided,In progress] - Assigned to Zhenyu Zheng (zhengzhenyu) | |
| 14:14:13 | cdent | sdague: from some reading just now it appears to happen when either the server side has closed the connection at the exact time the client is trying to read the status line, or is sending a response (because it is evil) without a status line | |
| 14:14:33 | bauzas | mriedem: was absorbed into a team meeting, but will do as soon as it finishes | |
| 14:14:34 | alex_xu | jaypipes: ah, thanks, I will push that change quickly | |
| 14:15:04 | cdent | sdague: TLS isn't invovled in these is it? | |
| 14:15:59 | mriedem | cdent: for the functional tests it shouldn't be | |
| 14:16:40 | cdent | do these functional tests spawn off a different process for the api server, or do it in process somehow? | |
| 14:17:29 | openstackgerrit | John Garbutt proposed openstack/nova master: Add description to policies in migrate_server.py https://review.openstack.org/459711 | |
| 14:18:02 | mriedem | cdent: they use OSAPIFixture which says "This spawns an OS API server as a fixture in a new greenthread in the current test." | |
| 14:18:20 | cdent | ah, then yeah, it could very well be eventlet | |
| 14:18:35 | mtreinish | mriedem: yeah, it seemed simple enough. Assuming I can figure out how to make the api-ref job pass | |
| 14:18:55 | mriedem | mtreinish: you'll have to use restructure text files rather than latex | |
| 14:18:59 | mriedem | so it could be a challenge | |
| 14:19:01 | mriedem | :P | |
| 14:19:06 | cdent | if for some reason the new greenthread doesn't get yielded to when it should, the client will try to read off the socket which nothing paying proper attention on the other side | |
| 14:19:22 | openstackgerrit | Alex Xu proposed openstack/nova master: Use plain routes list for flavors-access endpoint instead of stevedore https://review.openstack.org/456589 | |
| 14:19:22 | openstackgerrit | Alex Xu proposed openstack/nova master: Use plain routes list for '/servers' endpoint instead of stevedore https://review.openstack.org/445864 | |
| 14:19:23 | openstackgerrit | Alex Xu proposed openstack/nova master: Use plain routes list for flavor endpoint instead of stevedore[1] https://review.openstack.org/456549 | |
| 14:19:23 | openstackgerrit | Alex Xu proposed openstack/nova master: Use plain routes list for '/os-aggregates' endpoint instead of stevedore https://review.openstack.org/456825 | |
| 14:19:24 | openstackgerrit | Alex Xu proposed openstack/nova master: Use plain routes list for flavors-extraspecs endpoint instead of stevedore https://review.openstack.org/456585 | |
| 14:19:25 | openstackgerrit | Alex Xu proposed openstack/nova master: Use plain routes list for '/os-keypairs' endpoint instead of stevedore https://review.openstack.org/456478 | |
| 14:19:34 | alex_xu | jaypipes: ^ | |
| 14:20:22 | jaypipes | alex_xu: w00t. reviewing now. | |
| 14:20:30 | alex_xu | jaypipes: thanks | |
| 14:22:33 | openstackgerrit | Matthew Treinish proposed openstack/nova master: Remove nova-cert https://review.openstack.org/459460 | |
| 14:25:27 | cdent | mriedem one way to test the theory about eventlet is to inject a eventlet.greenthread.sleep(0) just before the failing call/test | |
| 14:26:00 | mriedem | cdent: the test that fails is pretty random though | |
| 14:26:04 | mriedem | so i'm not sure what is triggering it | |
| 14:26:22 | cdent | yes, the failure will be random in the eventlet case, that's why I'm wondering if that's the cause | |
| 14:26:41 | cdent | if you force a yield and it works consistently, then that might be it | |
| 14:26:47 | cdent | and we can consider "proper" solutions | |
| 14:26:54 | cdent | if it makes no difference, then ... | |
| 14:28:40 | cdent | mriedem: oh I see your saying not that a test fails randomly but rather different tests randomly fail | |
| 14:28:54 | mriedem | alex_xu: you're missing the GET versions samples https://review.openstack.org/#/c/457181/ | |
| 14:29:15 | mriedem | cdent: yeah | |
| 14:29:17 | cdent | in that case, is there some way to centralize the sleep(0) | |
| 14:29:38 | cdent | like in some central method that is making the request? | |
| 14:29:39 | openstackgerrit | Mathieu Gagné proposed openstack/nova master: Add ability to signal and perform online volume size change https://review.openstack.org/454322 | |
| 14:30:10 | mriedem | cdent: probably here File "nova/tests/functional/api/client.py", line 164, in api_request | |
| 14:30:15 | mriedem | also https://bugs.launchpad.net/nova/+bug/1686109 | |
| 14:30:15 | openstack | Launchpad bug 1686109 in OpenStack Compute (nova) "requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",)) in functional tests" [Medium,Confirmed] | |
| 14:30:19 | mriedem | is the bug i reported for tracking it | |
| 14:31:59 | erhudy | when an instance is launched and neutron ports are attached, does nova do that without calling the interface attach action from the API? not having os_compute_api:os-attach-interfaces disallows it after instance creation as expected, but at launch time it seems to take a different path | |
| 14:32:04 | erhudy | albeit this is on liberty so things may be different now | |
| 14:32:07 | cdent | mriedem i can poke at it after I'm done poking at devstack/uwsgi/nova/metadata, unless you're on the case? | |
| 14:33:25 | mriedem | cdent: i'm not no | |
| 14:33:41 | cdent | ✔ | |
| 14:33:44 | mriedem | erhudy: same internal code, different API entrypoint | |
| 14:34:01 | mriedem | erhudy: so no, server create does not call os-attach-interfaces | |
| 14:34:20 | alex_xu | mriedem: ah, thanks, rebase merge that with os-hosts change... | |
| 14:34:36 | mriedem | erhudy: i believe there is a separate policy rule for disabling networking on server create | |
| 14:34:43 | mriedem | alex_xu: ah | |
| 14:35:29 | erhudy | mriedem: thanks for the explanation | |
| 14:35:57 | mriedem | erhudy: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L533 | |
| 14:36:06 | mriedem | is the policy to disable creating networks on a server create request | |
| 14:36:15 | mriedem | well, | |
| 14:36:21 | mriedem | requesting specific networks i should say | |
| 14:36:38 | mriedem | if you omit networks, but one is available to the project, a port will still be created on that network | |
| 14:37:08 | mriedem | there is probably a bug there wrt the 2.37 microversion and requesting networks='none' | |
| 14:37:11 | mriedem | since we don't check for that case | |
| 14:37:24 | openstackgerrit | Alex Xu proposed openstack/nova master: Deprecate Multinic, floatingip action and os-virtual-interface API https://review.openstack.org/457181 | |
| 14:38:29 | erhudy | alright, thanks for pointing me to the right place | |
| 14:46:02 | dane-fichter | mriedem: barbican can handle either 'aes' or 'AES', you can see it in the functional tests | |
| 14:46:58 | mriedem | erhudy: fyi https://bugs.launchpad.net/nova/+bug/1686113 | |
| 14:46:58 | openstack | Launchpad bug 1686113 in OpenStack Compute (nova) "os_compute_api:servers:create:attach_network does not account for networks='none' and microversion 2.37" [Low,Triaged] | |
| 14:47:05 | mriedem | dane-fichter: ok | |
| 14:47:16 | mriedem | just reply to my question in the review i suppose | |
| 14:47:23 | dane-fichter | mriedem: will do | |
| 14:49:28 | openstackgerrit | John Garbutt proposed openstack/nova master: Add description to policies in migrations.py https://review.openstack.org/459727 | |
| 14:50:47 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add BDM to InstancePayload https://review.openstack.org/448779 | |
| 14:56:34 | openstackgerrit | Chris Dent proposed openstack/nova master: DNM: Use greenthread sleep to unlock api test threads https://review.openstack.org/459733 | |
| 14:57:11 | cdent | mriedem: one way around that ^ problem would be use to use wsgi-intercept instead of spawning a separate thread | |
| 14:57:19 | mriedem | dane-fichter: also note that the test that's dependent on this nova fix is still failing http://logs.openstack.org/59/455459/17/check/gate-barbican-simple-crypto-dsvm-tempest-ubuntu-xenial-nv/eb0372f/console.html#_2017-04-24_21_31_53_893425 | |
| 14:57:26 | mriedem | dane-fichter: so it'd be nice to see that passing | |
| 14:58:07 | openstackgerrit | John Garbutt proposed openstack/nova master: Fix rendering of several policy rule descriptions https://review.openstack.org/459734 | |
| 14:58:10 | dane-fichter | mriedem: yeah that's failing for other reasons, but agreed. | |
| 14:59:57 | openstackgerrit | Sean Dague proposed openstack/nova master: disable keepalive for functional tests https://review.openstack.org/459735 | |
| 15:03:21 | sfinucan | johnthetubaguy: RE: https://review.openstack.org/459734 | |
| 15:03:28 | openstackgerrit | Gábor Antal proposed openstack/nova master: Transform instance.volume_attach notification https://review.openstack.org/401992 | |
| 15:03:52 | sfinucan | johnthetubaguy: what's actually doing the rendering? Couldn't we change that instead to strip newlines or whatnot | |
| 15:03:55 | openstackgerrit | sahid proposed openstack/nova master: libvirt: update LibvirtConfigGuestDeviceAddress to provide XML https://review.openstack.org/459737 | |
| 15:03:56 | openstackgerrit | sahid proposed openstack/nova master: libvirt: set full description of the controller used by disk https://review.openstack.org/459738 | |
| 15:03:56 | openstackgerrit | sahid proposed openstack/nova master: libvirt: rearange how scsi controller is defined https://review.openstack.org/459739 | |