| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-05-04 | |||
| 12:10:26 | zigo | bauzas: The issue is only the functional test, not the api-paste.ini which i tested. | |
| 12:13:51 | bauzas | zigo: then I guess requests is not using parse | |
| 12:15:50 | bauzas | ahah, we have an APIPasteFixture object :) | |
| 12:16:20 | bauzas | bingo | |
| 12:16:22 | bauzas | zigo: ^ | |
| 12:17:10 | bauzas | zigo: look at this https://github.com/openstack/nova/blob/master/nova/tests/functional/api_paste_fixture.py#L51-L59 | |
| 12:25:42 | bauzas | zigo: you should probably pdb the fixture to see what you get as paste.ini | |
| 12:26:24 | zigo | Ok. | |
| 12:49:09 | bauzas | oh but today is a bank holiday in Ireland | |
| 12:49:12 | bauzas | stephenfin: my bad | |
| 12:50:34 | bauzas | stephenfin: my badhttps://review.opendev.org/#/c/725226/ ? | |
| 12:50:37 | bauzas | I can +W it | |
| 12:50:56 | bauzas | lyarwood: around maybe ? | |
| 13:01:35 | bauzas | zigo: fwiw, pdb'ing 724684 | |
| 13:33:41 | bauzas | zigo: ah, I think I found the problem | |
| 13:33:56 | zigo | :) | |
| 13:34:28 | bauzas | zigo: http://paste.openstack.org/show/793050/ | |
| 13:34:53 | bauzas | looks like it tries to call a /v2 endpoint | |
| 13:34:57 | zigo | Oh ! :) | |
| 13:35:11 | zigo | It should be just /healthcheck indeed... | |
| 13:35:29 | zigo | And probably without any auth as well. | |
| 13:38:30 | openstackgerrit | François Palin proposed openstack/nova stable/rocky: Add retry to cinder API calls related to volume detach https://review.opendev.org/725272 | |
| 13:38:30 | openstackgerrit | François Palin proposed openstack/nova stable/rocky: Add retry to cinder API calls related to volume detach https://review.opendev.org/725272 | |
| 13:48:32 | zigo | bauzas: What's the way to make it not prefix the URL then? | |
| 13:48:49 | bauzas | again, I'm not an API specialist unfortunately | |
| 13:49:06 | bauzas | I can't really answer why we transform this URL | |
| 13:49:21 | bauzas | but lemme try something | |
| 13:51:12 | bauzas | ahah found | |
| 13:51:13 | bauzas | https://github.com/openstack/nova/blob/master/nova/tests/functional/api/client.py#L147 | |
| 13:52:23 | zigo | strip_version=True and I'm good to go? | |
| 13:54:37 | bauzas | zigo: fwiw, I just tested to call 'http://728dd5ee-0231-4919-b991-a21cae822084:80/healthcheck' and I got a 200 :) | |
| 13:54:47 | bauzas | just trying strip_version | |
| 13:54:51 | zigo | :) | |
| 13:54:52 | zigo | Thanks. | |
| 13:54:54 | zigo | Trying. | |
| 13:55:29 | bauzas | got a 300 | |
| 13:55:40 | bauzas | that's... fun :) | |
| 13:55:56 | openstackgerrit | Thomas Goirand proposed openstack/nova master: Add a /healthcheck URL https://review.opendev.org/724684 | |
| 13:56:43 | bauzas | zigo: hold your change | |
| 13:56:47 | bauzas | it won't work (yet) | |
| 13:58:10 | bauzas | ahah ! | |
| 13:58:19 | bauzas | > /home/sbauza/git/openstack/nova/nova/tests/functional/api/client.py(172)api_request() | |
| 13:58:20 | bauzas | -> response = self.request(full_uri, **kwargs) | |
| 13:58:20 | bauzas | (Pdb) full_uri | |
| 13:58:20 | bauzas | 'http://94193b57-ca75-4e13-9d4c-2cc961eeaf5b:80/6f70656e737461636b20342065766572//healthcheck' | |
| 13:58:20 | bauzas | (Pdb) n | |
| 13:58:36 | bauzas | we don't want the project here | |
| 13:59:21 | bauzas | (Pdb) good_uri = 'http://94193b57-ca75-4e13-9d4c-2cc961eeaf5b:80/healthcheck' | |
| 13:59:21 | bauzas | (Pdb) self.request(good_uri, **kwargs).status_code | |
| 13:59:21 | bauzas | 200 | |
| 13:59:26 | bauzas | zigo: ^ | |
| 14:03:23 | bauzas | zigo: found how to solve it | |
| 14:03:27 | bauzas | zigo: https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L931 | |
| 14:03:29 | openstackgerrit | Thomas Goirand proposed openstack/nova master: Add a /healthcheck URL https://review.opendev.org/724684 | |
| 14:04:39 | bauzas | zigo: stop eating our CI resources, please :) | |
| 14:04:46 | zigo | Will do ! :) | |
| 14:04:53 | bauzas | zigo: I'm close to the resolution | |
| 14:05:13 | bauzas | zigo: I can propose a revision if you're okay with ? | |
| 14:05:20 | zigo | Sure, thanks ! | |
| 14:05:49 | bauzas | zigo: ok, I'll just propose another test class | |
| 14:09:15 | zigo | Indeed, it's probably fine in this file, but not the same class, I thought about fixing that too ! | |
| 14:17:12 | gibi | melwitt: you looked at this bug report before: https://bugs.launchpad.net/nova/+bug/1818671 There are some new data in there. I looked at it. I cannot reproduce it on master and the bugfix that seems relevant backported to the version of used by the last commenter | |
| 14:17:12 | openstack | Launchpad bug 1818671 in OpenStack Compute (nova) "Openstack usage list not showing all projects" [Undecided,New] | |
| 14:18:12 | gibi | melwitt: I don't have a queens devstack at hand. Do you have any idea what fixed the problem since Queens? | |
| 14:28:36 | bauzas | zigo: I'm done with your change | |
| 14:28:39 | bauzas | zigo: it works locally | |
| 14:28:47 | bauzas | zigo: but I'd like to add a negative test | |
| 14:29:12 | bauzas | zigo: how can I fake healthcheck to say "sorry but no" ? | |
| 14:29:12 | zigo | bauzas: Of what kind? What will you negatively test? | |
| 14:29:20 | zigo | Oh... | |
| 14:33:03 | openstackgerrit | Doug Hellmann proposed openstack/nova master: increase verbose level for release notes build https://review.opendev.org/725290 | |
| 14:34:56 | bauzas | ok, nevermind, you haven't enabled the disabled_per_file option | |
| 14:35:07 | bauzas | will just verify the JSON output | |
| 14:37:01 | bauzas | ahah, very interesting | |
| 14:37:01 | bauzas | /home/sbauza/git/openstack/nova/.tox/functional/lib/python3.7/site-packages/paste/deploy/util.py:55: DeprecationWarning: Using function/method 'Healthcheck.factory()' is deprecated: The healthcheck middleware must now be configured as an application, not as a filter | |
| 14:37:02 | zigo | bauzas: I don't see any reason why it would be needed, but yeah, we could set /etc/nova/disable_healthcheck by default. | |
| 14:37:14 | bauzas | zigo: let's not overthink this | |
| 14:37:21 | zigo | Oh ! :) | |
| 14:37:26 | bauzas | this being said, see the deprecation warning above | |
| 14:37:41 | zigo | I've seen it as an app somewhere. | |
| 14:38:13 | bauzas | zigo: https://github.com/openstack/oslo.middleware/blob/2e56b4379ab5a8bd5c42e885253c81d2a5f021c2/oslo_middleware/healthcheck/__init__.py#L242-L248 | |
| 14:38:22 | bauzas | zigo: just amending my revision | |
| 14:39:07 | zigo | Thanks a lot for all of this, this will really be helpful for operators. | |
| 14:40:03 | zigo | Typically, in haproxy, you'd setup: | |
| 14:40:03 | zigo | backend novabackend | |
| 14:40:03 | zigo | option httpchk GET /healthcheck | |
| 14:40:12 | zigo | If you don't, then you get crap in the nova-api.log ... | |
| 14:40:51 | zigo | As in, haproxy tries to open the connection, does no request, and close the connection, which is *very* hugly. | |
| 14:45:58 | gmann | bauzas: i feel we can keep those upgrade notes for V upgrade also as we will remove the old defaults in W cycle (along with U)- https://review.opendev.org/#/c/725226/1 | |
| 14:47:24 | bauzas | zigo: okay, you know what ? I can't really help on the paste issue, but I'll upload my new rev | |
| 14:47:41 | bauzas | zigo: then you can ask gmann or other API specialists to look at your paste config | |
| 14:51:42 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add a /healthcheck URL https://review.opendev.org/724684 | |
| 14:52:17 | bauzas | zigo: okay, I left the deprecated usage but the functional test should work | |
| 14:52:42 | bauzas | zigo: this being said, I'll -1 because of the deprecation usage, please try to look at it | |
| 14:53:57 | zigo | Will do, let's first wait to see if it passes the gate. | |
| 14:55:22 | bauzas | zigo: it will : | |
| 14:55:32 | bauzas | functional run-test: commands[1] | stestr slowest | |
| 14:55:32 | bauzas | Test id Runtime (s) | |
| 14:55:32 | bauzas | -------------------------------------------------------------------------------- ----------- | |
| 14:55:32 | bauzas | nova.tests.functional.test_middleware.TestHealthCheckMiddleware.test_healthcheck 4.460 | |