| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-07-27 | |||
| 17:52:34 | dansmith | omh | |
| 17:52:48 | dansmith | sean-k-mooney: it's checking to see that the console is actually empty in shutoff state/ | |
| 17:53:16 | sean-k-mooney | no output will be true if its not the empty string | |
| 17:53:33 | melwitt | on the command line I get "ResourceNotFound: 404: Client Error for url: http://127.0.0.1/compute/v2.1/servers/68b8344b-10dc-4a84-90e4-ad8be7a2bb60/action, Guest does not have a console available." | |
| 17:53:37 | melwitt | osc | |
| 17:53:48 | dansmith | wtf | |
| 17:54:00 | dansmith | so I guess I totally misread the test | |
| 17:54:02 | sean-k-mooney | ya thats what i get too | |
| 17:54:09 | sean-k-mooney | 404 | |
| 17:54:22 | sean-k-mooney | but i think the 404 has an empy body | |
| 17:54:33 | dansmith | I thought the test was asserting that the console was *still* available | |
| 17:54:33 | melwitt | yeah and it's bc | |
| 17:54:33 | melwitt | except (exception.InstanceNotFound, | |
| 17:54:33 | melwitt | exception.ConsoleNotAvailable) as e: | |
| 17:54:33 | melwitt | raise webob.exc.HTTPNotFound(explanation=e.format_message()) | |
| 17:54:48 | melwitt | wait what | |
| 17:54:53 | sean-k-mooney | dansmith: the test is | |
| 17:55:07 | sean-k-mooney | https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/api/compute/servers/test_server_actions.py#L211C7-L213 | |
| 17:55:10 | dansmith | it's not, it's asserting that the result is "console is empty" | |
| 17:55:14 | melwitt | """Test getting console output for a server in SHUTOFF status | |
| 17:55:14 | melwitt | Should be able to GET the console output for a given server_id | |
| 17:55:14 | melwitt | in SHUTOFF status. | |
| 17:55:24 | dansmith | sean-k-mooney: L213 | |
| 17:55:37 | frickler | you can add --debug to the osc call to verify the body | |
| 17:55:47 | melwitt | uh wtf self.assertTrue(output, "Console output was empty.") | |
| 17:55:50 | sean-k-mooney | no self.assertTrue takes an object to test and a message to return if it fails | |
| 17:55:56 | dansmith | melwitt: yeah I wonder if that means "doesn't fail" | |
| 17:56:08 | melwitt | this is asserting the exact opposite of what the docstring says | |
| 17:56:12 | dansmith | sean-k-mooney: ahhh, right okay | |
| 17:56:19 | dansmith | not assertEqual | |
| 17:56:29 | melwitt | ohh ok right | |
| 17:56:30 | dansmith | confusing | |
| 17:56:32 | sean-k-mooney | right | |
| 17:56:42 | sean-k-mooney | they shoudl not be useing assertTrue relaly in this case | |
| 17:56:48 | dansmith | yeah for serious | |
| 17:56:56 | sean-k-mooney | as i said they are relying on the boolean convertion to true of a non empty string | |
| 17:57:07 | melwitt | yeah bc they're saying "I got an error string, it must be a console!" | |
| 17:57:28 | sean-k-mooney | right so the fact this is failing | |
| 17:57:39 | sean-k-mooney | is telliing me the 404 had an emptyp body | |
| 17:57:48 | sean-k-mooney | im going to get the request id and check the api log | |
| 17:57:54 | dansmith | I wish frickler wasn't here watching this circus of three seasoned nova cores trying to figure out how one of the most basic calls works :/ | |
| 17:57:57 | sean-k-mooney | it should have the content lenght | |
| 17:58:19 | melwitt | seasoned curlies | |
| 17:58:39 | dansmith | frickler: I'm kidding of course, just .. not our best moment :) | |
| 17:59:14 | frickler | well I can't say I'm not enjoying it ;) | |
| 17:59:18 | dansmith | lol | |
| 17:59:49 | melwitt | this means not an empty string no? | |
| 17:59:49 | melwitt | DEBUG nova.api.openstack.wsgi [None req-2441eccd-ea63-4c9d-94a3-29aaf1ecf5c4 demo demo] Returning 404 to user: Guest does not have a console available. {{(pid=2011570) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:936}} | |
| 18:00:19 | melwitt | I guess I should do osc --debug | |
| 18:00:25 | sean-k-mooney | status: 404 len: 86 microversion: 2.91 time: 0.719216 | |
| 18:00:42 | melwitt | RESP BODY: {"itemNotFound": {"code": 404, "message": "Guest does not have a console available."}} | |
| 18:00:44 | sean-k-mooney | 86 bytes might just be the header | |
| 18:00:49 | gmann | status code is already checked in service client side so we can just remove this assert itself ? | |
| 18:00:54 | sean-k-mooney | oh actully no | |
| 18:00:58 | sean-k-mooney | RESP BODY: {"itemNotFound": {"code": 404, "message": "Guest does not have a console available."}} | |
| 18:01:11 | melwitt | so I think it really is going assertTrue("Guest does not have a console available.") | |
| 18:01:32 | sean-k-mooney | ya but why whoudl that be false | |
| 18:01:45 | dansmith | maybe it's ... AI... | |
| 18:01:48 | melwitt | maybe for lvm it's not returning a non empty string? I dunno why | |
| 18:01:57 | melwitt | AI yes | |
| 18:02:03 | sean-k-mooney | no that form lvm | |
| 18:02:09 | sean-k-mooney | i have it repoduced locally | |
| 18:02:11 | melwitt | oh... uh wtf | |
| 18:02:28 | sean-k-mooney | i ran osc with --debug | |
| 18:02:38 | sean-k-mooney | so it pring the actul repsocne body | |
| 18:03:13 | gmann | but how come it reach to this assert if it return 404 ? it should have failed here https://github.com/openstack/tempest/blob/master/tempest/lib/api_schema/response/compute/v2_1/servers.py#L456 | |
| 18:03:56 | sean-k-mooney | thats the fully output if your interested form osc https://termbin.com/fq3i | |
| 18:04:05 | dansmith | gmann: yeah I was looking to see if we were ignoring 404 or something but we're not | |
| 18:04:40 | melwitt | I don't get how this is passing though | |
| 18:04:40 | melwitt | lines = len(output.split('\n')) | |
| 18:04:40 | melwitt | self.assertEqual(lines, 3) | |
| 18:04:50 | gmann | yeah, status code in resposne schema fail if different status code than expected and it does not proceed further | |
| 18:05:33 | melwitt | this is some kind of black magic I think | |
| 18:05:47 | dansmith | obviously the solution is to remove the nova-lvm job | |
| 18:06:11 | melwitt | yeah. removing job and skipping tests makes things work :D | |
| 18:06:34 | gmann | :) | |
| 18:06:39 | melwitt | we definitely need more logging in that tempest test cause like... lol | |
| 18:08:11 | sean-k-mooney | we only validate if the respocne is scculs or redirection | |
| 18:08:13 | sean-k-mooney | https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/lib/common/rest_client.py#L1016 | |
| 18:08:24 | sean-k-mooney | we got a 404 | |
| 18:08:30 | sean-k-mooney | so the cleint does not validate it | |
| 18:08:57 | sean-k-mooney | that why its not failing earlier | |
| 18:09:04 | melwitt | and there's no else: noice | |
| 18:09:18 | sean-k-mooney | yep we fall off the end of the function | |
| 18:09:25 | melwitt | VALID@ | |
| 18:09:29 | melwitt | VALID! | |
| 18:12:18 | gmann | sean-k-mooney: it should have failed here https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/lib/common/rest_client.py#L852 | |
| 18:13:17 | sean-k-mooney | i assume _error_checker is a decortor or called else whewer | |
| 18:13:26 | sean-k-mooney | i didnt see that in the funcitons i was looking at | |
| 18:14:02 | sean-k-mooney | thats not called form action https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/lib/services/compute/servers_client.py#L218 | |
| 18:14:19 | sean-k-mooney | or get_console_output https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/lib/services/compute/servers_client.py#L648-L656 | |
| 18:14:57 | sean-k-mooney | which is what we are calling https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/api/compute/servers/test_server_actions.py#L211 | |
| 18:16:00 | gmann | it is called for every API call from tempest https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/lib/common/rest_client.py#L742 | |
| 18:16:33 | sean-k-mooney | so its called form this post https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/lib/services/compute/servers_client.py#L225C27-L225C31 | |
| 18:16:45 | frickler | only success is validated https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/lib/common/rest_client.py#L1016 | |
| 18:17:37 | sean-k-mooney | ya but gmann is correct | |
| 18:18:09 | sean-k-mooney | on https://github.com/openstack/tempest/blob/04cb0adc822ffea6c7bfccce8fa08b03739894b7/tempest/lib/common/rest_client.py#L742 we shoudl go to _error_checker | |
| 18:18:20 | gmann | that is for schema only tempest validate status code and it is <400 then only it proceed to test otherwise fail in API call itself | |
| 18:18:34 | gmann | that->validate_response | |
| 18:19:59 | sean-k-mooney | ok so the post will raise | |