| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-09-09 | |||
| 18:29:26 | sean-k-mooney | there are some yes | |
| 18:29:45 | sean-k-mooney | its not that devstack could not test this | |
| 18:29:58 | sean-k-mooney | its that its not in scope to test every combination of how openstack is deployed | |
| 18:30:04 | noonedeadpunk | yeah, fair | |
| 18:30:09 | noonedeadpunk | that I agree with | |
| 18:30:48 | noonedeadpunk | I guess I just not fully in agreement that serving consoles under subpath should be out of scope of nova-novncproxy service | |
| 18:31:14 | sean-k-mooney | for our new installer we deploy novnc on a subdomain | |
| 18:31:17 | sean-k-mooney | https://logserver.rdoproject.org/57/857/e3248dbf95dfb1878d852beef4475bfb68f4e811/github-check/nova-operator-tempest-multinode/616a00d/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/namespaces/openstack/routes/nova-novncproxy-cell1-public.yaml | |
| 18:32:16 | noonedeadpunk | yeah, subdomain works really nicely.... | |
| 18:32:45 | noonedeadpunk | ok, gotcha | |
| 18:32:48 | sean-k-mooney | yep so again using subpath need a reversproxy that support both http and websockets | |
| 18:32:58 | sean-k-mooney | as in it need to be able to rewirte both properly | |
| 18:33:15 | sean-k-mooney | it might be posibel to do some other way | |
| 18:33:16 | noonedeadpunk | the problem here of reverseproxy, that then you can't serve anything on root | |
| 18:33:36 | noonedeadpunk | so if you want to have smth like skyline/horizon on root - you can't do that jsut because of vnc console | |
| 18:33:53 | sean-k-mooney | right but normally you sever that on /dashboard | |
| 18:33:59 | sean-k-mooney | and have a redirect for root to it | |
| 18:34:39 | noonedeadpunk | but then redirecting is also fishy a bit, as you need to account for wss | |
| 18:34:48 | sean-k-mooney | you shoudl be able to have the proxy on /vnc by the way and use / for horizon aslong as there isnt a collition | |
| 18:34:53 | noonedeadpunk | which is just /?token= | |
| 18:35:09 | noonedeadpunk | but probably possible to come up with some regex | |
| 18:35:47 | sean-k-mooney | you shoudl be able to have the websoceket be wss://<domain>/novnc?token=... | |
| 18:36:54 | noonedeadpunk | so I just got `https://domain.com/console/vnc_lite.html` loading but still wss was just wss:// |
|
| 18:37:17 | noonedeadpunk | so it ended up with "something went wrong" | |
| 18:37:35 | sean-k-mooney | right but the websocket is not beeing writhe properly | |
| 18:38:07 | sean-k-mooney | you said the url looked correct form nova right | |
| 18:38:23 | sean-k-mooney | do you have an exmaple of the consol url returned | |
| 18:38:29 | noonedeadpunk | yes, but what does return wss URL? | |
| 18:39:02 | sean-k-mooney | at least for the serial console oepnstack server console show returns it | |
| 18:39:22 | noonedeadpunk | openstack console url show returns `https://domain.com/console/vnc_lite.html?path=%3Ftoken%3D76a1303f-c40c-4169-912d-19c05432dbc5` is what is returned | |
| 18:39:25 | sean-k-mooney | but i think we get https back by defualt for novnc | |
| 18:39:30 | noonedeadpunk | it loads, vnc console renders | |
| 18:40:00 | sean-k-mooney | so with devstack the dash board load at "http://192.168.16.127/dashboard/auth/login/?next=/dashboard/" | |
| 18:40:10 | noonedeadpunk | but then html5 tries to connect to wss on `wss://domain.com/?token=76a1303f-c40c-4169-912d-19c05432dbc5` | |
| 18:41:05 | noonedeadpunk | ok, I gues I'm jsut confused where this wss path is coming from | |
| 18:41:14 | noonedeadpunk | as indeed it's probably not from nova at all | |
| 18:41:30 | sean-k-mooney | its part of the html content provided to the javascript app | |
| 18:43:16 | sean-k-mooney | so in my case it returns http://192.168.16.127:6080/vnc_lite.html?path=%3Ftoken%3Dd8a68169-dbd6-4333-8b4a-f57bc3beb53c by edfault | |
| 18:43:44 | sean-k-mooney | if i open that direclty that is what nova is serving and what ironic is embding in an iframe | |
| 18:43:55 | noonedeadpunk | yup, true, and that's working as well (if clients don't have 6080 firewalled) | |
| 18:44:10 | noonedeadpunk | byt heir isp | |
| 18:44:26 | noonedeadpunk | ok, I'll try to do some rewrites indeed. | |
| 18:44:53 | noonedeadpunk | and look one more time what does construct wss uri | |
| 18:45:15 | sean-k-mooney | tehre are two path to how htis works. we use websoxkify to wrap th raw tcp connectoin to the vnc console create by qemu | |
| 18:45:41 | sean-k-mooney | and we use novnc ot provide the webpage that conenct to that and provde the vnc clinet | |
| 18:45:49 | noonedeadpunk | as htmlk5 console does have `const path = readQueryVariable('path', 'websockify');` | |
| 18:46:36 | noonedeadpunk | and it does `url += '/' + path;` - that's why I kind of complained that smth not passing html5 all details for it to perform | |
| 18:47:27 | sean-k-mooney | ok but that novnc that doing that | |
| 18:47:32 | sean-k-mooney | not nova | |
| 18:47:55 | sean-k-mooney | there might eb some header you can set like x_forwared_for or similar | |
| 18:48:35 | sean-k-mooney | in my case the path query arge is empty | |
| 18:48:45 | noonedeadpunk | I think it's just because path being used to pass token | |
| 18:48:49 | sean-k-mooney | well technialy path=%3F | |
| 18:48:58 | noonedeadpunk | ah | |
| 18:49:00 | noonedeadpunk | true | |
| 18:49:20 | noonedeadpunk | but it's what nova replies then | |
| 18:49:54 | sean-k-mooney | you may be able to add it to novncproxy_base_url | |
| 18:50:31 | sean-k-mooney | so instead of http://127.0.0.1:6080/vnc_auto.html | |
| 18:50:59 | sean-k-mooney | http://domain/subpath/vnc_auto.html?path=/subpath | |
| 18:51:55 | sean-k-mooney | can you try setting that in your broser and see if setting path=/subpath is enouch | |
| 18:51:58 | noonedeadpunk | hm, I somehow don't think that will work? | |
| 18:52:09 | noonedeadpunk | yeah, I'm trying now | |
| 18:52:24 | noonedeadpunk | in terms won't work - regarding defining novncproxy_base_url that way | |
| 18:52:28 | sean-k-mooney | well its being used by vnc_lite.html | |
| 18:52:32 | sean-k-mooney | whic is part of novnc | |
| 18:52:57 | sean-k-mooney | nova might not allow you pas qurey sting arge in baseurl | |
| 18:53:03 | sean-k-mooney | but it could be added by the proxy | |
| 18:53:16 | sean-k-mooney | *reverse_proxy when rewriteing | |
| 18:53:31 | sean-k-mooney | we coudl maybe add a cofnig option for that if it works | |
| 18:54:24 | sean-k-mooney | based on https://github.com/novnc/noVNC/blob/master/vnc_lite.html#L134-L153 | |
| 18:54:28 | sean-k-mooney | i dont see why it woudl not | |
| 18:54:51 | noonedeadpunk | yeah, adding manually shiould work | |
| 18:55:03 | noonedeadpunk | what probably wont - adjustin novncproxy_base_url as you said | |
| 18:55:29 | sean-k-mooney | ya we likely dont have logic to parse the query arges and append them | |
| 18:55:31 | noonedeadpunk | I don't think that nova gonna respect already defined path and wont'd add default path=%3F | |
| 18:55:34 | sean-k-mooney | but should be doable exterally | |
| 18:56:05 | noonedeadpunk | yeah, this should be not hard to do indeed | |
| 18:56:21 | noonedeadpunk | (I'd say would make sense for config option though) | |
| 18:56:51 | noonedeadpunk | or indeed parse for the option | |
| 19:00:29 | sean-k-mooney | the other thing you said is there isp blockes 6080 | |
| 19:00:47 | sean-k-mooney | we do hwave supprot for using alternitive ports alredy | |
| 19:00:58 | sean-k-mooney | so that might be an option in that case | |
| 19:01:46 | sean-k-mooney | it would be a pretty minor feature to add novncproxy_extra_query_args | |
| 19:05:45 | noonedeadpunk | ok, so `https://domain.com/console/vnc_lite.html?path=console%3Ftoken%3Dd7bf7a47-f081-4500-a961-05ddb7994e13` | |
| 19:05:48 | noonedeadpunk | does work indeed | |
| 19:07:05 | noonedeadpunk | so thanks a ton for your time | |
| 19:07:45 | noonedeadpunk | I know I was annoying but quite happy now as you said - it should be trivial both to rewrite and implement a "native" way | |
| 19:08:13 | sean-k-mooney | your bringing a pain point to our attention that not annoying | |
| 19:08:24 | sean-k-mooney | i still think subdomains are preferable | |
| 19:08:43 | sean-k-mooney | but if someone wants to add this minor feature im not agaisnt it | |
| 19:08:53 | sean-k-mooney | i just rpaobly wont go implemtnet it in my spare time | |
| 19:10:01 | sean-k-mooney | noonedeadpunk: you may have hit https://github.com/haproxy/haproxy/issues/829 by the way | |
| 19:12:11 | sean-k-mooney | there is a set-pathq to set items in teh query string now | |
| 19:14:13 | sean-k-mooney | i guess that migh now be https://www.haproxy.com/documentation/haproxy-configuration-tutorials/http-rewrites/#set-the-query-string | |
| 19:15:37 | sean-k-mooney | so i think you can jsut add "http-request set-query path=/subpath" | |
| 19:16:31 | sean-k-mooney | noonedeadpunk: im going to call it a day. if that works you could porpsoe a docs update if you were so inclidned or a devstack path | |
| 19:16:44 | sean-k-mooney | *patch | |
| 19:20:01 | noonedeadpunk | ++ thanks for the help as usual! | |
| 19:30:15 | noonedeadpunk | yeah, so here it goes: https://opendev.org/openstack/nova/src/branch/master/nova/objects/console_auth_token.py#L78 | |
| 19:49:53 | samcat116 | Hi all, whats a good place to start debugging the GET /servers api being slow? Running `openstack server list --all-projects --no-name-lookup` on a stack with about 1500 instances takes over 48s to return, with the GET https://example.com:8774/v2.1/servers/detail?deleted=False&all_tenants=True call taking over 30s itself. CPU usage across our 3 controllers is around 50% and I'm running with 10 api_workers, so don't think its a raw | |