Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2024-09-09
18:47:55 sean-k-mooney there might eb some header you can set like x_forwared_for or similar
18:48:35 sean-k-mooney in my case the path query arge is empty
18:48:45 noonedeadpunk I think it's just because path being used to pass token
18:48:49 sean-k-mooney well technialy path=%3F
18:48:58 noonedeadpunk ah
18:49:00 noonedeadpunk true
18:49:20 noonedeadpunk but it's what nova replies then
18:49:54 sean-k-mooney you may be able to add it to novncproxy_base_url
18:50:31 sean-k-mooney so instead of http://127.0.0.1:6080/vnc_auto.html
18:50:59 sean-k-mooney http://domain/subpath/vnc_auto.html?path=/subpath
18:51:55 sean-k-mooney can you try setting that in your broser and see if setting path=/subpath is enouch
18:51:58 noonedeadpunk hm, I somehow don't think that will work?
18:52:09 noonedeadpunk yeah, I'm trying now
18:52:24 noonedeadpunk in terms won't work - regarding defining novncproxy_base_url that way
18:52:28 sean-k-mooney well its being used by vnc_lite.html
18:52:32 sean-k-mooney whic is part of novnc
18:52:57 sean-k-mooney nova might not allow you pas qurey sting arge in baseurl
18:53:03 sean-k-mooney but it could be added by the proxy
18:53:16 sean-k-mooney *reverse_proxy when rewriteing
18:53:31 sean-k-mooney we coudl maybe add a cofnig option for that if it works
18:54:24 sean-k-mooney based on https://github.com/novnc/noVNC/blob/master/vnc_lite.html#L134-L153
18:54:28 sean-k-mooney i dont see why it woudl not
18:54:51 noonedeadpunk yeah, adding manually shiould work
18:55:03 noonedeadpunk what probably wont - adjustin novncproxy_base_url as you said
18:55:29 sean-k-mooney ya we likely dont have logic to parse the query arges and append them
18:55:31 noonedeadpunk I don't think that nova gonna respect already defined path and wont'd add default path=%3F
18:55:34 sean-k-mooney but should be doable exterally
18:56:05 noonedeadpunk yeah, this should be not hard to do indeed
18:56:21 noonedeadpunk (I'd say would make sense for config option though)
18:56:51 noonedeadpunk or indeed parse for the option
19:00:29 sean-k-mooney the other thing you said is there isp blockes 6080
19:00:47 sean-k-mooney we do hwave supprot for using alternitive ports alredy
19:00:58 sean-k-mooney so that might be an option in that case
19:01:46 sean-k-mooney it would be a pretty minor feature to add novncproxy_extra_query_args
19:05:45 noonedeadpunk ok, so `https://domain.com/console/vnc_lite.html?path=console%3Ftoken%3Dd7bf7a47-f081-4500-a961-05ddb7994e13`
19:05:48 noonedeadpunk does work indeed
19:07:05 noonedeadpunk so thanks a ton for your time
19:07:45 noonedeadpunk I know I was annoying but quite happy now as you said - it should be trivial both to rewrite and implement a "native" way
19:08:13 sean-k-mooney your bringing a pain point to our attention that not annoying
19:08:24 sean-k-mooney i still think subdomains are preferable
19:08:43 sean-k-mooney but if someone wants to add this minor feature im not agaisnt it
19:08:53 sean-k-mooney i just rpaobly wont go implemtnet it in my spare time
19:10:01 sean-k-mooney noonedeadpunk: you may have hit https://github.com/haproxy/haproxy/issues/829 by the way
19:12:11 sean-k-mooney there is a set-pathq to set items in teh query string now
19:14:13 sean-k-mooney i guess that migh now be https://www.haproxy.com/documentation/haproxy-configuration-tutorials/http-rewrites/#set-the-query-string
19:15:37 sean-k-mooney so i think you can jsut add "http-request set-query path=/subpath"
19:16:31 sean-k-mooney noonedeadpunk: im going to call it a day. if that works you could porpsoe a docs update if you were so inclidned or a devstack path
19:16:44 sean-k-mooney *patch
19:20:01 noonedeadpunk ++ thanks for the help as usual!
19:30:15 noonedeadpunk yeah, so here it goes: https://opendev.org/openstack/nova/src/branch/master/nova/objects/console_auth_token.py#L78
19:49:53 samcat116 Hi all, whats a good place to start debugging the GET /servers api being slow? Running `openstack server list --all-projects --no-name-lookup` on a stack with about 1500 instances takes over 48s to return, with the GET https://example.com:8774/v2.1/servers/detail?deleted=False&all_tenants=True call taking over 30s itself. CPU usage across our 3 controllers is around 50% and I'm running with 10 api_workers, so don't think its a raw
19:49:54 samcat116 horsepower issue
22:10:54 sean-k-mooney samcat116: the server detail list need to hit neutron for each server to lookup secuirty groups, but openstack client is doing more then jsut /v2.1/servers/detail?deleted=False&all_tenants=Tr
22:10:57 sean-k-mooney | ue
22:12:40 sean-k-mooney that specific query however does require conenctin to all cell db and pulling a lot of info so its one of the more expensive api calls you can make to nova
22:13:05 sean-k-mooney its not as expensieve as simple tenant usage but it s more expensive then just about any other read only call
22:19:20 samcat116 Ok I guess I’ll look on the neutron side as we are pretty port and network heavy as well
22:19:37 samcat116 I just don’t know why it would be that slow
22:20:08 sean-k-mooney it might not be related to that but its on eof tthe peices of data we lookup form other services
22:20:23 sean-k-mooney we cache alotof the info reatled to other services but not all
22:20:56 sean-k-mooney tehre are some profiling tools liek os-profiler
22:21:06 sean-k-mooney but im not famialr with who to properly use them
22:21:45 samcat116 Ok. I do have memcache all setup so I would assume this could be heavily cached
22:22:01 sean-k-mooney no
22:22:17 sean-k-mooney i think we have almsot no caching for this
22:22:38 sean-k-mooney we use memcache very sparingly
22:23:31 sean-k-mooney its main use in nova is for the metadata api
22:24:15 sean-k-mooney we generally dont cache api responces
22:25:30 sean-k-mooney the other main use of memcafhe is for keytone auth
22:27:47 samcat116 Good to know
22:30:08 samcat116 How far off of an estimate could this be. Just trying to figure out if this is an order of magnitude slower than it should be or not
22:30:41 sean-k-mooney that kind of depend on other factors like how many cells you have
22:30:53 sean-k-mooney you not the first person to ask about this
22:31:13 sean-k-mooney samcat116: can you try making the same query to /server instead of /server/detials
22:31:47 sean-k-mooney the delta is /server just hits the api db and list all the server we know about without going to the cell dbs or calling other services
22:32:17 sean-k-mooney that should be very fast as it basicaly a select instance_uuid from istance_mappings
22:35:01 sean-k-mooney if /servers/ is fast and /servers/detail is slow then one thing to check is if any of the cells are slow to respond
22:35:31 sean-k-mooney internally the api when repondign ot /servers/detail is calling into each cell db and geting the data form the instances table
22:38:06 sean-k-mooney the security group info i thin need a call to neutron for every port
22:38:41 samcat116 Ok I’ll try that. We only have one cell
22:39:09 sean-k-mooney that should simplfy thing then as the api is not making parralel requests
22:41:07 sean-k-mooney i think the security gorups are the only thing not cached in that responce. i.e. that is not just a direct db lookup
22:42:39 sean-k-mooney this has been know for a long time https://bugs.launchpad.net/nova/+bug/1923560 and a fix was prospoed here https://review.opendev.org/c/openstack/nova/+/786348
22:43:39 sean-k-mooney i was personaly on the side of removing the security groups form the instance obejct entirly
22:43:46 sean-k-mooney btu if we dont then we shoudl cache them
#openstack-nova - 2024-09-10
00:04:27 samcat116 So /server?all_tenants=True vs /servers/detail?all_tenants=True was 1.3s vs 26s
00:04:33 samcat116 s/server/servers/, s/all_tenants/all\_tenants/, s/all_tenants/all\_tenants/
00:18:06 sean-k-mooney we woudl expect there to be a delta but not over an order of magnitde.
07:04:49 bauzas because we don't call the cells DB for the first call :)
07:08:37 opendevreview Tobias Urdin proposed openstack/nova master: api-ref: Add new lines not allowed for server description https://review.opendev.org/c/openstack/nova/+/928758
09:06:50 bauzas well, looking at the relnotes, the cycle highlights will be quite simple to write...
09:26:29 opendevreview Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928456
09:26:30 opendevreview Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584
09:33:01 opendevreview Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584
09:39:03 opendevreview Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928456
09:39:03 opendevreview Balazs Gibizer proposed openstack/nova master: Refactor obj_make_compatible to reduce complexity https://review.opendev.org/c/openstack/nova/+/928590
09:39:04 opendevreview Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584
09:46:54 opendevreview Balazs Gibizer proposed openstack/os-traits master: Add support for hw_vif_model igb image property https://review.opendev.org/c/openstack/os-traits/+/928582
11:59:23 sean-k-mooney gibi: for the igb support can you allso add a patch to glance to update https://github.com/openstack/glance/blob/master/etc/metadefs/compute-libvirt-image.json#L112-L127
11:59:41 sean-k-mooney that used to autogenerate the ui in glance/heat ectra

Earlier   Later