Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-09-20
15:18:02 belmoreira Hi bauzas. Thanks, to me this new approach seems really heavy for deployments with a lot cells. For now I'm hacking something similar to [1] to not have a vncproxy per cell.
15:18:10 belmoreira [1] https://github.com/openstack/nova/blob/0bd61915ee1d96ca339f342a190e395a39afbcf9/nova/api/openstack/compute/console_auth_tokens.py#L42
15:18:30 belmoreira maybe we can discuss this in the PTG
15:26:51 dansmith it seems strange to me that someone with lots of cells would want to *not* shard that service across cells
15:27:04 dansmith especially with geo-distributed cells
15:31:35 bauzas agreed with dansmith
15:31:45 dansmith (he dropped)
15:32:04 bauzas hah, my internal meeting trampled this discussion
15:32:25 bauzas -ETOOMANYMEETINHS
15:32:29 bauzas :)
15:57:20 kashyap bauzas: Drop the needless ones on the floor like hot potatoes. And embrace JOMO (joy of missing out)
15:59:06 bauzas hah
15:59:20 bauzas nah, I'm still digesting my Friday-late meeting :p
16:06:51 sean-k-mooney if the central site has direct connectivity to the edge site then you could just centralise the novnc proxy instnace
16:07:03 sean-k-mooney but ya i would have assumed you would want them at each edge site too
16:07:11 sean-k-mooney well each cell
16:07:15 sean-k-mooney not nessisarly edge
16:08:04 sean-k-mooney i was assuming you would run the novnc proxy on the same host as the cell conductor
16:09:19 dansmith sean-k-mooney: I think the change he's referring to was one to make the service only look in one cell, which means you can centralize services, but not unify them (i.e. you need multiple ports and endpoints, regardless of where they are)
16:13:53 sean-k-mooney ah i see
16:14:51 sean-k-mooney unless we moved this to the api db, or allowed the proxy to connect to multiple cell dbs im not sure how we would adress that
16:15:06 dansmith it used to I think, that's the point
16:15:24 dansmith IIRC we removed that ability when we eliminated the consoleauth service
16:15:55 melwitt it (nova-consoleauth) used to use memcache (one instance) to store token auths for the entire deployment
16:16:08 dansmith I imagine that we could add back in just the api db lookup part (like metadata) but I think the expectation was was to make it shard, which I think is a better design, personally
16:16:12 dansmith ah right
16:18:02 sean-k-mooney so really without some way to pass the cell mapping info to a web server there is really no way to use a reverse proxy to expose it over one port/endpoint now
16:18:12 melwitt but yeah, adding a console_auth_token_mappings table would be one way to make it so you only need one console proxy
16:18:46 sean-k-mooney how i would proably try and set it up personaly is have it use a subdomain per cell in the url and have a reverse proxy bind to the single port
16:18:50 dansmith oh, do we not get the instance id as well?
16:18:53 sean-k-mooney then have it delegate to the correct backedn
16:18:58 dansmith that would suck to have to add another mapping :/
16:18:59 melwitt no we don't, token only
16:19:10 dansmith well then I'm pretty -1 on that plan
16:19:19 sean-k-mooney we dont need to add anything in nova
16:19:29 dansmith you could scatter/gather to find it
16:19:47 sean-k-mooney to find the inial url
16:19:53 sean-k-mooney ya you could
16:19:54 dansmith sean-k-mooney: that's not necessary, because we given them the url from the proxy anyway
16:20:04 dansmith sean-k-mooney: belmiro just doesn't want that
16:20:26 dansmith presumably because he doesn't want to run multiple services and have multiple firewall rules
16:20:29 melwitt oh yeah, I guess he said as much already (scatter gather)
16:21:03 sean-k-mooney ya if he does not want to run multiple proxy instances
16:21:10 dansmith melwitt: ah, I hadn't even clicked the link, but yeah
16:21:52 melwitt we could add another config option! for choosing whether you want a central console auth
16:21:55 dansmith melwitt: I think we'd want that as a toggle
16:21:57 dansmith yeah
16:22:11 dansmith because if you want most efficient and least-shared, you don't want it doing that
16:22:18 melwitt yeah
18:47:16 belmoreira bauzas dansmith melwitt sean-k-mooney I see that you discussed the vncproxy topic. Sorry I needed to leave the office (end of the working day here).
18:48:03 sean-k-mooney no worries. did you add it to the ptg adgenda
18:48:47 belmoreira not yet
18:48:55 sean-k-mooney was dansmith correct when ne assumed you did not want to run multiple novnc proxy instance (1 per cell)
18:49:22 belmoreira let me explain my concern.
18:49:27 sean-k-mooney or are you just concerned about how many port you need to open in the fire wall
18:49:29 sean-k-mooney sure
18:49:42 belmoreira having the vncproxy per cell in theory is good, because we are sharding the service per cell. But it depends in the deployment... For deployments that only expose the console_url in the internal network is ok.
18:50:05 belmoreira However, in my case I need to expose the vncproxy externally. Having only one set os vncproxies allow me to open only one port in the external firewall and have only one console_url address masked by the load balancer.
18:50:27 belmoreira The current approach of having a vncproxy per cell, means that I will have a different console_url per cell. Mapping this with the LB I will need at least to have a different frontend per cell. If I do it per port is a lot of open ports...
18:50:38 sean-k-mooney well you could
18:50:49 sean-k-mooney you can use a reverse proxy instead of a loadblance
18:51:06 sean-k-mooney and expose only one port and have it route the reuest to the backend based on a partil path match
18:51:17 sean-k-mooney or using a subdomain per cell
18:51:53 sean-k-mooney so the reverse proxy is the only thing you open the firewall too and have it dispatch internally to the per cell proxy based on a part of the url
18:51:56 belmoreira true, but in those cases we are also exposing the cell architecture to the user
18:52:21 sean-k-mooney yes at least to the extend needed to match on the url
18:52:30 belmoreira the console_url will be different per cell
18:52:39 sean-k-mooney ya it would be
18:53:02 sean-k-mooney so if we allowed a singel vnc proxy to connect to any of the cell dbs that would be your preference
18:54:27 sean-k-mooney belmoreira: dansmith and melwitt can correct me if i get this wrong but i think what they were suggesting was add a config option to denote if the novnc proxy should connect to on celldb or multiple and having it do a scater gater request to each cell db in the case of multi cell mode
18:55:07 belmoreira having that possibility would be great. I already patch it and have it working in my test infrastructure
18:55:19 sean-k-mooney belmoreira: that would assume that the novnc proxy can actully connect to all the hyperviors in any cell but i belive that is the cause for your env right
18:56:06 belmoreira yes
18:58:02 belmoreira my ideal setup is to have a set of vncproxies that can connect to any hypervisor in the region
19:00:10 sean-k-mooney ya which this would give you
19:00:51 sean-k-mooney i think the main issue is from the consol url we dont know which instance it for without looking up the token and to do that we need to check the cell db
19:01:10 sean-k-mooney so in this case we would need to check multiple cell dbs which increase the load on the db since only one will have the token
19:01:30 sean-k-mooney so we would not want to do that by default but we could allow you to opt into it
19:02:31 belmoreira basically is something similar to the code that I pointed earlier
19:03:01 belmoreira I agree that this shouldn't be the default. Small deployments would not benefit from it
19:04:59 dansmith yeah that's what I meant.. scatter/gather to find the cell that a token is in is not very efficient, but I think the alternative is a lot more work for the few people that might want it
19:05:19 dansmith I'd definitely prefer solving that at the load balancer level with a url suffix or something like that,
19:05:44 dansmith so I think adding a config to scatter/gather is okay and let's not add a new mapping table just for this until/unless performance becomes an issue
19:08:42 belmoreira I think it would be ok. Consoles are not a popular api call. I can report after in terms of performance.
19:11:47 belmoreira thank you all. I can add this into the ptg agenda we need to discuss it more
19:13:46 belmoreira I need to leave now. thank you again
21:15:36 zigo What's the problem with https://review.opendev.org/c/openstack/nova/+/809759 ? (ie what's happening with this nova-tox-validate-backport check?)
21:57:22 artom zigo, it checks that the "source" hash is merged in an upstream branch
21:57:45 artom zigo, so if you cherry pick from wallaby to victoria, the victoria check will fail until the wallaby one merged
21:59:07 artom zigo, ah, no, I had the completely wrong idea without even opening the link
21:59:11 artom "Stable branch requires either cherry-pick -x headers or [stable-only] tag!" is what explains it
21:59:15 artom In the job output
22:28:08 clarkb sounds like someone needs to update the bot or fix the job. Cherrypicking the gitreview change doesn't make sense as it is different than masters
22:28:30 clarkb in my personal opinion it seems like overkill to make people explicitly tag stuff stable only
22:28:38 clarkb its clearly stable only and reviewers can see that why do we need CI to -1?
23:08:48 artom clarkb, it's really a commit message linter, if you think about it
23:09:11 artom I'm very *shrug* about it, though I'd tend to err on the side of "more linting" over "less linting"
#openstack-nova - 2021-09-21
01:34:21 opendevreview Eric Xie proposed openstack/os-vif master: Fix typos https://review.opendev.org/c/openstack/os-vif/+/810137
07:15:59 opendevreview Balazs Gibizer proposed openstack/nova stable/xena: [stable-only]Update .gitreview for stable/xena https://review.opendev.org/c/openstack/nova/+/809759
07:16:30 opendevreview Balazs Gibizer proposed openstack/nova stable/xena: [stable-only]Update TOX_CONSTRAINTS_FILE for stable/xena https://review.opendev.org/c/openstack/nova/+/809760

Earlier   Later