Unified access indices from different clusters in Kibana #
Now there is such a demand, customers need to divide the data according to the business dimension, the index is split into three different clusters, to split the large cluster into multiple small clusters have many benefits, such as reduced coupling, bringing benefits to cluster availability and stability, but also to avoid the impact of a single business hotspot to affect other services, although splitting the cluster is a very common way to play, but the management is not so convenient, especially when querying data, it may be need to access the three sets of clusters separately APIs, even to switch between three different sets of Kibana to access the cluster’s data, is there a way to seamlessly unite them together?
A gateway! #
The answer is naturally yes, by switching the Elasticsearch address of Kibana to the address of the INFINI Gateway, we can intelligently route requests according to the index, that is, when accessing different business indexes, they will be intelligently routed to different clusters, as shown in the following figure:
Above, we have three different indexes：
- apm-*
- erp-*
- mall-*
Each corresponds to three different sets of Elasticsearch clusters:
- ES1-APM
- ES2-ERP
- ES3-MALL
Now let’s see how to configure the INFINI Gateway to meet this business requirements:
Configure clusters #
First configure the connection information for the three clusters.
elasticsearch:
- name: es1-apm
enabled: true
endpoints:
- http://192.168.3.188:9206
- name: es2-erp
enabled: true
endpoints:
- http://192.168.3.188:9207
- name: es3-mall
enabled: true
endpoints:
- http://192.168.3.188:9208
Configure Flow #
then, we define three flows that are used to access three different Elasticsearch clusters, as shown below:
flow:
- name: es1-flow
filter:
- elasticsearch:
elasticsearch: es1-apm
- name: es2-flow
filter:
- elasticsearch:
elasticsearch: es2-erp
- name: es3-flow
filter:
- elasticsearch:
elasticsearch: es3-mall
Then define a flow for path rule and forwarding, as follows:
- name: default-flow
filter:
- switch:
remove_prefix: false
path_rules:
- prefix: "apm-"
flow: es1-flow
- prefix: "erp-"
flow: es2-flow
- prefix: "mall-"
flow: es3-flow
- flow: #default flow
flows:
- es1-flow
Match different indexes based on the index prefix in the request path and forward to different flows.
Configure Router #
Next, we define the routing information as follows:
router:
- name: my_router
default_flow: default-flow
Point to the default flow defined above to unify the processing of requests.
Configure Entrypoint #
Finally, we define a service that listening on port 8000 to provide unified access to Kibana, as follows:
entry:
- name: es_entry
enabled: true
router: my_router
max_concurrency: 10000
network:
binding: 0.0.0.0:8000
Full Configuration #
The final complete configuration is as follows:
path.data: data
path.logs: log
entry:
- name: es_entry
enabled: true
router: my_router
max_concurrency: 10000
network:
binding: 0.0.0.0:8000
flow:
- name: default-flow
filter:
- switch:
remove_prefix: false
path_rules:
- prefix: "apm-"
flow: es1-flow
- prefix: "erp-"
flow: es2-flow
- prefix: "mall-"
flow: es3-flow
- flow: #default flow
flows:
- es1-flow
- name: es1-flow
filter:
- elasticsearch:
elasticsearch: es1-apm
- name: es2-flow
filter:
- elasticsearch:
elasticsearch: es2-erp
- name: es3-flow
filter:
- elasticsearch:
elasticsearch: es3-mall
router:
- name: my_router
default_flow: default-flow
elasticsearch:
- name: es1-apm
enabled: true
endpoints:
- http://192.168.3.188:9206
- name: es2-erp
enabled: true
endpoints:
- http://192.168.3.188:9207
- name: es3-mall
enabled: true
endpoints:
- http://192.168.3.188:9208
Start Gateway #
Start the gateway as follows:
➜ gateway git:(master) ✗ ./bin/gateway -config sample-configs/elasticsearch-route-by-index.yml
___ _ _____ __ __ __ _
/ _ \ /_\ /__ \/__\/ / /\ \ \/_\ /\_/\
/ /_\///_\\ / /\/_\ \ \/ \/ //_\\\_ _/
/ /_\\/ _ \/ / //__ \ /\ / _ \/ \
\____/\_/ \_/\/ \__/ \/ \/\_/ \_/\_/
[GATEWAY] A light-weight, powerful and high-performance elasticsearch gateway.
[GATEWAY] 1.0.0_SNAPSHOT, 2022-04-20 08:23:56, 2023-12-31 10:10:10, 51650a5c3d6aaa436f3c8a8828ea74894c3524b9
[04-21 13:41:21] [INF] [app.go:174] initializing gateway.
[04-21 13:41:21] [INF] [app.go:175] using config: /Users/medcl/go/src/infini.sh/gateway/sample-configs/elasticsearch-route-by-index.yml.
[04-21 13:41:21] [INF] [instance.go:72] workspace: /Users/medcl/go/src/infini.sh/gateway/data/gateway/nodes/c9bpg0ai4h931o4ngs3g
[04-21 13:41:21] [INF] [app.go:283] gateway is up and running now.
[04-21 13:41:21] [INF] [api.go:262] api listen at: http://0.0.0.0:2900
[04-21 13:41:21] [INF] [reverseproxy.go:255] elasticsearch [es1-apm] hosts: [] => [192.168.3.188:9206]
[04-21 13:41:21] [INF] [reverseproxy.go:255] elasticsearch [es2-erp] hosts: [] => [192.168.3.188:9207]
[04-21 13:41:21] [INF] [reverseproxy.go:255] elasticsearch [es3-mall] hosts: [] => [192.168.3.188:9208]
[04-21 13:41:21] [INF] [actions.go:349] elasticsearch [es2-erp] is available
[04-21 13:41:21] [INF] [actions.go:349] elasticsearch [es1-apm] is available
[04-21 13:41:21] [INF] [entry.go:312] entry [es_entry] listen at: http://0.0.0.0:8000
[04-21 13:41:21] [INF] [module.go:116] all modules are started
[04-21 13:41:21] [INF] [actions.go:349] elasticsearch [es3-mall] is available
[04-21 13:41:55] [INF] [reverseproxy.go:255] elasticsearch [es1-apm] hosts: [] => [192.168.3.188:9206]
After the gateway successfully started, you can access the target Elasticsearch cluster through the gateway’s IP+ port 8000.
Testing #
Let’s start with the API access test, as follows:
➜ ~ curl http://localhost:8000/apm-2022/_search -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /apm-2022/_search HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 21 Apr 2022 05:45:44 GMT
< content-type: application/json; charset=UTF-8
< Content-Length: 162
< X-elastic-product: Elasticsearch
< X-Backend-Cluster: es1-apm
< X-Backend-Server: 192.168.3.188:9206
< X-Filters: filters->elasticsearch
<
* Connection #0 to host localhost left intact
{"took":142,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}}%
You can see that apm-2022 points to the backend ES1-APM cluster.
To continue testing, access to the ERP index as follows:
➜ ~ curl http://localhost:8000/erp-2022/_search -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /erp-2022/_search HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 21 Apr 2022 06:24:46 GMT
< content-type: application/json; charset=UTF-8
< Content-Length: 161
< X-Backend-Cluster: es2-erp
< X-Backend-Server: 192.168.3.188:9207
< X-Filters: filters->switch->filters->elasticsearch->skipped
<
* Connection #0 to host localhost left intact
{"took":12,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}}%
Great!
Let’s continue testing, access to the mall index as follows:
➜ ~ curl http://localhost:8000/mall-2022/_search -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /mall-2022/_search HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 21 Apr 2022 06:25:08 GMT
< content-type: application/json; charset=UTF-8
< Content-Length: 134
< X-Backend-Cluster: es3-mall
< X-Backend-Server: 192.168.3.188:9208
< X-Filters: filters->switch->filters->elasticsearch->skipped
<
* Connection #0 to host localhost left intact
{"took":8,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}%
Perfect!
Another Option #
Besides using the switch filter, it is possible to use the path rules of the router itself, as shown in the following example configuration:
flow:
- name: default_flow
filter:
- echo:
message: "hello world"
- name: mall_flow
filter:
- echo:
message: "hello mall indices"
- name: apm_flow
filter:
- echo:
message: "hello apm indices"
- name: erp_flow
filter:
- echo:
message: "hello erp indices"
router:
- name: my_router
default_flow: default_flow
rules:
- method:
- "*"
pattern:
- "/apm-{suffix:.*}/"
- "/apm-{suffix:.*}/{any:.*}"
flow:
- apm_flow
- method:
- "*"
pattern:
- "/erp-{suffix:.*}/"
- "/erp-{suffix:.*}/{any:.*}"
flow:
- erp_flow
- method:
- "*"
pattern:
- "/mall-{suffix:.*}/"
- "/mall-{suffix:.*}/{any:.*}"
flow:
- mall_flow
INFINI Gateway has many powerful features, and there are many ways to achieve your need, go explore it by yourself.
Modify Kibana Configuration #
Modify the Kibana configuration file kibana.yml, replace the address of Elasticsearch with the gateway address (HTTP: 192.168.3.200/8000), as shown below:
elasticsearch.hosts: ["http://192.168.3.200:8000"]
Restart the Kibana。
Visit Kibana #
As you can see, in the Kibana developer tool we can already perform read and write operations from three different clusters as if it were one cluster.
Conclusion #
Through the INFINI Gateway, we can be very flexible for online traffic editing, dynamic combine requests of different cluster operations together on the fly.