搭建ES集群
搭建环境
ElasticSearch cluster 实验环境 6台VM, RHEL 7.6。角色分配如下
master[0,1]
data[2,3,4]
client[5]
也可以不分配角色，那么默认一个 node会承担的角色会包括master, data, ingest, ml. 在生成环境中，为了把负载分开，会分配role。
[注] ml 代表 Machine Learning，这里没有用到。
- 主节点 master节点
node.master: true
node.data: false
node.ingest: false
xpack.ml.enabled: false
2）数据节点 data节点
node.master: false
node.data: true
node.ingest: false
xpack.ml.enabled: false
3）负载均衡节点 client节点
当一个节点既不配置为主节点也不配置为数据节点时，该节点只能处理路由请求，处理搜索，分发索引操作；
node.master: false
node.data: false
node.ingest: false
xpack.ml.enabled: false
环境准备
源码自带Java，而且版本比较高，不需要提前安装，如果已经安装了java，需要改路径，防止版本冲突。
切换到root用户，编辑 /etc/sysctl.conf文件并在文件末尾追加如下内容
vm.max_map_count=262144
让它立即生效
sysctl -p
停掉防火墙
systemctl stop firewalld
systemctl disable firewalld
编辑/etc/security/limits.conf文件，在文件末尾追加下面内容后重启机器
# vi /etc/security/limits.conf
* soft nofile 65535
* hard nofile 65535
安装ES
Elasticsearch需要使用非root用户来运行，所以我们使用普通用户来操作，首先我们将安装包保存在/es目录下。该目录属主和属组均为普通用户。
从官网下载linux源码包 https://www.elastic.co/downloads/elasticsearch
解压并修改配置文件
$ pwd
/es/elasticsearch-7.8.0
$ ls
bin config data jdk lib LICENSE.txt logs modules NOTICE.txt pid plugins README.asciidoc
编辑config/elasticsearch.yml文件
这里以一个master node 为例。
cluster.name: roy-es 集群名字，集群中所有的节点中该名称要相同
node.name: roy-es-0 节点名称：集群中每一个节点名字要不相同
network.host: _site_
discovery.seed_hosts: 集群中其它节点IP地址
- 10.0.2.4
- 10.0.2.7
- 10.0.2.9
- 10.0.2.5
- 10.0.2.8
- 10.0.2.6
cluster.initial_master_nodes:
- roy-es-0
- roy-es-1
node.master: true
node.data: false
node.ingest: false
xpack.ml.enabled: false
其它node role 照着这样的格式改。
启动ES
$ ./bin/elasticsearch
查看集群状态
查看集群状态
$ curl roy-es-0:9200/_cat/health?v
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1599718050 06:07:30 roy-es green 6 3 0 0 0 0 0 0 - 100.0%
从中可以看到，es cluster，名字叫 roy-es。状态是绿色，总共有6个 nodes，其中data node 有3个，shards 是0，说明还没有真正开始用。
集群节点信息
$ curl roy-es-hdp-master-0:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.0.2.4 45 25 0 0.06 0.03 0.05 mr * roy-es-0
10.0.2.6 41 25 0 0.01 0.03 0.05 r - roy-es-5
10.0.2.5 59 25 0 0.00 0.01 0.05 drt - roy-es-3
10.0.2.8 36 25 0 0.00 0.01 0.05 drt - roy-es-4
10.0.2.9 24 24 0 0.00 0.01 0.05 drt - roy-es-2
10.0.2.7 31 24 0 0.00 0.01 0.05 mr - roy-es-1
看到了6个节点，以及它们承担的角色。
列出所有的索引
$ curl roy-es-0:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
这个结果意味着，在我们的集群中没有任何索引。
创建一个索引
$ curl -XPUT 'roy-es-0:9200/customer?pretty'
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "customer"
}
再次查看
$ curl roy-es-0:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open customer 3Kdnlx7zT1-RuCspavGN1Q 1 1 0 0 416b 208b
索引已经多了一个。
再看看shards
$ curl roy-es-0:9200/_cat/health?v
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1599718767 06:19:27 roy-es green 6 3 2 1 0 0 0 0 - 100.0%
shards 从 0 变成了 2.
配置自启动脚本
新建文件/etc/init.d/elasticsearch
$ cat /etc/init.d/elasticsearch
#!/bin/bash
#
#description: elasticsearch
#processname: elasticsearch-7.8.0
export ES_HOME=/es/elasticsearch-7.8.0
case $1 in
start)
su myuser<<!
cd $ES_HOME
./bin/elasticsearch -d -p pid
exit
!
echo "elasticsearach is started"
;;
stop)
pid=`cat $ES_HOME/pid`
kill -9 $pid
echo "elasticsearch is stopped"
;;
restart)
pid=`cat $ES_HOME/pid`
kill -9 $pid
echo "elasticsearch is stopped"
sleep 1
su dcpuser<<!
cd $ES_HOME
./bin/elasticsearch -d -p pid
exit
!
echo "elasticsearch is started"
;;
\*)
echo "start|stop|restart"
;;
esac
exit 0
配置权限，设置自启动并启动
chmod 755 /etc/init.d/elasticsearch
service elasticsearch enable
service elasticsearch start
进阶：集群安全配置
集群安全配置，即X-Pack TLS加密通信配置
为集群创建认证机构，为节点颁发证书。
通用方式
参考Self Signed Certificate with Custom Root CA （过程略）
ES自带工具
或者参考ES自带工具来实现，步骤更简单，但对应的配置跟下面的例子不一样。Encrypting communications in Elasticsearch
# cd ES-HOME-DIR
bin/elasticsearch-certutil ca
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
mv elastic-certificates.p12 config
vi config/elasticsearch.yml
针对证书，在配置文件中做修改。
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: elastic-certificates.p12
这个配置文件比起通用模式的配置（下一节）要更简洁一些。
把这些证书 拷贝到各个节点
（过程略）
各节点修改配置
elasticsearch.yml中增加一下配置，启用x-pack安全组件，启用ssl加密通信，并且配置认证证书：
xpack.security.enabled: true
xpack.security.audit.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /es/elasticsearch-7.8.0/config/certs/elasticsearch.key
xpack.security.transport.ssl.certificate: /es/elasticsearch-7.8.0/config/certs/elasticsearch.crt
xpack.security.transport.ssl.certificate_authorities: /es/elasticsearch-7.8.0/config/certs/rootCA.crt
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: /es/elasticsearch-7.8.0/config/certs/elasticsearch.key
xpack.security.http.ssl.certificate: /es/elasticsearch-7.8.0/config/certs/elasticsearch.crt
xpack.security.http.ssl.certificate_authorities: /es/elasticsearch-7.8.0/config/certs/rootCA.crt
设置密码
通过设置访问密码，这是elastic用户和其他一些系统内置用户的密码
bin/elasticsearch-setup-passwords auto
把密码保存下来。
如果不配置Kibana，下面的部分可以略过。
There are built-in users that you can use for specific administrative purposes: apm_system, beats_system, elastic, kibana_system, logstash_system, and remote_monitoring_user.
Run the following command from the Elasticsearch directory:
./bin/elasticsearch-setup-passwords interactive
After you setup the password for the kibana_system built-in user, configure Kibana to use it.
For example, run the following commands to create the Kibana keystore and add the kibana_system built-in user and its password in secure settings:
./bin/kibana-keystore create
./bin/kibana-keystore add elasticsearch.username
./bin/kibana-keystore add elasticsearch.password
When prompted, specify the kibana_system built-in user and its password for these setting values. The settings are automatically applied when you start Kibana.
重启ES
通过用户名密码访问es服务
$ curl -k --user elastic:CjSEHp33bcCHo8wRV25g https://roy-es-0:9200/_cat/health?v
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1599814335 08:52:15 roy-es green 6 3 4 2 0 0 0 0 - 100.0%
$ curl -k --user elastic:CjSEHp33bcCHo8wRV25g https://roy-es-0:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.0.2.6 42 25 0 0.00 0.01 0.05 r - roy-es-5
10.0.2.4 7 62 23 0.66 0.26 0.13 mr - roy-es-0
10.0.2.7 63 26 0 0.00 0.01 0.05 mr * roy-es-1
10.0.2.9 44 25 0 0.00 0.01 0.05 drt - roy-es-2
10.0.2.8 65 26 0 0.05 0.03 0.05 drt - roy-es-4
10.0.2.5 38 25 0 0.00 0.01 0.05 drt - roy-es-3
参考
https://yuuuuuy.top/2019/03/10/Centos7%E6%90%AD%E5%BB%BAES%E9%9B%86%E7%BE%A4/
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html
https://www.elastic.co/guide/en/elastic-stack-get-started/7.9/get-started-elastic-stack.html
