ElasticSearch-5.6.3安装
2017-10-30 本文已影响49人
Kent_Yao
下载
# 链接为个人私服
wget hzadg-jenkins.server.163.org:1234/elasticsearch-5.6.3.tar.gz
tar zxf elasticsearch-5.6.3.tar.gz
cd elasticsearch-5.6.3
配置
创建数据盘
sudo -iu root
mkdir /mnt/dfs/0/es /mnt/dfs/1/es /mnt/dfs/2/es /mnt/dfs/3/es /mnt/dfs/4/es /mnt/dfs/5/es /mnt/dfs/6/es /mnt/dfs/7/es /mnt/dfs/8/es /mnt/dfs/9/es /mnt/dfs/10/es /mnt/dfs/11/es
chown hzyaoqin /mnt/dfs/0/es /mnt/dfs/1/es /mnt/dfs/2/es /mnt/dfs/3/es /mnt/dfs/4/es /mnt/dfs/5/es /mnt/dfs/6/es /mnt/dfs/7/es /mnt/dfs/8/es /mnt/dfs/9/es /mnt/dfs/10/es /mnt/dfs/11/es
exit
配置
vi config/elasticsearch.yml
cluster.name: spark-es
node.name: bd23
node.master: true
node.data: true
path.data: /mnt/dfs/0/es/data,/mnt/dfs/1/es/data,/mnt/dfs/2/es/data,/mnt/dfs/3/es/data,/mnt/dfs/4/es/data,/mnt/dfs/5/es/data,/mnt/dfs/6/es/data,/mnt/dfs/7/es/data,/mnt/dfs/9/es/data,/mnt/dfs/10/es/data,/mnt/dfs/11/es/data
bootstrap.memory_lock: false
启动
take 1... failed
bin/elasticsearch
Elasticsearch requires at least Java 8 but your Java version from /usr/bin/java does not meet this requirement
# 升级jdk
export JAVA_HOME=/usr/jdk64/jdk1.8.0_77
take 2...sucess
bin/elasticsearch
[2017-10-30T10:44:05,478][WARN ][o.e.b.JNANatives ] unable to install syscall filter:
java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in
at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:351) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:639) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:258) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:111) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:195) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.6.3.jar:5.6.3]
linux内核过低，可忽略...
加入现有集群
配置网络参数
vi config/elasticsearch.yml
network.host: <ip of this node>
discovery.zen.ping.unicast.hosts: ["ip of existing node", "..."]
优化jvm参数
vi config/jvm.options
-Xms20g
-Xmx20g
再次启动
take 1...failed
bin/elasticsearch
ERROR: [3] bootstrap checks failed
[1]: max number of threads [1024] for user [hzyaoqin] is too low, increase to at least [2048]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[3]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
sudo -iu root
vi /etc/security/limits.conf
hzyaoqin soft nofile 65536
hzyaoqin hard nofile 131072
hzyaoqin soft nproc 2048
hzyaoqin hard nproc 4096
vi /etc/security/limits.d/90-nproc.conf
* soft nproc 1024 -> 2048
vi /etc/sysctl.conf
vm.max_map_count=655360
sysctl -p
exit
vi config/elasticsearch.yml
bootstrap.system_call_filter: false
take2...failed
bin/elasticsearch
ERROR: [1] bootstrap checks failed
[1]: max number of threads [1024] for user [hzyaoqin] is too low, increase to at least [2048]
weird...设置没生效，重启大法，重新登陆下...
take 3..sucess
[2017-10-30T11:25:33,080][INFO ][o.e.n.Node ] [bd21] initializing ...
[2017-10-30T11:25:33,669][INFO ][o.e.e.NodeEnvironment ] [bd21] using [11] data paths, mounts [[/mnt/dfs/3 (/dev/sde1), /mnt/dfs/2 (/dev/sdd1), /mnt/dfs/5 (/dev/sdg1), /mnt/dfs/0 (/dev/sdb1), /mnt/dfs/6 (/dev/sdh1), /mnt/dfs/11 (/dev/sdm1), /mnt/dfs/9 (/dev/sdk1), /mnt/dfs/10 (/dev/sdl1), /mnt/dfs/1 (/dev/sdc1), /mnt/dfs/4 (/dev/sdf1), /mnt/dfs/7 (/dev/sdi1)]], net usable_space [37.3tb], net total_space [39.3tb], spins? [possibly], types [ext4]
[2017-10-30T11:25:33,670][INFO ][o.e.e.NodeEnvironment ] [bd21] heap size [19.7gb], compressed ordinary object pointers [true]
[2017-10-30T11:25:33,678][INFO ][o.e.n.Node ] [bd21] node name [bd21], node ID [0ljTMuGtRE-mSmOmEnUBzw]
[2017-10-30T11:25:33,678][INFO ][o.e.n.Node ] [bd21] version[5.6.3], pid[7156], build[1a2f265/2017-10-06T20:33:39.012Z], OS[Linux/2.6.32-431.el6.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_77/25.77-b03]
[2017-10-30T11:25:33,679][INFO ][o.e.n.Node ] [bd21] JVM arguments [-Xms20g, -Xmx20g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/home/hzyaoqin/elasticsearch-5.6.3]
[2017-10-30T11:25:34,360][INFO ][o.e.p.PluginsService ] [bd21] loaded module [aggs-matrix-stats]
[2017-10-30T11:25:34,360][INFO ][o.e.p.PluginsService ] [bd21] loaded module [ingest-common]
[2017-10-30T11:25:34,360][INFO ][o.e.p.PluginsService ] [bd21] loaded module [lang-expression]
[2017-10-30T11:25:34,360][INFO ][o.e.p.PluginsService ] [bd21] loaded module [lang-groovy]
[2017-10-30T11:25:34,360][INFO ][o.e.p.PluginsService ] [bd21] loaded module [lang-mustache]
[2017-10-30T11:25:34,360][INFO ][o.e.p.PluginsService ] [bd21] loaded module [lang-painless]
[2017-10-30T11:25:34,360][INFO ][o.e.p.PluginsService ] [bd21] loaded module [parent-join]
[2017-10-30T11:25:34,361][INFO ][o.e.p.PluginsService ] [bd21] loaded module [percolator]
[2017-10-30T11:25:34,361][INFO ][o.e.p.PluginsService ] [bd21] loaded module [reindex]
[2017-10-30T11:25:34,361][INFO ][o.e.p.PluginsService ] [bd21] loaded module [transport-netty3]
[2017-10-30T11:25:34,361][INFO ][o.e.p.PluginsService ] [bd21] loaded module [transport-netty4]
[2017-10-30T11:25:34,361][INFO ][o.e.p.PluginsService ] [bd21] no plugins loaded
[2017-10-30T11:25:35,504][INFO ][o.e.d.DiscoveryModule ] [bd21] using discovery type [zen]
[2017-10-30T11:25:35,939][INFO ][o.e.n.Node ] [bd21] initialized
[2017-10-30T11:25:35,940][INFO ][o.e.n.Node ] [bd21] starting ...
[2017-10-30T11:25:36,064][INFO ][o.e.t.TransportService ] [bd21] publish_address {10.120.181.32:9300}, bound_addresses {10.120.181.32:9300}
[2017-10-30T11:25:36,073][INFO ][o.e.b.BootstrapChecks ] [bd21] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-10-30T11:25:39,445][INFO ][o.e.c.s.ClusterService ] [bd21] detected_master {bd23}{9mJYOtrcSTyTM6-yalEMVg}{G1cMAACGRQCJ_O6l3NacGg}{10.120.181.34}{10.120.181.34:9300}, added {{bd23}{9mJYOtrcSTyTM6-yalEMVg}{G1cMAACGRQCJ_O6l3NacGg}{10.120.181.34}{10.120.181.34:9300},{bd22}{TmZ_Is_GR9aFmHuPA10OrA}{vsgerZzsQcS9drNPCz8bxw}{10.120.181.33}{10.120.181.33:9300},}, reason: zen-disco-receive(from master [master {bd23}{9mJYOtrcSTyTM6-yalEMVg}{G1cMAACGRQCJ_O6l3NacGg}{10.120.181.34}{10.120.181.34:9300} committed version [6]])
[2017-10-30T11:25:40,047][INFO ][o.e.h.n.Netty4HttpServerTransport] [bd21] publish_address {10.120.181.32:9200}, bound_addresses {10.120.181.32:9200}
[2017-10-30T11:25:40,047][INFO ][o.e.n.Node ] [bd21] started
1 堆内存修改生效 -Xms20g, -Xmx20g
2 detected_master {bd23}
后台运行
take 1... sucess
bin/elasticsearch -d -p pid
