Docker部署ELK（四）-构建Logstash镜像

2017-05-07 本文已影响206人 Luomeng

Logstash

官网下载Logstash的tar包
解压到Dockerfile同级目录下。

修改配置文件：

# 绑定ip
http.host: "0.0.0.0"
# 日志存放路径
path.logs: /opt/product/data/logs

Dockerfile

FROM elkbase:v1.0
MAINTAINER Luomeng
VOLUME [ "/opt/product/data/" ]
ADD ./logstash-5.4.0 /opt/product/logstash-5.4.0
ADD logstash-simple.conf /opt/product/data/elk/
ADD init.sh /root/
RUN chmod +x /root/init.sh
EXPOSE 5044
EXPOSE 4560
EXPOSE 8080
ENTRYPOINT ["/root/init.sh"]
CMD ["/usr/sbin/init"]

init.sh

#!/bin/bash
export JAVA_HOME=/opt/product/tools/jdk1.8.0_131
export PATH=$JAVA_HOME/bin:$PATH
JAVA_OPTS="$JAVA_OPTS -Dfile.encoding=UTF8  -Duser.timezone=GMT+08"
# cd /opt/product/logstash-5.4.0/config/
# rm -rf logstash.yml
# cp /opt/product/data/elk/logstash.yml logstash.yml
/opt/product/logstash-5.4.0/bin/logstash -f /opt/product/data/logstash-simple.conf

logstash-simple.conf

input {        
    beats {
        port => "5044"
    }
}
output {
    elasticsearch {
        hosts => ["elasticsearch的ip:9200"]
        index => "logstash-tomcat-accesslog-%{+YYYY.MM.dd}"
    }          
}

文件存放在/Users/Luomeng/DockerProject/Logstash/data目录下。

构建镜像

docker build -t logstash:v1.0 .

启动容器

docker run -ti -p 5044:5044 -p 4560:4560 -p 18080:8080 logstash:v1.0 /bin/bash
查看日志是否成功启动并连接ES。

确认没问题后，用下面命令启动：

docker run --privileged --restart=always -d -ti -v /Users/Luomeng/DockerProject/Logstash/data:/opt/product/data -p 5044:5044 -p 4560:4560 -p 18080:8080 logstash:v1.0 /bin/bash

遇到问题:

我是在MacBook下安装的docker for mac，默认分配2G内存，先启动elasticsearch，然后启动logstash，es容器直接挂掉，docker stats命令监控了一下docker使用的资源，发现es启动就占用将近2G的内存，然后logstash启动，内存不足，es直接挂掉。解决办法：1.修改config目录下jvm.options配置文件中：-Xms1g -Xmx1g；2.增加Docker虚拟机的内存。
启动Logstash后，一直显示[2017-05-08T12:57:48,548][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x52d078 URL:http://127.0.0.1:9200/>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://127.0.0.1:9200/][Manticore::SocketException] Connection refused (Connection refused)"}，问题原因还在查找中...，原因找到了，比较2，直接在Logstash的Docker容器中链接http://127.0.0.1:9200/肯定连接失败啊，因为需要通过宿主机的IP地址加端口连接。
docker: Error response from daemon: driver failed programming external connectivity on endpoint hardcore_euclid (d00f191bc53e2d5c88ea32befc981678ab68b64e8bb4c58fd9da0c079acb4943): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 9100 -j DNAT --to-destination 192.168.0.4:9100 ! -i docker0: iptables: No chain/target/match by that name.