2020-12-08 Filebeat快速入门
2020-12-08 本文已影响0人
四喜临门
Filebeat简介
Filebeat是一款基于libbeat的轻量级日志采集器,用于监控日志文件、目录,收集事件并将其转发至Elasticsearch或Logstash。
工作基本原理
Filebeat支持多路输入。每个文件启动一个对应的harvester用于监控文件变化并将数据发送至libbeat。libbeat将日志数据汇聚,并发送至输出。
快速安装入门
linux版本下载地址
1. 安装
直接放到所需位置解压即可
tar xzvf filebeat-7.10.0-linux-x86_64.tar.gz
2. 配置连接到Elastic Static
filebeat.yml文件
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
# Array of hosts to connect to. ES集群的host和端口
hosts: ["localhost:9200"]
# Protocol - either `http` (default) or `https`. 如果是https协议需要打开切换协议头
#protocol: "https"
# Authentication credentials - either API key or username/password.
# 如果需要输入API key或者用户名密码则开启配置
#api_key: "id:api_key"
#username: "elastic"
#password: "changeme"
如果要使用kibana的默认dashboards
# =================================== Kibana ===================================
# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
#如果kibana部署在本机上就不用配置
host: "localhost:5601"
# Kibana Space ID
# ID of the Kibana Space into which the dashboards should be loaded. By default,
# the Default Space will be used.
#space.id:
3. 启动和配置数据采集模块
1. 确认需要启动的模块,linux下执行:
./filebeat modules list
列出原生的采集模块,如果没有所需的模块,还可以手动配置input section
2. 启动需要的模块,以elasticsearch为例
./filebeat modules enable elasticsearch
3. 在modules.d目录下,修改已启动模块的配置,elasticsearch.yml
# Module: elasticsearch
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.10/filebeat-module-elasticsearch.html
- module: elasticsearch
# Server log
server:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths:
- /u01/elastic/logs/log-application.log
gc:
enabled: false
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
#var.paths:
audit:
enabled: false
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
#var.paths:
slowlog:
enabled: false
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
#var.paths:
deprecation:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths:
- /u01/elastic/logs/log-application_deprecation.log
4. 上传ES索引模板,配置kibana dashboards
./filebeat setup -e
5. 启动filebeat
sudo ./filebeat -e
查看kibana界面
ES的日志大概就是这个样子
kibana界面