ElasticSearch使用SearchGuard插件
2017-11-30 本文已影响22人
3c69b7c624d9
背景
关于最近一段时间es的勒索新闻给广大开源组件使用者带来一丝不安。全球 500 亿条数据被 Elasticsearch 勒索者删除
基于这种情况，阿里云强制阿里云用户使用es组件的必须加强安全，否则直接关停服务器 ElasticSearch 多个高危漏洞及加固
阿里云提供方案为 shield插件ElasticSearch提供很多的安全插件供用户方便安装使用，在这里推荐1款安全插件：
Shield是Elastic公司为ElasticSearch开发的一个安全插件。在安装此插件后，Shield会拦截所有对ElasticSearch的请求，并加上认证与加密，保障ElasticSearch及相关系统的安全性。
Shield是商业插件，需要ElasticSearch的商业许可。第一次安装许可的时候，会提供30天的免费试用权限。30天后，Shield将会屏蔽clusterhealth, cluster stats, index stats这几个API，其余功能不受影响。
使用Shield可以定义一系列已知的用户，并用其认证用户请求。这些用户存在于抽象的“域”中。一个域可能是下面几种类型：
因此毫不犹豫选择了shield插件，结果一个月后
181918_zoSU_871390.png
插件过期了，几经波折找到原因Kibana won’t run with Shield LicenseExpiredException
Kibana requires Elasticsearch health and monitoring permissions to make sure Elasticsearch backend is available. So you’ll need to get a valid license to use Kibana with Shield.
方案
使用searchGuard方案（好吧，又可以省1600 $/year/cluster ）
github地址如下https://github.com/floragunncom
这里使用的是 elasticsearch-2.3.4
安装 ES 插件
<table>
<tbody>
<tr>
<td> <p><code>bin/plugin install -b com.floragunn/search-guard-ssl/</code><code>2.3</code><code>.</code><code>4.12</code></p> <p><code>bin/plugin install -b com.floragunn/search-guard-</code><code>2</code><code>/</code><code>2.3</code><code>.</code><code>4.4</code></p> </td>
</tr>
</tbody>
</table>
clone ssl project
<table>
<tbody>
<tr>
<td> <p><code>git clone https:</code><code>//github.com/floragunncom/search-guard-ssl.git</code></p> </td>
</tr>
</tbody>
</table>
修改脚本vim example.sh
#!/bin/bash
set -e
./clean.sh
./gen_root_ca.sh password password
./gen_node_cert.sh 0 password password
./gen_client_node_cert.sh admin password password
cp truststore.jks node-0-keystore.jks /usr/local/elasticsearch-2.3.4/config/
cp truststore.jks admin-keystore.jks /usr/local/elasticsearch-2.3.4/plugins/search-guard-2/sgconfig/
配置es配置
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: f6car
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# node.name: node-1
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /data/elastic-data/data
#
# Path to log files:
#
path.logs: /data/elastic-logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# bootstrap.mlockall: true
#
# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
# available on the system and that the owner of the process is allowed to use this limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#
# Set a custom port for HTTP:
#
# http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
#
# Require explicit names when deleting indices:
#
# action.destructive_requires_name: true
network.host: XXX
transport.tcp.port: 9300
transport.tcp.compress: true
http.port: 9200
#############################################################################################
# SEARCH GUARD #
# Configuration #
#############################################################################################
searchguard.enable: true
searchguard.authcz.admin_dn:
- CN=admin, OU=client, O=client, L=Test, C=DE
#############################################################################################
# SEARCH GUARD SSL #
# Configuration #
#############################################################################################
#############################################################################################
# Transport layer SSL #
# #
#############################################################################################
# Enable or disable node-to-node ssl encryption (default: true)
searchguard.ssl.transport.enabled: true
# JKS or PKCS12 (default: JKS)
#searchguard.ssl.transport.keystore_type: PKCS12
# Relative path to the keystore file (mandatory, this stores the server certificates), must be placed under the config/ dir
searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks
# Alias name (default: first alias which could be found)
#searchguard.ssl.transport.keystore_alias: my_alias
# Keystore password (default: changeit)
searchguard.ssl.transport.keystore_password: password
# JKS or PKCS12 (default: JKS)
#searchguard.ssl.transport.truststore_type: PKCS12
# Relative path to the truststore file (mandatory, this stores the client/root certificates), must be placed under the config/ dir
searchguard.ssl.transport.truststore_filepath: truststore.jks
# Alias name (default: first alias which could be found)
#searchguard.ssl.transport.truststore_alias: my_alias
# Truststore password (default: changeit)
searchguard.ssl.transport.truststore_password: password
# Enforce hostname verification (default: true)
searchguard.ssl.transport.enforce_hostname_verification: false
# If hostname verification specify if hostname should be resolved (default: true)
searchguard.ssl.transport.resolve_hostname: false
# Use native Open SSL instead of JDK SSL if available (default: true)
searchguard.ssl.transport.enable_openssl_if_available: false
运行es后执
./plugins/search-guard-2/tools/sgadmin.sh -cn f6car -h xxx -cd plugins/search-guard-2/sgconfig -ks plugins/search-guard-2/sgconfig/admin-keystore.jks -kspass password -ts plugins/search-guard-2/sgconfig/truststore.jks -tspass password -nhnv
显示如下
Clustername: f6car
Clusterstate: YELLOW
Number of nodes: 1
Number of data nodes: 1
Index does already exists
Populate config from /usr/local/elasticsearch-2.3.4/plugins/search-guard-2/sgconfig
Will update 'config' with plugins/search-guard-2/sgconfig/sg\_config.yml
SUCC Configuration for 'config' created or updated
Will update 'roles' with plugins/search-guard-2/sgconfig/sg\_roles.yml
SUCC Configuration for 'roles' created or updated
Will update 'rolesmapping' with plugins/search-guard-2/sgconfig/sg\_roles\_mapping.yml
SUCC Configuration for 'rolesmapping' created or updated
Will update 'internalusers' with plugins/search-guard-2/sgconfig/sg\_internal\_users.yml
SUCC Configuration for 'internalusers' created or updated
Will update 'actiongroups' with plugins/search-guard-2/sgconfig/sg\_action\_groups.yml
SUCC Configuration for 'actiongroups' created or updated
Done with success
修改logstash配置文件
output {
elasticsearch {
hosts => ["XXXX"]
index => "logstash-sql-{type}-%{+YYYY.MM.dd}"
document_type => "%{type}"
template_overwrite => true
user => "logstash"
password => "password"
}
}
修改kibana配置文件
elasticsearch.requestTimeout: 60000
# Time in milliseconds for Elasticsearch to wait for responses from shards.
# Set to 0 to disable.
# elasticsearch.shardTimeout: 0
# Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying
# elasticsearch.startupTimeout: 5000
# Set the path to where you would like the process id file to be created.
# pid.file: /var/run/kibana.pid
# If you would like to send the log output to a file you can set the path below.
logging.dest: /usr/local/kibana-4.5.3-linux-x64/logs/kibana.log
kibana_elasticsearch_username: f6car
kibana_elasticsearch_password: password
如存在既存数据，直接将既存数据文件夹重命名为kibana对应的用户(此处为f6car)
<table>
<tbody>
<tr>
<td> <p><code>mv elastic f6car</code></p> </td>
</tr>
</tbody>
</table>
如上，可以完成searchguard与elk的整合
