Elasticsearch学习笔记（一）
2021-08-31 本文已影响0人
dev_winner
- The Elastic Stack，包括 Elasticsearch、Kibana、Beats 和 Logstash（也称为 ELK Stack），能够安全可靠地获取任何来源、任何格式的数据，然后实时地对数据进行搜索、分析和可视化。
- Elaticsearch（ES）是一个开源的高扩展的分布式全文搜索引擎，是整个 Elastic
Stack 技术栈的核心。它可以近乎实时的存储、检索数据；本身扩展性很好，可以扩展到上百台服务器，处理 PB 级别的数据。 - Lucene 是 Apache 软件基金会 Jakarta 项目组的一个子项目，提供了一个简单却强大的应用程式接口，能够做全文索引和搜寻。但它只是一个提供全文搜索功能类库的核心工具包，而真正使用它还需要一个完善的服务框架搭建起来进行应用。
- 目前市面上流行的搜索引擎软件，主流的就两款：Elasticsearch 和 Solr，这两款都是基于 Lucene 搭建的，可以独立部署启动的搜索引擎服务软件。由于内核相同，所以两者除了服务器安装、部署、管理、集群以外，对于数据的操作 修改、添加、保存、查询等等都十分类似。
- 应用实例：GitHub、维基百科、SoundCloud、百度、新浪、阿里、Stack Overflow等。
- 安装包下载地址：传送门，下载window或其它版本解压后进入bin目录，点击
elasticsearch.bat文件启动 ES 服务。在此之前，先安装对应的JDK版本。若不能启动，可能是分配的内存不足，则可修改配置文件：config/jvm.options。 - 注意：
9300端口为 Elasticsearch 集群间组件的通信端口，9200端口为浏览器访问的 http协议 RESTful端口。
es服务启动成功
Elasticsearch与MySQL的类比
- 注意：这里 Types 的概念已经逐渐被弱化，Elasticsearch 6.X 中，一个 Index 下已经只能包含一个type，Elasticsearch 7.X 中，Type 的概念已经被删除了。
- 创建索引：
http://127.0.0.1:9200/playing，method：put
# response
{
"acknowledged": true, # 操作成功
"shards_acknowledged": true, # 分片操作成功
"index": "playing" #索引名称
}
- 注意：创建索引库的分片数默认为1片，在ES 7.0.0 之前的版本中，默认为 5 片。
- 由于
put操作具有幂等性，若重复添加索引，则会返回错误信息：
{
"error": {
"root_cause": [
{
"type": "resource_already_exists_exception",
"reason": "index [playing/2Rmw7bI-SZ-ntKHIbJGDoA] already exists",
"index_uuid": "2Rmw7bI-SZ-ntKHIbJGDoA",
"index": "playing"
}
],
"type": "resource_already_exists_exception",
"reason": "index [playing/2Rmw7bI-SZ-ntKHIbJGDoA] already exists",
"index_uuid": "2Rmw7bI-SZ-ntKHIbJGDoA",
"index": "playing"
},
"status": 400
}
-
post操作没有幂等性，意味着两次操作结果可能不一样，这种操作是不允许的：
{
"error": "Incorrect HTTP method for uri [/playing] and method [POST], allowed: [HEAD, DELETE, GET, PUT]",
"status": 405
}
- 查看某个索引：
http://127.0.0.1:9200/playing，method：get
{
"playing": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"creation_date": "1630457890587",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "2Rmw7bI-SZ-ntKHIbJGDoA",
"version": {
"created": "7060099"
},
"provided_name": "playing"
}
}
}
}
- 查看所有索引：
http://127.0.0.1:9200/_cat/indices?v，method：get
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open playing 2Rmw7bI-SZ-ntKHIbJGDoA 1 1 0 0 283b 283b
| 表头 | 含义 |
|---|---|
| health | 当前服务器健康状态：green(集群完整) yellow(单点正常、集群不完整) red(单点不正常) |
| status | 索引打开、关闭状态 |
| index | 索引名 |
| uuid | 索引统一编号 |
| pri | 主分片数量 |
| rep | 副本数量 |
| docs.count | 可用文档数量 |
| docs.deleted | 文档删除状态（逻辑删除） |
| store.size | 主分片和副分片整体占空间大小 |
| pri.store.size | 主分片占空间大小 |
- 删除某个索引：
http://127.0.0.1:9200/playing，method：delete
{
"acknowledged": true
}
- 重复访问索引时，服务器返回响应：索引不存在
{
"error": {
"root_cause": [
{
"type": "index_not_found_exception",
"reason": "no such index [playing]",
"resource.type": "index_or_alias",
"resource.id": "playing",
"index_uuid": "_na_",
"index": "playing"
}
],
"type": "index_not_found_exception",
"reason": "no such index [playing]",
"resource.type": "index_or_alias",
"resource.id": "playing",
"index_uuid": "_na_",
"index": "playing"
},
"status": 404
}
- 创建文档：
http://127.0.0.1:9200/playing/_doc，method：post，请求体内容：
{
"title":"一加phone",
"category":"手机",
"price":3999.00
}
- 服务器返回内容：
{
"_index": "playing",
"_type": "_doc", # 类型-文档
"_id": "ua3ynnsB4tbB67MkkEHa", #唯一标识
"_version": 1, # 版本
"result": "created", # 表示创建成功
"_shards": {
"total": 2, # 分片 - 总数
"successful": 1, # 分片 - 成功
"failed": 0 #分片 - 失败
},
"_seq_no": 0,
"_primary_term": 1
}
- 若采用
put操作，则返回的是不允许这样操作的提示，因为同样的post请求发送出去，都返回不用_id值唯一标识的文档信息，这就说明post不是幂等性的。
{
"error": "Incorrect HTTP method for uri [/playing/_doc] and method [PUT], allowed: [POST]",
"status": 405
}
- 由于没有指定数据唯一性标识（ID），默认情况下，ES 服务器会随机生成一个。若想要自定义唯一性标识，则需要在创建时指定：
http://127.0.0.1:9200/playing/_doc/1，method：post
{
"_index": "playing",
"_type": "_doc",
"_id": "1", # 自定义的id值
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 1,
"_primary_term": 1
}
- 多次发送上面的请求，会发现返回的结果中版本号、结果和序列号改变了：
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 2,
"_primary_term": 1
}
- 注意：若增加数据时明确数据主键，则请求方式也可以为
PUT，更改上一个操作的method请求为put，返回结果如下：
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 3, # 改变
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 3, # 改变
"_primary_term": 1
}
- 查询文档：
http://127.0.0.1:9200/playing/_doc/1，method：GET
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 3,
"_seq_no": 3,
"_primary_term": 1,
"found": true, # 表示找到
"_source": { # 文档源信息
"title": "一加phone",
"category": "手机",
"price": 3999
}
}
- 若查询一个不存在的文档（id=12），则返回以下内容：
{
"_index": "playing",
"_type": "_doc",
"_id": "12",
"found": false
}
- 查询某个索引下的全部文档：
http://127.0.0.1:9200/playing/_search，method：GET
{
"took": 2763,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "ua3ynnsB4tbB67MkkEHa",
"_score": 1,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 3999
}
},
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": 1,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 3999
}
}
]
}
}
- 修改文档：①完全覆盖：
http://127.0.0.1:9200/playing/_doc/1，method：PUT，请求体内容：
{
"title":"一加phone",
"category":"手机",
"price":4999.00
}
- 服务器返回内容：
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 4,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 4,
"_primary_term": 1
}
- 查询该文档：
http://127.0.0.1:9200/playing/_doc/1，method：GET，可以发现版本号和序列号都发生改变了。
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 4,
"_seq_no": 4,
"_primary_term": 1,
"found": true,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 4999
}
}
- 局部更新：
http://127.0.0.1:9200/playing/_update/1，method：POST，请求体内容：
{
"doc": {
"title": "小米phone"
}
}
- 服务器返回内容：
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 5,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 5,
"_primary_term": 1
}
- 查询该文档：
http://127.0.0.1:9200/playing/_doc/1，method：GET，可以发现版本和序列号都发生了改变。
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 5,
"_seq_no": 5,
"_primary_term": 1,
"found": true,
"_source": {
"title": "小米phone",
"category": "手机",
"price": 4999
}
}
- 删除文档：
http://127.0.0.1:9200/playing/_doc/1，method：DELETE
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 6,
"result": "deleted",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 6,
"_primary_term": 1
}
- 再删除一次，则返回找不到的提示：
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 7,
"result": "not_found", #找不到
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 7,
"_primary_term": 1
}
- 条件查询：
http://127.0.0.1:9200/playing/_search?q=category:手机，method：GET
{
"took": 1251,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.21072102,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "ua3ynnsB4tbB67MkkEHa",
"_score": 0.21072102,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 3999
}
},
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": 0.21072102,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 3999
}
}
]
}
}
- 一般不推荐放在query_string中，而是放在请求体中，全量查询：
http://127.0.0.1:9200/playing/_search，method：POST，请求体内容为：
{
"query":{
"match":{
"category": "手机"
}
}
}
- 服务器返回内容：
{
"took": 22,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.21072102,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "ua3ynnsB4tbB67MkkEHa",
"_score": 0.21072102,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 3999
}
},
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": 0.21072102,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 3999
}
}
]
}
}
- 分页查询：
http://127.0.0.1:9200/playing/_search，method：POST，请求体内容：
{
"query":{
"match_all":{
}
},
"from": 0,
"size": 1
}
- 服务器返回内容：
{
"took": 9,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "ua3ynnsB4tbB67MkkEHa",
"_score": 1,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 3999
}
}
]
}
}
- 分页投影排序查询：
http://127.0.0.1:9200/playing/_search，method：POST，请求体内容：
{
"query":{
"match_all":{
}
},
"from": 1,
"size": 1,
"_source": ["title"], #投影字段
"sort": {
"price": {
"order": "desc"
}
}
}
- 服务器返回内容：
{
"took": 10,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": null,
"_source": {
"title": "一加phone"
},
"sort": [
3999
]
}
]
}
}
- 多条件查询must操作：
http://127.0.0.1:9200/playing/_search，method：POST，请求体内容：
{
"query":{
"bool": {
"must": [ #表示 and 操作，必须同时满足
{
"match": {
"category": "手机"
}
}
]
}
}
}
- 多条件查询should和filter操作：
http://127.0.0.1:9200/playing/_search，method：POST，请求体内容：
{
"query":{
"bool": {
"should": [ #表示 or操作，只要符合条件都可以
{
"match": {
"title": "小米"
}
},
{
"match": {
"title": "一加"
}
}
],
"filter": { #按价格过滤
"range": {
"price": {
"gt": 4000
}
}
}
}
}
}
- 服务器返回内容：
{
"took": 8,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.3862942,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "2",
"_score": 1.3862942,
"_source": {
"title": "小米phone",
"category": "手机",
"price": 4999
}
}
]
}
}
- 对结果进行高亮显示：
http://127.0.0.1:9200/playing/_search，method：POST，请求体内容：
{
"query":{
"match_phrase": { #短词匹配，这里还能查询到估计是es分词了
"title": "phone"
}
},
"highlight": {
"fields": {
"title": {}
}
}
}
- 服务器返回内容：
{
"took": 9,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.18232156,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": 0.18232156,
"_source": {
"title": "一加phone",
"category": "手机",
"price": 3999
},
"highlight": {
"title": [
"一加<em>phone</em>"
]
}
},
{
"_index": "playing",
"_type": "_doc",
"_id": "2",
"_score": 0.18232156,
"_source": {
"title": "小米phone",
"category": "手机",
"price": 4999
},
"highlight": {
"title": [
"小米<em>phone</em>"
]
}
}
]
}
}
- 聚合查询：
http://127.0.0.1:9200/playing/_search，method：POST，请求体内容：
{
"aggs": { # 聚合操作
"price_group": { #分组名
"terms": { #分组操作
"field": "price" #分组字段
}
}
},
"size": 0 # 不用返回原始数据
}
- 服务器返回内容：
{
"took": 146,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": [] # 对应上面的 size 字段值
},
"aggregations": {
"price_group": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 3999,
"doc_count": 1
},
{
"key": 4999,
"doc_count": 1
}
]
}
}
}
- 平均值查询：
http://127.0.0.1:9200/playing/_search，method：POST，请求体内容：
{
"aggs": {
"price_avg": {
"avg": {
"field": "price"
}
}
},
"size": 0
}
- 服务器返回内容：
{
"took": 383,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": []
},
"aggregations": {
"price_avg": {
"value": 4499
}
}
}
- 创建映射：
http://127.0.0.1:9200/user/_mapping，method：PUT
{
"properties": {
"name":{
"type": "text",
"index": true
},
"sex":{
"type": "keyword", # 表示完全匹配
"index": true
},
"age":{
"type": "long",
"index": false # 表示不能被索引，也不能被查询
}
}
}
- 查询映射：
http://127.0.0.1:9200/user/_mapping，method：GET
{
"user": {
"mappings": {
"properties": {
"age": {
"type": "long",
"index": false
},
"name": {
"type": "text"
},
"sex": {
"type": "keyword",
"index": true
}
}
}
}
}
