Elasticsearch 搜索的实现
2018-10-16 本文已影响31人
溯水心生
[TOC]
一、空搜索
搜索（search） 可以做到：
- 在类似于 gender 或者 age 这样的字段 上使用结构化查询，join_date 这样的字段上使用排序，就像SQL的结构化查询一样。
- 全文检索，找出所有匹配关键字的文档并按照相关性（relevance） 排序后返回结果。
- 以上二者兼而有之。
映射（Mapping）
描述数据在每个字段内如何存储
分析（Analysis）
全文是如何处理使之可以被搜索的
领域特定查询语言（Query DSL）
Elasticsearch 中强大灵活的查询语言
空搜索语句：
GET /_search
返回结果：
{
"took": 36,
"timed_out": false,
"_shards": {
"total": 24,
"successful": 24,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 37795,
"max_score": 1,
"hits": [
{
"_index": ".kibana",
"_type": "doc",
"_id": "config:6.4.1",
"_score": 1,
"_source": {
"type": "config",
"updated_at": "2018-09-19T12:04:02.918Z",
"config": {
"buildNum": 17999,
"telemetry:optIn": false
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "AbWi9mUBXEvc_iP6owSi",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:59:38.481Z",
"interval_ms": 10000,
"type": "index_recovery",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:59:38.481Z"
},
"index_recovery": {
"shards": [
{
"index_name": ".monitoring-kibana-6-2018.09.20",
"id": 0,
"type": "EMPTY_STORE",
"stage": "DONE",
"primary": true,
"start_time_in_millis": 1537441095897,
"stop_time_in_millis": 1537441095978,
"total_time_in_millis": 80,
"source": {},
"target": {
"id": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE"
},
"index": {
"size": {
"total_in_bytes": 0,
"reused_in_bytes": 0,
"recovered_in_bytes": 0,
"percent": "0.0%"
},
"files": {
"total": 0,
"reused": 0,
"recovered": 0,
"percent": "0.0%"
},
"total_time_in_millis": 26,
"source_throttle_time_in_millis": 0,
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time_in_millis": 26
},
"verify_index": {
"check_index_time_in_millis": 0,
"total_time_in_millis": 0
}
},
{
"index_name": "megacorp",
"id": 1,
"type": "EXISTING_STORE",
"stage": "DONE",
"primary": true,
"start_time_in_millis": 1537440418788,
"stop_time_in_millis": 1537440418844,
"total_time_in_millis": 56,
"source": {},
"target": {
"id": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE"
},
"index": {
"size": {
"total_in_bytes": 261,
"reused_in_bytes": 261,
"recovered_in_bytes": 0,
"percent": "100.0%"
},
"files": {
"total": 1,
"reused": 1,
"recovered": 0,
"percent": "100.0%"
},
"total_time_in_millis": 1,
"source_throttle_time_in_millis": 0,
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time_in_millis": 44
},
"verify_index": {
"check_index_time_in_millis": 0,
"total_time_in_millis": 0
}
},
{
"index_name": "megacorp",
"id": 4,
"type": "EXISTING_STORE",
"stage": "DONE",
"primary": true,
"start_time_in_millis": 1537440418539,
"stop_time_in_millis": 1537440418820,
"total_time_in_millis": 281,
"source": {},
"target": {
"id": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE"
},
"index": {
"size": {
"total_in_bytes": 5786,
"reused_in_bytes": 5786,
"recovered_in_bytes": 0,
"percent": "100.0%"
},
"files": {
"total": 4,
"reused": 4,
"recovered": 0,
"percent": "100.0%"
},
"total_time_in_millis": 9,
"source_throttle_time_in_millis": 0,
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time_in_millis": 233
},
"verify_index": {
"check_index_time_in_millis": 0,
"total_time_in_millis": 0
}
},
{
"index_name": "megacorp",
"id": 3,
"type": "EXISTING_STORE",
"stage": "DONE",
"primary": true,
"start_time_in_millis": 1537440418568,
"stop_time_in_millis": 1537440418830,
"total_time_in_millis": 261,
"source": {},
"target": {
"id": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE"
},
"index": {
"size": {
"total_in_bytes": 5861,
"reused_in_bytes": 5861,
"recovered_in_bytes": 0,
"percent": "100.0%"
},
"files": {
"total": 4,
"reused": 4,
"recovered": 0,
"percent": "100.0%"
},
"total_time_in_millis": 8,
"source_throttle_time_in_millis": 0,
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time_in_millis": 239
},
"verify_index": {
"check_index_time_in_millis": 0,
"total_time_in_millis": 0
}
},
{
"index_name": "megacorp",
"id": 2,
"type": "EXISTING_STORE",
"stage": "DONE",
"primary": true,
"start_time_in_millis": 1537440418572,
"stop_time_in_millis": 1537440418819,
"total_time_in_millis": 247,
"source": {},
"target": {
"id": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE"
},
"index": {
"size": {
"total_in_bytes": 5795,
"reused_in_bytes": 5795,
"recovered_in_bytes": 0,
"percent": "100.0%"
},
"files": {
"total": 4,
"reused": 4,
"recovered": 0,
"percent": "100.0%"
},
"total_time_in_millis": 14,
"source_throttle_time_in_millis": 0,
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time_in_millis": 222
},
"verify_index": {
"check_index_time_in_millis": 0,
"total_time_in_millis": 0
}
},
{
"index_name": "megacorp",
"id": 0,
"type": "EXISTING_STORE",
"stage": "DONE",
"primary": true,
"start_time_in_millis": 1537440418577,
"stop_time_in_millis": 1537440418777,
"total_time_in_millis": 199,
"source": {},
"target": {
"id": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE"
},
"index": {
"size": {
"total_in_bytes": 261,
"reused_in_bytes": 261,
"recovered_in_bytes": 0,
"percent": "100.0%"
},
"files": {
"total": 1,
"reused": 1,
"recovered": 0,
"percent": "100.0%"
},
"total_time_in_millis": 6,
"source_throttle_time_in_millis": 0,
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time_in_millis": 183
},
"verify_index": {
"check_index_time_in_millis": 0,
"total_time_in_millis": 0
}
},
{
"index_name": ".kibana",
"id": 0,
"type": "EXISTING_STORE",
"stage": "DONE",
"primary": true,
"start_time_in_millis": 1537440418856,
"stop_time_in_millis": 1537440418924,
"total_time_in_millis": 67,
"source": {},
"target": {
"id": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE"
},
"index": {
"size": {
"total_in_bytes": 4123,
"reused_in_bytes": 4123,
"recovered_in_bytes": 0,
"percent": "100.0%"
},
"files": {
"total": 4,
"reused": 4,
"recovered": 0,
"percent": "100.0%"
},
"total_time_in_millis": 1,
"source_throttle_time_in_millis": 0,
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time_in_millis": 61
},
"verify_index": {
"check_index_time_in_millis": 0,
"total_time_in_millis": 0
}
},
{
"index_name": ".monitoring-es-6-2018.09.20",
"id": 0,
"type": "EMPTY_STORE",
"stage": "DONE",
"primary": true,
"start_time_in_millis": 1537441098833,
"stop_time_in_millis": 1537441098903,
"total_time_in_millis": 70,
"source": {},
"target": {
"id": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE"
},
"index": {
"size": {
"total_in_bytes": 0,
"reused_in_bytes": 0,
"recovered_in_bytes": 0,
"percent": "0.0%"
},
"files": {
"total": 0,
"reused": 0,
"recovered": 0,
"percent": "0.0%"
},
"total_time_in_millis": 36,
"source_throttle_time_in_millis": 0,
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time_in_millis": 20
},
"verify_index": {
"check_index_time_in_millis": 0,
"total_time_in_millis": 0
}
}
]
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "ArWi9mUBXEvc_iP6owSi",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:59:38.485Z",
"interval_ms": 10000,
"type": "index_stats",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:59:38.481Z"
},
"index_stats": {
"index": ".monitoring-es-6-2018.09.20",
"uuid": "0k6_GWrySXe6qRRHvUNVig",
"created": 1537441098799,
"status": "green",
"version": {
"created": "6.4.1",
"upgraded": "6.4.1"
},
"shards": {
"total": 1,
"primaries": 1,
"replicas": 0,
"active_total": 1,
"active_primaries": 1,
"active_replicas": 0,
"unassigned_total": 0,
"unassigned_primaries": 0,
"unassigned_replicas": 0,
"initializing": 0,
"relocating": 0
},
"total": {
"docs": {
"count": 88
},
"store": {
"size_in_bytes": 484070
},
"indexing": {
"index_total": 166,
"index_time_in_millis": 272,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 0,
"query_time_in_millis": 0
},
"merges": {
"total_size_in_bytes": 0
},
"refresh": {
"total_time_in_millis": 386
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 9,
"memory_in_bytes": 63010,
"terms_memory_in_bytes": 47376,
"stored_fields_memory_in_bytes": 2808,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 0,
"points_memory_in_bytes": 2326,
"doc_values_memory_in_bytes": 10500,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 0,
"evictions": 0,
"hit_count": 0,
"miss_count": 0
}
},
"primaries": {
"docs": {
"count": 88
},
"store": {
"size_in_bytes": 484070
},
"indexing": {
"index_total": 166,
"index_time_in_millis": 272,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 0,
"query_time_in_millis": 0
},
"merges": {
"total_size_in_bytes": 0
},
"refresh": {
"total_time_in_millis": 386
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 9,
"memory_in_bytes": 63010,
"terms_memory_in_bytes": 47376,
"stored_fields_memory_in_bytes": 2808,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 0,
"points_memory_in_bytes": 2326,
"doc_values_memory_in_bytes": 10500,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 0,
"evictions": 0,
"hit_count": 0,
"miss_count": 0
}
}
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "A7Wi9mUBXEvc_iP6owSi",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:59:38.485Z",
"interval_ms": 10000,
"type": "index_stats",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:59:38.481Z"
},
"index_stats": {
"index": "megacorp",
"uuid": "Bq-aXVvEReG4-_5VV0ypoA",
"created": 1537359244511,
"status": "yellow",
"version": {
"created": "6.4.1",
"upgraded": "6.4.1"
},
"shards": {
"total": 10,
"primaries": 5,
"replicas": 1,
"active_total": 5,
"active_primaries": 5,
"active_replicas": 0,
"unassigned_total": 5,
"unassigned_primaries": 0,
"unassigned_replicas": 5,
"initializing": 0,
"relocating": 0
},
"total": {
"docs": {
"count": 3
},
"store": {
"size_in_bytes": 17964
},
"indexing": {
"index_total": 0,
"index_time_in_millis": 0,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 40,
"query_time_in_millis": 282
},
"merges": {
"total_size_in_bytes": 0
},
"refresh": {
"total_time_in_millis": 0
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 3,
"memory_in_bytes": 8317,
"terms_memory_in_bytes": 6267,
"stored_fields_memory_in_bytes": 936,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 768,
"points_memory_in_bytes": 6,
"doc_values_memory_in_bytes": 340,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 12575,
"evictions": 0,
"hit_count": 15,
"miss_count": 15
}
},
"primaries": {
"docs": {
"count": 3
},
"store": {
"size_in_bytes": 17964
},
"indexing": {
"index_total": 0,
"index_time_in_millis": 0,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 40,
"query_time_in_millis": 282
},
"merges": {
"total_size_in_bytes": 0
},
"refresh": {
"total_time_in_millis": 0
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 3,
"memory_in_bytes": 8317,
"terms_memory_in_bytes": 6267,
"stored_fields_memory_in_bytes": 936,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 768,
"points_memory_in_bytes": 6,
"doc_values_memory_in_bytes": 340,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 12575,
"evictions": 0,
"hit_count": 15,
"miss_count": 15
}
}
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "BLWi9mUBXEvc_iP6owSi",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:59:38.485Z",
"interval_ms": 10000,
"type": "index_stats",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:59:38.481Z"
},
"index_stats": {
"index": ".monitoring-kibana-6-2018.09.20",
"uuid": "zafNbZ-GQwemzl9esmP-FA",
"created": 1537441095863,
"status": "green",
"version": {
"created": "6.4.1",
"upgraded": "6.4.1"
},
"shards": {
"total": 1,
"primaries": 1,
"replicas": 0,
"active_total": 1,
"active_primaries": 1,
"active_replicas": 0,
"unassigned_total": 0,
"unassigned_primaries": 0,
"unassigned_replicas": 0,
"initializing": 0,
"relocating": 0
},
"total": {
"docs": {
"count": 9
},
"store": {
"size_in_bytes": 115756
},
"indexing": {
"index_total": 9,
"index_time_in_millis": 130,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 0,
"query_time_in_millis": 0
},
"merges": {
"total_size_in_bytes": 0
},
"refresh": {
"total_time_in_millis": 394
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 9,
"memory_in_bytes": 34758,
"terms_memory_in_bytes": 30951,
"stored_fields_memory_in_bytes": 2808,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 0,
"points_memory_in_bytes": 387,
"doc_values_memory_in_bytes": 612,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 0,
"evictions": 0,
"hit_count": 0,
"miss_count": 0
}
},
"primaries": {
"docs": {
"count": 9
},
"store": {
"size_in_bytes": 115756
},
"indexing": {
"index_total": 9,
"index_time_in_millis": 130,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 0,
"query_time_in_millis": 0
},
"merges": {
"total_size_in_bytes": 0
},
"refresh": {
"total_time_in_millis": 394
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 9,
"memory_in_bytes": 34758,
"terms_memory_in_bytes": 30951,
"stored_fields_memory_in_bytes": 2808,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 0,
"points_memory_in_bytes": 387,
"doc_values_memory_in_bytes": 612,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 0,
"evictions": 0,
"hit_count": 0,
"miss_count": 0
}
}
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "BbWi9mUBXEvc_iP6owSi",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:59:38.485Z",
"interval_ms": 10000,
"type": "index_stats",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:59:38.481Z"
},
"index_stats": {
"index": ".kibana",
"uuid": "-FWp_rvnRjm6Nvu2dG9wdw",
"created": 1537358641597,
"status": "green",
"version": {
"created": "6.4.1",
"upgraded": "6.4.1"
},
"shards": {
"total": 1,
"primaries": 1,
"replicas": 0,
"active_total": 1,
"active_primaries": 1,
"active_replicas": 0,
"unassigned_total": 0,
"unassigned_primaries": 0,
"unassigned_replicas": 0,
"initializing": 0,
"relocating": 0
},
"total": {
"docs": {
"count": 1
},
"store": {
"size_in_bytes": 4123
},
"indexing": {
"index_total": 0,
"index_time_in_millis": 0,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 79,
"query_time_in_millis": 486
},
"merges": {
"total_size_in_bytes": 0
},
"refresh": {
"total_time_in_millis": 0
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 1,
"memory_in_bytes": 1346,
"terms_memory_in_bytes": 964,
"stored_fields_memory_in_bytes": 312,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 0,
"points_memory_in_bytes": 2,
"doc_values_memory_in_bytes": 68,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 3366,
"evictions": 0,
"hit_count": 62,
"miss_count": 4
}
},
"primaries": {
"docs": {
"count": 1
},
"store": {
"size_in_bytes": 4123
},
"indexing": {
"index_total": 0,
"index_time_in_millis": 0,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 79,
"query_time_in_millis": 486
},
"merges": {
"total_size_in_bytes": 0
},
"refresh": {
"total_time_in_millis": 0
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 1,
"memory_in_bytes": 1346,
"terms_memory_in_bytes": 964,
"stored_fields_memory_in_bytes": 312,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 0,
"points_memory_in_bytes": 2,
"doc_values_memory_in_bytes": 68,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 3366,
"evictions": 0,
"hit_count": 62,
"miss_count": 4
}
}
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "BrWi9mUBXEvc_iP6owSi",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:59:38.485Z",
"interval_ms": 10000,
"type": "indices_stats",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:59:38.481Z"
},
"indices_stats": {
"_all": {
"primaries": {
"docs": {
"count": 101
},
"store": {
"size_in_bytes": 621913
},
"indexing": {
"index_total": 175,
"index_time_in_millis": 402,
"is_throttled": false,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 119,
"query_time_in_millis": 768
}
},
"total": {
"docs": {
"count": 101
},
"store": {
"size_in_bytes": 621913
},
"indexing": {
"index_total": 175,
"index_time_in_millis": 402,
"is_throttled": false,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 119,
"query_time_in_millis": 768
}
}
}
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "B7Wi9mUBXEvc_iP6owSi",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:59:38.519Z",
"interval_ms": 10000,
"type": "cluster_stats",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:59:38.481Z"
},
"cluster_name": "elasticsearch",
"version": "6.4.1",
"license": {
"status": "active",
"uid": "a5d4acd3-c25e-4d39-94bb-07c0eda39493",
"type": "basic",
"issue_date": "2018-09-19T11:49:32.408Z",
"issue_date_in_millis": 1537357772408,
"max_nodes": 1000,
"issued_to": "elasticsearch",
"issuer": "elasticsearch",
"start_date_in_millis": -1
},
"cluster_stats": {
"timestamp": 1537441178489,
"status": "yellow",
"indices": {
"count": 4,
"shards": {
"total": 8,
"primaries": 8,
"replication": 0,
"index": {
"shards": {
"min": 1,
"max": 5,
"avg": 2
},
"primaries": {
"min": 1,
"max": 5,
"avg": 2
},
"replication": {
"min": 0,
"max": 0,
"avg": 0
}
}
},
"docs": {
"count": 101,
"deleted": 78
},
"store": {
"size_in_bytes": 636363
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"query_cache": {
"memory_size_in_bytes": 0,
"total_count": 0,
"hit_count": 0,
"miss_count": 0,
"cache_size": 0,
"cache_count": 0,
"evictions": 0
},
"completion": {
"size_in_bytes": 0
},
"segments": {
"count": 22,
"memory_in_bytes": 107431,
"terms_memory_in_bytes": 85558,
"stored_fields_memory_in_bytes": 6864,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 768,
"points_memory_in_bytes": 2721,
"doc_values_memory_in_bytes": 11520,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0,
"max_unsafe_auto_id_timestamp": -1,
"file_sizes": {}
}
},
"nodes": {
"count": {
"total": 1,
"data": 1,
"coordinating_only": 0,
"master": 1,
"ingest": 1
},
"versions": [
"6.4.1"
],
"os": {
"available_processors": 4,
"allocated_processors": 4,
"names": [
{
"name": "Linux",
"count": 1
}
],
"mem": {
"total_in_bytes": 3956957184,
"free_in_bytes": 139051008,
"used_in_bytes": 3817906176,
"free_percent": 4,
"used_percent": 96
}
},
"process": {
"cpu": {
"percent": 1
},
"open_file_descriptors": {
"min": 283,
"max": 283,
"avg": 283
}
},
"jvm": {
"max_uptime_in_millis": 808318,
"versions": [
{
"version": "1.8.0_111",
"vm_name": "Java HotSpot(TM) 64-Bit Server VM",
"vm_version": "25.111-b14",
"vm_vendor": "Oracle Corporation",
"count": 1
}
],
"mem": {
"heap_used_in_bytes": 451642584,
"heap_max_in_bytes": 1038876672
},
"threads": 59
},
"fs": {
"total_in_bytes": 49359622144,
"free_in_bytes": 15952596992,
"available_in_bytes": 15952596992
},
"plugins": [],
"network_types": {
"transport_types": {
"security4": 1
},
"http_types": {
"security4": 1
}
}
}
},
"cluster_state": {
"nodes_hash": 40395404,
"status": "yellow",
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"version": 13,
"state_uuid": "33d09pOdQnitL7qiaFxjRw",
"master_node": "O5BAVYE1SeeOGKzAdxo-Zw",
"nodes": {
"O5BAVYE1SeeOGKzAdxo-Zw": {
"name": "O5BAVYE",
"ephemeral_id": "TRKk0tbDSNuH4uSd4Jb0Sw",
"transport_address": "127.0.0.1:9300",
"attributes": {
"ml.machine_memory": "3956957184",
"xpack.installed": "true",
"ml.max_open_jobs": "20",
"ml.enabled": "true"
}
}
}
},
"stack_stats": {
"apm": {
"found": false
},
"xpack": {
"security": {
"available": false,
"enabled": true,
"realms": {
"file": {
"available": false,
"enabled": false
},
"ldap": {
"available": false,
"enabled": false
},
"native": {
"available": false,
"enabled": false
},
"saml": {
"available": false,
"enabled": false
},
"kerberos": {
"available": false,
"enabled": false
},
"active_directory": {
"available": false,
"enabled": false
},
"pki": {
"available": false,
"enabled": false
}
},
"roles": {
"native": {
"size": 0,
"fls": false,
"dls": false
},
"file": {
"size": 0,
"fls": false,
"dls": false
}
},
"role_mapping": {
"native": {
"size": 0,
"enabled": 0
}
},
"ssl": {
"http": {
"enabled": false
},
"transport": {
"enabled": false
}
},
"audit": {
"outputs": [
"logfile"
],
"enabled": false
},
"ipfilter": {
"http": false,
"transport": false
},
"anonymous": {
"enabled": false
}
},
"monitoring": {
"available": true,
"enabled": true,
"collection_enabled": true,
"enabled_exporters": {
"local": 1
}
},
"watcher": {
"available": false,
"enabled": true,
"execution": {
"actions": {
"_all": {
"total": 0,
"total_time_in_ms": 0
}
}
},
"watch": {
"input": {
"_all": {
"total": 0,
"active": 0
}
},
"trigger": {
"_all": {
"total": 0,
"active": 0
}
}
},
"count": {
"total": 0,
"active": 0
}
},
"ml": {
"available": false,
"enabled": true,
"jobs": {
"_all": {
"count": 0,
"detectors": {
"total": 0,
"min": 0,
"avg": 0,
"max": 0
},
"model_size": {
"total": 0,
"min": 0,
"avg": 0,
"max": 0
},
"forecasts": {
"total": 0,
"forecasted_jobs": 0
}
}
},
"datafeeds": {
"_all": {
"count": 0
}
}
},
"logstash": {
"available": false,
"enabled": true
},
"graph": {
"available": false,
"enabled": true
},
"rollup": {
"available": true,
"enabled": true
}
}
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "CLWi9mUBXEvc_iP6owSi",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:59:38.520Z",
"interval_ms": 10000,
"type": "node_stats",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:59:38.481Z"
},
"node_stats": {
"node_id": "O5BAVYE1SeeOGKzAdxo-Zw",
"node_master": true,
"mlockall": false,
"indices": {
"docs": {
"count": 101
},
"store": {
"size_in_bytes": 636363
},
"indexing": {
"index_total": 175,
"index_time_in_millis": 402,
"throttle_time_in_millis": 0
},
"search": {
"query_total": 119,
"query_time_in_millis": 768
},
"query_cache": {
"memory_size_in_bytes": 0,
"hit_count": 0,
"miss_count": 0,
"evictions": 0
},
"fielddata": {
"memory_size_in_bytes": 0,
"evictions": 0
},
"segments": {
"count": 22,
"memory_in_bytes": 107431,
"terms_memory_in_bytes": 85558,
"stored_fields_memory_in_bytes": 6864,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 768,
"points_memory_in_bytes": 2721,
"doc_values_memory_in_bytes": 11520,
"index_writer_memory_in_bytes": 0,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 0
},
"request_cache": {
"memory_size_in_bytes": 15941,
"evictions": 0,
"hit_count": 77,
"miss_count": 19
}
},
"os": {
"cpu": {
"load_average": {
"1m": 1.01,
"5m": 0.94,
"15m": 1.02
}
},
"cgroup": {
"cpuacct": {
"control_group": "/user.slice",
"usage_nanos": 513471132423
},
"cpu": {
"control_group": "/user.slice",
"cfs_period_micros": 100000,
"cfs_quota_micros": -1,
"stat": {
"number_of_elapsed_periods": 0,
"number_of_times_throttled": 0,
"time_throttled_nanos": 0
}
},
"memory": {
"control_group": "/user.slice",
"limit_in_bytes": "9223372036854771712",
"usage_in_bytes": "2889932800"
}
}
},
"process": {
"open_file_descriptors": 283,
"max_file_descriptors": 4096,
"cpu": {
"percent": 1
}
},
"jvm": {
"mem": {
"heap_used_in_bytes": 451642584,
"heap_used_percent": 43,
"heap_max_in_bytes": 1038876672
},
"gc": {
"collectors": {
"young": {
"collection_count": 10,
"collection_time_in_millis": 400
},
"old": {
"collection_count": 2,
"collection_time_in_millis": 118
}
}
}
},
"thread_pool": {
"generic": {
"threads": 4,
"queue": 0,
"rejected": 0
},
"get": {
"threads": 4,
"queue": 0,
"rejected": 0
},
"index": {
"threads": 0,
"queue": 0,
"rejected": 0
},
"management": {
"threads": 3,
"queue": 0,
"rejected": 0
},
"search": {
"threads": 7,
"queue": 0,
"rejected": 0
},
"watcher": {
"threads": 0,
"queue": 0,
"rejected": 0
},
"write": {
"threads": 4,
"queue": 0,
"rejected": 0
}
},
"fs": {
"total": {
"total_in_bytes": 49359622144,
"free_in_bytes": 15952596992,
"available_in_bytes": 15952596992
},
"io_stats": {
"total": {
"operations": 35430,
"read_operations": 32408,
"write_operations": 3022,
"read_kilobytes": 2113708,
"write_kilobytes": 201832
}
}
}
}
}
},
{
"_index": ".monitoring-es-6-2018.09.20",
"_type": "doc",
"_id": "jkldWwXZQiitMzm206BjKw:O5BAVYE1SeeOGKzAdxo-Zw:megacorp:1:p",
"_score": 1,
"_source": {
"cluster_uuid": "rw1yjlzkSgODXkUVgIxmxg",
"timestamp": "2018-09-20T10:58:18.704Z",
"interval_ms": 10000,
"type": "shards",
"source_node": {
"uuid": "O5BAVYE1SeeOGKzAdxo-Zw",
"host": "127.0.0.1",
"transport_address": "127.0.0.1:9300",
"ip": "127.0.0.1",
"name": "O5BAVYE",
"timestamp": "2018-09-20T10:58:18.473Z"
},
"state_uuid": "jkldWwXZQiitMzm206BjKw",
"shard": {
"state": "STARTED",
"primary": true,
"node": "O5BAVYE1SeeOGKzAdxo-Zw",
"relocating_node": null,
"shard": 1,
"index": "megacorp"
}
}
}
]
}
}
- hits
返回结果中最重要的部分是 hits ，它 包含 total 字段来表示匹配到的文档总数，并且一个 hits 数组包含所查询结果的前十个文档。
在 hits 数组中每个结果包含文档的 _index 、 _type 、 _id ，加上 _source 字段。这意味着我们可以直接从返回的搜索结果中使用整个文档。这不像其他的搜索引擎，仅仅返回文档的ID，需要你单独去获取文档。
每个结果还有一个 _score ，它衡量了文档与查询的匹配程度。默认情况下，首先返回最相关的文档结果，就是说，返回的文档是按照 _score 降序排列的。在这个例子中，我们没有指定任何查询，故所有的文档具有相同的相关性，因此对所有的结果而言 1 是中性的 _score 。
max_score 值是与查询所匹配文档的 _score 的最大值。
- took
took 值告诉我们执行整个搜索请求耗费了多少毫秒。
- shards
_shards 部分 告诉我们在查询中参与分片的总数，以及这些分片成功了多少个失败了多少个。正常情况下我们不希望分片失败，但是分片失败是可能发生的。如果我们遭遇到一种灾难级别的故障，在这个故障中丢失了相同分片的原始数据和副本，那么对这个分片将没有可用副本来对搜索请求作出响应。假若这样，Elasticsearch 将报告这个分片是失败的，但是会继续返回剩余分片的结果。
- timeout
timed_out 值告诉我们查询是否超时。默认情况下，搜索请求不会超时。 如果低响应时间比完成结果更重要，你可以指定 timeout 为 10 或者 10ms（10毫秒），或者 1s（1秒）：
GET /_search?timeout=10ms
在请求超时之前，Elasticsearch 将会返回已经成功从每个分片获取的结果。
应当注意的是 timeout 不是停止执行查询，它仅仅是告知正在协调的节点返回到目前为止收集的结果并且关闭连接。在后台，其他的分片可能仍在执行查询即使是结果已经被发送了。
使用超时是因为 SLA(服务等级协议)对你是很重要的，而不是因为想去中止长时间运行的查询。
二、多索引、多类型
- /_search
在所有的索引中搜索所有的类型 - /gb/_search
在 gb 索引中搜索所有的类型 - /gb,us/_search
在 gb 和 us 索引中搜索所有的文档 - /g,u/_search
在任何在任何以 g 或者 u 开头的索引中搜索所有的类型以 g 或者 u 开头的索引中搜索所有的类型 - /gb/user/_search
在 gb 索引中搜索 user 类型 - /gb,us/user,tweet/_search
在 gb 和 us 索引中搜索 user 和 tweet 类型 - /_all/user,tweet/_search
在所有的索引中搜索 user 和 tweet 类型
当在单一的索引下进行搜索的时候，Elasticsearch 转发请求到索引的每个分片中，可以是主分片也可以是副本分片，然后从每个分片中收集结果。多索引搜索恰好也是用相同的方式工作的--只是会涉及到更多的分片。
搜索一个索引有五个主分片和搜索五个索引各有一个分片准确来所说是等价的。
三、分页
Elasticsearch 接受 from 和 size 参数：
size
显示应该返回的结果数量，默认是 10
from
显示应该跳过的初始结果数量，默认是 0
如果每页展示 5 条结果，可以用下面方式请求得到 1 到 3 页的结果：
GET /_search?size=5
GET /_search?size=5&from=5
GET /_search?size=5&from=10
考虑到分页过深以及一次请求太多结果的情况，结果集在返回之前先进行排序。 但请记住一个请求经常跨越多个分片，每个分片都产生自己的排序结果，这些结果需要进行集中排序以保证整体顺序是正确的。
在分布式系统中深度分页
理解为什么深度分页是有问题的，我们可以假设在一个有 5 个主分片的索引中搜索。
当我们请求结果的第一页（结果从 1 到 10 ），每一个分片产生前 10 的结果，并且返回给 协调节点 ，协调节点对 50 个结果排序得到全部结果的前 10 个。
现在假设我们请求第 1000 页--结果从 10001 到 10010 。所有都以相同的方式工作除了每个分片不得不产生前10010个结果以外。 然后协调节点对全部 50050 个结果排序最后丢弃掉这些结果中的 50040 个结果。
可以看到，在分布式系统中，对结果排序的成本随分页的深度成指数上升。这就是 web 搜索引擎对任何查询都不要返回超过 1000 个结果的原因。
四、轻量搜索
有两种形式的 搜索 API：一种是 “轻量的” 查询字符串 版本，要求在查询字符串中传递所有的 参数，另一种是更完整的 请求体 版本，要求使用 JSON 格式和更丰富的查询表达式作为搜索语言。
查询字符串搜索非常适用于通过命令行做即席查询。例如，查询在 tweet 类型中 tweet 字段包含 elasticsearch 单词的所有文档：
GET /_all/tweet/_search?q=tweet:elasticsearch
下一个查询在 name 字段中包含 john 并且在 tweet 字段中包含 mary 的文档。实际的查询就是这样
+name:john +tweet:mary
但是查询字符串参数所需要的 百分比编码 （译者注：URL编码）实际上更加难懂：
GET /_search?q=%2Bname%3Ajohn+%2Btweet%3Amary
- 前缀表示必须与查询条件匹配。类似地， - 前缀表示一定不与查询条件匹配。没有 + 或者 - 的所有其他条件都是可选的——匹配的越多，文档就越相关。
- _all 字段
这个简单搜索返回包含 mary 的所有文档：
GET /_search?q=mary
之前的例子中，我们在 tweet 和 name 字段中搜索内容。然而，这个查询的结果在三个地方提到了 mary ：
有一个用户叫做 Mary
6条微博发自 Mary
一条微博直接 @mary
Elasticsearch 是如何在三个不同的字段中查找到结果的呢？
当索引一个文档的时候，Elasticsearch 取出所有字段的值拼接成一个大的字符串，作为 _all 字段进行索引。例如，当索引这个文档时：
{
"tweet": "However did I manage before Elasticsearch?",
"date": "2014-09-14",
"name": "Mary Jones",
"user_id": 1
}
这就好似增加了一个名叫 _all 的额外字段：
"However did I manage before Elasticsearch? 2014-09-14 Mary Jones 1"
除非设置特定字段，否则查询字符串就使用 _all 字段进行搜索。
在刚开始开发一个应用时，_all 字段是一个很实用的特性。之后，你会发现如果搜索时用指定字段来代替 _all 字段，将会更好控制搜索结果。当 _all 字段不再有用的时候，可以将它置为失效，正如在 元数据: _all 字段 中所解释的。
- 更复杂的查询
下面的查询针对tweents类型，并使用以下的条件： - name 字段中包含 mary 或者 john
- date 值大于 2014-09-10
- _all 字段包含 aggregations 或者 geo
+name:(mary john) +date:>2014-09-10 +(aggregations geo)
查询字符串在做了适当的编码后，可读性很差：
?q=%2Bname%3A(mary+john)+%2Bdate%3A%3E2014-09-10+%2B(aggregations+geo)
从之前的例子中可以看出，这种 轻量 的查询字符串搜索效果还是挺让人惊喜的。 它的查询语法在相关参考文档中有详细解释，以便简洁的表达很复杂的查询。对于通过命令做一次性查询，或者是在开发阶段，都非常方便。
但同时也可以看到，这种精简让调试更加晦涩和困难。而且很脆弱，一些查询字符串中很小的语法错误，像 - ， : ， / 或者 " 不匹配等，将会返回错误而不是搜索结果。
最后，查询字符串搜索允许任何用户在索引的任意字段上执行可能较慢且重量级的查询，这可能会暴露隐私信息，甚至将集群拖垮。
因为这些原因，不推荐直接向用户暴露查询字符串搜索功能，除非对于集群和数据来说非常信任他们。
我们经常在生产环境中更多地使用功能全面的 request body 查询API，除了能完成以上所有功能，还有一些附加功能。但在到达那个阶段之前，首先需要了解数据在 Elasticsearch 中是如何被索引的
