elastic 线程池设置解决logstash-429
2019-05-08 本文已影响0人
xyz098
logstash日志信息
Why am I seeing bulk rejections in my Elasticsearch cluster?
logstash | [2019-05-06T16:24:17,476][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of processing of [833922829][indices:data/write/bulk[s][p]]: request: BulkShardRequest [[crm-provider-permission-2019.05.06][4]] containing [153] requests, target allocation id: x8vav1ixQ_6ILskMY6IsYw, primary term: 1 on EsThreadPoolExecutor[name = node-hot-2/write, queue capacity = 5000, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@4958db63[Running, pool size = 17, active threads = 17, queued tasks = 5004, completed tasks = 598685703]]"})
elastic线程池设置
-
查看线程池状态
# GET /_cat/thread_pool/write?v node_name name active queue rejected node-7 write 0 0 0 node-master-2 write 0 0 0 # GET /_cat/thread_pool/write?v&h=node_name,name,active,rejected,completed node_name name active rejected completed node-8 write 0 2393 133703661 node-7 write 0 0 27376561 # curl -s -XGET "http://127.0.0.1:9200/_cat/thread_pool/write?v&h=node_name,name,active,rejected,completed"
-
调整线程池 Increasing the size of the queue in Elasticsearch?
-
es5以后不能动态调整
PUT _cluster/settings { "transient": { "thread_pool": { "bulk": { "queue_size": "1000" } } } } 不能调整 "reason": "transient setting [thread_pool.write.queue_size], not dynamically updateable"
-
配置文件修改
# cat elasticsearch.yml thread_pool: write: size: 32 // 默认是available processors,由17调整到32 queue_size: 100000 //由4000调整大小为100000 processors: 32
-
验证
查看reject的值 # curl -s -XGET "http://es.yw.zhenaioa.com:9201/_cat/thread_pool/write?v&h=node_name,name,active,rejected,completed" |grep node-hot 查看logstash日志,无429
-