码迷,mamicode.com
首页 > 其他好文 > 详细

elk批量删除索引

时间:2017-08-02 19:03:53      阅读:315      评论:0      收藏:0      [点我收藏+]

标签:str   style   时间   oms   pen   ted   也会   内容   服务   

用了一段时间elk发现如果索引长时间不删除,elk会越来越慢,重启elasticsearch服务器节点之前同步时间也会很长

第一步:在elasticsearch节点上使用curl -XGET ‘http://192.168.3.164:9200/_cat/shards‘查看索引

[root@192-168-3-164 ~]#  curl -XGET http://192.168.3.164:9200/_cat/shards |more
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 57452  100 57452    0     0   467k      0 --:--:-- --:--:-- --:--:--  471k
laravel-oms26-online-2017.08.02              1 r STARTED     48 101.8kb 192.168.3.164 node2-192.168.3.164
laravel-oms26-online-2017.08.02              1 p STARTED     48 101.8kb 192.168.3.163 node1-192.168.3.163
laravel-oms26-online-2017.08.02              4 r STARTED     65 118.1kb 192.168.3.165 node3-192.168.3.165
laravel-oms26-online-2017.08.02              4 p STARTED     65 118.1kb 192.168.3.163 node1-192.168.3.163
laravel-oms26-online-2017.08.02              2 p STARTED     42  61.4kb 192.168.3.164 node2-192.168.3.164
laravel-oms26-online-2017.08.02              2 r STARTED     42  61.4kb 192.168.3.165 node3-192.168.3.165
laravel-oms26-online-2017.08.02              3 r STARTED     52 130.5kb 192.168.3.165 node3-192.168.3.165
laravel-oms26-online-2017.08.02              3 p STARTED     52 130.5kb 192.168.3.163 node1-192.168.3.163
laravel-oms26-online-2017.08.02              0 p STARTED     37 105.2kb 192.168.3.164 node2-192.168.3.164
laravel-oms26-online-2017.08.02              0 r STARTED     37 105.2kb 192.168.3.163 node1-192.168.3.163
etl213-online-2017.08.01                     1 r STARTED    649 244.6kb 192.168.3.165 node3-192.168.3.165
etl213-online-2017.08.01                     1 p STARTED    649 244.6kb 192.168.3.163 node1-192.168.3.163
etl213-online-2017.08.01                     4 r STARTED    673 295.3kb 192.168.3.164 node2-192.168.3.164
etl213-online-2017.08.01                     4 p STARTED    673 295.3kb 192.168.3.165 node3-192.168.3.165
etl213-online-2017.08.01                     2 r STARTED    663 264.1kb 192.168.3.164 node2-192.168.3.164
etl213-online-2017.08.01                     2 p STARTED    663 264.1kb 192.168.3.163 node1-192.168.3.163
etl213-online-2017.08.01                     3 p STARTED    640   280kb 192.168.3.164 node2-192.168.3.164
etl213-online-2017.08.01                     3 r STARTED    640   280kb 192.168.3.163 node1-192.168.3.163
etl213-online-2017.08.01                     0 r STARTED    670 286.2kb 192.168.3.165 node3-192.168.3.165
etl213-online-2017.08.01                     0 p STARTED    670 286.2kb 192.168.3.163 node1-192.168.3.163

 

第二步:过滤出想要删除的索引存到临时文件里面 curl -XGET ‘http://192.168.3.164:9200/_cat/shards‘ |grep "2017\.08\.01" |grep 192.168.3.163 |awk ‘{print $1}‘ | uniq >  elk-index.tmp

[root@192-168-3-164 ~]# curl -XGET http://192.168.3.164:9200/_cat/shards |grep "2017.08\.01" |grep 192.168.3.163 |awk {print $1} | uniq > elk-index.tmp
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 57452  100 57452    0     0   123k      0 --:--:-- --:--:-- --:--:--  123k
[root@192-168-3-164 ~]# cat elk-index.tmp 
wcf.core50-online-2017.08.01
etl211-online-2017.08.01
laravel-dujia-online-2017.08.01
dms217-online-2017.08.01
laravel-oms27-online-2017.08.01

第三步:删除过滤出来的索引

for i in `cat elk-index.tmp`
   do 
      curl -XDELETE  http://192.168.3.164:9200/$i 
done
    

第四步为了方便可以在计划任务里面加定时任务删除五天之前的日志索引

[root@192-168-3-164 ~]# crontab -l
0 3 * * * bash /home/scripts/del_elasticseatch_index.sh

查看脚本文件内容cat  /home/scripts/del_elasticseatch_index.sh

#!/bin/bash
#The index five days ago
curl -XGET http://192.168.3.164:9200/_cat/shards |grep 192.168.3.164 |awk {print $1} |grep `date -d "5 days ago" +%Y.%m.%d` |uniq > /tmp/index_name.tmp

for index_name in `cat /tmp/index_name.tmp`  
do
    curl -XDELETE  http://192.168.3.164:9200/$index_name
    echo "${FILE_NAME} delete success" >> /home/scripts/del_elasticseatch_index.log
done

 

elk批量删除索引

标签:str   style   时间   oms   pen   ted   也会   内容   服务   

原文地址:http://www.cnblogs.com/xiewenming/p/7275283.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!