码迷,mamicode.com
首页 > 其他好文 > 详细

elk,centos7,filebeat,elasticsearch-head详细安装步骤

时间:2017-07-04 16:05:49      阅读:1072      评论:0      收藏:0      [点我收藏+]

标签:message   lin   mat   1.4   status   index   lte   成功   received   

 

先来张图,大致结构应该晓得,对吧!

技术分享

 

 

安装jdk:至少1.8以上

yum -y localinstall jdk-8u73-linux-x64.rpm

安装elasticsearch5.2.1 用普通用户安装es:

 tar -zxvf elasticsearch-5.2.1.tar.gz

 [es@node elasticsearch-5.2.1]$ ./bin/elasticsearch

处理错误:

ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

cat /etc/security/limits.conf 加上下面这个:

* soft nofile 65536

* hard nofile 131072

* soft nproc 2048

* hard nproc 4096

cat /etc/sysctl.conf

vm.max_map_count=262144
fs.file-max=65536

然后启动成功!

也可以指定名字启动:

./elasticsearch -Ecluster.name=my_cluster_name -Enode.name=my_node_name

健康基本检测

[es@node bin]$ curl -XGET ‘192.168.36.141:9200/_cat/master?v&pretty‘
id host ip node
lbWUxIvITHy_0SufW6fBtw 192.168.36.141 192.168.36.141 my_node_name

[es@node bin]$ curl -XGET ‘192.168.36.141:9200/_cat/health?v&pretty‘
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1499173093 08:58:13 my_cluster_name green 1 1 0 0 0 0 0 0 - 100.0%
[es@node bin]$

练习索引和文档:

新建索引,显示索引:

[es@node bin]$ curl -XPUT ‘localhost:9200/customer?pretty&pretty‘
[2017-07-04T09:02:16,302][INFO ][o.e.c.m.MetaDataCreateIndexService] [my_node_name] [customer] creating index, cause [api], templates [], shards [5]/[1], mappings []
{
"acknowledged" : true,
"shards_acknowledged" : true
}
[es@node bin]$ curl -XGET ‘localhost:9200/_cat/indices?v&pretty‘
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open customer gtxHBupjSQ-4DNgMAMnuqQ 5 1 0 0 650b 650b
[es@node bin]$

56 curl -XPUT ‘localhost:9200/customer?pretty&pretty‘
57 curl -XGET ‘localhost:9200/_cat/indices?v&pretty‘
58 curl -XPUT ‘localhost:9200/customer/external/1?pretty&pretty‘ -H ‘Content-Type: application/json‘ -d‘
{
"name": "John Doe"
}

59 curl -XGET ‘localhost:9200/customer/external/1?pretty&pretty‘
60 curl -XDELETE ‘localhost:9200/customer?pretty&pretty‘
61 curl -XGET ‘localhost:9200/_cat/indices?v&pretty‘
62 curl -XPUT ‘localhost:9200/customer/external/1?pretty&pretty‘ -H ‘Content-Type: application/json‘ -d‘
{
"name": "John Doe"
}

63 PUT /customer/external/1?pretty
64 { "name": "Jane Doe"; }
65 curl -XPUT ‘localhost:9200/customer/external/1?pretty&pretty‘ -H ‘Content-Type: application/json‘ -d‘
{
"name": "Jane Doe"
}

66 PUT /customer/external/2?pretty
67 { "name": "Jane Doe"; }
68 curl -XPUT ‘localhost:9200/customer/external/2?pretty&pretty‘ -H ‘Content-Type: application/json‘ -d‘
{
"name": "Jane Doe"
}

69 curl -XDELETE ‘localhost:9200/customer/external/2?pretty&pretty‘

官方文档说可以在kibana里面写更加直观。

反正这个装好了!下一个

 安装kibana:

tar zxvf kibana-5.2.1-linux-x86_64.tar.gz

./kibana &

http://192.168.36.141:5601/status#?_g=()

技术分享

先安装filebeat:

docker run -itd --name java -p 5000:22 bc74bce8afa1 镜像是我以前做的,直接用,在另外自己的一个网站上,有时间弄下来:

测试:启动我的java包nohup java -jar /data/exchange/exchange-1.4.jar & > nohup.out >/dev/null 

获取 nohup.out

 tar zxvf filebeat-5.2.1-linux-x86_64.tar.gz

配置:

[root@4e07f322e28f ~]# cat filebeat-5.2.1-linux-x86_64/filebeat.yml
#filebeat.prospectors:
#- input_type: log
# paths:
# - /root/logstash-tutorial.log
#output.logstash:
# hosts: ["192.168.36.150:5043"]
filebeat.prospectors:
- input_type: log
paths:
- /root/nohup.out
# - /var/log/secure
# - /var/log/messages
# - /var/log/*.log
fields:
type: syslog
output.logstash:
hosts: ["192.168.36.141:5043"]

[root@4e07f322e28f ~]#

启动filebeat:./filebeat-5.2.1-linux-x86_64/filebeat -e -c filebeat-5.2.1-linux-x86_64/filebeat.yml -d "publish"

 

安装logstach:

tar zxvf logstash-5.2.1.tar.gz 

配置pip:

[root@node logstash-5.2.1]# cat first-pipline.conf
input {
beats {
port => 5043
# ssl => true
# ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
# ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch {
hosts => ["192.168.36.141:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

 启动起来:

./bin/logstash -f first-pipline.conf --config.reload.automatic

最后在kibana上面建一个索引:

好了数据出来了:

技术分享

 

 

  ---------------我是有底线的-------------------- 
作者:jt
出处:http://www.cnblogs.com/Jt00/
本文版权归作者和博客园共有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接,否则保留追究法律责任的权利。
如果文中有什么错误,欢迎指出。以免更多的人被误导。

 

elk,centos7,filebeat,elasticsearch-head详细安装步骤

标签:message   lin   mat   1.4   status   index   lte   成功   received   

原文地址:http://www.cnblogs.com/Jt00/p/7116965.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!