(原)logstash-forwarder + logstash + elasticsearch + kibana

标签：des   style   http   color   os   ar   strong   for   文件   

[logstash-forwarder + logstash + elasticsearch + kibana]
------------------------------------------------------------------------------------------------------------------------------------------------
摘要:logstash-forwarder搜集日志，汇总给logstash，然后输出到elasticsearch,并由kibana展现web界面.
------------------------------------------------------------------------------------------------------------------------------------------------
一 安装
1.logstash-forwarder
see and install:
https://github.com/elasticsearch/logstash-forwarder

(logstash-forwarder有个坑. 虽然严格讲不算是logstash-forwarder的坑.
跟证书相关的:https://github.com/elasticsearch/logstash-forwarder/issues/221 <-可以不看.
下面的解决方案规避这个坑了. 下面会提到.)

2.logstash
see and install:
http://logstash.net/docs/1.4.2/tutorials/getting-started-with-logstash

3.elasticsearch

3.1.下载https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.2.tar.gz

3.2.解压到目录 elasticsearch-1.3.2

3.3. 测试安装是否成功
$ cd elasticsearch-1.3.2/
$ bin/elasticsearch
$ curl -X GET http://localhost:9200/
(保持elasticsearch一直运行. 下面将继续测试)

4.kibana:

4.1.下载https://download.elasticsearch.org/kibana/kibana/kibana-3.1.0.tar.gz

4.2. 解压到目录 kibana-3.1.0

4.3. 测试安装是否成功
$ cd kibana-3.1.0
$ vi config.js
第32行修改为：
elasticsearch: "http://localhost:9200",
注意后面有逗号.
在浏览器里打开这目录里的index.html.

------------------------------------------------------------------------------------------------------------------------------------------------
二 .方案:

client[logstash-forwarder]---|
client[logstash-forwarder]---|---log-server[logstash]--->[elasticsearch]
client[logstash-forwarder]---|

2.1 先启动elasticsearch
前面已经启动了.

2.2  开启logstash
先写logstash的配置文件:
$ cd logstash-1.4.2
$ vi test_logstash.conf
input {
  lumberjack {
    # The port to listen on
    port => 5000

    # The paths to your ssl cert and key
    ssl_certificate => "/home/xiaou/logstash-forwarder.crt"
    ssl_key => "/home/xiaou/logstash-forwarder.key"

    # Set this to whatever you want.
    type => "somelogsXXX"
  }
}
output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

还要产生自签证书:
$ openssl req -subj ‘/CN=localhost/‘ -x509 -batch -nodes -newkey rsa:2048 -keyout /home/xiaou/logstash-forwarder.key -out /home/xiaou/logstash-forwarder.crt
(这里用“-subj ‘/CN=localhost/‘”规避了上面提到的logstash-forwarder的坑)

然后启动logstash：
$ bin/logstash -f test_logstash.conf

2.3 启动logstash-forwarder
先写logstash-forwarder的配置文件:
$ cd logstash-forwarder
$ vi test_forwarder.conf
{
  "network": {
    "servers": [ "localhost:5000" ],
 "ssl ca": "/home/xiaou/logstash-forwarder.crt",
    "timeout": 5
  },
  "files": [
    {
      "paths": [
        "/var/log/linshi.txt",
        "/var/log/*.log"
      ],
      "fields": {
        "type": "linshiXX"
      }
    }
  ]
}
(这里配置文件的写法也是规避了前面提到的logstash-forwarder的坑:servers没用ip)
启动logstash-forwarder:
$ ./logstash-forwarder -config test_forwarder.conf
logstash-forwarder启动后就会与logstash建立tcp连接.

测试, 写日志，观察运行logstash的终端的输出:
$ echo 1234 >> /var/log/linshi.txt

2.4  打开kibana,展现最终汇总到elasticsearch的日志.
(唯kibana不能算是服务, 它只是一个“阅读器”.)
用浏览器打开kibana-3.1.0目录下的index.html,看右边倒数第五行有个链接。打开.
------------------------------------------------------------------------------------------------------------------------------------------------
End.

(原)logstash-forwarder + logstash + elasticsearch + kibana

标签：des   style   http   color   os   ar   strong   for   文件   

原文地址：http://www.cnblogs.com/xiaouisme/p/3977721.html