码迷,mamicode.com
首页 > 数据库 > 详细

logstash输出到influxdb

时间:2014-09-25 22:12:27      阅读:355      评论:0      收藏:0      [点我收藏+]

标签:blog   http   io   os   ar   文件   sp   div   on   

用了这个logstash扩展

https://github.com/PeterPaulH/logstash-influxdb/blob/master/src/influxdb.rb

把这个文件放到 logstash-1.4.2/lib/logstash/outputs

看一下午logstash的文档,终于解决了自己的需求

用python描述就是这样的

开发要求统计日志中各种类型的数量

while True:
    line = f.readline()
    try:
        if ‘"type":"text","receiver_id"‘ in line:
            type = ‘directmessage‘
        elif ‘"subtype":"unfollow"‘ in line:
            type = ‘unfollow‘
        elif ‘"subtype":"follow"‘ in line:
            type = ‘follow‘
        elif ‘"subtype":"status"‘ in line:
            type = ‘weibo‘
        elif ‘"subtype":"comment"‘ in line:
            type = ‘comment‘
        else:
            type = None

        if type:
            data = [
                {"name":"pingpong_processor",
                 "columns" : ["type"],
                 "points" : [[type]]

                }
            ] 

 logstash配置文件如下

input {
  stdin {}
}

filter {
  if ‘"type":"text"‘ in [message] {
      mutate {
        add_field => { "type" => "directmessage" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else if ‘"subtype":"unfollow"‘ in [message] {
      mutate {
        add_field => { "type" => "unfollow" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else if ‘"subtype":"follow"‘ in [message] {
      mutate {
        add_field => { "type" => "follow" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else if ‘"subtype":"status"‘ in [message] {
      mutate {
        add_field => { "type" => "weibo" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else if ‘"subtype":"comment"‘ in [message] {
      mutate {
        add_field => { "type" => "comment" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else {
      drop {}
  }
  
}

output {
  influxdb {
    host => "10.75.28.180"
    port => 4444
    name => ["pingpong_processor"]
    columns => ["type", "host"]
    points => [
        "%{type}", "c",
        "%{host}", "c"
    ]

  }

  stdout {}
}

 别忘记把influxdb的配置也修改下,因为默认upd协议是没有打开的

# Configure the udp api
  [input_plugins.udp]
  enabled = true
  port = 4444
  database = "pingpong_processor"

 我用的influxdb版本是 influxdb-0.8.2-1.x86_64,用/etc/init.d/influxdb这个启动报错,无奈手动启动的

/usr/bin/influxdb -pidfile /tmp/influxdb.pid -config config.toml

看一下udp端口是否启动了

netstat -anup|grep influxdb
udp        0      0 :::4444                     :::*                                    27512/influxdb

 完工

logstash输出到influxdb

标签:blog   http   io   os   ar   文件   sp   div   on   

原文地址:http://www.cnblogs.com/txwsqk/p/3993544.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!