码迷,mamicode.com
首页 > Web开发 > 详细

【原创】大数据基础之Logstash(3)应用之http(in和out)

时间:2019-03-20 01:15:24      阅读:245      评论:0      收藏:0      [点我收藏+]

标签:current   原创   inpu   一个   guide   应用   大数据   数据   elastic   

一个logstash很容易通过http打断成两个logstash实现跨服务器或者跨平台间数据同步,比如原来的流程是

logstash: nginx log -> kafka

打断成两个是

logstash1: nginx log -> http out

logstash2: http in ->kafka

具体如下

http out


  filter {
    ruby {
      code => "event.cancel if not event.get(‘message‘).include?‘something‘"
    }
  }

output {
    http {
      url => "http://test.server:10000"
      codec => "plain"
      format => "json"
      content_type => "application/json"
      http_method => "post"
    }
}

可以通过filter跳过不需要的记录

http in

input {
    http {
        host => "0.0.0.0"
        port => 10000
        additional_codecs => {"application/json"=>"json"}
        codec => "plain"
        threads => 4
        ssl => false
    }
}

 

http in

参考:https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html

http out

参考:https://www.elastic.co/guide/en/logstash/current/plugins-outputs-http.html

 

【原创】大数据基础之Logstash(3)应用之http(in和out)

标签:current   原创   inpu   一个   guide   应用   大数据   数据   elastic   

原文地址:https://www.cnblogs.com/barneywill/p/10562281.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!