码迷,mamicode.com
首页 > 数据库 > 详细

LogStash如何通过jdbc 从mysql导入elasticsearch

时间:2017-03-03 19:07:58      阅读:779      评论:0      收藏:0      [点我收藏+]

标签:cte   nload   .sql   odi   line   sql   ade   jdbc   remove   

input {
    stdin {
    }
    jdbc {
      # mysql jdbc connection string to our backup databse
      jdbc_connection_string => "jdbc:mysql://localhost:3306/userdb?useUnicode=true&characterEncoding=utf-8&useSSL=false"
      # the user we wish to excute our statement as
      jdbc_user => "user"
      jdbc_password => "pass"
      # the path to our downloaded jdbc driver
      jdbc_driver_library => "mysql-connector-java-5.1.40-bin.jar"
      # the name of the driver class for mysql
      jdbc_driver_class => "com.mysql.jdbc.Driver"
      jdbc_paging_enabled => "true"
      jdbc_page_size => "50000"
      statement_filepath => "mysqltestsql.sql"
      schedule => "* * * * *"
      type => "jdbc"
      use_column_value => true
      tracking_column => id
      last_run_metadata_path => "C:\logstash-5.2.0\bin\last_run_metadata_path\last_run_metadata_path"
    }
}

filter {
    json {
        source => "message"
        remove_field => ["message","_id"]
    }
}

output {  
    elasticsearch {
        action => "index"
        hosts => ["http://localhost:9200"] 

        index => "mysqltestdb2"
        #index => "user"
        workers => 1
    }
     #stdout {
         #codec => rubydebug
         #codec => json_lines
     #}
}

 

 mysqltestsql.sql

select id, name from userinfo WHERE id > :sql_last_value

 

LogStash如何通过jdbc 从mysql导入elasticsearch

标签:cte   nload   .sql   odi   line   sql   ade   jdbc   remove   

原文地址:http://www.cnblogs.com/haoliansheng/p/6497870.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!