logstash 6.6.0 读取nginx日志 插入到elasticsearch中

标签：stash   lap   插入   write   mutate   move   meta   配置   client   

logstash.conf

input {
# For detail config for log4j as input,
# See: https://www.elastic.co/guide/en/logstash/
file {
type => "log" # log 名
path => "/home/wwwlogs/a.log" # log 路径
}
}

filter {
grok {
match => {
"message" => "^%{IPORHOST:clientip} (?:-|%{USER:ident}) (?:-|%{USER:auth}) \[%{HTTPDATE:[@metadata]timestamp}\] %{NOTSPACE:method} %{NOTSPACE:url}"
}
remove_field => ["message"]
}
mutate {
split => ["url", "?"]
add_field => ["url_params", "%{url[1]}"]
remove_field => ["url"]
}

mutate {
split => ["url_params","&"]
add_field => ["cdid_info", "%{url_params[0]}"]
add_field => ["elapsedTime_info", "%{url_params[1]}"]
add_field => ["os_info", "%{url_params[2]}"]
add_field => ["time_info", "%{url_params[3]}"]
add_field => ["uid_info", "%{url_params[4]}"]
add_field => ["wt_info", "%{url_params[5]}"]
remove_field => ["url_params"]
}

mutate {
split => ["cdid_info", "="]
add_field => ["cdid", "%{cdid_info[1]}"]
remove_field => ["cdid_info"]
}

mutate {
split => ["elapsedTime_info", "="]
add_field => ["elapsedTime", "%{elapsedTime_info[1]}"]
remove_field => ["elapsedTime_info"]
}

mutate {
split => ["os_info", "="]
add_field => ["os", "%{os_info[1]}"]
remove_field => ["os_info"]
}

mutate {
split => ["time_info", "="]
add_field => ["time", "%{time_info[1]}"]
remove_field => ["time_info"]
}

mutate {
split => ["uid_info", "="]
add_field => ["uid", "%{uid_info[1]}"]
remove_field => ["uid_info"]
}

mutate {
split => ["wt_info", "="]
add_field => ["wt", "%{wt_info[1]}"]
remove_field => ["wt_info"]
}

}
output {
# For detail config for elasticsearch as output,
# See: https://www.elastic.co/guide/en/logstash/current
elasticsearch {
hosts => "39.100.100.100:9200" #ElasticSearch host, can be array. # elasticseach 的 host
index => "index_log" #The index to write data to.
}
# 该命令是将结果输出到控制台
#stdout { codec => rubydebug }
}
cd 到 conf 文件目录下 
 检查配置是否正确
../bin/logstash -f ./logstash.conf -t

有上面提示说明配置没有问题

启动

../bin/logstash -f ./logstash.conf

————————————————
版权声明：本文为CSDN博主「可爱的狼」的原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/adorablewolf/article/details/90210969

logstash 6.6.0 读取nginx日志 插入到elasticsearch中

标签：stash   lap   插入   write   mutate   move   meta   配置   client   

原文地址：https://www.cnblogs.com/ExMan/p/11853432.html