标签:cti nsf als 配置 class pat regex 说明 pos
说明:
mariadb audit log是 mariadb 的审计日志
目的是把日志拆分成 tab 键分隔的字段
直接附上 fluentd 配置文件
<system> log_level error </system> <source> @type tail path /data/logs/mariadb/server_audit.log tag mysql_audit pos_file /data/logs/mariadb/fluentd.pos <parse> @type multiline format_firstline /^\d{8}/ format1 /^(?<dt>\d{8} \d{2}:\d{2}:\d{2}),(?<hostname>[^,]+),(?<user>[^,]+),(?<ip>[^,]+),(?<connid>[^,]+),(?<queryid>[^,]+),(?<action>[^,]+),(?<db>[^,]+),(?<message>.*),(?<retcode>\d+)$/ </parse> </source> <filter mysql_audit> @type grep <regexp> key action pattern QUERY </regexp> <exclude> key user pattern lagou_status </exclude> <exclude> key db pattern information_schema </exclude> </filter> <filter mysql_audit> @type record_transformer enable_ruby <record> message ${record["message"].gsub(/\s/, ‘ ‘)} message ${record["message"].gsub(/\s+/, ‘ ‘)} </record> </filter> <match mysql_audit> #@type stdout @type file path "/data/logs/mariadb/#{ENV[‘HOSTNAME‘]}" time_slice_format %Y%m%d%H time_slice_wait 5m add_path_suffix false append true compress gzip <format> @type csv fields dt,hostname,user,ip,action,db,message,retcode delimiter ‘ ‘ </format> </match>
fluentd 比 logstash 内存占用大大下降
分析同样的日志 logstash 占用700M, fluentd 占用35M
不过 cpu 占用相当,对于日志量大的机器 cpu 到100%
看来对日志做正则过滤很损耗 cpu
标签:cti nsf als 配置 class pat regex 说明 pos
原文地址:https://www.cnblogs.com/txwsqk/p/9397853.html