码迷,mamicode.com
首页 > Web开发 > 详细

关于Flume中Chanel.Selector.header解释

时间:2016-09-02 06:31:24      阅读:199      评论:0      收藏:0      [点我收藏+]

标签:

flume内置的ChannelSelector有两种,分别是Replicating和Multiplexing。

Replicating类型的ChannelSelector会针对每一个Event,拷贝到所有的Channel中,这是默认的ChannelSelector。

replicating类型的ChannelSelector例子如下

 1 a1.sources = r1
 2 a1.channels = c1 c2 # 如果有100个Event,那么c1和c2中都会有这100个事件
 3 
 4 a1.channels.c1.type = memory
 5 a1.channels.c1.capacity = 1000
 6 a1.channels.c1.transactionCapacity = 100
 7 
 8 
 9 a1.channels.c2.type = memory
10 a1.channels.c2.capacity = 1000
11 a1.channels.c2.transactionCapacity = 100

 

Multiplexing类型的ChannelSelector会根据Event中Header中的某个属性决定分发到哪个Channel。

multiplexing类型的ChannelSelector例子如下:

1 a1.sources = r1
2 
3 a1.sources.source1.selector.type = multiplexing
4 a1.sources.source1.selector.header = validation # 以header中的validation对应的值作为条件
5 a1.sources.source1.selector.mapping.SUCCESS = c2 # 如果header中validation的值为SUCCESS,使用c2这个channel
6 a1.sources.source1.selector.mapping.FAIL = c1 # 如果header中validation的值为FAIL,使用c1这个channel
7 a1.sources.source1.selector.default = c1 # 默认使用c1这个channel
a1.sources.source1.selector.header = validation # 以header中的validation对应的值作为条件

同理,如下conf文件:
 1 # example.conf: A single-node Flume configuration
 2 
 3 # Name the components on this agent
 4 a1.sources = r1
 5 a1.sinks = k1
 6 a1.channels = c1
 7 
 8 # Describe/configure the source
 9 a1.sources.r1.type = netcat
10 a1.sources.r1.bind = localhost
11 a1.sources.r1.port = 44444
12 
13 a1.sources.r1.interceptors = i1
14 a1.sources.r1.interceptors.i1.type = regex_extractor
15 a1.sources.r1.interceptors.i1.regex = (\\d):(\\d):(\\d)
16 a1.sources.r1.interceptors.i1.serializers = s1 s2 s3
17 a1.sources.r1.interceptors.i1.serializers.s1.name = ip
18 a1.sources.r1.interceptors.i1.serializers.s2.name = domain
19 a1.sources.r1.interceptors.i1.serializers.s3.name = course
20 
21 a1.sources.r1.selector.type = multiplexing
22 a1.sources.r1.selector.header = course
23 a1.sources.r1.selector.default = c1
24 
25 # Describe the sink
26 a1.sinks.k1.type = logger
27 
28 # Use a channel which buffers events in memory
29 a1.channels.c1.type = memory
30 a1.channels.c1.capacity = 1000
31 a1.channels.c1.transactionCapacity = 100
32 
33 # Bind the source and sink to the channel
34 a1.sources.r1.channels = c1
35 a1.sinks.k1.channel = c1

source r1中的头部有IP、Domain和cource三种信息,而r1的selector.header = course,表示selector只对IP,Domain和Cource中的Cource进行判断选择,然后再划分channel。

关于Flume中Chanel.Selector.header解释

标签:

原文地址:http://www.cnblogs.com/Skyar/p/5831935.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!