【Flume】 flume 负载均衡环境的搭建 load_balance

标签：flume

flume的负载均衡即每次按照一定的算法选择sink输出到指定地方，如果在文件输出量很大的情况下，负载均衡还是很有必要的，通过多个通道输出缓解输出压力

flume内置的负载均衡的算法默认是round robin，轮询算法，按序选择

下面看一下具体实例：

 
# Name the components on this agent
a1.sources = r1
a1.sinks = k1 k2
a1.channels = c1
 
# Describe/configure the source
a1.sources.r1.type = exec
a1.sources.r1.channels=c1
a1.sources.r1.command=tail -F /root/dev/biz/logs/bizlogic.log 

#define sinkgroups
a1.sinkgroups=g1
a1.sinkgroups.g1.sinks=k1 k2
a1.sinkgroups.g1.processor.type=load_balance
a1.sinkgroups.g1.processor.backoff=true
a1.sinkgroups.g1.processor.selector=round_robin

#define the sink 1
a1.sinks.k1.type=avro
a1.sinks.k1.hostname=192.168.11.179
a1.sinks.k1.port=9876  

#define the sink 2
a1.sinks.k2.type=avro
a1.sinks.k2.hostname=192.168.11.178
a1.sinks.k2.port=9876


# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
 
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
a1.sinks.k2.channel=c1

以上是flume agent client的配置，server端的配置跟上一篇文章一样http://blog.csdn.net/simonchi/article/details/42494461

同样通过脚本去模拟生成文件内容，触发tail -F，执行两次，发现第一次输出在179，第二次输出在178，说明负载均衡已经生效了！！

很简单！！

【Flume】 flume 负载均衡环境的搭建 load_balance

标签：flume

原文地址：http://blog.csdn.net/simonchi/article/details/42495299