标签：
flume负载均衡配置
集群DNS配置如下：
hadoop-maser 192.168.177.162
machine-0192.168.177.158
machine-1191.168.177.167
配置主Flume，在hadoop-maser机上。配置文件为loadbalance.properties。
agent.sources=s1
agent.channels=c1
agent.sinks=k1 k2
agent.sinkgroups = g1
agent.sinkgroups.g1.sinks = k1 k2
agent.sinkgroups.g1.processor.type =load_balance
agent.sinkgroups.g1.processor.selector =round_robin
agent.sinkgroups.g1.processor.backoff =true
agent.sources.s1.type=avro
agent.sources.s1.channels=c1
agent.sources.s1.bind=0.0.0.0
agent.sources.s1.port=51515
agent.sources.s1.interceptors=i1
agent.sources.s1.interceptors.i1.type=timestamp
agent.channels.c1.type=memory
agent.sinks.k1.channel = c1
agent.sinks.k1.type = avro
agent.sinks.k1.hostname = machine-0
agent.sinks.k1.port = 51515
agent.sinks.k2.channel = c1
agent.sinks.k2.type = avro
agent.sinks.k2.hostname = machine-1
agent.sinks.k2.port = 51515
配置machine-0，定义文件conf/loadbalance.properties。
#loadbalance.properties。
agent.sources=s1
agent.channels=c1
agent.sinks=k1
agent.sources.s1.type=avro
agent.sources.s1.channels=c1
agent.sources.s1.bind=0.0.0.0
agent.sources.s1.port=51515
agent.channels.c1.type=memory
agent.channels.c1.capacity = 10000
agent.channels.c1.transactionCapacity =10000
agent.channels.c1.byteCapacityBufferPercentage= 20
agent.channels.c1.byteCapacity = 800000
agent.sinks.k1.type=hdfs
agent.sinks.k1.channel=c1#这台机上没有安装hadoop节点，请配位#hdfs://ip-address:8020/flume/%Y/%m
agent.sinks.k1.hdfs.path=/flume/%Y/%m
agent.sinks.k1.hdfs.filePrefix=flume
agent.sinks.k1.hdfs.fileSuffix=.log
agent.sinks.k1.hdfs.rollInterval=3600
agent.sinks.k1.hdfs.rollCount=0
agent.sinks.k1.hdfs.rollSize=0
agent.sinks.k1.hdfs.fileType=DataStream
agent.sinks.k1.hdfs.writeFormat=Text
agent.sinks.k1.hdfs.useLocalTimeStamp=false
与machine-0一样，配置machine-1。
启动flume-ng。进入flume目录下（将flume配置到linux环境变量中，不需要这一步骤，直接启动就可以了），输入下列命令。
bin/flume-ng agent -n agent -c conf –f \
conf/loadbalance.properties-Dflume.root.logger=DEBUG,console
在三台机上，启动flume。
标签：
原文地址：http://www.cnblogs.com/zwgblog/p/5733691.html
