标签:单机 系统 pac sof bin server 3.3 kafka war
搭建步骤:
1.环境准备:在win7系统 内存(8G)的硬件环境下,搭建1台centOS虚拟机
1.1给机器配置主机名(ip需要固定)
1.2 Xshell远程命令连接上该虚拟机
1.3 为该台虚拟机配置用户/密码登录
2.配置jdk
2.1指定目录下解压jdk压缩包 # /usr/software/
2.2 配置jdk环境变量
JAVA_HOME=/usr/software/jdk1.8.0_131
PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/sbin
3.配置flume
3.1指定目录下解压apache-flume-1.8.0-bin压缩包 # /usr/software/
3.2 复制apache-flume-1.8.0-bin/conf/flume-conf.properties.template文件至apache-flume-1.8.0-bin/http.conf,修改http.conf
编写配置文件(小案例)
#example.conf:单节点Flume配置
#命名Agent a1的组件
a1.sources = r1
a1.sinks = k1
a1.channels = c1
#描述/配置Source
a1.sources.r1.type = netcat
a1.sources.r1.bind = 0.0.0.0
a1.sources.r1.port = 44444
#描述Sink
a1.sinks.k1.type = logger
#描述内存Channel
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
#为Channle绑定Source和Sink
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
3.3 agent里三个组件并非耦合性较大,source必须存在,channel和sink也可以只配置其一,可直接从channel配置kafka,如下案例:
#Agent
a1.sources = r1
a1.channels = c1
#source
a1.sources.r1.type = spooldir
a1.sources.r1.spoolDir=/usr/files
a1.sources.r1.channels = c1
#channel
a1.channels.c1.type = org.apache.flume.channel.kafka.KafkaChannel
a1.channels.c1.kafka.bootstrap.servers = kafka01:9092,kafka02:9092,kafka03:9092
a1.channels.c1.kafka.topic = flume33
4.启动flume
在apache-flume-1.8.0-bin/目录下启动
./bin/flume-ng agent -c conf -f http.conf -n a1 -Dflume.root.logger=INFO,console
标签:单机 系统 pac sof bin server 3.3 kafka war
原文地址:http://www.cnblogs.com/runnerjack/p/7846753.html