Elastic Stack之kafka安装03

标签：kafka安装   防火   produce   receive   group   下载   nohup   ack   lan   

机器初始化

参考ES安装，安装java，关闭防火墙，selinux等

安装zookeeper

yum install -y wget
wget https://downloads.apache.org/kafka/2.6.0/kafka_2.13-2.6.0.tgz #下载zookeeper
mkdir /usr/share/kafka
tar zxvf kafka_2.13-2.6.0.tgz -C /usr/share/kafka/ #解压至/usr/share/kafka/下
mkdir -p /data/zookeeper/{data,logs} #创建zookeeper数据及日志存放路径
mkdir -p /data/kafka/{data,logs} #创建kafka数据及日志存放路径

配置zookeeper

vim /usr/share/kafka/kafka_2.13-2.6.0/config/zookeeper.properties #修改zookeeper配置文件
-----------------------------------------------------------------
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/logs
clientPort=2181
maxClientCnxns=0
admin.enableServer=false
tickTime=2000
initLimit=20
syncLimit=10
server.1=192.168.110.64:2888:3888
server.2=192.168.110.65:2888:3888
server.3=192.168.110.66:2888:3888

配置kafka

-----------------------------------------------------------------
#每台生成一个Myid 后期要用，我有三台kafka，第一台设置为1，第二台设置为2，第三台设置为3
echo 2 > /data/zookeeper/data/myid #创建Myid后面启动kafka时使用
vim /usr/share/kafka/kafka_2.13-2.6.0/config/server.properties
-----------------------------------------------------------------
broker.id=2 #修改为自己创建的myid
listeners=PLAINTEXT://192.168.110.66:9092 #修改为kafka本机地址
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/data/kafka/logs
num.partitions=6
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=2
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
zookeeper.connect=192.168.110.64:2181,192.168.110.65:2181,192.168.110.66:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0

启动kafka

cd /usr/share/kafka/kafka_2.13-2.6.0/
nohup bin/zookeeper-server-start.sh config/zookeeper.properties & #启动zookeeper
yum install -y net-tools
netstat -nlpt | grep -E "2181|2888|3888" #验证zookeeper群集，群集拥有2888端口即为leader
nohup bin/kafka-server-start.sh config/server.properties & #启动kafka
#创建一个topic验证kafka群集
bin/kafka-topics.sh --create --zookeeper 192.168.110.61:2181 --replication-factor 1 --partitions 1 --topic testtopic
#查看topic
bin/kafka-topics.sh --zookeeper 192.168.110.64 --list
#测试生产消息
bin/kafka-console-producer.sh --broker-list 192.168.110.64:9092 --topic testtopic
#测试消费消息
bin/kafka-console-consumer.sh --bootstrap-server 192.168.110.64:9092 --topic testtopic --from-beginning
#查看topic中的内容
bin/kafka-console-consumer.sh --bootstrap-server 192.168.36.194:9092,192.168.36.195:9092,192.168.36.196:9092 --topic TestEx01_Transport_Logs

Elastic Stack之kafka安装03

标签：kafka安装   防火   produce   receive   group   下载   nohup   ack   lan   

原文地址：https://blog.51cto.com/11258494/2565128