码迷,mamicode.com
首页 > 其他好文 > 详细

CentOS 7环境下Kafka的安装和基本使用

时间:2017-12-21 12:02:44      阅读:160      评论:0      收藏:0      [点我收藏+]

标签:runtime   available   blog   bre   centos7   expec   keyword   ble   using   

CentOS 7环境下Kafka的安装和基本使用

基础环境

Windows 10 X64

VMware-workstation-full-12.0.0-2985596

CentOS Linux release 7.4.1708

CentOS 7环境下安装Kafka

安装JDK

sudo tar -vxzf jdk-8u152-linux-x64.tar.gz
sudo mv jdk1.8.0_152 /opt/java
  • 配置java环境变量
vim /etc/profile
#在profile文件的最后面添加
export JAVA_HOME=/opt/jdk1.8.0_152
export PATH=$JAVA_HOME/bin:$PATH
#添加成功后通过java -version来查看jdk是否安装成功
java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)

安装Zookeeper

  • 通过wget下载zookeeper到本机
sudo wdget https://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.3.6/zookeeper-3.3.6.tar.gz
  • 解压并将文件移动到/opt/zookeeper目录下
# 解压zookeeper
sudo tar -zxvf zookeeper-3.3.6.tar.gz
# 将zookeeper移动到/opt/zookeeper目录下
sudo mv zookeeper-3.3.6 /opt/zookeeper
  • 编辑zookeeper的配置
# 复制一份zoo_sample.cfg文件并改名为zoo.cfg
sudo cp /opt/zookeeper/zoo_sample.cfg zoo.cfg
# 编辑zoo.cfg 文件
sudo vim /opt/zookeeper/zoo.cfg
#主要修改dataDir和server.1=127.0.0.1:2888:3888这2处
# the directory where the snapshot is stored.
dataDir=/opt/zookeeper/data
# the port at which the clients will connect
clientPort=2181
server.1=127.0.0.1:2888:3888
  • 配置zookeeper环境变量
sudo vim /etc/profile
#添加如下内容
ZOOKEEPER_HOME=/opt/zookeeper
PATH=$PATH:ZOOKEEPER_HOME/bin
export ZOOKEEPER_HOME
  • 启动zookeeper
zkServer.sh start
# 如果上面命令报错,则cd /opt/zookeeper/bin 目录
./zkServer.sh start

安装kafka

  • 通过wget下载kafka到本机
sudo wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/1.0.0/kafka_2.12-1.0.0.tgz
  • 解压并将文件移动到/opt/kafka目录下
# 解压
sudo tar -zxvf kafka_2.12-1.0.0.tgz
# 移动
sudo mv kafka_2.12-1 /opt/kafka
  • 编辑kafka的配置
#创建日志存放目录
cd /opt/kafka
mkdir -p log/kafka
#修改配置文件/opt/kafka/config/server.properties
sudo vim /opt/kafka/config/server.properties
#主要修改下面几项内容如下:
broker.id=181
delete.topic.enable=true
listeners = PLAINTEXT://127.0.0.1:9092
log.dirs=/opt/kafka/log/kafka
zookeeper.connect=127.0.0.1:2181
  • 配置kafka环境变量
sudo vim /etc/profile
#添加如下内容:
KAFKA_HOME=/opt/kafka
PATH=$PATH:$KAFKA_HOME/bin
export PATH KAFKA_HOME
  • 启动kafka
kafka-server-start.sh /opt/kafka/config/server.properties 

CentOS 7环境下Kafka的基本使用

  • 创建一个Topic (test)
kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 1 --topic test
  • Producer向Topic发送消息
kafka-console-producer.sh --broker-list 127.0.0.1:9092 --topic test
[yang@localhost bin]$ ./kafka-console-producer.sh --broker-list 127.0.0.1:9092 --topic test
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
>producer send one message
>producer send two message
>

技术分享图片

  • Consumer读取Topic的消息
kafka-console-consumer.sh --zookeeper 127.0.0.1:2181 --topic test --from-beginning
[yang@localhost bin]$ ./kafka-console-consumer.sh --zookeeper 127.0.0.1:2181 -topic test --from-beginning
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
[2017-11-19 20:32:13,047] WARN Connected to an old server; r-o mode will be unavailable (org.apache.zookeeper.ClientCnxnSocket)
[2017-11-19 20:32:13,085] WARN Connected to an old server; r-o mode will be unavailable (org.apache.zookeeper.ClientCnxnSocket)
[2017-11-19 20:32:13,100] WARN Connected to an old server; r-o mode will be unavailable (org.apache.zookeeper.ClientCnxnSocket)
[2017-11-19 20:32:13,265] WARN Connected to an old server; r-o mode will be unavailable (org.apache.zookeeper.ClientCnxnSocket)
producer send one message
producer send two message

技术分享图片

参考资料

CentOS 搭建JDK环境

CentOS7安装配置ZooKeeper集群

CentOS7.0安装配置Kafka集群

kafka环境搭建1-单机版

Confluent.Kafka (Kafka .NET client)

CentOS 7环境下Kafka的安装和基本使用

标签:runtime   available   blog   bre   centos7   expec   keyword   ble   using   

原文地址:http://www.cnblogs.com/Yang2012/p/8078586.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!