大数据：从入门到XX(六)

时间：2016-07-15 13:58:01 阅读：244 评论：0 收藏：0 [点我收藏+]

标签：zookeeper3.4.8集群部署在三台redhat虚拟机上同时演练leader选举过程

什么是ZooKeeper，看看ZooKeeper官网怎么说：

Apache ZooKeeper is an effort to develop and maintain an open-source server which enables highly reliable distributed coordination.

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

点此进入：ZooKeeper Overview

ZooKeeper是分布式应用环境中的基础组件，在hadoop集群中，hdfs和yarn都要用到ZooKeeper来配置可自动切换的高可用（HA）系统，本文根据ZooKeeper官方文档，选用三台虚拟机，部署ZooKeeper集群，用到的虚拟机为前文提到的： hadoop01、hadoop02、hadoop03。以下为详细安装步骤。

1、在hadoop01机器上，创建zookeeper用户，加入hadoop组（本系列文档中有关于创建hadoop组的部分)

#以root用户登录hadoop01机器

[root@hadoop01 hadoop]# useradd -m -g hadoop zookeeper

#首次启用，设置zookeeper用户密码
[root@hadoop01 hadoop]# passwd zookeeper

2、下载最新的ZooKeeper发布包：官网推荐的镜像地址

#zookeeper-3.4.8.tar.gz拷贝至 /home/zookeeper 目录
[zookeeper@hadoop01 ~]$ tar zxvf zookeeper-3.4.8.tar.gz
解压后多一个文件夹 zookeeper-3.4.8

3、使用zookeeper用户登录，设置环境变量

#在hadoop01机器上以zookeeper用户身份操作
[root@hadoop01 hadoop]# su - zookeeper

[zookeeper@hadoop01 ~]$ vi .bash_profile

export PATH
#add start 20160628
export ZOOKEEPER_HOME=/home/zookeeper/zookeeper-3.4.8
export PATH=$PATH:$ZOOKEEPER_HOME/bin

#add end 20160628

#保存以上环境配置，退出vi后，执行以下命令是环境变量生效

[zookeeper@hadoop01 ~]$ source .bash_profile

4、在hadoop01机器上设置zookeeper相关参数

[zookeeper@hadoop01 ~]$ cd zookeeper-3.4.8/conf/

#从样例复制一个配置文件模板

[zookeeper@hadoop01 zookeeper-3.4.8]$ cp zoo_sample.cfg zoo.cfg
#编辑配置文件
[zookeeper@hadoop01 conf]$ vi zoo.cfg

# The number of milliseconds of each tick

tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just

# example sakes.

#modify start 20160628

#dataDir=/tmp/zookeeper

dataDir=/home/zookeeper/zookeeper-3.4.8/zkdata

#modify end 20160628

# the port at which the clients will connect

clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature

#autopurge.purgeInterval=1

#添加的配置信息

#modify start 20160628

server.1=hadoop01:2888:3888
server.2=hadoop02:2888:3888

server.3=hadoop03:2888:3888

#modify end 20160628

5、在hadoop01机器上创建工作目录，设置zookeeper ID值（在不同服务器上ID不能重复）

#在hadoop01机器上，以zookeeper用户执行以下命令

[zookeeper@hadoop01 ~]$ cd /home/zookeeper/zookeeper-3.4.8

[zookeeper@hadoop01 zookeeper-3.4.8]$ mkdir zkdata
[zookeeper@hadoop01 zkdata]$ cd zkdata/
[zookeeper@hadoop01 zkdata]$ echo ‘1‘ >myid

6、在hadoop02上执行相关操作

#在hadoop02机器上，创建zookeeper用户
[root@hadoop02 hadoop]# useradd -m -g hadoop zookeeper

#在hadoop01机器上，执行以下操作

[zookeeper@hadoop01 ~]$ cd /home/zookeeper

[zookeeper@hadoop01 ~]$ scp -r zookeeper-3.4.8 hadoop02:$PWD

#在hadoop02机器上，切换zookeeper用户身份

[root@hadoop02 hadoop]# su - zookeeper

[zookeeper@hadoop02 ~]$ vi .bash_profile

export PATH
#add start 20160628
export ZOOKEEPER_HOME=/home/zookeeper/zookeeper-3.4.8
export PATH=$PATH:$ZOOKEEPER_HOME/bin

#add end 20160628

#保存以上环境配置，退出vi后，执行以下命令使环境变量生效

[zookeeper@hadoop02 ~]$ source .bash_profile

7、创建工作目录，设置zookeeper ID值

#在hadoop02机器上，以zookeeper用户执行以下命令

[zookeeper@hadoop02 ~]$ cd /home/zookeeper/zookeeper-3.4.8

[zookeeper@hadoop02 zookeeper-3.4.8]$ mkdir zkdata
[zookeeper@hadoop02 zkdata]$ cd zkdata/
[zookeeper@hadoop02 zkdata]$ echo ‘2‘ >myid

8、在hadoop03上执行相关操作

#在hadoop03机器上，创建zookeeper用户
[root@hadoop03 hadoop]# useradd -m -g hadoop zookeeper

#在hadoop01机器上，执行以下操作

[zookeeper@hadoop01 ~]$ cd /home/zookeeper

[zookeeper@hadoop01 ~]$ scp -r zookeeper-3.4.8 hadoop03:$PWD

#在hadoop03机器上，切换zookeeper用户身份

[root@hadoop03 hadoop]# su - zookeeper

[zookeeper@hadoop03 ~]$ vi .bash_profile

export PATH
#add start 20160628
export ZOOKEEPER_HOME=/home/zookeeper/zookeeper-3.4.8
export PATH=$PATH:$ZOOKEEPER_HOME/bin

#add end 20160628

#保存以上环境配置，退出vi后，执行以下命令使环境变量生效

[zookeeper@hadoop03 ~]$ source .bash_profile

9、创建工作目录，设置zookeeper ID值

#在hadoop03机器上，以zookeeper用户执行以下命令

[zookeeper@hadoop03 ~]$ cd /home/zookeeper/zookeeper-3.4.8

[zookeeper@hadoop03 zookeeper-3.4.8]$ mkdir zkdata
[zookeeper@hadoop03 zkdata]$ cd zkdata/
[zookeeper@hadoop03 zkdata]$ echo ‘3‘ >myid

10、启动zookeeper集群

#在hadoop01机器上运行

[zookeeper@hadoop01 ~]$ zkServer.sh start

ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

#在hadoop02机器上运行

[zookeeper@hadoop02 ~]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

#在hadoop03机器上运行

[zookeeper@hadoop03 ~]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

11、查看zookeeper集群状态

[zookeeper@hadoop03 ~]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg
Mode: follower

[zookeeper@hadoop02 ~]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.8/bin/../conf/zoo.cfg
Mode: leader

12、查看执行的进程

[zookeeper@hadoop02 ~]$ jps -l
5449 org.apache.zookeeper.server.quorum.QuorumPeerMain

13、关闭zookeeper集群

#在hadoop01机器上运行

[zookeeper@hadoop01 ~]$ zkServer.sh stop

#在hadoop02机器上运行

[zookeeper@hadoop02 ~]$ zkServer.sh stop

#在hadoop03机器上运行

[zookeeper@hadoop03 ~]$ zkServer.sh stop

14、下图演示了开启zookeeper集群，以及杀掉leader服务器的进程后，自动选举新的leader服务器情况。