1、分布式系统
2、ZooKeeper介绍
Paxos算法（读帕克西）
ZAB协议
3、ZooKeeper部署运行
部署
ZooKeeper可执行脚本
脚本说明：
zkServer.sh：ZooKeeper服务器的启动、停止和重启脚本； zkCli.sh：ZooKeeper的简易客户端； zkEnv.sh：设置ZooKeeper的环境变量； zkCleanup.sh：清理ZooKeeper历史数据，包括事务日志文件和快照数据文件。
启动服务：
[hadoop@dnode2 ~]$ zkServer.sh start JMX enabled by default Using config: /usr/local/zookeeper3.4.6/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [hadoop@dnode2 ~]$
停止服务：
[hadoop@dnode2 ~]$zkServer.sh stop
4、ZooKeeper客户端
客户端脚本
[hadoop@nnode ~]$ zkCli.sh Connecting to localhost:2181 2015-08-04 22:07:22,645 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2015-08-04 22:07:22,650 [myid:] - INFO [main:Environment@100] - Client environment:host.name=nnode 2015-08-04 22:07:22,650 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.7.0_75 2015-08-04 22:07:22,658 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2015-08-04 22:07:22,658 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/local/jdk1.7/jre
当看到如下信息时，表示已经成功连接上本地的ZooKeeper服务器了。
注意：上面的命令没有显示指定ZooKeeper服务器地址，那么默认是连接本地的ZooKeeper服务器。如果希望连接指定的ZooKeeper服务器，可以通过如下方式实现：
[hadoop@nnode ~]$zkCli.sh -server ip:port
通过zkCli.sh连接后查看帮助：
[zk: localhost:2181(CONNECTED) 4] help ZooKeeper -server host:port cmd args connect host:port get path [watch] ls path [watch] set path data [version] rmr path delquota [-n|-b] path quit printwatches on|off create [-s] [-e] path data acl stat path [watch] close ls2 path [watch] history listquota path setAcl path acl getAcl path sync path redo cmdno addauth scheme auth delete path [version] setquota -n|-b val path [zk: localhost:2181(CONNECTED) 5]
ls：
# 此时ZooKeeper已经启动，而Hadoop HA还未启动时调用zkCli.sh [zk: localhost:2181(CONNECTED) 0] ls / [hadoop-ha, hbase, zookeeper] [zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha [cluster] [zk: localhost:2181(CONNECTED) 2] ls /hadoop-ha/cluster [ActiveBreadCrumb] [zk: localhost:2181(CONNECTED) 3] ls /hadoop-ha/cluster/ActiveBreadCrumb [] # 当Hadoop HA启动后，再次调用zkCli.sh [zk: localhost:2181(CONNECTED) 0] ls / [hbase, hadoop-ha, zookeeper] [zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha [cluster] [zk: localhost:2181(CONNECTED) 2] ls /hadoop-ha/cluster [ActiveBreadCrumb, ActiveStandbyElectorLock] [zk: localhost:2181(CONNECTED) 3] ls /hadoop-ha/cluster/ActiveBreadCrumb [] [zk: localhost:2181(CONNECTED) 4] ls /hadoop-ha/cluster/ActiveStandbyElectorLock [] [zk: localhost:2181(CONNECTED) 5] # 查看hadoop-daemon.sh start zkfc时的日志输出 # 我的Hadoop集群环境有两个namenode节点，分别为主机：nnode和dnode1 # 在启动Hadoop集群时首选在nnode主机上执行命令hadoop-daemon.sh start namenode， # 再在主机dnode1上执行命令hadoop-daemon.sh start namenode # 启动yarn等的操作省略，最后在nnode和dnode1上依次执行hadoop-daemon.sh start zkfc，查看日志 [hadoop@nnode ~]$ tail -n50 /usr/local/hadoop2.6.0/logs/hadoop-hadoop-zkfc-nnode.log 2015-08-04 23:25:43,137 INFO org.apache.hadoop.hdfs.tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at nnode/192.168.137.117:53310 2015-08-04 23:25:44,304 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2015-08-04 23:25:44,304 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=nnode 2015-08-04 23:25:44,304 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.7.0_75 2015-08-04 23:25:44,304 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2015-08-04 23:25:44,304 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/local/jdk1.7/jre 2015-08-04 23:25:44,304 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=这里为引入所有的jar包，具体引入的jar包略过 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/local/hadoop2.6.0/lib/native 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-431.el6.x86_64 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=hadoop 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop 2015-08-04 23:25:44,305 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/usr/local/hadoop2.6.0 2015-08-04 23:25:44,312 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=nnode:2181,dnode1:2181,dnode2:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@63a4d48c 2015-08-04 23:25:44,829 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server nnode/192.168.137.117:2181. Will not attempt to authenticate using SASL (unknown error) 2015-08-04 23:25:44,844 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to nnode/192.168.137.117:2181, initiating session 2015-08-04 23:25:44,858 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server nnode/192.168.137.117:2181, sessionid = 0x14ef907cf510003, negotiated timeout = 5000 2015-08-04 23:25:44,872 INFO org.apache.hadoop.ha.ActiveStandbyElector: Session connected. 2015-08-04 23:25:44,899 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-08-04 23:25:44,941 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8019 2015-08-04 23:25:45,012 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2015-08-04 23:25:45,014 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8019: starting 2015-08-04 23:25:45,231 INFO org.apache.hadoop.ha.HealthMonitor: Entering state SERVICE_HEALTHY 2015-08-04 23:25:45,234 INFO org.apache.hadoop.ha.ZKFailoverController: Local service NameNode at nnode/192.168.137.117:53310 entered state: SERVICE_HEALTHY 2015-08-04 23:25:45,281 INFO org.apache.hadoop.ha.ActiveStandbyElector: Checking for any old active which needs to be fenced... 2015-08-04 23:25:45,298 INFO org.apache.hadoop.ha.ActiveStandbyElector: Old node exists: 0a07636c757374657212036e6e311a056e6e6f646520bea00328d33e 2015-08-04 23:25:45,299 INFO org.apache.hadoop.ha.ActiveStandbyElector: But old node has our own data, so don‘t need to fence it. 2015-08-04 23:25:45,299 INFO org.apache.hadoop.ha.ActiveStandbyElector: Writing znode /hadoop-ha/cluster/ActiveBreadCrumb to indicate that the local node is the most recent active... 2015-08-04 23:25:45,308 INFO org.apache.hadoop.ha.ZKFailoverController: Trying to make NameNode at nnode/192.168.137.117:53310 active... 2015-08-04 23:25:47,881 INFO org.apache.hadoop.ha.ZKFailoverController: Successfully transitioned NameNode at nnode/192.168.137.117:53310 to active state [hadoop@dnode1 ~]$ tail -n50 /usr/local/hadoop2.6.0/logs/hadoop-hadoop-zkfc-dnode1.log 2015-08-04 23:25:54,379 INFO org.apache.hadoop.hdfs.tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at dnode1/192.168.137.118:53310 2015-08-04 23:25:55,035 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2015-08-04 23:25:55,035 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=dnode1 2015-08-04 23:25:55,035 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.7.0_75 2015-08-04 23:25:55,035 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2015-08-04 23:25:55,035 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/local/jdk1.7/jre 2015-08-04 23:25:55,035 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=这里为引入所有的jar包，具体引入的jar包略过 2015-08-04 23:25:55,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/local/hadoop2.6.0/lib/native 2015-08-04 23:25:55,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2015-08-04 23:25:55,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2015-08-04 23:25:55,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux 2015-08-04 23:25:55,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64 2015-08-04 23:25:55,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-431.el6.x86_64 2015-08-04 23:25:55,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=hadoop 2015-08-04 23:25:55,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop 2015-08-04 23:25:55,041 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/usr/local/hadoop2.6.0 2015-08-04 23:25:55,041 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=nnode:2181,dnode1:2181,dnode2:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@7b2c4c4b 2015-08-04 23:25:55,099 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server nnode/192.168.137.117:2181. Will not attempt to authenticate using SASL (unknown error) 2015-08-04 23:25:55,113 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to nnode/192.168.137.117:2181, initiating session 2015-08-04 23:25:55,132 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server nnode/192.168.137.117:2181, sessionid = 0x14ef907cf510004, negotiated timeout = 5000 2015-08-04 23:25:55,140 INFO org.apache.hadoop.ha.ActiveStandbyElector: Session connected. 2015-08-04 23:25:55,165 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-08-04 23:25:55,201 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8019 2015-08-04 23:25:55,267 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2015-08-04 23:25:55,269 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8019: starting 2015-08-04 23:25:55,481 INFO org.apache.hadoop.ha.HealthMonitor: Entering state SERVICE_HEALTHY 2015-08-04 23:25:55,481 INFO org.apache.hadoop.ha.ZKFailoverController: Local service NameNode at dnode1/192.168.137.118:53310 entered state: SERVICE_HEALTHY 2015-08-04 23:25:55,508 INFO org.apache.hadoop.ha.ZKFailoverController: ZK Election indicated that NameNode at dnode1/192.168.137.118:53310 should become standby 2015-08-04 23:25:55,523 INFO org.apache.hadoop.ha.ZKFailoverController: Successfully transitioned NameNode at dnode1/192.168.137.118:53310 to standby state
Java客户端API
开源客户端
ZkClient
Curator
5、ZooKeeper应用场景
6、ZooKeeper实现分析
7、ZooKeeper运维
本文出自 “鲁春利的工作笔记” 博客，请务必保留此出处http://luchunli.blog.51cto.com/2368057/1681841
原文地址：http://luchunli.blog.51cto.com/2368057/1681841
