标签：des style blog io ar color os sp for
本文基于HBase-0.94.1分析HMaster的主要类成员.
HMaster是HBase主/从集群架构中的中央节点。通常一个HBase集群存在多个HMaster节点,其中一个为Active Master,其余为Backup Master.
HMaster的主要类成员如下:
这些类都继承自ZookeeperListener.
/******************************ZooKeeperListener and ZooKeeperWatcher*********************************************/ // Our zk client. private ZooKeeperWatcher zooKeeper; // Manager and zk listener for master election//继承ZooKeeperListener private ActiveMasterManager activeMasterManager; // Region server tracker // 继承ZooKeeperListener private RegionServerTracker regionServerTracker; // Draining region server tracker//继承ZooKeeperListener private DrainingServerTracker drainingServerTracker; /** manager of assignment nodes in zookeeper*/ AssignmentManager assignmentManager;
ZooKeeperWatcher是HBase里唯一一个实现Watcher接口的类，其他需要感知ZooKeeper的内部类都要通过ZooKeeperWatcher.registerListener(ZooKeeperListener listener)方法向ZooKeeperWatcher实例注册。
zooKeeper是一个ZooKeeperWatcher对象，它在HMaster构造函数中初始化; 它是一个单一的ZooKeeper Watcher ，每个HMaster，RegionServer,Client都会实例化一个ZooKeeperWatcher。
ZooKeeperListener是HBase用来侦听Zookeeper事件的基类。
(1)一个进程的ZookeeperWatcher将执行ZookeeperListener父类的某个方法。为了从watcher收到事件，每个listener必须通过ZookeeperWatcher.registerListener注册自己.
(2)ZooKeeperListener的子类需要重写自己感兴趣的方法 (注意,wather在调用listeners里的方法时将会被阻塞，所以listener里的方法不要long-running)
ActiveMasterManager
该类用来处理master端所有与master选举相关的事情.
(1).侦听并响应关于master znode的zk通知(ZK Notifications),包括nodeCreated和nodeDeleted.
(2).包括一个阻塞方法:持有backup masters，等待active master挂掉.
(3).这个class在HMaster里被初始化,HMaster调用blockUntilBecomingActiveMaster()以阻塞等待成为active master.
RegionServerTracker
该类用来通过ZK跟踪所有OnLine状态的RegionServer
DrainingServerTracker
该类用来通过ZK跟踪所有处于上线/下线流水状态的RegionServer.
AssignmentManager
assignmentManager 负责管理region的分配工作.
HMaster通过HBase RPC机制，将自己封装成一个RPC Server,对外提供RPC调用服务.
// RPC server for the HMaster private final RpcServer rpcServer; /** * This servers address. */ private final InetSocketAddress isa;
MasterFileSystem 抽象了HMaster与低层文件系统交互所需的一系列接口
它在HMaster.finishInitialization()方法中被初始化
// file system manager for the master FS operations//抽象了HMaster与底层文件系统交互所需的一系列操作 private MasterFileSystem fileSystemManager
private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
//将是否为active master标识置true
isActiveMaster = true;
/*我们已经是active master了，开始初始化相关组件.
...
this.masterActiveTime = System.currentTimeMillis();
// TODO: Do this using Dependency Injection, using PicoContainer, Guice or Spring.
// 1.创建HBase文件系统 MasterFileSystem.(抽象了HMaster与低层文件系统交互所需的一系列接口)
this.fileSystemManager = new MasterFileSystem(this, this, metrics, masterRecovery);
...
}ServerManager
ServerManager管理所有的regionServer信息。实际上，RegionServerTracker,DrainingServerTracker,AssignmentManager 初始化时需要serverManager作为构造函数的参数.
/******************************ServerManager****************************************************/ /** server manager to deal with region server info*/ private ServerManager serverManager;RegionServerTracker,DrainingServerTracker,AssignmentManager在initializeZKBasedSystemTracker()时进行初始化.
private void initializeZKBasedSystemTrackers() throws IOException,
InterruptedException, KeeperException {
this.catalogTracker = new CatalogTracker(this.zooKeeper, this.conf, this);
this.catalogTracker.start();
this.balancer = LoadBalancerFactory.getLoadBalancer(conf);
this.assignmentManager = new AssignmentManager(this, serverManager,
this.catalogTracker, this.balancer, this.executorService);
zooKeeper.registerListenerFirst(assignmentManager);
this.regionServerTracker = new RegionServerTracker(zooKeeper, this,
this.serverManager);
this.regionServerTracker.start();
this.drainingServerTracker = new DrainingServerTracker(zooKeeper, this,
this.serverManager);
this.drainingServerTracker.start();
catalogTracker 跟踪"目录表"(-ROOT-和.META.表)的可用性
ClusterStatusTracker
clusterStatusTracker跟踪集群在zookeeper上的配置(Tracker on cluster settings up in zookeeper). ClusterStatusTracker和ClusterStatus不同，后者只是一个存储cluster当前视图的镜像的数据结构，而ClusterStatusTracker是用于跟踪集群在zookeeper上配置的属性信息.
// manager of catalog regions private CatalogTracker catalogTracker; // Cluster status zk tracker and local setter private ClusterStatusTracker clusterStatusTracker;
ExecutorService是一个通用的执行器服务类,这个组件抽象了一个threadPool，一个队列，和一个Runnable线程(Handler角色)
/******************************ExecutorService****************************************************/ // Instance of the hbase executor service. ExecutorService executorService;该变量在finishInitialization方法中初始化
private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
......
if (!masterRecovery) {
//初始化ExecutorService和serverManager
this.executorService = new ExecutorService(getServerName().toString());//维护一个threadPool和队列
this.serverManager = new ServerManager(this, this);//管理所有的regionserver
}
LoadBalancer的职责是维护regionServers之间的负载均衡; balancerChore线程做一些Balancer相关的清理工作.
用户可以通过实现LoadBalancer接口来定制自己的负载均衡策略. 默认情况下，HBase采用的是org.apache.hadoop.hbase.master.DefaultLoadBalancer类做负载均衡.
/******************************LoadBalancer, BalancerChore****************************************/ private LoadBalancer balancer; private Thread balancerChore;loadBalancer是在initializeZKBasedSystemTrackers()方法中调用LoadBalancerFactory.getLoadBalancer(conf)初始化的.
private void initializeZKBasedSystemTrackers() throws IOException,
InterruptedException, KeeperException {
this.balancer = LoadBalancerFactory.getLoadBalancer(conf);
...
}public class LoadBalancerFactory {
/**
* Create a loadblanacer from the given conf.
* @param conf
* @return A {@link LoadBalancer}
*/
public static LoadBalancer getLoadBalancer(Configuration conf) {
// Create the balancer
Class<? extends LoadBalancer> balancerKlass = conf.getClass(
HConstants.HBASE_MASTER_LOADBALANCER_CLASS,
DefaultLoadBalancer.class, LoadBalancer.class);
return ReflectionUtils.newInstance(balancerKlass, conf);
}
}TableDescriptor 接口描述了用来管理Table描述符的一系列操作,FSTableDescriptor是该接口的实现类。
/******************************TableDescriptors***************************************************/ private TableDescriptors tableDescriptors;
在HMaster.finishInitialization方法中初始化
private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
....
// 2.初始化tableDescriptors，从文件系统读取HTable的描述信息.
this.tableDescriptors =
new FSTableDescriptors(this.fileSystemManager.getFileSystem(),
this.fileSystemManager.getRootDir());
FTableDescriptor的构造函数如下: FSTableDescriptors根据fs和rootdir可以读取、更改所有的table描述符
public FSTableDescriptors(final FileSystem fs, final Path rootdir,
final boolean fsreadOnly) {
super();
this.fs = fs;
this.rootdir = rootdir;
this.fsreadonly = fsreadOnly;
}
以FSTableDescriptor。FSTableDescriptor获取rootdir下面的Path(这些Path经过FileUtils.getTableDirs(fs,rootdir)获得，经过了过滤) . 将Path的Name(也即文件名)作为tableName.
@Override
public Map<String, HTableDescriptor> getAll()
throws IOException {
Map<String, HTableDescriptor> htds = new TreeMap<String, HTableDescriptor>();
//获取所有的tableName
List<Path> tableDirs = FSUtils.getTableDirs(fs, rootdir);
for (Path d: tableDirs) {
HTableDescriptor htd = null;
try {
htd = get(d.getName());
} catch (FileNotFoundException fnfe) {
} if (htd == null)
continue;
htds.put(d.getName(), htd);
} return htds;
}
SnapshotManager管理snapshots的生成和装载(taking and restoring)过程.
// monitor for snapshot of hbase tables private SnapshotManager snapshotManager; //HBase tables的镜像管理者
/******************************protocolHandlers 管理coprocessor protocol的注册*********************/
// Registered master protocol handlers
private ClassToInstanceMap<CoprocessorProtocol>
protocolHandlers = MutableClassToInstanceMap.create();
/** The health check chore. */ private HealthCheckChore healthCheckChore;//健康检查 /******************************垃圾回收:CatalogJanitor,LogCleaner,HFileCleaner*********************/ private CatalogJanitor catalogJanitorChore; //定时扫描-META-，对无用的region进行垃圾回收 private LogCleaner logCleaner; private HFileCleaner hfileCleaner;
//todo
标签：des style blog io ar color os sp for
原文地址：http://blog.csdn.net/mango_song/article/details/41803353
