标签：hadoop   源码   文件系统   hdfs   

前言

在linux文件系统中，i-node节点一直是一个非常重要的设计，同样在HDFS中，也存在这样的一个类似的角色，不过他是一个全新的类，INode.class，后面的目录类等等都是他的子类。最近学习了部分HDFS的源码结构，就好好理一理这方面的知识，帮助大家更好的从深层次了解Hadoop分布式系统文件。

HDFS文件相关的类设计

在HDFS中与文件相关的类主要有这么几个

1.INode--这个就是最底层的一个类，抽象类，提炼一些文件目录共有的属性。

2.INodeFile--文件节点类，继承自INode，表示一个文件

3.INodeDirectory--文件目录类，也是继承自INode.他的孩子中是文件集也可能是目录

4.INodeDirectoryWithQuota--有配额限制的目录，这是为了适应HDFS中的配额策略。

5.INodeFileUnderConstruction--处于构建状态的文件类，可以从INodeFile中转化而来。

我大体上分为了以上这么几个类，后续将从上述类中挑出部分代码，来了解作者的巧妙的设计思想。

INode

INode基础抽象类，保存了文件，目录都可能会共有的基本属性，如下

/**
 * We keep an in-memory representation of the file/block hierarchy.
 * This is a base INode class containing common fields for file and 
 * directory inodes.
 */
abstract class INode implements Comparable<byte[]> {
  //文件/目录名称
  protected byte[] name;
  //父目录
  protected INodeDirectory parent;
  //最近一次的修改时间
  protected long modificationTime;
  //最近访问时间
  protected long accessTime;

//Only updated by updatePermissionStatus(...).
  //Other codes should not modify it.
  private long permission;

  //使用long整数的64位保存，分3段保存，分别为mode模式控制访问权限，所属组，所属用户
  private static enum PermissionStatusFormat {
    MODE(0, 16),
    GROUP(MODE.OFFSET + MODE.LENGTH, 25),
    USER(GROUP.OFFSET + GROUP.LENGTH, 23);

    final int OFFSET;
    final int LENGTH; //bit length
    final long MASK;

    PermissionStatusFormat(int offset, int length) {
      OFFSET = offset;
      LENGTH = length;
      MASK = ((-1L) >>> (64 - LENGTH)) << OFFSET;
    }

    //与掩码计算并右移得到用户标识符
    long retrieve(long record) {
      return (record & MASK) >>> OFFSET;
    }

    long combine(long bits, long record) {
      return (record & ~MASK) | (bits << OFFSET);
    }
  }

要获取这些值，需要与内部的掩码值计算并作左右移操作。在这里存储标识符的好处是比纯粹的字符串省了很多的内存，那么HDFS是如何通过标识符数字获取用户组或用户名的呢，答案在下面

/** Get user name */
  public String getUserName() {
    int n = (int)PermissionStatusFormat.USER.retrieve(permission);
    //根据整形标识符，SerialNumberManager对象中取出，避免存储字符串消耗大量内存
    return SerialNumberManager.INSTANCE.getUser(n);
  }
  /** Set user */
  protected void setUser(String user) {
    int n = SerialNumberManager.INSTANCE.getUserSerialNumber(user);
    updatePermissionStatus(PermissionStatusFormat.USER, n);
  }
  /** Get group name */
  public String getGroupName() {
    int n = (int)PermissionStatusFormat.GROUP.retrieve(permission);
    return SerialNumberManager.INSTANCE.getGroup(n);
  }
  /** Set group */
  protected void setGroup(String group) {
    int n = SerialNumberManager.INSTANCE.getGroupSerialNumber(group);
    updatePermissionStatus(PermissionStatusFormat.GROUP, n);
  }

/**
   * Check whether this is the root inode.
   * 根节点的判断标准是名字长度为0
   */
  boolean isRoot() {
    return name.length == 0;
  }

/** Simple wrapper for two counters : 
   *  nsCount (namespace consumed) and dsCount (diskspace consumed).
   */
  static class DirCounts {
    long nsCount = 0;
    long dsCount = 0;
    
    /** returns namespace count */
    long getNsCount() {
      return nsCount;
    }
    /** returns diskspace count */
    long getDsCount() {
      return dsCount;
    }
  }

//移除自身节点方法
  boolean removeNode() {
    if (parent == null) {
      return false;
    } else {
      
      parent.removeChild(this);
      parent = null;
      return true;
    }
  }

.

INodeDirectory

下面来分析分析目录节点类，说到目录，当然一定会有的就是孩子节点了，下面是目录类的成员变量：

/**
 * Directory INode class.
 */
class INodeDirectory extends INode {
  protected static final int DEFAULT_FILES_PER_DIRECTORY = 5;
  final static String ROOT_NAME = "";

  //保存子目录或子文件
  private List<INode> children;

//移除节点方法
  INode removeChild(INode node) {
    assert children != null;
    //用二分法寻找文件节点
    int low = Collections.binarySearch(children, node.name);
    if (low >= 0) {
      return children.remove(low);
    } else {
      return null;
    }
  }

/**
   * Add a child inode to the directory.
   * 
   * @param node INode to insert
   * @param inheritPermission inherit permission from parent?
   * @return  null if the child with this name already exists; 
   *          node, otherwise
   */
  <T extends INode> T addChild(final T node, boolean inheritPermission) {
    if (inheritPermission) {
      FsPermission p = getFsPermission();
      //make sure the  permission has wx for the user
      //判断用户是否有写权限
      if (!p.getUserAction().implies(FsAction.WRITE_EXECUTE)) {
        p = new FsPermission(p.getUserAction().or(FsAction.WRITE_EXECUTE),
            p.getGroupAction(), p.getOtherAction());
      }
      node.setPermission(p);
    }

    if (children == null) {
      children = new ArrayList<INode>(DEFAULT_FILES_PER_DIRECTORY);
    }
    //二分查找
    int low = Collections.binarySearch(children, node.name);
    if(low >= 0)
      return null;
    node.parent = this;
    //在孩子列表中进行添加
    children.add(-low - 1, node);
    // update modification time of the parent directory
    setModificationTime(node.getModificationTime());
    if (node.getGroupName() == null) {
      node.setGroup(getGroupName());
    }
    return node;
  }

//递归删除文件目录下的所有block块
  int collectSubtreeBlocksAndClear(List<Block> v) {
    int total = 1;
    //直到是空目录的情况，才直接返回
    if (children == null) {
      return total;
    }
    for (INode child : children) {
      //递归删除
      total += child.collectSubtreeBlocksAndClear(v);
    }
    
    //删除完毕之后，置为空操作，并返回文件数计数结果
    parent = null;
    children = null;
    return total;
  }

/** {@inheritDoc} */
  DirCounts spaceConsumedInTree(DirCounts counts) {
    counts.nsCount += 1;
    if (children != null) {
      for (INode child : children) {
        child.spaceConsumedInTree(counts);
      }
    }
    return counts;    
  }

INodeDirectoryWithQuota

与上个类相比就多了Quota这个单词，Quota在英文中的意思就是”配额“，有容量限制，在这里的容量限制有2个维度，第一个命名空间限制，磁盘空间占用限制，前者避免你创建过多的目录文件，后者避免你占用过大的空间。变量定义如下，他是继承自目录类的

/**
 * Directory INode class that has a quota restriction
 * 存在配额限制的目录节点,继承自目录节点
 */
class INodeDirectoryWithQuota extends INodeDirectory {
  //命名空间配额
  private long nsQuota; /// NameSpace quota
  //名字空间计数
  private long nsCount;
  //磁盘空间配额
  private long dsQuota; /// disk space quota
  //磁盘空间占用大小
  private long diskspace;

一般此类都可以通过非配额类以参数的形式构造而来，如下

/** Convert an existing directory inode to one with the given quota
   *  给定目录,通过传入配额限制将之转为配额目录
   * @param nsQuota Namespace quota to be assigned to this inode
   * @param dsQuota Diskspace quota to be assigned to this indoe
   * @param other The other inode from which all other properties are copied
   */
  INodeDirectoryWithQuota(long nsQuota, long dsQuota, INodeDirectory other)
  throws QuotaExceededException {
    super(other);
    INode.DirCounts counts = new INode.DirCounts();
    other.spaceConsumedInTree(counts);
    this.nsCount= counts.getNsCount();
    this.diskspace = counts.getDsCount();
    setQuota(nsQuota, dsQuota);
  }

/** Verify if the namespace count disk space satisfies the quota restriction
   * 给定一定的误差限制,验证命名空间计数和磁盘空间是否使用超出相应的配额限制,超出则抛异常
   * @throws QuotaExceededException if the given quota is less than the count
   */
  void verifyQuota(long nsDelta, long dsDelta) throws QuotaExceededException {
    //根据误差值计算新的计数值
    long newCount = nsCount + nsDelta;
    long newDiskspace = diskspace + dsDelta;
    if (nsDelta>0 || dsDelta>0) {
      //判断新的值是否超出配额的值大小
      if (nsQuota >= 0 && nsQuota < newCount) {
        throw new NSQuotaExceededException(nsQuota, newCount);
      }
      if (dsQuota >= 0 && dsQuota < newDiskspace) {
        throw new DSQuotaExceededException(dsQuota, newDiskspace);
      }
    }
  }

INodeFile

与目录相对应的类就是文件类，在HDFS中文件对应的就是许多个block块嘛，所以比如会有block列表组，当然他也可能会定义每个block的副本数，HDFS中默认是3,

class INodeFile extends INode {
  static final FsPermission UMASK = FsPermission.createImmutable((short)0111);

  //Number of bits for Block size
  //48位存储block数据块的大小
  static final short BLOCKBITS = 48;

  //Header mask 64-bit representation
  //Format: [16 bits for replication][48 bits for PreferredBlockSize]
  //前16位保存副本系数,后48位保存优先块大小,下面的headermask做计算时用
  static final long HEADERMASK = 0xffffL << BLOCKBITS;

  protected long header;
  //文件数据block块
  protected BlockInfo blocks[] = null;

/**
   * Get block replication for the file 
   * @return block replication value
   * 得到副本系数通过与掩码计算并右移48位
   */
  public short getReplication() {
    return (short) ((header & HEADERMASK) >> BLOCKBITS);
  }

 @Override
  DirCounts spaceConsumedInTree(DirCounts counts) {
    //命名空间消耗加1
    counts.nsCount += 1;
    //累加磁盘空间消耗大小
    counts.dsCount += diskspaceConsumed();
    return counts;
  }

//计算磁盘空间消耗的大小
  long diskspaceConsumed(Block[] blkArr) {
    long size = 0;
    for (Block blk : blkArr) {
      if (blk != null) {
        size += blk.getNumBytes();
      }
    }
    /* If the last block is being written to, use prefferedBlockSize
     * rather than the actual block size.
     * 如果最后一个块正在被写,用内部设置的prefferedBlockSize的值做替换
     */
    if (blkArr.length > 0 && blkArr[blkArr.length-1] != null && 
        isUnderConstruction()) {
      size += getPreferredBlockSize() - blocks[blocks.length-1].getNumBytes();
    }
    
    //每个块乘以相应的副本数
    return size * getReplication();
  }

/**
   * Return the last block in this file, or null if there are no blocks.
   * 获取文件的最后一个block块
   */
  Block getLastBlock() {
    if (this.blocks == null || this.blocks.length == 0)
      return null;
    return this.blocks[this.blocks.length - 1];
  }

/**
   * add a block to the block list
   * 往block列表中添加block块
   */
  void addBlock(BlockInfo newblock) {
    if (this.blocks == null) {
      this.blocks = new BlockInfo[1];
      this.blocks[0] = newblock;
    } else {
      int size = this.blocks.length;
      BlockInfo[] newlist = new BlockInfo[size + 1];
      System.arraycopy(this.blocks, 0, newlist, 0, size);
      newlist[size] = newblock;
      this.blocks = newlist;
    }
  }

INodeFileUnderConstruction

这个类的名字有点长，他的意思是处于构建状态的文件类，是INodeFile的子类，就是当文件被操作的时候，就转变为此类的形式了，与block读写操作的关系比较密切些。变量定义如下

//处于构建状态的文件节点
class INodeFileUnderConstruction extends INodeFile {
  //写文件的客户端名称,也是这个租约的持有者
  String clientName;         // lease holder
  //客户端所在的主机
  private final String clientMachine;
  //如果客户端同样存在于集群中,则记录所在的节点
  private final DatanodeDescriptor clientNode; // if client is a cluster node too.
  
  //租约恢复时的节点
  private int primaryNodeIndex = -1; //the node working on lease recovery
  //最后一个block块所处的节点组,又名数据流管道成员
  private DatanodeDescriptor[] targets = null;   //locations for last block
  //最近租约恢复时间
  private long lastRecoveryTime = 0;

//设置新的block块,并且为最后的块赋值新的targes节点
  synchronized void setLastBlock(BlockInfo newblock, DatanodeDescriptor[] newtargets
      ) throws IOException {
    if (blocks == null || blocks.length == 0) {
      throw new IOException("Trying to update non-existant block (newblock="
          + newblock + ")");
    }
    BlockInfo oldLast = blocks[blocks.length - 1];
    if (oldLast.getBlockId() != newblock.getBlockId()) {
      // This should not happen - this means that we‘re performing recovery
      // on an internal block in the file!
      NameNode.stateChangeLog.error(
        "Trying to commit block synchronization for an internal block on"
        + " inode=" + this
        + " newblock=" + newblock + " oldLast=" + oldLast);
      throw new IOException("Trying to update an internal block of " +
                            "pending file " + this);
    }
    
    //如果新的block时间比老block的还小的话,则进行警告
    if (oldLast.getGenerationStamp() > newblock.getGenerationStamp()) {
      NameNode.stateChangeLog.warn(
        "Updating last block " + oldLast + " of inode " +
        "under construction " + this + " with a block that " +
        "has an older generation stamp: " + newblock);
    }

    blocks[blocks.length - 1] = newblock;
    setTargets(newtargets);
    //重置租约恢复时间,这样操作的话,下次租约检测时将会过期
    lastRecoveryTime = 0;
  }

 /**
   * remove a block from the block list. This block should be
   * the last one on the list.
   * 在文件所拥有的block列表中移动掉block,这个block块应该是最后一个block块
   */
  void removeBlock(Block oldblock) throws IOException {
    if (blocks == null) {
      throw new IOException("Trying to delete non-existant block " + oldblock);
    }
    int size_1 = blocks.length - 1;
    if (!blocks[size_1].equals(oldblock)) {
      //如果不是最末尾一个块则将会抛异常
      throw new IOException("Trying to delete non-last block " + oldblock);
    }

    //copy to a new list
    BlockInfo[] newlist = new BlockInfo[size_1];
    System.arraycopy(blocks, 0, newlist, 0, size_1);
    blocks = newlist;
    
    // Remove the block locations for the last block.
    // 最后一个数据块所对应的节点组就被置为空了
    targets = null;
  }

/**
   * Update lastRecoveryTime if expired.
   * @return true if lastRecoveryTimeis updated. 
   * 设置最近的恢复时间
   */
  synchronized boolean setLastRecoveryTime(long now) {
    boolean expired = now - lastRecoveryTime > NameNode.LEASE_RECOVER_PERIOD;
    if (expired) {
      //如果过期了则设置为传入的当前时间
      lastRecoveryTime = now;
    }
    return expired;
  }

总结

HDFS中的INode文件相关源码分析就是上述所说的了，上面只是部分我的代码分析，全部代码链接如下

https://github.com/linyiqun/hadoop-hdfs,后续将会继续更新HDFS其他方面的代码分析。

参考文献

《Hadoop技术内部–HDFS结构设计与实现原理》.蔡斌等

HDFS源码分析(一)-----INode文件节点

标签：hadoop   源码   文件系统   hdfs   

原文地址：http://blog.csdn.net/androidlushangderen/article/details/47427925