Hadoop 2.7.3 完全分布式维护-动态增加datanode篇

$ cat slaves 
sht-sgmhadoopdn-01
sht-sgmhadoopdn-02
sht-sgmhadoopdn-03
sht-sgmhadoopdn-04

$ hostname 
sht-sgmhadoopnn-01
$ rsync -az --progress /usr/local/hadoop-2.7.3/* hduser@sht-sgmhadoopdn-04:/usr/local/hadoop-2.7.3/

$ hostname 
sht-sgmhadoopdn-04
$ rm -rf /usr/local/hadoop-2.7.3/logs/*
$ rm -rf /usr/local/hadoop-2.7.3/data/*

$ hadoop-daemon.sh start datanode
starting datanode, logging to /usr/local/hadoop-2.7.3/logs/hadoop-hduser-datanode-sht-sgmhadoopdn-04.out
$ jps
31875 Jps
31821 DataNode

$ hdfs dfsadmin -report
Configured Capacity: 303324561408 (282.49 GB)
Present Capacity: 83729309696 (77.98 GB)
DFS Remaining: 83081265152 (77.38 GB)
DFS Used: 648044544 (618.02 MB)
DFS Used%: 0.77%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0

-------------------------------------------------
Live datanodes (4):

Name: 172.16.101.66:50010 (sht-sgmhadoopdn-04)
Hostname: sht-sgmhadoopdn-04
Decommission Status : Normal
Configured Capacity: 75831140352 (70.62 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 35573932032 (33.13 GB)
DFS Remaining: 40257183744 (37.49 GB)
DFS Used%: 0.00%
DFS Remaining%: 53.09%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Sep 01 22:50:16 CST 2017


Name: 172.16.101.60:50010 (sht-sgmhadoopdn-03)
Hostname: sht-sgmhadoopdn-03
Decommission Status : Normal
Configured Capacity: 75831140352 (70.62 GB)
DFS Used: 216006656 (206 MB)
Non DFS Used: 61714608128 (57.48 GB)
DFS Remaining: 13900525568 (12.95 GB)
DFS Used%: 0.28%
DFS Remaining%: 18.33%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Sep 01 22:50:15 CST 2017


Name: 172.16.101.59:50010 (sht-sgmhadoopdn-02)
Hostname: sht-sgmhadoopdn-02
Decommission Status : Normal
Configured Capacity: 75831140352 (70.62 GB)
DFS Used: 216006656 (206 MB)
Non DFS Used: 62057410560 (57.80 GB)
DFS Remaining: 13557723136 (12.63 GB)
DFS Used%: 0.28%
DFS Remaining%: 17.88%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Sep 01 22:50:14 CST 2017


Name: 172.16.101.58:50010 (sht-sgmhadoopdn-01)
Hostname: sht-sgmhadoopdn-01
Decommission Status : Normal
Configured Capacity: 75831140352 (70.62 GB)
DFS Used: 216006656 (206 MB)
Non DFS Used: 60249300992 (56.11 GB)
DFS Remaining: 15365832704 (14.31 GB)
DFS Used%: 0.28%
DFS Remaining%: 20.26%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Fri Sep 01 22:50:15 CST 2017

$ hdfs dfsadmin -setBalancerBandwidth 67108864
Balancer bandwidth is set to 67108864
$ start-balancer.sh -threshold 5
starting balancer, logging to /usr/local/hadoop-2.7.3/logs/hadoop-hduser-balancer-sht-sgmhadoopnn-01.out

$ yarn-daemon.sh start nodemanager
starting nodemanager, logging to /usr/local/hadoop-2.7.3/logs/yarn-hduser-nodemanager-sht-sgmhadoopdn-04.out
$ jps
32562 NodeManager
32599 Jps
31821 DataNode