标签：
系统： Ubuntu14.04
Hadoop版本： 2.7.2
参照http://www.cnblogs.com/taichu/p/5264185.html中的分享，来学习运行第一个hadoop程序。
在hadoop的安装文件夹 /usr/local/hadoop下创建input文件夹
hadoop@hadoopmaster:/usr/local/hadoop$ mkdir ./input
然后copy几个文档到input文件夹中作为WordCount的输入
hadoop@hadoopmaster:/usr/local/hadoop$ cp *.txt ./input
执行WordCount
hadoop@hadoopmaster:/usr/local/hadoop$ hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.2-sources.jar org.apache.hadoop.examples.WordCount input/ output
运行时报错
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://hadoopmaster:9000/user/hadoop/input
按照http://rangerwolf.iteye.com/blog/1900615中提供的解决方案把usr/local/hadoop/input文件夹put到hdfs中
hadoop@hadoopmaster:/usr/local/hadoop$ hdfs dfs -put ./input/ input (这里在input前加上 / 就不会报错了，后同)
报错：put: `input‘: No such file or directory
直接mkdir也会出现同样的问题
hadoop@hadoopmaster:~$ hdfs dfs -mkdir input
mkdir: `input‘: No such file or directory
加上 / 成功put了后再执行WordCount，仍然报错，Input path does not exist: hdfs://hadoopmaster:9000/user/hadoop/input
这里的input文件夹路径是/user/hadoop/input，但我的路径是/usr/local/input，是不是这个原因导致找不到路径呢
参考http://stackoverflow.com/questions/20821584/hadoop-2-2-installation-no-such-file-or-directory中的回答，在hdfs中创建/user/hadoop/
hadoop@hadoopmaster:hdfs dfs -mkdir -p /user/hadoop
将input文件夹添加到该文件夹下（这一步是在eclipse中进行的）
现在我的hdfs的文件结构是这样的
hadoop@hadoopmaster:/usr/local/hadoop/etc/hadoop$ hdfs dfs -ls -R /
drwxr-xr-x - hadoop supergroup 0 2016-03-16 21:39 /input
-rw-r--r-- 2 hadoop supergroup 15429 2016-03-16 21:39 /input/LICENSE.txt
-rw-r--r-- 2 hadoop supergroup 101 2016-03-16 21:39 /input/NOTICE.txt
-rw-r--r-- 2 hadoop supergroup 1366 2016-03-16 21:39 /input/README.txt
drwx------ - hadoop supergroup 0 2016-03-16 21:17 /tmp
drwx------ - hadoop supergroup 0 2016-03-16 21:17 /tmp/hadoop-yarn
drwx------ - hadoop supergroup 0 2016-03-16 21:17 /tmp/hadoop-yarn/staging
drwx------ - hadoop supergroup 0 2016-03-16 21:17 /tmp/hadoop-yarn/staging/hadoop
drwx------ - hadoop supergroup 0 2016-03-16 21:41 /tmp/hadoop-yarn/staging/hadoop/.staging
drwxr-xr-x - hadoop supergroup 0 2016-03-16 21:51 /user
drwxr-xr-x - hadoop supergroup 0 2016-03-16 22:02 /user/hadoop
drwxr-xr-x - hadoop supergroup 0 2016-03-16 21:57 /user/hadoop/input
-rw-r--r-- 3 hadoop supergroup 15429 2016-03-16 21:57 /user/hadoop/input/LICENSE.txt
-rw-r--r-- 3 hadoop supergroup 101 2016-03-16 21:57 /user/hadoop/input/NOTICE.txt
-rw-r--r-- 3 hadoop supergroup 1366 2016-03-16 21:57 /user/hadoop/input/README.txt
执行
hadoop@hadoopmaster:/usr/local/hadoop/etc/hadoop$ hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.2-sources.jar org.apache.hadoop.examples.WordCount input/ output
Not a valid JAR: /usr/local/hadoop/etc/hadoop/share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.2-sources.jar
换成hdfs jar则会
hadoop@hadoopmaster:/usr/local/hadoop/etc/hadoop$ hdfs jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.2-sources.jar org.apache.hadoop.examples.WordCount input/ output
Error: Could not find or load main class jar
尚不清楚原因，改天来解决。有高手知道为什么会出这样的问题的请不吝赐教。
在eclipse中执行WordCount，运行成功！下面是部分输出结果。
16/03/16 22:02:46 INFO mapreduce.Job: map 100% reduce 100%
16/03/16 22:02:46 INFO mapreduce.Job: Job job_local1837130715_0001 completed successfully
16/03/16 22:02:46 INFO mapreduce.Job: Counters: 35
File System Counters
FILE: Number of bytes read=29752
FILE: Number of bytes written=1200391
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=66016
HDFS: Number of bytes written=8983
HDFS: Number of read operations=33
HDFS: Number of large read operations=0
HDFS: Number of write operations=6
Map-Reduce Framework
Map input records=322
Map output records=2347
Map output bytes=24935
Map output materialized bytes=13001
Input split bytes=355
Combine input records=2347
Combine output records=897
Reduce input groups=840
Reduce shuffle bytes=13001
Reduce input records=897
Reduce output records=840
Spilled Records=1794
Shuffled Maps =3
Failed Shuffles=0
Merged Map outputs=3
GC time elapsed (ms)=17
Total committed heap usage (bytes)=1444937728
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=16896
File Output Format Counters
Bytes Written=8983
嗯，目前还不是很看得懂这个结果，明天开始看书了，基础还是最重要的。
备忘：
hadoop命令：
列出hdfs文件系统根目录下的目录和文件
hdfs dfs -ls / (有 / )
hdfs dfs -ls (没有有 / )
这两个的结果是不一样的，不理解为什么，这就是为什么明天要看书的原因
Error: Could not find or load main class jar
hadoop@hadoopmaster:/usr/local/hadoop/etc/hadoop$ hdfs dfs -ls（在hdfs dfs -mkdir -p /user/[current login user]）
Found 2 items
drwxr-xr-x - hadoop supergroup 0 2016-03-16 21:57 input
drwxr-xr-x - hadoop supergroup 0 2016-03-16 22:02 output
hadoop@hadoopmaster:/usr/local/hadoop/etc/hadoop$ hdfs dfs -ls /
Found 3 items
drwxr-xr-x - hadoop supergroup 0 2016-03-16 21:39 /input
drwx------ - hadoop supergroup 0 2016-03-16 21:17 /tmp
drwxr-xr-x - hadoop supergroup 0 2016-03-16 21:51 /user
列出hdfs文件系统所有的目录和文件
hdfs dfs -ls -R /（有 / 和没有 / 的结果也是不一样的，对应上面-ls的文件夹）
标签：
原文地址：http://www.cnblogs.com/loganlee/p/5285521.html
