标签:style class blog c code java
Hadoop Eclipse 开发工具 主要分为
14/05/21 23:06:47 INFO
input.FileInputFormat: Total input paths to process : 2
14/05/21
23:06:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
14/05/21 23:06:47 WARN snappy.LoadSnappy: Snappy
native library not loaded
14/05/21 23:06:48 INFO mapred.JobClient: Running
job: job_201405220635_0009
14/05/21 23:06:49 INFO mapred.JobClient: map 0%
reduce 0%
14/05/21 23:06:59 INFO
mapred.JobClient: map 50% reduce 0%
14/05/21
23:07:00 INFO mapred.JobClient: map 100% reduce 0%
14/05/21
23:07:09 INFO mapred.JobClient: map 100% reduce 33%
14/05/21
23:07:11 INFO mapred.JobClient: map 100% reduce 100%
14/05/21
23:07:13 INFO mapred.JobClient: Job complete:
job_201405220635_0009
14/05/21
23:07:13 INFO mapred.JobClient: Counters: 29
14/05/21
23:07:13 INFO mapred.JobClient: Job Counters
14/05/21
23:07:13 INFO mapred.JobClient: Launched reduce tasks=1
14/05/21 23:07:13 INFO mapred.JobClient:
SLOTS_MILLIS_MAPS=17386
14/05/21
23:07:13 INFO mapred.JobClient: Total time spent by all reduces waiting
after reserving slots (ms)=0
14/05/21 23:07:13 INFO mapred.JobClient:
Total time spent by all maps waiting after reserving slots
(ms)=0
14/05/21 23:07:13 INFO
mapred.JobClient: Launched map tasks=2
14/05/21
23:07:13 INFO mapred.JobClient: Data-local map tasks=2
14/05/21 23:07:13 INFO mapred.JobClient:
SLOTS_MILLIS_REDUCES=12160
14/05/21 23:07:13 INFO mapred.JobClient: File
Output Format Counters
14/05/21
23:07:13 INFO mapred.JobClient: Bytes Written=15
14/05/21
23:07:13 INFO mapred.JobClient: FileSystemCounters
14/05/21
23:07:13 INFO mapred.JobClient: FILE_BYTES_READ=52
14/05/21
23:07:13 INFO mapred.JobClient: HDFS_BYTES_READ=252
14/05/21
23:07:13 INFO mapred.JobClient: FILE_BYTES_WRITTEN=177419
14/05/21 23:07:13 INFO mapred.JobClient:
HDFS_BYTES_WRITTEN=15
14/05/21
23:07:13 INFO mapred.JobClient: File Input Format Counters
14/05/21 23:07:13 INFO mapred.JobClient:
Bytes Read=22
14/05/21 23:07:13
INFO mapred.JobClient: Map-Reduce Framework
14/05/21
23:07:13 INFO mapred.JobClient: Map output materialized
bytes=58
14/05/21 23:07:13 INFO
mapred.JobClient: Map input records=2
14/05/21
23:07:13 INFO mapred.JobClient: Reduce shuffle bytes=58
14/05/21 23:07:13 INFO mapred.JobClient:
Spilled Records=8
14/05/21
23:07:13 INFO mapred.JobClient: Map output bytes=38
14/05/21
23:07:13 INFO mapred.JobClient: CPU time spent (ms)=5610
14/05/21 23:07:13 INFO mapred.JobClient:
Total committed heap usage (bytes)=336338944
14/05/21
23:07:13 INFO mapred.JobClient: Combine input records=4
14/05/21 23:07:13 INFO mapred.JobClient:
SPLIT_RAW_BYTES=230
14/05/21
23:07:13 INFO mapred.JobClient: Reduce input records=4
14/05/21 23:07:13 INFO mapred.JobClient:
Reduce input groups=2
14/05/21
23:07:13 INFO mapred.JobClient: Combine output records=4
14/05/21 23:07:13 INFO mapred.JobClient:
Physical memory (bytes) snapshot=428146688
14/05/21
23:07:13 INFO mapred.JobClient: Reduce output records=2
14/05/21 23:07:13 INFO mapred.JobClient:
Virtual memory (bytes) snapshot=2140233728
14/05/21
23:07:13 INFO mapred.JobClient: Map output records=4
错误解决方法:
14/05/21 23:14:11 INFO mapred.JobClient:
Cleaning up the staging area
hdfs://192.168.1.53:9000/app/hadoop/hadoop/tmp/mapred/staging/hadoop/.staging/job_201405220635_0010
14/05/21 23:14:11 ERROR
security.UserGroupInformation: PriviledgedActionException as:hadoop
cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
newout already exists
Exception
in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output
directory newout already exists
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
at
org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:973)
at
org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at
java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at
org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
at
org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
at
org.apache.hadoop.examples.WordCount.main(WordCount.java:93)
cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory newout already exists
说明目录已经存在了,需要删除后再执行,命令如下:
hadoop fs -rmr newout
2.1 Hadoop Eclipse Plugin 配置及安装,布布扣,bubuko.com
2.1 Hadoop Eclipse Plugin 配置及安装
标签:style class blog c code java
原文地址:http://www.cnblogs.com/scwanglijun/p/3742617.html