码迷,mamicode.com
首页 > 其他好文 > 详细

topk记录

时间:2014-05-13 06:53:12      阅读:448      评论:0      收藏:0      [点我收藏+]

标签:java   c   tar   int   get   com   

lk@lk-virtual-machine:~/hadoop-1.0.1/bin$ ./hadoop dfs -rmr output
Deleted hdfs://localhost:9000/user/lk/output
lk@lk-virtual-machine:~/hadoop-1.0.1/bin$ ./hadoop jar ~/mytopk.jar top.Top  input output
****hdfs://localhost:9000/user/lk/input
14/05/12 05:14:03 INFO input.FileInputFormat: Total input paths to process : 4
14/05/12 05:14:18 INFO mapred.JobClient: Running job: job_201405120333_0004
14/05/12 05:14:20 INFO mapred.JobClient:  map 0% reduce 0%
14/05/12 05:17:32 INFO mapred.JobClient:  map 50% reduce 0%
14/05/12 05:17:36 INFO mapred.JobClient: Task Id : attempt_201405120333_0004_m_000001_0, Status : FAILED
java.lang.RuntimeException: Error while running command to get file permissions : java.io.IOException: Cannot run program "/bin/ls": java.io.IOException: error=12, Cannot allocate memory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:488)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:200)
    at org.apache.hadoop.util.Shell.run(Shell.java:182)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
    at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:703)
    at org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:443)
    at org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.getOwner(RawLocalFileSystem.java:426)
    at org.apache.hadoop.mapred.TaskLog.obtainLogDirOwner(TaskLog.java:251)
    at org.apache.hadoop.mapred.TaskLogsTruncater.truncateLogs(TaskLogsTruncater.java:124)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:260)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.io.IOException: java.io.IOException: error=12, Cannot allocate memory
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:164)
    at java.lang.ProcessImpl.start(ProcessImpl.java:81)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:470)
    ... 15 more

    at org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:468)
    at org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.getOwner(RawLocalFileSystem.java:426)
    at org.apache.hadoop.mapred.TaskLog.obtainLogDirOwner(TaskLog.java:251)
    at org.apache.hadoop.mapred.TaskLogsTruncater.truncateLogs(TaskLogsTruncater.java:124)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:260)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

14/05/12 05:17:58 INFO mapred.JobClient:  map 0% reduce 0%
14/05/12 05:18:10 INFO mapred.JobClient: Task Id : attempt_201405120333_0004_m_000000_0, Status : FAILED
java.lang.RuntimeException: Error while running command to get file permissions : java.io.IOException: Cannot run program "/bin/ls": java.io.IOException: error=12, Cannot allocate memory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:488)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:200)
    at org.apache.hadoop.util.Shell.run(Shell.java:182)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
    at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:703)
    at org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:443)
    at org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.getOwner(RawLocalFileSystem.java:426)
    at org.apache.hadoop.mapred.TaskLog.obtainLogDirOwner(TaskLog.java:251)
    at org.apache.hadoop.mapred.TaskLogsTruncater.truncateLogs(TaskLogsTruncater.java:124)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:260)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.io.IOException: java.io.IOException: error=12, Cannot allocate memory
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:164)
    at java.lang.ProcessImpl.start(ProcessImpl.java:81)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:470)
    ... 15 more

    at org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:468)
    at org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.getOwner(RawLocalFileSystem.java:426)
    at org.apache.hadoop.mapred.TaskLog.obtainLogDirOwner(TaskLog.java:251)
    at org.apache.hadoop.mapred.TaskLogsTruncater.truncateLogs(TaskLogsTruncater.java:124)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:260)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

14/05/12 05:18:14 INFO mapred.JobClient: Task Id : attempt_201405120333_0004_m_000001_1, Status : FAILED
14/05/12 05:20:01 INFO mapred.JobClient:  map 25% reduce 0%
14/05/12 05:20:27 INFO mapred.JobClient: Task Id : attempt_201405120333_0004_m_000000_1, Status : FAILED
attempt_201405120333_0004_m_000000_1: log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapred.Task).
attempt_201405120333_0004_m_000000_1: log4j:WARN Please initialize the log4j system properly.
14/05/12 05:20:41 INFO mapred.JobClient:  map 0% reduce 0%
14/05/12 05:20:43 INFO mapred.JobClient: Task Id : attempt_201405120333_0004_m_000001_2, Status : FAILED
java.lang.NumberFormatException: For input string: ""
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:493)
    at java.lang.Integer.parseInt(Integer.java:514)
    at top.Top$TopKMapper.map(Top.java:28)
    at top.Top$TopKMapper.map(Top.java:1)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

14/05/12 05:23:55 INFO mapred.JobClient:  map 25% reduce 0%
14/05/12 05:24:06 INFO mapred.JobClient:  map 50% reduce 0%
14/05/12 05:24:19 INFO mapred.JobClient: Task Id : attempt_201405120333_0004_m_000000_2, Status : FAILED
attempt_201405120333_0004_m_000000_2: log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapred.Task).
attempt_201405120333_0004_m_000000_2: log4j:WARN Please initialize the log4j system properly.
14/05/12 05:24:33 INFO mapred.JobClient:  map 0% reduce 0%
14/05/12 05:24:58 INFO mapred.JobClient: Job complete: job_201405120333_0004
14/05/12 05:25:01 INFO mapred.JobClient: Counters: 7
14/05/12 05:25:01 INFO mapred.JobClient:   Job Counters
14/05/12 05:25:01 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=979673
14/05/12 05:25:01 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/05/12 05:25:01 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/05/12 05:25:01 INFO mapred.JobClient:     Launched map tasks=7
14/05/12 05:25:01 INFO mapred.JobClient:     Data-local map tasks=7
14/05/12 05:25:01 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
14/05/12 05:25:01 INFO mapred.JobClient:     Failed map tasks=1
lk@lk-virtual-machine:~/hadoop-1.0.1/bin$


topk记录,布布扣,bubuko.com

topk记录

标签:java   c   tar   int   get   com   

原文地址:http://blog.csdn.net/huaweitman/article/details/25562103

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!