码迷,mamicode.com
首页 > 其他好文 > 详细

spark运行任务报错:Container [...] is running beyond physical memory limits. Current usage: 3.0 GB of 3 GB physical memory used; 5.0 GB of 6.3 GB virtual memory used. Killing container.

时间:2019-07-10 23:08:14      阅读:187      评论:0      收藏:0      [点我收藏+]

标签:gcd   rac   问题   rop   ast   stderr   stdout   mamicode   tmp   

spark版本:1.6.0

scala版本:2.10

报错日志:

Application application_1562341921664_2123 failed 2 times due to AM Container for appattempt_1562341921664_2123_000002 exited with exitCode: -104
For more detailed output, check the application tracking page: http://winner-offline-namenode:8088/cluster/app/application_1562341921664_2123 Then click on links to logs of each attempt.
Diagnostics: Container [pid=8891,containerID=container_e12_1562341921664_2123_02_000001] is running beyond physical memory limits. Current usage: 3.0 GB of 3 GB physical memory used; 5.0 GB of 6.3 GB virtual memory used. Killing container.
Dump of the process-tree for container_e12_1562341921664_2123_02_000001 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 8891 8889 8891 8891 (bash) 0 0 112881664 367 /bin/bash -c LD_LIBRARY_PATH=/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64: /usr/java/jdk1.8.0_162/bin/java -server -Xmx2048m -Djava.io.tmpdir=/disk1/hadoop/yarn/local/usercache/hadoop/appcache/application_1562341921664_2123/container_e12_1562341921664_2123_02_000001/tmp ‘-XX:+PrintGCDetails‘ ‘-XX:+PrintGCTimeStamps‘ -Dhdp.version=2.6.4.0-91 -Dspark.yarn.app.container.log.dir=/disk1/hadoop/yarn/log/application_1562341921664_2123/container_e12_1562341921664_2123_02_000001 org.apache.spark.deploy.yarn.ApplicationMaster --class ‘com.winner.wifi.WinWifiAPIncre‘ --jar file:/hadoop/datadir/guozy/jar/winnercloud-assembly.jar --arg ‘2019-07-02‘ --arg ‘bg‘ --arg ‘St_P00094#St_P00087#Xsj_00001P00016#Cqhm_P00091#Langold_00004P00001#Xcjt_00004P00010#Djzx_P00001#Xsj_00001P00035#Xsj_00001P00013#Whc_P00081#Sygc_00001P00001#Szc_P00017#Lyyysc_P00001#Longfor_00001P00013#Lgjt_P00040#Wfjjt_P00028#Qzkyss_00001P00001#Hfjt_P00002#Xxhjgjgc_P00001#Xsj_00001P00024#Wdl_P00103#Hualian_00008P00020#Xl_P00067#Bfc_P00114#Snjt_00001P00001#Lfjt_P00002#Xuhui_P00050#Bailian_00002P00051#Xinyuan_P00077#Wdl_P00105#Hualian_P00004#Wfjjt_P00095#Fxhyjt_00001P00001#Xsj_00001P00023#Xsj_00001P00020#Xhbh_P00089#Szc_P00001#Yxgw_00001P00001#Wfjjt_P00035#Mybh_P00001#Yhjt_00002P00001#Wdl_P00106#Nydtjt_00001P00001#Xsj_00001P00002#Xuhui_P00038#Xsj_00001P00017#Xsj_00001P00012#Xsj_00001P00027#Lfjt_P00001#Xsj_00001P00034#Xhbh_P00108#Wjjt_P00022#Hjgc_P00001#Zxtfgc_P00001#Xhzb_P00001#Txjt_P00001#Mybh_P00003#Hdjt_00002P00018#Bailian_00002P00026#Hrzd_00002P00001#Bailian_00002P00052#Hxmkl_P00007#Ayjt_P00002#Wdl_P00107#Bailian_00002P00004#Ayjt_00004P00013#Hxmkl_P00009#Xsj_00001P00006#Wfjjt_P00027‘ --arg ‘1‘ --arg ‘150‘ --executor-memory 2048m --executor-cores 1 --properties-file /disk1/hadoop/yarn/local/usercache/hadoop/appcache/application_1562341921664_2123/container_e12_1562341921664_2123_02_000001/__spark_conf__/__spark_conf__.properties 1> /disk1/hadoop/yarn/log/application_1562341921664_2123/container_e12_1562341921664_2123_02_000001/stdout 2> /disk1/hadoop/yarn/log/application_1562341921664_2123/container_e12_1562341921664_2123_02_000001/stderr
|- 8906 8891 8891 8891 (java) 51299 2993 5230977024 787116 /usr/java/jdk1.8.0_162/bin/java -server -Xmx2048m -Djava.io.tmpdir=/disk1/hadoop/yarn/local/usercache/hadoop/appcache/application_1562341921664_2123/container_e12_1562341921664_2123_02_000001/tmp -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Dhdp.version=2.6.4.0-91 -Dspark.yarn.app.container.log.dir=/disk1/hadoop/yarn/log/application_1562341921664_2123/container_e12_1562341921664_2123_02_000001 org.apache.spark.deploy.yarn.ApplicationMaster --class com.winner.wifi.WinWifiAPIncre --jar file:/hadoop/datadir/guozy/jar/winnercloud-assembly.jar --arg 2019-07-02 --arg bg --arg St_P00094#St_P00087#Xsj_00001P00016#Cqhm_P00091#Langold_00004P00001#Xcjt_00004P00010#Djzx_P00001#Xsj_00001P00035#Xsj_00001P00013#Whc_P00081#Sygc_00001P00001#Szc_P00017#Lyyysc_P00001#Longfor_00001P00013#Lgjt_P00040#Wfjjt_P00028#Qzkyss_00001P00001#Hfjt_P00002#Xxhjgjgc_P00001#Xsj_00001P00024#Wdl_P00103#Hualian_00008P00020#Xl_P00067#Bfc_P00114#Snjt_00001P00001#Lfjt_P00002#Xuhui_P00050#Bailian_00002P00051#Xinyuan_P00077#Wdl_P00105#Hualian_P00004#Wfjjt_P00095#Fxhyjt_00001P00001#Xsj_00001P00023#Xsj_00001P00020#Xhbh_P00089#Szc_P00001#Yxgw_00001P00001#Wfjjt_P00035#Mybh_P00001#Yhjt_00002P00001#Wdl_P00106#Nydtjt_00001P00001#Xsj_00001P00002#Xuhui_P00038#Xsj_00001P00017#Xsj_00001P00012#Xsj_00001P00027#Lfjt_P00001#Xsj_00001P00034#Xhbh_P00108#Wjjt_P00022#Hjgc_P00001#Zxtfgc_P00001#Xhzb_P00001#Txjt_P00001#Mybh_P00003#Hdjt_00002P00018#Bailian_00002P00026#Hrzd_00002P00001#Bailian_00002P00052#Hxmkl_P00007#Ayjt_P00002#Wdl_P00107#Bailian_00002P00004#Ayjt_00004P00013#Hxmkl_P00009#Xsj_00001P00006#Wfjjt_P00027 --arg 1 --arg 150 --executor-memory 2048m --executor-cores 1 --properties-file /disk1/hadoop/yarn/local/usercache/hadoop/appcache/application_1562341921664_2123/container_e12_1562341921664_2123_02_000001/__spark_conf__/__spark_conf__.properties
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143.
Failing this attempt. Failing the application. 

报错截图:

    技术图片

主要资源配置:

  --driver-memory 2g

       --executore-memory 2g

临时解决方案:

      增大driver端的内存

       --driver-memory 3g

       --executore-memory 2g

问题解决。

这里有一点还没有搞太明白,就是为什么只要driver端内存比executore内存大就可以呢,具体深层次的原因还没找到,如果网友知道,还望赐教,不胜感激

   

 

spark运行任务报错:Container [...] is running beyond physical memory limits. Current usage: 3.0 GB of 3 GB physical memory used; 5.0 GB of 6.3 GB virtual memory used. Killing container.

标签:gcd   rac   问题   rop   ast   stderr   stdout   mamicode   tmp   

原文地址:https://www.cnblogs.com/Gxiaobai/p/11166986.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!