2)提示出错
Error: Java heap space
问题产生原因:
"if too much time is being spent in garbage collection: if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered, an OutOfMemoryError will be thrown."
jvm gc行为中超过98%以上的时间去释放小于2%的堆空间时会报这个错误。
处理方法:
1. 在jvm启动参数中添加 "-XX:-UseGCOverheadLimit",该参数在JDK6中默认启用("-XX:+UseGCOverheadLimit")。
调整后的生产环境中使用的参数为:
JAVA_OPTS=‘-Xms512m -Xmx4096m -XX:MaxPermSize=128m -XX:-UseGCOverheadLimit -XX:+UseConcMarkSweepGC‘
2. 检查是否有使用了大量内存的代码或死循环
oozie中的处理方法:
configuration中添加节点数目
<configuration>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Djava.net.preferIPv4Stack=true -Xmx12734003200</value>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>10240</value>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Djava.net.preferIPv4Stack=true -Xmx12468006400</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>1024</value>
</property>
<property>
<name>oozie.launcher.mapreduce.map.memory.mb</name>
<value>1536</value>
</property>
<property>
<name>oozie.launcher.mapreduce.map.java.opts</name>
<value>-Xmx1536m</value>
</property>
</configuration>
注: mapreduce.map.java.opts增加map jvm内存
mapreduce.map.memory.mb map执行内存