码迷,mamicode.com
首页 > 其他好文 > 详细

Hadoop Kernel tunning

时间:2015-12-16 09:24:32      阅读:152      评论:0      收藏:0      [点我收藏+]

标签:

/etc/security/limits.conf

@ochadoop soft nofile 102642
@ochadoop hard nofile 102642
@ochadoop soft nproc unlimited
@ochadoop hard nproc unlimited
@ochadoop soft memlock unlimited
@ochadoop hard memlock unlimited

/etc/sysctl.conf

#尽可能少地让应用把内容交换到硬盘
echo "vm.swappiness = 0" >> /etc/sysctl.conf
#可以让应用在请求内存的时候使用交换
echo vm.overcommit_memory = 1 >> /etc/sysctl.conf

Mount options:disabling file access time

The real problem with maintaining access time (or
atime as it’s commonly called) is that every time a file is read, the metadata needs to be
updated. That is, for each read, there’s also a mandatory write. This is relatively expensive at scale and can negatively impact the overall performance of Hadoop

sample:

LABEL=/ / ext3 noatime,nodiratime 1 1
LABEL=/data/1 /data/1 ext3 noatime,nodiratime 1 2
LABEL=/data/2 /data/2 ext3 noatime,nodiratime 1 2
LABEL=/data/3 /data/3 ext3 noatime,nodiratime 1 2
LABEL=/data/4 /data/4 ext3 noatime,nodiratime 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SWAP-sda2 swap swap defaults 0 0

Hadoop Kernel tunning

标签:

原文地址:http://www.cnblogs.com/huaxiaoyao/p/5050033.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!