码迷,mamicode.com
首页 > 其他好文 > 详细

一、Hadoop伪分布式环境搭建

时间:2016-05-07 23:41:36      阅读:175      评论:0      收藏:0      [点我收藏+]

标签:

Hadoop 2.x伪分布式环境搭建步骤:

1、修改hadoop-env.sh、yarn-env.sh、mapred-env.sh

方法:使用notepad++(beifeng用户)代开这三个文件

添加代码:export JAVA_HOME=/opt/modules/jdk1.7.0_67

2、修改core-site.xml、hdfs-site.xml、yarn-site.xml、mapred-site.xml配置文件

1)修改core-site.xml

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://Hadoop-senior02.beifeng.com:8020</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/opt/modules/hadoop-2.5.0/data</value>
  </property>
</configuration>

2)修改hdfs-site.xml

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
    <name>dfs.namenode.http-address</name>
    <value>Hadoop-senior02.beifeng.com:50070</value>
  </property>
</configuration>

3)修改yarn-site.xml

<configuration>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>Hadoop-senior02.beifeng.com</value>
  </property>
  <property>
    <name>yarn.log-aggregation-enable</name>
    <value>true</value>
  </property>
  <property>
    <name>yarn.log-aggregation.retain-seconds</name>
    <value>86400</value>
  </property>
</configuration>

4)修改mapred-site.xml

<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>0.0.0.0:19888</value>
  </property>
</configuration>

3、启动hdfs

1)格式化namenode:$ bin/hdfs namenode -format

2)启动namenode:$sbin/hadoop-daemon.sh start namenode

3)启动datanode:$sbin/hadoop-daemon.sh start datanode

4)hdfs监控web页面:http://hadoop-senior02.beifeng.com:50070

4、启动yarn

1)启动resourcemanager:$sbin/yarn-daemon.sh start resourcemanager

2)启动nodemanager:sbin/yarn-daemon.sh start nodemanager

3)yarn监控web页面:http://hadoop-senior02.beifeng.com:8088

5、测试wordcount jar包

1)定位路径:/opt/modules/hadoop-2.5.0

2)代码测试:

一、Hadoop伪分布式环境搭建

标签:

原文地址:http://www.cnblogs.com/really0612/p/5469443.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!