hadoop 开发环境搭建

添加java环境变量

 

 

二，修改文件句柄数

vi  /etc/security/limits.conf

 

# add by tank

* soft nofile 65536
* hard nofile  65536

 

 

三，设置ssh无密码登录

 

  参考：http://www.cnblogs.com/tankaixiong/p/4172942.html

 

四，安装hadoop 环境

 

这里用的是hadoop2.20版本

目录结构：

修改配置文件：

[tank@192 hadoop]$ vi core-site.xml 

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
  <property>
     <name>fs.default.name</name>
     <value>hdfs://192.168.149.128:9000</value>
  </property>
</configuration>

 

[tank@192 hadoop]$ vi hdfs-site.xml 

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
      <name>dfs.namenode.name.dir</name>
      <value>file:/data/soft/hadoop/hadoop-2.2.0/hdfs/name</value>
        <final>true</final>
   </property>
   <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/data/soft/hadoop/hadoop-2.2.0/hdfs/data</value>
   </property>
</configuration>

 

 

[tank@192 hadoop]$ vi mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
   <property>
     <name>mapred.job.tracker</name>
     <value>192.168.149.128:9001</value>
   </property>
</configuration>
~
~

 

 

 

启动:

sbin/start-all.sh

 

查看状态：

5295 ResourceManager

 

总共有5个hadoop线程 

 

访问地址查看：

http://192.168.149.128:50070/dfshealth.jsp