码迷,mamicode.com
首页 > 系统相关 > 详细

install hadoop on xubuntu

时间:2017-01-06 22:07:36      阅读:329      评论:0      收藏:0      [点我收藏+]

标签:property   version   ext   word   home   nod   file   without   编辑   

0. install xubuntu

we recommend to set username as "hadoop"

after installation, set user "hadoop" as administrator

sudo addgroup hadoop  
sudo adduser --ingroup hadoop hadoop 

打开/etc/sudoers文件

sudo gedit /etc/sudoers  

在root  ALL=(ALL:ALL)  ALL下添加hadoop  ALL=(ALL:ALL)  ALL

 

1. install java

1.解压java压缩包到usr/java(新建的文件夹)中。解压后就可使用

2.配置环境变量。如下  
在etc/profile 文件中。在最后添加如下内容    
#set java environment  
export JAVA_HOME=/usr/java/jdk1.7.0_67  
export JRE_HOME=/usr/java/jdk1.7.0_67/jre    
export PATH=$PATH:/usr/java/jdk1.7.0_67/bin    
export CLASSPATH=./:/usr/java/jdk1.7.0_67/lib:/usr/java/jdk1.7.0_67/jre/lib    
  
3.配置立即生效命令  
  source /etc/profile  

4.检测是否配置成功  
  java -version  
  
如果不行,重启linux  

 

2. configure login in ssh without entering password

please operate under user "hadoop"

su - hadoop  
sudo apt-get install openssh-server 
sudo /etc/init.d/ssh start  

cd ~/.ssh
ssh-keygen -t rsa -P ""  
cat ~/.ssh/id_rsa.pub>> ~/.ssh/authorized_keys  

 

3. install hadoop

1. unzip hadoop.tar.gz into /usr/hadoop
  then, ensure user "hadoop" owns /usr/hadoop
  
sudo chown -R hadoop:hadoop hadoop  
2. edit environment
    2.1 gedit /etc/profile    append these: 

export JAVA_HOME=/usr/java/  
export JRE_HOME=/usr/java/jre    
export HADOOP_INSTALL=/usr/hadoop
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin
export CLASSPATH=./:/usr/java/lib:/usr/java/jre/lib

    2.2 gedit /usr/hadoop/conf/hadoop-env.sh    append these:

# The java implementation to use.  Required.
export JAVA_HOME=/usr/java  
export HADOOP_INSTALL=/usr/hadoop
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin

3. restart linux

 

4. test

hadoop@ms:~$ 
hadoop@ms:~$ java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)
hadoop@ms:~$ hadoop version
Hadoop 1.2.1
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152
Compiled by mattf on Mon Jul 22 15:23:09 PDT 2013
From source with checksum 6923c86528809c4e7e6f493b6b413a9a
This command was run using /usr/hadoop/hadoop-core-1.2.1.jar
hadoop@ms:~$ 

 

5. hadoop 伪分布式

技术分享
编辑三个文件:
1). core-site.xml:

<configuration>  
    <property>  
        <name>fs.default.name</name>  
        <value>hdfs://localhost:9000</value>  
    </property>  
    <property>  
        <name>hadoop.tmp.dir</name>  
        <value>/usr/local/hadoop/tmp</value>  
    </property>  
</configuration>  


2).hdfs-site.xml:

<configuration>  
    <property>  
        <name>dfs.replication</name>  
        <value>2</value>  
    </property>  
    <property>  
        <name>dfs.name.dir</name>  
        <value>/usr/local/hadoop/datalog1,/usr/local/hadoop/datalog2</value>  
    </property>  
    <property>  
        <name>dfs.data.dir</name>  
        <value>/usr/local/hadoop/data1,/usr/local/hadoop/data2</value>  
    </property>  
</configuration>  

3). mapred-site.xml:

<configuration>     
    <property>    
        <name>mapred.job.tracker</name>  
        <value>localhost:9001</value>     
    </property>  
</configuration>  


2. 启动Hadoop到相关服务,格式化namenode, secondarynamenode, tasktracker:
hadoop@derekUbun:/usr/local/hadoop$ source /usr/local/hadoop/conf/hadoop-env.sh   
hadoop@derekUbun:/usr/local/hadoop$ hadoop namenode -format  
View Code

 

6*. install hbase[伪分布式]

1. unzip hbase.tar.gz into /usr/hbase
  then, ensure user "hadoop" owns /usr/hbase
  
sudo chown -R hadoop:hadoop hbase  

2. edit environment
    2.1 gedit /etc/profile    append these: 

export HBASE_HOME="/usr/hbase"
export PATH=$HBASE_HOME/bin:$PATH

    2.2 gedit /usr/hbase/conf/hbase-site.xml    append these:

<property>
     <name>hbase.rootdir</name>
     <!-- 对应hadoop中hdfs的配置项 -->
     <value>hdfs://localhost:9000/hbase</value>
 </property>
 <property>
     <name>hbase.cluster.distributed</name>
     <value>true</value>
</property>
<property>
     <name>hbase.master.info.port</name>
     <value>60010</value>
</property>
  
    2.3 gedit /usr/hbase/hbase-env.sh    modify these:

# The java implementation to use.  Java 1.6 required.
export JAVA_HOME=/usr/java/

# Extra Java CLASSPATH elements.  Optional.
export HBASE_CLASSPATH=/usr/hadoop/conf

# Tell HBase whether it should manage its own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=true

3. restart linux
    

 

 

 

#. references

http://blog.csdn.net/zhaoyl03/article/details/8657104#

http://www.tuicool.com/articles/VZn6zi

http://blog.csdn.net/zhaoyl03/article/details/8657104#

http://blog.csdn.net/pdw2009/article/details/21261417

http://www.th7.cn/db/nosql/201510/134214.shtml

 

install hadoop on xubuntu

标签:property   version   ext   word   home   nod   file   without   编辑   

原文地址:http://www.cnblogs.com/pdev/p/6257007.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!