ambari本地源自动化安装hortonworks hadoop(转)

标签：

原文地址：http://blog.chinaunix.net/xmlrpc.php?r=blog/article&uid=26230811&id=4023821

环境配置

操作系统建议以software server 方式安装，base 方式安装后，需要安装Development tools，本文以base举例。

一：制作ambari，hadoop本地源

首先需要有一台能够联网的linux机器

1：制作ambari（1.4.1.25）本地源

[root@controller ~]# wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.4.1.25/ambari.repo

[root@controller ~]# cp ambari.repo /etc/yum.repos.d/

将网络源中的包同步到本地：

reposync -r HDP-UTILS-1.1.0.16

reposync -r Updates-ambari-1.4.1.25

reposync -r ambari-1.x

会得到相应的三个目录，每个目录中有相应的rpm包

准备一台http服务器，本文以172.16.235.164作为http服务器发布安装源

mkdir /var/www/html/ambari

将 HDP-UTILS-1.1.0.16 Updates-ambari-1.4.1.25 ambari-1.x复制到 /var/www/html/ambari下

cd /var/www/html/ambari

createrepo ./    #没有createrepo命令自己去安装

这样ambari安装源就做好了，下载好jdk-6u31-linux-x64.bin备用

2：准备hortonworks hadoop （HDP）2.0.6本地源

准备HDP网络源

vim /etc/yum.repos.d/HDP.repo，添加以下内容

[HDP-2.0.6]

name=HDP

baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.0.6.0

enabled=1

gpgcheck=0

同步网络源到本地：

reposync -r HDP-2.0.6  #将近1G

讲 HDP-2.0.6 复制到http server的/var/www/html/hdp下

cd /var/www/html/hdp

createrepo ./  

这样hadoop的本地源就做好了

3：制作系统本地源（安装ambari和hadoop时，可能会依赖些系统包，这时需要有系统的本地源来安装）

http server 挂载系统光盘

mount /dev/cdrom  /var/www/html/centos/

二：安装ambari

系统准备：关闭iptables、selinux,启动ntpd服务

[root@ambari ]vim /etc/hosts

172.16.235.164  ambari

172.16.235.165  master.hadoop

172.16.235.166  datanode1.hadoop

配置与所有主机的无密码登入

[root@ambari ~]# ssh-keygen  #一路回车（所有主机都要执行）

[root@ambari ~]# cd .ssh/

[root@ambari .ssh]# ls

id_rsa  id_rsa.pub

[root@ambari .ssh]# scp id_rsa.pub master.hadoop:/root/.ssh/1

The authenticity of host ‘master.hadoop (172.16.235.165)‘ can‘t be established.

RSA key fingerprint is fc:33:53:5f:f2:20:86:92:ac:96:09:e2:dc:e7:33:ea.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added ‘master.hadoop,172.16.235.165‘ (RSA) to the list of known hosts.

root@master.hadoop‘s password:

id_rsa.pub                                                                                        100%  393     0.4KB/s   00:00   

[root@ambari .ssh]# scp id_rsa.pub datanode1.hadoop:/root/.ssh/1

The authenticity of host ‘datanode1.hadoop (172.16.235.166)‘ can‘t be established.

RSA key fingerprint is fc:33:53:5f:f2:20:86:92:ac:96:09:e2:dc:e7:33:ea.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added ‘datanode1.hadoop,172.16.235.166‘ (RSA) to the list of known hosts.

root@datanode1.hadoop‘s password:

id_rsa.pub                                                                                        100%  393     0.4KB/s   00:00  

登入master.hadoop

[root@master ~]# cd .ssh/

[root@master .ssh]# cat 1> authorized_keys

登入datanode1.hadoop

[root@datanode1 ~]# cd .ssh/

[root@datanode1 .ssh]# cat 1> authorized_keys

回到ambari 测试无密码登入

[root@ambari .ssh]# ssh master.hadoop

Last login: Wed Dec  4 11:11:15 2013 from 172.16.38.48

[root@master ~]# exit

logout

Connection to master.hadoop closed.

[root@ambari .ssh]# ssh datanode1.hadoop

Last login: Wed Dec  4 10:38:06 2013 from 172.16.38.48

[root@datanode1 ~]# exit

logout

Connection to datanode1.hadoop closed.

[root@ambari .ssh]#

安装与配置ambari服务：

[root@ambari ]vim /etc/yum.repos.d/ambari.repo  添加以下内容

[ambari-1.x]

name=Ambari 1.x

baseurl=http://172.16.235.164/ambari/

gpgcheck=0

enabled=1

[HDP-UTILS-1.1.0.16]

name=Hortonworks Data Platform Utils Version - HDP-UTILS-1.1.0.16

baseurl=http://172.16.235.164/ambari/

gpgcheck=0

enabled=1

[Updates-ambari-1.4.1.25]

name=ambari-1.4.1.25 - Updates

baseurl=http://172.16.235.164/ambari/

gpgcheck=

enabled=1

[root@ambari ]vim /etc/yum.repos.d/centos.repo  添加以下内容

[centos]

baseurl=http://172.16.235.164/centos/

enabled=1

gpgcheck=0

执行安装ambari：

[root@ambari yum.repos.d]# yum -y install ambari-server

Loaded plugins: fastestmirror, security

Repository ‘Updates-ambari-1.4.1.25‘: Error parsing config: Error parsing "gpgcheck = ‘‘": invalid boolean value

Repository ‘centos‘ is missing name in configuration, using id

Loading mirror speeds from cached hostfile

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package ambari-server.noarch 0:1.4.1.25-1 will be installed

--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.4.1.25-1.noarch

--> Running transaction check

---> Package postgresql-server.x86_64 0:8.4.13-1.el6_3 will be installed

--> Processing Dependency: postgresql-libs(x86-64) = 8.4.13-1.el6_3 for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Processing Dependency: postgresql(x86-64) = 8.4.13-1.el6_3 for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Running transaction check

---> Package postgresql.x86_64 0:8.4.13-1.el6_3 will be installed

---> Package postgresql-libs.x86_64 0:8.4.13-1.el6_3 will be installed

--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================================

Package                           Arch                   Version                          Repository                          Size

====================================================================================================================================

Installing:

ambari-server                     noarch                 1.4.1.25-1                       HDP-UTILS-1.1.0.16                  37 M

Installing for dependencies:

postgresql                        x86_64                 8.4.13-1.el6_3                   centos                             2.8 M

postgresql-libs                   x86_64                 8.4.13-1.el6_3                   centos                             200 k

postgresql-server                 x86_64                 8.4.13-1.el6_3                   centos                             3.4 M

Transaction Summary

====================================================================================================================================

Install       4 Package(s)

Total download size: 43 M

Installed size: 88 M

Downloading Packages:

(1/4): ambari-server-1.4.1.25-1.noarch.rpm                                                                   |  37 MB     00:00    

(2/4): postgresql-server-8.4.13-1.el6_3.x86_64.rpm                                                           | 3.4 MB     00:00    

(3/4): postgresql-libs-8.4.13-1.el6_3.x86_64.rpm                                                             | 200 kB     00:00    

(4/4): postgresql-8.4.13-1.el6_3.x86_64.rpm                                                                  | 2.8 MB     00:00    

------------------------------------------------------------------------------------------------------------------------------------

Total                                                                                                43 MB/s |  43 MB     00:00    

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

  Installing : postgresql-libs-8.4.13-1.el6_3.x86_64                                                                            1/4

  Installing : postgresql-8.4.13-1.el6_3.x86_64                                                                                 2/4

  Installing : postgresql-server-8.4.13-1.el6_3.x86_64                                                                          3/4

  Installing : ambari-server-1.4.1.25-1.noarch                                                                                  4/4

  Verifying  : postgresql-server-8.4.13-1.el6_3.x86_64                                                                          1/4

  Verifying  : postgresql-libs-8.4.13-1.el6_3.x86_64                                                                            2/4

  Verifying  : ambari-server-1.4.1.25-1.noarch                                                                                  3/4

  Verifying  : postgresql-8.4.13-1.el6_3.x86_64                                                                                 4/4

Installed:

  ambari-server.noarch 0:1.4.1.25-1                                                                                                

Dependency Installed:

  postgresql.x86_64 0:8.4.13-1.el6_3     postgresql-libs.x86_64 0:8.4.13-1.el6_3     postgresql-server.x86_64 0:8.4.13-1.el6_3   

Complete!

开始配置ambari 服务：

首先上传jdk文件

[root@ambari ]cd /var/lib/ambari-server/resources  将jdk-6u31-linux-x64.bin上传到该目录

[root@ambari resources]# chmod +x jdk-6u31-linux-x64.bin

[root@ambari yum.repos.d]# ambari-server setup

Using python  /usr/bin/python2.6

Initializing...

Setup ambari-server

Checking SELinux...

SELinux status is ‘enabled‘

SELinux mode is ‘permissive‘

WARNING: SELinux is set to ‘permissive‘ mode and temporarily disabled.

OK to continue [y/n] (y)?

Customize user account for ambari-server daemon [y/n] (n)?

Adjusting ambari-server permissions and ownership...

Checking iptables...

WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.

OK to continue [y/n] (y)?

Checking JDK...

JDK already exists, using /var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin

Installing JDK to /usr/jdk64

Successfully installed JDK to /usr/jdk64/jdk1.6.0_31

Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-6.zip to /var/lib/ambari-server/resources/jce_policy-6.zip

To download the JCE Policy files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.htmlNot accepting will result in errors when configuring Kerberos security.

Do you accept the Oracle Binary Code License Agreement [y/n] (y)?

Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-6.zip

Completing setup...

Configuring database...

Enter advanced database configuration [y/n] (n)?

Default properties detected. Using built-in database.

Checking PostgreSQL...

Running initdb: This may take upto a minute.

About to start PostgreSQL

Configuring local database...

Connecting to the database. Attempt 1...

Configuring PostgreSQL...

Restarting PostgreSQL

Ambari Server ‘setup‘ completed successfully.

[root@ambari ~]# ambari-server start

Using python  /usr/bin/python2.6

Starting ambari-server

Ambari Server running with ‘root‘ privileges.

Server PID at: /var/run/ambari-server/ambari-server.pid

Server out at: /var/log/ambari-server/ambari-server.out

Server log at: /var/log/ambari-server/ambari-server.log

Ambari Server ‘start‘ completed successfully.

到此ambari安装好

三：本地安装hadoop

给ambari配置本地hadoo源地址：

[root@ambari ~]# cd /var/lib/ambari-server/resources/stacks/HDPLocal/2.0.6/repos/

[root@ambari repos]# vim repoinfo.xml  修改红色部分为本地源地址

<reposinfo>

< os type="centos6">

   < repo>

     < baseurl>http://172.16.235.164/hdp/</baseurl>

     < repoid>HDP-2.0.6</repoid>

     < reponame>HDP</reponame>

   < /repo>

< /os>

这样在ambari执行hadoop安装时，会将本地源地址配置到所有主机上，在为所有主机安装ambari-agent时，会将ambari server下的/etc/yum.repo/ambari.repo文件复制到所有主机

主机准备（master.hadoop、datanode1.hadoop）配置相同，只举例master.hadoop

关闭iptables、selinux、启动ntpd服务（略）

[root@master ~]# vim /etc/hosts 添加以下

172.16.235.164  ambari

172.16.235.165  master.hadoop

172.16.235.166  datanode1.hadoop

[root@master ~]# vim /etc/yum.repos.d/centos.repo 添加以下内容

[root

[centos]

baseurl=http://172.16.235.164/centos/

enabled=1

gpgcheck=0

@master ~]# yum -y groupinstall "Development tools"

登入ambari管理页面

http://172.16.235.164:8080/ 账户：admin 密码：admin

配置集群名称

选择HDP版本

选择目标主机，一行一个主机。要让ambar server自动在所以主机安装ambari-agent，需要提供ambari server root账户的私钥，即/root/.ssh/ id_rsa文件，选中使用本地源安装hadoop，然后下一步

确认并且选择主机

确认OK

下一步选择要安装的组件，选好后下一步

选择组件分配在哪些主机，都是master 的服务

选择安装client，主机有限，master同时安装client

配置服务，其中nagios中要填写密码及报警接受邮箱必填，其他选择性配置

显示配置信息

开始安装，过程稍微有点长

安装过程

Google浏览器反应比较慢，用另一某浏览器已经显示succes了

Ganglia：

nagios：

ambari本地源自动化安装hortonworks hadoop(转)

标签：

原文地址：http://www.cnblogs.com/xwj517537691/p/4502628.html