适合国内网速的CDH5安装

1）进入yum源配置文件夹

cd /etc/yum.repos.d

2）备份系统自带的yum源

mv CentOS-Base.repo CentOS-Base.repo.bk

3）下载163的yum源：

wget http://mirrors.163.com/.help/CentOS-Base-163.repo

mv CentOS6-Base-163.repo CentOS-Base.repo

3）更新完yum源后，运行下边命令更新yum配置，使操作马上生效

yum makecache

yum clean all

1）进入 yum 源文件夹

cd /etc/yum.repos.d

2）查看cloudera-manager yum 源文件是否已经下载下来

多了一个 cloudera-manager.repo 文件

3）得到rpm下载地址

    解决方法：rm -rf /usr/share/cmf/

7.1 将前面下载的CDH parcels文件放到 /opt/cloudera/parcel-repo 文件夹下

7.2 生成相应的sha文件

1）在第5步下载的 manifest.json 文件里依照版本号“CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel”找到相应的 hash值

"hash": "67fc4c86b260eeba15c339f1ec6be3b59b4ebe30"

2）hash值保存在sha文件里

echo ‘67fc4c86b260eeba15c339f1ec6be3b59b4ebe30‘>CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel.sha

8. 启动Cloudera Manager

安装中出现例如以下问题，详细解决方案见 “问题列表” 问题1：
python -c ‘import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect(("localhost", int(7182))); s.close();‘

9. 加入服务

參考文档:

说明：尽量把全部的參考文档都列了出来，假设有遗漏欢迎提醒。

Cloudera Manager 和CDH 4 终极安装http://www.tuicool.com/articles/AnuiUra

C?l?o?u?d?e?r?a?M?a?n?a?g?e?r?和?C?D?H?4?安?装:http://wenku.baidu.com/link?

url=SOOI3r56NN7Un55Z3jsNprQp9PpOc-F8_ByXPJ7v4GJmAioEMLM6vL0Hkc2c0HSxztlWWvPOA13Grs1vf2-0wJdbueQfbEAvuNbGIldxxou

CDH套件半手动安装流程http://www.douban.com/note/352772895/

使用yum安装CDH Hadoop集群(cdh5 禁用ipv6、hostname设置、设置yum源、时钟同步)： http://blog.javachen.com/2013/04/06/install-cloudera-cdh-by-yum/ 

查看 SELinux状态及关闭SELinux:http://bguncle.blog.51cto.com/3184079/957315

CentOS6.5中改动yum源：http://www.cnblogs.com/liuling/p/2014-4-14-001.html

问题列表：

描写叙述：

DNS反向解析错误，不能正确解析Cloudera Manager Server主机名
日志：
Detecting Cloudera Manager Server...
Detecting Cloudera Manager Server...

BEGIN host -t PTR 192.168.1.198

198.1.168.192.in-addr.arpa domain name pointer localhost. 

END (0) 

using localhost as scm server hostname 

BEGIN which python 

/usr/bin/python 

END (0) 

BEGIN python -c ‘import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();‘ localhost 7182

Traceback (most recent call last): 

File "<string>", line 1, in <module> 

File "<string>", line 1, in connect 

socket.error: [Errno 111] Connection refused

END (1) 

could not contact scm server at localhost:7182, giving up 

waiting for rollback request

不优雅的解决方式：

将连不上的机器 /usr/bin/host 文件删掉

BEGIN host -t PTR 192.168.1.198

/tmp/scm_prepare_node.8OX5y7is/scm_prepare_node.sh: line 100: /usr/bin/host: 权限不够

END (126) 

BEGIN which python 

/usr/bin/python 

END (0) 

BEGIN python -c ‘import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();‘ 192.168.1.198 7182

END (0) 

BEGIN which wget 

/usr/bin/wget 

END (0) 

BEGIN wget -qO- -T 1 -t 1 http://169.254.169.254/latest/meta-data/public-hostname && /bin/echo

END (4)

说明：

不明确cloudera的初衷，这里已经得到 Cloudera Manager Server的ip了，却还要把ip解析成主机名来连接

因为DNS反向解析没有配置好。依据Cloudera Manager Server 的ip解析主机名却得到了localhost，造成之后的连接错误

这里的解决方式是直接把/usr/bin/host删掉，这样Cloudera Manager就会直接使用 ip进行连接，就没有错了

參考：

cloudera manager 4.8 加入设备DNS反向解析有关问题 

http://www.reader8.cn/jiaocheng/20140419/2307406.html

问题 2.1 

问题描写叙述： 

Bad Health --Clock Offset

The host‘s NTP service did not respond to a request for the clock offset.

解决：

配置NTP服务

步骤參考：

CentOS配置NTP Server:

http://www.hailiangchen.com/centos-ntp/

国内经常使用NTPserver地址及IP

http://www.douban.com/note/171309770/

改动配置文件：

[root@work03 ~]# vim /etc/ntp.conf

# Use public servers from the pool.ntp.org project.

# Please consider joining the pool (http://www.pool.ntp.org/join.html).

server s1a.time.edu.cn prefer

server s1b.time.edu.cn

server s1c.time.edu.cn

restrict 172.16.1.0 mask 255.255.255.0 nomodify   <===放行局域网来源

启动ntp

#service ntpd restart    <===启动ntp服务

client同步时间(work02,work03）：

ntpdate work01

说明：NTP服务启动须要大约五分钟时间，服务启动之前，若client同步时间，则会出现错误“no server suitable for synchronization found”

定时同步时间：

在work02和 work03上配置crontab定时同步时间

crontab -e

00 12 * * * root /usr/sbin/ntpdate 192.168.56.121 >> /root/ntpdate.log 2>&1 

问题 2.2

描写叙述：

     Clock Offset

• Ensure that the host‘s hostname is configured properly.

• Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules).

• Ensure that ports 9000 and 9001 are free on the host being added.

• Check agent logs in /var/log/cloudera-scm-agent/ on the host being added (some of the logs can be found in the installation details).

问题定位：

在相应host（work02、work03）上执行 ‘ntpdc -c loopinfo‘

[root@work03 work]# ntpdc -c loopinfo

ntpdc: read: Connection refused

解决：

开启ntp服务：

三台机器都开机启动 ntp服务

chkconfig ntpd on

问题 3 heartbeat:

错误信息：

Installation failed. Failed to receive heartbeat from agent.

解决：关闭防火墙

问题 4 Unknow Health：

Unknow Health
重新启动后：Request to theHost Monitor failed.
service --status-all| grep clo
机器上查看scm-agent状态：cloudera-scm-agent dead but pid file exists
解决：重新启动服务
 service cloudera-scm-agent restart

 service cloudera-scm-server restart

---

问题 5 canonial name hostname consistent：

Bad Health

The hostname and canonical name for this host are not consistent when checked from a Java process.

canonical name：

4092 Monitor-HostMonitor throttling_logger WARNING  (29 skipped) hostname work02 differs from the canonical name work02.xinzhitang.com

解决：改动hosts 使FQDN和 hostname同样

ps：尽管攻克了可是不明确为什么主机名和主机别名要一样

/etc/hosts

192.168.1.185 work01 work01

192.168.1.141 work02 work02

192.168.1.198 work03 work03

问题 6 Concerning Health：

Concerning Health Issue 

--  Network Interface Speed --

描写叙述：The host has 2 network interface(s) that appear to be operating at less than full speed. Warning threshold: any.

具体：

This is a host health test that checks for network interfaces that appear to be operating at less than full speed.

A failure of this health test may indicate that network interface(s) may be configured incorrectly and may be causing performance problems. Use the ethtool command to check and configure the host‘s network interfaces to use the fastest available link speed and duplex mode.

解决：

本次測试改动了 Cloudera Manager 的配置。应该不算是真正的解决