标签:
编译过程漫长无比,错误百出,需要耐心耐心!!
操作系统:Centos6.4 64位
jdk:jdk-7u80-linux-x64.rpm,不要使用1.8
maven:apache-maven-3.3.3-bin.tar.gz
protobuf:protobuf-2.5.0.tar.gz 注:谷歌的产品,最好是提前百度准备一下这个文件
hadoop src:hadoop-2.5.2-src.tar.gz hadoop的官网下载
ant:apache-ant-1.9.6-bin.tar.gz 编译common的时候会用到
openssl devel
ncurses-devel
CMake:2.6 or later
编译过程保证网络畅通,需要从外网下载其他的依赖包
比较简单通过rpm -ivh jdk-7u80-linux-x64.rpm安装即可
tar zxvf hadoop-2.5.2-src.tar.gz
查看文件夹hadoop-2.5.2-src内的BUILDING.txt ,里面有编译所需要的环境要求
vi BUILDING.txt
直接解压缩到目录/usr
tar zxvf apache-maven-3.3.3-bin.tar.gz -C /usr
设置环境变量
vi + /etc/profile export MAVEN_HOME=/usr/apache-maven-3.3.3 export PATH=.:$PATH:$MAVEN_HOME/bin
使用source命令使配置文件生效
source /etc/profile
验证maven是否安装成功mvn -version,显示maven,java,linux等信息
解压缩到/usr目录
tar zxvf protobuf-2.5.0.tar.gz -C /usr
切换到目录/usr/protobuf-2.5.0中然后执行./configrue,make命令
./configure
需要安装gcc c++执行yum进行安装
yum install gcc yum install gcc-c++ #会两次提示输入y(yes/no) yum install make
我在进行上面第二个命令,对configure进行预编译时报错,查看异常后,是c++没有安装成功,便再一次执行了:yum install gcc-c++ 命令,OK
安装完成后执行./configure通过后执行下面的make命令
make make install
命令执行的时间比较长,最后一个命令的执行安装,如果出现错误,重新来过即可
执行完成后测试是否安装成功
protoc --version
CMake 需要2.6以上的版本,在安装的时候会提示输入 y/N,同时由于安装这个组件是需要联网的,故根据网速不同,安装的进度也不一样
yum install cmake yum install openssl-devel yum install ncurses-devel
解压文件
tar zxvf apache-ant-1.9.6-bin.tar.gz -C /usr
设置环境变量
vi + /etc/profile export ANT_HOME=/usr/apache-ant-1.9.6 export PATH=.:$PATH:$ANT_HOME/bin
使用source命令使配置文件生效
source /etc/profile
验证是否安装成功ant -version,显示版本信息
切换为hadoop用户
切换到目录hadoop-2.5.2-src,执行命令
mvn package-DeskipTests -Pdist,native
之后是漫长的等待过程,中间过程会下载很多的文件,具体作用不详,经常因为下载网络问题失败,再次执行编译命令即可,之前下载过不会重新执行下载
出现失败情况,如果发现在输出[INFO]前的日志是停留在某个下载链接上,直接重试
出现编译测试错误,如下图,暂时未查明原因,直接重试一次
意外中断编译后再次开始后,编译异常,如下图
为了解决问题,通过以下命令重新尝试,分为两步操作
第一步、执行
mvn clean install –DskipTests
命令,等待完成(会自动联网下载很多东西)网络通畅后迅速搞定,不通畅的时候反复了多次
[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop Main ................................. SUCCESS [ 4.721 s] [INFO] Apache Hadoop Project POM .......................... SUCCESS [ 1.135 s] [INFO] Apache Hadoop Annotations .......................... SUCCESS [ 4.990 s] [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 0.814 s] [INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.552 s] [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 4.834 s] [INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 4.277 s] [INFO] Apache Hadoop Auth ................................. SUCCESS [ 5.709 s] [INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 2.516 s] [INFO] Apache Hadoop Common ............................... SUCCESS [ 53.258 s] [INFO] Apache Hadoop NFS .................................. SUCCESS [ 1.175 s] [INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.070 s] [INFO] Apache Hadoop HDFS ................................. SUCCESS [01:05 min] [INFO] Apache Hadoop HttpFS ............................... SUCCESS [ 4.430 s] [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [ 28.926 s] [INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 0.851 s] [INFO] Apache Hadoop HDFS Project ......................... SUCCESS [ 0.092 s] [INFO] hadoop-yarn ........................................ SUCCESS [ 0.063 s] [INFO] hadoop-yarn-api .................................... SUCCESS [ 4.983 s] [INFO] hadoop-yarn-common ................................. SUCCESS [01:01 min] [INFO] hadoop-yarn-server ................................. SUCCESS [ 0.092 s] [INFO] hadoop-yarn-server-common .......................... SUCCESS [ 20.415 s] [INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [01:24 min] [INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [ 0.619 s] [INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [ 1.173 s] [INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 5.231 s] [INFO] hadoop-yarn-server-tests ........................... SUCCESS [ 1.075 s] [INFO] hadoop-yarn-client ................................. SUCCESS [ 1.853 s] [INFO] hadoop-yarn-applications ........................... SUCCESS [ 0.129 s] [INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [ 0.563 s] [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [ 0.515 s] [INFO] hadoop-yarn-site ................................... SUCCESS [ 0.112 s] [INFO] hadoop-yarn-project ................................ SUCCESS [ 0.132 s] [INFO] hadoop-mapreduce-client ............................ SUCCESS [ 0.151 s] [INFO] hadoop-mapreduce-client-core ....................... SUCCESS [ 8.638 s] [INFO] hadoop-mapreduce-client-common ..................... SUCCESS [ 5.152 s] [INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [ 1.165 s] [INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 4.623 s] [INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 1.714 s] [INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [ 18.468 s] [INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [ 0.856 s] [INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 1.955 s] [INFO] hadoop-mapreduce ................................... SUCCESS [ 0.096 s] [INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 9.471 s] [INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [01:08 min] [INFO] Apache Hadoop Archives ............................. SUCCESS [ 0.841 s] [INFO] Apache Hadoop Rumen ................................ SUCCESS [ 1.113 s] [INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 2.258 s] [INFO] Apache Hadoop Data Join ............................ SUCCESS [ 0.981 s] [INFO] Apache Hadoop Extras ............................... SUCCESS [ 1.185 s] [INFO] Apache Hadoop Pipes ................................ SUCCESS [ 0.098 s] [INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 1.246 s] [INFO] Apache Hadoop Client ............................... SUCCESS [ 0.427 s] [INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 0.336 s] [INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 21.003 s] [INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 2.185 s] [INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.097 s] [INFO] Apache Hadoop Distribution ......................... SUCCESS [ 0.227 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 08:47 min [INFO] Finished at: 2016-01-14T09:05:59+08:00 [INFO] Final Memory: 96M/237M [INFO] ------------------------------------------------------------------------
执行编译
mvn package -Pdist,native -DskipTests -Dtar
编译成功
[INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop Main ................................. SUCCESS [ 6.143 s] [INFO] Apache Hadoop Project POM .......................... SUCCESS [ 3.923 s] [INFO] Apache Hadoop Annotations .......................... SUCCESS [ 6.345 s] [INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.466 s] [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 3.457 s] [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 8.004 s] [INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 6.740 s] [INFO] Apache Hadoop Auth ................................. SUCCESS [ 7.214 s] [INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 6.145 s] [INFO] Apache Hadoop Common ............................... SUCCESS [02:51 min] [INFO] Apache Hadoop NFS .................................. SUCCESS [ 26.889 s] [INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.107 s] [INFO] Apache Hadoop HDFS ................................. SUCCESS [04:44 min] [INFO] Apache Hadoop HttpFS ............................... SUCCESS [04:57 min] [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [01:01 min] [INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 15.992 s] [INFO] Apache Hadoop HDFS Project ......................... SUCCESS [ 0.307 s] [INFO] hadoop-yarn ........................................ SUCCESS [ 0.160 s] [INFO] hadoop-yarn-api .................................... SUCCESS [02:30 min] [INFO] hadoop-yarn-common ................................. SUCCESS [01:06 min] [INFO] hadoop-yarn-server ................................. SUCCESS [ 0.111 s] [INFO] hadoop-yarn-server-common .......................... SUCCESS [ 27.082 s] [INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [ 40.111 s] [INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [ 9.432 s] [INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [ 15.236 s] [INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 38.984 s] [INFO] hadoop-yarn-server-tests ........................... SUCCESS [ 0.889 s] [INFO] hadoop-yarn-client ................................. SUCCESS [ 13.748 s] [INFO] hadoop-yarn-applications ........................... SUCCESS [ 0.165 s] [INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [ 7.001 s] [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [ 4.640 s] [INFO] hadoop-yarn-site ................................... SUCCESS [ 0.177 s] [INFO] hadoop-yarn-project ................................ SUCCESS [ 13.099 s] [INFO] hadoop-mapreduce-client ............................ SUCCESS [ 0.141 s] [INFO] hadoop-mapreduce-client-core ....................... SUCCESS [ 39.011 s] [INFO] hadoop-mapreduce-client-common ..................... SUCCESS [ 33.940 s] [INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [ 7.769 s] [INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 18.003 s] [INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 16.896 s] [INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [ 7.284 s] [INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [ 3.377 s] [INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 11.164 s] [INFO] hadoop-mapreduce ................................... SUCCESS [ 5.954 s] [INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 8.246 s] [INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 14.047 s] [INFO] Apache Hadoop Archives ............................. SUCCESS [ 3.657 s] [INFO] Apache Hadoop Rumen ................................ SUCCESS [ 12.067 s] [INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 8.998 s] [INFO] Apache Hadoop Data Join ............................ SUCCESS [ 5.860 s] [INFO] Apache Hadoop Extras ............................... SUCCESS [ 5.582 s] [INFO] Apache Hadoop Pipes ................................ SUCCESS [ 11.003 s] [INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 11.071 s] [INFO] Apache Hadoop Client ............................... SUCCESS [ 16.352 s] [INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 0.241 s] [INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 8.179 s] [INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 7.388 s] [INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.048 s] [INFO] Apache Hadoop Distribution ......................... SUCCESS [01:07 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 27:05 min [INFO] Finished at: 2016-01-14T09:56:35+08:00 [INFO] Final Memory: 77M/237M [INFO] ------------------------------------------------------------------------
检查编译的本地文件,完成编译
标签:
原文地址:http://www.cnblogs.com/xizhunet/p/5136569.html