标签:des style blog http io ar color os 使用
引言
Oozie是Hadoop生态系统中的工作流控制组件,可以调度MR,Hive,Shell等各类任务,并配置复杂的工作流程。
今天介绍如何安装部署,使用的是单机测试环境。
欢迎转载,请注明来源:
http://blog.csdn.net/u010967382/article/details/41355953
下载解压
下载安装过程需要的相关文件:
oozie4.0.0安装包
ext2.2包
存在在/home/casliyang/Downloads/ext-2.2.zip
将oozie4.0.0安装包解压拷贝到:
/home/casliyang/oozie/oozie-4.0.0-cdh5.2.0
这也也是即将配置环境变量的OOZIE_HOME
配置环境变量
向/etc/profile中加入如下三行,注意加完回到原用户后执行source /etc/profile
export OOZIE_HOME=/home/casliyang/oozie/oozie-4.0.0-cdh5.2.0
export OOZIE_CONFIG=$OOZIE_HOME/conf
export CLASSPATH=$CLASSPATH:$OOZIE_HOME/bin
将ext2.2.0和hadoop相关jar包加入war包,并发布到自带的web server下
在$OOZIE_HOME/bin目录下执行
./addtowar.sh -inputwar $OOZIE_HOME/oozie.war -outputwar $OOZIE_HOME/oozie-server/webapps/oozie.war -hadoop 2.5.0 $HADOOP_HOME -extjs /home/casliyang/Downloads/ext-2.2.zip
生成oozie数据库脚本并执行
这里使用的是自带的derby数据库
在$OOZIE_HOME/bin目录下
执行以下命令创建数据库脚本
./ooziedb.sh create -sqlfile oozie.sql
通过以下命令执行sql脚本初始化数据库
./oozie-setup.sh db create -run -sqlfile $OOZIE_HOME/bin/oozie.sql
配置oozie
配置$OOZIE_HOME/conf/oozie-site.xml
修改以下配置项,将值指向真实hadoop配置文件路径,实现oozie和hadoop的关联:
<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/home/casliyang/hadoop2/hadoop-2.5.0-cdh5.2.0/etc/hadoop</value>
<description>
Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
the Hadoop service (JobTracker, HDFS). The wildcard ‘*‘ configuration is
used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
the relevant Hadoop *-site.xml files. If the path is relative is looked within
the Oozie configuration directory; though the path can be absolute (i.e. to point
to Hadoop client conf/ directories in the local filesystem.
</description>
</property>
配置$OOZIE_HOME/conf/oozie-default.xml
将oozie.services属性中的org.apache.oozie.service.JobsConcurrencyService值提到第一个位置。
HDFS上创建共享目录
在$HADOOP_HOME/etc/hadoop/core-site.xml中加入以下配置,红色的casliyang为用户名,蓝色的casliyang为组名,singlehadoop为oozie的host(注意不能用localhost,会报错):
<!-- for oozie,add by casliyang,2014-11-21 -->
<property>
<name>hadoop.proxyuser.casliyang.hosts</name>
<value>singlehadoop</value>
</property>
<property>
<name>hadoop.proxyuser.casliyang.groups</name>
<value>casliyang</value>
</property>
启动HADOOP,该步骤不详述。
在$OOZIE_HOME/bin目录下执行:
casliyang@singlehadoop:~/oozie/oozie-4.0.0-cdh5.2.0/bin$ ./oozie-setup.sh sharelib create -fs hdfs://singlehadoop:8020 -locallib $OOZIE_HOME/oozie-sharelib-4.0.0-cdh5.2.0-yarn.tar.gz
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/casliyang/oozie/oozie-4.0.0-cdh5.2.0/libtools/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/casliyang/oozie/oozie-4.0.0-cdh5.2.0/libtools/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
the destination path for sharelib is: /user/casliyang/share/lib/lib_20141121184647
启动oozie
在$OOZIE_HOME/bin目录下
前台运行oozie:
./oozied.sh run
后台运行oozie:
./oozied.sh start
查看oozie网页:
停止oozie:
./oozied.sh stop
【甘道夫】Oozie-4.0.0-CDH5.2.0 安装部署
标签:des style blog http io ar color os 使用
原文地址:http://blog.csdn.net/u010967382/article/details/41355953