标签:
上次实现了flume+kafka+hbase+ELK:http://www.cnblogs.com/super-d2/p/5486739.html
这次我们可以加上storm:
storm-0.9.5简单配置如下:
安装依赖
wget http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.tar.gz tar zxvf jdk-8u45-linux-x64.tar.gz cd jdk-8u45-linux-x64 sudo vi /etc/profile 添加如下内容: export JAVA_HOME=/home/dir/jdk1.8.0_45 export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export PATH=$PATH:$JAVA_HOME/bin
安装 Python
sudo apt-get install python
安装 zookeeper
wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz tar -xzvf zookeeper-3.4.6.tar.gz chown -R root:root zookeeper-3.4.6 cd zookeeper-3.4.6 cp conf/zoo_sample.cfg conf/zoo.cfg sudo vi /etc/profile 添加如下内容: export ZOOKEEPER_HOME=/home/dir/Downloads/zookeeper-3.4.6 export PATH=$ZOOKEEPER_HOME/bin:$PATH
安装storm
wget http://www.apache.org/dyn/closer.cgi/storm/apache-storm-0.9.5/apache-storm-0.9.5.tar.gz tar zxvf apache-storm-0.9.5.tar.gz cd apache-storm-0.9.5 sudo vi /etc/profile 添加如下内容: export STORM_HOME=/home/dir/Downloads/apache-storm-0.9.5 export PATH=$STORM_HOME/bin:$PATH
使环境变量生效
source /etc/profile
修改storm配置
vi conf/storm.yaml 修改如下: storm.zookeeper.servers: - "127.0.0.1" # - "server2" storm.zookeeper.port: 2181 //zookeeper端口默认就是 2181 nimbus.host: "127.0.0.1" # # storm.local.dir: "/home/dir/storm" ui.port: 8088
启动storm
先要启动 zookeeper
zkServer.sh start 启动 nimbus storm nimbus >/dev/null 2>&1 & 启动 supervisor storm supervisor >/dev/null 2>&1 & 启动 ui storm ui >/dev/null 2>&1 & jps 命令可以看到 29929 nimbus 17754 core 20714 Jps 30222 supervisor ui 启动后 可以在浏览器 查看结果 localhost:8088/index.html
运行 wordcount example
cd examples/storm-starter 提交任务 storm jar storm-starter-topologies-0.9.5.jar storm.starter.WordCountTopology wordcountTop 杀掉topology命令 storm kill wordcountTop
大数据平台架构(flume+kafka+hbase+ELK+storm+redis+mysql)
标签:
原文地址:http://www.cnblogs.com/super-d2/p/5663044.html