码迷,mamicode.com
首页 > 其他好文 > 详细

storm安装配置

时间:2016-05-07 10:50:18      阅读:247      评论:0      收藏:0      [点我收藏+]

标签:

系统环境

  • Cent OS 6.5
  • hadoop-2.5.0
  • zookeeper-3.4.5
  • storm-0.9.2


tar zxvf apache-storm-0.9.2-incubating.tar.gz -C /home/hadoop/hadoop/src
[hadoop@nameNode src]$ ls -a
.  ..  flume-1.6.0  hadoop-2.5.0  hive-0.12.0-bin  hive-2.0.0  kafka-2.9.2  sqoop-1.4.6.bin  <span style="color:#ff0000;">storm-0.9.2-incubating</span>  zookeeper-3.4.5  zookeeper.out


  1. 配置    storm.yaml
    1. cd ./storm-0.9.2-incubating/conf/

      主要修改配置

      storm.zookeeper.servers:
           - "nameNode"  配置zookeeper
           - "dataNode1"
           - "dataNode2"

       nimbus.host: "nameNode" 配置nimbus主机地址

  2. 分配到子节点
    1. scp -r /home/hadoop/hadoop/src/storm-0.9.2-incubating hadoop@dataNode1:/home/hadoop/hadoop/src/
    2. scp -r /home/hadoop/hadoop/src/storm-0.9.2-incubating hadoop@dataNode2:/home/hadoop/hadoop/src/
  3. 启动Storm测试
    1. 在主节点 nimbus
      1. [hadoop@nameNode storm-0.9.2-incubating]$ ./bin/storm nimbus
        [hadoop@nameNode storm-0.9.2-incubating]$	./bin/storm ui  WEB页面服务



    2. 启动从节点 supervisor
      1. [hadoop@dataNode1 storm-0.9.2-incubating]$ 	./bin/storm supervisor 
        [hadoop@dataNode2 storm-0.9.2-incubating]$ 	./bin/storm supervisor 
  1. 执行代码测试Storm集群
  代码:链接:http://share.weiyun.com/bf3cd590b3962d9052dbaadb2c6d41fb (密码:f7NT)
将jar上传到主节点 nameNode nimbus主节点上
sftp> put D:/stormchone.jar .
Uploading stormchone.jar to /home/hadoop/hadoop/data/stormchone.jar
Skipping directory d:/Documents
  100% 5KB      5KB/s 00:00:00     
D:/stormchone.jar: 5149 bytes transferred in 0 seconds (5149 bytes/s)

调用代码 JAR
[hadoop@<span style="color:#ff0000;">nameNode</span> storm-0.9.2-incubating]$<span style="color:#ff0000;"> ./bin/storm jar /home/hadoop/hadoop/data/stormchone.jar cn.canbot.storm.RunMain</span>
Running: /usr/java/jdk1.8.0_73/bin/java -client -Dstorm.options= -Dstorm.home=/home/hadoop/hadoop/src/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/hadoop/data/stormchone.jar:/home/hadoop/hadoop/src/storm-0.9.2-incubating/conf:/home/hadoop/hadoop/src/storm-0.9.2-incubating/bin -Dstorm.jar=/home/hadoop/hadoop/data/stormchone.jar cn.canbot.storm.RunMain
836  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
861  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar /home/hadoop/hadoop/data/stormchone.jar to assigned location: storm-local/nimbus/inbox/stormjar-39e6739a-9741-4443-8e9f-f3d1693d976a.jar
872  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: storm-local/nimbus/inbox/stormjar-39e6739a-9741-4443-8e9f-f3d1693d976a.jar
873  [main] INFO  backtype.storm.StormSubmitter - Submitting topology demotopo in distributed mode with conf {"topology.acker.executors":0,"topology.workers":4,"topology.debug":true}
952  [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: demotopo
当启动Storm以后由于代码中设置 woker 数量为4个 ,Storm内置负载均衡 我这里Storm子节点有 两台。
JAVA源代码中设置
fileWriter=new FileWriter("/home/hadoop/hadoop/tmp/storm/data/"+UUID.randomUUID());
<span style="white-space:pre"></span><pre name="code" class="java" style="font-size: 14px; line-height: 21px;"><span style="background-color: rgb(255, 0, 0);">/home/hadoop/hadoop/tmp/storm/data/  </span><span style="background-color: rgb(255, 255, 255);"> 指定数据写入地址</span>
<span style="background-color: rgb(255, 255, 255);">动态查看 Storm 写入数据</span>
[hadoop@dataNode1 storm]$ tail -f datace85f944-3586-499e-9cd7-52d20409290a 
</pre><pre>

Storm集群配置
待更新····












storm安装配置

标签:

原文地址:http://blog.csdn.net/qq_26840065/article/details/51333702

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!