标签:style blog java color os art
1. 打开spark-shell
2. 建立StreamingContext
import org.apache.spark.streaming._
import org.apache.spark.streaming.StreamingContext._
import org.apache.spark.api.java.function._ import org.apache.spark.streaming._ import org.apache.spark.streaming.api._ // Create a StreamingContext with a local master val ssc = new StreamingContext(sc, Seconds(1))
3.用StreamingContext建立一个DStream
val lines = ssc.socketTextStream("localhost", 9999)
4.运算
// Split each line into words val words = lines.flatMap(_.split(" ")) import org.apache.spark.streaming.StreamingContext._ // Count each word in each batch val pairs = words.map(word => (word, 1)) val wordCounts = pairs.reduceByKey(_ + _) // Print a few of the counts to the console wordCounts.print()
5.启动DStream
ssc.start() // Start the computation ssc.awaitTermination() // Wait for the computation to terminate
6.启动一个新控制台,并执行如下命令
$ nc -lk 9999
7.在新控制台上输入任意语句,并在spark-shell窗口观察输出
hello world how are you hello are you there
you are the son of a bitch
Spark Streaming之旅,布布扣,bubuko.com
标签:style blog java color os art
原文地址:http://www.cnblogs.com/littlesuccess/p/3794225.html