码迷,mamicode.com
首页 > 数据库 > 详细

SparkSQL demo

时间:2017-12-26 19:56:57      阅读:205      评论:0      收藏:0      [点我收藏+]

标签:reg   imp   sparksql   each   sele   rip   beijing   lda   order by   

1、数据样本:data1.txt

xiaoming,25,chengdu
xiaohua,23,beijing
liuyang,16,hangzhou
xiaoqiang,19,zhejiang

2、demo例子

package com.test.sparksql

import org.apache.spark.sql.SQLContext
import org.apache.spark.{SparkConf, SparkContext}

/**
  * Created by admin on 2017/12/26-16:45.
  * Description : 
  */
object SparkSqlApplication {

  case class Person(name:String, age:Int, addr:String)

  def main(args: Array[String]): Unit = {
    val conf = new SparkConf().setAppName("Spark Sql Test").setMaster("local")
    val sc = new SparkContext(conf)
    val sqlContext = new SQLContext(sc)

    import sqlContext._
    import sqlContext.implicits._

    val people = sc.textFile("D:\\SparkSqlDataTest\\data1.txt").map(_.split(",")).map( p => Person(p(0),p(1).trim.toInt,p(2))).toDF()

    people.registerTempTable("people")

    val teenagers = sql("SELECT name, age, addr FROM people ORDER BY age")

    teenagers.map( t => "name:" + t(0) + " age:" + t(1) + " addr:" + t(2)).collect().foreach(println)

    sc.stop();
  }
}

 

3、输出结果

name:liuyang age:16 addr:hangzhou
name:xiaoqiang age:19 addr:zhejiang
name:xiaohua age:23 addr:beijing
name:xiaoming age:25 addr:chengdu

 

SparkSQL demo

标签:reg   imp   sparksql   each   sele   rip   beijing   lda   order by   

原文地址:https://www.cnblogs.com/xubiao/p/8119521.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!