[Spark][Python][DataFrame][SQL]Spark对DataFrame直接执行SQL处理的例子

标签：data   table   ext   直接   dfs   --   car   alice   led   

[Spark][Python][DataFrame][SQL]Spark对DataFrame直接执行SQL处理的例子

$cat people.json

{"name":"Alice","pcode":"94304"}
{"name":"Brayden","age":30,"pcode":"94304"}
{"name":"Carla","age":19,"pcoe":"10036"}
{"name":"Diana","age":46}
{"name":"Etienne","pcode":"94104"}

$ hdfs dfs -put people.json

$pyspark

sqlContext = HiveContext(sc)
peopleDF = sqlContext.read.json("people.json")

peopleDF.registerTempTable("people")

tmpDF=sqlContext.sql(""" select * FROM people WHERE name like "A%" """)

tmpDF.limit(3).show()

+----+-----+-----+----+
| age| name|pcode|pcoe|
+----+-----+-----+----+
|null|Alice|94304|null|
+----+-----+-----+----+

[Spark][Python][DataFrame][SQL]Spark对DataFrame直接执行SQL处理的例子

标签：data   table   ext   直接   dfs   --   car   alice   led   

原文地址：http://www.cnblogs.com/gaojian/p/7634267.html