标签:
SparkSQL是指整合了Hive的spark-sql cli, 本质上就是通过Hive访问HBase表,具体就是通过hive-hbase-handler, 具体配置参见:Hive(五):hive与hbase整合
目录:
SparkSql 访问 hbase配置:
guava-14.0.1.jar
htrace-core-3.1.0-incubating.jar
hbase-common-1.1.2.2.4.2.0-258.jar
hbase-common-1.1.2.2.4.2.0-258-tests.jar
hbase-client-1.1.2.2.4.2.0-258.jar
hbase-server-1.1.2.2.4.2.0-258.jar
hbase-protocol-1.1.2.2.4.2.0-258.jar
hive-hbase-handler-1.2.1000.2.4.2.0-258.jar
export SPARK_CLASSPATH=/usr/hdp/2.4.2.0-258/spark/lib/guava-11.0.2.jar: /usr/hdp/2.4.2.0-258/spark/lib/hbase-client-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/spark/lib/hbase-common-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/spark/lib/hbase-protocol-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/spark/lib/hbase-server-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/spark/lib/hive-hbase-handler-1.2.1000.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/spark/lib/htrace-core-3.1.0-incubating.jar: /usr/hdp/2.4.2.0-258/spark/lib/protobuf-java-2.5.0.jar:${SPARK_CLASSPATH}
<property>
<name>hbase.zookeeper.quorum</name>
<value>r,hdp2,hdp3</value>
<description>HBase使用的zookeeper节点</description>
</property>
<property>
<name>hbase.client.scanner.caching</name>
<value>100</value>
<description>HBase客户端扫描缓存,对查询性能有很大帮助</description>
</property>
测试验证:
标签:
原文地址:http://www.cnblogs.com/tgzhu/p/5829242.html