标签：大数据平台
在网上看过的安装教程真的是有点心酸呢。好多都失败了。
分享下，自己实验成功可以用的博文供大家参考：
推荐1给力星：http://www.powerxing.com/install-hadoop/ hadoop+spark 完全参考的他的博文，相当给力推荐指数5个星
hive的话，我这边参考的文章太多了。没个能用的，不知道是我自己弄的不对还是怎么回事.
.....
最后无意间看到hive变成指南这本书，里面的教程安装成功的。
root@iZ254fu6ocuZ:/usr/local/hadoop/hive# hive
hive>
root@iZ254fu6ocuZ:/usr/local/hadoop/hive# jps
8100 ResourceManager
8533 JobHistoryServer
7709 SecondaryNameNode
18406 Jps
7514 DataNode
7410 NameNode
8204 NodeManager
root@iZ254fu6ocuZ:/usr/local/hadoop/hive#
root@iZ254fu6ocuZ:/usr/local/spark# ./bin/spark-shell
scala>
文档太多了，我就分享下我初次安装所踩的坑吧：
不得不说的就是环境变量，踩了好多， 例如路径明明是/usr/local 写成/usr/lacol
启动程序时总报莫名奇妙的错误，这种错误，看到此处的同学也都注意下吧，很坑的。
还有就网上很多配置文件中，有些是需要改成你自己的主机名的，别粘贴过来了，不改参数。
我在分享几个我用过的一些包的链接吧；
wget http://www.eu.apache.org/dist/hive/hive-1.1.1/apache-hive-1.1.1-bin.tar.gz hive
wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.39.tar.gz hive连接JDBC时依赖的包
wget http://archive.apache.org/dist/spark/spark-1.6.0/spark-1.6.0-bin-without-hadoop.tgz spark
wget http://mirrors.cnnic.cn/apache/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz hadoop
hive的本地模式分享： 已实验，无问题：hive的安装包是1.1.1
主要就是一个配置文件，本地模式不用修改特别负责。只需要配置入下内容即可，
目的就是为了，防止每次执行hive命令时元数据存储都在不同的目录下。
root@iZ254fu6ocuZ:/usr/local/hadoop/hive/conf# cat hive-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/usr/local/hadoop/hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<!--表示使用嵌入式的derby，create为true表示自动创建数据库，数据库名为metastore_db-->
<value>jdbc:derby:;databaseName=/usr/local/hadoop/hive/metastore_db;create=true</value>
<!--表示使用客服模式的derby，hadoopor为数据库名，192.168.0.3为derby服务端的IP地址，而4567为服务端的端口号-->
<!--<value>jdbc:derby://192.168.0.3:4567/hadoopor;create=true</value>-->
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>org.apache.derby.jdbc.EmbeddedDriver</value>
<!--<value>org.apache.derby.jdbc.ClientDriver</value>-->
<description>Driver class name for a JDBC metastore</description>
</property>
</configuration>
root@iZ254fu6ocuZ:/usr/local/hadoop/hive/conf#
使用JDBC管理元数据
需要有一台有mysql的服务器：我是另外准备了一台进行测试
hadoop@ubuntu:~$ mysql -uroot -pmysql mysql> CREATE USER ‘hive‘ IDENTIFIED BY ‘hive‘; mysql> GRANT ALL PRIVILEGES ON *.* TO ‘hive‘@‘%‘ WITH GRANT OPTION; mysql> flush privileges;
使用JDBC管理元数据需要准备JDBC的驱动，上面已经提供链接可以使用：
将mv mysql-connector-java-5.1.39/mysql-connector-java-5.1.39-bin.jar /usr/local/hadoop/hive/lib/
将上面的hive-site.xml备份，重新编写该文件：
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://120.27.7.76/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>
</configuration>
root@iZ254fu6ocuZ:/usr/local/hadoop/hive/conf#
root@iZ254fu6ocuZ:~# hive
hive>
Ubuntu + hadoop 2.7 + hive 1.1.1 + sprk 初装成功分享 有什么问题大家一起讨论
标签：大数据平台
原文地址：http://bjzby.blog.51cto.com/4084070/1826083
