码迷,mamicode.com
首页 > 系统相关 > 详细

Ubuntu 14.10 下Hive配置

时间:2015-08-26 22:16:46      阅读:290      评论:0      收藏:0      [点我收藏+]

标签:

1 系统环境

Ubuntu 14.10 

JDK-7

Hadoop 2.6.0

2 安装步骤

2.1 下载Hive

我第一次安装的时候,下载的是Hive-1.2.1,配置好之后,总是报错

[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
    at jline.TerminalFactory.create(TerminalFactory.java:101)
    at jline.TerminalFactory.get(TerminalFactory.java:158)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
    at org.apache.hadoop.hive.cli.CliDriver.getConsoleReader(CliDriver.java:773)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:715)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:230)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
    at org.apache.hadoop.hive.cli.CliDriver.getConsoleReader(CliDriver.java:773)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:715)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

对于这个错误,很多人的做法都是将hadoop/share/hadoop/yarn/lib中的jline 替换成高版本的,如hive中的2.×

1.Delete jline from the Hadoop lib directory (it‘s only pulled in transitively from ZooKeeper).
2.export HADOOP_USER_CLASSPATH_FIRST=true

但是我替换之后,还是报错,后来换了一个低版本的HIve-1.0.1,就好了,所以如果出现如上错误,不一定非的是jar问题,也可能是版本问题。

下载地址:http://mirrors.hust.edu.cn/apache/hive/hive-1.0.1/apache-hive-1.0.1-bin.tar.gz

2.2 配置hive

2.2.1 拷贝一份hive-env.sh,设置HADOOP_HOME

2.2.2 拷贝一份hive-site.xml

主要修改如下参数

<property> 
   <name>javax.jdo.option.ConnectionURL </name> 
   <value>jdbc:mysql://localhost:3306/hive </value> 
</property> 
 
<property> 
   <name>javax.jdo.option.ConnectionDriverName </name> 
   <value>com.mysql.jdbc.Driver </value> 
</property>

<property> 
   <name>javax.jdo.option.ConnectionPassword </name> 
   <value>hive </value> 
</property> 
 
<property> 
   <name>hive.hwi.listen.port </name> 
   <value>9999 </value> 
   <description>This is the port the Hive Web Interface will listen on </descript ion> 
</property> 

<property> 
   <name>datanucleus.autoCreateSchema </name> 
   <value>true</value> 
</property> 
 
<property> 
   <name>datanucleus.fixedDatastore </name> 
   <value>false</value> 
</property> 
</property> 

  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>Username to use against metastore database</description>
  </property>

  <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/home/hdpsrc/hive/iotmp</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/hdpsrc/hive/iotmp</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>
  <property>
    <name>hive.querylog.location</name>
    <value>/home/hdpsrc/hive/iotmp</value>
    <description>Location of Hive run time structured log file</description>
  </property>

2.3 拷贝一份hive-log4j.properties

这里我只是设置了log文件的路径

2.4 建立相关文件夹,就是在2.2中设置的

2.5 在HDFS中建立相关文件夹

  $ $HADOOP_HOME/bin/hadoop fs -mkdir       /tmp
  $ $HADOOP_HOME/bin/hadoop fs -mkdir       /user/hive/warehouse
  $ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /tmp
  $ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /user/hive/warehouse

 

2.3 安装MySQL

因为这里的关系型数据库是用的mysql,那么需要安装mysql,同时按照上面配置的新建hive用户,新建数据库。

安装mysql参考:http://www.cnblogs.com/liuchangchun/p/4099003.html

新建用户参考:http://www.cnblogs.com/liuchangchun/p/4431426.html

因为之前安装mysql后将编码设置成了utf8的,在启动hive的时候报了如下错:

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes

这个错误,需要修改hive数据库的编码:

alter database hive character set latin1;

2.4 拷贝mysql的驱动jar包到hive/lib下面

如果不拷贝,那么会报错:找不到jar包

Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
    at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58)
    at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
    at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:238)
    ... 67 more

 

2.5 启动hive

启动之前要确保hdfs已经起来。

在命令行执行:hive,即可进入hive命令行界面,如果没有错误,恭喜你,OK了。

如果HDFS没起来,那么可能报错.。。

 

参考:

https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-InstallationandConfiguration

http://blog.csdn.net/an342647823/article/details/46048403

http://blog.csdn.net/keljony/article/details/43371995

http://sunjia-704471770-qq-com.iteye.com/blog/1631430

http://blog.csdn.net/jdplus/article/details/46493553

 

Ubuntu 14.10 下Hive配置

标签:

原文地址:http://www.cnblogs.com/liuchangchun/p/4761730.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!