Building hive from source

clone and build from source

downlaod

git clone https://git-wip-us.apache.org/repos/asf/hive.git
or 
git clone git@github.com:wankunde/hive.git

build

git branch -va
// checkout the branch which you are intrest.
git checkout -b branch-0.14 origin/branch-0.14
// or git checkout --track origin/branch-0.14

// compile and dist
mvn clean install -DskipTests -Phadoop-2 -Pdist

//  generate protobuf code
cd ql
mvn clean install -DskipTests -Phadoop-2,protobuf

// generate Thrift code
mvn clean install -Phadoop-2,thriftif -DskipTests -Dthrift.home=/usr/local

Tips

• By default,before compile,maven will download many dependency packages and meet timeout exception.I use nexus and add <timeout>120000</timeout> configuration to <server> configuration item. Not Test
• Hive HiveDeveloperFAQ Wiki

Test Unit

Change hive log level

bin/hive -hiveconf hive.root.logger=DEBUG,console

Or change log4j properties.

cp conf/hive-log4j.properties.template conf/hive-log4j.properties

Connecting a Java Debugger to hive

Example java remote debug

• Run remote java program using script

JVM_OPTS="-server -XX:+UseParNewGC -XX:+HeapDumpOnOutOfMemoryError"
DEBUG="-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=2345"
JVM_OPTS="$JVM_OPTS $DEBUG"
$JAVA_HOME/bin/java $JVM_OPTS -cp tools-1.0.jar com.wankun.tools.hdfs.Test2

• Add and run new remote debug configuration in eclipse

Start hive debug

help command
hive --help --debug

Run hive without a hadoop cluster

export HIVE_OPTS=‘--hiveconf mapred.job.tracker=local --hiveconf fs.default.name=file:///tmp     --hiveconf hive.metastore.warehouse.dir=file:///tmp/warehouse     --hiveconf javax.jdo.option.ConnectionURL=jdbc:derby:;databaseName=/tmp/metastore_db;create=true‘

Hive test unit

Two kind of unit tests

• Normal unit test

mvn test -Dtest=ClassName#methodName -Phadoop-2

For example,
mvn test -Dtest=TestAbc -Phadoop-2 which TestAbc is the test case.
mvn test -Dtest=‘org.apache.hadoop.hive.ql.*‘ -Phadoop-2 .

Help Links : Maven Surefire Plugin

• Query files

There are many test scripts. Not successed

$ ls ql/src/test/queries/
clientcompare  clientnegative  clientpositive  negative  positive

// run test unit,ql as example
cd ql
mvn test -Dtest=TestCliDriver -Dqfile=groupby1.q -Phadoop-2

//Take src/test/queries/clientpositive/groupby1.q for example.

mvn test -Dmodule=ql -Phadoop-2 -Dtest=TestCliDriver -Dqfile=groupby1.q -Dtest.output.overwrite=true

Help Links1

Help Links2