Pig环境安装
pig的安装非常简单,将pig-0.14.0.tar.gz解压到合适的目录下
tar -zxvf pig-0.14.0.tar.gz
修改环境变量:
#pig export PIG_HOME=/usr/local/cloud/pig-0.11.1/pig-0.11.1 export PATH=.:$PIG_HOME/bin:$PATH export PIG_CLASSPATH=$HADOOP_HOME/etc/hadoop #为了让pig识别你的hadoop版本,如果只想用pig的本地模式,不用配置这项。使环境变量修改生效:
本地模式启动方式pig -x local
[root@leaf pig-0.11.1]# pig -x local 2014-11-24 07:50:19,622 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53 2014-11-24 07:50:19,622 [main] INFO org.apache.pig.Main - Logging error messages to: /usr/local/cloud/pig-0.11.1/pig-0.11.1/logs/pig_1416844219621.log 2014-11-24 07:50:19,663 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /root/.pigbootup not found 2014-11-24 07:50:19,901 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:/// 2014-11-24 07:50:19,903 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used 2014-11-24 07:50:19,907 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS 2014-11-24 07:50:19,907 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2014-11-24 07:50:20,188 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 2014-11-24 07:50:20,190 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS修改pig默认的日志存放目录
pig.logfile=/usr/local/cloud/pig-0.11.1/pig-0.11.1/logs
[root@leaf pig-0.11.1]# pig 2014-11-24 07:57:16,370 [main] INFO org.apache.pig.Main - Apache Pig version <span style="font-family: Arial, Helvetica, sans-serif;">0.11.1</span><span style="font-family: Arial, Helvetica, sans-serif;">(r1459641) compiled Mar 22 2013, 02:13:53</span> 2014-11-24 07:57:16,370 [main] INFO org.apache.pig.Main - Logging error messages to: /usr/local/cloud/pig-0.11.1/pig-0.11.1/logs/pig_1416844636369.log 2014-11-24 07:57:16,410 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /root/.pigbootup not found 2014-11-24 07:57:16,681 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://192.168.1.240:9000 2014-11-24 07:57:16,684 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used 2014-11-24 07:57:16,685 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS 2014-11-24 07:57:17,634 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS测试是否安装成功:
grunt> ls hdfs:/// hdfs://192.168.1.240:9000/source <dir> hdfs://192.168.1.240:9000/testdata <dir> hdfs://192.168.1.240:9000/tmp <dir> hdfs://192.168.1.240:9000/user <dir> hdfs://192.168.1.240:9000/usr <dir>pig常见命令:
grunt> cd /user/root/output
grunt> ls hdfs://192.168.1.240:9000/user/root/output/_policy<r 1> 194 hdfs://192.168.1.240:9000/user/root/output/clusteredPoints <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-0 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-1 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-10-final <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-2 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-3 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-4 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-5 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-6 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-7 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-8 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-9 <dir> hdfs://192.168.1.240:9000/user/root/output/data <dir> hdfs://192.168.1.240:9000/user/root/output/random-seeds <dir>
grunt> pig -help 2014-11-24 08:04:11,969 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Encountered " <IDENTIFIER> "pig "" at line 1, column 1. Was expecting one of: <EOF> "cat" ... "clear" ... "fs" ... "sh" ... "cd" ... "cp" ... "copyFromLocal" ... "copyToLocal" ... "dump" ... "describe" ... "aliases" ... "explain" ... "help" ... "history" ... "kill" ... "ls" ... "mv" ... "mkdir" ... "pwd" ... "quit" ... "register" ... "rm" ... "rmf" ... "set" ... "illustrate" ... "run" ... "exec" ... "scriptDone" ... "" ... "" ... <EOL> ... ";" ...
原文地址:http://blog.csdn.net/u013361361/article/details/41458903