第一次调用 Hadoop Java API

标签：

环境：Hadoop1.2.1

例：直接使用FileSystem以标准输出格式显示Hadoop文件系统中的文件
编写JAVA文件

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;

import java.io.InputStream;
import java.net.URI;

public class FileSystemCat {
    public static void main(String[] args) throws Exception {
        String uri = args[0];
        Configuration conf = new Configuration();
        FileSystem fs = FileSystem.get(URI.create(uri), conf);
        InputStream in = null;
        try {
            in = fs.open(new Path(uri));
            IOUtils.copyBytes(in, System.out, 4096, false);
        } finally {
            IOUtils.closeStream(in);
        }
    }
}

[grid@hadoop1 hadoop-1.2.1]# vi conf/hadoop-env.sh
# The java implementation to use.  Required.
export JAVA_HOME=/usr/java/jdk1.7.0_72

# Extra Java CLASSPATH elements.  Optional.
export HADOOP_CLASSPATH=/home/grid/hadoop-1.2.1/myclasses

上传编辑好的JAVA文件

[grid@hadoop1 hadoop-1.2.1]$ mkdir myclasses
[grid@hadoop1 hadoop-1.2.1]$ cd myclasses
[grid@hadoop1 myclasses]$ rz

编译java文件

[grid@hadoop1 myclasses]$ javac -classpath ../hadoop-core-1.2.1.jar  FileSystemCat.java

执行：

[grid@hadoop1 hadoop-1.2.1]$ ./bin/hadoop FileSystemCat hdfs://hadoop1:9000/user/grid/in/test3.txt
Hello Hadoop Java API

第一次调用 Hadoop Java API

标签：

原文地址：http://my.oschina.net/zc741520/blog/365318