码迷,mamicode.com
首页 > 其他好文 > 详细

MapReduce单元测试

时间:2017-07-25 19:53:25      阅读:127      评论:0      收藏:0      [点我收藏+]

标签:tput   input   span   output   throws   reduce   inpu   exception   hadoop2   

MapReduce进行单元测试的步骤:

1. 在POM中添加MRUnit

<dependency>
            <groupId>org.apache.mrunit</groupId>
            <artifactId>mrunit</artifactId>
            <version>1.1.0</version>
            <classifier>hadoop2</classifier>
            <scope>test</scope>
 </dependency>

2. Mapper测试用例

    @Test
    public void testCountMapper() throws IOException {
        LongWritable key = new LongWritable(0);
        Text value = new Text("hadoop yarn");
        new MapDriver<LongWritable,Text,Text,IntWritable>()
                .withMapper(new WordCountMapper())
                .withInput(key,value)
                .withOutput(new Text("hadoop"),new IntWritable(1))
                .withOutput(new Text("yarn"),new IntWritable(1))
                .runTest();
    }

 3. Reducer测试用例

    @Test
    public void testCountReducer() throws IOException {
        new ReduceDriver<Text,IntWritable,Text,IntWritable>()
                .withReducer(new WordCountReducer())
                .withInput(new Text("hadoop"), Arrays.asList(new IntWritable(1),new IntWritable(1)))
                .withOutput(new Text("hadoop"),new IntWritable(2))
                .runTest();
    }

 

MapReduce单元测试

标签:tput   input   span   output   throws   reduce   inpu   exception   hadoop2   

原文地址:http://www.cnblogs.com/xdlaoliu/p/7235675.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!