码迷,mamicode.com
首页 > 其他好文 > 详细

Mapreduce部署与第三方依赖包管理

时间:2019-07-05 14:22:36      阅读:105      评论:0      收藏:0      [点我收藏+]

标签:dep   pre   include   mvn   pid   set   jar   coding   pen   

参考了几篇文章,总算搞定打包第三方jar了,记录一下

 

https://segmentfault.com/a/1190000016237395

https://www.cnblogs.com/Dhouse/p/6595344.html

https://yq.aliyun.com/articles/308777

https://blog.csdn.net/u013177446/article/details/54134583

 

pom.xml加入maven-assembly-plugin

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
	                <artifactId>maven-assembly-plugin</artifactId>
	                <configuration>
<!-- 	                    <descriptorRefs> -->
<!-- 	                        <descriptorRef>jar-with-dependencies</descriptorRef> -->
<!-- 	                    </descriptorRefs> -->
	                    <descriptors>
                            <descriptor>src/assembly.xml</descriptor>
                        </descriptors>
	                </configuration>
	                <executions>
		                <execution>
		                    <id>make-assembly</id>
		                    <phase>package</phase>
		                    <goals>
		                        <goal>single</goal>
		                    </goals>
		                </execution>
		            </executions>
	            </plugin>

 

src/assembly.xml指定需要打包哪些第三方包

<?xml version="1.0" encoding="UTF-8"?>
<assembly>
    <id>cascading</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
<!--             默认为添加为.class,指定为false,则以jar包的方式导入 -->
            <unpack>false</unpack>
            <scope>runtime</scope>
            <!--
            <excludes>
                <exclude>org.apache.hadoop:*</exclude>
            </excludes>
            -->
            <!-- very small jar -->
<!--             指定路径,放到lib目录下 -->
            <outputDirectory>lib</outputDirectory>
            <includes>
                <include>fastjson:*</include>
            </includes>
        </dependencySet>
    </dependencySets>
    <fileSets>
        <fileSet>
            <directory>${project.build.outputDirectory}</directory>
            <outputDirectory>/</outputDirectory>
        </fileSet>
    </fileSets>
</assembly>

 

打包

mvn assembly:assembly -Dmaven.test.skip=true

生成的jar包中,第三方jar包fastjson放在lib目录下。直接上传生成的jar到hadoop,可以直接运行

 

Mapreduce部署与第三方依赖包管理

标签:dep   pre   include   mvn   pid   set   jar   coding   pen   

原文地址:https://www.cnblogs.com/jerryzh/p/11137927.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!