码迷,mamicode.com
首页 > 其他好文 > 详细

【原创】大叔经验分享(100)Atlas导入hive元数据

时间:2020-01-11 16:58:15      阅读:411      评论:0      收藏:0      [点我收藏+]

标签:http   under   XML   data   otf   password   info   mod   mon   

首先要有HIVE_HOME环境变量,

如果是apache,直接配置为解压目录;如果是CDH,设置如下:

# export HIVE_HOME=/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hive

执行导入

# bin/import-hive.sh
...
Failed to import Hive Meta Data!!!

报错,查看日志

# more logs/import-hive.log 
2020-01-11 14:42:38,951 INFO  - [main:] ~ Looking for atlas-application.properties in classpath (ApplicationProperties:110)
2020-01-11 14:42:38,955 INFO  - [main:] ~ Looking for /atlas-application.properties in classpath (ApplicationProperties:115)
2020-01-11 14:42:38,956 INFO  - [main:] ~ Loading atlas-application.properties from null (ApplicationProperties:123)
2020-01-11 14:42:38,984 ERROR - [main:] ~ Import failed (HiveMetaStoreBridge:176)
org.apache.atlas.AtlasException: Failed to load application properties
    at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:134)
    at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:86)
    at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:120)
Caused by: org.apache.commons.configuration.ConfigurationException: Cannot locate configuration source null
    at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:259)
    at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:238)
    at org.apache.commons.configuration.AbstractFileConfiguration.<init>(AbstractFileConfiguration.java:197)
    at org.apache.commons.configuration.PropertiesConfiguration.<init>(PropertiesConfiguration.java:284)
    at org.apache.atlas.ApplicationProperties.<init>(ApplicationProperties.java:69)
    at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:125)
    ... 2 more

提示找不到atlas-application.properties,将其拷贝到hive conf目录

# cp conf/atlas-application.properties /etc/hive/conf/

再次执行

# bin/import-hive.sh
...
Enter username for atlas :- admin
Enter password for atlas :- 
Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/jaxrs/json/JacksonJaxbJsonProvider
    at org.apache.atlas.AtlasBaseClient.getClient(AtlasBaseClient.java:270)
    at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:453)
    at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:448)
    at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:132)
    at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:82)
    at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:131)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 6 more
Failed to import Hive Meta Data!!!

还是报错,提示找不到类,从server目录下拷贝到hook/hive目录下

# cp server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-1.8.3.jar hook/hive/atlas-hive-plugin-impl/
# cp server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.2.jar hook/hive/atlas-hive-plugin-impl/
# cp server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.8.jar hook/hive/atlas-hive-plugin-impl/

再次执行成功,到atlas里可以看到hive相关数据

技术图片

 

导入成功

【原创】大叔经验分享(100)Atlas导入hive元数据

标签:http   under   XML   data   otf   password   info   mod   mon   

原文地址:https://www.cnblogs.com/barneywill/p/12180277.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!