码迷,mamicode.com
首页 > 其他好文 > 详细

[sqoop] sqoop命令

时间:2016-12-19 13:40:37      阅读:313      评论:0      收藏:0      [点我收藏+]

标签:关系   ade   tail   hadoop   配置   ssi   option   direction   数据导入   

 

官网文档:http://sqoop.apache.org/docs/1.99.7/user/CommandLineClient.html#delete-link-function

 

一、了解sqoop数据导入的几个重要概念

(1)connector:sqoop2中预定了各种链接,这些链接是一些配置模板。

#查看预定制的连接器

sqoop:000> show connector
+------------------------+---------+------------------------------------------------------------+----------------------+
|          Name          | Version |                           Class                            | Supported Directions |
+------------------------+---------+------------------------------------------------------------+----------------------+
| oracle-jdbc-connector  | 1.99.7  | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO              |
| sftp-connector         | 1.99.7  | org.apache.sqoop.connector.sftp.SftpConnector              | TO                   |
| kafka-connector        | 1.99.7  | org.apache.sqoop.connector.kafka.KafkaConnector            | TO                   |
| kite-connector         | 1.99.7  | org.apache.sqoop.connector.kite.KiteConnector              | FROM/TO              |
| ftp-connector          | 1.99.7  | org.apache.sqoop.connector.ftp.FtpConnector                | TO                   |
| hdfs-connector         | 1.99.7  | org.apache.sqoop.connector.hdfs.HdfsConnector              | FROM/TO              |
| generic-jdbc-connector | 1.99.7  | org.apache.sqoop.connector.jdbc.GenericJdbcConnector       | FROM/TO              |
+------------------------+---------+------------------------------------------------------------+----------------------+


其中,最基本的是generic-jdbc-connector,是mysql等关系型数据库的连接器。支持数据的从关系型数据库的导入导出。

除此之外,支持导入导出的还有:hdfs-connector、kite-connector和oracle-jdbc-connector。

仅支持数据导入的连接器有: sftp-connector 、 ftp-connector 以及kafka-connector 。


通过这些模板,可以创建出对应数据源的link,比如我们链接mysql,就是使用JDBC的方式进行链接,这时候就从这个generic-jdbc-connector模板继承出一个link。那如果我们需要连接hdfs,则需要使用hdfs-connector模版。


(2)link:从connector继承出的对象,用于指定的对数据源的链接。

(3)job:指定一个导入导出作业,必须指定数据源和数据目的地,并配置各项job参数,用于提交给mapreduce。

 

二、几个常用的重要命令

#设置
set [server|option|truststore]

#查看
show [server|version|connector|driver|link|job|submission|option|role|principal|privilege]

#创建
create [link|job|role]

#删除
delete [link|job|role]

#更新
update [link|job]

#克隆
clone [link|job]

#执行任务
start [job]

#结束任务
stop [job]

#查看任务状态
status [job]

#启用
enable [link|job]

#禁用
disable [link|job]

#授权
grant [role|privilege]

#回收授权
revoke [role|privilege]

 

set option --name verbose --value true
set option --name poll-timeout --value 20000

 

三、查看

 

查看服务器

sqoop:000> show server -all
Server host: localhost
Server port: 12000
Server webapp: sqoop

 

查看版本

sqoop:000> show version -all
client version:
  Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb 
  Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
server version:
  Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb 
  Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
API versions:
  [v1]

 

查看内置连接器

sqoop:000> show connector 
+------------------------+---------+------------------------------------------------------------+----------------------+
|          Name          | Version |                           Class                            | Supported Directions |
+------------------------+---------+------------------------------------------------------------+----------------------+
| oracle-jdbc-connector  | 1.99.7  | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO              |
| sftp-connector         | 1.99.7  | org.apache.sqoop.connector.sftp.SftpConnector              | TO                   |
| kafka-connector        | 1.99.7  | org.apache.sqoop.connector.kafka.KafkaConnector            | TO                   |
| kite-connector         | 1.99.7  | org.apache.sqoop.connector.kite.KiteConnector              | FROM/TO              |
| ftp-connector          | 1.99.7  | org.apache.sqoop.connector.ftp.FtpConnector                | TO                   |
| hdfs-connector         | 1.99.7  | org.apache.sqoop.connector.hdfs.HdfsConnector              | FROM/TO              |
| generic-jdbc-connector | 1.99.7  | org.apache.sqoop.connector.jdbc.GenericJdbcConnector       | FROM/TO              |
+------------------------+---------+------------------------------------------------------------+----------------------+

 

sqoop:000> show link
0    [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
+------+----------------+---------+
| Name | Connector Name | Enabled |
+------+----------------+---------+
+------+----------------+---------+

 

其他查看命令

show server --all

show option --name verbose

show version --all

show connector --all or show connector

show driver

show link --all
show link --name linkName

show job --all 
show job --name jobName

show submission
show submission -j jobName
show submission --job jobName --detail

 

四、创建

创建link

create link --connector connectorName
create link -c connectorName
例如:
create link -connector generic-jdbc-connector
create link -connector hdfs-connector

 

创建job

create job --from fromLinkName --to toLinkName 
create job -f fromLinkName -t toLinkName

 

五、更新

update link --name linkName
update job --name jobName

 

六、删除

delete link --name linkName
delete job --name jobName

 

七、克隆

clone link --name linkName
clone job --name jobName

 

八、启动

start job --name jobName
start job --name jobName --synchronous

 

九、停止

stop job --name jobName

 

十、查看任务执行状态

status job --name jobName

 

[sqoop] sqoop命令

标签:关系   ade   tail   hadoop   配置   ssi   option   direction   数据导入   

原文地址:http://www.cnblogs.com/avivaye/p/6196922.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!