具体的安装和jdbc的驱动准备你们看第6课。现在我用一个例子讲解sqoop2的具体使用方法
有一个mysql的表叫worker,里面有三条数据,我们要将其导入hadoop
这是建表语句
CREATE TABLE `workers` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(20) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8;
insert into workers (name) values ('jack'); insert into workers (name) values ('vicky'); insert into workers (name) values ('martin');
$ sqoop2 Sqoop home directory: /usr/lib/sqoop2 Sqoop Shell: Type 'help' or '\h' for help. sqoop:000> create connection --cid 1
Creating connection for connector with id 1 Please fill following values to create new connection object Name: First connection Configuration configuration JDBC Driver Class: com.mysql.jdbc.Driver JDBC Connection String: jdbc:mysql://mysql.server/database Username: sqoop Password: ***** JDBC Connection Properties: There are currently 0 values in the map: entry# Security related configuration options Max connections: 0 New connection was successfully created with validation status FINE and persistent id 1
sqoop:000> create job --xid 1 --type import
Creating job for connection with id 1 Please fill following values to create new job object Name: First job Database configuration Table name: workers Table SQL statement: Table column names: Partition column name: Boundary query: Output configuration Storage type: 0 : HDFS Choose: 0 Output format: 0 : TEXT_FILE 1 : SEQUENCE_FILE Choose: 0 Compression format: 0 : NONE 1 : DEFAULT 2 : DEFLATE 3 : GZIP 4 : BZIP2 5 : LZO 6 : LZ4 7 : SNAPPY Choose: 0 Output directory: /user/jarcec/users New job was successfully created with validation status FINE and persistent id 1
sqoop:000> start job --jid 1 Submission details Job ID: 1 Server URL: http://localhost:12000/sqoop/ Created by: root Creation date: 2014-11-26 16:41:30 CST Lastly updated by: root External ID: job_1406097234796_0006 N/A 2014-11-26 16:41:30 CST: BOOTING - Progress is not available
$ hdfs dfs -ls /user/jarcec/workers/ Found 3 items -rw-r--r-- 2 sqoop2 supergroup 0 2014-11-26 16:42 /user/jarcec/workers/_SUCCESS -rw-r--r-- 2 sqoop2 supergroup 9 2014-11-26 16:41 /user/jarcec/workers/part-m-00000 -rw-r--r-- 2 sqoop2 supergroup 21 2014-11-26 16:42 /user/jarcec/workers/part-m-00001
$ hdfs dfs -cat /user/jarcec/workers/part-m-00000 1,'jack' $ hdfs dfs -cat /user/jarcec/workers/part-m-00001 2,'vicky' 3,'martin'
Alex 的 Hadoop 菜鸟教程: 第7课 Sqoop2 导入教程
原文地址:http://blog.csdn.net/nsrainbow/article/details/41518451