码迷,mamicode.com
首页 > 数据库 > 详细

sqlldr批量生成控制文件

时间:2016-07-01 13:31:16      阅读:520      评论:0      收藏:0      [点我收藏+]

标签:

参考文档:http://askjoey.blog.51cto.com/7594056/1671852

环境:oracle11g,redhat linux

1.建立基础表

create table TAB_COLUMN_UNION
(
  TAB_COLUMN_ORA    VARCHAR2(4000),
  TAB_COLUMN_SOURCE VARCHAR2(4000),
  TABLE_NAME        VARCHAR2(500)
)
tablespace SDATA
  pctfree 10
  initrans 1
  maxtrans 255
  storage
  (
    initial 16
    next 8
    minextents 1
    maxextents unlimited
  );

table_name 中插入需要生成控制文件的表名。

TAB_COLUMN_ORA 通过存储过程生成表对应的每列

TAB_COLUMN_SOURCE 通过存储过程生成导出成数据文件的语句,这里假设源库和目标库都是oracle

存储过程如下:

CREATE OR REPLACE PROCEDURE ETL_DOWNLOAD as



n_count integer;

v_column_ora varchar2(4000);


begin

update tab_column_union set tab_column_ora=null;

update tab_column_union set TAB_COLUMN_SOURCE=null;


for c1 in (select column_name,a.table_name,column_id,data_type from user_tables a,user_tab_columns b
  where a.table_name=b.TABLE_NAME and b.DATA_TYPE not in (‘CLOB‘,‘BLOB‘) order by a.table_name,column_id)

loop

n_count:=0;

v_column_ora:=null;

select ‘‘‘"‘‘||‘||c1.column_name||‘||‘‘"‘‘‘ into v_column_ora from dual;

update tab_column_union set TAB_COLUMN_SOURCE=TAB_COLUMN_SOURCE||v_column_ora||‘||‘‘,‘‘||‘ where table_name=c1.table_name;

--select count(*) into n_count from tab_column_ex where table_name=c1.table_name and tab_column=c1.column_name;

case when  c1.data_type=‘DATE‘ then

   select c1.column_name||‘ DATE "YYYY-MM-DD HH24:MI:SS"‘ into v_column_ora from dual;

   update tab_column_union set tab_column_ora=tab_column_ora||v_column_ora||‘,‘||chr(10) where table_name=c1.table_name;

when c1.data_type=‘NUMBER‘ then

   select c1.column_name||‘ INTEGER EXTERNAL‘ into v_column_ora from dual;

   update tab_column_union set tab_column_ora=tab_column_ora||v_column_ora||‘,‘||chr(10) where table_name=c1.table_name;

else


 update tab_column_union set tab_column_ora=tab_column_ora||c1.column_name||‘,‘||chr(10) where table_name=c1.table_name;


end case;


END LOOP;

--update tab_column_union t set tab_column_db2=substr(t.tab_column_db2,1,length(t.tab_column_db2)-1);

update tab_column_union t set tab_column_ora=substr(t.tab_column_ora,1,length(t.tab_column_ora)-2);

update tab_column_union t set TAB_COLUMN_SOURCE=substr(t.TAB_COLUMN_SOURCE,1,length(t.TAB_COLUMN_SOURCE)-7);

update tab_column_union t set TAB_COLUMN_SOURCE=‘select ‘||TAB_COLUMN_SOURCE||‘from ‘||table_name||‘;‘;

    COMMIT;

end ETL_DOWNLOAD;

步骤:1.先把需要从源库导入到目标库的表 在 目标库中建立起来,表结构保持一致。

         2.建立基础表TAB_COLUMN_UNION

         插入需要生成控制文件的表名,注意表名要大写,否则执行存储过程会找不到表。

         3.建立存储过程ETL_DOWNLOAD

执行存储过程后会写入TAB_COLUMN_ORA和TAB_COLUMN_SOURCE

再在sqlplus 下执行:

set lines 1000

set pages 0

select ctl_name from (
select ‘spool /home/oracle/‘||lower(table_name)||‘.ctl‘ ctl_name,table_name, 0 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select ‘select ‘‘LOAD DATA‘,table_name,0.1 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select ‘INFILE  ‘‘‘‘$ZM_PATH/‘||table_name||‘_$rq.CSV‘‘‘‘‘,table_name,0.11 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select ‘TRUNCATE into table ‘||table_name,table_name,0.12 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select ‘FIELDS TERMINATED BY  ‘||‘‘‘‘||‘‘‘‘||‘,‘||‘‘‘‘||‘‘‘‘||‘ OPTIONALLY ENCLOSED BY ‘||‘‘‘‘||‘‘‘‘||‘"‘||‘‘‘‘||‘‘‘‘,table_name, 0.2 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select ‘TRAILING NULLCOLS‘,table_name,0.3 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select ‘(‘ ,table_name,0.4 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select b.tab_column_ora,a.table_name,0.5 cid from user_tables a ,tab_column_union b where a.table_name=b.TABLE_NAME
union all
select ‘) ‘‘ from dual;‘ ,table_name,0.6 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select ‘spool off‘,table_name,0.7 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
) aa order by table_name,cid;

把执行结果复制出来写入到一个文件里,文件名ctlout1.sql

通过linux命令

sed  ‘/^$/d‘ ctlout1.sql> ctlout2.sql

去掉ctlout1.sql中的空行。

然后在通过sqlplus连接上目标库,

先执行:

set lines 1000
set pages 0
set echo off
set feedback off
set heading off
set termout off
set trims on

然后再

@ctlout2.sql,会在/home/oracle下生成控制文件。

 

另外如果源库也是oracle,那么可以在sqlplus下执行下列语句生成导出文件。

----生成导出文件

sqlplus  -s 下执行

set lines 1000
set pages 0
set echo off
set feedback off
set heading off
set termout off
set trims on

spool /home/oracle/dataout.sql
select txt_name from (
select ‘spool /datafile/zmcf/ZMCF_CSV/‘||upper(table_name)||‘.CSV‘ txt_name,table_name, 0 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
union all
select t.tab_column_source,t.table_name,0.1 from tab_column_union t
union all
select ‘spool off‘,table_name,0.7 cid from user_tables a where a.TABLE_NAME in (select table_name from tab_column_union)
) aa order by table_name,cid;

spool off

连接到源库,

set lines 1000
set pages 0
set echo off
set feedback off
set heading off
set termout off
set trims on

@dataout.sql

生成数据文件,

需要先建立目录:/datafile/zmcf/ZMCF_CSV

 

另外在我的环境源库字符集gbk,目标库字符集utf8

导出的时候设置export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK

导出txt文件再导入到目标库,

同样在导入时先设置字符集

export NLS_LANG=AMERICAN_AMERICA.AL32UTF8

发现导入时总是报分隔符不正确,怀疑是字符集问题。

file -i 查看导出文件的字符编码,发现内含中文的文件字符集并不是gbk,而是其他的字符集,可能问题就出现在这里。

那么就换种思路,再导出时,设置客户端字符集为:

export NLS_LANG=AMERICAN_AMERICA.AL32UTF8

然后再导出,查看导出文件,含中文的文件字符集已经是utf-8

再把文件正常导入到目标库,发现导入正常。

问题解决

 

sqlldr批量生成控制文件

标签:

原文地址:http://www.cnblogs.com/leipeng/p/5630513.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!