数据泵导入数据,完成测试数据的迁移

先看看物理内存多大，去判断测试库选择多大内存！ 
$ free -m
             total       used       free     shared    buffers     cached
Mem:         16079      14978       1100          0        137      13930
-/+ buffers/cache:        910      15168
Swap:         8039          0       8039 
$ echo 3 > /proc/sys/vm/drop_caches
# free -g
             total       used       free     shared    buffers     cached
Mem:            15          0         15          0          0          0
-/+ buffers/cache:          0         15
Swap:            7          0          7
发现测试库有15G内存大小，但是内存都已使用，和客户沟通是新环境随便搞之后，直接清理cache内存

软件已经安装，检查后，查询oralce_home，及操作系统空间是否充足,查询后是500G，足够。

$echo ORACLE_HOME
# su - psoft
--配置数据库响应文件参数
echo ‘[GENERAL]
RESPONSEFILE_VERSION = "11.2.0"
OPERATION_TYPE = "createDatabase"
[CREATEDATABASE]
GDBNAME = "tt1"
SID = "tt1"
TEMPLATENAME = "General_Purpose.dbc"
STORAGETYPE=FS
DATAFILEDESTINATION =/xx/datafile
CHARACTERSET = "UTF8"
NATIONALCHARACTERSET= "AL16UTF16"
TOTALMEMORY = "8192"
SYSPASSWORD = "oracle"
SYSTEMPASSWORD = "oracle" ‘>>/home/xx/dbca_install.rsp

上述静默安装参数简要说明，字符集UTF8，内存8G,sys密码oracle，数据库名称tt1

--静默安装数据库
$ORACLE_HOME/bin/dbca -silent -responseFile /home/xx/dbca_install.rsp

--静默安装监听配置
$ netca -silent -responsefile $ORACLE_HOME/assistants/netca/netca.rsp

安装完毕！

日志格式化,具体操作可以找以前写的博客日志格式化内容，操作啥的不细说了。

alter database add logfile thread 1 group 4(‘/home/xx/hcm/oracledb/datafile/tt1/redo04.log‘) size 200m;
alter database add logfile thread 1 group 5(‘/home/xx/hcm/oracledb/datafile/tt1/redo05.log‘) size 200m;
alter database add logfile thread 1 group 6(‘/home/xx/hcm/oracledb/datafile/tt1/redo06.log‘) size 200m;
alter database add logfile thread 1 group 7(‘/home/xx/hcm/oracledb/datafile/tt1/redo07.log‘) size 200m;
alter database add logfile thread 1 group 8(‘/home/xx/hcm/oracledb/datafile/tt1/redo08.log‘) size 200m;
alter database add logfile thread 1 group 9(‘/home/xx/hcm/oracledb/datafile/tt1/redo09.log‘) size 200m;
alter database drop logfile group 1;
alter database drop logfile group 2;
alter database drop logfile group 3;

生产库创建对象(得到客户认可)模拟后进行删除,测试库进行导入。
生产库
> env|grep LANG
LANG=en_US.UTF-8
NLS_LANG=AMERICAN_AMERICA.UTF8

SQL> create table system.a_yz_test(id int,cc varchar2(20));
SQL> insert into system.a_yz_test values(1,‘呵呵‘);
1 row created.
SQL> commit;
Commit complete.
SQL> select * from system.a_yz_test;
ID CC
---------- --------------------
1 呵呵

SQL>select * from dba_directories;
SQL>create directory dump as ‘/bak‘;

> expdp \‘/ as sysdba\‘ directory=dump dumpfile=yang%u.dmp logfile=yang.log TABLES=SYSTEM.A_YZ_TEST
> scp /bak/yang01.dmp root@xx:/home/xx/.

测试库导入

SQL>select * from dba_directories;
SQL>create directory dump as ‘/home/xx‘;
$ env|grep LANG
NLS_LANG=simplified chinese_china.al32utf8
LANG=zh_US.utf-8
$ export ORACLE_SID=tt1
$ impdp \‘/ as sysdba\‘ directory=dump dumpfile=yang%u.dmp logfile=yang.log
ORA-39083: 对象类型 PRE_TABLE_ACTION 创建失败, 出现错误:
ORA-23327: 导入的延迟 rpc 数据与导入 db 的GLOBAL NAME不匹配
ORA-39083: 对象类型 PRE_TABLE_ACTION 创建失败, 出现错误:
ORA-23327: 导入的延迟 rpc 数据与导入 db 的GLOBAL NAME不匹配
SQL> select * from system.a_yz_test;
ID CC
---------- ------------------------------------------------------------
1 呵呵   --说明导入正常，可以进行全库导出导入了。

流程比较细
1.生产库查询需要导出的用户名称
SQL> select username ,DEFAULT_TABLESPACE,to_char(CREATED,‘yyyy-mm-dd hh24:mi:ss‘) as "c_date" from dba_users
 where ACCOUNT_STATUS=‘OPEN‘ order by 3;
排除SYS,SYSTEM均需要迁移

2.需要考虑迁移的数据量，大小200g以内，生产库及测试库操作系统本地均大于200g【小于需要考虑使用压缩导出】

SQL> select sum(bytes)/1024/1024/1024 from dba_segments;

SQL> select sum(bytes)/1024/1024/1024 from dba_data_files;

 3.查询需要迁移用户使用了什么表空间

SQL> select owner,sum(bytes)/1024/1024/1024,tablespace_name from dba_segments where owner in(‘A‘,‘H‘)
group by owner,tablespace_name;

SQL>select tablespace_name,sum(bytes)/1024/1024/1024 from dba_segments where owner in(‘PS‘,‘SYSADM‘,‘PEOPLE‘,‘RBDB‘,‘G3DB‘,‘KAOQIN_DB‘,‘DBMONITOR‘,‘RBHRDB‘,‘HRFANRAN‘)
group by tablespace_name order by 2;

本次客户比较奇葩，足足有50多个表空间，实际操作，使用了remap_tablespace进行转换后，导入还是报错，create table xx tablespace ccx 不存在，导入报错！
因此这次迁移，是同步所有生产环境的数据库进行操作！！！
那么也就是需要迁移同步表空间！ 我们发现表空间使用最大<30g，因此只需要对每个表空间创建一个数据文件即可！

select distinct ‘create tablespace ‘||tablespace_name||‘ datafile ‘‘/home/psoft/hcm/oracledb/datafile/tt1/‘||tablespace_name||‘.dbf‘‘ size 1m autoextend on next 50m;‘ as "sql_text" from dba_data_files
group by tablespace_name;

生产环境执行，输出脚本内容，测试库执行，创建表空间！

4.测试库导入的用户密码呢？

  通过数据泵impdp 参数sqlfile  以及include=user  可以得到创建用户的信息，编辑脚本后，可以创建用户，客户也不需要让测试人员修改密码，直接用生产环境的用户密码操作即可！

impdp \‘/ as sysdba\‘ dumpfile=tt%u.dmp directory=dump logfile=tt_user.log include=user sqlfile=impdp_user.sql

真正的操作
1.导出

> expdp \‘/ as sysdba\‘ directory=dump dumpfile=tt%u.dmp logfile=tt.log SCHEMAS=A,B,C PARALLEL=4
. . exported "SYSCC"."TBCC"                       0 KB       0 rows
Processing object type SCHEMA_EXPORT/TABLE/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC
Processing object type SCHEMA_EXPORT/FUNCTION/FUNCTION
Processing object type SCHEMA_EXPORT/PROCEDURE/PROCEDURE
Processing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC
Processing object type SCHEMA_EXPORT/FUNCTION/ALTER_FUNCTION
Processing object type SCHEMA_EXPORT/PROCEDURE/ALTER_PROCEDURE
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/INDEX/FUNCTIONAL_INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/FUNCTIONAL_INDEX/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/VIEW/VIEW
Processing object type SCHEMA_EXPORT/VIEW/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY
Processing object type SCHEMA_EXPORT/TABLE/TRIGGER
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type SCHEMA_EXPORT/MATERIALIZED_VIEW
Master table "SYS"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded
******************************************************************************
Dump file set for SYS.SYS_EXPORT_SCHEMA_01 is:
  /bak/tt01.dmp
  /bak/tt02.dmp
  /bak/tt03.dmp
  /bak/tt04.dmp
Job "SYS"."SYS_EXPORT_SCHEMA_01" successfully completed at Sat Apr 4 20:08:35 2020 elapsed 0 00:12:20
2.SCP ！省略
3.测试库
3.1 创建表空间
3.2 准备创建用户

impdp \‘/ as sysdba\‘ dumpfile=tt%u.dmp directory=dump logfile=tt_user.log include=user sqlfile=impdp_user.sql

编辑create_user.sql脚本

修改默认表空间名称！

CREATE USER "A" IDENTIFIED BY VALUES ‘XXXX40F0056DAB96165BC21989CC23;0AE52ADF439D30BD‘ default tablespace users
TEMPORARY TABLESPACE "TEMP";

随后创建脚本
3.3 授权
有的用户没权限，导入会报错，最简单粗暴的就是直接grant dba

SQL> select ‘grant dba to ‘||USERNAME||‘;‘ from dba_users where ACCOUNT_STATUS=‘OPEN‘;
SQL> grant dba to a;
授权成功。

3.4 导入
本次导入很多次失败
failed 1
使用多次转换，但是有些表在dba_segments 不存在，并未真正创建segments
但是呢，表的元数据空表，有其他的表空间，当时没有创建dba_segments不存在的表空间，最终导致导入失败。

nohup time impdp \‘/ as sysdba\‘ dumpfile=tt%u.dmp directory=dump logfile=tt.log parallel=8 EXCLUDE=STATISTICS REMAP_TABLESPACE=BNAPP:USERS,PSIMAGE2:USERS,HRLARGE:USERS,COAPP:USERS,EOAPP:USERS,PTTLRG:USERS,HRAPP2:USERS,PTTBL:USERS,PSIMAGE:USERS,PTAMSG:USERS,PTWORK:USERS,PTPRJWK:USERS,CULARG1:USERS,SYSAUX:USERS,PTPRC:USERS,CULARGE:USERS,EPAPP:USERS,PAAPP:USERS,PIAPP:USERS,HRAPP3:USERS,HRAPP6:USERS,SAAPP:USERS,HRSWORK:USERS,BNLARGE:USERS,TLAPP:USERS,HPAPP:USERS,GIAPP:USERS,HRSAPP:USERS,AAAPP:USERS,PTLOCK:USERS,CULARG2:USERS,PSDEFAULT:USERS,PSINDEX:USERS,HRAPP:USERS,PTAPPE:USERS,PTAPP:USERS,GPAPP:USERS,EOEWAPP:USERS,ERAPP:USERS,CCAPP:USERS,HRWORK:USERS,ADAPP:USERS,TLLARGE:USERS,PYAPP:USERS,PTRPTS:USERS,HRAPP7:USERS,EOECAPP:USERS,EOECLRG:USERS,EOLARGE:USERS,FGAPP:USERS,HRAPP4:USERS,HRAPP1:USERS,AALARGE:USERS,HRAPP5:USERS,FAAPP:USERS,STAPP:USERS,CULARG3:USERS,EOIUAPP:USERS,PTTREE:USERS,PY0LRG:USERS metrics=yes &

结束导出进程

$ impdp \‘/ as sysdba\‘ attach=SYS_IMPORT_FULL_01
Import> kill
是否确实要停止此作业 ([Y]/N): y

select ‘drop table ‘||owner||‘.‘||table_name||‘ purge;‘ from dba_tables where owner in(select USERNAME from dba_users where ACCOUNT_STATUS=‘OPEN‘ and username not in(‘SYS‘,‘SYSTEM‘))
group by owner,table_name ;

删除！ drop user xx cascade;即可，但是导入对象太多，删除太慢！！！

  failed 2 

nohup time impdp \‘/ as sysdba\‘ dumpfile=tt%u.dmp directory=dump logfile=tt.log parallel=8 EXCLUDE=STATISTICS

  并行太多！！！ 导入20分钟？？？  session event???   还在分配通道！！！

SQL_ID EVENT
-- -------------------- ------------------------------ --
afcz9s4uazbpk wait for unread message on bro adcast channel

afcz9s4uazbpk wait for unread message on bro adcast channel

 停止导出进程，删除表，删除用户及数据，重新导入

 成功！ 导出多少进程，导入多少！ 排除统计信息！

nohup time impdp \‘/ as sysdba\‘ dumpfile=tt%u.dmp directory=dump logfile=tt.log parallel=4 EXCLUDE=STATISTICS &

 确认没啥问题，OK,DBA权限是否回收，客户说测试库，那就不需要咯。