轻描淡写随着数据表越来越大,历史数据的处理将越来越困难.且过滤字段没有索引(如时间)字段,而创建索引是需要临时表空间排序的.有越来越大,创建的维护工作也随之越来越难.
除了本文介绍的方法以外,还可以通过以下几种方法来维护历史数据:
1. 创建历史表,将历史数据定期移至历史表,让源表保持"瘦小身材"(源表会有碎片,需要定期对表和索引进行重建)
2. ORACLE系统包DBMS_REDEFINITION实现表的在线重定义
3. 把源表做分区处理,历史数据的维护可以转化成表/索引分区的维护.
本文将对以下几种方法做比较:
1. 普通删除语句(delete from tabname where condition1...condition2...)
2. 通过存储过程(每次提取一条)
3. 通过存储过程(批量提取)
以一张近500M的表作为实验对象,该表没有任何索引.总条数400多万:
SQL> select bytes/1024/1024 from user_segments where segment_name=‘ROBO‘;
BYTES/1024/1024
---------------
472 ---
SQL> select count(*) from robo;
COUNT(*)
----------
4154240 ----约400万条数据
SQL> select count(*) from robo where owner=‘TEST‘;
COUNT(*)
----------
4992 ---近5000条数据
SQL> select count(*) from robo where owner=‘SYS‘;
COUNT(*)
----------
1909760 ---近200万条数据
1. 普通删除的方法:
SQL> delete from robo where owner=‘TEST‘;
delete from robo
where
owner=‘TEST‘
性能数据:
-------------
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 1 1 0 0
Execute 1 0.65 0.64 59842 60092 5280 4992
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.65 0.64 59843 60093 5280 4992
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 52
Rows Row Source Operation
------- ---------------------------------------------------
0 DELETE ROBO (cr=60092 pr=59842 pw=0 time=642682 us)
4992 TABLE ACCESS FULL ROBO (cr=60092 pr=59840 pw=0 time=485683 us)
只要全表扫描一次，删除效率很快
2. 游标处理（每次提取一条）
declare
v_rowid varchar2(25);
v_sqltext varchar2(200);
cursor c1 is select rowid from robo where owner=‘TEST‘;
begin
v_sqltext := ‘delete from robo where rowid=:1‘;
open c1;
loop
fetch c1 into v_rowid;
execute immediate v_sqltext using v_rowid;
exit when c1%notfound;
end loop;
dbms_output.put_line(c1%rowcount||‘ row deleted.‘);
close c1;
end;
/
4992 row deleted.
PL/SQL procedure successfully completed.
性能数据:
-------------
delete from robo
where
rowid=:1
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 4993 0.33 0.31 1 4993 5280 4992
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4994 0.33 0.31 1 4993 5280 4992
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 52 (recursive depth: 1)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 1 0.00 0.00
********************************************************************************
declare
v_rowid varchar2(25);
v_sqltext varchar2(200);
cursor c1 is select rowid from robo where owner=‘TEST‘;
begin
v_sqltext := ‘delete from robo where rowid=:1‘;
open c1;
loop
fetch c1 into v_rowid;
execute immediate v_sqltext using v_rowid;
exit when c1%notfound;
end loop;
dbms_output.put_line(c1%rowcount||‘ row deleted.‘);
close c1;
end;
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.26 0.22 0 0 0 1
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.26 0.22 0 0 0 1
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 52
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 1 0.00 0.00
********************************************************************************
SELECT ROWID
FROM
ROBO WHERE OWNER=‘TEST‘
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 4993 0.70 0.73 60078 69059 0 4992
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4995 0.70 0.73 60078 69059 0 4992
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 52 (recursive depth: 1)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 1 0.00 0.00
SQL*Net message from client 1 3.13 3.13
db file sequential read 1 0.00 0.00
db file scattered read 3784 0.00 0.27
性能数据:
-----------------
delete from robo
where
rowid=:1
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 4993 0.33 0.31 1 4993 5280 4992
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4994 0.33 0.31 1 4993 5280 4992
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 52 (recursive depth: 1)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 1 0.00 0.00
********************************************************************************
全表扫描次数为一次，但游标提取和删除次数等于结果集数目+1次。执行效率比普通删除来得低
1. 普通删除方法
SQL> delete from robo where owner=‘SYS‘;
1909760 rows deleted.
Elapsed: 00:01:20.23
性能数据:
-------------
delete from robo
where
owner=‘SYS‘
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 31 87 0 0
Execute 1 17.52 78.34 27888 60322 2123625 1909760
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 17.53 78.34 27919 60409 2123625 1909760
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 52
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 2998 0.00 0.02
db file scattered read 2318 0.00 0.07
log file switch completion 17 0.97 14.43
log file switch (checkpoint incomplete) 84 0.97 45.38
log buffer space 4 0.19 0.29
SQL*Net message to client 1 0.00 0.00
********************************************************************************
主要耗时是等待在线日志的切换，过程当中产生大量在线日志，执行效率大。
2. 游标处理(每交提取一条)
SQL> declare
v_rowid varchar2(25);
v_sqltext varchar2(200);
cursor c1 is select rowid from robo where owner=‘SYS‘;
begin
v_sqltext := ‘delete from robo where rowid=:1‘;
open c1;
loop
fetch c1 into v_rowid;
execute immediate v_sqltext using 2 3 4 5 6 7 8 9 10 v_rowid;
exit when c1%notfound;
end loop;
dbms_output.put_line(c1%rowcount||‘ row deleted.‘);
close c1;
end;
/
11 12 13 14 15 16
PL/SQL procedure successfully completed.
Elapsed: 00:04:12.89
性能数据:
-------------
delete from robo
where
rowid=:1
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1909761 112.14 162.39 344 1909874 2124331 1909760
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 1909762 112.14 162.39 344 1909874 2124331 1909760
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 52 (recursive depth: 1)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 342 0.00 0.00
log file switch completion 19 0.97 12.66
log file switch (checkpoint incomplete) 76 0.97 39.72
********************************************************************************
declare
v_rowid varchar2(25);
v_sqltext varchar2(200);
cursor c1 is select rowid from robo where owner=‘SYS‘;
begin
v_sqltext := ‘delete from robo where rowid=:1‘;
open c1;
loop
fetch c1 into v_rowid;
execute immediate v_sqltext using v_rowid;
exit when c1%notfound;
end loop;
dbms_output.put_line(c1%rowcount||‘ row deleted.‘);
close c1;
end;
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 77.46 75.43 0 0 0 1
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 77.46 75.43 0 0 0 1
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 52
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 1 0.00 0.00
********************************************************************************
SELECT ROWID
FROM
ROBO WHERE OWNER=‘SYS‘
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 1 1 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 1909761 11.68 11.27 59936 2035069 0 1909760
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 1909763 11.69 11.27 59937 2035070 0 1909760
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 52 (recursive depth: 1)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 1 0.00 0.00
SQL*Net message from client 1 18.98 18.98
db file sequential read 7 0.00 0.00
db file scattered read 3798 0.00 0.21
游标执行一次,将所有符合条件的ROWID一次性查询出来，但提取次数和删除次数与结果集相当。比起直接删除效率来得慢.
3. 游标处理(批量提取)
declare
type var_tab is table of varchar2(25) index by pls_integer;
v_rowid var_tab;
v_sqltext varchar2(200);
cursor c1 is select rowid from robo where owner=‘SYS‘;
begin
v_sqltext := ‘delete from robo where rowid=:1‘;
open c1;
loop
fetch c1 bulk collect into v_rowid limit 50000;
for i in 1..v_rowid.count loop
execute immediate v_sqltext using v_rowid(i);
end loop;
exit when c1%notfound;
end loop;
dbms_output.put_line(c1%rowcount||‘ row deleted.‘);
close c1;
end;
/
性能数据:
------------
delete from robo
where
rowid=:1
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1909760 113.98 154.81 36737 2053459 2121671 1909760
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 1909761 113.98 154.81 36737 2053459 2121671 1909760
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 52 (recursive depth: 1)
Rows Row Source Operation
------- ---------------------------------------------------
1909760 DELETE ROBO (cr=2053679 pr=36737 pw=0 time=79817096 us)
1909760 TABLE ACCESS BY USER ROWID ROBO (cr=2053394 pr=36266 pw=0 time=12527588 us)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 36735 0.00 0.22
log file switch completion 21 0.97 12.30
log file switch (checkpoint incomplete) 72 0.97 30.14
********************************************************************************
declare
type var_tab is table of varchar2(25) index by pls_integer;
v_rowid var_tab;
v_sqltext varchar2(200);
cursor c1 is select rowid from robo where owner=‘SYS‘;
begin
v_sqltext := ‘delete from robo where rowid=:1‘;
open c1;
loop
fetch c1 bulk collect into v_rowid limit 50000;
for i in 1..v_rowid.count loop
execute immediate v_sqltext using v_rowid(i);
end loop;
exit when c1%notfound;
end loop;
dbms_output.put_line(c1%rowcount||‘ row deleted.‘);
close c1;
end;
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 49.21 47.42 0 0 0 1
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 49.21 47.42 0 0 0 1
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 52
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 2 0.00 0.00
SQL*Net message from client 2 0.00 0.00
********************************************************************************
SELECT ROWID
FROM
ROBO WHERE OWNER=‘SYS‘
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 39 2.48 2.42 126813 221222 1 1909760
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 41 2.48 2.42 126813 221222 1 1909760
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 52 (recursive depth: 1)
Rows Row Source Operation
------- ---------------------------------------------------
1909760 TABLE ACCESS FULL ROBO (cr=221222 pr=126813 pw=0 time=1910539 us)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 306 0.00 0.00
db file scattered read 7945 0.00 0.47
latch: object queue header operation 2 0.00 0.00
db file parallel read 1 0.00 0.00
全表扫描一次,但由于是批量提取数据(每次定为5万条)提高,减少提取次数,比起上一种方法节省了很大一部分时间.删除的次数依然与结果集相当.
普通删除和游标处理两种方法，删除操作的执行次数都相同。
1. 普通删除的方法：
较游标处理的方法相比较，虽然时间快，但记录数不好控制，从头删到尾，加上在线日志的归档，和不能批量提交给回滚段的压力非常大。
2. 游标处理的方法：
结果集的查询次数为一次,但提取次数跟方法相关.
批量提取的带来很大优势,主要体现在节省两个方面的时间： 1） 游标提取次数的减少，结果集提取时间缩短；2）游标执行时间的缩短
游标处理的最大优势是，虽然花费更多的时间，但能通过游标自由控制结果集，如批量删除和提交，减少回滚段的压力。
---------------------------------------------------
道行尚浅，欢迎拍砖。
原文地址：http://blog.csdn.net/robo23/article/details/44005773
