MySQL逗号分割字段的行列转换测试改进

标签：

<p>由于很多业务表因为历史原因或者性能原因，都使用了违反第一范式的设计模式。即同一个列中存储了多个属性值（具体结构见下表）。</p><p>这种模式下，应用常常需要将这个列依据分隔符进行分割，并得到列转行的结果。</p>

<span class="cnblogs_code_copy"></span><p style="margin: 10px auto; line-height: 19px; font-family: verdana, sans-serif; font-size: 13px;">表数据：</p><table style="border: 1px solid rgb(192, 192, 192); border-image: none; width: 143px; height: 142px; font-family: verdana, sans-serif; border-collapse: collapse;" border="0"><tbody><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">ID</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">Value</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">1</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">tiny,small,big</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">2</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">small,medium</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">3</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">tiny,big
</td></tr></tbody></table><p> </p><p style="margin: 10px auto; line-height: 19px; font-family: verdana, sans-serif; font-size: 13px;">期望得到结果：</p><table style="border: 1px solid rgb(192, 192, 192); border-image: none; font-family: verdana, sans-serif; border-collapse: collapse;" border="0"><tbody><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">ID</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">Value</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">1</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">tiny</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">1</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">small</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">1</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">big</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">2</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">small</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">2</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">medium</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">3</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">tiny</td></tr><tr><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">3</td><td style="padding: 3px; border: 1px solid rgb(192, 192, 192); border-image: none; border-collapse: collapse;">big
</td></tr></tbody></table><p> </p><p>
</p><pre><span style="line-height: 1.5;">#需要处理的表
</span><span style="color: rgb(0, 0, 255); line-height: 1.5;">create</span> <span style="color: rgb(0, 0, 255); line-height: 1.5;">table</span> tbl_name (ID <span style="color: rgb(0, 0, 255); line-height: 1.5;">int</span> ,mSize <span style="color: rgb(0, 0, 255); line-height: 1.5;">varchar</span>(<span style="color: rgb(128, 0, 0); line-height: 1.5; font-weight: bold;">100</span><span style="line-height: 1.5;">));
</span><span style="color: rgb(0, 0, 255); line-height: 1.5;">insert</span> <span style="color: rgb(0, 0, 255); line-height: 1.5;">into</span> tbl_name <span style="color: rgb(0, 0, 255); line-height: 1.5;">values</span> (<span style="color: rgb(128, 0, 0); line-height: 1.5; font-weight: bold;">1</span>,<span style="color: rgb(255, 0, 0); line-height: 1.5;">'</span><span style="color: rgb(255, 0, 0); line-height: 1.5;">tiny,small,big</span><span style="color: rgb(255, 0, 0); line-height: 1.5;">'</span><span style="line-height: 1.5;">);
</span><span style="color: rgb(0, 0, 255); line-height: 1.5;">insert</span> <span style="color: rgb(0, 0, 255); line-height: 1.5;">into</span> tbl_name <span style="color: rgb(0, 0, 255); line-height: 1.5;">values</span> (<span style="color: rgb(128, 0, 0); line-height: 1.5; font-weight: bold;">2</span>,<span style="color: rgb(255, 0, 0); line-height: 1.5;">'</span><span style="color: rgb(255, 0, 0); line-height: 1.5;">small,medium</span><span style="color: rgb(255, 0, 0); line-height: 1.5;">'</span><span style="line-height: 1.5;">);
</span><span style="color: rgb(0, 0, 255); line-height: 1.5;">insert</span> <span style="color: rgb(0, 0, 255); line-height: 1.5;">into</span> tbl_name <span style="color: rgb(0, 0, 255); line-height: 1.5;">values</span> (<span style="color: rgb(128, 0, 0); line-height: 1.5; font-weight: bold;">3</span>,<span style="color: rgb(255, 0, 0); line-height: 1.5;">'</span><span style="color: rgb(255, 0, 0); line-height: 1.5;">tiny,big</span><span style="color: rgb(255, 0, 0); line-height: 1.5;">'</span><span style="line-height: 1.5;">);

#用于循环的自增表
</span><span style="color: rgb(0, 0, 255); line-height: 1.5;">create</span> <span style="color: rgb(0, 0, 255); line-height: 1.5;">table</span> incre_table (AutoIncreID <span style="color: rgb(0, 0, 255); line-height: 1.5;">int</span><span style="line-height: 1.5;">);
</span><span style="color: rgb(0, 0, 255); line-height: 1.5;">insert</span> <span style="color: rgb(0, 0, 255); line-height: 1.5;">into</span> incre_table <span style="color: rgb(0, 0, 255); line-height: 1.5;">values</span> (<span style="color: rgb(128, 0, 0); line-height: 1.5; font-weight: bold;">1</span><span style="line-height: 1.5;">);
</span><span style="color: rgb(0, 0, 255); line-height: 1.5;">insert</span> <span style="color: rgb(0, 0, 255); line-height: 1.5;">into</span> incre_table <span style="color: rgb(0, 0, 255); line-height: 1.5;">values</span> (<span style="color: rgb(128, 0, 0); line-height: 1.5; font-weight: bold;">2</span><span style="line-height: 1.5;">);
</span><span style="color: rgb(0, 0, 255); line-height: 1.5;">insert</span> <span style="color: rgb(0, 0, 255); line-height: 1.5;">into</span> incre_table <span style="color: rgb(0, 0, 255); line-height: 1.5;">values</span> (<span style="color: rgb(128, 0, 0); line-height: 1.5; font-weight: bold;">3</span>);

select a.ID,substring_index(substring_index(a.mSize,‘,‘,b.AutoIncreID),‘,‘,-1) 
from 
tbl_name a
join
incre_table b
on b.AutoIncreID <= (length(a.mSize) - length(replace(a.mSize,‘,‘,‘‘))+1)
order by a.ID;

原理分析：

这个join最基本原理是笛卡尔积。通过这个方式来实现循环。

以下是具体问题分析：

length(a.Size) - length(replace(a.mSize,‘,‘,‘‘))+1  表示了，按照逗号分割后，改列拥有的数值数量，下面简称n

select a.ID,substring_index(substring_index(a.mSize,‘,‘,b.AutoIncreID),‘,‘,-1) 
from 
tbl_name a
join
incre_table b
on b.AutoIncreID <= (length(a.mSize) - length(replace(a.mSize,‘,‘,‘‘))+1)
order by a.ID;

原理分析：

这个join最基本原理是笛卡尔积。通过这个方式来实现循环。

以下是具体问题分析：

length(a.Size) - length(replace(a.mSize,‘,‘,‘‘))+1  表示了，按照逗号分割后，改列拥有的数值数量，下面简称n

join过程的伪代码：

根据ID进行循环

{

判断：i 是否 <= n

{

获取最靠近第 i 个逗号之前的数据， 即 substring_index(substring_index(a.mSize,‘,‘,b.ID),‘,‘,-1)

i = i +1 

}

ID = ID +1 

}

总结：

这种方法的缺点在于，我们需要一个拥有连续数列的独立表（这里是incre_table)。并且连续数列的最大值一定要大于符合分割的值的个数。

例如有一行的mSize 有100个逗号分割的值，那么我们的incre_table 就需要有至少100个连续行。

当然，mysql内部也有现成的连续数列表可用。如mysql.help_topic： help_topic_id 共有504个数值，一般能满足于大部分需求了。

改写后如下:

select a.ID,substring_index(substring_index(a.mSize,‘,‘,b.help_topic_id+1),‘,‘,-1) 
from 
tbl_name a
join
mysql.help_topic b
on b.help_topic_id < (length(a.mSize) - length(replace(a.mSize,‘,‘,‘‘))+1)
order by a.ID;

测试实例：

</pre><pre class="sql" name="code">-- SELECT  help_topic_id  FROM mysql.help_topic
-- eg.把一个字段用“，”分隔开组合
select group_concat(user_id ORDER BY user_id ASC) as nids from admin_user
SELECT b.did,GROUP_CONCAT(b.sid ORDER BY adjustment DESC,similar DESC) FROM test b GROUP BY b.did 


-- 1.如果多个导购同１张单的先分解
-- 加时间段
select a.DJBH,a.je,substring_index(substring_index(a.dgy_list_id,',',b.help_topic_id+1),',',-1) 
from 
ipos_qtlsd  a
join
mysql.help_topic b
on b.help_topic_id < (length(a.dgy_list_id) - length(replace(a.dgy_list_id,',',''))+1) and a.djbh='BP0102_qtsy000070'
order by a.DJBH;

-- 2.取平均值

-- SELECT  help_topic_id  FROM mysql.help_topic
-- 1.如果多个导购同１张单的先分解
-- @zddm
-- @ rq
select a.DJBH,
substring_index(substring_index(a.dgy_list_id,',',b.help_topic_id+1),',',-1) AS FJID,
substring_index(substring_index(a.dgy_list_mc,',',b.help_topic_id+1),',',-1) AS FJMC,
FORMAT(a.je/(length(a.dgy_list_id) - length(replace(a.dgy_list_id,',',''))+1),2) AS FJJE,
je 
from 
ipos_qtlsd  a
join
mysql.help_topic b
on b.help_topic_id < (length(a.dgy_list_id) - length(replace(a.dgy_list_id,',',''))+1) 
and a.rq BETWEEN UNIX_TIMESTAMP('2016-04-01') and UNIX_TIMESTAMP('2016-05-01')
and a.djbh='gd_151125000001'
order by a.DJBH;

-- gd_151125000001 
--3.分解后的指标
-- SELECT  help_topic_id  FROM mysql.help_topic
-- 1.如果多个导购同１张单的先分解
-- @khdm_change 终端代码
-- @start_time　开始时间
-- @end_time　　结束时间
-- SELECT * FROM ipos_qtlsd WHERE djbh='gd_151125000001'
set @khdm_change ='BP0102';
set @start_time=UNIX_TIMESTAMP('2016-04-01');
set @end_time=UNIX_TIMESTAMP('2016-05-01');
SELECT FJID,FJMC,SUM(FJJE) 
FROM(
select a.zddm,a.zdmc,a.DJBH,
 substring_index(substring_index(a.dgy_list_id,',',b.help_topic_id+1),',',-1) AS FJID,
 substring_index(substring_index(a.dgy_list_mc,',',b.help_topic_id+1),',',-1) AS FJMC,
 FORMAT(a.je/(length(a.dgy_list_id) - length(replace(a.dgy_list_id,',',''))+1),2) AS FJJE,
 je 
from ipos_qtlsd  a
 join mysql.help_topic b on b.help_topic_id < (length(a.dgy_list_id) - length(replace(a.dgy_list_id,',',''))+1) 
and a.rq BETWEEN  @start_time and @end_time
and a.zd_id=(SELECT id from com_base_kehu where khdm=@khdm_change)
) AA
GROUP BY FJID,FJMC
-- and a.djbh='gd_151125000001'
-- order by a.DJBH;

MySQL逗号分割字段的行列转换测试改进

标签：

原文地址：http://blog.csdn.net/luoxq111/article/details/51363904