码迷,mamicode.com
首页 > 其他好文 > 详细

SSIS 日志记录的机制和查询sysssislog

时间:2015-09-16 19:58:44      阅读:360      评论:0      收藏:0      [点我收藏+]

标签:

一,MSDN Logging

SQL Server Integration Services includes log providers that you can use to implement logging in packages, containers, and tasks. With logging, you can capture run-time information about a package, helping you audit and troubleshoot a package every time it is run.

Logs are associated with packages and are configured at the package level. Each task or container in a package can log information to any package log. The tasks and containers in a package can be enabled for logging even if the package itself is not. For example, you can enable logging on an Execute SQL task without enabling logging on the parent package. A package, container, or task can write to multiple logs. You can enable logging on the package only, or you can choose to enable logging on any individual task or container that the package includes.

 

选择Provider Type为SSIS log provider for sql server,将log记录到Sql Server的Table中。

The SQL Server log provider, which writes log entries to the sysssislog table in a SQL Server database.

sysssislog Contains one row for each logging entry that is generated by packages or their tasks and containers at run time. This table is created in the msdb database when you install Microsoft SQL Server Integration Services. If you configure logging to log to a different SQL Server database, a sysssislog table with this format is created in the specified database.

 

二,将Package的执行情况存储在表sysssislog中,sysssislog表的结构

Element

Description

 ID

 The unique identifier of the logging entry.

 Event

 The name of the event that generated the logging entry.

Computer

The name of the computer on which the log event occurred.

Operator

The identity of the user who launched the package.

SourceName

The name of the container or task in which the log event occurred.

SourceID

The unique identifier of the package; the For Loop, Foreach Loop, or Sequence container; or the task in which the log event occurred.

ExecutionID

The GUID of the package execution instance.

技术分享                                 Note                              

Running a single package might create log entries with different values for the ExecutionID element. For example, when you run a package in SQL Server Data Tools, the validation phase might create log entries with an ExecutionID element that corresponds to SQL Server Data Tools. However, the execution phase might create log entries with an ExecutionID element that corresponds to dtshost.exe. For another example, when you run a package that contains Execute Package tasks, each of these tasks runs a child package. These child packages might create log entries that have a different ExecutionID element than the log entries that the parent package creates.

MessageText

A message associated with the log entry.

DataBytes

A byte array specific to the log entry. The meaning of this field varies by log entry.

StartTime

The time at which the container or task starts to run.

EndTime

The time at which the container or task stops running.

This feature is not implemented. The value in the endtime column is always the same as the value in the starttime column.

DataCode

An optional integer value that typically contains a value from the DTSExecResult enumeration that indicates the result of running the container or task:

  • 0 - Success

  • 1 - Failure

  • 2 - Completed

  • 3 - Canceled

 

Event 是产生日志的事件,不是每一个组件都能产生所有的Event,一个些Event只能由某些组件来产生,例如PacakgeStart只能由Package来触发,通过定位该事件,可以通过Source来确定Package的Name.

SourceID 是Package,以及Package中组件的ID属性值。

SourceName 是Package,以及Package中组件的Name属性。

ExecutionID 是Package一次execution的标识ID,如果一个package内有很多Task,那么每一个Task的ExecutionID字段是相同的。同一个Package,每次执行的ExecutionID都是不同的。

 

三,使用sysssislog查看数据的脚本

查看每一个Package及其内的组件执行的情况

select 
    s.[PackageName], 
    s.executionid,
    s.PackageStartTime,
    s.PackageEndTime,
    s.PackageDurationInMinutes,
    k.TaskName, 
    k.TaskStartTime,
    k.TaskEndTime,
    k.TaskDurationInMinutes

from 
(
    select s.source [PackageName], 
        s.sourceid as PackageID,
        s.executionid,
        min(s.starttime) PackageStartTime,
        max(s.endtime) PackageEndTime,
        DATEDIFF(MINUTE,min(s.starttime),max(s.endtime)) as PackageDurationInMinutes
    from dbo.sysssislog as s
    where s.event in (PackageStart, PackageEnd)
    group by s.source,s.sourceid, s.executionid
)as s
left join 
(
    SELECT t.source TaskName, 
        t.sourceid as TaskID,
        t.executionid,
        min(t.starttime) TaskStartTime,
        max(t.endtime) TaskEndTime,
        DATEDIFF(MINUTE,min(t.starttime),max(t.endtime)) as TaskDurationInMinutes
    from dbo.sysssislog t
    where t.event in (OnPostExecute,OnPreExecute)
    group by t.source,t.sourceid,t.executionid

) as k
on s.executionid=k.executionid and k.TaskID<>s.[PackageID]


汇总每一个package的执行情况

/**
Author: Troy Witthoeft
Date: 2014-01-06
Description: SSIS log package overview query.
**/
  
-- The following CTE &amp;amp;quot;numbers off&amp;amp;quot; the packages inside a single executionid.
-- The numbers are ordered by time.
;WITH MultiPackageFinderCTE AS (
    SELECT DISTINCT executionid, sourceid, source, MIN(starttime) AS MinStartTime, computer,
        ROW_NUMBER() over (PARTITION BY executionid ORDER BY MIN(starttime) DESC) AS PackageOrdinal
    FROM dbo.sysssislog
    WHERE sourceid IN (SELECT DISTINCT sourceid FROM dbo.sysssislog WHERE event = PackageStart)
    GROUP BY executionid, sourceid, source, computer
)
  
--Main Query
SELECT * 
FROM 
(
    SELECT A.executionid, A.computer, A.operator, A.starttime,
        A.Duration, A.endtime, A.Messages, A.MessageSources,
        COALESCE(B.PackageID,A.executionid) AS PackageID, B.PackageName, B.ExecutionNo,
        C.PackageStart, D.PackageEnd, D.datacode,
        E.ErrorCount,  F.FirstError, G.ConfigMessage, H.PackageRows,
        Status = ( CASE
            --Infer a status using PackageStart, PackageEnd, the endtime, and OnError messages.
            WHEN PackageStart IS NOT NULL AND PackageEnd IS NOT NULL AND ErrorCount IS NULL THEN Success
            WHEN PackageStart IS NOT NULL AND PackageEnd IS NULL AND ErrorCount IS NULL AND endtime > DATEADD(s,-10,GETDATE()) THEN Executing
            WHEN PackageStart IS NOT NULL AND PackageEnd IS NULL AND ErrorCount IS NULL AND endtime < DATEADD(s,-10,GETDATE()) THEN Stalled            
            WHEN PackageStart IS NOT NULL AND ErrorCount > 0 THEN Failure
            WHEN PackageStart IS NULL AND PackageEnd IS NULL AND ErrorCount IS NULL THEN Config Success
            WHEN PackageStart IS NULL AND PackageEnd IS NULL AND ErrorCount > 0 THEN Config Failure
            ELSE Other
        END )
    FROM (
        SELECT executionid
            ,MAX(computer) AS computer
            ,MAX(operator) AS operator
            ,MIN(starttime) AS starttime
            ,DATEDIFF(second, MIN(starttime), MAX(endtime)) As Duration
            ,MAX(endtime) AS endtime
            ,COUNT(message) AS Messages
            ,COUNT(DISTINCT sourceid) AS MessageSources
        FROM dbo.sysssislog
        GROUP BY executionid
    ) AS A
  
    LEFT JOIN (
        SELECT executionid
            ,sourceid AS PackageID
            ,source AS PackageName
            ,ROW_NUMBER() OVER (PARTITION BY computer, sourceid ORDER BY MinStartTime) AS ExecutionNo
        FROM MultiPackageFinderCTE
    ) AS B
    ON A.executionid = B.executionid
  
    -- Get the PackageStart event time.
    LEFT JOIN (
        SELECT executionid
            ,starttime AS PackageStart
        FROM dbo.SysSSISLog
        WHERE event = PackageStart
    ) AS C
    ON A.executionid = C.executionid
  
    -- Get the PackageEnd event time.
    LEFT JOIN (
        SELECT executionid
            ,datacode
            ,endtime AS PackageEnd
        FROM dbo.SysSSISLog
        WHERE event = PackageEnd
    ) AS D
    ON A.executionid = D.executionid
  
    -- Count the error messages.
    LEFT JOIN (
        SELECT executionid
            ,COUNT(message) AS ErrorCount
        FROM dbo.SysSSISLog
        WHERE event = OnError
        GROUP BY executionid
    ) AS E
    ON A.executionid = E.executionid
  
    -- Promote the first error message inside the executionid.
    LEFT JOIN (
        SELECT executionid
            ,message AS FirstError
        FROM (
            SELECT executionid
                ,message
                ,ROW_NUMBER() OVER(PARTITION BY executionid ORDER BY starttime ASC) AS ErrorNumber
            FROM dbo.sysssislog
            WHERE event = OnError
            ) AS F1
        WHERE ErrorNumber=1
    ) AS F
    ON A.executionid = F.executionid
  
    -- Get the configuration message.
    -- Optional: If using eternal XML configs,
    -- uncomment the subtring method to parse out the UNC path.
    LEFT JOIN (
        SELECT executionid
            ,message as ConfigMessage
            --,SUBSTRING(message, charindex(‘configure from the XML file "‘, message) + 29, charindex(‘".‘, message) - charindex(‘configure from the XML file "‘, message) - 29) AS dtsConfig
        FROM dbo.SysSSISLog
        WHERE event = OnInformation
        AND message LIKE %package%attempting%configure%
        GROUP BY executionid, message
    ) AS G
    ON A.executionid = G.executionid
  
    -- Get package-level OnInformation messages having the words "wrote rows"
    -- Extract integers from these messages. Sum them per executionid.
    LEFT JOIN (
        SELECT executionid
              ,SUM(ISNULL(CONVERT(INT, SUBSTRING(message, charindex(wrote , message) + 5, charindex(rows., message) - charindex(wrote , message) - 5)),0)) As PackageRows
        FROM dbo.SysSSISLog
        WHERE event = OnInformation
        AND message LIKE %wrote%rows%
        AND sourceid IN (SELECT DISTINCT sourceid FROM dbo.sysssislog WHERE event = PackageStart)
        GROUP BY executionid
    ) AS H
    ON A.executionid = H.executionid
) AS X
  
ORDER BY starttime DESC

 

SSIS 日志记录的机制和查询sysssislog

标签:

原文地址:http://www.cnblogs.com/ljhdo/p/4813379.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!