标签：
? ?
SQL statement execution on an SAP system
2014年4月7日
0:28
? ?
SQL execution plans, part 1: SQL statement execution on an SAP system
9 Feb 2009 2:52 PM
The most common root causes of a performance issue in an SAP system are slow I/O subsystems and inadequate execution plans. Rarely do we encounter cases where CPU resources would be scarce. In order to analyze slow SQL statements you first?need to figure out the statement text and the execution plan. SAP provides a rich set of database monitoring tools which makes it easy to figure out a slow SQL statement. It is also easy to get an execution plan for it. However, this might not be the currently used execution plan, but an estimated plan based on the given parameters. This causes a lot of confusion for database administrators and SAP consultant. Without a deeper knowledge how SAP executes SQL statements and how the SAP database monitors retrieve an execution plan you will not be able to understand performance issues based on bad or unstable execution plans.
Since this will be an extensive discussion, I had to divide it into several parts. Today we want to discuss how SAP executes SQL statements on SQL Server.
SAP was originally ported to Microsoft SQL Server on version 6.0 in the year 1995. In these days SQL Server had no SQL statement cache, but only a Stored Procedure Cache. Each time an ad-hoc query or prepared statement was sent to SQL Server, a new execution plan had to be created. However, without re-using execution plans the overall SAP performance on SQL Server (and other database platforms) would not have been acceptable. Therefore the SAP database interface always wrapped a stored procedure (SP) around an ABAP Open SQL Statement for execution?against Microsoft SQL Server.
This had several advantages: The statement text sent to SQL Server (over the network) was shorter. It only contained the SP name and the parameters. Existing execution plans could be reused (which was the main intention of using SPs). In addition, it was quite simple to get the current execution plan of an existing SP.
On the other hand, using SPs also resulted in a lot of overhead in the SAP database interface. A mapping between the Open SQL statement of ABAP and the SP executing the native SQL statement on SQL Server had to be implemented. Once the Open SQL statement in ABAP changed, SAP had to make sure that also the SP changed. Before executing a SP, you first have to create it. This has to be performed on a separate database connection in order to commit the SP creation. To minimize the SP creation attempts SAP implemented a SP name cache which contained the existing SPs already created on the database. As you can clearly see, this resulted in a quite complex SAP database interface.
Just as the SQL Server features increased with new releases (SQL Server 6.5, 7.0, 2000, 2005, 2008), the SAP database interface made several improvements within new releases. It was decided to simplify the SAP database interface and get rid of SPs, because the original limitations of SQL Server 6.0 regarding re-usage of execution plans do not exist anymore.
Up to SAP release 4.6D SAP always used stored procedures:
????????Permanent stored procedures for SQL statements which have a statement id (a SAP internally well-defined identification of the corresponding Open SQL statement). Those stored procedures usually started with a ‘Y‘ in their name.
????????Global temporary stored procedures (starting with "##") for SQL statements which do not have a statement id (this is typically the ABAP command SELECT FOR ALL ENTRIES)
As a result the global temporary stored procedures have been replaced with parameterized statements as of SAP kernel version 620. Starting with SAP kernel version 700 all stored procedures have been replaced with parameterized statements.
There are two special cases. In all SAP releases there is a small set of stored procedures (starting with "sap_") used by the SAP database monitors. Secondly you may execute Native SQL statements using the ABAP command EXEC SQL. This is a very?rarely used feature in ABAP. On SQL Server it results in ad-hoc queries without any parameter.
The following table gives an overview how SQL statements are executed on Microsoft SQL Server for the particular SAP NetWeaver releases.
? ?
?? | SAP ABAP release? 4.6D and older | SAP ABAP release? 620, 640 | SAP ABAP release? 700 and newer | SAP JAVA (all releases) |
Typical statement (90% case) | Permanent stored procedure | Permanent stored procedure | Parameterized statement | Parameterized statement |
FOR ALL ENTRIES (10% case) | Temporary stored procedure | Parameterized statement | Parameterized statement | ?? |
SAP database? monitor SPs | sap_* stored procedure | sap_* stored procedure | sap_* stored procedure | ?? |
Native SQL | Ad-hoc query | Ad-hoc query | Ad-hoc query | Ad-hoc query |
? ?
Using?parameterized statements simplified the SAP database interface. However, it is very difficult to figure out the current execution plan of a?parameterized SQL statement on SQL Server 2000. Starting with SQL Server 2005 you can easily query the SQL Server dynamic management views (or use the SAP database monitors). The only practicable method on SQL Server 2000 is a SQL Server profiler trace, which results in a huge overhead. Therefore it is recommended to upgrade a SQL Server 2000 system to SQL Server 2005 or SQL Server 2008?when running on SAP Basis releases of 6.20 or newer independent of ABAP or JAVA usage. Next week we will discuss when and how SQL Server creates an execution plan.
? ?
? ?
? ?
SQL Server的特性
早起的版本中不支持缓存执行计划
所以SAP自己在早起的Database Interface中做sql到SP的映射(sp有自己的执行计划)
虽然在数据库层面的查询效率上说有提高
但是在sap的DB Interface上的开销太大
两者相抵 效率还是不高
在后来的sql server版本中
支持缓存执行计划
所以sap的db interface也设计成了parameterized statement
参数化的查询 ?直接找该语句对应的执行计划?
所以建议使用2005以后的版本
? ?
? ?
? ?
SQL compilation
2014年4月7日
0:28
? ?
SQL execution plans, part 2: SQL compilation
22 Feb 2009 3:07 PM
Today we want to have a closer look how an execution plan for a given SQL statement is created within SQL Server and how this affects an SAP system. The idea of this discussion is not to highlight in detail the phases of the plan creation. The intention is to give an SAP administrator or ABAP/JAVA programmer an idea, which factors have an impact on the execution plan.
The component of SQL Server which creates the execution plan is called (query) optimizer. The procedure of creating an execution plan is called compilation. Once a new plan is re-created for an existing query it is called a re-compilation. The most important decision which has to be made by the optimizer is to decide which database indexes are used and how they are used. SQL Server may perform one or several index seeks. In some cases an index range scan on the index may be even faster. For JOINs the optimizer has to decide the join method (nested-loop, merge-join, hash-join) and the join order.
A compilation can be a very time consuming operation. That‘s the reason why SAP uses stored procedures or parameterized statements, which allow SQL Server to re-use an existing execution plan. However, a new execution plan has to be created?once no valid execution plan exists:
????????After restarting SQL Server.
????????After clearing the procedure cache using DBCC FREEPROCCACHE. This command can also be executed in the SAP transaction DBACOCKPIT.
????????Once an execution plan is?flushed out of the SQL Servers?statement cache due to memory pressure. This can happen at any time.
????????For SAP applications running 4.6D kernel or older kernel releases,?once the name of a global temporary stored procedure is?flushed out of SAP‘s stored procedure (SP) name cache. This can happen at any time. As a result SAP creates a new temporary SP which a different name. SQL Server will then create a new execution plan for the new SP.
????????For SAP applications running 4.6D kernel or older kernel releases, once the connection is closed, which originally created a global temporary SP. When SAP restarts a work process, all its database connections are certainly closed. As a result the dependent SPs will be dropped. SAP has to re-create these SPs and SQL Server has to re-create their execution plans.
An existing execution plan may become invalid:
????????After updating the index (or column) statistics, which were relevant for the query optimizer during plan creation. It makes no difference here whether this is an automatic or manual update statistics.
????????Once the structure of the table changes. This includes creating or dropping an index.
????????Once the SP executing a query or a table accessed by the query is explicitly marked for recompilation. This can be done using the system stored procedure sp_recompile. It is not recommended to mark a table for recompilation, because a "sp_recompile <TABLE NAME>" holds and requests database locks. We have seen blocking situations for hours at SAP systems caused by this.?
????????For SAP applications running 6.40 kernel or older kernel releases,?If the SAP profile parameter "dbs/mss/max_duration" was set. Whenever the average runtime of a query exceeded the specified threshold, SAP executed a "sp_recompile <PROCEDURE NAME>" for the stored procedure taking too long. This parameter caused rather trouble than solving issues. Therefore it was removed from the SAP kernel.
? ?
Generally speaking, the decisions made by the query optimizer are based on the estimation of the number of rows selected and cost calculations (CPU/memory/IO needs) for the potential execution plans. There are several factors which have an influence on the chosen execution plan. Let‘s have a closer look on them:
????????Existing indexes
This is the most obvious factor. Without a proper index you cannot expect a fast query execution. In the other hand, having too many indexes is much more often an issue on an SAP system. This may result in unstable (changing) execution plans. Inserts and deletes become more expensive the more indexes you have. Defining proper indexes is therefore often a compromise. You have to find a balance between speeding up a single query and minimizing the impact on the other queries. Typically you therefore create a few combined indexes to tune your most important queries running in dialog. Queries running in a batch job are often not expected to have optimal performance.
? ?
????????Clustered index property
SAP (almost) always creates the primary key as a clustered index on SQL Server. You can have at most one clustered index per table since the data is physically sorted by the clustered index. Having a clustered primary key has several advantages and reduces the cost for using the primary key. All other indexes are non-clustered and called secondary indexes. A secondary index stores the clustered index key (which typically does not change) rather than a physical pointer to the data row. This results in better update performance. On the other hand you have to traverse two index trees when using a secondary index: First of all the index tree of the non-clustered index to receive the clustered index key and then the clustered index to access the data row. In a nutshell, having a primary clustered key in SAP reduces the cost of using the primary key and increases the cost of using a secondary index. In extreme cases a secondary index will not be chosen by the optimizer due to higher costs of accessing the secondary index.
? ?
????????Index statistics
The index statistics of SQL Server basically consists of the density information of the combined n index columns and a histogram of the first index column. A density is the reciprocal value of the number of distinct values. If a combined index consist of three columns A, B and C then the index statistics contains the density of (A), the combination of (A,B) and the combination of (A,B,C). The histogram is only available for column A. It contains a sample of up to 200 values and the number of rows which have this value. If required, SQL Server automatically creates additional column statistics including a histogram for other columns, too. However, the database option "auto create statistics" is to be set for it, as recommended by SAP.
Having not up-to-date index statistics is the most common suspect when seeing bad query performance. However, on an SAP system the index statistics are typically innocent. SAP strongly recommends to turn on the database options auto update/create statistics. This results in automatically updated statistics and works pretty well on an SAP system. There are only a few scenarios where SAP recommends performing a manual update statistics in addition. One example is a client import of a relatively small client (column "MANDT" means "Mandant" in German) into a huge SAP database. The same may be necessary when creating a new?company code (column "BUKRS" means "Buchungskreis" in German) on a system which only has a few distinct customer codes.
Updating the statistics only makes sense when the selectivity of the updated columns changes significantly. For example, the document number (column "BELNR" means "Belegnummer" in German) is almost always very selective. Even when updating a one year old index statistics of this column you will not see any significant difference in the selectivity. Therefore updating the statistics of this column is not relevant for optimizer decisions.
? ?
????????Parameters used for compilation
The SQL Server query optimizer estimates the number of selected rows based on the parameters passed at compilation time. This is very useful in many scenarios. A typical example is a delete flag (column "LOEKZ" means "Loeschkennzeichen" in German). There are many tables in ABAP which have such a column, containing either the value " " (for delete flag not set) and "X" (for delete flag set). There is often an index containing "LOEKZ" in order to speed-up selecting the rows, with have the delete flag set. Just having a look on the density 0.5 of this field, this index looks useless. Once you take the parameter value "X" into account and know from the histogram that only 0.01% of the rows have the value "X" (and 99.99% have the value " "), the index becomes very attractive. This SQL Server feature of creating execution plans dependent on the parameters passed at compilation time is called parameter sniffing.
Although parameter sniffing helps in many cases, it may result in unexpected execution plans in some scenarios. Running a query the first time with untypical parameters results in an execution plan which may not fit for the subsequent executions of the same query. Dependent on the parameters passed by chance at compilation time, totally different execution plans may be used.?
? ?
????????Other factors
It is self-evident that an optimizer hint or a plan guide has a huge impact on the execution plan. But also SQL Server configuration and connection settings are important. SAP recommends to turn off parallel query execution by setting the configuration option "max degree of parallelism" to 1. Setting this to a different value may result in different (parallel) execution plans.
All of the above factors influence the execution plan. For an SAP system you will rather see an issue with parameter sniffing than with outdated index statistics. However, customers often think that they have solved a performance issue by updating the statistics. They are not aware of the side effect of the statistics update. The optimizer will create a new execution plan due to the new statistics. This time the parameters used for compilation are probably different from the previous compilation. It‘s just like rolling the dice. Probably a DBBCC FREEPROCCACHE would also have (temporarily) solved the issue.
You may probably ask yourself what to do when running into a performance issue. It is the same as with any other issue. You first have to analyze it before taking action. With SQL Server 2005 you can easily figure both, the execution plan and the parameters used for compilation. Once you detect instable execution plans due to varying parameters you should force an execution plan, for example by adding an optimizer hint. You can add all possible SQL Server optimizer hints into SAP ABAP code. You will even find SAP Standard coding which will force certain indexes for different databases by using ABAP Query hints (please see OSS notes #129385, 133381)
? ?
? ?
? ?
how to get the plan
2014年4月7日
0:29
? ?
SQL execution plans, part 3: how to get the plan
10 May 2009 2:52 PM
We already discussed when an execution plan is created and how long it is valid. We also had a look on the factors which have an influence on the execution plan. SAP almost always executes parameterized statements on SQL Server. For details see:
Figuring out the execution plan of a long running SQL statement in an SAP system is much more complicated than one would expect in the first instance.
? ?
? ?
There are several places in an SAP NetWeaver system, where you can see SQL Server execution plans. The most famous place is the SAP SQL trace. Since this trace is started using SAP transaction ST05 it is often simply called ST05 trace. Every single ABAP programmer knows how to use this trace and to see the execution plan of a traced SQL statement. However, almost no one is aware of the fact, that you will always get an?estimatedexecution plan based on the parameters submitted to the queries while the traces was running. The?actual used?execution plan is only visible in the DBA Cockpit (SAP transaction DBACOCKPIT). You can either see the actual execution plan of a currently running request or an execution plan which is stored in the SQL Server statement cache. In the DBA Cockpit you can also start a SQL Profiler trace (including the execution plan). Due to the performance overhead of a SQL Profiler trace we typically do not recommend this.
The user interface of the SAP database monitor has changed a lot over the years. In the following we will have a closer look at SAP transaction DBACOCKPIT, which is available in NetWeaver 700 basis support package 13 and newer SAP releases. However, the concepts described here also apply for older SAP releases and for transaction ST04 (the predecessor of DBACOCKPIT).
? ?
? ?
? ?
SAP SQL trace (ST05 trace)
A ST05 trace can be activated per SAP instance (application server), SAP user and database table. The trace is running on the SAP application server and storing all executed SQL statements in a file. It contains the SQL statement, the actual used parameters, the number of rows returned and the time needed to run the statement in microseconds. The measured time includes all kind of waits, such as I/O, blocking database locks and network traffic. The ST05 trace is very popular because the user interface is identical for all database platforms supported by SAP (SQL Server, Oracle, DB2, Informix, MaxDB). An ABAP programmer can perform a ST05 trace without having a deeper knowledge of the underlying database system.??However, in order to check and understand a SQL Server execution plan you actually need to get a better idea how ST05 works for SQL Server.
Let‘s have a closer look how a ST05 trace looks like. One great feature of a ST05 trace is the summarization function. You can easily figure out the SQL statements with the highest overall time, time per execution or time per row returned:
? ?
?When pressing the "Explain" button an?estimated?execution plan is displayed. Therefore SAP creates temporarily a stored procedure for the traced SQL statement, using a SP name starting with sap_XPL. Then the stored procedure is executed with the traced parameters, using a database session with option SET SHOWPLAN_XML ON. As a result, SQL Server returns a XML execution plan for the query. This estimated execution plan is not necessarily identical to the execution plan used by SQL Server when the traced SQL statement was originally executed. It is the plan SQL Server would create now, considering the used parameters and the actual index statistics.
? ?
The XML execution plan is parsed by SAP in order to create the "Explain Tree" as you can see in the screenshot above. This kind of display is often better readable, since you can collapse branches of the tree individually. Alternatively you can display the execution plan as plain text by changing to the "Text Explain" tab. By pressing the "XML" button the original XML explain is opened using Internet Explorer (or the application which is associated with the .xml extension). You can even display the execution plan graphically using Management Studio. When pressing the "SSMS" button, SAP stores the XML plan as a file with the extension .sqlplan and opens it with?SQL?Server?Management?Studio which is installed on your computer:
? ?
? ?
? ?
"SQL statements" in DBA Cockpit
Using the SAP transaction DBACOCKPIT you can easily access the runtime statistics of all parameterized SQL statements and the currently cached execution plan. SAP retrieves this information from SQL Server‘s Dynamic Management View (DMV)"sys.dm_exec_query_stats".
When opening the screen "SQL statements" in DBA Cockpit you will get a list with the 300 most expensive SQL statements regarding the total elapsed time. Changing the number of rows or the criteria results in re-reading the list from SQL Server‘s DMVs. Therefore a specific SQL statement may not be in the result list any more after changing the criteria from "Total elapsed time" to "Average Logical Reads".
? ?
DBA Cockpit can use the standard list functionality of SAP. When clicking on the icons in the list above, you can sort by any column or apply an additional filter on the current list (without re-reading the data from SQL Server). After sorting and filtering the screen may look like this:
? ?
? ?
By pressing the "Explain" button the XML execution plan is read from the SQL Server statement cache (procedure cache). This is the execution plan which is currently in use. If you want to know how long it has been in use, then you can simply look at the last column of the list. This column called "Comp date" contains date and time of the plan creation (compilation). The SQL Server statement cache is typically large enough to keep most execution plans for days or weeks.
? ?
The user interface in SAP for the SQL explain is the same in ST05 and DBA cockpit. Per default the execution plan is displayed as an explain tree. You can use Internet Explorer to see it as plain XML or use SQL Management Studio to get a graphical view of it. The used explain method is explicitly mentioned. When clicking on the tab "SQL Code" you will get the SQL statement and the parameters which were used during the compilation. These parameters are extracted by SAP from the XML execution plan.
? ?
"SAP SQL Statistics" in DBA Cockpit
In SQL Server 2005 and newer releases you can easily figure out expensive SQL queries. They can be queried from the SQL Server statement cache using DMVs. In older SQL Server releases there was no feature like this. Therefore SAP implemented its own statement cache in the SAP database interface for OLEDB. The cache is used to collect statement execution statistics on the application server. In DBA Cockpit you can see the SQL Server statement cache in "SQL Statements" and the SAP statement cache in "SAP SQL Statistics". Since it is not needed any more, the SAP statement cache was removed for SQL Server 2008 and newer releases. Therefore "SAP SQL Statistics" is grayed-out in DBA Cockpit for these SQL releases.
? ?
Each SAP instance (application server) has its own statement cache, containing statistical information like the number of executions and min/max/average execution time per query. Since this data is measured by SAP, it cannot include the number of physical/logical reads per query or the used execution plan. The size of the SAP statement cache is configurable, but has a fixed size. Therefore a query may be flushed out of the SAP statement cache. The statement caches of the SAP instances are independent from each other and from the statement cache in SQL Server. Therefore a particular query may be found in the SAP statement cache and not in the SQL Server statement cache, or visa verse.
You can reset the "SAP SQL Statistics" per SAP instance. This is useful for tracing a long running SAP batch jobs without the overhead of a SQL Profiler trace. Unlike the SQL Server 2005 DMVs, it contains the number of rows selected per statement.
The "SAP SQL Statistics" has the same user interface as in "SQL Statements" and ST05 to display the execution plan of a query. However, you should be aware that this is an estimated execution plan. It is created using the parameters stored in the SAP statement cache of the current SAP instance.??The SAP statement cache contains the parameters of longest execution for each query. If there is a problem with a changing execution plan caused by changing parameters then you typically will not see the right execution plan here. Therefore you should search for the same query in "SQL Statements" and check the execution plan there.
? ?
"Database Processes" in DBA Cockpit?
To check the current activity on SQL Server you can use "Database Processes" in DBA Cockpit. For SQL Server 2000 you can see a list with the content of the SQL Server system table "sysprocesses". For SQL Server 2005 and newer a join of the DMVs "sys.dm_exec_sessions", "sys.dm_exec_connections" and "sys.dm_exec_requests" is displayed. You can filter by SAP system (using the SQL Server login), application server and SAP work process number. Each SQL Session contains the total number of physical reads, writes and consumed CPU time. Using the "Reset" and "Since Reset" buttons you can easily see which SQL session is currently consuming CPU.
If you want to check the execution plan of a currently running query then you should first press the button "Active Requests". This filters the output list in DBA cockpit. As a result, only active SQL requests are visible. After this you can display the currently used execution plan of an active request by pressing the "Explain" button.
As long as you do not press "Active Requests" you can see all SQL sessions and connections, including those which do not have an active request. For those connections you also see a SQL statement. However, this is not an actual running statement. It is the most recent statement which ran in the past. Since SQL Server does not have a valid plan handle any more for those SQL statements, you can only create an estimated execution plan for them. Therefore the "Explain" button for the most recent SQL statement is not available any more in the newest release of DBA Cockpit.
? ?
? ?
Best practices for SAP NetWeaver 700 and newer
The easiest way to find slow SQL statements for an ABAP programmer is the ST05 trace. A database administrator should check "SQL Statements" in DBA Cockpit. For SQL Server 2000 you should use "SAP SQL Statistics" instead.
The best way to figure out an execution plan is "SQL Statements". For SQL Server 2000 (and SAP NetWeaver 700) you can only be sure about the execution plan when performing a SQL Server profiler trace. For supportability reasons we strongly recommend to upgrade SQL Server 2000 to a newer release when running SAP NetWeaver 700 (or newer).
? ?
? ?
? ?
标签：
原文地址：http://www.cnblogs.com/rootbin/p/4463474.html
