标签:des style blog class code c
To speed up bulk loading of data, InnoDB implements an insert buffer, a special index in the InnoDB system tablespace that buffers modifications to secondary indexes when the leaf pages are not in the buffer pool. Batched merges from the insert buffer to the index pages result in less random access patterns than when updating the pages directly. This speeds up the operation on hard disks.
In MySQL 5.5, the insert buffer has been extended to a change buffer, which covers all modifications of secondary index leaf pages. This will improve the performance of bulk deletes and updates, transaction rollback and the purging of deleted records (reducing the “purge lag”).
To assess the benefits of the extended buffering, you may want to run benchmarks with the settingsinnodb_change_buffering=all, innodb_change_buffering=inserts, and innodb_change_buffering=none. Users of solid-state storage, where random reads are about as fast as sequential reads, might benefit from disabling the buffering altogether.
Read on to learn how the change buffering works.
InnoDB can perform three kinds of modifications on secondary index records. If the affected index page is not in the buffer pool, the modifications can be buffered in the change buffer. When an index lookup or scan needs a page that is not in the buffer pool, the page will be read from the tablespace and any buffered changes will be merged to it.
The following operations can modify secondary index pages:
Before MySQL 5.5, UPDATE
, DELETE
and
purge operations were performed directly on the index pages, resulting in
random-access I/O. In MySQL 5.5, all of these operations can be buffered.
In InnoDB, there are two types of indexes: the clustered index B-tree, where the records are stored in the PRIMARY KEYorder, and secondary index B-trees, which identify rows by primary key. InnoDB multiversion concurrency control (MVCC) treats these indexes differently.
Records in the clustered index can be updated in place, and their hidden
system columns DB_TRX_ID
, DB_ROLL_PTR
point
to undo log entries from which earlier versions can be reconstructed. InnoDB
secondary index records do not contain any system columns, and their data is
never updated in place. An UPDATE
of an indexed column
requires the
operations Delete-mark(old), Insert(new)
and eventually Purge(old) in the secondary index.
An UPDATE
of a PRIMARY KEY
results
in Delete-mark, Insert and
eventually Purge in all indexes.
When a secondary index record has been marked for deletion or when the page
has been updated by a newer transaction, InnoDB will look up the clustered index
record. In the clustered index, it suffices to check
the DB_TRX_ID
and only retrieve the correct version from the
undo log when the record was modified after the reading transaction started.
When a page is in the buffer pool, it will always be updated directly. When a page is loaded to the buffer pool, any buffered changes will be merged to it, so that users never see unmerged changes.
Because change buffering works on individual leaf pages, we cannot buffer changes that would result into page splits or merges, but must perform such changes on the B-tree pages directly.
The insert buffer bitmap keeps track on the available space on pages and prevents overflows when buffering inserts. Delete-marking records can always be buffered, because the flag will be updated in place. Purging a delete-marked record could result in an empty page, something that we do not allow. We determine the non-emptiness of a page from previously buffered operations on the same page. If there are no previously buffered operations, the purge will have to load the index page to the buffer pool.
InnoDB refuses to buffer an operation when the on-disk change buffer tree would grow bigger than ? of the in-memory buffer pool (innodb_buffer_pool_size). This might be a good rule-of-thumb, but some setups could benefit from the ability of setting the change buffer size independently of the buffer pool size.
The InnoDB change buffer is a persistent data structure and a complex mechanism that comes into play when the workload does not fit in the buffer pool. Because it trades random I/O with a larger amount of sequential I/O, it speeds up operation on hard disks, where random access is much slower than sequential access.
On solid-state storage, there is not much difference between sequential and random access times. Change buffering may still be useful if writes to solid-state storage are expensive, either in terms of speed or the consumption of limited program/erase cycles. Change buffering could reduce the write load on user tablespaces and cause more writes to the system tablespace (which contains the insert buffer) and the redo log. These should be placed on a hard disk.
change buffering
The general term for the
features involving the change
buffer, consisting of insert
buffering, delete
buffering, and purge
buffering. Index changes resulting from SQL statements, which
could normally involve random I/O operations, are held back and performed
periodically by a background thread. This sequence of operations can
write the disk blocks for a series of index values more efficiently than if each
value were written to disk immediately. Controlled by the innodb_change_buffering
and innodb_change_buffer_max_size
configuration
options.
insert buffering
The technique of storing
secondary index changes due to INSERT
operations in the insert buffer rather than writing them
immediately, so that the physical writes can be performed to minimize random
I/O. It is one of the types of change
buffering; the others are delete
buffering and purge
buffering.
Insert buffering is not used if the secondary index is unique, because the uniqueness of new values cannot be verified before the new entries are written out. Other kinds of change buffering do work for unique indexes.
unique index
An index on a column or set of columns that have a unique constraint. Because the index is known not to contain any duplicate values, certain kinds of lookups and count operations are more efficient than in the normal kind of index. Most of the lookups against this type of index are simply to determine if a certain value exists or not. The number of values in the index is the same as the number of rows in the table, or at least the number of rows with non-null values for the associated columns.
The insert
buffering optimization does not apply to unique indexes. As
a workaround, you can temporarily
set unique_checks=0
while doing a bulk
data load into an InnoDB table.
delete buffering
The technique of storing
index changes due to DELETE
operations in
the insert buffer rather
than writing them immediately, so that the physical writes can be performed to
minimize random I/O. (Because delete operations are a two-step process, this
operation buffers the write that normally marks an index record for deletion.)
It is one of the types of change
buffering; the others are insert
buffering and purge
buffering.
purge buffering
The technique of storing
index changes due to DELETE
operations in
the insert buffer rather
than writing them immediately, so that the physical writes can be performed to
minimize random I/O. (Because delete operations are a two-step process, this
operation buffers the write that normally purges an index record that was
previously marked for deletion.) It is one of the types of change
buffering; the others are insert
buffering. and delete
buffering
When INSERT
, UPDATE
, and DELETE
operations are done to a table, often the
values of indexed columns (particularly the values of secondary keys) are not in
sorted order, requiring substantial I/O to bring secondary indexes up to date.
InnoDB has an insert
buffer that caches changes to secondary index entries when the
relevantpage is
not in the buffer
pool, thus avoiding I/O operations by not reading in the page
from the disk. The buffered changes are merged when the page is loaded to the
buffer pool, and the updated page is later flushed to disk using the normal
mechanism. The InnoDB main thread
merges buffered changes when the server is nearly idle, and during a slow
shutdown.
Because it can result in fewer disk reads and writes, this feature is most valuable for workloads that are I/O-bound, for example applications with a high volume of DML operations such as bulk inserts.
However, the insert buffer occupies a part of the buffer pool, reducing the memory available to cache data pages. If the working set almost fits in the buffer pool, or if your tables have relatively few secondary indexes, it may be useful to disable insert buffering. If the working set entirely fits in the buffer pool, insert buffering does not impose any extra overhead, because it only applies to pages that are not in the buffer pool.
Command-Line Format | --innodb_change_buffering=# | ||
Option-File Format | innodb_change_buffering | ||
System Variable Name | innodb_change_buffering |
||
Variable Scope | Global | ||
Dynamic Variable | Yes | ||
Permitted Values (<= 5.5.3) | |||
Type | enumeration | ||
Default | inserts |
||
Valid Values | inserts |
||
none | |||
Permitted Values (>= 5.5.4) | |||
Type | enumeration | ||
Default | all | ||
Valid Values | inserts |
||
deletes | |||
purges | |||
changes | |||
all | |||
none |
Whether InnoDB
performs change buffering, an
optimization that delays write operations to secondary indexes so that the I/O
operations can be performed sequentially. The permitted values are inserts
(buffer insert operations), deletes
(buffer delete operations; strictly
speaking, the writes that mark index records for later deletion during a purge
operation), changes
(buffer insert and
delete-marking operations), purges
(buffer purge operations, the writes when
deleted index entries are finally garbage-collected), all
(buffer insert, delete-marking, and purge
operations) and none
(do not buffer any
operations). The default is all
.
innodb_change_buffer_max_size
Introduced | 5.6.2 | ||
Command-Line Format | --innodb_change_buffer_max_size=# | ||
Option-File Format | innodb_change_buffer_max_size | ||
System Variable Name | innodb_change_buffer_max_size |
||
Variable Scope | Global | ||
Dynamic Variable | Yes | ||
Permitted Values | |||
Type | numeric | ||
Default | 25 | ||
Range | 0 .. 50 |
Maximum size for the InnoDB change buffer, as a percentage of the total size of the buffer pool. You might increase this value for a MySQL server with heavy insert, update, and delete activity, or decrease it for a MySQL server with unchanging data used for reporting.
参考:
https://blogs.oracle.com/mysqlinnodb/entry/mysql_5_5_innodb_change
http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_change_buffering
http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_insert_buffer
http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_unique_index
http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_delete_buffering
http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_purge_buffering
http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_change_buffer_max_size
http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_change_buffering
http://dev.mysql.com/doc/innodb/1.1/en/innodb-performance-change_buffering.html#
MySQL 5.5: InnoDB Change Buffering,布布扣,bubuko.com
MySQL 5.5: InnoDB Change Buffering
标签:des style blog class code c
原文地址:http://www.cnblogs.com/xiaotengyi/p/3725812.html