数据库优化-基准测试（四）

基准测试工具：sysbench

简述

sysbench多线程基准测试工具，可以测试：

• 文件I/O性能
• Scheduler性能
• 内存分配和转换速度
• POSIX线程实现性能
• 数据库服务器性能(OLTP)

工具初始开发用于MySQL的性能测试，现在已经扩展到其它数据库。
获取工具的网址：
https://code.launchpad.net/~sysbench-developers/sysbench/0.5
http://sysbench.sourceforge.net/
http://dev.mysql.com/downloads/benchmarks.html
源码下载：
bzr branch lp:sysbench
http://sourceforge.net/projects/sysbench/ (0.4.12)
http://downloads.mysql.com/source/sysbench-0.4.12.5.tar.gz

Sysbench的安装

$ bzr branch lp:sysbench
$ cd sysbench
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install

Sysbench CPU测试

例子1：处理素数的一个简单测试

$ sysbench --test=cpu --cpu-max-prime=20000 run
Running the test with following options:
Number of threads: 1
Primer numbers limit: 20000
General statistics:
total time: 54.5050s
total number of events: 10000
total time taken by event execution: 54.4840s
response time:
min: 5.20ms
avg: 5.45ms
max: 45.02ms
approx. 95 percentile: 5.69ms
Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 54.4840/0.00

例子2：处理素数的多线程测试，线程数为4

$ sysbench --num-threads=4 --test=cpu --cpu-max-prime=20000 run
Running the test with following options:
Number of threads: 4
Primer numbers limit: 20000
General statistics:
total time: 16.5565s
total number of events: 10000
total time taken by event execution: 66.1771s
response time:
min: 5.89ms
avg: 6.62ms
max: 101.86ms
approx. 95 percentile: 8.43ms
Threads fairness:
events (avg/stddev): 2500.0000/136.15
execution time (avg/stddev): 16.5443/0.00

例子3：处理素数的多线程测试，线程数为8

$ sysbench --num-threads=8 --test=cpu --cpu-max-prime=20000 run
Running the test with following options:
Number of threads: 8
Primer numbers limit: 20000
General statistics:
total time: 16.1659s
total number of events: 10000
total time taken by event execution: 129.1411s
response time:
min: 5.89ms
avg: 12.91ms
max: 49.95ms
approx. 95 percentile: 25.30ms
Threads fairness:
events (avg/stddev): 1250.0000/54.65
execution time (avg/stddev): 16.1426/0.01

Sysbench 线程测试(scheduler)

例子：Lock mutex => Yield CPU => Run queue => Unlock mutex

$ sysbench --num-threads=64 --test=threads run
Running the test with following options:
Number of threads: 64
General statistics:
total time: 3.4055s
total number of events: 10000
total time taken by event execution: 217.2030s
response time:
min: 0.56ms
avg: 21.72ms
max: 162.65ms
approx. 95 percentile: 59.37ms
Threads fairness:
events (avg/stddev): 156.2500/10.98
execution time (avg/stddev): 3.3938/0.01

Sysbench mutexs测试

例子：线程试图获取相同的mutex集合，测试mutexe实现

$ sysbench --num-threads=128 --test=mutex --mutex-locks=100000 --mutex-loops=100
--mutex-num=2048 run
Running the test with following options:
Number of threads: 128
General statistics:
total time: 10.4538s
total number of events: 128
total time taken by event execution: 1274.8660s
response time:
min: 8069.55ms
avg: 9959.89ms
max: 10453.19ms
approx. 95 percentile: 10436.39ms
Threads fairness:
events (avg/stddev): 1.0000/0.00
execution time (avg/stddev): 9.9599/0.52

Sysbench memory测试

例子1：顺序读或写内存

$ sysbench --num-threads=1 --test=memory run –memory-block-size=16k
--memory-total-size=4G --memory-oper=read
Running the test with following options:
Number of threads: 1
Operations performed: 262144 (302035.23 ops/sec)
4096.00 MB transferred (4719.30 MB/sec)
General statistics:
total time: 0.8679s
total number of events: 262144
total time taken by event execution: 0.3994s
response time:
min: 0.00ms
avg: 0.00ms
max: 0.11ms
approx. 95 percentile: 0.00ms
Threads fairness:
events (avg/stddev): 262144.0000/0.00
execution time (avg/stddev): 0.3994/0.00

内存OPS：

数字的谎言！
前一个测试报告在8MB块大小时读速度1.8TB/s
32MB块大小时读速度上升到7.2TB/s
恒定的OPS是真正的读瓶颈。

内存写OPS：

内存写带宽MB/S：

Sysbench 文件IO测试

IO测试包括两个阶段：

• 准备：在文件系统中创建文件
• 运行：在之前创建的文件上执行测试

支持测试的I/O模式( –file-test-mode)，包括：

• seqwr : 顺序写
• seqrewr : 顺序重写
• seqrd : 顺序读
• rndrd : 随机读
• rndwr : 随机写
• rndrw : 随机读/写

其它选项：

--file-num : 文件数量
--file-block-size：文件块大小
--file-total-size : 所有合并的文件
--file-io-mode : sync , async, mmap
--file-extra-flags (odirect)
--file-fsync-freq : fsync()的频率
--file-fsync-all
--file-rw-ratio : rndrw模式中的R/W比例

准备文件：

$ sysbench --test=fileio --file-num=4 --file-total-size=1G prepare
4 files, 262144Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: 0
Creating file test_file.0
Creating file test_file.1
Creating file test_file.2
Creating file test_file.3
1073741824 bytes written in 40.32 seconds (25.40 MB/sec).

执行测试1（文件数1，顺序重写，文件块大小4K）：

sysbench --test=fileio --file-num=1 --file-total-size=2G --file-fsync-all=on --file-test-mode=seqrewr --max-time=100 –file-block-size=4096 --max-requests=0 run
Number of threads: 1
Extra file open flags: 0
1 files, 2Gb each
2Gb total file size
Block size 4Kb
Calling fsync() after each write operation.
Using synchronous I/O mode
Doing sequential rewrite test
Operations performed: 0 Read, 524288 Write, 524288 Other = 1048576 Total
Read 0b Written 2Gb Total transferred 2Gb (52.616Mb/sec)
13469.70 Requests/sec executed
Test execution summary:
total time: 38.9235s
total number of events: 524288
total time taken by event execution: 38.2213
per-request statistics:
min: 0.06ms
avg: 0.07ms
max: 8.60ms
approx. 95 percentile: 0.12ms

执行测试2（文件数29，随机读写，文件块大小16K）：

sysbench --test=fileio --file-num=29 --file-total-size=290G --file-fsync-all=on --file-test-mode=rndrw --max-time=100 --file-block-size=16384 --max-requests=0 run
Number of threads: 1
Extra file open flags: 0
29 files, 10Gb each
290Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Calling fsync() after each write operation.
Using synchronous I/O mode
Doing random r/w test
Operations performed: 449824 Read, 299883 Write, 299883 Other = 1049590 Total
Read 6.8638Gb Written 4.5759Gb Total transferred 11.44Gb (117.14Mb/sec)
7497.05 Requests/sec executed
Test execution summary:
total time: 100.0003s
total number of events: 749707
total time taken by event execution: 98.9223
per-request statistics:
min: 0.01ms
avg: 0.13ms
max: 7.34ms
approx. 95 percentile: 0.16ms

执行测试3（文件数29，随机读写，读写比率为4，文件块大小16K，线程数8）：

sysbench --num-threads=8 --test=fileio --file-num=29 –file-total-size=290G --file-fsync-all=on --file-test-mode=rndrw --max-time=100 --file-rw-ratio=4 --file-block-size=16384 --max-requests=0 run
Number of threads: 8
Extra file open flags: 0
29 files, 10Gb each
290Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 4.00
Calling fsync() after each write operation.
Using synchronous I/O mode
Doing random r/w test
Operations performed: 1734215 Read, 433552 Write, 433552 Other = 2601319 Total
Read 26.462Gb Written 6.6155Gb Total transferred 33.077Gb (338.71Mb/sec)
21677.60 Requests/sec executed
Test execution summary:
total time: 100.0003s
total number of events: 2167767
total time taken by event execution: 796.3734
per-request statistics:
min: 0.01ms
avg: 0.37ms
max: 15.18ms
approx. 95 percentile: 1.77ms

执行测试4（文件数29，随机读写，读写比率为4，文件块大小16K，线程数16, –file-fsync-all 每执行完一次写操作,就执行一次fsync,默认是off）：

sysbench --num-threads=16 --test=fileio --file-num=29 –file-total-size=290G --file-fsync-all=on --file-test-mode=rndrw --max-time=100 –file-rw-ratio=4 --file-block-size=16384 --max-requests=0 run
Number of threads: 16
Extra file open flags: 0
29 files, 10Gb each
290Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 4.00
Calling fsync() after each write operation.
Using synchronous I/O mode
Doing random r/w test
Operations performed: 2028938 Read, 507238 Write, 507238 Other = 3043414 Total
Read 30.959Gb Written 7.7398Gb Total transferred 38.699Gb (396.26Mb/sec)
25360.46 Requests/sec executed
Test execution summary:
total time: 100.0051s
total number of events: 2536176
total time taken by event execution: 1594.7509
per-request statistics:
min: 0.01ms
avg: 0.63ms
max: 17.41ms
approx. 95 percentile: 1.84ms

结论

• 更多的线程 = 更多的吞吐量
  1个线程：117.14MB/sec
  8个线程：338.71 MB/s ( 2.89x )
  16个线程：396.26 MB/s ( 3.41x )
• 更多的线程 = 更长的响应时间
  1个线程：平均0.13ms , 95% 0.16ms
  8个线程：平均0.37ms , 95% 1.77ms
  16个线程：平均0.63ms , 95% 1.84ms

Sysbench – OLTP

OLTP工作负载：

 --oltp-tables-count：模拟多个表(Percona扩展)
 --oltp-test-modes： [complex] 
    simple : 非常简单的查询模式，PK查找
    complex : 事务模式
    nontrx : 非事务查询模式
 --oltp-read-only：on为只读测试，off为读写混合模式[off]
 --oltp-skip-trx：忽略BEGIN/COMMIT语句[off]

延迟：

--oltp-reconnect-mode : [session] 
    session : 从不断开
    query : 每次查询后断开
    transaction : 每次事务后断开
    random : 随即地
--oltp-connect-delay：在连接数据库后延迟的时间（毫秒）[10000] 
--oltp-user-delay-min：每次请求后延迟的最小时间（毫秒）[0]
--oltp-user-delay-max：每次请求后延迟的最大时间（毫秒）[0]

例子1：表数量4，线程数1

#--test：指定Lua脚本
#--oltp-table-size：指定表的大小，即表的行数
$ sysbench --test=tests/db/oltp.lua --oltp-tables-count=4 --oltp-table-size=500000 --oltp-test-mode=simple run
Number of threads: 1
OLTP test statistics:
queries performed:
read: 140000
write: 40000
other: 20000
total: 200000
transactions: 10000 (217.87 per sec.)
deadlocks: 0 (0.00 per sec.)
read/write requests: 180000 (3921.61 per sec.)
other operations: 20000 (435.73 per sec.)
General statistics:
total time: 45.8995s
total number of events: 10000
total time taken by event execution: 45.7667s
response time:
min: 2.62ms
avg: 4.58ms
max: 1273.69ms
approx. 95 percentile: 4.86ms
Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 45.7667/0.00

例子2：表数量16，线程数16

$ sysbench --test=tests/db/oltp.lua –oltp-tables-count=16
--oltp-table-size=500000 --oltp-test-mode=simple –num-threads=16 run
Number of threads: 16
OLTP test statistics:
queries performed:
read: 140000
write: 40000
other: 20000
total: 200000
transactions: 10000 (777.78 per sec.)
deadlocks: 0 (0.00 per sec.)
read/write requests: 180000 (13999.96 per sec.)
other operations: 20000 (1555.55 per sec.)
General statistics:
total time: 12.8572s
total number of events: 10000
total time taken by event execution: 204.7219s
response time:
min: 3.07ms
avg: 20.47ms
max: 178.74ms
approx. 95 percentile: 44.42ms
Threads fairness:
events (avg/stddev): 625.0000/10.07
execution time (avg/stddev): 12.7951/0.01

例子3：更复杂的测试

#--oltp-point-selects：在一个事务里面Point select的数量，默认为10
#--oltp-range-size：range查询的范围大小，默认100，应该小于oltp-table-size
#--oltp-index-updates：在单个事务中index update的数量，默认1
#--max-time：限制最大的整体执行时间（秒）
$ sysbench --test=sysbench/sysbench/sysbench/tests/db/oltp.lua --oltp-tables-count=4 --oltp-table-size=500000 --oltp-test-mode=complex –mysql-user=root --mysql-password=password --num-threads=16 --oltp-point-selects=1000 --oltp-range-size=1000 --oltp-index-updates=1000 --max-time=300 run
Number of threads: 16
OLTP test statistics:
queries performed:
read: 5803120
write: 2015732
other: 7585
total: 7826437
transactions: 1805 (5.98 per sec.)
deadlocks: 3975 (13.18 per sec.)
read/write requests: 7818852 (25916.23 per sec.)
other operations: 7585 (25.14 per sec.)
General statistics:
total time: 301.6972s
total number of events: 1805
total time taken by event execution: 4815.5190s
response time:
min: 383.54ms
avg: 2667.88ms
max: 18964.65ms
approx. 95 percentile: 6059.93ms
Threads fairness:
events (avg/stddev): 112.8125/6.89
execution time (avg/stddev): 300.9699/0.48

查询的类型

• 单记录查询
  SELECT c FROM sbtest WHERE id=N
• 索引列UPDATE
  UPDATE sbtest SET k=k+1 WHERE id=N
• 非索引列UPDATE
  UPDATE sbtest SET c=N WHERE id=M
• 范围查询
  SELECT c FROM sbtest WHERE id BETWEEN N AND M
• SUM()范围查询
  SELECT SUM(k) FROM sbtest WHERE id between N and M
• ORDER BY范围查询
  SELECT c FROM sbtest WHERE id between N and M ORDER BY c
• DISTINCT范围查询
  SELECT DISTINCT c FROM sbtest WHERE id BETWEEN N and M
  ORDER BY c