Benchmarking MySQL - High Performance MySQL - page 52

MySQL Benchmark Suite (sql-bench)
MySQL distributes its own benchmark suite with the MySQL server, and you can
use it to benchmark several different database servers. It is single-threaded and
measures how quickly the server executes queries. The results show which types
of operations the server performs well.
The main benefit of this benchmark suite is that it contains a lot of predefined tests
that are easy to use, so it makes it easy to compare different storage engines or
configurations. It's useful as a high-level benchmark, to compare the overall per-
formance of two servers. You can also run a subset of its tests (for example, just
testing UPDATE performance). The tests are mostly CPU-bound, but there are short
periods that demand a lot of disk I/O.
The biggest disadvantages of this tool are that it's single-user, it uses a very small
dataset, you can't test your site-specific data, and its results might vary between
runs. Because it's single-threaded and completely serial, it will not help you assess
the benefits of multiple CPUs, but it can help you compare single-CPU servers.
Perl and DBD drivers are required for the database server you wish to benchmark.
Documentation is available at http://dev.mysql.com/doc/en/mysql-benchmarks
.html/ .
Super Smack
Super Smack ( http://vegan.net/tony/supersmack/ ) is a benchmarking, stress-testing,
and load-generating tool for MySQL and PostgreSQL. It is a complex, powerful
tool that lets you simulate multiple users, load test data into the database, and
populate tables with randomly generated data. Benchmarks are contained in
“smack” files, which use a simple language to define clients, tables, queries, and
so on.
Database Test Suite
The Database Test Suite, designed by The Open Source Development Labs (OSDL)
and hosted on SourceForge at http://sourceforge.net/projects/osdldbt/ , is a test kit
for running benchmarks similar to some industry-standard benchmarks, such as
those published by the Transaction Processing Performance Council (TPC). In
particular, the dbt2 test tool is a free (but uncertified) implementation of the
TPC-C OLTP test. We used to use it a lot, but we have developed purpose-built
tools for MySQL that we now use instead.
Percona's TPCC-MySQL Tool
We have created a fair-usage implementation of a benchmark similar to the
TPC-C test, with tools specifically designed for benchmarking MySQL. This is the
tool we use most often for evaluating how MySQL behaves on nontrivial work-
loads. (For simpler benchmarks, we usually use sysbench instead.) The source code
is available at https://launchpad.net/perconatools , and there is brief usage docu-
mentation in the source repository.