Databases Reference
In-Depth Information
int id = resultSet.getInt(1);
}
resultSet.close();
\\Get finish time
Also, exclude writing output from your benchmark timings. For example,
suppose that your benchmark writes data to a console so that you can verify the
results of each
Select
statement. For example, what if your benchmark includes
the following line of code:
System.Console.WriteLine("Value of Column 2: " +
dataReader.GetInt32(2));
If done once, it may add only a second or two to your benchmark results, but
if done repeatedly, that time can add up, skewing your true results. Make sure
that the console output occurs outside your timing loop.
Design benchmarks so that they measure tasks over a sufficient duration.
Benchmarks that are run over short durations make it difficult to reproduce
meaningful and reliable results for the following reasons:
•
They produce results that often do not scale. In most cases, you cannot
extrapolate the results from a short duration and apply them to the larger
context of your application.
•
Computer system clocks, used to time benchmark runs, are notoriously
imprecise because of design limitations, temperature changes, and dimin-
ished battery voltage over time. In fact, time kept by computer system clocks
can fluctuate from the real time as much as several minutes a day. If a bench-
mark is run over a short duration, perhaps 10 seconds or less, the drift
caused by a system clock can produce inconsistent results.
•
Factors such as Java class loaders and the .NET Just-in-Time (JIT) compiler
cause application start-up performance costs that skew performance results
over short durations.
For example, suppose you want to measure the throughput of an application
that retrieves 1,000-byte rows from a database table containing a million rows.
First, the benchmark is run over 5 seconds, resulting in a throughput of 5 rows
per second. What if another short-term process running in the background
caused a “blip” in the system during that 5 seconds? You could run the same
benchmark a second time for 5 seconds, and the outcome may result in a com-
pletely different metric—for example, 10 rows per second, which is a huge vari-
ance on this scale.
