Java Reference
In-Depth Information
Figure 12.5. Results Biased by Dynamic Compilation.
Code may also be decompiled (reverting to interpreted execution) and recompiled for various
reasons, such as loading a class that invalidates assumptions made by prior compilations, or
gathering sufficient profiling data to decide that a code path should be recompiled with dif-
ferent optimizations.
One way to to prevent compilation from biasing your results is to run your program for a
long time (at least several minutes) so that compilation and interpreted execution represent
a small fraction of the total run time. Another approach is to use an unmeasured “warm-up”
run, in which your code is executed enough to be fully compiled when you actually start
timing. On HotSpot, running your program with
-XX:+PrintCompilation
prints out a
message when dynamic compilation runs, so you can verify that this is prior to, rather than
during, measured test runs.
Running the same test several times in the same JVM instance can be used to validate the
testing methodology. The first group of results should be discarded as warm-up; seeing in-
consistent results in the remaining groups suggests that the test should be examined further
to determine why the timing results are not repeatable.
The JVM uses various background threads for housekeeping tasks. When measuring multiple
unrelated
computationally intensive activities in a single run, it is a good idea to place ex-
plicit pauses between the measured trials to give the JVM a chance to catch up with back-
ground tasks with minimal interference from measured tasks. (When measuring multiple re-
lated activities, however, such as multiple runs of the same test, excluding JVM background
tasks in this way may give unrealistically optimistic results.)
