Java Reference
In-Depth Information
Take baseline measurements.
Before you attempt anything else, you
must measure the system. The baseline will show either progression or
regression in subsequent tests. The baseline measurements should
include enough detail so that you can isolate problems to individual
components. Code profilers can assist with this process. Other tools,
such as Page Detailer, can break down the individual load times for
objects on a web page. The point is to have enough information to
spot the bottlenecks.
4
Test with repeatable scenarios.
Testing with repeatable scenarios makes
it much easier to understand when a change has improved the perfor-
mance of the system.
5
Work on the most important scenarios first.
This should go without say-
ing, but many programmers waste valuable time on meaningless
administrator functions or unimportant error conditions, and ignore
the critical test cases.
6
Work on the bottlenecks with the highest reward/cost ratio.
Again, it
should go without saying that many smaller low-cost, medium-reward
fixes can often improve the system as much as a high-reward fix with a
high cost.
7
Quit when you are done.
Performance testing can go on indefinitely if
you let it. Establish criteria that will make your customers happy and
quit once you've satisfied those criteria.
8
Once a system is in production, do not stop measuring and tuning.
Pro-
duction systems will probably have different behavior from sterile
tests. It's difficult to predict how caching models will hold up, how
workloads will be distributed, or how end users will use a system. You
should periodically measure your production systems and, where pos-
sible, tune the systems with information learned from the sterile test
environment. If a system must be tuned in production, it's important
to keep the methodology strong; change one variable at a time, and
measure between changes. For production systems, response time in a
vacuum is not enough, since the workload can also change. Other
metrics, such as page views per second, may be more valuable.
9
If necessary, get help.
Because Internet performance tuning can be such
a demanding discipline and is becoming more specialized, it pays to
spend a little money up front to prevent a long-term disaster.
10
