Information Technology Reference
In-Depth Information
if application response times had remained in the multiple-minute range, the tester
would have been challenged to apply a stopwatch to several servers that would
most likely be dispersed geographically, one of the primary advantages of distrib-
uted computing. The advent of viable Internet business applications has caused the
span of these geographically distributed computing environments to stretch around
the world. Test tools can and do provide millisecond response time measurements
across these widely distributed computing environments without the tester leaving
his chair.
11.6.5 Achievement of Impossible Manual Tests—a
Return on Investment Issue
Choosing to do manual performance testing in preference to tool testing is likened
to going hunting for elephants with a rock instead of a rifl e. Although stopwatch
timing may not be the most precise way to measure an application's response times,
stopwatch timing can still get the job done if no testing tools are available. The true
limitation of this approach resides in the size and complexity of the workload to be
tested as described in Chapter 9.
Consider the situation of a workload to be tested in the range of 10-100 active
business transactions and no test tools available. This situation can be tested manu-
ally by calling in the entire offi ce staff on a Saturday, give them manual scripts to
run, feed them pizza and beer, and have them pound on the system all day. The
results are not particularly precise or repeatable, but the job does get done. The
authors know of large companies who still do “Saturday Special” timing of their
applications.
Any performance workload requirement beyond 100 active business transac-
tions cannot be tested manually. The good news is that performance test tools have
matured to the point that they can test workloads well beyond the 100 transaction
breakpoint for manual testing. It is commonplace to achieve performance tool test-
ing of workloads in the thousands of transactions. Some of the more specialized
Internet performance testing tools can drive workloads of 500,000 transactions
or more.
11.6.6 Reusability (Regression Testing)—a Return on
Investment Issue
Test tools have been designed from the beginning to easily and reliably reuse test
scripts. As testing processes have matured, this reusability feature has greatly en-
abled the concept and practice of regression testing described in Chapters 7 and 8.
If the tester manages and maintains all test scripts produced for the fi rst ver-
sion of an application, then this set of test scripts for Version 1 can be applied to
an early copy of Version 2. Code untouched by Version 2 should pass the Version 1
script tests. Code touched to accommodate Version 2 changes should also pass the
Version 1 script tests. New code for Version 2 will require test script additions or
Search WWH ::




Custom Search