Performance Testing - Software Testing: Testing Across the Entire Software Development Life Cycle - page 140

Box B still represents the peak workload response time for 350 active transac-
tions. At the new 350 transaction peak, the worst transaction response time is about
6.8 seconds, still well below the 10 seconds maximum response time (arrow). Know-
ing the knee of the curve is lurking somewhere out there, you extend your peak
workload test to 450 transactions as illustrated with Box C and, in fact, see the knee
of the curve push your transaction response time to 26.4 seconds, well above the
10 seconds maximum response. So you have just discovered that there is no margin
of safety in the 350 peak workload. As soon as something in the business forces
more than 350 transactions active at the same time, the transaction will begin to slow
down unacceptably.
What do you do with this information ? You should discuss your fi ndings with
the development team. They may want to analyze the code for the apparent response
bottleneck, make some programming changes, and attempt to push the knee well
to the right by some agreed safety margin. They may want to cap the system just
to the left of the knee. Because the knee is 350 transactions in this case, the de-
veloper might put a governor on the transaction with a 10% safety margin so that
when 315 transactions have become active, the user attempting to submit the 316th
transaction will get a message like “The system is responding to the maximum num-
ber of catalog browse requests. Please wait a moment and try your request again.”
Regardless of which alternative the developers choose to implement, you will need
to retest their solution to confi rm that the changes keep the system off the knee of
the performance curve.
Recapping your performance testing activities so far, you have developed the
needed peak workload plans; you have successfully ramped your workloads up and
down, and you have run your round trip performance tests for each transaction that
will be in the peak workload mix. The next step is to run the round trip performance
tests with the peak workload mix.

9.5.2 Saturday Peak Workload in an Empty Test System

We return to our performance workload Draft 4 plans to demonstrate what can hap-
pen when you test a workload mix. We will focus fi rst on the workload mix for Sat-
urday purchasing. For simplicity of discussion, we will place just three transaction
groups in our demonstration: log on/log off, purchase steps, and catalog search. We
choose the Saturday workload to mix fi rst because, of the two workload plans, the
Saturday plan requires fewer transactions at the testing peak. This is what we know
about these three transaction groups for the Saturday workload:

Transaction
group

Response
time max

Peak
workload

Log on/log off
Purchase steps
Catalog search

3 seconds
7 seconds
10 seconds

500
500
200