Performance Testing - Software Testing: Testing Across the Entire Software Development Life Cycle - page 147

the required maximums before any transaction mixing is attempted. If all
transactions pass this first hurdle, then systematic mixing of workloads is at-
tempted until all mixes are successful or until the testing hits a showstopper.
The showstopper/correction/regression/retest cycle is repeated until the re-
quired performance is achieved. Rather than using more charts to demonstrate
what might happen with the weekday workload testing, we will discuss the out-
comes you might expect and why you might expect them. The associated charts
should leap to mind.
The most noticeable difference between the Saturday and weekday work-
loads is the volume of work, almost four times more transactions. This would
lead you to expect new volume-related issues to be revealed by the weekday
performance testing. For example, we learned that the logon/logoff code was de-
signed to operate primarily from lists (active user ID and password) in memory
as opposed to lists on disk fi les. While it is true that everything runs faster in
memory than from disk, it is also true that memory usually has a much smaller
storage capacity. One possible outcome of this design strategy tradeoff is that
the logon/logoff transaction group could become a memory exhaustion show-
stopper before the 2,000 weekday log on workload is achieved. Buying 2,000
memory chips costing $100 each may be a less desirable solution than revising
the logon/logoff performance maximums to a slower response and revising the
logon/logoff code to maintain some of its user ID/password information on disk
fi les. The tradeoff faced by the application owner is the cost of additional hard-
ware versus the cost of revising the logon/logoff code and attendant slower disk
access performance expectation.
Because we will test fewer purchase step transaction groups during the
weekday workload tests than we did during the Saturday workload tests, the
purchase steps should rightfully be expected to stay below the performance
maximums or at least not exceed them. The catalog browse transaction group
is just the opposite situation from the purchase steps. The weekday workload
will require many more catalog browse transaction groups than the Saturday
workload tests. There is a distinct possibility that the catalog browse plot will
reveal a curve knee before its workload peak is achieved. Aided by performance
test results, the developers should be able to make the corrections that will ul-
timately allow the software to pass the weekday workload performance tests as
well.
9.6 PUTTING PERFORMANCE TESTING
IN PERSPECTIVE
Software performance is the ugly stepchild of software functionality. Both de-
velopers and testers spend signifi cant time and effort to ensure that the required
software functionality is delivered. Many development teams have discovered
that if the software delivers correct functionality too slowly, the end-user is just