Testing Strategy - Software Testing: Testing Across the Entire Software Development Life Cycle - page 69

correctly to get the expected results ?” If the answer is “yes” to both questions, then
it is the developer who must resolve the failed test as either a specifi cation error or
an implementation error.
Black box testing is also referred to as “behavior” testing because of the domi-
nance of expected/actual results analysis for this technique. We will examine these
black box techniques in a subsequent chapter. You just need a general appreciation
now for what black box testing is in order to do your test planning.
4.2.4 PerformanceTesting
Performance testing is that kind of software testing you can do once the software has
been shown to operate correctly. The testing emphasis changes from correctness to
response time and throughput. This change in emphasis occurs at some point near the
end of the development process regardless of the kind of development (custom, soft-
ware for sale) being done. Off-the-shelf software products are certainly candidates
for performance testing validation, especially because most software vendors adver-
tise the performance you should expect from their software under various operating
conditions. You do test the performance of software products you buy, don't you ?
A team of experienced testers is the best group to plan and execute performance
tests. The tester is really the only development team member who understands and
can execute performance tests correctly. Performance testing is complex and re-
quires a number of testing skills that are usually found in a team of specialized
testers rather than one individual. A testing environment with performance test tools
must be created. The maximum mix (peak workload) of transactions (how many
of which kind) for performance testing must be determined with the assistance of
business experts. The system under test must be started empty and brought up to the
maximum mix of transactions in some controlled way. Measurements are taken on
the transaction responses individually and collectively during the maximum mix
execution. The system under test is allowed to idle down gracefully. When a perfor-
mance test fails, that is, transactions run too slow or not enough transactions com-
pleted per unit of time, the tester needs to provide the stakeholder and development
team with the results so that corrective action can be determined and taken.
Here is another departure for performance testing. With white box and black
box testing, the system behavior is tightly associated with particular business activi-
ties that, in turn, are dictated by that business (accounting principles, supply chain
principles, sales principles, and so forth).
So a defect found by white box or black box testing strongly implies program-
m ing cor rections. With per for ma nce testing, the system's response time a nd th rough-
put are dictated more by the competition targeted by the system's business. Examples
of competition targets would be cash register response times for a retail store, gas
pump authorization response times for an auto gas station, or web page rendering
response time for internet sales companies.
When performance tests indicate that the application will not be as fast as required, the
fi rst suggested solution from the development is usually a “quick fi x” such as buying faster
hardware, buying more memory (more cache for i/o), buying more network bandwidth, or