Overview of Testing - Software Testing: Testing Across the Entire Software Development Life Cycle - page 18

market. The cost of the needed software development is forecast, usually with some
precision, if the effort is similar to prior software development efforts. The question
typically missed at this juncture is, “What will it cost if the software does not work as
it is advertised?” The unspoken assumption is that the software will work fl awlessly
this time, even though no prior software development has been fl awless. Therefore,
a strong connection should be made early in the process between looking for defects
and avoiding risk.
Testing principle 2: Positive and negative testing contribute to risk reduction.
Positive testing is simply the verifi cation that the new software works as advertised.
This seems like common sense, but based on the authors' experience with software
during the past 20 years, new off-the-shelf software continues to have defects right out
of the package that scream, “Nobody tested me!” There is no reason to expect that
new corporate software systems have a better track record. Similarly, negative testing
is simply the verifi cation that customers can not break the software under normal
business situations. This kind of testing is most often omitted from the software
development because it is more time consuming than positive testing; it requires more
tester creativity to perform than positive testing, and it is not overtly risk-driven.
Testing principle 3: Static and execution testing contribute to risk reduction.
The preponderance of software testing conducted today involves executing the pro-
gram code under development. Functional, structural (nonfunctional), and performance
testing must execute program code to complete the tests. A small but growing number
of testing teams and organizations have awakened to the fact that there are a large num-
ber of documents produced during software development that, if reviewed for defects
(static testing), could signifi cantly reduce the number of execution defects before the
code is written. The corollary statement is that the best programmers in the organization
cannot overcome bad requirements or bad specifi cations by writing good code.
Testing principle 4: Automated test tools can contribute to risk reduction.
As software has become orders of magnitude more complex than the COBOL,
PL/1, or FORTRAN systems of yesterday, new types of business risks have arisen.
These new risks are most often found in the performance area where system response
times and high volumes of throughput are critical to business success. This makes
them impossible to test manually. It is true that performance testing tools are quite
expensive. It is also true that the potential risk due to poor performance can exceed the
cost of the performance test tools by several orders of magnitude. As of 2004, some
companies still consider a performance test that involves calling in 200 employees
on a Saturday, feeding them pizza, and asking them to pound on a new application
all at the same time for several hours. As we will discuss in Chapter 9, this kind of
manual testing has severe limitations, including typically an inadequate number of
employees that volunteer to test (What happens if you need to test 3000 users and
have only 200 employees?) and the nonrepeatability of test results because no one
performs a manual test exactly the same way twice. The last 5 years of automated
performance test tool maturity has prompted the strong consideration of testing tools
to replace other kinds of manual testing when conditions are favorable.