Analyzing and Interpreting Test Results - Software Testing: Testing Across the Entire Software Development Life Cycle - page 201

12.9 PUTTING TEST RESULTS IN PERSPECTIVE
The analysis of testing results can benefi t a development project in two different
ways. The fi rst and most obvious way is to track individual defects to correction.
This tracking provides the development manager with the number of defects discov-
ered, the number of defects corrected, and the number of defects awaiting correction.
From these tracking reports, the development manager can make informed decisions
about the effort and advisability of correcting all known defects before the develop-
ment is declared completed.
The second and less obvious way that test results analysis benefi ts a development
project is by noting trends and trend comparisons that can indicate possible issues with
successful testing completion. These possible issues can take the form of unusually
buggy software components, testing effort that may not keep pace with the develop-
ment effort, and anticipation of numbers of defects that may escape the development
project and be discovered by end users. These analytical results represent additional
management decision inputs available while the software is being developed.
12.10 SUMMARY
This chapter describes the kinds of test execution results you might want to collect,
the ways you might want to consider analyzing these results, and the interpretations
you may place on your analysis outcome.
The fi rst step in discussing test execution results is to assume that you have done a
thorough job of test planning. If you have unintentionally omitted certain application
functions or business activities or structural components or performance aspects of
your application from your test planning, then your test coverage will be inadequate.
One planning key to successful test results analysis is the clear defi nition of suc-
cess for each test case. It is common for a test case to have a number of expected re-
sults. If the actual results obtained from a test execution all match the expected results,
then the test case is normally considered “attempted and successful.” If only some of
the actual results obtained from a test execution match the expected results, then the
test case is normally considered “attempted but unsuccessful.” A test case may be “at-
tempted but unsuccessful” because the actual results do not match the expected results
or because the software halted with an error message before the test case was com-
pleted. Test cases that have not been executed are initially marked “unattempted.”
The incremental discovery and correction retesting of software defects is the
primary way that software testers help software developers implement the develop-
ment requirements. The fewer the latent defects in the delivered software, the closer
the software comes to fulfi lling the requirements. The success of incremental defect
discovery is directly related to the management process used to track defects from
discovery to correction.
Successful incremental defect tracking can provide a third testing metric nor-
mally called the defect backlog. If your testing has discovered 300 software defects
and development has successfully corrected 100 of these defects, then a backlog of
200 defects still need to be corrected. The challenge for the development team is