Analyzing and Interpreting Test Results - Software Testing: Testing Across the Entire Software Development Life Cycle - page 199

Visually, the Rayleigh curve shows you that when the testing team doubles its
effort, they do indeed “push the envelope” or fi nd more defects in the software than
normally expected at that time in the project based on the Rayleigh formula. The
fact that the prior project curve then falls below the Rayleigh curve shows you that
the follow-up testing effort may have been less effective than expected at fi nding the
defects also based on the Rayleigh formula.
Now that you have a sense of what the curve comparison tells you, look at
the total defects discovered by each curve. The prior project discovered a total of
10,682 defects. The Rayleigh curve formula predicts 12,328 defects, 15% more
defects than the prior project found. The next question you ask is, “How can we
fi nd 15% more defects with our testing ?” You may fi nd one area of the soft-
ware that test planning consistently misses across the projects like security or
backup/recovery that could account for the majority of the 15% missed defects.
You may fi nd several areas of the software that test planning consistently misses
across the projects that collectively begin to approach the missing 15% of defect
discoveries. You may fi nd that your test planning is very comprehensive, but the
way you execute some of your tests may allow some defects to go undiscovered.
Finally, you may fi nd that your testing is planned well and executed well, leading
you to conclude that the Rayleigh curve is simply not predictive for your projects.
The purpose of the Rayleigh curve comparison is to prompt you to ask probing
questions about your testing effectiveness, then act on any fi ndings to improve
your testing.
Finally, we need to show you one example of a kind of prior project de-
fect curve that defi es Rayleigh curve comparison from the outset. Instead of
looking like Figure 12.15, assume that your prior project defect curve looks like
Figure 12.18.

Figure 12.18 Prior development project defect tracking log with a gap in defect discovery