Analyzing and Interpreting Test Results - Software Testing: Testing Across the Entire Software Development Life Cycle - page 200

The challenge in comparing the Rayeigh curve to these results is the accu-
rate placement of the gunsight. Instead of a clearly defi ned peak, defect discovery
dropped off dramatically between weeks 9 and 13. The proper location of the gun-
sight becomes problematic in the area of the dotted box.

12.8 MORE DEFECT TRACKING METRICS

It is worthwhile to take inventory of the metrics that we have found to be helpful in
analyzing defect discovery trends and prioritizing defect backlogs.

Development metrics discussed and demonstrated

1.
2.
3.
4.
5.

unique development defect identifi er
development defect discovery date
development defect severity
development defect correction date
development defect correction code earmark

HelpDesk metrics discussed and demonstrated

1.
2.
3.
4.
5.

unique customer defect identifi er
customer defect discovery date
customer defect severity
support defect correction date
support defect correction code earmark

These metrics come from four sources: the test team, the development team, the
HelpDesk team, and the software support team. As simple as this list of metrics ap-
pears, the effort to establish uniform collection and reporting of this list is intensive
and nontrivial. Many software organizations think that the value of the results do not
justify the effort. Consider a small pilot project to establish the value of these metrics
to your organization. Then make an informed decision about continuing the metrics
collection and analysis on a progressively larger scale.
Some software organizations have proven to themselves that the value of the
results do justify extending the list of metrics further, thereby adding effort but also
providing additional analysis opportunities. One of the more successful techniques
for adding to the above list of metrics is IBM's Orthogonal Defect Classifi cation
which is called “ODC.” [50] ODC is the result of IBM software defect research
started around 1994 and has become a mature technique. ODC suggests adding two
more metrics to the defect log that the testing team provides at defect discovery time
and four more at defect correction time. IBM has developed a complementary analy-
sis and reporting tool called JMYSTIQ ( Java—managing your software to improve
quality ) to help the ODC user analyze this formidable list of metrics over thousands
of defects. Expect further development of tools and techniques in software defect
analysis, a rich area of practical research that just might produce our silver bullet.