Other Related Topics - Automatic Generation of Combinatorial Test Data - page 83

8.3 Test Case Prioritization and Test Suite Reduction
When the strength t is high, a combinatorial test suite achieving t -way coverage can
be very large. The tester may not be able to run the entire test suite, due to time or
budget constraints. Then, it is essential to prioritize the test cases [ 4 ].
To address the problem, Bryce and Colbourn [ 5 ] proposed an algorithm, which
tries to maximize the number of t -tuples covered by the earlier tests so that if a
tester only runs a partial test suite, he/she will be able to test as many t -tuples as
possible. This work is extended in [ 6 ]. The authors note that “minimum test suite
size and lowest expected time to fault detection are incompatible objectives”. They
differentiate between two goals, which are not so compatible: constructing a test suite
of the smallest size versus covering as many t -way interactions as possible in the
earliest tests. They developed a hybrid approach to test generation so as to reduce the
expected time to fault detection. In this approach, a fast but effective greedymethod is
adopted to produce an initial candidate, and then it is modified repeatedly to improve
the coverage. The test suite is constructed one test at a time while attempting to
maximize the number of t -way interactions covered by the earliest tests.
Normally, a tester executes the whole test suite in sequence. But sometimes we
need to reorder the test cases for some purposes. For example, some time and effort
is needed when the tester changes from one test case to another. This is called the
switching cost . Recently, several authors [ 7 , 8 ] proposed methods for minimizing
the switching cost while generating the covering array. On the other hand, Wu et al.
[ 9 ] presented algorithms for prioritizing a given test suite, so as to minimize the total
switching cost.
For some applications, using a complete new test suite might be too expensive,
because the cost of implementing a test case is high (e.g., when it requires a large
amount of specific data). To alleviate this problem, Blue et al. [ 10 ] proposed a method
called Interaction-based Test-Suite Minimization (ITSM), to obtain a desired test
suite from existing test cases. It is applicable when the company already has many
test cases, which are quite extensive and representative. Instead of constructing a
new test suite from scratch, ITSM reduces an existing test suite, while preserving its
interaction coverage.
To use ITSM, one still needs to specify the parameters of the SUT and their values,
but it is not necessary to define the constraints. Of course, there should also be a test
suite. ITSM tries to select a subset of the test suite that preserves its t -combination
coverage. ITSMdoes not guarantee full interaction coverage. The quality of the result
depends on the existing test suite. But it may save a lot of effort when applicable. It
can still be considered as a test generation method.
8.4 Fault Localization
Testing and debugging are different activities in the software development process,
but they are closely related. After several bugs (defects) have been detected during
testing, they need to be fixed. This bug-fixing process is called debugging. Typically,