Other Related Topics - Automatic Generation of Combinatorial Test Data - page 82

Modeling

SUT Model

Test Generation

Faulty Combinations Test Results Test Suite

Fault Localization Test Execution

Faulty combination 1
Faulty combination 2
...

Test case 1 => pass
Test case 2 => fail
Test case 3 => pass
…

Test case 1
Test case 2
Test case 3
…

Test Reduction &
Minimization

Fig. 8.1 Related activities

8.2 Modeling

To use the test data generation methods described in the previous chapters, we should
have a suitable model of the software/system, including the constraints. Currently, in
most cases, the model is constructed by the test designers manually. The model can
be built from the SUT itself, or from the system requirements or design specification.
We assume that there are a number of parameters in the CT model, each of which
can take values from a discrete domain, which is not too large. However, in real
software systems, many input variables have very large ranges of values. Thus, to
use combinatorial testing techniques, we have to build an abstract model using various
other methods, e.g., that based on equivalence classes.
Ghandehari et al. [ 1 ] described a three-step approach to apply CT to real code.
First, an abstract model is created, which consists of abstract parameters and values.
Then, a combinatorial test suite is generated from the abstract model. Finally, concrete
tests are derived from the abstract tests. These concrete tests are used to perform the
actual testing.
They used the Siemens programs to illustrate their approach. These programs
were introduced to the software engineering community by several staff members
of the Siemens corporation [ 2 ], and became famous benchmarks in software testing
and analysis. The programs are relatively small in terms of lines of code, and they
have a small number of input parameters. But the input spaces are quite complex. For
example, one input parameter of the program replace is a regular expression, and we
need to deal with some metacharacters in it. This program has less than 1,000 lines
of code and just 3 input parameters. But its abstract model for combinatorial testing
[ 1 ] contains 20 abstract parameters and 36 constraints.
Instead of analyzing the input parameters of the code, we may also obtain a CT
model from the software design specification. Recently, Satish et al. [ 3 ] presented
an approach to extracting key information (such as parameters and values) of the CT
model from UML sequence diagrams.