Quality Assurance in the Presence of Variability - Intentional Perspectives on Information Systems Engineering - page 319

The benefit of this approach is that existing techniques from single-system devel-
opment can be used as they are. In order to implement the sample strategy, the
following challenges have to be faced:
1. Selection of representative sample products is necessary. The sample products
should be chosen in such a way that checking those sample products will allow
drawing conclusions about the overall quality of the software product line.
2. Keeping the number of selected sample products manageable. The number of
sample products should be kept as small as possible while guaranteeing a rep-
resentative coverage of the software product line. Otherwise, the effort for
checking the sample products will be infeasible and several redundant checks
(e.g. due to commonalities) have to be performed.
3.3 Comprehensive Strategy
The comprehensive strategy aims at ensuring the quality of all potential products
of the software product line. A “brute-force” realization [ 19] of the comprehensive
strategy could be as follows:
1. Bind the variability in the reusable artifacts for each of the potential products of
the software product line.
2. Apply techniques from the development of single systems to the derived artifacts
of each of these products.
The comprehensive strategy is the strategy that leads to the best coverage of the
domain artifacts. Although the sample strategy (see the previous sub-section) allows
checking all variants of the software product line by determining representative sam-
ple products, those variants are not checked in all potential reuse contexts, i.e. they
are not checked for all products of the software product line. As an example, it may
be the case that v 1 and v 2 have been checked in one sample product and v 2 and
v 3 in another sample product and have not exposed any failures, while a product
which contains v 1 and v 3 might fail, for instance, due to some undesired feature
interactions [ 16] .
Obviously, the number of potential products in a software product line of
industry-relevant size prevents any “brute-force” approach from being used for
realizing the comprehensive strategy in practice. We illustrate this with a simple
example. Assume a set of reusable artifacts that contains 10 variation points each
with 2 optional variants. Approximately 1 million possible software products can be
derived from these artifacts if there are no further constraints for combining the vari-
ants (20 independent variants lead to 2 20 possible combinations). Industry reports on
software product lines with up to tens of thousands of variation points and variants
(see [ 4, 14] ) which leads to a much larger set of possible products.
Thus, a significant challenge for realizing the comprehensive strategy is the ques-
tion how to deal with the complexity that is involved in checking all potential