STATISTICAL TECHNIQUES IN SOFTWARE SIX SIGMA AND DESIGN FOR SIX SIGMA (DFSS) - Software Design for Six-Sigma: A Roadmap for Excellence - page 141

such experiments and methods of analysis provide the DFSS team with insight, data,
and necessary information for making decisions, allocating resources, and setting
optimization strategies.
An experimental design is a plan that is based on a systematic and efficient ap-
plication of certain treatments to an experimental unit or subject, an object, or a
source code. Being a flexible and efficient experimenting platform, the experimenta-
tion environment (hardware or software) represents the subject of experimentation at
which different treatments (factorial combinations) are applied systematically and ef-
ficiently. The planned treatments may include both structural and parametric changes
applied to the software. Structural changes include altering the type and configuration
of hardware elements, the logic and flow of software entities, and the structure of the
software configuration. Examples include adding a new object-oriented component,
changing the sequence of software operation, changing the concentration or the flow,
and so on. Parametric changes, however, include making adjustments to software
size, complexity, arguments passed to functions or calculated from such functions,
and so on.
In many applications, parameter design is more common in software experimental
design than that of structural experimental design. In practical applications, DFSS
teams often adopt a certain concept structure and then use the experimentation to
optimize its functional requirement (FR) performance. Hence, in most designed
experiments, design parameters are defined as decision variables and the experiment
is set to receive and run at different levels of these decision variables in order to study
their impact on certain software functionality, an FR. Partial or full factorial design
is used for two purposes:
- Finding those design parameters (variables) of greatest significance on the sys-
tem performance.
- Determining the levels of parameter settings at which the best performance level
is obtained. Direction of goodness (i.e., best) performance can be maximizing,
minimizing, or meeting a preset target of a functional requirement.
The success of experimental design techniques is highly dependent on providing
an efficient experiment setup. This includes the appropriate selection of design param-
eters, functional requirements, experimentation levels of the parameters, and number
of experimental runs required. To avoid conducting a large number of experiments,
especially when the number of parameters (a.k.a. factors in design of experiment
terminology) is large, certain experimental design techniques can be used. An exam-
ple of such handling includes using screening runs to designate insignificant design
parameters while optimizing the software system.
Experimental design, when coupled with software available testing tools and
techniques, is very insightful. An abundance of software testing tools exist. The
correctness testing tools often are specialized to certain systems and have limited
ability and generality. Robustness and stress testing tools are more likely to be
made generic. Mothora (DeMillo, 1991) is an automated mutation testing tool set