Design of Experiments - Design of Experiments for Reinforcement Learning - page 57

those previously described. Analysis of variance is used to analyze experimentally
collected data to test for differences in group means for more than two groups.
ANOVA works by partitioning the observed variance into that which can be explained
(based on the data and an associated regression model) and that which cannot be
explained. Using sum-of-squares decomposition and statistical tests comparing the
explained and unexplained variance, one can determine the significance of model
terms, whether they are single main effects or interaction effects. ANOVA is based
on three assumptions, those being: that the response variable is normally distributed,
that each group has equal variance (i.e., homoscedasticity), and that observations are
independent. It should be noted, however, that truly normally distributed data are
rarely seen in practice, and that ANOVA can still provide useful information with
deviations from the normality assumption. Additionally, the simplest use of ANOVA
requires equal numbers of observations at each factor-level and uses Type I sum-of-
squares, but Type II and III sum-of-squares can be used with unequal numbers of
factor-level observations.
While ANOVA is relatively independent of the experimental design, alternative
design of experiments techniques, such as the response surface methodology (RSM),
use experimental designs that are complementary to the analysis methods.
Response Surface Methods As the name implies, response surface methods are
used to create a surface that approximates the behavior of a response variable based
on a set of continuous and independent factors (Box and Wilson 1951 ; Box 1954 ;
Myers and Montgomery 2002 ). Response surfaces can be used to identify significant
effects and the form of effects, to act as a surrogate model for the system under study,
or to determine a set of operating conditions that produce and optimal response. In the
case of optimization, a preliminary linear model is developed using a simplistic exper-
imental design (e.g., fractional factorial or simplex designs), and this model is used
to direct and guide subsequent experimentation toward the optimal operating point.
Once sufficiently close to the optimum, a second-order model, consisting of linear,
quadratic, and two-way interaction effects, is developed with a more sophisticated
experimental design, such as central composite designs (CCD) or Box-Behnken de-
signs (BBD) (Box and Wilson 1951 ; Box 1954 ; Myers and Montgomery 2002 ). The
approximate optimal operating point can then be determined through canonical anal-
ysis of this second-order model. While response surface methods have been found
to be sufficient for physical and real-world experiments (Box and Draper 1987 ), its
low-order model terms may limit its applicability to more complex response surfaces
(such as from computer simulations) (Sacks et al. 1989 ; Vining 2008 ).
Taguchi Methods Taguchi design of experiment techniques were originally devel-
oped to improve the quality of manufactured goods, but their application has extended
to numerous domains, including biotechnology (Antolín et al. 2002 ), material sci-
ence (Gell et al. 2001 ), and supply chain optimization (Shang et al. 2004 ), amongst
many others. This approach is based on the assumption that only single factors (and
not interactions among multiple factors) have an effect on response variables. Con-
sequently, this drastically reduces the number of experimental runs that are required
to determine the effects of these factors, and this is the main attractiveness of Taguchi