Design of Experiments - Design of Experiments for Reinforcement Learning - page 60

design that attempts to uniformly fill the design space by minimizing a discrep-
ancy measure that is based on the distribution of the design points (Pronzato and
Müller 2012 ). These designs are based on low-discrepancy random number se-
quences, which have the quality that the number of points falling into an arbitrary
portion of the design space is proportional to a measure of this design subspace.
Such sequences are often based on quasi-random numbers, which can be constructed
sequentially (Pronzato and Müller 2012 ), and which result in a more uniformly dis-
tribution than conventional pseudo-random numbers (Tuffin 1998 ; Caflisch 1998 ).
The improved sampling uniformity ultimately reduces the total computational ex-
pense of Monte Carlo simulations because quasi-random sequences converge faster
than pseudo-random sequences. Examples of low-discrepancy sequences include
the Halton (Halton 1960 ), Hammersley (Hammersley 1960 ), Sobol' (Sobol' 1967 ),
Faure (Faure 1982 ), and Niederreiter (Niederreiter 1992 ) sequences. While quasi-
random sequences were largely developed for Monte Carlo integration (Niederreiter
1992 ), their use in experimental design and statistics has been growing (Fang and
Wang 1994 ; Chen et al. 2006 ).
Statistical Form of Metamodels The statistical form of the metamodel can vary
greatly depending on experiment and response characteristics, the desired informa-
tion from the metamodel, and the desired use of the metamodel. Here, we briefly
describe some types of models that have been used for metamodeling. While conven-
tional response surface models were largely developed for physical experiments, they
have been applied to computer experiments. However, their success is rather mixed,
and this may be due to their low-order model, which assumes that the response
surface is relatively smooth when in fact computational models often have more
complex response surfaces (Vining 2008 ). Nonetheless, response surface models
may be of use for understanding the behavior of a few ( < 4) variables under specific
circumstances.
Classification and regression trees (CART) create a decision tree-like representa-
tion of the response by recursively partitioning the design space into hyper-rectangles,
with each partition having its own approximation of the response surface (Breiman
et al. 1984 ; Loh 2011 ). These models, however, are not robust to outliers or skewed
response variables (Galimberti et al. 2011 ). Random forests attempt to improve the
robustness of CARTs by using an ensemble of CARTs where the predicted response
values are a consensus value from all of the individual CARTs (Breiman 2001 ).
This approach, however, significantly increases the computational expense of model
creation. Multi-variate adaptive regression splines (MARS) can be thought of as an
extension of CARTs that use piecewise linear surfaces that are splined together to ap-
proximate a complete, and potentially complex, response surface (Friedman 1991 ).
Each piecewise surface is parameterized by knot locations (i.e., the extent of their
domain) and either single variables or interactions between variables.
Artificial neural networks are another approach to metamodeling when they are
used as a regression model, and they are appealing because they can model highly
nonlinear responses. Neural networks are not often used in traditional design of
experiments approaches, possibly due to the disconnect between the design of