Methodology - Design of Experiments for Reinforcement Learning - page 82

x 2

−2

228 228

2

223 223

113 113

x 1

−3 3 228 228

223 223

113 113

Fig. 4.5 Parameter ranges of the convergent subregions of the 2-D camelback function. For each
variable or dimension of the domain, this plot shows the extent and location of each of the convergent
subregions.

Table 4.4 Mean convergent subregions boundaries ( 1 standard deviation) for the 2-D camelback

function for 20 separate replications of the sequential CART procedure.
Subregion

±

Count x y

13 (1) 19 [ − 0.654 ± 0.154, 0.639 ± 0.0978) [ − 0.977 ± 0.108, 1.030 ± 0.106)

23 (2) 4 [ − 1.900 ± 0.0591, − 1.280 ± 0.131) [0.460 ± 0.0476, 0.996 ± 0.032)

± ± − ± − ±

28 (3) 7 [1.350 0.213, 1.870 0.0701) [ 0.949 0.010, 0.488 0.099)

serve as a surrogate model that is computationally quick to evaluate and that has
a similar response to the true response of a computationally intensive simulation,
such as finite element models, computational fluid dynamics models, partial differ-
ential equation solvers that rely on Monte Carlo methods, or even partially converged
simulations. Such models may be either deterministic or stochastic, where a deter-
ministic model will output the same response for a set of input parameters, and
where a stochastic model will have an output that is variable for a fixed set of input
parameters.
In this section, we describe kriging, one type of metamodeling that has been
successful in approximating both deterministic and stochastic computer simulations.
We begin with an overview of deterministic kriging, and then cover stochastic kriging,
which is used later in this work to approximate the response surface of reinforcement
learning in two domains.

4.2.1 Kriging

Kriging, also known as spatial correlation modeling, is a type of metamodel that
was initially developed for geospatial modeling by Krige ( 1951 ) for approximating