Methodology - Design of Experiments for Reinforcement Learning - page 80

−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3

x 1 x 1 x 1

a b c

Iteration 0 Iteration 1 Iteration 2

−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3

x 1 x 1 x 1

d e f

Iteration 3 Iteration 4 Iteration 5

−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3

x 1 x 1

g

h

Iteration 6 Iteration 7

Fig. 4.3 Sequential CART modeling of the 2-D 6-hump camelback function. Each plot shows
one iteration of the sequential CART process where subregions progressively become smaller. The
underlying deterministic surface is shown in blue and white on each plot, where white indicates
lower values and blue indicates higher values. In each subfigure, the initial bounding subregion is
outlined by a thick black line and the subregions of the CART model that correspond to the low
points ˉ L are outlined in dotted lines . The convergent subregions are outlined with red dotted lines ,
and these regions are not explored in subsequent iterations.

low regions of the function. Table 4.4 shows the resulting parameter bounds for these
subregions. The count column indicates in how many (of the 20 replications) that
particular subregion was found, and we can see that the smaller subregions 2 and 3
are found less often than the large subregion 1. In general, we see that the variability,