Methodology - Design of Experiments for Reinforcement Learning - page 78

−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3

x 1 x 1

a Deterministic function. b Noisy function.

Fig. 4.2 Deterministic ( a ) and noisy ( b ) versions of the 6-hump camelback function.

Table 4.2 Parameter settings
for sequential CART
modeling and CART models.

Number of variables 2

Initial number of design points 40

Replications per design point 4

Number of sequential CART iterations 7

Number of design points per convergent subregion 20

Sample design method LHS

Minimum design points per split a

y

0 . 10

Minimum design points per leaf a 0 . 05 y

Minimum complexity change per split 0.01

Maximum number of surrogates 5

Proportion of low points per leaf ( ˆ ) 0.8

Convergence proportion threshold ( ʸ ˆ ) 0.75

a y is the response vector for the current iteration

testing and development of the sequential CART algorithm in each of the problems.
At this point, we do not claim that these settings are ideal, and future work should
explore the effects of each of these settings in a variety of domains. However, we do
find that most of the settings used in this example problem are able to successfully
find convergent parameter subregions.
An initial sample size of 40 points (20 × number of variables) were sampled

over x 1 ∈ 2, 2] using Latin Hypercube sampling (LHS), with

4 replications at each design point, resulting in a total of 160 initial design points.
Sequential CART modeling was run for 7 iterations following the initialization design

[ − 3, 3] and x 2 ∈ [ −