The Truck Backer-upper Problem - Design of Experiments for Reinforcement Learning - page 118

Table 6.6 Kriging metamodel statistics for the TBU domain.
Subregion # No. points p conv y min y med y max RMSE MAE RMSE loo MAE loo R 2 R adj Q 2

12 145 0.595 50.7 59.0 65.0 1.090 0.829 1.490 1.180 0.906 0.902 0.826

25 156 0.641 54.0 58.6 62.2 0.693 0.504 0.800 0.605 0.784 0.759 0.712

31 198 0.76 51.9 58.9 62.5 0.867 0.658 0.978 0.749 0.837 0.832 0.793

32 180 0.683 51.0 58.6 62.3 0.813 0.627 1.110 0.859 0.898 0.895 0.811

35 178 0.739 49.0 57.9 62.4 1.040 0.803 1.430 1.120 0.871 0.866 0.758

63 183 0.711 57.6 61.2 65.6 0.978 0.750 1.140 0.891 0.451 0.432 0.260

70 167 0.653 57.4 61.0 66.2 0.975 0.761 1.270 1.030 0.662 0.628 0.427

The sensitivity indices (Fig. 6.6 ) generally show consistent results across all
three sensitivity methods for each convergent subregion and for each parameter (see
Sect. 4.2.6 for a description of the sensitivity indices). The sensitivity indicies of
subregions 12, 25, 31, 35, and 63 all indicate that ʳ is the most important parameter,
and all other parameters have nearly negligible importance. Subregion 63 shows that,
while ʳ is also the most important parameter, the number of hidden nodes, ʻ , and
are also somewhat important. Subregion 70 shows that ʱ ratio , ʻ , ʳ , and all have
modest importance.
From the FANOVA graphs (Fig. 6.7 ), we again see that ʳ overwhelmingly has the
most dominant influence with respect to the performance in most of the subregions.
The interaction structure between variables, however, is quite different across the
subregions. In some subregions (12, 31, 32, and 35), ʳ also has a large two-way
interaction, though the other variables that participate in this interaction are different.
Additionally, subregions 25 and 70 don't include ʳ in the main two-way interaction
at all.
Figures 6.8 and 6.9 show 2-dimensional projections of the response surface from
the metamodel for each of the convergent subregions. The most distinguishing feature
of most these surface projections are the stark qualitative differences for the parameter
ʳ compared to all other parameters. The projections along the ʳ dimension have a
severe and relatively quick transition from high values to low values, and this holds
when ʳ is compared to any other parameter. All other parameters have relatively flat
response surfaces, with some peaks and valleys, and that are all generally around
the median response value. These observations hold for domains 12, 25, 31, and
32 (Fig. 6.8 a-d) and domain 35 (Fig. 6.9 a). The visual significance of ʳ in these
plots for the shape of the response surface is also consistent with the findings of the
sensitivity analysis (Fig. 6.6 ) and the FANOVA plots (Fig. 6.7 a-e).
However, domains 63 and 70 (Fig. 6.9 b-c) are qualitatively different than the
other domains because the sharp transition of the response surface for ʳ is absent.
These surfaces show oblique transitions between high and low valued regions across
a number of parameters. Additionally, the surfaces along the ʱ mag dimension are
generally more complex and less smooth than along other dimensions. The impor-
tance of other parameters in these subregions is also supported by their sensitivity
indices (Fig. 6.6 ) and the FANOVA plots (Fig. 6.7 f-g).