Methodology - Design of Experiments for Reinforcement Learning - page 89

n − 1

R adj = R 2 )

1 − (1 −

n − d − 1

where y i is the i th response value, y is the

mean of the response values. The adjusted- R 2 adjusts the value of R 2 by penalizing
the addition of variables to the model; in this computation, n is the number of data
points in the model, and d is the number of regressors. Additionally, similar met-
rics are computed from a leave-one-out (LOO) cross validation, which also include
RMSE, MAE, and Q 2 , which is a LOO-computed version of the R 2 . LOO cross
validation consisted of removing design points one at a time without re-estimating
the kriging model parameters. The computation of Q 2 is similar to that of R 2 , except
that Q is used to indicate this metric was computed from the LOO analysis, and that
in this computation, the predicted values

y i is the model-predicted response, and

y i are computed from a model that does not
contain observation i . Due to the local dependence on each data point of the kriging
model, the removal of a single data point can severely change the model, and this
is especially so at the location of said data point. It is therefore not surprising if the
values of Q 2 is much lower than the values of R 2 , and that the RMSE and MAE
values are greater for LOO. We use the R 2 and related measures because they assess
the linearity of the predictions relative to the true responses while also considering
the prediction errors; the commonly used alternative measure of Pearson's correla-
tion coefficient merely assesses the linearity of predicted points relative to the true
response values.
We use a series of global sensitivity indices (Sobol' 2001 ; Saltelli et al. 2004 )
to assess the importance of each variable. Global sensitivity analysis decomposes
the variance of the response (i.e., mean number of time steps to reach the goal) and
apportions it to the input parameters. This type of sensitivity analysis differs from
a conventional local sensitivity analysis that is gradient-based. In global sensitivity
analysis, parameters that have greater sensitivity indices have a greater effect on
the variation in the response. We provide a brief review of how Sobol' indices are
computed, from which many sensitivity indices are based on similar computations,
although we direct the reader to the literature, namely Sobol' ( 2001 ) and Saltelli
et al. ( 2004 ), for a more comprehensive review.
Suppose that f (

dž

) is a square-integrable function that is defined over the hypercube
[0, 1] d , where d is the dimensionality of the function. The function can be represented
as a sum of functions:

·

d d

f ( X ) = f 0 + f i ( X i ) + f ij ( X i , X j ) + ... + f 12 ...d ( X )

i = 1 i<j

This decomposition is called an ANOVA decomposition if:

1

f i 1 ...i s x i 1 , ... , x i s dx i k

= 0, 1 ≤ k ≤ s , { i 1 , ... , i s }⊆{ 1, ... , d }

0

The variance of the output Y ( X i , ... , X d ) with mutually in-

dependent variables, can be decomposed as a functional decomposition, also known

= f ( X ), where X =