Methodology - Design of Experiments for Reinforcement Learning - page 90

as a functional ANOVA:

d d

Va r ( Y ) = D i ( Y ) + D ij ( Y ) + ... + D 12 ...d ( Y )

i = 1 i<j

where D i ( Y ) = Var[ E ( Y | X i )], D ij ( Y ) = Var[ E ( Y | X i , X j )] − D i ( Y ) − D j ( Y ), and

so on for higher order terms. Sobol' indices can then be computed as:

D i ( Y )
Var( Y ) , S ij

D ij ( Y )
Var( Y ) , ...

S i = =

which apportion the variance of the output Y to each input or combination of inputs.
Rather than relying on a single sensitivity index, we use three sensitivity indices
that are computed using slightly different methods, and we then take a consensus of
these indices. These sensitivity measures include Fourier amplitude sensitivity testing
(FAST) (Saltelli et al. 1999 ), an asymptotically efficient approach to estimating
Sobol' indices (Eff) (Monod et al. 2006 ), and Jansen's approach for estimating
Sobol' indices (Jansen 1999 ). Each of these approaches estimates the first order
and total order global sensitivity indices using slightly different methods. Only the
first order sensitivity estimates are analyzed here because the total order indices
are more variable, and we instead use FANOVA graphs to understand low order
interactions between variables. All sensitivity indices were computed with the R
package sensitivity (Pujol et al. 2012 ).
FANOVA graphs are a visual tool for easily understanding the main effects and
interaction structure among variables of a function. We provide a brief overview of
FANOVA graphs and interested readers are directed to Fruth et al. ( 2013 ) for a more
comprehensive description. The variables are presented as a connected graph, where
the size of the nodes represent the magnitude of their main effects, and the size of
the edges between variables represent the magnitude of their pairwise interactions.
The main effects or influence of each variable are estimated by first-order Sobol'
indices, and the interaction among variables, also called the total interaction index
(TII) (Fruth et al. 2013 ), is computed from the Sobol' decomposition as the sum of
all Sobol' indices which contain both variables i and j :

D ij : = D I

I ↇ{ i , j }

All computations for the FANOVA graphs use the R package fanovaGraph (Fruth
et al. 2013 ).
The experiments in this work include more than two variables, and thus visualizing
their response surfaces is not possible. In lieu of this, we use a method that is used
for structural reliability modeling that projects the probability of excursion (i.e.,
exceeding some threshold value) onto pairs of variables. In these plots, we use the
median response value as the threshold value. These figures are very useful for gaining
insight to the shape of the response function and often are intuitively consistent with
the sensitivity indices.