Geoscience Reference
In-Depth Information
The HSY approach is essentially a
nonparametric
method of sensitivity analysis in that it makes no
prior assumptions about the variation or covariation of different parameter values, but only evaluates sets
of parameter values in terms of their performance. Other nonparametric methods include approaches
based on fuzzy set theory, in which fuzzy measures are used to reflect the imprecision of knowledge
about individual parameter values and boundary conditions (e.g. Dou
et al.
, 1995; Schulz and Huwe,
1997, 1999). There is also a variance based Sobol' Global Sensitivity Analysis, which aims to decompose
the variance of a predicted variable from a sample of model runs into primary effects (associated with
individual parameters), secondary effects (associated with the joint variation of two parameters) and so on
(see Saltelli
et al.
, 2004, 2006). It would be useful to have such a decomposition, and in some applications
it might be very instructive, particularly where within the structure of the model there is a strong hidden
interaction of two or more parameters. However, it is a linear decomposition and, in nonlinear models,
parameter interactions also exhibit strong nonlinearities such that the analysis is sometimes misleading
(for example, where two parameters show a positive interaction in one part of the parameter space and a
negative interaction in another part).
There have been some comparative studies of sensitivity analysis methods as applied to hydrological
models (e.g. Pappenberger
et al.
, 2006a, 2008). These have tended to show that sensitivities of different
parameters depend on the method of sensitivity analysis used, the predicted variable to which sensitivity
is examined and the period of data used in the comparison (although with some consistency in identifying
a small number of the most sensitive parameters).
7.3 Performance Measures and Likelihood Measures
The definition of a parameter response surface as outlined above and shown in Figures 7.1 and 7.2 requires
a quantitative measure of performance, goodness of fit or likelihood. It is not too difficult to define the
requirements of a rainfall-runoff model in words: we want a model to predict the hydrograph peaks
correctly, to predict the timing of the hydrograph peaks correctly, and to give a good representation of the
form of the recession curve to set up the initial conditions prior to the next event. We may also require
that, over a long simulation period, the relative magnitudes of the different elements of the water balance
should be predicted accurately. The requirements might be somewhat different for different projects, so
there may not be any universal measure of performance that will serve all purposes.
Most measures of goodness of fit used in hydrograph simulation in the past have been based on the sum
of squared errors or error variance. Taking the squares of the residuals results in a positive contribution
of both overpredictions and underpredictions to the final sum over all the time steps. The error variance
(assuming a zero mean error),
σ
ε
, is defined as
T
y
t
−
y
t
2
1
T
−
1
σ
ε
=
(7.2)
t
=
1
y
t
is the predicted value of variable
y
at time step
t
=
1
,
2
,...,T.
Usually the predicted variable is
discharge,
Q
(as shown in Figure 7.4), but it may be possible to evaluate model performance with respect
to other predicted variables so we use the general variable
y
in the following discussion. A widely used
goodness-of-fit measure based on the error variance is the modelling “efficiency” of Nash and Sutcliffe
(1970), defined as
where
1
σ
ε
σ
o
E
=
−
(7.3)
where
σ
o
is the variance of the observations. The efficiency is like a statistical coefficient of determination.
It has the value of 1 for a perfect fit when
σ
ε
=
zero; it has the value of 0 when
σ
ε
=
σ
o
,
which is equivalent
