Geoscience Reference
In-Depth Information
Observed
Simulated
Error or
Residual
Time, t
Figure 7.4
Comparing observed and simulated hydrographs.
to saying that the hydrological model is no better than a one parameter “no-knowledge” model that gives
a prediction of the mean of the observations for all time steps! Negative values of efficiency indicate
that the model is performing worse than this “no-knowledge” model (which has one parameter of the
mean flow).
The sum of squared errors and modelling efficiency are not ideal measures of goodness of fit for
rainfall-runoff modelling for three main reasons. The first is that the largest residuals tend to be found
near the hydrograph peaks. Since the errors are squared, this can result in the predictions of peak discharge
being given greater weight than the prediction of low flows (although this may clearly be a desirable
characteristic for some flood forecasting purposes). Secondly, even if the peak magnitudes were to be
predicted perfectly, this measure may be sensitive to timing errors in the predictions. This is illustrated
for the second hydrograph in Figure 7.4 which is well predicted in shape and peak magnitude but the
slight difference in time results in significant residuals on both rising and falling limbs.
Figure 7.4 also illustrates the third effect, that the residuals at successive time steps may not be
independent but may be
autocorrelated
in time. The use of the simple sum of squared errors as a goodness-
of-fit measure has a strong theoretical basis in statistical inference, for cases where the samples (here,
the residuals at each time step) can be considered as independent and of constant variance. In many
hydrograph simulations there is also a suggestion that the variance of the residuals may change in a
consistent way over time, with a tendency to be higher for higher flows. This has led to significant
criticism of the Nash-Sutcliffe efficiency and sum of squared errors as performance measures (see
Beran, 1999; Houghton-Carr, 1999; McCuen
et al.
, 2006; Schaefli and Gupta, 2007; Smith
et al.
2008a;
Reusser
et al.
, 2009). In general, measures based on the sum of squared errors lead to biased parameter
estimates when there is structure in the residual series, such as autocorrelated values in time. This has not
stopped a flow of papers in rainfall-runoff modelling that still use the Nash-Sutcliffe efficiency measure
as the basic measure of performance (almost certainly because of its rather natural range of values aiding
interpretation).
So, it should be possible to do better and the obvious thing to do is to borrow from the theory of
likelihood in statistics, which attempts to take account of structure in the residuals, such as correlation
and changing variance of the errors (
heteroscedastic errors
, e.g. Sorooshian
et al.
, 1983; Hornberger
et
al.
, 1985). Maximum likelihood, in frequentist statistics, aims to maximise the probabilty of predicting
an observation, given the model. These probabilities are specified on the basis of a likelihood function,
which is a goodness-of-fit measure that has the advantage that it can be interpreted directly in terms
of such prediction probabilities. Bayesian statistics combines this likelihood function with some prior
distributions of the parameters to produce posterior distributions for the parameters that can be used
