Geology Reference
In-Depth Information
consistently lie within limits of acceptability,
perhaps because there are 'outliers' in either input
data or evaluation observations (which can be dif-
ficult to detect) or because of fundamental error
in model structures (see Blazkova & Beven, 2009;
Dean
et al
., 2009; Krueger
et al
., 2009; Liu
et al
.,
2009). What this approach does allow is the rejec-
tion of all the models tried if necessary, whereas
in statistical approaches, error can always be
accommodated by expanding the variance of the
residual error model.
naturally within the GLUE framework: a particu-
lar model is either behavioural or not according
to the criteria set by the user. Within the statisti-
cal framework this has led to the development of
Bayesian model averaging or Bayesian melding
techniques (although these generally require that
the same form of error model be used for all the
model structures considered (and for all parame-
ter sets within those model structures) to allow a
common basis for assessing likelihood).
4.4
The Information Content of Observations
in Constraining Uncertainty
4.3
Model Evaluation as Hypothesis Testing
in the Face of Uncertainty
The discussion so far has revealed that there
might be different ways of evaluating likelihood
in assessing the performance of a model. Formal
statistical likelihood follows from assumptions
about the error model; informal likelihood meas-
ures are a more subjective choice. Mantovan and
Todini (2006) stressed how the formal frame-
work allows the objective evaluation of the
information content of observations in condi-
tioning parameter distributions in ideal cases,
but in the response of Beven
et al
. (2008) it is
shown how this can lead to an overestimation of
information content in even mildly non-ideal
cases. There is then the possibility of making a
Type II error, of giving a good model a low likeli-
hood because of a particular realisation of the
errors, particularly the input errors. The global
informal likelihood measures in that study
seemed to underestimate the information con-
tent of observations. This safeguards against
Type II errors but makes a Type I error (of accept-
ing a poor model because of uncertainty) more
likely. In real applications, with an expectation
of input and model structural error, there are
good reasons to expect that the real information
content of observations will be overestimated by
formal likelihood measures.
This is essentially a problem of differentiating
between aleatory (statistical) and epistemic
(knowledge) uncertainties. The first we can hap-
pily treat in terms of probabilities and formal
likelihoods. They should be 'well-behaved' in
In the previous section we have described two
fundamentally different approaches to uncer-
tainty estimation. They overlap, in that GLUE
can be used with formal statistical likelihood
measures if the strong assumptions can be justi-
fied, but they are essentially based on rather dif-
ferent philosophies. In formal statistical methods,
the primary aim (relaxed somewhat in Bayesian
MCMC methodology) is to find the maximum
likelihood model and allow for random error by a
statistical distribution. In GLUE the aim is to
make predictions with all models that can be
considered behavioural and allow for the errors
associated with each model, with all their com-
plexity, implicitly. The latter is therefore more
rejectionist in its approach, in that it is possible
that all the models tried might be rejected (e.g.
Choi & Beven, 2007; Dean
et al
., 2009), while if
the set of behavioural models is large then they
can be considered as multiple working hypothe-
ses about how the system responds (Beven, 1996a,
2002a, 2008, 2009). It is readily seen that this can
easily be extended to consideration of multiple
model structures which might all have parameter
sets that can be accepted as behavioural in evalu-
ation (or not!), but which might imply quite dif-
ferent process controls and predictions. It follows
that, given the limitations of the type of input
and observational data that are available, it will
not necessarily be clear that one model is dis-
tinctly better than another. This can be handled
