Geoscience Reference
In-Depth Information
signatures (e.g. Sivapalan
et al.
, 2003; Sivapalan, 2005; Winsemius
et al.
, 2009). Most
qualitative measures of performance can be set up as either a binary likelihood measure (one
if behavioural according to the qualitative measure, zero if not) or as a fuzzy measure. They
may be inherently subjective, in that different modellers might choose to emphasise dif-
ferent aspects of the behaviour in making such a qualitative assessment (see, for example,
Houghton-Carr, 1999).
Box 7.2 Combining Likelihood Measures
There may be a need to combine different likelihood measures for different types of model
evaluation (such as, one measure calculated for the prediction of discharges and one calculated
for the prediction of water table or soil moisture levels) or to update an existing likelihood
estimate with a new measure calculated for the prediction of a new observation or set of
observations.
Most cases can be expressed as successive combinations of likelihoods, where a prior like-
lihood estimate is updated using a new likelihood measure to form a posterior likelihood. This
is demonstrated by one form of combination using
Bayes equation
, which may be expressed in
the form:
L
o
(
M
(
))
L
(
M
(
)
|
Y
)
L
p
(
M
(
)
|
Y
)
=
(B7.2.1)
C
where
L
o
(
M
(
)) is the prior likelihood of a model with parameters
,
L
(
M
(
)
|
Y
) is the likelihood
calculated for the current evaluation given the set of observations
Y
,
L
p
(
M
(
)
|
Y
) is the posterior
likelihood and
is a scaling constant to ensure that the cumulative posterior likelihood is unity.
In the GLUE procedure, this type of combination is used for the likelihoods associated with
individual parameter sets so that for the
C
i
th parameter set:
L
o
(
M
(
i
))
L
(
M
(
i
)
|
Y
)
L
p
(
M
(
i
)
|
Y
)
=
(B7.2.2)
C
and
is taken over all parameter sets. Strictly, in the theory of Bayesian statistics, this form
should be used only where a sampled likelihood (for a particular parameter set) can be assumed
to be independent of other samples. In GLUE, the parameter sets are chosen randomly so as
to be independent samples of the parameter space.
The use of Bayes equation to combine likelihoods has a number of characteristics that may,
or may not, be attractive in model evaluation. Since it is a multiplicative operation, if any
evaluation results in a zero likelihood, the posterior likelihood will be zero regardless of how
well the model has performed previously. This may be considered as an important way of
rejecting nonbehavioural models; it may cause a re-evaluation of the data for that period or
variable; it may lead to the rejection of all models.
Statistical likelihoods, however, never go to zero but only get very, very small (in some
cases with time series of residuals leading to hundreds of orders of magnitude difference in
likelihood between good and poor models which is why statisticians tend to work with log
likelihood rather than likelihood so as to make the calculations feasible within the rounding
error of digital computers).
The successive application of Bayes equation will tend to gradually reduce the impact of
earlier data and likelihoods relative to later evaluations. This may be an advantage if it is thought
that the system is changing over time. It has the disadvantage that if a period of calibration
is broken down into smaller periods, and the likelihoods are evaluated and updated for each
period in turn, for many likelihood measures the final likelihoods will be different from using
a single evaluation for the whole period. An exception is a likelihood measure based on
C
