Assessing Model Adequacy - Environmental Modelling: Finding Simplicity in Complexity - page 433

Table 26.1 Standard deviations of internal, external and total model discrepancy at 13 different hours for the fast model and of
external model discrepancy for the slow model, as depicted in Figure 26.7.

100 160 220 280 340 400 460 520 580 640 700 760 820

Fast internal 0.15 0.16 0.16 0.06 0.06 0.07 0.06 0.23 0.20 0.16 0.12 0.05 0.04

Fast external 0.14 0.00 0.00 0.00 0.00 0.08 0.00 0.00 0.00 0.00 0.10 0.00 0.02

Fast total 0.21 0.16 0.16 0.06 0.06 0.11 0.06 0.23 0.20 0.16 0.16 0.05 0.05

Slow external 0.15 0.09 0.10 0.06 0.06 0.11 0.07 0.15 0.15 0.09 0.15 0.15 0.13

The upper panel in Figure 26.7 shows the 250 runs used
to build the 13 emulators, the 203 candidate runs and the
observed discharges: the error bars are based on the sumof
the external model discrepancy and measurement error.
The lower panel in Figure 26.7, summarized in
Table 26.1, compares the fast and slow model results
and shows standard deviations for fast internal model
discrepancy, fast external model discrepancy, fast total
model discrepancy and slow external model discrepancy,
the later equalling the slow total model discrepancy, as
there is no internal model discrepancy. Observe that the
fast total model discrepancy and slow external model
discrepancy are of a similar order of magnitude, with
the fast total model discrepancy being mostly larger
due to the fast internal discrepancy contribution, which
could not be assessed in the slow model situation. Other
deviations are due to the best run selection process
being slightly different in the fast and slow cases: in
particular, the presence of the internal discrepancy alters
the definition of an acceptable run. As a further check
on the quality of the emulators, we found that the 203
candidate runs suggested by the emulator did in fact
include the best eight runs that would have been found
had we evaluated all 100 000 runs used in Section 26.3.4.1
using the runoff model directly. Thus, using the 13
emulators we have only had to evaluate the runoff model
453

to the Haute Mentue catchment and to Leanna House
for explaining the model and providing her R code. This
chapter was produced with the support of the Basic Tech-
nology Initiative as part of the 'Managing Uncertainty
for Complex Models' project, and with the support of an
EPSRC Mobility Fellowship (IV).

References

Craig, P.S., Goldstein, M., Rougier, J.C., and Seheult, A.H. (2001)
Bayesian forecasting for complex systems using computer simu-
lators. Journal of the American Statistical Association , 96 , 717-29.
Craig, P.S., Goldstein, M., Seheult, A.H., and Smith, J.A. (1997)
Pressurematching for hydrocarbon reservoirs: a case study in the
use of Bayes linear strategies for large computer experiments,
in Case Studies in Bayesian Statistics (eds C. Gatsonis et al .),
Springer-Verlag, New York, vol. 3, pp. 36-93.
Iorgulescu, I., Beven, K.J., and Musy, A. (2005) Data-based mod-
elling of runoff and chemical tracer concentrations in the
Haute-Mentue research catchment (Switzerland), Hydrological
Processes 19, 2557-73.
Kennedy, M.C. and O'Hagan, A. (2001) Bayesian calibration of
computer models, Journal of the Royal Statistical Society, Series
B , 63 , 425-64.
MUCM (2009), MUCM Toolkit Release 3, Aston, England.
http://mucm.aston.ac.uk (accessed 12 December 2011).
O'Hagan, A. (2006), Bayesian analysis of computer code outputs: a
tutorial. Reliability Engineering and SystemSafety , 91 , 1290-300.
R Development Core Team (2008) R: A Language and Environ-
ment for Statistical Computing , R Foundation for Statistical
Computing, Vienna.
Rougier, J. (2009) Formal Bayes methods for model calibration
with uncertainty, in Applied Uncertainty Analysis for Flood Risk
Management (eds Beven, K. and Hall, J.) Imperial College
Press/World Scientific, London.
Venables, W.N. and Ripley, B.D. (2002) Modern Applied Statistics
with S , 4th edn, Springer, New York.

203 times to achieve the same results as
running the runoff model 100 000 times! As the model
discrepancy is similar in magnitude to that for the fast
simulator, our conclusions regarding model adequacy
are consistent with those given in Section 26.3.4.1.

= 250 +

26.5 Acknowledgements

We are extremely grateful to Keith Beven for many help-
ful discussions on the runoff model and its application