Geoscience Reference
In-Depth Information
the dynamics of the biophysical and socio-economic components of the system.
LUCC scenarios were developed with local land users using the role-playing game
and the model, and were refi ned by repeated interactions between the researchers
and the land users. The role-playing game helped the researchers to improve their
understanding of farmers' decision making and how the actors deal with the risks
engendered by uncertainty; it built trust and facilitated communication, and hence,
model development.
Evaluating models: confrontation and experimentation
Verifi cation and validation of models are much contested issues. Verifi cation focuses
on assessment of a model's structure (i.e., is the model free of logical, mathematical
or coding errors?), whereas validation addresses on how exactly a model reproduces
observed system dynamics (i.e., a model's predictions are confronted with observa-
tional data to assess its empirical adequacy). While some researchers believe that
validation is central to modelling, others have argued that it is a logical impossibility
(see Rykiel, 1996). Both verifi cation and validation are, in essence, concerned with
evaluating a model's adequacy against some criteria; what is 'adequate' will vary
with a model's purpose. I will use the term 'evaluation' to encompass this broad(er)
range of processes.
Models and their outcomes can be evaluated in many ways (table 20.2; Gardner
and Urban (2003)). Kleindorfer et al. (1998) distinguishes objectivist, or founda-
Table 20.2
Some common methods of model evaluation and analysis, and their purpose
Method
Description and purpose
Structural
- Error propagation: Analysis of error in model output(s) as a function
of the uncertainty associated with each parameter input to the model.
- Sensitivity analysis: Identifi cation of components of a model most
sensitive to uncertainty and error in parameterisation.
Confrontational
- Visual 'diagnostics': Visual comparison of empirical observations and
model predictions (i.e. by graphs).
- Visual inspection for systematic bias, etc.
- Statistical methods: Summary of differences between observations and
predictions.
- Quantitative comparison of predictions and observations (via
correlation, regression and residual analysis,
t
-tests, difference
measures, etc.).
- Assessment of spatio-temporal trends in model performance and
error.
Experimental
- Pattern-oriented modelling: Use of multiple observed patterns to
evaluate and refi ne models and select between alternate
representations (this will include structural and confi rmatory
evaluation).
- 'Social' validation: Accepting a model as legitimate on the basis of
consensus that it is valid by its users (this may or may not include
structural and confi rmatory evaluation).
Note: These methods are not mutually exclusive and most models are evaluated using a combina-
tion of the three.
