Developing predictive models - Conservation and Sustainable Use - page 176

● Use your sensitivity analyses as policy outputs. If you find that the model is very
sensitive to the value of the lag in harvester effort as it responds to changes in
prey population size, and that there is also no information about this parameter,
then you have made a useful contribution to the research agenda.
5.3.6 Validation
You may think that your model is a good representation of reality, but you need
evidence to back this up if you expect its outputs to be used in conservation.
Validation of the model against independent data is a crucial component of the
modelling process. In an ideal world, models should be developed with one set
of data and then tested with another set, to demonstrate that the model is able to
predict datasets other than the one which is used for model development. For
example, when looking at the relationship between the presence or absence of a
protected species and various factors such as habitat type and hunting level, you
can use half your data to develop a model of the relationship between the param-
eters, and the other half to test the ability of the model to predict correctly (Carroll
et al . 1999). Or if there is a long time-series of fish catches, half the time-series can
be used to develop a model of stock dynamics, which can then be used to predict
the other half of the time-series, and the prediction compared to reality.
If you are comparing between models with different structural assumptions ,
validation data can help you to decide which model best predicts the data, and so
which one the weight of evidence suggests is most likely to be true. This can be
done using the AIC to measure the deviance between the output data that each
model produces and the real data (Section 4.4.1). Harrison et al . (2006) used this
approach to distinguish between a set of models with different assumptions about
grey seal movement behaviour.
There are very few conservation models in the literature that are robustly vali-
dated, which is a major concern. Usually there are good reasons for not validating
properly. There may be few spatial replicates and time-series are short. Often there
is little opportunity to learn about the system through adaptive management due
to the species being highly endangered (Section 7.5.2.6). But partial validation is
still possible, and must be attempted if people are to take your model seriously as a
tool for management. Some suggestions are:
● Although it is not true validation, it can still be helpful to show that the
model is internally consistent , i.e. it is properly predicting the data from
which it came. Often there are parts of a dataset that are not used directly as
model inputs, which can be tested against model outputs. For example, in the
deer model, it may be that there is a time-series of population sizes or age
structures, but the model is only developed using the data on survival and
fecundity. It is then possible to initialise the model with the age structure and
population size that was observed in the first year of the time-series and see if
it is able to produce the age structure and population size observed in the last
year (Figure 5.5).