Environmental Engineering Reference
In-Depth Information
Traditionally, the probability predictions from the model were converted into
presences and absences and then a confusion matrix could be used to calculate
various parameters of choice (e.g. commission and omission error, kappa, etc:
Fielding 2002). The AUC (“area under curve”) is currently the most commonly
used measure of discriminatory power of a model. Its value (between 0.5 for
random and 1 for perfect) quantifies the ability of the model to put the data points
into the correct class (i.e. presence or absence), independent of the threshold
required by the other measures mentioned. It has recently received justified criti-
cism because its values are not comparable across different prevalences (and the
criticism extends to kappa, too; see Lobo et al. 2008). Currently, misclassification
rates, commission and omission errors are more
en vogue
again, because they can
be intuitively interpreted. Furthermore, by assigning different weights to false
negatives (omission error) and to false positives (commission error), conservation
management can come to more sophisticated and balanced decisions (Rondinini
et al. 2006).
Only rarely will a second set of data be available to investigate the quality of our
model(s) through external validation. A different recording strategy, another time
slice or data from a different geographic location represent really independent data,
and could thus be considered an external validation. The internal validation
(described above as cross-validation) is an optimistic assessment of model quality.
When using SDMs to infer underlying mechanisms, external validation is less of
an issue than when using them to extrapolate to a future climate or other sites.
Because the cross-validated models are optimistic, they give narrower error bands
than they should.
13.2.3
Interpretation
Once we have arrived at what we regard as a final model, we should make every
effort to understand what it means. A first and most relevant step is to visualize the
functional relationships within the model. The plot of how occurrence probability
is related to, say, annual precipitation should be accompanied by a confidence band
around this line. It may be useful to plot the data as rug (ticks on the axis representing
positions of the data values) into this figure to visualise the support at each point in
parameter space (Fig.
13.3
).
For interactions, visualization becomes more difficult. Two-way interactions can
still be plotted (e.g. as a 3-D plot or as a contour plot). No confidence bands can be
included, though. Here it is again very important to indicate the position of the
samples to identify regions of the parameter space that have not been sampled. For
higher dimensions, or for a model that averages across many sub-models, we can do
the same plots (called marginal plots for main effects because they represent the
marginal changes to a predictor, averaging across all other predictors). We can also
slice through higher dimensions, i.e. calculate a marginal plot for specific values of
other predictors (often their median).
