Biomedical Engineering Reference
In-Depth Information
think that these universality results apply to most complex systems, let alone
ones with adaptive agents!
6.
EVALUATING MODELS OF COMPLEX SYSTEMS
We do not build models for their own sake; we want to see what they do,
and we want to compare what they do both to reality and to other models. This
kind of evaluation of models is a problem for all areas of science, and as such
little useful general advice can be given. However, there are some issues that are
peculiar to models of complex systems, or especially acute for them, and I will
try to provide some guidance here, moving from figuring out just what your
model does, to comparing your model to data, to comparing it to other models.
6.1. Simulation
The most basic way to see what your model does is to run it; to do a simula-
tion. Even though a model is entirely a human construct, every aspect of its be-
havior following logically from its premises and initial conditions, the frailty of
human nature is such that we generally cannot perceive those consequences, not
with any accuracy. If the model involves a large number of components that
interact strongly with each other—if, that is to say, it's a good model of a com-
plex system—our powers of deduction are generally overwhelmed by the mass
of relevant, interconnected detail. Computer simulation then comes to our aid,
because computers have no trouble remembering large quantities of detail, nor
in following instructions.
6.1.1.
Direct Simulation
Direct simulation—simply starting the model and letting it go—has two
main uses. One is to get a sense of the typical behavior, or of the range of behav-
ior. The other, more quantitative, use is to determine the distribution of impor-
tant quantities, including time series. If one randomizes initial conditions, and
collects data over multiple runs, one can estimate the distribution of desired
quantities with great accuracy. This is exploited in the time-series method of
surrogate data (above), but the idea applies quite generally.
Individual simulation runs for models of complex systems can be reasona-
bly expensive in terms of time and computing power; large numbers of runs,
which are really needed to have confidence in the results, are correspondingly
more costly. Few things are more dispiriting than to expend such quantities of
time and care, only to end up with ambiguous results. It is almost always
