Biomedical Engineering Reference
In-Depth Information
The story of autopoesis is especially rich in morals. (1) Replication is essen-
tial. (2) It is a good idea to share not just data but programs. (3)
Always
test the
robustness of our model to changes in its parameters. (This is fairly common.)
(4)
Always
test your model for robustness to small changes in qualitative as-
sumptions. If your model calls for a given effect, there are usually several
mechanisms that could accomplish it. If it does not matter which mechanism
you actually use, the result is that much more robust. Conversely, if it does mat-
ter, the overall adequacy of the model can be tested by checking whether
that
mechanism is actually present in the system. Altogether too few people perform
such tests.
6.3.3.
Comparing Macro-data and Micro-models
Data are often available only about large aggregates, while models, espe-
cially agent-based models, are about individual behavior. One way of comparing
such models to data is to compute the necessary aggregates, from direct simula-
tion, Monte Carlo, etc. The problem is that many different models can give the
same aggregated behavior, so this does not provide a powerful test between dif-
ferent models. Ideally, we'd work back from aggregate data to individual behav-
iors, which is known, somewhat confusingly, as
ecological inference
. In
general, the ecological inference problem itself does not have a unique solution.
But the aggregate data, if used intelligently, can often put fairly tight constraints
on the individual behaviors, and micro-scale can be directly checked against
those constraints. Much of the work here has been done by social scientists, es-
pecially American political scientists concerned with issues arising from the
Voting Rights Act (154), but the methods they have developed are very general,
and could profitably be applied to agent-based models in the biological sciences,
though, to my knowledge, they have yet to be.
6.4. Comparison to Other Models
Are there other ways of generating the data? There generally are, at least if
"the data" are some very gross, highly summarized pattern. This makes it impor-
tant to look for differential signatures, places where discrepancies between dif-
ferent generative mechanisms give one some
leverage
. Given two mechanisms
that can both account for our phenomenon, we should look for some
other
quan-
tity whose behavior will be different under the two hypotheses. Ideally, in fact,
we would look for the statistic on which the two kinds of model are
most
diver-
gent. The literature on experimental design is relevant here again, since it con-
siders such problems under the heading of
model discrimination
, seeking to
