Biomedical Engineering Reference
In-Depth Information
On MDL, see Rissanen's book (185), and Grünwald's lecture notes (270).
Vapnik (22) argues that when MDL converges on the optimal model, SRM will
too, but he assumes independent data.
On statistical complexity and causal states, see (195) for a self-contained
treatment, and (188) for extensions of the theory.
10.
ACKNOWLEDGMENTS
This work has been supported by a grant from the James S. McDonnell
Foundation, and was finished while enjoying the hospitality of the Institut des
Systèmes Complexes, the Laboratoire de l'Informatique du Parallèlisme and the
Exystence Thematic Institute on Discrete and Computational Aspects of Com-
plex Systems at the École Normale Supériere de Lyon. I am grateful to Michel
Morvan and Cris Moore, organizers of the "Science et Gastronomie 2003"
workshop, for allowing me to present some of this material there, and to my
fellow participants for their critical comments. It is a pleasure to acknowledge
discussions with Dave Albers, Satinder Singh Baveja, Philippe Binder, Sven
Bruckener, Sandra Chapman, Markus Christen, Michael Cohen, Jim Crutchfield,
Gunther Eble, Dave Feldman, Walter Fontana, Peter Grassberger, Rob Haslin-
ger, John Holland, Herbert Jaeger, Jürgen Jost, Michael Lachmann, Alex Lan-
caster, Norm Margolus, Cris Moore, Mark Newman, Scott Page, Mitchell
Porter, Kristina Shalizi, Carl Simon, Eric Smith, Ricard Solé, Bill Tozier, Erik-
van Nimwegen and Nick Watkins. Special thanks to Prof. Cohen for permission
to take a triangle he drew and add a corner to it; to Profs. Drenzer and Farrell for
permission to use their data on weblogs; to Bill Tozier for suggesting that the
leitmotifs of complex systems are analysis patterns, and for advice on agents; to
Kara Kedi for daily validation of von Neumann's remarks on the complexity of
cats; to Mathew Dafilis and Nigel Phillips for spotting misprints; to Kristina
Shalizi for assistance on linear models of time series and for careful reading of a
draft; and to Kristina and Rob for resisting the idea that there
is
such a thing as
complex systems science. Finally, I wish to acknowledge the patience of the
editors.
11.
NOTES
1. Several topics pretend to give a unified presentation of the topics. To
date, the only one worth reading is (222), which however omits all models of
adaptive systems.
2. Not all data mining is strictly for predictive models. One can also mine
for purely descriptive models, which try to, say, cluster the data points so that
more similar ones are closer together, or just assign an overall likelihood score.
These, too, can be regarded as minimizing a cost function (e.g., the dissimilarity
within clusters plus the similarity across clusters). The important point is that
