Biomedical Engineering Reference
In-Depth Information
One can show (195, theorems 5 and 6) that the statistical complexity is always at
least as large as the predictive information, and generally that it measures how
far the system departs from statistical independence.
The causal states have, from a statistical point of view, quite a number of
desirable properties. The maximal prediction property corresponds exactly to
that of being a sufficient statistic (159); in fact they are minimal sufficient statis-
tics (159,165). The sequence of states of the process form a Markov chain. Re-
ferring back to our discussion of filtering and state estimation (§3.5), one can
design a recursive filter that will eventually estimate the causal state without any
error at all; moreover, it is always clear whether the filter has "locked on" to the
correct state or not.
All of these properties of the causal states and the statistical complexity
extend naturally to spatially extended systems, including, but not limited to,
cellular automata (196,197). Each point in space then has its own set of causal
states, which form not a Markov chain but a Markov field, and the local causal
state is the minimal sufficient statistic for predicting the future of that point. The
recursion properties carry over, not just temporally but spatially: the state at one
point, at one time, helps determine not only the state at that same point at later
times, but also the state at neighboring points at the same time. The statistical
complexity, in these spatial systems, becomes the amount of information needed
about the past of a given point in order to optimally predict its future. Systems
with a high degree of local statistical complexity are ones with intricate spatio-
temporal organization, and, experimentally, increasing statistical complexity
gives a precise formalization of intuitive notions of self-organization (197).
Crutchfield and Young were inspired by analogies to the theory of abstract
automata, which led them to call their theory, somewhat confusingly,
computa-
tional mechanics
. Their specific initial claims for the causal states were based
on a procedure for deriving the minimal automaton capable of producing a given
family of sequences
28
known as Nerode equivalence classing (198). In addition
to the theoretical development, the analogy to Nerode equivalence-classing led
them to describe a procedure (102) for estimating the causal states and the F-
machine from empirical data, at least in the case of discrete sequences. This
Crutchfield-Young algorithm has actually been successfully used to analyze
empirical data, for instance, geomagnetic fluctuations (199). The algorithm has,
however, been superseded by a newer algorithm that uses the known properties
of the causal states to guide the model discovery process (105) (see §3.6.3
above).
Let me sum up. The Grassberger-Crutchfield-Young statistical complexity
is an objective property of the system being studied. This reflects the
intrinsic
difficulty of predicting it, namely the amount of information that is actually
relevant to the system's dynamics. It is low both for highly disordered and trivi-
ally ordered systems. Above all, it is calculable, and has actually been calculated
for a range of natural and mathematical systems. While the initial formulation
