Information Technology Reference
In-Depth Information
Computation in Neural Nets*
HEINZ VON FOERSTER
Department of Electrical Engineering and Department of Biophysics, University of
Illinois, Urbana, Illinois, USA
A mathematical apparatus is developed that deals with networks of ele-
ments which are connected to each other by well defined connection rules
and which perform well defined operations on their inputs. The output of
these elements either is transmitted to other elements in the network or—
should they be terminal elements—represents the outcome of the compu-
tation of the network. The discussion is confined to such rules of connection
between elements and their operational modalities as they appear to have
anatomical and physiological counter parts in neural tissue. The great lati-
tude given today in the interpretation of nervous activity with regard to
what constitutes the “signal” is accounted for by giving the mathematical
apparatus the necessary and sufficient latitude to cope with various inter-
pretations. Special attention is given to a mathematical formulation of
structural and functional properties of networks that compute invariants in
the distribution of their stimuli.
1. Introduction
Ten neurons can be interconnected in precisely 1,267,650,500,228,229,401,
703,205,376 different ways. This count excludes the various ways in which
each particular neuron may react to its afferent stimuli. Considering this
fact, it will be appreciated that today we do not yet possess a general theory
of neural nets of even modest complexity.
It is clear that any progress in our understanding of functional and struc-
tural properties of nerve nets must be based on the introduction of
constraints into potentially hyper-astronomical variations of connecting
pathways. These constraints may be introduced from a theoretical point of
view, for reasons purely esthetic, in order to develop an “elegant” mathe-
Search WWH ::
Custom Search