Computation in Neural Nets - Understanding Understanding: Essays on Cybernetics and Cognition - page 35

Our next task is, of course, to give a description of the operational
possibilities of our elements in order to put some life into the as yet dead
structure of connective pathways.
3. General Properties of Network Elements
Today our globe is populated by approximately 3 ¥ 10 9 people, each with
his own cherished personality, his experiences and his peculiarities. The
human brain is estimated to have approximately 10 ¥ 10 9 neurons in oper-
ation, each with its own structural peculiarities, its scars and its metabolic
and neuronal neighborhood. Each neuron, in turn, is made up of approxi-
mately 4 ¥ 10 9 various building blocks—large organic compounds of about
10 6 atoms each—to which we deny individuality, either of ignorance or of
necessity. When reducing neurons to a common denominator we may end
up with a result that is not unlike Aristotle's reduction of man to a feather-
less biped. However, since it is possible to set up categories of man, say,
homo politicus , homo sapiens and homo faber , categories which do not
overlap but do present some human features, it might be possible to set up
categories in the operation of neurons which do not overlap but do repre-
sent adequately in certain domains the activity of individual neurons. This
is the method we shall employ in the following paragraphs.
We shall select some operational modalities as they have been reason-
ably well established to hold for single neurons under specified conditions,
and shall derive from these operational modalities all that may be of
significance in the subsequent discussion of neural nets.
Peculiar as it may seem, the neuron is usually associated with two oper-
ational principles that are mutually exclusive. One is known as the “All or
Nothing” law, which certainly goes back to Bowditch (1871) and which
states that a neuron will respond with a single pulse whose amplitude is
independent of the strength of stimulus if, and only if, the stimulus equals
or exceeds a certain threshold value. Clearly, this description of the behav-
ior of a neuron attaches two states to this basic element, namely, “zero” for
producing no pulse, ad “one” for producing the pulse. Since modern com-
puter jargon has crept into neurophysiology, this neural property is usually
referred to as its “digital” characteristic, for if a record of the activity of a
neuron in these terms is made, the record will present itself in form of a
binary number whose digits are “ones” and “zeros”:
. . . 01100111011110...
When we adopt this operational modality of a neuron as being crucial in
its processing of information, we also associate with the string of “ones” and
“zeros” the code in which information is transmitted in a network.
The other operational principle, which is diametrically opposed to the
one just mentioned, derives its legitimacy from the observation that—at