Computation in Neural Nets - Understanding Understanding: Essays on Cybernetics and Cognition - page 36

least in sensory fibers—information is coded into the lengths of time inter-
vals between pulses. Since the length of a time interval is a continuous vari-
able, and since under certain conditions this interval may represent
monotonically a continuously varying stimulus, this behavior of a neuron is
usually referred to—by again invoking computer jargon—as its “analog”
characteristic. Under these conditions we may regard the behavior of a
neuron as the transfer function of a more or less linear element whose
input and output signal is a pulse interval code and whose function is pulse-
interval modulation. A “Weber-Fechner neuron” simply has a logarithmic
transfer function, a “Stevens neuron” a power-law response (Stevens, 1957)
and a “Sherrington neuron” has neat, almost linear properties with thre-
shold (Sherrington, 1906).
Although it is not at all difficult—as we shall see—to propose a single
mechanism that reconciles all types of operation in neurons discussed so
far (analog as well as digital), it is important to separate these operational
modalities, because the overall performance of a network may change
drastically if its elements move, from one operational modality to another.
Consequently, we shall discuss these different modalities under two differ-
ent headings: first “The Neuron as an 'All or Nothing' Element” with special
attention to synchronous and a-synchronous operations, and second “The
Neuron as an 'Integrating Element' ”. After this we shall be prepared to
investigate the behavior of networks under various operation conditions of
its constituents.

3.1. The Neuron as an “All or Nothing” Element
3.1.1. Synchronism

This exposé follows essentially the concepts of a “formal neuron” as pro-
posed by McCulloch and his school (McCulloch and Pitts, 1943; McCulloch,
1962), who define this element in terms of four rules of connection and four
rules of operation.

Rules of connection:
A “McCulloch formal neuron”:
i) receives N input fibers X i ( i = 1, 2, N ), and has precisely one output

fiber *.

ii) Each input fiber X i may branch into n i facilitatory (+) or inhibitory

(-) synaptic junctions, but fibers may not combine with other fibers.

iii) Through the neuron, signals may travel in one direction only.

iv) Associated with this neuron is an integer q (-•<q<+•), which rep-

resents a threshold.

Rules of operation:

v) Each input fiber X i may be in only one of two states ( x i = 0, 1), being

either OFF (0) or ON (1).