Computation in Neural Nets - Understanding Understanding: Essays on Cybernetics and Cognition - page 43

and strength, because for a given synaptic distribution the logical strength
increases with increasing threshold. This observation will be of importance
in our discussion of adaptive nets, because by just raising the threshold to
an appropriate level, the elements will be constrained to those functions
which “education” accepts as “proper”.
Since we have shown that all logical functions with two variables can be
represented by McCulloch formal neurons, and since in drawing networks
composed of elements that compute logical functions it is in many cases of
no importance to refer to the detailed synaptic distributions or threshold
values, we may replace the whole gamut by a single box with appropriate
inputs and outputs, keeping in mind, however, that the box may contain a
complex network of elements operating as McCulloch formal neurons. This
box represents a universal logical element, and the function it computes
may be indicated by attaching to it any one of the many available symbolic
representations.
We shall make use of this simplified formalism by introducing an element
that varies the functions it computes, not by manipulation of its thresholds
but according to what output state was produced, say, one computational
step earlier. Without specifying the particular functions this element
computes, we may ask what we can expect from such an element, from an
operational point of view. The mathematical formalism that represents the
behavior of such an element will easily show its salient features. Let X ( t )
be the N -tuple ( x 1 , x 2 , x 3 ,..., x N ) representing the input state at time t for
N input fibers, and Y ( t ) the M -tuple ( y 1 , y 2 ,..., y M ) representing its output
state at time t . Call Y ¢ its output state at t -D t . Hence,

( )

(19)

YXY = F , ¢ .

In order to solve this expression we have to know the previous output state
Y ¢ which, of course, is given by the same relation only one step earlier in
time. Call X ¢ the previous input state, then:

( )

YXY

¢= F ¢ ≤ , .

and so on. If we insert these expressions into eq. (19), we obtain a telescopic
equation for Y in terms of its past experience X ¢, X ≤, X ≤¢,...and Y o ,the
birth state of our element:

( )

YXXXX = F , ¢ , ≤ , ≤¢ , ..., Y .

o

In other words, this element keeps track of its past and adjusts its modus
operandi according to previous events. This is doubtless a form of
“memory” (or another way of adaptation) where a particular function from
a reservoir of available functions is chosen. A minimal element that is
sufficient for the development of cumulatively adaptive systems has been
worked out by Ashby (see Fitzhugh, 1963) (see fig. 13). It is composed of
at least one, at most three, McCulloch formal neurons, depending upon the
functions to be computed in the unspecified logical elements. We shall call