Computation in Neural Nets - Understanding Understanding: Essays on Cybernetics and Cognition - page 96

stimulus field—as has been shown in fig. 25—and the loop activity is
reduced to the small amount that remains when objects happen to be in the
visual field. Since it is clear that the output of the whole system represents
the number of edges present at any moment, it represents at the same time
a count of the number of objects, regardless of their size and position, and
independent of the strength of illumination.
Consider this system as being in contact with an environment which has
the peculiar property of being populated by a fixed number of objects that
freely move about so that the limited visual field consisting of, say, N recep-
tors perceives only a fraction of the number of these objects. In the long
run, our system will count objects normally distributed around a mean value
with a standard deviation of, say, s.
The amount of information (entropy) H OUT as reported by the system
about its environment is (Shannon and Weaver, 1949):

H

OUT =2

ln p e

with

e = 2.71828....

On the other hand, with N binary receptors its input information is

HNH
IN

= > .

OUT

This represents a drastic reduction in information—or reduction in
uncertainty—which is performed by this network and one may wonder why
and how this is accomplished. That such a reduction is to be expected may
have been suggested by the system's indifference to a variety of environ-
mental particulars as, e.g., size and location of objects, strength of illumina-
tion, etc. But this indifference is due to the network's abstracting powers,
which it owes to its structure and the functioning of its constituents.

5.4. Information Storage in Network Structures
Let us make a rough estimate of the information stored by the choice of a
particular function computed by the network elements. As we have seen,
these abstractions are computed by sets of neighbor elements that act upon
one computer element. Let n s and n h be the number of neighbors of the k th
order in a two-dimensional body-centered square lattice and hexagonal
lattice respectively (see fig. 39):

2

= () ( )

nkn 2 , = 3 k + 1 . (116)

s h

The number of logical functions with n inputs is (eq. (18))

n

= 2 2

N ,

and the amount of information necessary to define a particular function is:

n

(117)

H

F =

log N = 2 .

2