Computation in Neural Nets - Understanding Understanding: Essays on Cybernetics and Cognition - page 47

Let us demonstrate this mathematical apparatus in the simple example
of fig. 10. With letters A and B for the names of fibers X 1 and X 2 we have,
of course, A ( x 1 = 0, 1) and B ( x 2 = 0, 1) and the four possible input states are
again:

x 1 x 2
0 0
1 0
0 1
1 1
The four corresponding Bernoulli products are after eq. (25):

( ) - ( )

1 - p 1 p

1 2

( )

p p

pp
pp

◊ 1 -

1 2

( ) ◊

1 -

1 2

◊

1 2

In order to find which of these products will contribute to our sum
(eq. (27)) that defines the desired output activation probability, we simply
consult the truth table for the function that is computed after a specific
threshold has been selected, and add those terms to this sum for which the
truth tables give a “one” ( y = 1). The following table lists for the four values
of q as chosen in fig. 10 the resulting probabilities of output excitation
according to eq. (27).
With eq. (28) we have at once what we called the internal frequency of
the element when its threshold is set to compute a particular logical func-
tion. Using the notation as suggested in column 5 of table 1 for denoting
logical functions as subscript, and denoting with T tautology and with C
contradiction, the computer frequencies representing the various logical
functions of the arguments f 1 and f 2 , again represented as frequencies, are
as follows.
This table indicates an interesting relationship that exists between the
calculus of propositions and the calculus of probabilities (Landahl et al.,
1943). Furthermore, it may be worthwhile to draw attention to the fact—
which may be shown to hold in general—that low threshold values, result-
ing in “weak” logical functions, give this element essentially a linear
characteristic; it simply adds the various stimuli f i . This is particularly true,
if the stimuli are weak and their cross-products can be neglected. For higher
threshold values the element is transformed into a highly non-linear device,

TABLE 2.
q Sum of Bernoulli products = p

0 (1 - p 1 ) (1 - p 2 ) + (1 - p 1 ) p 2 + p 1 (1 - p 2 ) + p 1 p 2 = 1

1 p 1 (1 - p 2 ) + (1 - p 1 ) p 2 + p 1 p 2 = p 1 + p 2 - p 1 p 2

2 p 1 p 2 = p 1 p 2

3 -=0