Information Technology Reference
In-Depth Information
Hidden_Acts Pattern: 0
Set
env_type
to
LETTERS
and
Run
.
Then do a
Y
cluster plot on the resulting hidden units.
Although this network clearly does a better job of dis-
tinguishing between the different letters than the localist
network, it still collapses many letters into the same hid-
den representation. Thus, we have evidence that these
distributed feature detectors are appropriate for repre-
senting distinctions among digits, but not letters.
,
!
30.00
9
28.00
26.00
24.00
9
22.00
20.00
18.00
16.00
Question 3.5
The distributed network achieves a use-
ful representation of the digits using half the number of
hidden units as the localist network (and this number of
hidden units is 1/7th the number of input units, greatly
compressing the input representation) — explain how
this efficiency is achieved.
14.00
12.00
10.00
6
8.00
6.00
4.00
2.00
Go to the
PDP++Root
window. To continue on to
the next simulation, close this project first by selecting
.projects/Remove/Project_0
. Or, if you wish to
stop now, quit by selecting
Object/Quit
.
0.00
X
,
!
0.00
1.00
2.00
3.00
4.00
Figure 3.13:
Cluster plot of the distributed representations of
the noisy digit inputs. Some “errors” are made where different
digits are collapsed together, and not all versions of the same
digit are in a single cluster. Nevertheless, it does a pretty good
job of emphasizing the distinctions between digits and deem-
phasizing those among different versions of the same digit.
3.4
Bidirectional Excitatory Interactions
In many ways, bidirectional excitatory connectivity
(also known as
interactivity
or
recurrence
) behaves
in just the same way as feedforward or unidirectional
excitatory connectivity in producing transformations of
the input patterns, except that the transformation can go
both ways. Thus, in addition to having the ability to ac-
tivate a digit category unit from an image of a digit (as
in the feedforward networks explored above), a bidirec-
tionally connected network can also produce an image
of a digit given the digit category. This is like
mental
imagery
where you produce an image based on a con-
ception (e.g., “visualize the digit 8”), which involves
processing going
top-down
instead of the
bottom-up
direction explored previously.
With
lateral
connectivity (i.e., connections among
units within the same layer), one part of a pattern can
be used to activate other parts. This process goes by
thenameof
pattern completion
, because the pattern
is being completed (made full) by processing a part of
it. Some closely related phenomena are
mutual sup-
port
,
resonance
(Grossberg, 1976), and
amplifica-
tion
, where either lateral connectivity or bidirectional
do a better job than our simple hand-set weight values
at emphasizing and deemphasizing the appropriate dis-
tinctions as a function of the task we train it on.
Another aspect of the distributed cluster plot is that
the digit categories are not equally separate elements of
a single cluster group, as with the localist representa-
tion. Thus, there is some residual similarity structure
between different digits reflected in the hidden units,
though less than in the input images, as one can tell
because of the “flatter” cluster structure (i.e., the clus-
ters are less deeply nested within each other, suggest-
ing more overall equality in similarity differences). Of
course, this residual similarity might be a good thing in
some situations, as long as a clear distinction between
different digits is made. Again, we typically rely on
learning to ensure that the representations capture the
appropriate transformations.
Another test we can perform is to test this network on
the letter input stimuli.
Search WWH ::
Custom Search