Information Technology Reference
In-Depth Information
a)
b)
NoisyDigits Pattern: 0
Hidden_Acts Pattern: 0
Y
Y
30.00
30.00
6
9
28.00
28.00
26.00
26.00
24.00
4
24.00
22.00
22.00
1
20.00
20.00
18.00
8
18.00
16.00
16.00
14.00
14.00
12.00
5
12.00
10.00
10.00
2
8.00
8.00
6.00
9
6.00
4.00
4.00
7
2.00
2.00
0.00
0.00
X
X
0.00
5.00
10.00
15.00
20.00
0.00
0.50
1.00
Figure 3.8:
Cluster plots of digits with three different images of each digit.
a)
Shows cluster plot of the noisy digit images.
b)
Shows cluster plot of hidden layer digit detectors when shown these images. Note that the zero-length lines for the digit subclusters
here indicates that these all have exactly the same pattern, with zero distance between them. While the input represents each image
as a distinct pattern (though clustered nicely by digit), the hidden layer collapses across the differences within a given digit category,
but preserves distinctions between categories.
in the cluster plots of figure 3.8 as a function of over-
lap between the circles (though doing so accurately is
impossible in two dimensions). Thus, the overlapping
circles in the input indicate the similarity relationships
(shared pixels) among the images of different digits,
with each such circle corresponding to the set of differ-
ent images of a digit that will activate the corresponding
detector. The hidden layer circles do not overlap, indi-
cating distinctions emphasized between digits.
One way to see how the transformations implemented
by our digit network detectors emphasize distinctions
between digits at the expense of deemphasizing other
distinctions is to present them with non-digit input im-
ages — these images would contain distinctions that the
network deemphasizes. For example, if we presented
images of letters to this network, we would not expect
to get a very sensible hidden layer representation. Fig-
ure 3.10 illustrates this phenomenon, showing that the
digit hidden layer almost always collapses across the
distinctions present in the letter inputs. In this case
the collapsing happened because the hidden units sim-
ply did not respond to the letters (with the exception of
0
1
2
3
4
5
6
7
8
9
Hidden
2
7
8
3
1
6
5
0
9
4
Input
Figure 3.9:
Sketch summarizing results of previous figure
showing the transformation from the overlapping patterns of
digit representations in the input layer to the categorical digit
representations in the hidden layer. Each circle in the input
loosely represents the collection of pixels that make up the
different versions of a digit image, with overlap indicating
similarity resulting from shared pixels. The hidden layer rep-
resentations have no overlap, so that the digits are more clearly
and categorically represented.
Search WWH ::
Custom Search