Information Technology Reference
In-Depth Information
bottom-up and top-down connectivity (or both) can en-
hance the activations of interconnected units, making
them more strongly activated.
Many phenomena associated with bidirectionally
connected networks can be described in terms of
at-
tractor dynamics
, where the network appears to be
at-
tracted
to a particular final activity state given a range
of initial activity patterns. This range of initial config-
urations that lead to the same final pattern is called the
attractor basin
for that attractor. Although we can de-
scribe the terminology of the attractor in this section, a
full exploration of this phenomenon awaits the introduc-
tion of inhibitory mechanisms, and is thus postponed
until section 3.6.
Select the
s.wt
button, and then select various hid-
den units.
These sending weights look just like the receiving
ones — templates of appropriate digit image. We next
verify that in fact these weights are
symmetric
(the
same in both directions).
,
!
Alternately click on
r.wt
and
s.wt
while a given
hidden unit is selected.
You should notice no difference in the display. The
interesting thing about symmetric weights is that, as you
can see,
a given unit activates the same things that ac-
tivate it
. This guarantees a kind of
consistency
in what
things get activated in bidirectional networks. We will
discuss the biological basis of weight symmetry in de-
tail in chapter 5, where it plays an important role in
the mathematical derivation of our error-driven learning
mechanism.
Now we can run the network and see what these
weights produce. First, let's replicate the previous feed-
forward results.
,
!
3.4.1
Bidirectional Transformations
We begin our introduction to the bidirectional case by
exploring a bidirectional version of the the localist digit
network. We will see that this network can perform the
same kinds of transformations as the unidirectional one,
but in both directions.
First, open the GridLog by doing
View
and selecting
GRID_LOG
. Then, select the
act
button to view the unit
activities in the network, and press
Run
.
This will present the images to the input layer for all
of the digits. You may notice that the network displays
the activations during the
settling
process (as they are
being updated on their way to equilibrium) instead of
just at the final equilibrium state as before. This will
come in handy later. Note also that the grid log shows
both the input and hidden activation states now. The
results for this run should be exactly the same as for
the feedforward network — the bidirectionality of the
weights has no additional effect here because the input
units are
clamped
(fixed) to the pattern in the event, and
are not actually computing their activations or otherwise
paying any attention to their weights.
Now let's go in the opposite direction.
,
!
Open the project
bidir_xform.proj.gz
in
chapter_3
to begin.
Note that this project looks very similar to the
previous digit networks, having the same windows,
etc. The main control panel for this one is called
bd_xform_ctrl
.
We first examine the network. Note that there are
two arrows connecting the
Input
and
Hidden
layers
— one going up and the other coming back down. This
indicates bidirectional connectivity. We can view this
connectivity as before.
Select the
r.wt
button in the network window and
click on the hidden units and the input units.
When you click on the hidden units, you will see the
now-familiar digit image templates. When you click on
the input units, you will see that they now receive from
the hidden units as well. Note that different input units
receive from different numbers of sending units — this
reflects the different types of pattern overlap among the
digit images.
A more direct way to see what would happen if you
activated a hidden unit is to view its
sending weights
.
,
!
Set
env_type
to
CATEGS
(
Apply
) and
Run
.
This will run the network by clamping the digit cat-
egory units in the hidden layer, instead of the digit im-
ages in the input. The resulting input patterns are those
driven by the top-down weights from these units, as you
might have noticed if you saw the dynamics of activa-
tion updating in the network window during the settling
for each pattern.
Otherwise, it is somewhat difficult
Search WWH ::
Custom Search