Networks of Neurons - Computational Explorations in Cognitive Neuroscience - page 89

Observe that the connectivity is symmetric, so that
the left unit receives from the left and center hidden1
units, while the right one receives from the center and
right hidden1 units.
Thus, the connectivity pattern can be interpreted
as representing 3 separable features in the input and
hidden1 units, with the hidden2 units representing
2 “objects” each consisting of 2 out of these 3 features.
As labeled in the simulation, you can think of the first
object as a TV , which has the two features of a CRT and
speakers , while the other is a synthesizer , which has
speakers and a keyboard . Thus, there is a one-feature
overlap between these objects, and it is this shared fea-
ture that will cause the network trouble.
Now we will present activity to only the left input
unit, which is unique to the TV object, and observe the
network's response. To see the trajectory of settling in
the network, we will open a grid log.

trial Event cycle hidden1 hidden2

0 Event_0 9

0 Event_0 19

0 Event_0 29

0 Event_0 39

0 Event_0 49

0 Event_0 59

0 Event_0 69

0 Event_0 79

0 Event_0 89

0 Event_0 99

0 Event_0 109

0 Event_0 119

0 Event_0 129

0 Event_0 139

0 Event_0 149

Press View in the amp_td_dist_ctrl control
panel, and select GRID_LOG . Then, select the act but-
ton to view the unit activities in the network, and then
press RunUniq in the control panel.
As you can see by watching the network settle,
and by looking at a trace of it in the grid log to the
right (showing the activations for the hidden1 and
hidden2 units every 10 updates (cycles) of settling,
see figure 3.17), the CRT hidden unit (on the left) first
activates the TV unit, and then this comes back down to
activate the Speakers feature. This is a good example
of a pattern completion -like phenomenon that uses top-
down activation instead of lateral activation. However,
once the Speakers unit becomes activated, it then ac-
tivates the Synth unit in the hidden2 layer, which
then does the same kind of top-down activation of the
Keyboard unit. The result is the uninterpretable acti-
vation of all 3 hidden units.

0 Event_0 159

0 Event_0 169

0 Event_0 179

0 Event_0 189

0 Event_0 199

Figure 3.17: The GridLog display of the two layers of hid-
den unit activations in response to the unique input (left-most
input unit, CRT). Activity inevitably spreads over to the other
feature units via top-down activation from the TV unit and
Synth units in the second hidden layer.

Try increasing the leak current (search with larger
steps first and then narrow your way down, and don't
search in finer steps than .001) to see if you can get the
network to just activate the features associated with TV
(left and center hidden features, CRT and Speakers ).

Question 3.9 (a) List the values of g_bar_l where
the network's behavior exhibited a qualitative transi-
tion in what was activated at the end of settling, and
describe these network states. (b) Using the value of
g_bar_l that activated only the desired two hidden
units at the end of settling, try increasing the dt_vm
parameter from .03 to .04, which will cause the network
to settle faster in the same number of cycles by increas-