Networks of Neurons - Computational Explorations in Cognitive Neuroscience - page 78

forward button at the top of the log (right-most VCR-
style button, >| ), to move to the end of the data that
was recorded during the previous stepping.
You should see that the GridLog has recorded the ac-
tivities (and net inputs) of the hidden units for each of
the digits, so you can easily see the overall pattern of
activation across all the inputs.

Object Act ions LogFile .views View: Object Actions

Display Toggle Update Init Clear

trial Event act net

0 0

1 1

2 2

Bias Weights

3 3

Before we continue, it is important to understand the
role of the bias weights in this simulation (see sec-
tion 2.5.1 from the previous chapter for the definition
of bias weights). The digit images used as input pat-
terns have somewhat different numbers of active units
(as you might have observed in the detector exercise in
the previous chapter).

4 4

5 5

6 6

7 7

Press View in the control panel and select EVENTS
to see all of the digit input patterns.
You might have expected these differences in the
number of active input units to result in different net
input and activation levels for the corresponding hidden
units. Instead, all the activations shown in the Grid-
Log appear roughly similar. The discrepancy here is
attributable to the use of bias weights, which compen-
sate for these differences in overall activity level coming
from the inputs.

8 8

9 9

−1.091 1.091

Figure 3.11: The GridLog display of hidden unit activations
in response to the 10 digit inputs. Level of activity is encoded
as the size of the white square within each grey unit grid box.

Iconify (minimize) the events window, and select
bias.wt in the NetView window.
You should see a pattern of different valued bias
weights on the hidden units. Let's evaluate the contri-
butions of these bias weights by running the network
without them.

As before, this presents the first input pattern (0) to
the network, and updates the activations in the network
over a series of cycles until equilibrium activations have
effectively been reached for all units (see section 2.4.5).
Note that we are using the noisy XX1 rate coded activa-
tion function, as with most of our simulations.

, !

On the xform_ctrl , select BIASES_OFF for the
biases field, hit Apply , and then Run .
This will run through all of the digits, and you can
view the GridLog to see the resulting activities and net
inputs.

Proceed to Step through the entire sequence of
digits.
You should have observed that each unit was acti-
vated when its matching digit was presented, and not
when any of the other digits were presented.
To record this pattern of hidden unit activations over
the different inputs, we will use a window called a
GridLog . This window displays the activity states of
the network over time as rows of colored squares (fig-
ure 3.11).

, !

Question 3.1 How did the lack of bias weights affect
the hidden unit activities, and their relation to the num-
ber of active units in the input patterns?

Turn the bias weights back on ( BIASES_ON and
Apply ), and select bias.wt to view in the Network
window if it isn't already selected (you may need to

Press View on the control panel and select
GRID_LOG . When the window appears, press the full-

, !