Networks of Neurons - Computational Explorations in Cognitive Neuroscience - page 97

1−

0.8−

0.6−

Inhib2

Hidden2

top−down

0.4−

inhib
hidden

0.2−

0−

Inhib

Hidden

0 20 40 60 80 100 120 140 160

Figure 3.23: Plot of average hidden (excitatory) and in-
hibitory unit activities for the first hidden layer, showing
where the top-down excitation comes in. Although detectable,
the extra excitation is well controlled by the inhibition.

Input

Figure 3.22: Inhibition network with bidirectional excitatory
connectivity.

Next, we will see that inhibition is differentially im-
portant for bidirectionally connected networks.

Set the g_bar_i.hidden parameter to 3, and Run .
This reduces the amount of inhibition on the excita-
tory neurons. Note that this has a relatively small impact
on the initial, feedforward portion of the activity curve,
but when the second hidden layer becomes active, the
network becomes catastrophically over activated — an
epileptic fit!

In extending the network to the bidirectional case, we
also have to extend our notions of what feedforward in-
hibition is. In general, the role of feedforward inhibition
is to anticipate and counterbalance the level of excita-
tory input coming into a layer. Thus, in a network with
bidirectional excitatory connectivity, the inhibitory neu-
rons for a given layer also have to receive the top-down
excitatory connections, which play the role of “feedfor-
ward” inhibition.

Set the g_bar_i.hidden parameter back to 5.

Set Point Behavior

Verify that this network has both bidirectional exci-
tatory connectivity and the “feedforward” inhibition com-
ing back from the second hidden layer by examining the
r.wt weights as usual.

Our final exploration of inhibition provides some mo-
tivation for the summary inhibition functions presented
in the next section. Here, we explore what happens to
the activity levels when different overall levels of exci-
tatory input are presented to the network.

, !

Now Run this network.
The graph log (figure 3.23) shows the average activ-
ity for only the first hidden and inhibitory layers (as be-
fore). Note that the initial part up until the point where
the second hidden layer begins to be active is the same
as before, but as the second layer activates, it feeds back
to the first layer inhibitory neurons, which become more
active, as do the excitatory neurons. However, the over-
all activity level remains quite under control and not
substantially different than before, which is in distinct
contrast to the earlier simulations with just a leak cur-
rent operating.

First, Clear the graph log and Run the network for
purposes of comparison.
The input pattern is set to have the default of 20 (out
of 100) units active, which is what we have been using
all along.

Change the input_pct field in the control panel
to 15 instead of 20, Apply , and then hit the NewInput
button to make a new input pattern with this new per-
centage activity. Then do Run .
This changes the input pattern to have 15 units active.
When you Run now, the activity level is not substan-

, !