Individual Neurons - Computational Explorations in Cognitive Neuroscience - page 52

whether the noise happens to move the membrane po-
tential up or down when it approaches the threshold,
which can delay or advance the spike timing in a ran-
dom fashion. Thus, the threshold greatly magnifies
small differences in membrane potential by making a
large distinction between subthreshold and superthresh-
old potentials. On average, however, the spikes are
equally likely to be early or late, so these random tim-
ing differences end up canceling out in the rate code
average. This robustness of the rate code in the face
of random noise (relative to the detailed spike timing)
is one important argument for why it is reasonable to
think that neurons rely primarily on rate code informa-
tion (see section 2.8 for more discussion).
Now, let's explore some of the properties of the noisy
XX1 rate-code activation function, compared to other
possible such functions. We will compare XX1 (equa-
tion 2.20), which is the non-noisy version of noisy XX1,
and LINEAR , which is just a threshold-linear function
of the difference between the membrane potential and
the threshold:

values. Thus, the two XX1 based functions have a sat-
urating nonlinearity that allows them to gradually ap-
proach a maximal value, instead of just being clipped
off at this maximum. However, these XX1 functions ap-
proximate the threshold-linear function for lower levels
of excitation.
You should also notice that XX1 and NOISY_XX1
get closer to each other as g_e gets larger. The noise
convolution has much less of an effect when the func-
tion gets flatter, as it does in the saturating nonlinearity
region. Convolving noise with a linear function gives
you back the linear function itself, so wherever the func-
tion is approximately linear, noise has much less of an
effect.
When you are done with this simulation, you can ei-
ther close this project in preparation for loading the next
project, or you can quit completely from the simulator.

Locate the PDP++Root window. To continue on to
the next simulation, close this project first by selecting
.projects/Remove/Project_0 . Or, if you wish to
stop now, quit by selecting Object/Quit .

(2.21)

2.6.3 The Neuron as Detector

where [x]+ is again the positive component of x or 0 if

Having explored the basic equations that govern the
point neuron activation function, we can now explore
the basic function of the neuron: detecting input pat-
terns. We will see how a particular pattern of weights
makes a simulated neuron respond more to some in-
put patterns than others. By adjusting the level of ex-
citability of the neuron, we can make the neuron re-
spond only to the pattern that best fits its weights, or
in a more graded manner to other patterns that are close
to its weight pattern. This provides some insight into
why the point neuron activation function works the way
it does.

is negative (i.e., if the membrane potential is below
threshold).

Press Defaults . Then, change the excitatory input
g_bar_e from .4 to .375, and press Apply and then
Run . Then, run XX1 with the same parameters (under
act_fun , select XX1 ,press Apply and then Run ). Next
run LINEAR inthesameway.
Notice that NOISY_XX1 starts earlier than XX1 or
LINEAR, because it has a soft threshold. This results
from convolving the XX1 function with noise, such that
even at sub-threshold values, there is a certain chance of
getting above threshold, as reflected in a small positive
activation value in the rate code.

Open the project detector.proj.gz in
chapter_2 to begin. (If you closed the previous
project and did not quit completely from the simula-
tor, do PDP++Root/.projects/OpenIn/root and
select detector.proj.gz .)
As before, the three main windows (NetView, PDP++
Root, control panel) will open up. We begin by exam-
ining the NetView window. The network has an input
which will have patterns of activation in the shape of

Change the excitatory input g_bar_e from .375 to
.42, and press Apply and then Run . Then, as in the
previous procedure, run the other two activation func-
tions with the same parameters.
Notice that LINEAR goes up to ceiling (where
it is clipped at a maximum of 1), while XX1 and
NOISY_XX1 increase but stay below their maximum