Individual Neurons - Computational Explorations in Cognitive Neuroscience - page 41

quire very strong inputs to fire (e.g., many inputs active
and closely matching the weight pattern), while others
may only require very weak inputs (e.g., only a few in-
puts active and/or not too closely matching the weights).
Both types (and everything in between) can be useful
and/or necessary in solving particular tasks, so it is im-
portant to include these differences in excitability in the
model. It is also important to enable these biases to
adapt with learning so that they can become tuned to
the problem at hand (see section 3.3.1 in the next chap-
ter for further discussion).
To keep the implementation simple, and because of
the uncertainty regarding which biological mechanism
is responsible, we implement the bias input in the way
that most artificial neural network models do, as an ad-
ditional bias input term in the input equation. Specifi-
cally, we introduce a bias weight ( ￿ ), which determines
the amount of bias input, and is modified with learning
much like the other weights in the network.
We also include time averaging in the net input com-
putation, which reflects the sluggishness involved in
propagating and aggregating synaptic inputs over the
entire dendritic membrane. Time averaging is also im-
portant when discrete spiking is used for having a win-
dow of temporal summation , where spiking inputs that
arrive around the same time period will produce a larger
excitatory effect than if those same inputs were dis-
tributed across time. In more pragmatic terms, time av-
eraging smoothes out rapid transitions or fluctuations
that might otherwise cause the network to oscillate or
generally fail to propagate activation effectively.
The next subsection presents the details of the excita-
tory input computation. These details are mostly imple-
mentational, not conceptual, so you may wish to skip
this section on your first pass through the topic, return-
ing to it later if necessary. The main features of the
implementation are that inputs are averaged together by
projections, and then combined using various scaling
parameters that allow the different projections to be dif-
ferentially weighted relative to each other. This scaling
is much more important in the Leabra model than in
more abstract ANN models because the weight values
are naturally bounded between 0 and 1, and thus can-
not grow arbitrarily large through learning to achieve a
useful balance of influences from different inputs.

Details of Input Conductance Computation

At each synaptic input, the fraction of excitatory input
channels open is computed as just the sending activation
times the weight: x i w ij (weights are wt in the simula-
tor). All of the individual synaptic conductances that
come from the same input projection k are then aver-
aged together:

(2.14)

(note that we assume that all of the variables discussed
in this section are a function of time (except the con-
stants as noted), and so we generally omit the explicit
time dependency (t) for simplicity, unless different time
states of the variable are used within an equation).
The factor in the above equation is usually, but not

always, equal to the number of connections a unit has
within a given projection. When a given projection has
partial connectivity from a sending layer, it is typically
useful to treat the missing connections as though they
simply have weights permanently set to 0. Thus, the

factor is set to be the number of units in the sending
layer of the projection by default, and not the number of
actual connections in the projection. By so doing, one
can easily use mechanisms for automatically configur-
ing the network connectivity as a shortcut for setting
a pattern of weights, without affecting the normaliza-
tion properties of the net input compared to an other-
wise identical network with full connectivity. Where
this is not appropriate, the div_gp_n flag in the sim-
ulator divides by the actual number of connections in a
projection (group).
The excitatory conductance value for a given projec-
tion k , which we write as g ek , is just the average of
the individual inputs times a normalizing factor based
on the expected activity level of the sending projection,
which is represented by the variable ￿ k :

(2.15)

This normalization is useful because the different pro-
jections into a neuron may have very different baseline
levels of activity due to relatively arbitrary aspects of
the model, but as noted previously the weights cannot