Hebbian Model Learning - Computational Explorations in Cognitive Neuroscience - page 133

Contrast Enhancement

following sigmoidal function:

1.0

(4.21)

0.8 ￿ ￿

w

ij
1 ￿ w

0.6

where ￿ is the weight gain parameter ( wt_gain in the
simulator, stored in the wt_sig field) that controls the
extent of contrast enhancement performed.

0.4

Note that

0.2

this function can be derived from the same form

that has been used repeatedly throughout the text:

0.0

0.0 0.2 0.4 0.6 0.8 1.0

Linear Weight Value

(4.22)

Figure 4.12: Effective weight value as a function of under-
lying linear weight value, showing contrast enhancement of
correlations around the middle values of the conditional prob-
ability as represented by the linear weight value. Note that
the midpoint of the function is shifted upward by the offset
parameter wt off .

(though it is more difficult to have a gain parameter for
the equation in this latter form). See figure 4.12 for a
plot of this function for the standard wt_gain param-
eter of 6.
The effective weight value w is used for computing
the net inputs to units and is the standard wt value in the
simulator. The original linear weight value is only used
as an internal variable for computing weight changes (it
basically makes sure that the averaging over events re-
sults in the appropriate conditional probability values).
Thus, it would probably be very difficult to measure
something that reflects the linear weight value biolog-
ically, as it could just be subsumed into the dynamics
of the synaptic modification process. Implementation-
ally, we get the original linear weight value from the
effective weight value by applying the inverse of the
sigmoidal function.
One can add an additional parameter to the sigmoid
function that controls its offset . Thisoffsetactsmuch
like a threshold and can be useful for imposing a higher
threshold for correlation to further enhance the con-
trast between the different features present in the input.
This offset parameter ￿ ( wt_off in the simulator, also
stored in the wt_sig field) is introduced into the effec-
tive weight value equation as follows:

for a given weight value. The results for the standard
wt_off value of 1.25 are shown in figure 4.12.
Note that the point around threshold (nominally .5,
but this is affected by wt_off ) becomes a dividing
line in the contrast enhancement process — values
above are enhanced, while those below are weakened.
Thus, when the sigmoidal nonlinearity is introduced,
it suddenly becomes much more important where nu-
merically different conditional probability values fall
relative to this contrast enhancement threshold. The
savg_cor renormalization parameter plays an impor-
tant role in determining this point, in addition to the ob-
vious importance of the wt_off parameter. We will

see this in the following simulations.

4.7.3 Exploration of Renormalization and Contrast

Enhancement in CPCA

Open the project hebb_correl.proj.gz in
chapter_4 to begin (if it is still open from the previous
exercises, you will want to close and reopen it to start
with a clean slate). View the r.wt weights of the hidden
unit as before. View the GRAPH_LOG .
We a r e first going to explore the renormalization of
the weights by taking into account the ￿ expected ac-

, !

(4.23)

￿ ￿

ij
1 ￿ w

with values of ￿ greater than 1 imposing a higher
threshold on the underlying linear correlation values