Hebbian Model Learning - Computational Explorations in Cognitive Neuroscience - page 113

Chapter 4

Hebbian Model Learning

4.9.5 Learning Based Primarily on Hidden Layer
Constraints .................144
4.9.6 GenerativeModels .............145
4.10Summary ......................145
4.11FurtherReading ..................1 46

Contents

4.1 Overview ......................115
4.2 BiologicalMechanismsofLearning........116
4.3 ComputationalObjectivesofLearning ......118
4.3.1 Simple Exploration of Correlational Model
Learning...................121
4.4 PrincipalComponentsAnalysis..........122
4.4.1 Simple Hebbian PCA in One Linear Unit . . 122
4.4.2 Oja'sNormalizedHebbianPCA ......124
4.5 Conditional Principal Components Analysis . . . 125
4.5.1 TheCPCALearningRule..........127
4.5.2 DerivationofCPCALearningRule.....128
4.5.3 Biological Implementation of CPCA Heb-
bianLearning................129
4.6 ExplorationofHebbianModelLearning.....130
4.7 Renormalization and Contrast Enhancement . . 132
4.7.1 Renormalization...............133
4.7.2 Contrast Enhancement . ..........134
4.7.3 Exploration of Renormalization and Con-
trast Enhancement in CPCA ........135
4.8 Self-OrganizingModelLearning .........137
4.8.1 Exploration of Self-Organizing Learning . . 138
4.8.2 SummaryandDiscussion ..........142
4.9 OtherApproachestoModelLearning ......142
4.9.1

4.1 Overview

Learning is perhaps the single most important mecha-
nism in a neural network, because it provides the pri-
mary means of setting the weight parameters, of which
there are often thousands in a model and trillions in
the brain. Learning depends on both individual neuron-
level mechanisms and network-level principles (devel-
oped in the previous chapter) to produce an overall
network that behaves appropriately given its environ-
ment. The importance of a mathematically rigorous
treatment of learning mechanisms (also called algo-
rithms or rules ) was driven home by the critical analyses
of Minsky and Papert (1969), and such mathematical
analyses have played a key role in most learning algo-
rithms developed since. These mathematical analyses
must be complemented with biological, psychological,
and more pragmatic constraints in achieving a useful
learning mechanism for cognitive neuroscience model-
ing. Thus, we use many levels of analysis in developing
ideas about how learning should and does occur in the
human cortex.
We begin with the biological mechanisms that un-
derlie learning, long-term potentiation (LTP) and long-
term depression (LTD) , which refer to the strengthening

Algorithms That Use CPCA-Style Hebbian
Learning...................143

4.9.2 Clustering..................143

4.9.3 Topography .................143

4.9.4 InformationMaximizationandMDL ....144

115