Networks of Neurons - Computational Explorations in Cognitive Neuroscience - page 69

Chapter 3

Networks of Neurons

3.7 Summary ......................112
3.8 FurtherReading ..................1 14

Contents

3.1 Overview ...................... 71
3.2 GeneralStructureofCorticalNetworks ..... 72
3.3 Unidirectional Excitatory Interactions: Trans-
formations ..................... 75
3.3.1 ExplorationofTransformations....... 79
3.3.2 Localist versus Distributed Representations 82
3.3.3 Exploration of Distributed Representations . 84
3.4 BidirectionalExcitatoryInteractions ....... 85
3.4.1 Bidirectional Transformations ........ 86
3.4.2 Bidirectional Pattern Completion ...... 87
3.4.3 Bidirectional Amplification ......... 89
3.4.4 AttractorDynamics ............. 92
3.5 InhibitoryInteractions............... 93
3.5.1

3.1 Overview

Although the neuron provides the basic unit of process-
ing, a network of such neurons is required to accom-
plish everything but the most simple tasks. In the previ-
ous chapter we were able to describe the essential com-
putation performed by the neuron in terms of the de-
tector model. No such simple, overarching computa-
tional metaphor applies to the computation performed
by the entire network. Instead, we adopt a two-pronged
approach to understanding how networks work: first,
in this chapter, we identify and explore several impor-
tant principles that govern the general behavior of net-
works; second, in the next chapter, we show how learn-
ing, which is responsible for setting the detailed weight
values that specify what each unit detects, can shape
the behavior of networks according to various princi-
ples that build upon those developed in this chapter.
We begin with a summary of the general structure and
patterns of connectivity within the mammalian cortex
(neocortex), which establishes a biological basis for the
general types of networks we use. There are many com-
monalities across all cortical areas in terms of neuron
types and general patterns of connectivity, which give
rise to a canonical or generic cortical network struc-
ture that can be used for modeling all kinds of different
psychological phenomena. As explained in the previ-
ous chapter (section 2.3.3), excitation and inhibition are
separated in the cortex, because they are implemented

General Functional Benefits of Inhibition . . 94

3.5.2 Exploration of Feedforward and Feedback
Inhibition .................. 95
3.5.3 The k-Winners-Take-All Inhibitory Functions 100
3.5.4 Exploration of kWTA Inhibition .......103
3.5.5 Digits Revisited with kWTA Inhibition . . . 104
3.5.6 Other Simple Inhibition Functions .....105
3.6 ConstraintSatisfaction...............106
3.6.1 AttractorsAgain...............108

3.6.2 TheRoleofNoise ..............108

3.6.3 The Role of Inhibition . . ..........109

3.6.4 Explorations of Constraint Satisfaction:

CatsandDogs................109

3.6.5 Explorations of Constraint Satisfaction:

NeckerCube.................111

71