Information Technology Reference
In-Depth Information
this evaluation. This is the well-known integrate-and-
fire model of neural function. The output then provides
the input to other neurons, continuing the information-
processing cascade through a network of interconnected
neurons.
This chapter provides an overview of the computa-
tional-level description of a neuron (as a detector), and
the biological mechanisms that underlie this neural in-
formation processing. We focus on the canonical pyra-
midal neuron in the cortex , which is consistent with the
general focus throughout this topic on the cortex. The
sum of these biological mechanisms is known as the ac-
tivation function , as the resulting output of a neuron
is called its activation value. There is actually quite a
bit known about the neural activation function, and the
Leabra algorithm strikes a balance between using bio-
logically based mechanisms on the one hand, and keep-
ing the computational implementation relatively simple
and tractable on the other. Thus, we use a point neuron
activation function, which uses the same basic dynam-
ics of information processing as real biological neurons,
while shrinking the spatial extent (geometry) of the neu-
ron to a single point, which vastly simplifies the com-
putational implementation. Simulations in this chapter
illustrate the basic properties of the activation function,
and how it arises from the underlying biological prop-
erties of neurons. Further, we show how this activation
function can be understood in terms of a mathematical
analysis based on Bayesian hypothesis testing.
information must be retrieved from the memory mod-
ule, sent to the CPU, processed, and then stored back
into memory. In contrast, the brain appears to employ
parallel distributed processing ( PDP ), where process-
ing occurs simultaneously (in parallel) across billions of
neurons distributed throughout the brain. Memory, like
processing, is similarly distributed throughout the brain.
Thus, our computational level description of neural pro-
cessing must explain how each neuron provides mem-
ory and processing functions in a distributed way, while
still producing something useful when all the neurons
work together.
The central idea we use to explain what the neuron is
doing is that of a detector . As a simplification, we can
think of a neuron as detecting the existence of some set
of conditions, and responding with a signal that commu-
nicates the extent to which those conditions have been
met. Think of a smoke detector, which is constantly
sampling the air looking for conditions that indicate the
presence of a fire. In the brain, there are neurons in
the early stages of the visual system that are constantly
sampling the visual input looking for conditions that in-
dicate the presence of very simple visual features such
as bars of light at a given position and orientation in the
visual scene. Higher up in the visual system, there are
neurons that detect different sets of objects.
We must emphasize that although it is useful to view
the function of a neuron as a detector, the content of ex-
actly what it is detecting is not well captured by the rel-
atively simplistic notions evoked by things like smoke
detectors. In contrast with most synthetic detectors,
a neuron is considerably more complex, having many
thousands of different inputs, and living in a huge and
dynamic network of other neurons. Thus, although it is
sometimes possible to describe roughly what a neuron
is detecting, this need not necessarily be the case — as
we will see later, neurons can contribute usefully to the
overall computation if a number of them are detecting
different hard-to-label subsets or combinations.
A neuron's response is also often context sensitive ,
in that it depends on other things that you might not
have otherwise anticipated. For example, one of the ori-
ented bar detectors in the early visual system may not
respond in some visual scenes that have a bar of light
appropriately oriented and positioned for that detector,
2.2
Detectors: How to Think About a Neuron
In a standard serial computer, the basic information pro-
cessing operations are simple arithmetic and memory
manipulations (e.g., storage and retrieval). Although it
is conceivable that the brain could have been based on
the same kinds of operations, this does not appear to be
the case. Thus, to understand what functions the biolog-
ical mechanisms of the neuron are serving, we need to
come up with a computational level description of what
the neuron is doing. That is the purpose of this section,
which uses the standard computer as a point of contrast.
In a standard computer, memory and processing are
separated into distinct modules, and processing is cen-
tralized in the central processing unit or CPU . Thus,
Search WWH ::




Custom Search