Biomedical Engineering Reference
In-Depth Information
animal must not simply strengthen all perceptions and responses that occur for
whatever reason, but must apply the Hebb rule or any other learning rule selec-
tively, perhaps regulated by some sort of "value" signal that would play a role
similar to that of the error feedback signal in the back-propagation rule, but
without its per-connection specificity. This requirement has been clearly set
forth by Edelman (55,56) in his theory of neuronal group selection or "neural
Darwinism" and given mathematical form and tested in working network mod-
els by Reeke et al. (57) and Friston et al. (58). More recently, these ideas have
been implemented in synthetic neuronal-network-based control systems for ro-
botic devices performing tasks in a real-world environment (59,60). In its sim-
plest form, value can be implemented by replacing Eq. [4] with a formulation
such as
¯
¯
%
c
=
s
¸ ¸
R
s
R
v
,
[5]
¡
°
¡
°
¡
°
ij
i
i
j
j
where
v
is the nonspecific value signal, which can be either positive (represent-
ing a positive reward for the behavior) or negative (a penalty). Physiologically,
the value signal is most likely delivered via widely broadcast modulatory trans-
mitters such as dopamine or acetylcholine, but blood-borne signals (for example,
hormones involved in homeostasis) can also be imagined to play a role. (Map-
ping of reward systems in the brain via MRI imaging is discussed in detail in
Part IV, chapter 5 [by Breiter, Gasic, and Makris], this volume). Psychologi-
cally, the value signal can be considered not only as responding to intrinsic
evaluations of the consequences of behavior, but also as being subject to exter-
nal manipulation via the administration of rewards and punishments. However,
it should be noted that this simple formulation neglects the basic problem known
in learning theory as the "credit assignment" problem (61,62), one aspect of
which is that by the time the value signal arrives the neural activity levels
s
i
and
s
j
may have deviated significantly from the values they had at the time the be-
havior was produced. For other than trivially short delays in assessing value,
some sort of "memory trace" of past activity is required at the cellular level in
order for this learning scheme to work. (Part III, chapter 5.3 [by Kolb and Tim-
mann], this volume, contains a discussion of how the classical conditioning of
the eyeblink response can be understood in terms of neuronal mechanisms of the
kind discussed here operating in the cerebellum.)
1.4. Computational Considerations
In practice, network models are often implemented with
ad hoc
computer
codes, especially when new architectural or dynamical principles are being
tested. However, where applicable, development is quicker (although execution
may be slower) with a general-purpose neuron- or neuronal network-simulation
tool that can be programmed with some sort of
problem-description language
to
specify the elements needed for a particular simulation. A number of such pro-
