Biomedical Engineering Reference
In-Depth Information
4.6. Distributed Processing in the Nervous System
The connectionist modeling paradigm has become the dominant theoretical
framework for thinking about information processing by the nervous system
(20,41). While the mapping from neural network to neural systems is highly
approximate, the objective in connectionist models is to explore the properties
and limits of a "
gedankenexperiment
" in which information is distributed over a
population of homogeneous, computationally trivial units. Out of this research
have arisen the following robustness observations: (1) pattern recognition of
corrupted inputs, (2) categorization or generalization of noisy inputs, and (3)
graceful degradation in response to graded perturbations in network input or
network structure. There is some sense that network models are intrinsically
fault-tolerant as a result of the distributed nature of the information representa-
tion. The aforementioned principles of redundancy and modularity are likely to
participate in connectionist robustness but do not exactly capture the distributed
nature of the information in a neural network model.
The canonical representation of a feedforward neural network is
¬
-
-
Sf
=
wSR
,
[23]
-
-
-
i
ij
j
i
®
j
where
S
i
is the output of unit
i
,
w
ij
are the weights from unit
j
to unit
i
, and R
i
is
the activation threshold of unit
i
. The function
f
(.) is most often of the form of a
nonlinear squashing function or a step-function. The robustness of a network can
be assessed as the deviation of the actual output vector (
S
) from an desired out-
put vector (
O
). A common metric is the RMS error:
1
N
F
=
(
SO
)
2
.
[24]
i
i
N
i
Perturbations in
S
j
or
w
ij
can then be assessed quantitatively. An alternative error
function for binary or "bipolar" units is to use the Hamming distance between
S
and
O
.
4.6.1. Joanisse and Seidenberg on Verb Morphology
There has been some debate on whether brain-injured patients have a
greater difficulty in constructing the irregular past tense of familiar verbs or the
regular past tense of nonsense (nonce) words. The impairment has been used to
discriminate between damage to rule-following (regular) versus damage to asso-
ciative memories (irregular). The construction of the past tense has become a
