ARTIFICIAL NEURAL NETWORKS FOR DATA MINING AND FEATURE EXTRACTION - Quantitative Medical Data Analysis Using Mathematical Tools and Statistical Techniques - page 329

Large l predict input neurons (genes) most signicant to the training of
the network, while small l predicts neurons with lesser signicance 16 .
Both cases imply a natural criteria for pruning input neurons with lesser
signicances. Thus, a general algorithm for dimensionality reduction and
feature extraction is to alternately train the network to predict classica-
tions and prune weights which are relatively unchanged over large periods
of time.

4. Neuron Inspired Neural Networks

Neural networks are powerful tools for exploring mining data, but there are
also many problems that can arise. It is necessary in Cybenko's theorem
for the hidden layer to become arbitrarily large, which may also lead to net-
works that converge poorly and slowly at best. Overtting is problematic,
as it so often is with nonlinear techniques. A training set in which some
(p; q) pairs are errantly associated with each other (mislabeled data) may
also lead to slow convergence of the back propagation algorithm. Misla-

beled data can also produce errant classications in general.
Many of these issues are addressed in the literature by using known
mathematical techniques to modify ANN algorithms to address such dif-
culties. However, we conclude by suggesting how models of real-world
neurons can be used to suggest modications to articial neurons.
In particular, models have been developed which incorporate ion chan-
nels (i.e., active properties) into dendritic electrotonic cable models 17 . In
these models, the dendritic membrane voltage V (X; t) at a dimensionless
distance X from the soma and at time t satises

Z n

X

t

V (X; t) = V initial + G (X; X j ; t) I j () d

0

j=1

where X j corresponds to the location of an ionic channel, I j () is the
activation at that channel, and G (X; X j ; t) is a multi-exponential decay 18 .
Since V (0; t) is the voltage at the soma and since G (X; X j ; t) is a multi-
exponential decay, the somatic voltage is of the form

Z

X

n

X

1

t

w js C js e js t e js I j () d V soma = V initial + (2)

0

j=1 s=1

where js is the s th rate of decay at the j th synaptic channel.