Biomedical Engineering Reference
In-Depth Information
(b) Remove a small number of input neurons (genes) with weights closest
to 0 to create a reduced data set
p
i
k+1
; q
i
k+1
; i = 1; : : : ; N
The support of p
i
k
is a prediction of the input pattern (i.e., the expressed
genes). As a variation on this procedure, it may be more appropriate to
remove a xed percentages of genes which correspond to weights that are
large in magnitude and then continue until convergence becomes very poor
or no weights remain with magnitudes suciently distant from 0.
Perceptrons can be used as classiers only on data which can be divided
into separate classes by a hyperplane
12
, thus making it desirable to use an
MLP instead. Although there is no obvious correspondence between input
neurons and weights in an MLP, there are strategies for using MLP's to
reduce the dimensionality of the classication data.
Let's begin by deriving a simple algorithm for feature extraction in
multilayer perceptrons. To do so, let us notice that in the back propagation
algorithm, the change in the weights w
kl
is
w
kl
= "
k
x
l
so that if x
l
= p
l
and if
l
= 0; then after some large number of training
sessions we have
X
w
kl
=
"
k
R
l
where the sum is over the training sessions. If is chosen so thatj"
k
j< 1,
then w
kl
is close to 0 when the l
th
neuron is not in the pattern. Such a
criteria in combination with the algorithm above allows the use of a MLP
in reducing the dimensionality of a set of classied data.
This method is similar to methods that use sensitivity analysis to predict
the relative importance of a given input neuron. Specically, for each i =
1; : : : ; n and l = 1; : : : ; r; the partial derivative
X
m
@y
j
@x
l
=
2
y
j
(1y
j
)
jk
w
kl
k
(1
k
)
k=1
measures the sensitivity of the output neuron y
j
to variations in the in-
put neuron x
l
(see [15]).
p
i
; q
i
Given a training set
; i = 1; : : : ; N, the
signicance of the l
th
input neuron is dened to be
0
1
A
1=2
@y
j
2
y=q
i
;x
l
=
(
p
l
l
)
X
n
1
n
@
l
=
max
i2f1;:::;Ng
:
@x
l
j=1
Search WWH ::
Custom Search