Digital Signal Processing Reference
In-Depth Information
n
,
n
input dimension,
w
0
∈ R
with weight
w
the bias, and as
activation function
θ
, the Heaviside function (
θ
(
x
)=0for
x<
0and
θ
(
x
)=1for
x
∈ R
0). Often, the bias
w
0
is added as additional weight to
w
with fixed input 1.
Learning in a perceptron means minimizing the error energy function
shown above. This can be done, for example, by gradient descent with
respect to
w
and
w
0
. This induces the well-known
delta rule
for the
weight update,
≥
t
)
x
,
Δ
w
=
η
(
y
(
x
)
−
where
η
is a chosen learning rate parameter,
y
(
x
) is the output of the
neural network at sample
x
,and
t
is the observation of input
x
.Itiseasy
to see that a perceptron separates the data linearly, with the boundary
hyperplane given by
n
w
x
+
w
0
=0
{
x
∈ R
|
}
.
Results
We wanted to approximate the diagnosis function
d
:
R
30
→{
}
that classifies each parameter set to one of the two diagnoses. It turned
out that we achieved best results in terms of approximation quality by
using the 13-dimensional column subset with parameters RANTESRO,
RANTESZZ, RANBALLY, IP101RO, IP101ZZ, IP102RO, IP1O2ZZ,
CD8, CD4/CD8, CX3CD8, NG, ZZ and CORTISONE, as explained
earlier in this section. The diagnosis of each patient in this sample set
was known; so we really wanted to approximate the now 13-dimensional
diagnosis function
d
:
0
,
1
.
We had to omit 10 of the original 39 samples because too many
parameters of those samples were missing. Of the remaining 29 samples,
one parameter of one sample was unknown, so we replaced it with the
mean value of this parameter of the other samples.
After centering the data, further preprocessing was performed by
applying a PCA to the 13-D data set in order to normalize and whiten
the data and to reduce their dimension. With only this small number
of samples, learning in a 13-D neural network can easily result in very
low generalization quality of the network. In figure 6.20, we give a plot
of reduction dimension versus the output error of a perceptron trained
with all 29 samples after reduction to the given dimension. We see that
dimension reduction as low as five dimensions still yields quite good
R
13
→{
0
,
1
}
Search WWH ::
Custom Search