Information Technology Reference
In-Depth Information
theoretical MEE error rates for Thyroid
2
and Ionosphere
2
: in this last case,
characterized by a large distribution overlap, the theoretical MEE algorithm
assigned all errors to one of the classes and was unable to converge to a more
balanced solution. Further details on these experiments are provided in [219].
Figure 3.4.2 shows the datasets with the decision borders achieved by the
hypersphere (circle) neuron. It is clear that the empirical MEE always finds
a good solution close to the min
Pe
solution. The theoretical MEE algorithm
exhibits the diculties that were already seen in the preceding section and
correspond to getting stuck in a local minimum solution which is inecient.
This effect is basically the one that we had already found in the simple
Example 3.2. Further details on the theoretical MEE solutions are presented
in [219].
3.5 The Data Splitter
The data-splitter is one of the simplest classifiers one can think of; its task is
to find an optimal split point in univariate data. The continuous output data
splitter can be viewed as a perceptron with only one adjustable parameter,
the bias
w
0
, with output expressed by
y
=
ϕ
(
x
w
0
).Wethenhave,by
virtue of (2.22), that the perceptron is trained to find a split point in the
error distribution.
Assuming the perceptron uses the tanh(
−
·
) activation function, we apply
Theorem 3.2 noting that:
tanh
2
(
x
ϕ
(
x
)=1
−
−
w
0
)
=0
∀
x
(3.72)
ϕ
−
1
(
y
)=
w
0
+ atanh(
y
)
,
(3.73)
ϕ
(
ϕ
−
1
(
y
)) = 1
y
2
.
−
(3.74)
For the tanh(
·
) activation function, since
y
∈
]
−
1
,
1[, we use the target value
set
T
=
.
Using the above formulas let us see how the data splitter theoretically
performs for the simple case of uniform class-conditional inputs:
{−
1
,
1
}
1
1
f
X|−
1
(
x
)=
[
a,b
]
(
x
)
,f
X|
1
(
x
)=
[
c,d
]
(
x
)
(3.75)
b
−
a
d
−
c
with
a<c
b<d
.
The error class-conditional PDFs can be obtained using Theorem 3.2, and
both Shannon's and Rényi's quadratic entropies can be thereafter derived
[212]. The expression of the quadratic Rényi information potential is
≤
