Information Technology Reference
In-Depth Information
w
0
−5.4626
w
0
−11,864
−5.4628
−5.463
−11,866
−5.4632
1
w
1
1
w
1
0.5
10.926
4.747
0.5
x 10
−3
x 10
−4
0
4.7465
0
10.9259
4.746
−0.5
−0.5
w
2
−1
4.7455
w
2
−1
−1.5
4.745
10.9258
(a)
(b)
−23.4754
w
0
−5.4628
−23.4755
w
0
−5.4629
−23.4756
2
−5.4629
−5.463
w
1
1
11.1
w
1
x 10
−4
−5.4631
9.3903
11
0
9.3903
−5.4631
9.3902
10.9
−1
w
2
1.5
x 10
−4
w
2
9.3902
1
0.5
0
9.3901
−0.5
−2
−1
10.8
9.3901
−1.5
(c)
(d)
Fig. 3.15
H
S
for bivariate Gaussian inputs, computed on a
(
w
1
,w
2
,w
0
)
grid
around
w
∗
(central dot). The dot size represents
H
S
value (larger size for higher
H
S
). (a) The distant-classes case, μ
1
=[50]
T
,with
w
∗
the
min
H
S
solution. (b)
T
,with
w
∗
a minimizer for
w
1
and
w
2
and a
maximizer for
w
0
. (c) A zoom of the central layer showing the maximum for
w
0
.
(d) The same as (a) for the logistic activation function.
The close-classes case,
μ
1
=[10]
Similar results are obtained if the logistic activation function
ϕ
(
x
)=1
/
(1+
e
−x
) is considered. In this case
exp
m
t
2
ln
t−e
1
−t
+
e
1
2
σ
t
−
−
2
πσ
t
(
t
f
E|t
(
e
)=
1+
t,t
[
(
e
)
.
(3.45)
]
−
−
e
)(1
−
t
+
e
)
For the same “distant classes” setting we find the
H
S
-MEE solution
[
w
1
0
w
0
]
T
23
.
4755]
T
=[9
.
3902 0
−
(corresponding to the optimal solu-
tion
w
0
/w
1
=2
.
5) which is a local minimum as shown in Fig. 3.15(d). For
the “close classes” setting, and in the same way as for the tanh activation
function, a saddle point is found.
−
