Information Technology Reference
In-Depth Information
1.
i
=
k
:
6
w
i
σ
2
2
A
+
σ
2
w
i
∂
2
V
∂w
i
=
1
σ
3
−
−
⇒
⎧
⎨
w
i
=
k
=0
=
−
8
μ
2
√
1+2
μ
2
∂
2
V
∂w
i
<
0
2
√
2
w
k
=0
=
⇒
(3.65)
⎩
w
i
∂
2
V
∂w
i
√
2
>
0
6
A
σ
2
−
B
w
i
=
k
=0
w
k
=0
∂
2
V
∂w
k
∂w
i
∂
2
V
∂w
k
∂w
i
∂
2
V
∂w
k
∂w
i
w
i
w
k
σ
3
=
⇒
=
=0
(3.66)
2.
i
=
k
:
σ
3
6
w
k
2
1
σ
2
∂
2
V
∂w
k
4
w
k
μ
2
σ
2
w
k
=
A
+
B
σ
−
−
−
⇒
σ
2
⎧
⎨
w
i
=
k
=0
=
∂
2
V
∂w
k
4
σ
3
>
0
w
k
=0
=
4
μ
2
⇒
(3.67)
⎩
∂
2
V
∂w
k
√
2
3.
j
=
i
and
i, j
=
k
:
⎧
⎨
w
i
=
k
=0
=0
6
A
σ
2
−
1
∂
2
V
∂w
j
∂w
i
∂
2
V
∂w
j
∂w
i
w
i
w
j
σ
3
w
k
=0
=
w
i
w
j
=
⇒
.
(3.68)
⎩
∂
2
V
∂w
j
∂w
i
√
2
Therefore, the Hessian for the
w
i
=
k
=0critical points is a diagonal matrix
whose elements are precisely the eigenvalues. Thus, we get saddle points. In
what concerns the
w
k
=0critical points they are either minima or saddle
points.
Note that the
μ
=0setting would produce an infinity of
i
w
i
=2critical
points, but this is an uninteresting degenerate configuration with no distinct
classes. The
σ
=0setting is discussed in the following example.
Example 3.9.
We apply the results of Theorem 3.3 to the two-dimensional
case. For whitened Gaussian inputs with means lying on
x
1
, symmetric about
x
2
and with
μ
=1, we have two critical points of
V
R
2
(
w
1
,w
2
) located at
w
0
=0and at the following weight vectors:
±
2
/
30
T
,
0
±
√
2
T
.
Figure 3.18 shows a segment of the
V
R
2
(
w
1
,w
2
) surface with the two crit-
ical points for positive weights signalled by vertical lines. Figure 3.19 shows
the error PDFs at these two critical points. The [0
±
√
2]
T
critical points
are minima (confirming the above results) and correspond to a line passing
through the means; one indeed expects the entropy to be a maximum for this
setting, and the information potential a minimum.
