Information Technology Reference
In-Depth Information
4.5
0.4
f(y|t=1)
H(V)
4
0.2
3.5
0
3
−0.2
2.5
2
−0.4
1.5
−0.6
1
−0.8
0.5
V
y
0
−1
−1
−0.5
0
0.5
1
0
0.05
0.1
0.15
0.2
0.25
0.3
(a)
(b)
Fig. D.1 The tanh-neuron PDF for
a
=0
.
7
:a)PDFsfor
b
=0
.
05
(solid),
b
=0
.
3
(dashed),
b
=1
.
2
(dash-dot),
b
=2
(dotted); b)
H
(
V
)
for
b ∈
[0
.
05
,
2]
.
f
(
x
)=
K
(
a
+
bx
+
cx
2
)
−
1
/
2
c
exp
(
b
+2
cm
)atan((
b
+2
cx
)
/
√
4
ac
.
−
b
2
)
c
√
4
ac
−
b
2
(D.2)
From (D.1) and (D.2) one obtains several families of distributions by imposing
restrictions on the parameters
a, b, c
,and
m
, which control the shape of
f
.
The roots of
a
+
bx
+
cx
2
=0define distribution types. For some types one has
to restrict the support of
f
in order to guarantee
f>
0.
K
is a normalization
constant guaranteeing
f
=1.
The Pearson system is extraordinarily rich, in the sense that it includes
many well-known distributions, as we shall see, and also allows accurate mod-
eling of complex univariate PDFs of real-world datasets (see e.g., [10]).
Besides the Gauss distribution which is a special case of (D.2), there are
twelve Pearson types of density families, depending on
Δ
=
b
2
4
ac
and
conditions on
a, b
,and
c
. For several types there are no known formulas for
the variance and entropy (no known explicit integral), except sometimes for
particular subtypes. Many entropy formulas can be found in [133] and [162]
(in this last work the Rényi entropies are also presented). Others can be
computed with symbolic mathematical software. The variance and entropy
of the Pearson types are as follows:
Type 0:
Δ
=0and
b
=
c
=0
,a>
0.
Corresponds to the Gauss distribution family with mean
−
m
and
σ
=
√
a
.
−
The entropy
H
(
σ
)=ln(
σ
√
2
πe
) is a USF of
V
.
Type I:
Δ>
0,realroots
a
1
and
a
2
of opposite signs (
a
1
<
0
<a
2
),and
x
[
a
1
,a
2
].
There are no known formulas for the variance and entropy of the general
solution,
f
(
x
)=
K
(1 +
x/a
1
)
m
1
(1
∈
x/a
2
)
m
2
,
−
−
a
1
<x<a
2
,m
1
,m
2
>
−
1. However, a particular subtype is the generalized Beta distribution
