Information Technology Reference
In-Depth Information
iii If
f
is a saturating function and
a
=0is a constant,
af
is a saturating
function; of the same type if
a>
0 and of different type otherwise.
iv If
f
(
x
) is a saturating function and
a>
0 is a constant,
f
(
ax
) is a satu-
rating function of the same type.
vIf
f
1
and
f
2
are USFs of the same variable and with domains
D
1
and
D
2
,f
1
+
f
2
defined on
D
1
∩
D
2
=
∅
is USF. A similar result holds for
DSFs (yielding a DSF).
Remarks:
1. The above properties (i) through (iv) are useful when one has to deal
with annoying normalization factors. Let us suppose a PDF
f
(
x
)=
kg
(
x
),
where
k
is the normalization factor. We have
H
=
−
f
ln
f
=
−
kG
−
k
ln
k
with
G
=
g
ln
g
. Let us further suppose that
V
=
av
,where
a>
0
is a constant. We then have the following implications with annotated
property:
G
(
v
) DSF
iv
⇒
i
⇒−
iii
⇒−
ii
⇒
G
(
V
) DSF
G
(
V
) USF
kG
(
V
) USF
H
(
V
) USF.
2. By property (ii) if
H
(
V
) is USF then
H
is also USF in terms of the second-
order moment, i.e., the “MSE risk”.
Proposition D.1.
If
f
(
x
)
is a saturating function and
g
(
x
)
is USF, with
the codomain of
g
contained in the support of
f
,then
h
(
x
)=
f
(
g
(
x
))
is a
saturating function of the same type of
f
(
x
)
.
Proof.
Let us analyze the up-saturating case of
f
(
x
) and set
y
=
g
(
x
).Since
g
is strictly concave we have for
t
∈
]0
,
1[ and
x
1
=
x
2
,g
(
tx
1
+(1
−
t
)
x
2
)
>
ty
1
+(1
−
t
)
y
2
. Therefore, because
f
is a strictly increasing function,
h
(
tx
1
+
(1
−
t
)
x
2
)
>f
(
ty
1
+(1
−
t
)
y
2
), the strict concavity of
f
implying
h
(
tx
1
+
(1
t
)
f
(
y
2
). Furthermore, since the composition of two
strictly increasing functions is a strictly increasing function,
h
is USF. The
result for a down-saturating
f
(
x
) is proved similarly.
−
t
)
x
2
)
>tf
(
y
1
)+(1
−
Remark: The preceding Proposition allows us to say that if entropy is a
saturating function of the standard deviation,
σ
, it will also be a saturating
function of the variance,
V
,since
σ
=
√
V
is USF. The converse is not true.
For instance,
f
(
V
)=
V
0
.
7
is USF of
V
but not of
σ
(
f
(
σ
)=
σ
1
.
4
).
Proposition D.2.
A differentiable USF (DSF)
f
(
x
)
, has a strictly decreas-
ing positive (increasing negative) derivative and vice-versa.
Proof.
We analyze the up-saturating case. Since
f
(
x
) is strictly concave, we
have for
t
t
)
x
+
t
(
x
+
t
)) =
f
(
x
+
t
2
);
∈
]0
,
1[, (1
−
t
)
f
(
x
)+
tf
(
x
+
t
)
<f
((1
−
<
f
(
x
+
t
2
)
−f
(
x
)
f
(
x
+
t
)
−f
(
x
)
t
therefore,
t
2
, and since
f
(
x
) is strictly increasing
the strict decreasing positive derivative follows. The down-saturating case is
proved similarly.
Remark: The preceding Proposition is sometimes useful to decide whether
the entropy,
H
, is a saturating function of the variance,
V
,bylookinginto
dH/dV
.
