Digital Signal Processing Reference
In-Depth Information
3. When a large number of trials is run, statistical regularity must be
observed in the outcome, i.e., an average behavior must be identified
if the experiment is repeated a large number of times.
The key point of analyzing a random experiment lies exactly in the
representation of the statistical regularity. A simple measure thereof is
the so-called relative frequency. In order to reach this concept, let us define
the following:
•
The space of outcomes
, or sample space, which is the set of all
possible outcomes of the random experiment.
•
An event
A
, which is an element, a subset or a set of subsets of
.
Relative frequency is the ratio between the number of occurrences of a
specific event and the total number of experiment trials. If an event
A
occurs
N
(
A
)
times over a total number of trials
N
, this ratio obeys
N
(
A
)
0
≤
≤
1
(2.26)
N
We may state that an experiment exhibits statistical regularity if, for any
given sequence of
N
trials, (2.26) converges to the same limit as
N
becomes a
large number. Therefore, the information about the occurrence of a random
event can be expressed by the
frequency definition of probability
, given by
N
(
A
)
Pr
(
A
)
=
lim
N
(2.27)
N
→∞
On the other hand, as stated by Andrey Nikolaevich Kolmogorov in his
seminal work [170], “The probability theory, as a mathematical discipline,
can and should be developed from axioms in exactly the same way as Geom-
etry and Algebra.” Kolmogorov thus established the axiomatic foundation of
probability theory. According to this elegant and rigorous approach, we can
define a
field of probability
formed by the triplet
{
)
}
,
F
,Pr
(
A
, where
is the
F
space of outcomes,
is a field that contains all possible events of the ran-
dom experiment,
∗
and Pr
is the probability of event
A
. This measure is so
chosen as to satisfy the following axioms.
(
A
)
Axiom 1:
Pr
(
A
)
≥
0
Axiom 2:
Pr
()
=
1
Axiom 3:
If
A
∩
B
= ∅
, then Pr
(
A
∪
B
)
=
Pr
(
A
)
+
Pr
(
B
)
, where
∩
and
∪
stand
for the set operations intersection and union, respectively.
∗
In the terminology of mathematical analysis, the collection of subsets
is referred to as a
F
σ-algebra [110].
