Databases Reference
In-Depth Information
Probability and Random Processes
In this appendix we will look at some of the concepts relating to probability and
random processes that are important in the study of systems. Our coverage will
be highly selective and somewhat superficial, but enough to use probability and
random processes as a tool in understanding data compression systems.
A.1 Probability
There are several different ways of defining and thinking about probability. Each approach
has some merit; perhaps the best approach is the one that provides the most insight into the
problem being studied.
A.1.1 Frequency of Occurrence
The most common way that most people think about probability is in terms of outcomes, or
sets of outcomes, of an experiment. Let us suppose we conduct an experiment
E
that has
N
possible outcomes. We conduct the experiment
n
T
times. If the outcome
ω
i
occurs
n
i
times,
n
i
n
T
we say that the frequency of occurrence of the outcome
ω
i
is
. We can then define the
probability of occurrence of the outcome
ω
i
as
n
i
n
T
P
(ω
i
)
=
lim
n
T
→∞
In practice we do not have the ability to repeat an experiment an infinite number of times,
so we often use the frequency of occurrence as an approximation to the probability. To make
this more concrete consider a specific experiment. Suppose we turn on a television 1,000,000
times. Of these times, 800,000 times we turn the television on during a commercial and 200,000
times we turn it on and don't get a commercial. We could say the frequency of occurrence, or
