Digital Signal Processing Reference
In-Depth Information
1
p
(
m
1
)
p
1
p
1
p
2
I
(
m
2
/
m
1
)
=
log
2
m
1
)
=
log
2
m
1
,
m
2
)
=
log
2
(8.16)
p
(
m
2
/
p
(
where p
1
and p
2
are independent probabilities of m
1
and m
2
.
Plain English text is a good example of a Markov source. Probability of any letter
depends upon the earlier letter/s. For example, probability of a 'u' after 'q' can be
considered 100%. Hence, once a 'q' has occurred, a 'u' thereafter brings no new
information.
Mathematically speaking, p(u/q)
=
1. Hence, information content of 'u' after
1
1
1
=
'q'
=
I
)
=
log
2
)
=
log
2
0
(
u
/
q
p
(
u
/
q
8.5 Discrete Memoryless Channel (DMC)
8.5.1 Channel Representation
As we know, a communication channel may be defined as the medium through
which the symbols flow from the transmitter to the receiver end. A
discrete memo-
ryless channel
is a statistical model with an input random variable X and an output
random variable Y, as shown in Fig.
8.3
. At each unit duration, one symbol from
input is accepted and one symbol at the output responds. In discrete channel, the
number of alphabets in X and Y are both finite and in memoryless channel, the
current output only depends on the current input not the delayed (latched) previous
one.
According to the Fig.
8.2
, the input X consist of
x
i
s with i
=
...
m with a pri-
ory known probabilities
P
(
x
i
) s respectively. Similarly, output Y consist of
y
i
s with
i
1, 2,
=
1, 2,
...
m. Each input-to-output path is indicated alongwith the conditional prob-
ability
P
y
j
|
x
i
is the conditional probability of obtaining output
y
j
given that the input is
x
i
, and is called a
channel transition probability.
x
i
, where
P
y
j
|
x
1
Fig. 8.3
Representation of
discrete memoryless channel
y
1
y
2
x
2
(
)
P
y
j
⏐
x
i
X
Y
M
M
x
m
y
n
8.5.2 The Channel Matrix
A channel is specified by a complete set of transition probabilities. According to the
Fig.
8.2
, the channel is specified by the matrix of transition probabilities[
P
(
Y
|
X
)
].
The matrix is given by
Search WWH ::
Custom Search