Digital Signal Processing Reference
In-Depth Information
equally distributed, meaning each of the 256 possible values has
a probability of 1/256, and an entropy of eight bits per pixel. The
possible outcomes of the differential encoder tend to be much
more probable for small values (due to the correlation to nearby
pixels) and much less likely for larger values. As we saw in our
simple example above, when the probabilities are not evenly
distributed, the entropy is lower. Lower entropy means less infor-
mation. Therefore, on average, significantly fewer bits are required
to represent the image, and the only cost is increased complexity
due to the predictor and differential encoding computations.
Let's assume that averaged over a complete video frame, the
probabilities work out thus:
Pixel color value Probability
¼
1/256 for values in range of
0 to 255
And that after differential encoding, the probability distribu-
tion comes out as:
Differential color
Probability
1/16
for value equal to 0
[
Differential color
Probability ¼ 1/25
for values in the range 8to1, 1 to 8
Differential color
Probability ¼ 1/400
for values in the range 32 to 9, 9 to 32
Differential color
Probability ¼ 1/5575
for values in the range 255 to 33, 33 to 255
We earlier defined entropy as:
Entropy
¼
þ
prob (outcome
1
)
info (outcome
1
)
prob (outcome
2
)
info (outcome
2
)
þ
.
prob (outcome
n
)
info
(outcome
n
)
with info of outcome
log
2
(1 / probability of that outcome)
In the first case, we have 256 outcomes, each of probability
1/256:
Entropy
¼
8 bits
In the second case, we have 511 possible outcomes, with one
of four probabilities:
Entropy
¼
256
(1 / 256)
log
2
(1 / (1 / 256) )
¼
¼
1
(1 / 16)
log
2
(1 / (1 / 16))
þ
16
(1 / 25)
log
2
(1 / (1 / 25))
þ
48
(1 / 400)
log
2
(1 / (1 / 400))
þ
446
(1 / 5575)
log
2
(1 / (1 / 5575))
¼
0.250
þ
2.972
þ
1.037
þ
0.996
¼
5.255 bits
11.6 Lossless Compression
The entropy has been significantly reduced through the use of
the differential coder. While this step function probability
distribution is obviously contrived, typical entropy values for
various differential encoders across actual video frame data tend
