Digital Signal Processing Reference
In-Depth Information
11
ENTROPY, PREDICTIVE CODING
AND QUANTIZATION
CHAPTER OUTLINE
11.1 Entropy 69
11.2 Huffman Coding 71
11.3 Markov Source 72
11.4 Predictive Coding 73
11.5 Differential Encoding
74
11.6 Lossless Compression
75
11.7 Quantization
76
11.8 Decibels
79
In this chapter, we will discuss some of the basic concepts of
data compression, including video and image compression. Until
now we have considered only uncompressed video formats, such
as RGB or YCrCb, where each pixel is individually represented
(although this is not strictly true for 4:2:2 or 4:2:0 forms of YCrCb).
However, high levels of compression are possible with little loss of
video quality. Reducing the data needed to represent an indi-
vidual image or a sequence of video frames is very important
when considering how much storage is needed on a camera flash
chip or computer hard disk, or the bandwidth needed to transport
cable or satellite television, or stream video to a computer or
handheld wireless device.
11.1 Entropy
We will start with the concept of entropy. Some readers may
recall from studies in thermodynamics or physics that entropy is
a measure of the disorderliness of a system. Further, the second
law of thermodynamics states that in a closed system entropy can
only increase, and never decrease. In the study of compression,
and also the related field of error correction, entropy can be
thought of as the measure of unpredictability. Entropy can be
applied to a set of digital data.
