HEVC Transform and Quantization - High Efficiency Video Coding (HEVC) - page 155

The alternate transform provides around 1 % bit-rate reduction while coding
intra pictures [ 25 ]. In intra-picture prediction, a block is predicted from left and/or
top neighboring samples. The prediction quality is better near the left and/or top
boundary resulting in an intra-prediction residual that tends to have lower amplitude
near the boundary samples and higher amplitudes away from the boundary samples.
The DST basis functions are better than the DCT basis functions in modeling this
spatial characteristic of the intra prediction residual. This can be seen from the first
row (basis function) of the alternate transform matrix which increases from left to
right as opposed to the DCT transform matrix that has a flat first row. A theoretical
analysis of the optimality of DST for intra-prediction residual is provided in [ 25 ].
During the course of the development of HEVC, alternate transforms for
transform block sizes of 8 8 and higher were also studied. However, only the
4 4 alternate transform was adopted in HEVC since the additional coding gain
from using the larger alternate transforms was not significant (also, their complexity
is higher since there is no symmetry in the transform matrix and a full matrix
multiplication is needed to implement them for transform sizes 8 8 and larger).

6.3 Quantization and De-quantization

Quantization consists of division by a quantization step size ( Qstep ) and subsequent
rounding while inverse quantization consists of multiplication by the quantization
step size. Here, Qstep refers to the equivalent step size for an orthonormal transform,
i.e. without the scaling factors of Tables 6.2 and 6.3 . Similar to H.264/AVC [ 27 ],
a quantization parameter ( QP ) is used to determine the quantization step size in
HEVC. QP can take 52 values from 0 to 51 for 8-bit video sequences. An increase
of 1 in QP means an increase of the quantization step size by approximately 12 %
(i.e., 2 1/6 ). An increase of 6 leads to an increase in the quantization step size by a
factor of 2. In addition to specifying the relative difference between the step-sizes
of two consecutive QP values, there is also a need to define the absolute step-size
associated with the range of QP values. This was done by selecting Qstep D 1for
QP D 4.
The resulting relationship between QP and the equivalent quantization step size
for an orthonormal transform is now given by:

Qstep.QP / D 2 1=6 QP 4 (6.6)

Figure 6.6 shows how the quantization step size increases non-linearly with QP .
Equation ( 6.6 ) can be also be expressed as:

QP
6

Qstep.QP / D G QP %6 << (6.7)