Game Development Reference
In-Depth Information
2
q
reflected by 2
q
/
[−
(
−
)
·
Q
,
+
(
−
Q
, and the dead-zone range is represented by
1
f
1
2
q
. To reconstruct the quantized transform coefficients, a de-quantization is
applied formulated as below:
f
)
·
Q
]
2
q
−
1
Q
+
q
,
x
˜
=
sign
(
x
)
·
(
|
x
|·
)
(5.6)
Q
2
(
q
+
q
)
Q
·
where
should be as close as possible to 1 to minimize the reconstruction error.
Among all components of a video coding system, quantization is one of the most
open parts for users to control the behaviors of an encoder. As
2
q
Q
has a direct impact
on the magnitude of quantization values, and directly controls the compression ratio
which is the most important feature of a video encoding process. To meet different
requirements in various application scenarios, a series of
Q
and q can be configured,
so that a wide range of compression ratio can be supported.
Since the standard only specifies the decoding part, it leaves much flexibility
for the quantization design at encoder. For high performance video encoding, a rate-
distortion optimized quantization (RDOQ) method (Karczewicz et al.
2008
)isalways
applied which allows quantization outputs to be further tuned around the output of the
most straightforward quantization process Eq.
5.5
based on rate-distortion criterion.
After the quantization, the transform coefficient values are converted to much
smaller levels. The quantized transform coefficients of a block are further scanned
to a one-dimensional vector and fed into the entropy coding engine. In next two
sections, we will discuss more on the detailed technical design of transform and
quantization of AVS1 and AVS2, and the transform coefficient coding part will be
introduced in the next chapter.
5.1.3 Recent Development of Transform Design
Although it has been proven that
DCT
is efficient for coding natural imagery sources,
there still has limitations since it is a fixed transformwhich is impossible to cover the
dynamic variations of image statistics. Nowadays, since the computing capabilities
of various devices are increasing rapidly, researchers have been exploiting novel
transforms to further improve the coding efficiency on top of
DCT
. Several advanced
new transforms have been proposed in recent years, and the possibility of further
improvement on the coding efficiency of transform has been verified in various ways.
In this subsection, we will introduce several new transforms including directional
DCT
, Mode-Dependent Directional Transform (
MDDT
),
DST
and
RDOT
.
The conventional 2D-DCT assumes separate statistics along the horizontal and
vertical directions, however, in natural images, edges can be arbitrarily directed. To
break though the limitation of conventional 2D-DCT, a directional
DCT
is proposed
in Zeng and Fu (
2008
) which aims at capturing the directionality of intrapredicted
residues. The basic idea of directional
DCT
is illustrated in Fig.
5.2
, where a 4
4
block is predicted using the diagonal down-left intraprediction mode. At the first
×
