Transform and Quantization - Advanced Video Coding Systems - page 85

B ij , and CS is the combination of S , S , Q
indicates the quantization step size. In Eq. 5.11 , CS is a constant matrix (Yu et al.
2005 ) and Q can be configured to control the quantization step size. According to the
forward transformand quantization process defined in Eqs. 5.7 and 5.11 , respectively,
the inverse and dequantization process is defined as below,

= // =

A ij /

where C A B means C ij

X = ( T 8 × ( Y T 8 ⊗ Q ) × ), (5.12)

It is noted that at the decoder end, no scaling operation is involved and both S and
S have been dealt with at the encoder end, as shown in Eq. 5.12 . For practical imple-
mentation of video codec, where divisions are usually avoided, Eqs. 5.11 and 5.9 are
reformulated to an identical expression with only multiplications and shifts. Also for
low-complexity inverse transform and dequantization, in AVS1, shifting operations
are applied separately in the horizontal and vertical inverse transform to keep the
intermediate data within the range of 16-bit representation.

5.3 Transform and Quantization in AVS2

In AVS2, a transform unit (TU) is defined for transform coding for a coding block,
concluding one luma transform block and two chroma transform blocks. For intra
luma CU, the TU size is related with the PU type. Specifically, if the PU is 2 N

×

2 N ,

× ×

transform block size is 2 N 2 N , and if the PU is N N , the transform size is also

16 CU), non-square
transform will be used. For inter luma CU, the TU can be set to 2 N

N × N . If the SDIP prediction is used (only for 32 × 32 and 16 ×

× 2 N transform

size or split into four square N 2 N transform

blocks according to the prediction unit partitions, as shown in Fig. 3.6 . The TU size is
typically decided at the encoder using rate-distortion optimization, called two-level
transform in AVS2.
For chroma transform block, as the chroma component generally presents much
smoother textures than luma component, the transform block is restricted to be the
same size of the coding block. Therefore, the transform block size selection for luma
and chroma may be not corresponding since there is flexibility for luma transform
size but no flexibility for chroma transform size. It is observed in (He et al., 2013 )
that this approach is able to achieve coding gains for complex texture picture regions
which generate semi-random residuals and less complexity is needed for decoding
a chroma transform block since the transform block size is always hooked up with
the coding block size.

× N or non-square 2 N × 0 . 5 N / 0 . 5 N ×