Graphics Reference
In-Depth Information
a
b
First stage of forward transform
Second stage of forward transform
32768
32768
32768
32768
256
256
256
256
0
0
0
0
256
256
256
256
0
0
0
0
256
256
256
256
0
0
0
0
256
256
256
256
[]
[]
×
T
×
D
D
65536
65536
65536
65536
8388608
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
S T 1 = 2 −( B + M −9) = 2 −1
>>
>>
8
1
S T 2 = 2 −( M +6) = 2 −8
32768
32768
32768
32768
32768
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Fig. 6.4 Intermediate scaling factor determination for the forward transform so that the interme-
diate and output values fit within 16-bits. B is video bit depth and M
log 2 ( N )where N is the
transform size. Worst case bit-depth analysis is done assuming a residual block with all samples
having maximum amplitude equal to
D
2 B
8 is the video bit depth), as input to the first
stage of the forward transform. ( a ) First stage of the forward transform, ( b ) Second stage of the
forward transform. Reproduced with permission from [ 6 ]. © IEEE 2013
(where B
D
In summary the constraints imposed in this section result in the following scale
factors after different transform stages:
￿ After the first forward transform stage: S T 1 D 2 ( B C M 9)
￿ After the second forward transform stage: S T 2 D 2 ( M C 6)
￿ After the first inverse transform stage: S IT 1 D 2 6
￿ After the second inverse transform stage: S IT 2 D 2 (21 B )
where B is the bit depth of the input/output signal (e.g. 8 bit) and M D log 2 ( N ).
Without quantization/de-quantization, this choice of scale factors ensures a bit
depth of 16 bit after all transform stages. However, quantization errors introduced by
the quantization/de-quantization process might increase the dynamic range before
each inverse transform stage to more than 16 bit. For example, consider the situation
where B D 8 and all input samples to the forward transform are equal to 255. In this
case, the output of the forward transform will be a DC coefficient with value equal to
255 << 7 D 32640. For high QP values and with a quantizer rounding upwards, the
input to each inverse transform stage can easily exceed the allowed 16 bit dynamic
range of [ 32768, 32767]. While clipping to 16 bit range was considered trivial
after the de-quantizer, it was considered undesirable after the first inverse transform
stage. In order to allow for quantization error of some reasonable magnitude and at
 
Search WWH ::




Custom Search