Game Development Reference
In-Depth Information
the signal to frequency domain. Then each channel is normalized by the perceptual
importance function, such as the contrast sensitivity function. The final measure is
derived by pooling the error in each sub-band. This complex distortion measure sys-
tem is trying to precisely simulate all components of HVS, so it is usually defined
to be “bottom-up” bit rate reductionapproach (Wang and Bovik
2006
). Analogous
to this system, in video coding the residuals are also decomposed into several sub-
bands, and each transform coefficient is then subject to quantization and entropy
coding. Therefore, the bottom-up approach can be directly employed in quantization
optimization.
It is found that the natural image is highly structured and HVS is adapted to the
structural information in images. This leads to the well-known structural similarity
(SSIM) index. The SSIM index brings image quality assessment from pixel-based
stage to structural-based stage. It is defined to be the product of the luminance
comparison, contrast comparison, and structure comparison between two patches,
(
2
μ
x
μ
y
+
C
1
)(
2
˃
xy
+
C
2
)
S
(
x
,
y
)
=
(12.6)
x
+
μ
y
+
x
+
˃
y
+
(μ
C
1
)(˃
C
2
)
where
μ
x
and
˃
x
denote the expectation and standard deviation of the local signal
x, and
˃
xy
denotes the correlation coefficient. SSIM index of the whole image is
obtained by averaging the local SSIM indices calculated using a sliding window.
The SSIM index follows a “top-down” philosophy, which assumes the entire func-
tion of the HVS (Wang and Bovik
2006
). By contrast, in the “bottom-up” schemes,
the functionality of each component in HVS is considered, and the computational
model aims to simulate exactly what HVS does. Yet, due to the very limit knowledge
about HVS, the “bottom-up” schemes may not be able to fully function the way as
HVS. In the future, better understanding about HVS can be very helpful in quality
assessment.
Another way to evaluate the image quality is to quantify the visual information
contained in the image. In Sheikh and Bovik (
2006
), the researchers proposed the
Visual Information Fidelity (VIF) approach, which models the distorted image by
a divisive normalization-based masking model for the HVS. Compared with other
image quality assessment algorithms, VIF can accurately quantify the visual qual-
ity improvement when the visual image quality has been enhanced by a contrast
enhancement operation. More recently, low level features such as edge and gradient
information have been recognized as important cues for HVS to interpret the scene.
In Zhang et al. (
2011
), the phase congruency and image gradient magnitude are
employed to quantify the image quality. In Liu et al. (
2012
), the gradient similarity
measure is proposed based on the observation that structural and contrast changes
can be effectively captured with gradient information.
For natural video, both spatial and temporal information should be considered
for quality assessment. In Seshadrinathan and Bovik (
2010
), the video quality is not
only evaluated in spatial and temporal domain, but also in spatial-temporal domain,
in which the motion quality along the motion trajectories is taken into account.
