Image Processing Reference
In-Depth Information
comparisons are calculated using individual color and depth map images. The
Structural SIMilarity maps for color and depth map sequences (i.e., E-SSIM
C
and
E-SSIM
D
) are therefore defined by (
9.9
) and (
9.10
) respectively.
α
ʲ
ʳ
E-SSIM
C
¼
½
l
C
x
ðÞ
;
y
½
c
C
x
ðÞ
;
y
½
s
e
x
ðÞ
;
y
ð
9
:
9
Þ
α
ʲ
ʳ
E-SSIM
D
¼
½
l
D
x
ðÞ
;
y
½
c
D
x
ðÞ
;
y
½
s
e
x
ðÞ
;
y
ð
9
:
10
Þ
where
l
C
(
x
,
y
) and
l
D
(
x
,
y
) are
luminance
comparisons of color and depth map
sequences respectively,
C
C
(
x
,
y
) and
C
D
(
x
,
y
) are contrast comparisons of color and
depth map sequences respectively, and
S
e
(
x
,
y
) is the
structural
comparison between
the edge/gradient maps of the received color and depth map sequences.
Since the proposed method heavily depends on the accuracy of edge detection, a
suitable edge detection scheme has to be utilized [
33
]. In this work, the
Sobel
operator is selected to obtain edge information (i.e., the binary edge mask) due to its
simplicity and efficiency [
34
]:
Sobel
filtering is typically used when real time
operation is needed [
35
]. This is important in scenario where “on-the-fly” system
adaptation is performed based on video quality feedback.
In the proposed method, initially the horizontal and vertical edges (i.e.,
E
Horizontal
(
x
,
y
) and
E
Vertical
(
x
,
y
)) are calculated by applying the
Sobel
filter in
each image dimension. Then the gradient image (
G
(
x
,
y
)) is calculated as follows;
q
E
Horizontal
x
E
Vertical
x
Gx
ðÞ
¼
;
y
ðÞ
;
y
2
þ
ðÞ
;
y
2
ð
9
:
11
Þ
The extracted edge/gradient information (
G
(
x
,
y
)) from the processed/received
color and depth maps images is employed to quantify the
structural
degradation
of the image.
The amount of edges detected in the sequence depends on a number of factors,
including the edge detection threshold used and the amount of compression (
QP
values being used), besides the characteristics of the sequence. For instance, when
compressed with higher
QP
values, images are smoothed (due to the removal of high
frequencies) and
blockiness
can occur when using
DCT
based image encoders, hence
detection of accurate edges can be difficult. Therefore, the
structural
comparison
performed with the proposed method is sensitive to the compression level being used.
This may tend to get saturated
structural
degradation ratings at higher
QP
values.
Even though the gradient/edge information of color and depth map images
provides a significant indication of the structural degradation of the processed
images, due to the abstract level of information being used with the proposed
method, the quality ratings may slightly vary compared to the
FR
SSIM method
(the
FR
method uses the complete, original image sequence as the reference).
In order to minimize the difference between the proposed method and
FR
quality
evaluation methods, relationships can be derived between them based on experi-
mental findings. For instance, the measured quality with the proposed method (i.e.,
structural similarity maps E-SSIM
C
and E-SSIM
D
) can be approximated based on
