Image Processing Reference
In-Depth Information
intensity of the image (using the mean value) whereas
contrast
comparison pro-
vides an indication about the variation of pixel intensity of the image frame (using
the standard deviation of individual images).
Structural
comparison provides an
indication about structural degradation compared to the original image by calcu-
lating the covariance between the original and processed images. Mean SSIM
(MSSIM) index for overall image quality evaluation is defined as;
M
X
M
SSIM
x
j
,
y
j
1
MSSIM
X
ð
;
Y
Þ
¼
ð
9
:
5
Þ
j
¼1
where
X
and
Y
are the reference and the distorted images, respectively;
x
j
and
y
j
are
the image contents at the
j
th local window; and
M
is the number of local windows in
the image. MSSIM represents the mean SSIM and it is a measure of the structure
similarity between images
X
and
Y
.
In the proposed method,
luminance
and
contrast
comparisons are performed
based on image statistics (
mean
and
standard deviation
of individual images)
generated at the sender and receiver ends (see [
31
] for further details about
calculating
luminance
and
contrast
comparisons). The
structural
comparison is
performed between the generated edge information of the processed color and
corresponding depth map images. If
X
0
and
Y
0
represent the edge/gradient maps of
the received color and the received depth map image respectively, the edge/gradient
based structural comparison
s
e
(
x
,
y
) can be defined as;
ðÞ
¼
˃
x
0
y
0
þ
C
˃
x
0
˃
y
0
þ
S
e
x
;
y
ð
9
:
6
Þ
C
where
x
0
and
y
0
are block vectors of
X
0
and
Y
0
respectively,
˃
x
0
and
˃
y
0
are the
standard deviation of vector
x
0
and
y
0
respectively,
˃
x
0
y
0
is the covariance of vector
x
0
and
y
0
, and
C
is a small constant to avoid the denominator being zero. Then the edge
based structural similarity metric (E-SSIM) can be defined as follows;
α
ʲ
ʳ
E-SSIM
¼
½
lx
ðÞ
;
y
½
cx
ðÞ
;
y
½
s
e
x
ðÞ
;
y
ð
9
:
7
Þ
Then the mean image quality is calculated as;
M
X
M
E-SSIM
x
j
,
y
j
1
M
E-SSIM
X
½
ð
;
Y
Þ
¼
ð
:
Þ
9
8
j
¼1
A similar SSIM metric has been proposed for 2D video in [
32
]. However, in this
case the edge information of the original and processed images has been used for
structural comparison.
The
structural
comparison term (i.e.,
s
e
(
x
,
y
)) is common for both color and depth
map quality evaluation and only the terms associated with
luminance
and
contrast
