Image Processing Reference
In-Depth Information
right video or color and depth video) [
15
-
17
]. For instance, “depth perception” is
highly correlated to the average PSNR of the rendered left and right image sequences
[
15
]. This means that we could use individual objective quality measures of different
3D video components to predict the true user perception in place of subjective quality
evaluation, through a suitable approximation derived based on correlation analysis.
However, with some 3D source representations such as color and depth map 3D
image format, it may be difficult to derive a direct relationship between objective
measures and subjective quality ratings. For instance, objective quality of depth map
may have a very weak correlation on its own for overall subjective quality, because
depth map is used for projecting the corresponding color image into 3D coordinates
and it is not directly viewed by the end users. All the methods described above are FR
methods and need the original 3D image sequence to measure the quality, hence they
are less useful in real-time transmission applications. The solution is to use Reduced-
Reference (RR) or No-Reference (NR) metrics which need fewer or zero bits
respectively to transmit the side-information for the original image sequence. RR
and NR quality metrics are proposed in the literature for 2D video [
18
-
21
], but only a
few are reported so far for 3D video [
22
]. For instance, [
22
] proposed a NR quality
metric for 3D images based on left and right stereoscopic images. The proposed Near
NR quality assessment in this chapter considers instead stereoscopic image sequences
based on color plus depth 3D video format. Due to recent advances in free viewpoint
3D video delivery with multi-color and multi-depth map 3D format [
23
], quality
metrics as proposed in this chapter could be effectively used in emerging
applications.
Color plus depth 3D video enables us to render virtual left and right views based
on a 3D image warping method commonly known as Depth Image Based Render-
ing (DIBR) [
24
]. This representation consists of two images per video frame, one
representing color information, similar to 2D video, the other representing depth
information. The edge information of both color and depth map images can
represent different depth levels and object boundaries (see Fig.
9.1
). The major
boundaries of image objects of both color and depth are coincident. This informa-
tion can be exploited to quantify irregularities of color and depth map sequences.
This chapter therefore proposes a Near NR quality evaluation approach for color
plus depth format 3D video based on edge information characteristics. Edge
information extracted from images has been employed in the past in measuring
2D image quality (e.g., [
25
,
26
]). In our previous work, a RR metric utilizing
gradient information was proposed for depth maps associated with color plus
depth based 3D video [
27
,
28
]. It shows comparable results to FR methods, but
limited only for measuring depth map quality. Furthermore, the overhead for side-
information, although limited, was of the order of kbps. In this chapter, we propose
a global quality evaluation method for color plus depth 3D video based on edge
extraction, requiring only a few bytes per second for the transmission of side-
information (Near NR).
This chapter is organized as follows: Sect.
9.2
reports the proposed Near NR
quality metric for color plus depth 3D video transmission. The experimental setup,
results, and discussion are presented in Sect.
9.3
. Section
9.4
concludes the chapter.
