Rendering, Adaptation and 3D Displays - 3DTV: Processing and Transmission of 3D Video Signals - page 131

Stereoscopic
camera pair

Leftmost camera Rightmost camera

LR

5 -View 3 D display

9 -View 3 D display

virtual view

Figure 5.4 View interpolation exploiting the complete multi-camera baseline

side of the virtual view and the other reference is on the right-hand side
of it. First, the left and the right reference images are segmented into a
main layer, a foreground layer and a background layer based on the edge
characteristics of the depth maps and the texture of both reference views.
The major segment is the main layer that comprises all pixels that are not
located in the vicinity of edges. The foreground layer comprises the pixels
that are in the close vicinity of the edges and whose depth value is lower than
that of the pixels on the other side of the edge, i.e. closer to the camera. The
remaining pixels are classified as the background layer. Figure 5.5 depicts
an exemplary segmented frame from the multi-view shooting of one of the
European Union (EU) FP7 projects MUSCADE (Multimedia Scalable 3D for
Europe) [3]. Grey pixels represent the main layer, white pixels represent the
foreground layer and the black pixels represent the background layer.
The main idea behind segmenting the reference views first is to differentiate
between reliable and unreliable depth values that are located across the
boundaries of sharp edges (i.e. depth transition areas). Subsequently, the
unreliable regions that consist of the foreground and the background layers
are treated separately from the main layer during the forward warping
and blending processes. Canny edge detector can be used to mark an
area as unreliable around the detected edges, where finding the threshold
value for the edge detector depends on the shot multi-view content and

Figure 5.5 Segmented image on the left side and the texture image on the

right side