Digital Signal Processing Reference
In-Depth Information
object-based segmentation in demand. A comprehensive review on the object-based
multiview image segmentation and our proposed algorithm will discuss in the
following sections.
5.2.2
Object-Based Multiview Image Segmentation
Localizing and extracting the OOIs in MVIs is the objective for object-based mul-
tiview image segmentation. Based on the different methodologies involved, the
existing algorithms of object-based multiview image segmentation can be grouped
into depth-based segmentation and silhouette-based segmentation.
5.2.2.1
Depth-Based Segmentation
Depth information reconstructed from MVIs usually serves as a valuable source
in various related techniques such as 3D reconstruction [
18
], image-based render-
ing [
32
], freeview video generation [
31
], MVI/V compression [
45
] and virtual view
synthesis [
46
]. Comparing with the 2D analysis and processing, the recovered depth
information from the geometric relationship of MVIs assists in understanding and
visualizing the 3D world in more efficient way. Accurate object segmentation in the
clutter scene and complicated scenario is almost impossible or error-prone with-
out any semantic knowledge about the scene or only relying on the 2D information
(color, texture, and spatial location) from single-view images, since the semantic
object is not always homogenous with these low-level features. By assuming that
object locates in the different depth layer in the 3D scene and the depth value over
one object forms smooth and consistent distributions, semantic objects can be ex-
tracted with known depth and segmentation performance using 2D features can be
improved. However, object segmentation only exploiting the depth data is problem-
atic due to the inaccuracy of the depth reconstruction resulting from the inherent
difficulties of stereo matching such as the lack of textures and occlusion. Thus, to
obtain more precise and robust segmentation for object-level manipulation, intelli-
gent fusion of depth with other features should be taken into account.
Depth reconstruction and multiview segmentation is generally addressed in
the sequential, joint or iterative fashion in a number of literatures. The most
straightforward way for depth-based segmentation is to perform depth estima-
tion beforehand, and then incorporate the depth information into the segmentation
framework. Kolmogorov et al. in [
19
] described models and algorithms for bi-
layer segmentation of stereoscopic frames. Stereo disparity is obtained by dynamic
programming in Layered Dynamic Programming algorithm, and stereo match like-
lihood is then probabilistically fused with contrast-sensitive color model to segment
each frame by ternary graph cut. Good quality segmentation of temporal sequence
can be achieved by marginalizing explicit temporal consistency in the realtime
system. To automatically align the panoramic images and segment building from
multiview city-scale street view, a fast and accurate method for multiview alignment
