Multiview Image Segmentation and Video Tracking - Video Segmentation and Its Applications - page 122

and segmentation is proposed in [ 34 ]. Buildings in each panoramic image are
labeled using graph cut image segmentation based on color and orientation fea-
tures. The mistakes in single-view segmentation are corrected by aggregating the
results over multiple views with the help of available depth, in which a much better
segmentation can be obtained.
To prevent the propagation of error from stereo estimation to foreground ex-
traction in the sequential approaches, depth reconstruction and object segmentation
problems are simultaneously solved by joint optimization. For the challenging out-
door environments analysis with moving cameras, for example, rugby and soccer
scenes [ 13 ], multiview scene reconstruction and segmentation are dealt with by
joint graph-cut optimization. Segmentation and depth labeling field are formulated
into the unified energy function, which involves color and contrast term for seg-
mentation, as well as the match and smoothness term for reconstruction. Joint
segmentation and reconstruction enables the high-quality scene representation of
the sport scene. By exploiting strong interdependency between 3D reconstruction
and foreground extraction, Golducke et al. [ 10 ] proposed a flexible and homoge-
nous approach to simultaneous depth estimation and background subtraction in a
multiview setting, assisted by a static background image with known depth for each
camera. The results of depth reconstruction and background separations algorithm
is obtained as minimization of energy functional, to generation a dense depth map
and foreground map.
The iterative depth-based segmentation receives the segmentation feedback from
current estimation to improve the depth reconstruction and vice verse. In order to
create the intermediate synthesized view using depth and segmentation information,
an iterative algorithm is developed in [ 26 ] which continuously performance the dis-
parity estimation and the image segmentation in the iterative circle, and improve the
result of each other. In [ 11 ], the estimated depth map and segmentation mask are
iteratively computed using an Expectation-Maximization (EM) algorithm.

5.2.2.2 Silhouette-Based Segmentation

Object segmentation and reconstruction of 3D shapes are highly related topics in the
field of computer vision and graphics. On the one hand, the acceptable segmentation
of object with accurate silhouette from considerate amount of MVIs is required to
accomplish the 3D object reconstruction, namely Shape-from-silhouette [ 5 , 22 ]to
combine the multiple silhouettes of same object from different viewpoints as source
of shape information to reconstruct the 3D models. On the other hand, the recon-
structed visual hull [ 20 ] from silhouette images, which approximately represents the
geometry of object by linking the object shape in MVIs, is capable of refining the
segmentation by projecting the visual hull onto the image plane and exploiting the
silhouette coherence across MVIs.
Silhouette-based object segmentation from MVIs has been addressed in the
recently extensive literatures. Tsai et al. [ 39 ] proposed a semiautomatic MVIs seg-
mentation algorithm for 3D modeling by integrating with visual hull reconstruction.
In the segmentation process, the automatic segmentation initialization is first carried