Multiview Image Segmentation and Video Tracking - Video Segmentation and Its Applications - page 123

out by graph-cut based image segmentation activated by trimap labelling. Then,
it asks for user's interaction to choose a subset of the segmentation results with
satisfactory quality for 3D reconstruction using volumetric graph cut and learning
shape priors. At last, those discontent segmentations will be refined with the help
of 3D model projection and the learned shape priors to propagate the successful
segmentation and rectify the segmentation errors. An automatic algorithm in [ 6 ]
dedicated to obtain the 3D segmentation of rigid object using volumetric graph-cut.
Camera fixation constraint is adopted to initialize the OOI and the color model. Iter-
ative refinement of silhouette extraction and visual hull estimation is performed by
volumetric graph-cut optimization, which ensures that the resulting silhouettes by
propagating the computed visual hull back to the individual view are consistent with
one another at every iteration. Grauman et al. [ 12 ] presented a Bayesian approach to
visual hull reconstruction using image-based representation of extracted silhouette
from pedestrian images. The basic background subtraction results in rough seg-
mentation corrupted by noise of each viewpoint in the simple color background.
The visual hull of pedestrian is reconstructed by PPCA-based Bayesian model from
problematic silhouettes. The used class-specific prior in visual hull reconstruction
reduces the effect of segmentation errors in the silhouette extraction process.
The multiview segmentation algorithms using silhouette and visual hull as
mentioned above are developed for equi-tilt set or tunable sequence relying on
the known background, or simple background for a specific object class. For the
silhouette extraction with arbitrarily unknown background, object segmentation
methods are proposed in semiautomatic or fully automatic manner. By employing
the intersection consistency in 3D space and projection consistency in 2D images,
Zeng and Quan [ 48 ] proposed a silhouette extraction algorithm from multiple im-
ages of unknown background, and a silhouette carving algorithm for robust visual
hull reconstruction as extension. In [ 30 ], provided the minimum user input to hardly
constraint the “target object” and “background” pixels in only one of MVI, tentative
segmentation of all views is achieved using traditional graph cut technique. Then,
the visual hull of an object with calibrated cameras is reconstructed from silhouette
of MVI, and final results are acquired by back-projection 3D model to 2D images to
eliminate the segmentation errors in the tentative stage. Lee et al. [ 24 ] proposed an
automatic foreground extraction method which can simultaneously identify region
of interest in MVIs without any a priori knowledge on the background and user
interaction. Driven by the initial segmentation from the intersection of viewing
volumes, iterative optimal segmentation of all views is conducted using graph cut
method, where the prior term in the energy function encodes the spatial consistency
exploiting the multiview silhouette coherence.

5.2.3 Proposed Multiview Image Segmentation Algorithm

Detection and Localization of OOIs is the first but important step for the video
tracking. To initialize the tracked OOIs in the sequence, we propose a depth-based