Image Matting - Computer Vision for Visual Effects - page 42

Figure 2.23. Images with shadows don't obey the assump-
tions of the matting equation and pose challenges for the
algorithms described so far.

Li et al. [ 279 ] proposed a natural generalization of the Lazy Snapping work from
Section 2.8.1 to video. The Gibbs energy formulation is similar to the methods in
Section 2.8 , but thenodes in the graph (here, image superpixels) are connectedboth in
space and time, with inter-frame edgeweights estimated similarly to intra-frame edge
weights. Criminisi et al. [ 107 ] also posed video segmentation as a conditional random
field energy minimized with graph cuts, but added an explicit learned prior on the
foreground likelihood at a pixel based on its label in the previous two frames. Non-
binary values in these techniques are typically obtained independently per frame by

applying a “bordermatting” algorithmsimilar toGrabCut (Figure 2.21 b-c).Wang et al.
[ 530 ] also proposed a graph-cut-based video segmentation method, but extended
the superpixel formation, user stroking, and border matting algorithms to operate
natively in the space-time “video volume” formed by stacking the frames at each
time instant. Finally, Bai et al. [ 26 ] proposed to propagate and update local classifiers
applied at points distributed around the foreground boundary of the previous frame
to generate constraints for the graph cut at the current frame. This was followed by
a space-time version of robust matting (Section 2.6.1 ) that rewards consistency with
the

α

α values from the previous frame.

2.10 MATTING EXTENSIONS

2.10.1 Violations of the Matting Equation

When a matte of an object needs to be created from an image containing strong
directional lighting, the shadow of the object should intuitively be part of the fore-
ground instead of part of the background. However, shadows don't obey the matting
equation, since they arise from a different method of image formation. That is, shad-
ows come from the occlusion of light by a physical object, not by the translucent
mixture of a gray foreground object with the background (Figure 2.23 ).
Chuang et al. [ 97 ] proposed an algorithm for pulling shadow mattes based on a
shadow compositing equation similar to Equation ( 2.2 ). This approach relied on a