Graphics Reference
In-Depth Information
time
time
(a)
(b)
Figure 3.37.
(a) Generalizing optimized scale-and-stretch to video encourages temporal coher-
ence between the quads in each frame. (b) Generalizing seam carving to video results in a
spatio-temporal seam that separates the left half of each image from the right half.
and introduced new constraints based on user keyframing of important objects and
structures.
Wang et al. [
534
,
535
] generalized the optimized scale-and-stretch algorithm of
[
536
] to video. In [
534
], all the frames are first aligned to compensate for camera
motion. Moving objects are detected and tracked across video frames, and con-
straints are imposed toensure that thequads on theseobjects are resizedconsistently.
The single-frame salience map is also replaced by a moving average across several
neighboring frames. Overall, the approach avoids artifacts that would be created by
retargeting each frame independently. In [
535
], the constraints on separating camera
and object motionwere relaxed, replaced by a simpler optical-flow-basedmethod for
determining a critical region in each frame that will not be removed. A cost function is
proposed that penalizes the deviation of each quad transformation from a similarity
transform and encourages temporal coherence in the quad transformations based
on the estimated optical flow, as sketched in Figure
3.37
a.
Rubinstein et al. also showed how seam carving could be applied to resize video;
the generalizations of the graphs in Figures
3.31
b and
3.33
ato
pixels are
straightforward. In this case, a cut on the graph defines a spatio-temporal seam
that cuts through the video volume, as illustrated in Figure
3.37
b. Alternately, we
couldmake the video shorter in the temporal direction by removing seams “parallel”
to the
(
x
,
y
,
t
)
plane. Since the number of vertices in the graph can be very large, a
coarse-to-fine strategy for computing the cut may be required (e.g., [
299
]).
(
x
,
y
)
3.7
INDUSTRY PERSPECTIVES
Paul Lambert, compositing supervisor, and Blake Sloan, software engineer fromDig-
ital Domain in Venice, California, Shankar Chatterjee, software developer at Cinesite
