Graphics Reference
In-Depth Information
where the second image was taken. If the images are taken of the same object at the
same time, the effect is similar to the “bullet time” effect from
The Matrix
; objects
appear frozen in time as the camera moves around them. If the images are actually of
different objects from different perspectives, the morphing result seems to simulta-
neously transform one object into the other while moving the virtual camera. Radke
et al. [
380
] generalized view morphing to create an interpolated virtual video from
two source videos, allowing independent temporal manipulation effects and virtual
camera motion.
Mahajan et al. [
310
] noted that the warping and cross-dissolving steps required
in morphing and view synthesis tend to blur synthetic views due to repeated pixel
resampling and averaging operations (this blurriness is visible in the rectified and
synthetic views inFigure
5.30
). Instead, inspiredby some of themethodswe discussed
inChapter
3
, they proposed to synthesize sharper virtual images so that eachpixel in a
synthetic view is taken fromexactly one of the source images. Thus, aswemorph from
one image to the next, the problem is determining a
transition point
for each pixel
that specifies which source image should be used for that pixel, and which location
in that source image generates the pixel's intensity. The algorithm involves a graph-
cut optimization for determining the transition points and Poisson reconstruction to
generate the final synthetic view from its gradients.
5.9
INDUSTRY PERSPECTIVES
Marty Ryan, senior software engineer at Rhythm & Hues Studios in El Segundo, CA
and Shankar Chatterjee, software developer at Cinesite inHollywood, CA, discuss the
role of optical flow in visual effects.
RJR: What are the most common applications of optical flow in visual effects?
Ryan: The number-one use is for retiming a sequence — speeding it up or slowing it
down. The technology is at the point where retiming is fairly routine; it's not just for
stunts where they film a slow-speed crash but want it to appear faster. These days,
even in non-action films they'll move you forward in time or slow you down. You
barely notice it, but the filmmaker often uses it to make a point.
A less glamorous example is that we do a lot of films where we make animals like
dogs appear to talk. To make the talking effect, we need to replace the jaw of the
dog and estimate the background behind it. Since painting a single clean plate of the
background is very labor-intensive, the artist does it for one image, warps it into later
frames using the optical flow until it no longer looks good, then paints a new clean
plate, warps that one forward, and so on. It's a very low-tech application, but one of
those invisible effects that we use a lot.
Chatterjee: We also commonly use optical flow for retiming — one example is
Thir-
teen Days
, a movie about the Cuban missile crisis. In the very last frames there was a
slowdown based on my optical flow algorithm. Optical flow was also used for retim-
ing in the movie
Shanghai Noon
; they wanted to show Jackie Chan's movement in
slightly slower than real time, to get the full impact.
