Graphics Reference
In-Depth Information
We next describe the estimation of
dense correspondence
between a pair of
images, and the applications of this correspondence (Chapter
5
). In general, this
problem is called
optical flow
and is used in visual effects for retiming shots and cre-
ating interesting image transitions. When two cameras simultaneously film the same
scene from slightly different perspectives, such as for a live-action 3Dmovie, the cor-
respondence problem is called
stereo
. Once the dense correspondence is estimated
for a pair of images, it can be used for visual effects including video matching, image
morphing, and view synthesis.
The second part of the topic moves into three dimensions, a necessity for real-
istically merging computer-generated imagery with live-action plates. We describe
the problem of camera tracking or
matchmoving
, the estimation of the location and
orientation of a moving camera from the image sequence it produces (Chapter
6
).
We also discuss the problems of estimating the lens distortion of a camera, calibrat-
ing a camera with respect to known 3D geometry, and calibrating a stereo rig for 3D
filming.
Next, we discuss the acquisition and processing of
motion capture
data, which
is increasingly used in films and video games to help in the realistic animation of
computer-generatedcharacters (Chapter
7
).Wediscuss technology for capturing full-
body and facial motion capture data, as well as algorithms for cleaning up and post-
processing the motion capture marker trajectories. We also overview more recent,
purely vision-based techniques for markerless motion capture.
Finally, we overview the main methods for the direct acquisition of
three-
dimensional data
(Chapter
8
). Visual effects personnel routinely scan the 3D
geometry of filming locations to be able to properly insert 3D computer-generated
elements afterward, and also scan in actors' bodies and movie props to create con-
vincing digital doubles. We describe laser range-finding technology such as LiDAR
for large-scale 3D acquisition, structured-light techniques for closer-range scanning,
and more recent multi-view stereo techniques. We also discuss key algorithms for
dealing with 3D data, including feature detection, scan registration, and multi-scan
fusion.
Of course, there are many exciting technologies behind the generation of
computer-generated imagery for visual effects applications not discussed in this
topic. A short list of interesting topics includes the photorealistic generation of water,
fire, fur, and cloth; the physically accurate (or visually convincing) simulation of how
objects crumble or break; and the modeling, animation, and rendering of entirely
computer-generated characters. However, these are all topics better characterized as
computer graphics
than computer vision, in the sense that computer vision always
starts from real images or video of the natural world, while computer graphics can be
created entirely without reference to real-world imagery.
Each chapter includes a short
Industry Perspectives
section containing inter-
views with experts from top Hollywood visual effects companies including Digital
Domain, Rhythm & Hues, LOOK Effects, and Gentle Giant Studios. These sections
relate the chapter topics to real-world practice, and illuminate which techniques are
commonplace and which are rare in the visual effects industry. These interviews
should make interesting reading for academic researchers who don't know much
about filmmaking.
