Graphics Reference
In-Depth Information
RJR: Has markerless motion capture made any inroads into visual effects production?
Apostoloff: It's used sometimes in the film industry, mostly for body double work.
You can get a very high-resolution densemesh from that kind of setup. If you're going
to have a character that looks very similar to your actor, and you want to capture all
the nuanced performance of your actor without doing a lot of animation on top of
that, it's great.
For example, you may want some high-profile actors to appear to fall out of a
plane and perform some stunts, but you don't actually want to throw your high-
profile actors out of the plane. Instead, you might take them to one of those free-fall,
vertical wind tunnel places and capture a bunch of footage from twenty different
cameras around them, reconstruct the volume around them, and insert it into a new
scene like a blue-screen kind of effect. That kind of voxel-carving effect was used for a
sequence in
Quantum of Solace
. While the reconstructed geometry may not be great,
you'd be amazed at how believable it looks if you've got great texture. While it wasn't
for a feature film, the video game
L.A. Noire
used a kind of facial markerless mocap
for a digital body double effect. They captured high-resolution textured geometry of
people and just streamed it directly into the game to get some amazing results.
On the other hand, markerless mocap isn't a viable technique for replacing tra-
ditional mocap on a large film production at this point in time. What you can get
from a traditional motion capture system like a Vicon is incredibly accurate, and the
retargeting to different characters works very well. It's a seamless pipeline and there
are real-time systems that can do live pre-visualization of the animated characters
on set in the virtual production environment, which is very useful for the directors.
While markerless systems are incredibly convenient, because you don't have to
dress somebody up in a ridiculous outfit, what you get back from such systems is
often quite jerky and noisy. Also, you're usually restricted in the size of the volume
that you can film the actors in. If you want to shoot a scene with twenty different
actors doing a dance sequence you can't do that with markerless mocap.
One of the biggest issues is that we need to have a system that we can use for
animation after we process the mocap data. If your character looks a lot different
fromyour actor, it's not often clear how youwould use the high-resolutionmesh data
from markerless motion capture to animate a character. Taking traditional mocap
data and mapping it to a complicated network of animation controls is already a
really hard problem! We haven't reached the point where we canmap the data across
frommotion capture into animation accurately enough to worry about that little bit
of information that we lose from not having a complete mesh.
7.9
NOTES AND EXTENSIONS
Motion capture is related to the classical perceptual experiments described by
Johansson in the 1970s [
223
]. Markers were attached to the joints of a performer
wearing tight clothing (i.e., a primitive motion capture suit). The performer was
filmed by a static camera while acting out different motions in a studio, and the
resulting video was processed so that only the markers were visible in each frame.
Untrained observers watching the videos could immediately identify themotions as a
