Learning 3D Face Deformation Model: Methods and Applications - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 318

audio-to-visual mapping using neural networks for real-time, speech-
driven, 3D face animation. Moreover, the framework includes parts-based
MUs because of the local facial motion and an interpolation scheme to
adapt MUs to arbitrary face geometry and mesh topology. Experiments
show we can achieve natural face animation and robust non-rigid face
tracking in our framework.
Introduction
A synthetic human face provides an effective solution for delivering and
visualizing information related to the human face. A realistic, talking face is
useful for many applications: visual telecommunication (Aizawa & Huang,
1995), virtual environments (Leung et al., 2000), and synthetic agents (Pandzic,
Ostermann & Millen, 1999).
One of the key issues of 3D face analysis (tracking and recognition) and
synthesis (animation) is to model both temporal and spatial facial deformation.
Traditionally, spatial face deformation is controlled by certain facial deformation
control models and the dynamics of the control models define the temporal
deformation. However, facial deformation is complex and often includes subtle
expressional variations. Furthermore, people are very sensitive to facial appear-
ance. Therefore, traditional models usually require extensive manual adjustment
for plausible animation. Recently, the advance of motion capture techniques has
sparked data-driven methods (e.g., Guenter et al., 1998). These techniques
achieve realistic animation by using real face motion data to drive 3D face
animation. However, the basic data-driven methods are inherently cumbersome
because they require a large amount of data for producing each animation.
Besides, it is difficult to use them for facial motion analysis.
More recently, machine learning techniques have been used to learn compact
and flexible face deformation models from motion capture data. The learned
models have been shown to be useful for realistic face motion synthesis and
efficient face motion analysis. In order to allow machine-learning-based ap-
proaches to address the problems of facial deformation, analysis and synthesis
in a systematic way, a unified framework is demanded. The unified framework
needs to address the following problems: (1) how to learn a compact model from
motion capture data for 3D face deformation; and (2) how to use the model for
robust facial motion analysis and flexible animation.
In this chapter, we present a unified machine-learning-based framework on
facial deformation modeling, facial motion analysis and synthesis. The frame-
work is illustrated in Figure 1. In this framework, we first learn from extensive