Learning 3D Face Deformation Model: Methods and Applications - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 322

Machine Learning Techniques for Facial Deformation
Modeling, Analysis and Synthesis
Artificial Neural Network (ANN) is a powerful tool to approximate functions.
It has been used to approximate the functional relationship between motion
capture data and the parameters of pre-defined facial deformation models
(Morishima, Ishikawa & Terzopoulos, 1998). This helps to automate the con-
struction of a physics-based face muscle model. Moreover, ANN has been used
to learn the correlation between facial deformation and other related signals,
such as speech (Morishima & Harashima, 1991; Lavagetto, 1995; Massaro et al.,
1999).
Because facial deformation is complex, yet structured, Principal Component
Analysis (PCA) (Jolliffe, 1986) has been applied to learn a low-dimensional
linear subspace representation of 3D face deformation (Kshirsagar, Molet &
Thalmann, 2001; Reveret & Essa, 2001). Then, arbitrary complex face deforma-
tion can be approximated by a linear combination of just a few basis vectors.
Moreover, the low-dimensional linear subspace can be used to constrain noisy
low-level motion estimation to achieve more robust 3D facial motion analysis
(Reveret & Essa, 2001).
The dynamics of facial motion is complex, so it is difficult to model with analytic
equations. A data-driven model, such as the Hidden Markov Model (HMM)
(Rabiner, 1989), provides an effective alternative. One example is “voice
puppetry” (Brand, 1999), where an HMM trained by entropy minimization is used
to learn a dynamic model of facial motion during speech.
Learning 3D Face Deformation Model
In this section, we introduce the methods for a learning 3D face deformation
model in our framework. 3D face deformation model describes the spatial and
temporal deformation of 3D facial surface. Efficient and effective facial motion
analysis and synthesis requires a compact, yet powerful, model to capture real
facial motion characteristics. For this purpose, analysis of real facial motion data
is needed because of the high complexity of human facial motion.
In this section, we first introduce the motion capture database we used. Then,
we present our methods for learning holistic and parts-based spatial facial
deformation models, respectively. Next, we describe how we adapt the learned
models to arbitrary face mesh. Finally, we describe the temporal facial deforma-
tion modeling. The face models used for MU-based animation are generated by