Analysis and Synthesis of Facial Expressions - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 238

Facial expression modeling
Once a 3-D head model is available, new views can be generated by rotating and
translating the 3-D object. However, for the synthesis of facial expressions, the
model can no longer be static. In general, two different classes of facial
expression modeling can be distinguished in model-based coding applications: the
clip-and-paste method and algorithms based on the deformation the 3-D sur-
faces.
For the clip-and-paste method (Aizawa et al., 1989; Welsh et al., 1990; and
Chao et al., 1994), templates of facial features like eyes and the mouth are
extracted from previous frames and mapped onto the 3-D shape model. The
model is not deformed according to the facial expression, but remains rigid and
is used only to compensate for the global motion given by head rotation and
translation. All local variations in the face must, therefore, be described by
texture changes of the model. During encoding of a video sequence, a codebook
containing templates for different facial expressions is built. A new expression
can then be synthesized by combining several feature templates that are
specified by their position on the model and their template index from the
codebook. As a result, a discrete set of facial expressions can be synthesized.
However, the transmission of the template codebook to the decoder consumes
a large number of bits, which makes the scheme unsuitable for coding purposes
(Welsh et al., 1990). Beyond that, the localization of the facial features in the
frames is a difficult problem. Pasting of templates extracted at slightly inaccu-
rate positions leads to an unpleasant “jitter” in the resulting synthetic sequence.
The deformation method avoids these problems by using the same 3-D model
for all facial expressions. The texture remains basically constant and facial
expressions are generated by deforming the 3-D surface (Noh et al., 2001). In
order to avoid the transmission of all vertex positions in the triangle mesh, the
facial expressions are compactly represented using high-level expression pa-
rameters. Deformation rules associated with the 3-D head model describe how
certain areas in the face are deformed if a parameter value changes. The
superposition of many of these local deformations is then expected to lead to the
desired facial expression. Due to the advantages of the deformation method over
the clip-and-paste method (Welsh et al., 1990), it is used in most current
approaches for representing facial expressions. The algorithms proposed in this
chapter are also based on this technique and, therefore, the following review of
related work focuses on the deformation method for facial expression modeling.
One of the first systems of facial expression parameterization was proposed by
Hjortsjö (1970) and later extended by the psychologists Ekman and Friesen
(1978). Their facial action coding system (FACS) is widely used today for the
description of facial expressions in combination with 3-D head models (Aizawa