Analysis and Synthesis of Facial Expressions - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 236

are becoming more available with today's computers. In this chapter, the state-
of-the-art in facial animation and analysis is reviewed and new techniques for the
estimation of 3-D human motion, deformation, and facial expressions from
monocular video sequences are presented. The chapter starts with an overview
of existing methods for representing human heads and facial expressions three-
dimensionally in a computer. Algorithms for the determination of facial expres-
sions from images and image sequences are reviewed, focusing on feature-
based and optical-flow based methods. For natural video capture conditions,
scene lighting often varies over time. This illumination variability has a consid-
erable influence not only on the visual appearance of the objects in the scene, but
also on the performance of the estimation algorithms. Therefore, methods for
determining lighting changes in the scene are discussed for the purpose of robust
facial analysis under uncontrolled illumination settings. After this overview, an
example of a hierarchical, gradient-based method for the robust estimation of
MPEG-4 facial animation parameters is given, illustrating the potential of model-
based coding. This method is able to simultaneously determine both global and
local motion in the face in a linear, low-complexity framework. In order to
improve the robustness against lighting changes in the scene, a new technique for
the estimation of photometric properties based on Eigen light maps is added to
the system. The performance of the presented methods is evaluated in some
experiments given in the application section. First, the concept of model-based
coding is described, where head-and-shoulder image sequences are represented
by computer graphics models that are animated according to the facial motion
and deformation extracted from real video sequences. Experiments validate that
such sequences can be encoded at less than 1 kbit/s enabling a wide range of new
applications. Given an object-based representation of the current scene, changes
can easily be made by modifying the 3-D object models. In that context, we will
show how facial expression analysis can be used to synthesize new video
sequences of arbitrary people, who act exactly in the same way as the person
in a reference sequence, which, e.g., enables applications in facial animation for
film productions.
Review of Facial Analysis and Synthesis
Techniques
Facial Animation
Modeling the human face is a challenging task because of its familiarity. Already
early in life, we are confronted with faces and learn how to interpret them. We