Analysis and Synthesis of Facial Expressions - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 245

Hierarchical Model-Based Facial
Expression Analysis

The most challenging part of facial expression analysis is the estimation of 3-D
facial motion and deformation from two-dimensional images. Due to the loss of
one dimension caused by the projection of the real world onto the image plane,
this task can only be solved by exploiting additional knowledge of the objects in
the scene. In particular, the way the objects move can often be restricted to a
low number of degrees of freedom that can be described by a limited set of
parameters. In this section, an example of a new 3-D model-based method for
the estimation of facial expressions is presented that makes use of an explicit
parameterized 3-D human head model describing shape, color, and motion
constraints of an individual person (Eisert, 2000). This model information is
jointly exploited with spatial and temporal intensity gradients of the images. Thus,
the entire area of the image showing the object of interest is used, instead of
dealing with discrete feature points, resulting in a robust and highly accurate
system. A linear and computationally efficient algorithm is derived for different
scenarios. The scheme is embedded in a hierarchical analysis-synthesis frame-
work to avoid error accumulation in the long-term estimation.

Optical-Flow Based Analysis

In contrast to feature-based methods, gradient-based algorithms utilize the
optical flow constraint equation:

( ) ( ) ∂ I X , Y ∂ I X , Y

( ) ( )

′

d + d = I X , Y − I X , Y ,

(1) x y

∂ X ∂ Y

∂ I

and I

∂

where are the spatial derivatives of the image intensity at pixel

∂ X ∂

position [X, Y]. I ′ -I denotes the temporal change of the intensity between two

-t corresponding to two successive frames in an image
sequence. This equation, obtained by Taylor series expansion up to first order of
the image intensity, can be set up anywhere in the image. It relates the unknown
2-D motion displacement d =[d x , d y ] with the spatial and temporal derivatives of
the images.
The solution of this problem is under-determined since each equation has two
new unknowns for the displacement coordinates. For the determination of the

time instants ∆ t=t ′