Analysis and Synthesis of Facial Expressions - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 242

High-level facial animation parameters can finally be derived from the estimated
set of 2-D motion parameters. Even higher robustness can be expected by
directly estimating the facial animation parameters using more sophisticated
motion models. In Choi et al. (1994), a system is described that utilizes an explicit
3-D head model. This head model directly relates changes of facial animation
parameters to surface deformations. Orthographic projection of the motion
constraints and combination with optical flow information result in a linear
estimator for the unknown parameters. The accuracy problem of separate global
and local motion estimation is here relaxed by an iterative framework that
alternately estimates the parameters for global and local motion.
The joint estimation of global head motion together with facial expressions is
rarely addressed in the literature. In Li et al. (1993; 1994), a system for the
combined estimation of global and local motion is presented that stimulated the
approaches presented in the next section. A 3-D head model based on the
Candide (Rydfalk, 1978) model is used for image synthesis and provides explicit
3-D motion and deformation constraints. The affine motion model describes the
image displacements as a linear function of the six global motion parameters and
the facial action units from the FACS system, which are simultaneously
estimated in an analysis-synthesis framework. Another approach that allows a
joint motion and deformation estimation has been proposed by DeCarlo et al.
(1996, 1998). A deformable head model is employed that consists of ten separate
face components that are connected by spring-like forces incorporating anthro-
pometric constraints (DeCarlo et al., 1998b; Farkas, 1995). Thus, the head shape
can be adjusted similar to the estimation of local deformations. For the determi-
nation of motion and deformation, again a 3-D motion model is combined with the
optical flow constraint. The 3-D model also includes a dynamic, Lagrangian
description for the parameter changes similar to the work of Essa (Essa et al.,
1994; Essa et al., 1997). Since the head model lacks any color information, no
synthetic frames can be rendered which makes it impossible to use an analysis-
synthesis loop. Therefore, additional edge forces are added to avoid an error
accumulation in the estimation.
Illumination Analysis
In order to estimate the motion of objects between two images, most algorithms
make use of the brightness constancy assumption (Horn, 1986). This assump-
tion, which is an inherent part of all optical flow-based and many template-based
methods, implies that corresponding object points in two frames show the same
brightness. However, if the lighting in the scene changes, the brightness of
corresponding points might differ significantly. But, also, if the orientation of the
object surface relative to a light source changes due to object motion, brightness