Graphics Reference
In-Depth Information
(about 35,000), but the number of vertices used to represent the face region is considerably
lower as a result of the large outliers acquired in the hair and shoulder regions (see Figure
5.18
b
). The lack of facial geometric details makes the 3D sequences quite challenging to be
used for facial expression recognition and face recognition.
5.5.2 3D Shape Motion Analysis
The raw data obtained from even the most accurate 3D scanners is far from being perfect
and clean because it contains spikes, holes, and significant noise. A preprocessing step must
be applied to remove these anomalies before any further operations can be performed. Thus
the preprocessing is an important stage of the recognition systems, especially when knowing
that all the features will be extracted from the output of this step. An automatic preprocessing
pipeline is developed and applied according to the following steps:
1. Filling holes: Several BU-4DFE scans are affected with holes that often lie in the mouth
region and that take part in an acquisition session that meets an open mouth expression. In
this case the mouth area is not visible and cannot be acquired by the stereo sensors, which
causes missing data. A linear interpolation technique is used to fill the missing regions of
a given raw 3D face image.
2. Nose detection: The nose tip is a keypoint that is needed for both preprocessing and facial
surface representation stages. Knowing that in most 3D images, the nose is the closest
part of the face to the 3D acquisition system, in this step the nose tip is detected using
horizontal and vertical slicing of the facial scan in order to search for the maximum value
of the
z
-coordinate along these curved profiles. Once this is done for the first frame, for the
remaining frames of the sequence this detection technique is refined and the search area
in a current frame is reduced to a small sphere centered on the nose tip detected in the
previous frame.
3. Cropping: Face boundaries, hair, and shoulders are irrelevant parts for the study, and they
are usually affected with outliers and spikes. The nose tip detected in the previous step is
used to crop out the required facial area from the raw face image. Using a sphere, centered
on the nose tip and of a radius determined empirically, the 3D range model is cut and the
mesh structure kept inside the sphere is retained.
4. Pose correction: In this step a global registration technique (i.e., ICP) is applied to align
meshes of the current frame and the first frame of the sequence. After this rigid alignment,
the pose of the 3D face is adjusted and made similar enough to the pose in the first frame.
Geometric Facial Deformation
One basic idea to capture facial deformations across 3D video sequences is to track meshes
vertices densely along successive 3D frames. Because the meshes resolutions vary across
3D video frames, establishing a dense matching on consecutive frames is necessary. For this
purpose, Sun and Yin (2008) proposed to adapt a generic model (a tracking model) to each 3D
frame using a set of 83 predefined facial landmarks to control the adaptation based on radial
basis functions. The main limitation of this approach was that 83 landmarks were manually
annotated in the first frame of each sequence. Moreover, the adaptation decreased the accuracy
