Modeling and Synthesis of Realistic Visual Speech in 3D - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 282

Figure 12. The personalization of visemes follows two steps symbolized by
the two horizontal transitions: 1) The linear combination of the example
visemes as described in the text, and 2) A residual adaptation, following the
cloning technique described in Noh et al. (2001).

F is still a bit different from the original novel face no F , expression
cloning is applied as a last step to the visemes found from projection (Noh et al.,
2001). We have found that the direct application of viseme cloning from an
example face to other faces yields results that are less convincing. This is
certainly the case for faces that differ substantially. According to the proposed
strategy the complete set of examples is exploited, and cloning only has to deal
with a small residue. The process of personalizing visemes is summarized in
Figure 12.

As the face

nov

Automatic, Audio-Based Animation

Once the visemes for a face have been determined, animation can be achieved
as a concatenation of visemes. The visemes, which have to be visited, the order
in which this should happen, and the time intervals in between are generated
automatically from an audio track containing speech. First a file is generated that
contains the ordered list of allophones and their timing. “Allophones” correspond
to a finer subdivision of phonemes. This transcription has not been our work and
we have used an existing tool, described in Traber (1995). The allophones are
then translated into visemes (the list of visemes is provided in Figure 2). The
vocals and silence are directly mapped to the corresponding visemes. For the