Image Processing Reference
In-Depth Information
Fig. 2. The dominance function for the diphtong case (a) and the weights diagram (b) for the
diphthong case
Fig. 3. The same of Fig.2 for the vowel-consonant case.
4.3 The emotional talking head
Emotions can be considered as particular visemes, called emotional visemes. They must be
“mixed” with the phonetic visemes to express an emotion during the facial animation. Such a
process can be performed in two different ways. FaceGen can generate also facial modification
to express an emotion, so a phonetic viseme can be modified using FaceGen to include an
emotion. As result, different sets of modified phonetic visemes can be produced. Each of them
are different both as type and intensity of a given emotion. Such a solution is very accurate
but it requires an adequate amount of memory and time to create a large emotional/phonetic
visemes database. The second approach considers a single emotional viseme whose mesh
vertexes coordinate are blended with a viseme to produce a new keyframe. Even though such
a solution is less accurate than the previous one, it is less expensive on the computational side,
and allows to include and mix “on the fly” emotional and phonetic visemes at run-time.
4.4 Audio streaming synchronization
Prosody contains all the information about the intonation and duration to be assigned to each
phoneme in a sentence. In our talking head model, the prosody is provided by Espeak espeak
(2010), a multilanguage and multiplatform tool that is able to convert the text into a .pho
prosody file. The Talking Head is intrinsically synchronized with the audio streaming because
the facial movements are driven by the .pho file, which determines the phoneme (viseme) and
its duration. Espeak provides a variety of options to produce the prosody for the language
and speech synthesizer to use. As an example it can generate a prosody control for the couple
Italian/Mbrola, which is a speech synthesizer based on concatenation of diphones. It takes as
