Digital Signal Processing Reference
In-Depth Information
extract salient features of the face whereas the lip movement is represented
by DCT coefficients of the corresponding optical flow vectors in the lip
region. Face and lip features are then stored as biometric templates and
classified through a set of algorithms so-called synergetic computer. The
acoustic information on the other hand is represented by cepstral coefficients
that are then classified by vector quantization using a minimum distance
classifier.
In our biometric speaker identification system, we use three different
modalities: speech, lip trace and face texture. Lip movement is a natural by-
product of the speaking act. Information inherent in lip movement has so far
been exploited mostly for the speech recognition problem, establishing a
one-to-one correspondence with the phonemes of speech and the visemes of
lip movement. It is quite natural to assume that lip movement would also
characterize an individual as well as what that individual is speaking. Only
few articles in the literature incorporate lip information for the speaker
identification problem [11, 16, 19]. Although these works demonstrate some
improvement over unimodal techniques, they use a decision-fusion strategy
and hence do not fully exploit the mutual dependency between lip movement
and speech. In a recent work [12], bimodal data and decision fusion of audio
and eigenlip stream has been studied with encouraging results. In this chapter
we present an HMM-based speaker identification scheme for joint use of the
face, the lip trace and the audio signal of a speaking individual through data
and multilevel decision fusion.
2.
MULTIMODAL DECISION FUSION
The speaker identification problem is often formalized by using
probabilistic approach: Given a feature vector
f
representing the sample data
of an unknown individual, compute the a posteriori probability for
each class
n
= 0,1,…,
N
, i.e. for each speaker's model. The sample feature
vector is then assigned to the class
that maximizes the a posteriori
probability:
