Gabor-Like Image Filtering for Transient Feature Detection and Global Energy Estimation Applied to Multi-expression Classification (Computer Vision,Imaging and Computer Graphics) Part 3

TBM Belief Modeling

In order to recognize the undergoing expression, a fusion process of the states of all the visual cues (permanent and transient facial features) is performed using the Transferable Belief Model (TBM). The TBM can be considered as an extension of Bayes’s theorem in which probability measures are replaced by belief functions, and where no prior knowledge is assumed [13], [21]. The TBM considers the definition of the frame of discernment of N exclusive and exhaustive hypotheses characterizing the six basic facial expressions and neutral Ω = {happiness ( E1), surprise ( E2), disgust (E3), fear (E4), anger (e5), sadness (E6), neutral (E7)}. The TBM requires the definition of the Basic Belief Assignment (BBA) associated to each independent source of information.

Belief Modeling

The belief definition means the definition of the BBAs of each visual cue and is equivalent to the probabilities definition in the Bayesian model.

Fig. 9. (a) Model of BBAs for the characteristic distances [1]; (b) Model of BBAs for the transient features detection; (c) Model of BBAs of the Nasolabial furrow angles

Beliefs of the Permanent Facial Features. The BBA of the permanent facial features (i.e. characteristic distance states) is based on the work of [1]. The BBA m^Di of each characteristic distance state Di is defined as:

Whereis the power set,the frame of discernment,the doubt state betweenandthe belief in the propositionwithout favoring any proposition of A in case of doubt proposition. This is the main difference with the Bayesian model, which implies equiprobability of the propositions of A. The piece of evidenceassociated with each symbolic state given the value of the characteristic distance Di is defined by the model depicted in Fig. 9.a.

Beliefs of the Transient Facial Features: Presence. The BBA of the states of each transient feature TFj is defined as:

Where TFi means the nasal root wrinkles, TF2, the nasolabial furrow,

From the frame of discernmentonly the states Pj (the wrinkles are present without any doubt) and the state (there is a doubt in their detection and notedare considered. Then if the wrinkles are detected as present (the energy threshold is higher than the defined value) the corresponding state is P j if not, the corresponding state isThe piece of evidenceof each state is derived according to the model depicted in Fig. 9.b. The nasal root wrinkles are used as a refinement process and are associated to disgust and anger expressions (without favoring any of them). If they are present the current expression is disgust or anger with the piece of evidence:

If they are not present, the current expression can be one of the 7 studied expressions with the piece of evidence:

Compared to the nasal root wrinkles, if present, the nasolabial furrows are associated to happiness, disgust and anger expressions with the piece of evidence:

Beliefs of the Transient Facial Features: Orientation. The BBAs of the nasolabial furrow angle states are defined as:

Where An is the angle, mean opened and closed angles (see section 2.2), {Op,Cl} means Op or Cl and corresponds to the doubt between Op and Cl (noted Op u Cl). The pieces of evidence associated to the states of the detected angles are defined using the model proposed in Fig. 9.c. Based on the BBAs of the nasolabial furrow angle states, the piece of evidence associated to each one of the 3 expressions happiness ( E1), anger ( E3) and disgust ( E5) is computed based on the Table 1 as:

In the other cases the piece of evidence of the expression or subset of expressions is equal to the projection of the angle value on the proposed model.

Temporal Information

The dynamic and asynchronous behavior of the facial features is introduced by combining at each time t their previous deformations from the beginning until the end of each emotional segment (see section 2) to take a decision. The analysis of the facial feature states is made inside an increasing temporal window At (Fig. 10. a). The size of the window At increases progressively at each time from the detection of the beginning until the detection of the end of the current segment. At each time t inside the window At, the current state of each facial feature (i.e. characteristic distances and transient features) is selected based on the combination of their current state at time t and of the whole set of their past states since the beginning which then takes into account their dynamic and asynchronous deformations (Fig. 10.a). The dynamic fusion of the BBAs is made according to the number of appearance of each symbolic state notedand    their    integral (sum) of plausibility noted computed progressively inside the temporal window Δΐ. For instance, tor a characteristic distance Di and for the state = C- :

From the two parametersthe selected states of each visual cues at each time t inside the temporal window At are chosen as:

Fig. 10.b shows an example of the temporal selection of the states of the characteristic distance D2 during a sequence of disgust expression. One can see the correction of the false detection state ( C+) by the temporal fusion process (equation 13). The piece of evidence associated to each chosen state corresponds to its maximum piece of evidence inside the temporal window as:

Then at each time t from the beginning to the end of the expression sequence, once the BBAs of all the visual cues are refined, the corresponding expression is selected according to the rules Table 1.

Fig. 10. (a) Example of the increasing temporal window during a sequence of disgust expression; (b) Example of the selection of the characteristic distance states at each time inside the increasing temporal window

Beliefs Fusion

The fusion process of all the visual cue states is done at each time (Fig. 10.a) using the conjunctive combination rule [21] and results in m° the BBA of the corresponding expression or subset of expressions:

From Table 1 and the BBAs of the sensor states: the characteristic distance states the transient features’ statesand the angles’ statesa set of

BBAs on facial expressions is derived for each sensor as:The

fusion process of the BBAsis performed successively using the conjunctive combination rule (equation. 17). For example, for two characteristic distances Di and Dt the joint BBAusing the conjunctive combination is:

The obtained results are then combined to the BBAs of the transient features’ states as:

Where a, B, E, F, G, H, C denote propositions andthe conjunction (intersection) between the corresponding propositions. This leads to propositions with a lower number of elements and with more accurate pieces of evidence.

The decision is the ultimate step and consists in making a choice between various hypotheses E e and their possible combinations. The decision is made using the maximum of belief (which favors the mixture of hypotheses in case of doubt) as: