Dynamic Facial Expression Recognition Using Boosted Component-Based Spatiotemporal Features and Multi-classifier Fusion - Advanced Concepts for Intelligent Vision Systems - page 311

Dynamic Facial Expression Recognition Using Boosted
Component-Based Spatiotemporal Features and
Multi-classifier Fusion

Xiaohua Huang 1 , 2 , Guoying Zhao 1 , Matti Pietikainen 1 , and Wenming Zheng 2

1 Machine Vision Group, Department of Electrical and Information Engineering,
University of Oulu, Finland
2 Research Center for Learning Science, Southeast University, China
{ huang.xiaohua,gyzhao,mkp } @ee.oulu.fi ,
wenming [e-mail]
http://www.ee.oulu.fi/mvg

Abstract. Feature extraction and representation are critical in facial expression
recognition. The facial features can be extracted from either static images or dy-
namic image sequences. However, static images may not provide as much dis-
criminative information as dynamic image sequences. On the other hand, from
the feature extraction point of view, geometric features are often sensitive to the
shape and resolution variations, whereas appearance based features may contain
redundant information. In this paper, we propose a component-based facial ex-
pression recognition method by utilizing the spatiotemporal features extracted
from dynamic image sequences, where the spatiotemporal features are extracted
from facial areas centered at 38 detected fiducial interest points. Considering
that not all features are important to the facial expression recognition, we use
the AdaBoost algorithm to select the most discriminative features for expres-
sion recognition. Moreover, based on median rule, mean rule, and product rule
of the classifier fusion strategy, we also present a framework for multi-classifier
fusion to improve the expression classification accuracy. Experimental studies
conducted on the Cohn-Kanade database show that our approach that combines
both boosted component-based spatiotemporal features and multi-classifier fu-
sion strategy provides a better performance for expression recognition compared
with earlier approaches.

Keywords: Component, facial interest point, feature selection, multi-classifier
fusion, spatiotemporal features.

1 Introduction

A goal of facial expression recognition is to determine the emotional state, e.g. happi-
ness, sadness, surprise, neutral, anger, fear, and disgust, of human beings based on the
facial images, regardless of the identity of the face. To date, most of facial expression
recognition are based on static images or dynamic image sequences [1,2,3], where dy-
namic image sequences based approaches provide more accurate and robust recognition
of facial expressions than the static image based approaches.