An Unsupervised Ensemble Approach for Emotional Scene Detection from Lifelog Videos - Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing - page 141

In order to solve the issue, we propose an impressive scene detection method for
lifelog video retrieval. Impressive scenes are considered to be useful since they con-
tain recordings of important events that need to be retrieved during the lifelog video
retrieval. We attempt to obtain impressive scenes by detecting emotional scenes
based on the fact that it is important for human communication to utilize nonverbal
communications including emotional expressions [4]. We focus on facial expression
recognition for the detection of emotional scenes because most of emotions can be
reflected in the facial expressions.
Facial expression recognition has been extensively studied and can be applied
to video-scene detection [5][6][7]. However, most of facial expression recognition
techniques are based on supervised learning. They generally require a large amount
of training data to construct a good facial expression recognition model. Since it
is quite difficult to automatically collect training data, preparing sufficient training
data requires considerable human effort.
In our approach, we aim to improve the efficiency of the facial expression recog-
nition by introducing an unsupervised learning framework using a clustering tech-
nique. The facial expression is recognized by classifying the facial image into one
of the clusters corresponding to a certain facial expression.
For the purpose of improving the recognition accuracy, we introduce an ensemble
learning approach called cluster ensemble [8]. The cluster ensemble can enhance
and stabilize the accuracy of facial expression recognition by combining diverse
sets of clusters into a single set of more discriminative clusters.
In order to detect emotional scenes, we introduce an efficient emotional scene de-
tection method on the basis of the result of the facial expression recognition for each
frame image in a video. Because the ensemble clustering is unsupervised, the pro-
posed method neither requires learning data nor the predefinition of facial expres-
sions. We show the effectiveness of the the proposed method through an emotional
scene detection experiment.
The remainder of this paper is organized as follows. Section 2 presents some
related works. Section 3 describes the facial features used to recognize facial ex-
pressions. Section 4 elaborates the facial expression recognition method using the
cluster ensemble. Section 5 explains the emotional scene detection method. Sec-
tion 6 shows the emotional scene detection experiment using several lifelog videos.
Finally, Section 7 concludes this study.

2 Related Works

Facial expression recognition plays the most important role in our emotional scene
retrieval. In order to precisely and efficiently recognize facial expressions, various
kinds of facial expression recognition techniques have been proposed. The perfor-
mance of the facial expression recognition can be greatly influenced by the facial
features. Currently, there are two major types of facial features, appearance features
and geometric features [9].