Gesture recognition in cooking video based on image features and motion features using Bayesian network classifier - Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition - page 313

CHAPTER 24

Gesture recognition in
cooking video based on image
features and motion features
using Bayesian network
classiier

Nguyen Tuan Hung 1 ; Pham The Bao 1 ; Jin Young Kim 2

1 Faculty of Mathematics and Computer Science, Ho Chi Minh University of

Science, Ho Chi Minh City, Viet Nam
2 School of Electrical and Computer Engineering, Chonnam National University, Gwangju, South Korea

Abstract
In this chapter, we propose an advanced method, which combines image features and motion features,
for gesture recognition in cooking video. First of all, the image features including global and local fea-
tures of Red-Green-Blue color images are extracted, then represented using bag of features method. Con-
currently, motion features are also extracted from these videos and represented through some dense tra-
jectories descriptors such as histogram of oriented gradient, histogram of optical flow, or motion bound-
ary histogram. In addition, we use relative positions between objects and also their positions are detected
in each frame to decrease misclassification. Next, we combine both image features and motion features
to describe the cooking gestures. At the last step, Bayesian network models are applied to predict which
action class a certain frame belongs to, base on the action class of previous frames and the cooking ges-
ture in current frame. Our method has been approved through Actions for Cooking Eggs dataset that it
can recognize human cooking actions as we expected. We evaluate our method as a general method for
solving many different action recognition problems. In near future, therefore, we are going to apply it to
solve other action recognition problems.

Keywords

Action recognition
Bayesian network
Features combination
Image features