Event Understanding of Human-Object Interaction: Object Movement Detection via Stable Changes - Intelligent Video Event Analysis and Understanding - page 198

this stage, (1) the method extracts changed pixels by a background subtraction tech-
nique and categorizes them into “something inserted” state, called the foreground
state, and “something removed” state, called removed-layer state. (2)The method
then employs the blob detection algorithm to the pixels and extracts changed re-
gions. After extracting changed regions, (3) the method tracks the extracted regions.
Second, the method categorizes the extracted regions into non-objects and objects
via their motion, and finally detects object placement and object removal. In this
stage, (4) the method discriminates between the non-object state and the object state
via the regions' motion detection result for past some frames (we call this motion
detection results as “motion history”). (5) The method rejects the stable changes
caused without object movements, then detects object movements. Finally, (6) the
system updates its background model according to the object detection result.
The proposed method detects object movements from the stable changes ex-
tracted by a background subtraction method. Common background subtraction
methods have only one background model, so background subtraction methods
with only one model have only “changed” state , called foreground state, and “not
changed” state, called background state. In this research, we need to classify “ob-
ject placement” and “object removal” from “changed” state, so we adopt a multiple-
layered background model [6, 7] (we call the multiple-layered background model
as “layered background model”). Moreover, the proposed method adopts an object
placement / removal classification method based on edge subtraction, to handle “ob-
ject removal which exists in the initial state”, which the method with only layered
background model cannot handle properly.
The proposed method employs motion history of extracted regions to discrim-
inate between object regions and non-object regions. In the household environ-
ment, objects or non-objects are sometimes largely occluded by furniture, so track-
ing methods must be robust for sudden occlusion. The proposed method adopts
keypoint-based tracking method for this problem. In addition, to detect motion ro-
bustly even when people move only parts of their body (e.g. when a human is sitting
on a sofa and read a book, the whole body of the human hardly move but only their
arms moves), we adopt the partial frame subtraction technique.

3 Attentive Region Detection

This section describes the method to extract changed regions from input images and
to identify these regions by comparing them to stored regions.
First, we discuss detection method of image changes via a region-level back-
ground subtraction technique, then describe how the proposed method categorizes
these changes into object placement state and object removal state. Finally, we dis-
cuss our tracking method.