Event Understanding of Human-Object Interaction: Object Movement Detection via Stable Changes - Intelligent Video Event Analysis and Understanding - page 201

(1) Extract changed pixels (2) Classify changed pixels

Changed

Foreground State

Input Image Input Image

Changed Pixels Removed-Layer State

(Object removal)

Not changed

Layered Background Base Background

it records placed objects it records initial state

before object detection starts

Fig. 3 Layered background model. Each pixel on the input image is compared to the layered
background and the base background. First, the input image is compared to the layered back-
ground, and changed pixels on the input images are extracted. Second, the changed pixels
are compared to the base background, and then these changed pixels are classified into the
foreground state or the removed-layer state

layered background and the base background, the pixel is classified as the fore-
ground state.
Classification of object placement and object removal via the layered background
model can detect easily which object in the detected objects is removed. But, if
objects which exists in the initial state are removed, the regions of the removed
objects change from both of the layered background and the base background, so
the pixels of the removed object are classified as the foreground state. Therefore,
only with the layered background model, removal of objects in the initial state can't
be handled. To solve this problem, we employ edge subtraction technique. Details
of the classification by edge subtraction technique is discussed in Section 4.2.
After pixel-level state detection, the blob detection algorithm is employed for
foreground state pixels and removed-layer state pixels. Blobs of foreground state
pixels whose size is more than R th pixels are extracted as the foreground region, and
blobs of removed-layer state pixels whose size is more than R th pixels are extracted
as the removed-layer region.
Foreground regions extracted in this operation are then tracked. The size of the
removed-layer region is checked in the background update process. If the ratio of
the size of the removed-layer region to the size of its original layered background
is greater than a fixed threshold, object removal is detected. In our implementation,
the threshold ratio is set to 0.80.

3.3 Region Tracking

After the method extracts changed regions, then the method tracks the extracted
regions. To track robustly under occlusion, we apply keypoint-based tracking ap-
proach via FAST-10 corner detector [15]. Fig. 4 shows an overview of keypoint
based tracking and the detection result.