Event Understanding of Human-Object Interaction: Object Movement Detection via Stable Changes - Intelligent Video Event Analysis and Understanding - page 202

Foreground
Region List

...

Keypoint-based Matching

Extracted
Region

Image

(b)

(a)

Fig. 4 Keypoint based tracking. (a): Overview of keypoint based tracking. Extracted regions
are matched with the list of stored foreground regions by using their keypoint patch templates.
(b): Tracking result of human using keypoint based tracking. The circles represent extracted
patch template area

First, the method extracts the FAST keypoints in the inner foreground regions. To
detect keypoint patches in sufficient scales, the method generates a five-level image
pyramid, and then extracts keypoints in each level. Level zero of the image pyramid
stores the full resolution image, and images in the image pyramid are sub-sampled
in each level. The method extracts keypoints from grayscale-converted images on
each image level and generates patch templates of 5 5 pixel for these keypoints

from RGB images on the extracted level. To be robust under illumination changes,
each RGB pixel is converted to a color invariants [16, 17].
The patch templates are compared to the patch templates of the foreground re-
gions extracted in previous frames, which are stored in the system's foreground
region list. To reduce the computational cost, only patches in the same pyramid
level are compared. This comparison is done by evaluating the sum-of-squared-
differences (SSD) score at all keypoints within the search regions and selecting the
keypoint with smallest difference score. If the best matching score is under the fixed
threshold (0.05 in our implementation) and if the ratio of the best matching score
to the second-best matching score is less than the fixed threshold (0.8 in our imple-
mentation), the patch is considered to be matched.
Up to n matched points (10 in our implementation) are searched in the extracted
foreground region, and the extracted region is identified as the region with the largest
number of matched points in the foreground region list. To avoid tracking failure
caused by wrong keypoint matching, if the ratio of each region size is over a fixed
threshold (10 in our implementation), the regions are regarded as not matched. Ad-
ditionally, to track very low textured changed regions (e.g. when objects in the initial
state are removed, the changed regions are usually the part of low textured furniture),
if the distance of the center of each region is small (2 pixel in our implementation)
and the ratio of each region size is under the threshold mentioned above, the regions
are matched.
If an extracted region does not match any regions, then it is added to the fore-
ground region list. In some cases, several extracted regions match to one stored

×