An Appearance-Based Prior for Hand Tracking - Advanced Concepts for Intelligent Vision Systems - page 293

Fig. 2. The color segmentation is useless whereas the appearance probability correctly picks out
the hand in front of the walkway

hand, integrate two or more modalities such as texture and color information, allowing
recognition and tracking of fixed shapes despite arbitrary backgrounds [3]. The FoF
method also employs a multimodal technique to track hands even while changing shape.
Here, we add a third modality to further improve tracking.
Pyramid-based LK feature tracking [11,16] describes how to find and track small
image artifacts from frame to frame. If the feature match correlation between two con-
secutive frames is below a threshold, the feature is considered “lost.” (FoF makes use
of this feature tracker to track objects, which are larger than, and a composition of,
features.)

2.2 Flock of Features Tracking

Flock of Features tracking [8] (FoF) is motivated by the seemingly chaotic flight be-
havior of a flock of birds such as pigeons. No single bird sets the flock's direction and
the birds frequently exchange relative positions in the flock. Yet, the entire flock stays
tightly together as a large “cloud” and is able to perform quick maneouvers and direc-
tion changes. Reynolds [13] found that this decentralized organization can be modeled
with two constraints: birds like to maintain a minimum safe flying distance to the other
birds, but desire not to stray too far from the flock.
LK features, tracked over time, exhibit “flight paths” similar to a flock of birds. In-
dividual features attach to arbitrary artifacts of the object, such as the fingers of a hand.
They can then move independently along with the artifact, without disturbing most other
features and without requiring the explicit updates of model-based approaches, result-
ing in flexibility, speed, and robustness. FoF features are constrained to stay a minimum
distance apart, yet no more than a maximum distance from their median. Features in
violation are repositioned to a conforming location that also has a high skin color prob-
ability (see Fig. 2), avoiding dense clusters that ignore parts of the object, and avoiding
tracking background artifacts by falling back on a second modality. The FoF can be
seen in the various figures and the video as clouds of little dots, their mean (and hand
location) as the big dot. Note that FoF tracking-with or without our extension-makes
no attempt at estimating the articulation of the hand's digits (fingers) as model-based
approaches do (see for example [21,17]).
One of its strengths is also a weakness: FoF does not rely on an object model beyond
object color; the myriad of possible hand configurations does not have to be modeled