Optical flow-based representation for video action detection - Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition - page 279

In our approach, Lucas-Kanade algorithm is applied to the corner points detected with Shi-
Tomasi algorithm. Video frames are selected according to a frequency of 6 fps (30 fps videos
are used) from Hollywood Human Actions dataset [ 41 ] . In Figure 1 , two frequently sequential
frames obtained from the mentioned dataset are shown.
FIGURE 1 Consecutive frames for optical flow estimation.
Figure 2 shows the optical flow vectors estimated for the detected points in the former video
frame in the sequence.
FIGURE 2 Frame with optical flow vectors.
In our method, optical flow vectors are calculated for every detected point in all frequently
selected frames. The set of optical flow vectors is the temporal information source for our rep-
resentation.
The model below forms the back bone for our representation formalism. Optical flow vector
set with an operator constructs the representation.
(13)
S ( V ) is the set of optical flow vectors while Φ is the descriptor operator. Operator defines the
relation of the elements of the optical flow vector set of the frames. This relation exposes the
temporal representation of video information. The operator may change according to the com-
plexity of the model. It may vary from just counting the vectors to complex relations between
the optical flow vectors. This generic representation can easily be adapted to different prob-
lems such as segment classification or cut detection. Choice of the operator and the optical
flow representation may change drastically in different problems.
5.2 Proposed Representation
Usage of optical flow in video information representation is encountered in many studies in-
cluding [ 14 , 17 , 42 ] . These studies are the state-of-the-art techniques motivating us for an optic-
al flow-based representation. Optical flow histogram is the most common way of optical flow-