Optical flow-based representation for video action detection - Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition - page 287

mensionality problem. On the other hand, representing a single summary will cause the prob-
lem of lacking the flow of temporal information. In these cases, the focus of approaches will
be finding supportive information from different sources and integration of these sources in a
singular representation.
We aimed to solve the temporal information representation problem in video domain. As
the video information is a perfect example of high-frequency temporal information, represent-
ation of video information is essential for the purposes based on video information retrieval.
Video action recognition is selected as our specific domain. The problem domain is reduced
to the temporal video segment classification. The study is shaped on visual features of the
video information for the automaticity concerns. As it is mentioned below, the representa-
tion level determines the reduced problem. In this context, our aim is to represent the video
scenes avoiding the lack of temporal information flow while without causing the curse of di-
mensionality problem. Therefore, using more descriptive and high-level visual features hav-
ing the ability to host the additional temporal nature of the simpler features such as color,
edge, corner, etc. becomes unavoidable. This will pass the high load of temporal information
residing in high-dimensional representation to the mentioned high-level features.
The discussion summarized above took us to the complex visual features having temporal
dimension. In our research, we observed that space-time related 3D features obtained from
combining 2D features with temporal information [ 16 , 18 ] are proved to be successful. Space-
time interest points and space-time shapes for actions are proposed in these studies. We also
observed high-level state-of-the-art features such as optical flow describing the motion of
frame features are calculated and used in temporal video information representation as in
Refs. [ 13 , 14 , 21 ] . Curse of dimensionality problem occurs as all frames are represented using
optical flow vectors [ 14 ] . The problem is solved by using time-series analysis and metrics.
An optical flow-based approach is proposed in this paper for representing temporal video
information by inspiring from the above studies. This generic approach is applied in both tem-
poral video segment classification and temporal video segmentation. The adaptation of the
model to video segment classification is presented. The weighted frame velocity concept is
proposed to strengthen the representation with the velocity of video frames. This represent-
ation formalism is tested with SVM-based classification of video segments. The results show
that the proposed method produces encouraging results.
The main advantage of the method is the multipurpose temporal video representation mod-
el proposed for video action recognition domain. The new formalism described here is espe-
cially important for simplifying the computational complexity for high-dimensional informa-
tion.
References
[1] Burton A, Radford J. Thinking in perspective: critical essays in the study of thought pro-
cesses. London, UK: Routledge; 1978.
[2] Warren DH, Strelow ER. In: Electronic spatial sensing for the blind: contributions from
perception, rehabilitation, and computer vision. Dordrecht, Netherlands: Martinus Nijhof
Publishers; 1985: Nato Science Series.
[3] Ibson JJ. The perception of the visual world. Boston, MA, USA: Houghton Mifflin; 1950.
[4] Royden CS, Moore KD. Use of speed cues in the detection of moving objects by mov-
ing observers. Vis Res. 2012;59:17-24.
[5] Vasileios TC, Aristidis CL, Nikolaos PG. Scene detection in videos using shot cluster-
ing and sequence alignment. IEEE Trans Multimed. 2009;11(1):89-100.