Optical flow-based representation for video action detection - Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition - page 277

mentioned differential equation for a set of neighboring pixels together by using a weighted
window. Nagel [ 28 ] and Uras et al. [ 29 ] use second-order derivatives generating the optical
low equations. Global smoothness concept is also used as well as the Horn-Shunck method.
Niebles et al. [ 30 ] propose a distance-based method efficient for real-time systems. The method
is analyzed according to time-space complexity and its tradeoff. Harris and Stephens Proes-
mans et al. [ 31 ] suggest a classical differential approach. But, it is combined with correlation-
based motion descriptors.
4.2.2 Region-Based Matching
Region-based matching approaches alternate the differential techniques in case diferentiation
and numerical operations is not useful due to noise or small number of frames [ 25 ] . In region-
based matching, the concepts such as velocity and similarity are defined between image re-
gions. Shi and Tomasi [ 32 ] and Ali [ 33 ] propose region-based matching methods for optical
low estimation. In Ref. [ 32 ] , the matching is based on Laplacian pyramid while [ 33 ] recom-
mends a method based on sum of squared distance computation.
4.2.3 Energy-Based Methods
Energy-based methods are based on the output energy of filters tuned by the velocity [ 25 ] .
Laptev and Lindeberg [ 34 ] propose an energy-based method iting spatiotemporal energy to
a plane in frequency space. Gabor filtering is used in the energy calculations.
4.2.4 Phase-Based Techniques
Diferent from energy-based methods velocity is defined as filter outputs having phase beha-
vior. References [ 35 - 37 ] are the examples of phase-based techniques using spatiotemporal il-
ters.
5 Optical flow-based segment representation
In this study, an optical flow-based temporal video information representation is proposed.
Optical flow vectors are needed to be calculated for the selected sequential frames. Optical
flow estimation is important as the basic element of the model is optical flow vectors. As men-
tioned in Section 4 , detection of features and estimation of optical flow according to these fea-
tures are the main steps of optical flow estimation. The methods and approaches for both steps
are discussed below [ 38 ] .
5.1 Optical Flow Estimation
In our approach, Shi-Tomasi algorithm proposed in Ref. [ 39 ] is used for feature detection. As
it is mentioned before, Shi-Tomasi algorithm is based on Harris corner detector [ 40 ] and inds
corners as interest points. Harris matrix shown in Equation (10) is obtained from the Harris
corner detector: