Understanding Sports Video Using Players Trajectories - Intelligent Video Event Analysis and Understanding - page 127

To perform motion video interpretation using such trajectories, a general and
original hierarchical parallel semi-Markov model (HPaSMM) is here proposed. A
lower level is used to model players motions and interactions using parallel hidden
Markov models, while an upper level relying on semi-Markov chains is considered
to describe activity phases. This probabilistic framework helps handling temporal
causalities of low level features within an upper level architecture that models tem-
poral transitions between activity phases. Hence, such a modeling provides an ef-
ficient and extensible machine learning tool for sports semantic-based video event
applications such that segmentation, summarization or indexing.
Spatio-temporal features modeled in the lower layer and used to model visual
semantics ( i.e. , the observed activity phases) are estimated from trajectory data. In
order to proceed activity recognition under motion clutters, a kernel-based filter-
ing is proposed to handle noisy spatio-temporal features. The considered features
characterize on one side single players motions using velocity information, while
distances between trajectories are used to describe interactions between players.
To show the versatility and efficiency of the proposed modeling, the methodol-
ogy described in the first part of this paper is then applied to two very different
sports: handball and squash. In order to represent squash activities, three features
and two activity phases have been defined on the players trajectories, whereas hand-
ball video interpretation relies on five different trajectory-based features and eight
activity phases. In the video context however and in order to propose a system that
may be able to process a large variety of sports video content, invariance of the
activity feature representation to some appropriate transformations is considered.
Evaluations of our sports event reasoning method has been performed on real video
sequences, and satisfying activity understanding results have been reached on both
sports videos. We also have favorably compared with the hierarchical parallel hidden
Markov models method (HPaHMM). In our previous works on sports video under-
standing [10], such a method was already considered for handball. A quite similar
scheme was also considered, with a different low level feature modeling though, for
squash video processing [8] . In this chapter however, we propose a generalization
of the method and describes an overall algorithm for sports video understanding that
may easily be extended to other team and racquet sports. We also go into important
details in depth while providing updated results, comparisons and perspectives.
The structure of the chapter stands as follows. In Section 2, we will describe the
hierarchical parallel semi-Markovian framework proposed for trajectory-based sport
video understanding. Application of this framework to squash video processing is
presented in Section 3 as well as corresponding experiments and results. Section 4
further describes the use of the novel modeling to a more complex set of activities for
handball video understanding. Experimental results are also reported and discussed.
In conclusion of this work, perspectives of extension to “general” high level analysis
of videos are considered.