Video Scene Analysis: A Machine Learning Perspective - Video Segmentation and Its Applications - page 103

Fig. 4.9 The framework of our approach. In our approach, scenes with similar contents are
grouped into the same cluster. For each cluster, a ranking function is optimized to give ranks for all
subsets in a scene, while these estimated ranks are expected to approximate the ground-truth ranks
a multi-task learning algorithm to infer multiple saliency models simultaneously.
Different from the traditional single-task learning approach, the multi-task learning
approach can carry out multiple training tasks simultaneously with fewer training
data per task. In this framework, the appropriate sharing of information across train-
ing tasks can be used to effectively improve the performance of each model.
The system framework of this approach is illustrated in Fig. 4.9 . In this frame-
work, the training scenes are grouped into M clusters and a ranking function is
optimized for each cluster. Here the ranking function is optimized as in the pre-
vious approach (i.e., the same pair-wise losses, the same optimization strategies).
However, several penalty terms are added into the optimization process to improve
the performance of each of the M ranking functions, especially for the general-
ization ability. These penalty terms mainly consist of scene clustering penalty (to
group scenes with similar contents into the same cluster), model diversity penalty
(to improve the generalization ability of each ranking function), and model complex-
ity penalty (to avoid over-complex model). By introducing these penalty terms, the
training process can optimize M ranking functions simultaneously with an appropri-
ate sharing of information across them. Therefore, the performance of this approach
is much better than the other approaches (e.g., [ 13 , 15 , 16 , 19 , 20 , 24 , 30 , 44 , 46 , 68 ])
and can reach a ROC score of 0.811 on the video eye-fixation dataset [ 22 ]. Some
representative examples are illustrated in Fig. 4.10 .FromFig. 4.10 , we can see that
our approach can adapt to various scenes and demonstrates a higher accuracy in
locating the most salient targets.