Information Technology Reference
In-Depth Information
Chapter 2
User-Perceptive Multimedia Content Analysis
Abstract
Typical social multimedia services allow users as uploaders, viewers,
taggers, and commenters to interact and collaborate with each other in a commu-
nication dialog. The wisdom of crowds provides a huge resource for understanding
social multimedia content. In this chapter, we explicitly model user interaction in
the tag generation process and propose a regularized tensor factorization solution
to refine the ternary correlations among user, image, and tag. While the traditional
social tag analysis work focus on analyzing the image-tag binary correlation, taking
user factor into consideration shows superior performance in image tag refinement
task.
2.1 Introduction
Multimedia content analysis is the first step for most traditional multimedia comput-
ing tasks. In social multimedia computing, current social multimedia platforms allow
users interacting with multimedia through uploading, annotating, commenting, and
interacting with each other through social dialogs. These interactions capture what
user perceive themultimedia content, and can be exploited towardmultimedia content
analysis, e.g., user-contributed picture tags indicate user-perceived visual semantics,
user browsing behaviors, such as pause, fast-forward, indicate video structure infor-
mation. In this section, we will review existing work in user-perceptive multimedia
content analysis based on the exploited interactions.
The idea of exploiting the crowdwisdomfromuser interaction formultimedia con-
tent analysis has been realized into several popular systems. The best example goes
to the ESP game [
38
], which is designed to make collaboratively people label images
as a side-effect of playing a game. The derived image labels can be used as training
samples for image annotation and to help improve image search on the Web. Another
example is Waze, which is another system based on user collaboration. Acquired by
Google in 2013, Waze is a free turn-by-turn GPS application for mobile phones that
uses crowdsourcing to provide routing and real-time traffic updates. Other success-
ful systems include Wikipedia, the world's largest free encyclopedia that is written
collaboratively by anonymous Internet contributors, and Facebook Translations, by
