Database Reference
In-Depth Information
Chapter 4
Interactive Mobile Visual Search
and Recommendation at Internet Scale
Abstract
Mobile-based visual search and recognition has been an emerging topic
for both research and engineering communities. Among various methods, visual
search has its merit in providing an alternative solution, where text/voice searches
are not applicable. Combining the Bag-of-word (BoW) model with advanced
retrieval algorithms, a mobile-based visual search and social activity recommenda-
tion system is presented at internet scale. The merit of the BoW model in large-scale
image retrieval is integrated with the flexible user interface provided by the mobile
platform. Instead of text or voice input, the system takes visual images captured
from the built-in camera and attempts to understand users' intents through inter-
actions. Subsequently, such intents are recognized through a retrieval mechanism
using the BoW model. Finally, visual results are mapped onto contextually relevant
information and entities (i.e. local business) for social task suggestions. Hence, the
system offers users the ability to search information and make decisions on-the-go.
4.1
Introduction
Mobile devices are becoming ubiquitous. People use them as personal concierge to
search information and make decisions. Therefore, understanding user intent and
subsequently provide meaningful and personalized suggestions is important. While
existing efforts have predominantly focused on understanding the intent expressed
by a textual or a voice query, this chapter presents a new and alternative perspective
which understands user intent
visually
, i.e., via visual signal captured by the built-
in camera. This kind of intent is named as “visual intent” as it can be naturally
expressed through a visual form.
The bag-of-words (BoW) model and its application in content-based retrieval
has shown promising results in desktop-based visual searches at large-scale. In
this chapter, a mobile visual search algorithm is presented, by combining the
BoW model's merit with user interaction through a mobile platform. An innovative
context-aware search-tree is described based on the BoW paradigm, which includes
both user specified region of interest (ROI) and surrounding pictorial context. There
is a mutual benefit by combining the visual search using the BoW model with mobile
devices.
Search WWH ::
Custom Search