Introduction - Multimedia Database Retrieval: Technology and Applications - page 7

1.3.2.2 BoW in Retrieval and Mobile Image Search

Chapter 4 of this topic explores the merits of using BoW in mobile visual search
by effectively incorporating user interaction. Efficient and scalable indexing and
non-linear fast retrieval algorithms are adopted in handling large-scale images.
Human interaction is included in the loop. Therefore, specific user perception and
distinguishing requests are used to lead the system into achieving a customized
search result.
Based on the above idea, an interactive mobile visual search application aimed
at social activity suggestion is developed using a coined term “visual intent”,
which can be naturally expressed through a visual query incorporating human
specification. To accomplish the discovery of visual intent on the phone, Tap Tell
was developed, as an exemplary real application. This prototype takes advantage of
user interaction and rich context to enable interactive visual search and contextual
recommendation. Through the Ta p Te l l system, a mobile user can take a photo and
indicate an object-of interest within the photo via a circle gesture. Then, the system
performs search-based recognition by retrieving similar images based on both the
object-of-interest and surrounding image context. Finally, the contextually relevant
entities (i.e. local businesses) are recommended to complete social tasks.

1.3.3 Mobile Visual Search

The widespread availability of networks has enabled portable multimedia devices,
particularly mobile phones, to capture, share, and access vast amount of multimedia
content. This has led to the emergence of technologies providing improved mul-
timedia data management in mobile environments. Among others, mobile visual
search technology has been at the center of mobile applications. Mobile phones are
equipped with camera and imaging functionality, which enable a visual query that
can be naturally expressed in a visual form instead of by text or voice. The user
can capture the objects/scenes that he or she is interested in, and obtain relevant
information about the captured objects/scenes [ 9 ].
In an advanced system for mobile visual search, instead of sending the whole
query image over the network, a compact signature is used, achieving a low-bit-
rate for the search. Figure 1.2 shows the vocabulary coding process of obtaining a
compact signature for mobile visual search. The utilization of a compact signature
overcomes the significant limitation of the battery power of mobile terminals, and
achieves better uplink bandwidth at the servers and latency network access. To date,
BoW models with scalable vocabulary tree (SVT) form the basis for research into
the development of compact signatures [ 10 ]. However, the BoW model is limited by
its homogenous process in treating all paths/regions without distinction. Features
are extracted homogeneously, and local features are treated without emphasis.
Therefore, a query image with unprioritized information can mislead a computer