Interactive Mobile Visual Search and Recommendation at Internet Scale - Multimedia Database Retrieval: Technology and Applications - page 117

4.3.3 Social Activity Recommendations

Recently, Jain and Sinha proposed to re-examine the fundamental issue between
content and context and why researchers should utilize both of them to bridge
the semantic gap [ 123 ]. From the perspective of visual content analysis, Hua and
Tian surveyed the importance of visual features to help text-based searches [ 122 ].
Although the aforementioned two studies focused on context and visual contents,
respectively, they both advocate on a multi-modality structure to achieve various
tasks. On the other hand, Guy et al. suggest that while machine learning and
human computer interactions play key roles in recommendations, personalization
and context-awareness are also crucial in establishing an efficient recommendation
system [ 121 ]. Authors agree with their arguments that it is necessary to connect
data and users. It is also believe that smart-phones provide perfect platforms for
such data-users connection, from human computer interaction, to visual search, and
finally, to the recommendation.
In the TapTell system, after the visual intent expression and identification, rich
metadata is utilized as a better feature to search. Powerful context is used to re-
rank metadata-based search result for the final task completion. To be specific,
the metadata associated with the top image search result is adopted as a textual
query. Then, social activity recommendations are obtained based on the text retrieval
results. The Okapi BM25 ranking function is used to compute a ranking score based
on text similarity [ 132 ]. Keywords Q t = {

are extracted by projecting
the text query to a quantized text dictionary. Subsequently, relevance score of query
Q t and database image descriptions D t are computed. Detailed score computation
techniques can be referred to in [ 132 ]. In the last step, search results are ranked

q t 1 , q t 2 ,..., q t n }

Fig. 4.10 Result of recommendation list, which is visualized in a map to help users to picture the
distances between the query and the results