State-of-the-Art: Semantics Acquisition Games - Semantic Acquisition Games: Harnessing Manpower for Creating Semantics - page 36

probable reason for this is that metadata creation comprises mostly working with
text and textual games are easy to create: most of the existing SAGs do not comprise
elaborate graphics or 3D modeling as is common in most of today's games.
3.1 Image Description Games
First, we look at SAGs for multimedia metadata acquisition, in particular at image
description SAGs. Image description is a task that humans can do much more effec-
tively than machines, even today, when much work has been done in the field of
automated image classification [ 10 ]. Generally, the image description is done by two
steps: (1) symbolic interpretation of the (sub-symbolic) image bitmap (this is easily
done by human brain with almost no effort) and (2) transcription of the symbols to
the textual form. The latter is the step, which humans are not normally motivated to
do, so SAGs focus on that part.
The already mentioned ESP Game [ 24 ] produces textual annotations in form of
tags to images given as game input. This online game is played by two players
(coupled randomly from the large pool of players) at the same time. The players are
anonymous to each other, they cannot communicate. Both of them are shown the
same image (as the one seen in the Fig. 3.2 ). Their task is to type in some words,
describing the given image. When they have both typed the same word, they “win”
the round.
The deployment of the ESP resulted in a massive collection of new metadata to
images (partially because the game was adopted by Google company, which adver-
tised it and used its results). The collected metadata also had a surprisingly good
quality, which, in the end, had a significant impact on proliferation of the whole
SAGs research field.
Motivated by several dishonest player behavior issues of the ESP game, a modi-
fication of its principle was made by Ho et al. in the game named KissKissBan [ 8 ]. It
introduces a third player, blocker , as opponent to the first two players (couple). His
task is to write tags relevant to the image earlier than the remaining players prevent-
ing them to use them for reaching agreement. According to authors, this modification
brought two benefits: (1) third player effectively supervised any attempts of cheat-
ing by partner couple, (2) more specific and richer set of tags was used to describe
images (since the obvious tags have been quickly banned). The authors evaluated
the game in experiments (with 500 games) showing the increase in diversity of the
acquired labels.
The SAGs (described above) focus on retrieval of proper tags describing the
images. Luis von Ahn, however, went further toward identification of exact bitmap
areas which objects occupy in the image. He designed the game Peekaboom [ 24 ],
which uses the already tagged images from ESP Game and outputs contours of
objects mentioned in the description tags.
Peekaboom is played by two players with two roles: peek and boom which are
again two anonymous players, capable of communication only through the game