Game Development Reference
In-Depth Information
Participants. In this study, we used two groups of participants with each having
four members. The purpose of having two groups was to observe the influence of
awareness about the game's purpose (one group was told that they are creating
metadata for their images, the other not). Each group comprised members of the
same social group, which has several common interests as well as images. Two of
the participants in each groupwere players in this study, one served as a tag evaluation
judge and one helped in preparation of the image set prior to the playing ( preparator ).
Data. Each preparator created a set of 48 images to be used in the game. Images
were drawn froma larger pool comprising three albums. Themembers of preparator's
group were familiar with the contents of the images. The preparator drawn 12 images
for each of the defined categories (portraits, groups,…).
Methodology and process. The players played the game individually, under our
supervision. The playing was followed by an interview. First, the players rules and
features of the game were explained to the players. Then, for one of the groups, the
purpose was explained. During the gameplay, the players were allowed to comment
their actions, if they considered it necessary. After the gameplay, we asked them to
comment on the game in general. All answers were recorded and analyzed post hoc.
Each player played three game rounds, with different board sizes: 6
×
6, 8
×
8
and 10
×
10 cards. Each card pair of the 48 pairs was featured twice in the game.
For the 10
10 game the remaining two pairs (48 was not enough to cover 100 card
slots) were selected randomly and were not considered in the evaluation.
After the game sessions, the tag extractionwas automatically run and the tags were
prepared to be evaluated by judges. For this, we implemented a simple application,
where images were consecutively presented with their assigned tags and judges were
asked to fill in the information about them. Prior to this, the judges were introduced
to the concept of multimedia metadata (need for their creation and ways of use).
They were not introduced to the game itself. For each assigned tag, the judges had
to answer the following:
×
￿
Is the tag describing the image correctly?
￿
In case of correct tag, is it specific for the group (i.e. do you think it is possible,
that a person not familiar with your group would be able to provide this tag)?
￿
In case of correct tag, is it understandable for people outside your group?
￿
In case of specific tag, assign one of the following type: person, place, event or
other.
Results. For both groups together, 366 tags were extracted by the filtering heuristics.
From this number, only one third (122 tags) passed according to the cross-player
validation rule. The rest passed by second (repeated use of a tag on an image by single
player, 196 tags) and third rule (repeated use of a tag for an album). This demonstrated
a substantial increase in quantity of acquired tags through less restrictive heuristics.
There was also a significant difference between the quantities of tags between the
participating groups: the purpose aware group provided annotations, from which 2.5
times more tags were extracted, than from the purpose unaware group logs.
The overall correctness of the acquired tags was 90%. Here, a difference between
the two groups was noted: 96% for the purpose aware and 86% for unaware group.
Search WWH ::




Custom Search