Introduction - Semantic Acquisition Games: Harnessing Manpower for Creating Semantics - page 4

There is still a lack of sufficient semantics for domain models, especially in
specialized domains (as opposite to the well establishing general domain models of
linked data). The ever increasing number of multimedia resources (images, music)
is not covered with sufficient descriptive metadata creation (in both quantity and
quality). In connection to the above, the SAG approaches have trouble to solve
more specific human intelligence tasks for which only small groups of sufficiently
experienced players are usually available. Based on these challenges, we formulated
our first goal:
Goal 1: Add to semantics acquisition with new effective and functioning, SAG-based
approaches and, if possible, for specific domains, where the lack of the semantics is more
severe and where only limited number of players is available.
Secondly, our work orients itself on the state-of-the-art of the design of the semantics
acquisition games and names its main open challenges.
The SAG design and development is a non-trivial task and there is only a little of
existing guidance on how to create these games. The SAGs are created ad-hoc and
have to deal with cold-start problems (or they fail to provide feedback to the players
according to the quality of artifacts they are producing), popularity (the games look
more or less like a work) and player cheating problems (which hamper not only the
fairness of the game but also damages their “useful” output value). Thus, a major
challenge for researchers is to come up with a complex methodology for SAG design.
Validation schemes for player-created artifacts (such as synchronous consensus of
the crowd), which SAGs use for ensuring the quality of their output, are not sufficient
in acquisition of correct solutions for human intelligence tasks that require certain
degree of expertise of the workers. Even if there is a minority of experts in the crowd,
their voice is “overrun” by the lay majority. The research challenge is therefore to
identify experts and authorities within the crowd of players, and assign them with
more voting power. Based on these challenges, we formulate our second goal:
Goal 2: Improve the effectiveness of semantics acquisition games by developing design
principles, independent on the problem domain, which the SAG is dealing with. In particular,
we focus on the possibilities of
1. reducing the cold start problems of SAGs,
2. preventing malicious player behavior and
3. taking advantage of players with more expertise and confidence for solving the game's
purpose.
1.2 Topic Outline
This topic is split into two major parts to cover our two goals. The part one covers our
workwith semantics acquisition games from the perspective of semantics acquisition.
It focuses on the state-of-the-art in semantics acquisition in general (expert-based,
automated and crowd-based approaches) and for semantics acquisition games. After,