CityLights: A Method for Music Metadata Validation - Semantic Acquisition Games: Harnessing Manpower for Creating Semantics - page 81

Fig. 6.2 The CityLights game interface after the player chooses one answer option. Before con-
firming the choice he may once again review the tags it contains, set the height of the bet or take
his decision back
The “unfortunate” circumstances for the player in terms of drawn tags can occur
quite often if the input metadata corpus is of bad quality. From the game perspective,
it is important that the player encounters a balanced portion of “easy” or “not tricky”
questions to the “unfortunate” questions such as the one with the 5th symphony .The
player must not get bored, but also not get overwhelmed.
6.1.2 Validating Individual Tags
So far, we know that each player answer represent a relative comparison of two groups
of tags (in terms of their correct assignment). At the start of the crowdsourcing
process, each of the tags present in the input corpus is assigned with the support
value, which will later act as a sum of positive or negative feedback acquired via
player interactions with this tag. Using the example from the previous Sect. 6.1.1 ,
when player selects the wrong choice, each tag in the “correct” option has its support
decreased (because they were not convincing). On the other hand, if the player selects
the “correct” choice, all of its tags will have its support increased. We define two
thresholds (positive and negative) that define at what values, the support marks
a definitive correctness: when the support value of a certain tag reaches one of the
pre-defined thresholds, it is declared as confirmed or rejected tag. Alternatively, the
support value may also act as a property of ranking the tags assigned to a single
music resource in the input dataset.