Game Development Reference
In-Depth Information
1
0.9
0.8
0.7
0.6
0.5
0.4
0.5 0.6 0.7 0.8 0.9
1
1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9
2
Consensus threshold
W e ig h tin g w ith u s e fu ln e s s
W eighting with consensus
Fig. 8.2 The precision of the image tags produced by PexAce for different consensus thresholds:
(1) when usefulness is considered in suggestion weighting, (2) when consensus is considered in
suggestion weighting and (3) a baseline when no weighting is applied. We can see that using both
measures overdo the baseline, however, the consensus ratio is weaker, catching up with only higher
thresholds
much slower (i.e. requires higher strictness). Nevertheless, both approaches achieved
about 10% increase of precision, which we consider very promising (see Fig. 8.2 ).
We consider these experiments a first step toward potentially heavy employment
of player expertise information in SAGs. The principles we used are general: they
can easily by adapted to other SAGs where mutual player artifact validation (offline
too) is used.
We have shown a significant improvement in PexAce's output quality, if player
expertise is considered, even if we use the consensus ratio, which can be measured
continuously throughout the game deployment (except the very early stages, when
not much logs exist). Also, the SAG can possibly use some gold standard dataset
tasks to test players skills, measure their usefulness directly and project it also onto
task instances on which no golden datasets exist.
Also, any external information on the player may be used, if it has an arguable
correlation with player's in-game skills. For example, if there are user interests user
models available, a multimedia resource description game or domain modeling game
can use it to decide what particular tasks would be useful for what player. By this,
we come to another dimension of the player expertise information: differentiating
between task types or task domains. For example, within a image annotation SAG
a player may prove skilled when describing cars and miserable when describing ani-
mals. We therefore see a large, yet “uncharted” research area comprising issues like:
￿
How to categorize task instances for game (how to create the categories and how
to fill them)?
￿
How to assess the skills of players regarding these categories?
￿
How to effectively match the players to the tasks, particularly when there is not
much manpower in the game?
 
Search WWH ::




Custom Search