Game Development Reference
In-Depth Information
yet a small term network was possible to construct. It contained more than 100 terms.
The players were recruited through social network.
For the sake of experimentation, to raise the number of players (and thus games
played and logs collected) we opted for implementing a tournament mode of the
game. Its main purpose was to bring in more incentives for potential players. In it,
players compete directly over the set of same task terms. The tournament mode was
put into practice at a student conference at our university, where we also awarded
material prizes for the winners. Another occasion, when the game was played was
during a showcase.
In total, we have recorded about 3,800 games together by 300 players, who sub-
mitted 27,200 queries. In total, 3,200 term connections were suggested (with 40 task
terms featured).
The filtering procedure resulted in term network containing 400 nodes and 560
edges, yet we have to admit, that the distribution of relationship to task terms was
not regular due to fact that the tournament task terms were bit “overused” by players.
When we applied an additional limitation, that all, apart the 10 strongest (according
to weight
w
) relationships per task were pruned, the resulting graph contained only
183 nodes and 220 edges.
4.3.2 Term Network Soundness
To validate the soundness of relationships in the network, we conducted an experi-
ment with a group of judges evaluating a sample set of created relationships.
Hypothesis
. Every (oriented) edge in the term network created by Little Search
Game reflects a real semantic relationship of the source term with the target term.
To do so, we conducted a survey evaluating the soundness of a subset of the created
term network.
Participants
. The survey was conducted with 18 participants of both genders aged
between 18 and 30. The participants were of various professions. We considered no
further knowledge about them.
Data
. We randomly chose 12 relationships from the Little Search Game term
network for evaluation. To create some “noise”, so that participants would not realize
they were expected to mark each relationship with a positive vote, we created 8 more
random term pairs and shuffled them into the original 12 to create a list of 20 ordered
term pairs.
Ta s k
. Participants were presented with a list of ordered term pairs, with the task:
“Do you consider the term B as being related to the term A (in other words: would
you include the term B in the top 10 most related terms for term A)? (1—Definitely
irrelevant, 2—More likely irrelevant, 3—More likely relevant, 4—Definitely rele-
vant, 5—Unsure).”
We indirectly stressed the importance of evaluating a one-way
relationship from A to B, since our term network is an oriented graph.
Process
. All participants answered all questions. We computed whether the par-
ticipants as a group rejected or admitted the relevance of term pairs based on vote
Search WWH ::
Custom Search