Game Development Reference
In-Depth Information
If we drive the selection with real co-occurrence (i.e. we offer the player terms
highly co-occurring with the task term), we can mitigate the effect of frustration,
because player would feel more rewarded. On the other hand if the player identi-
fies only the high co-occurrence relationships, the game would not explore hidden
relationships and would merely clear-up some noise (it can be expected that players
wont pick the high co-occurrence relationships with low or no relatedness).
To find a balance between these extremes, our initial proposal was to compose the
candidate term set according to real co-occurrence with three, same-sized groups:
1. High co-occurrence group. This group contains terms with co-occurrences (to the
task term) above the noise level. We expected that these options would be picked
the most times and would also ensure that players will not get frustrated.
2. Medium co-occurrence group. This group represents the term couples with mod-
erate co-occurrence percentages but from the interval already spoiled by the noise.
This group was the primary potential source of “hidden” relationships.
3. Low co-occurrence group, which composed from term couple with marginal co-
occurrences.
The exact co-occurrence percentage thresholds between these three groups depend on
the characteristics of the given problemdomain. If, for instance, the domain document
corpus contains many documents and relatively few terms we want to track, then the
noise levels would probably be higher, than with domains with opposite parameters
(especially regarding number of documents). It is therefore necessary, prior to the
deployment to a specific domain, to set these thresholds carefully.
In our case (the domain of software engineering course) we have hundreds of terms
we wanted to track and hundreds of documents. This resulted in the situation that
most of the term couples had no co-occurrence at all and the non-zero co-occurrences
were counted in units. Therefore, we decided to work with only two groups of terms,
those which had and those which had not any co-occurrence with the task term.
4.4.2 TermBlaster Validation
In a preliminary experiment executed with limited number of players (38) in a period
of two days, we aimed to evaluate the setup of the TermBlaster, mainly for its capa-
bility of acquiring valid term relationships. We have recorded 732 rounds played over
6 task terms (distributed equally). For each term approximately 400 terms were dis-
played (multiple times) as candidates and averagely 15 relationships were acquired
for each (a threshold of 5 needed votes to pass was used).
The acquired term relationships were evaluated for overall soundness and whether
they were of “hidden” type (i.e. they were from the “no co-occurrence” group). The
soundness was firstly evaluated against existing data set—a lightweight model of
the domain created for learning framework used in the software engineering course.
This data set contained all terms with which the game worked (600 terms) and the
relationships between them, counted 700. This however, represented only the most
Search WWH ::
Custom Search