Game Development Reference
In-Depth Information
important relationships between the terms and not all possible, so we expected the
precision to be very low. Nevertheless, we pursued the test, which rendered 21%
of relationships as correct (21% precision with 43% recall) and 11% of them as
“hidden”.
To assess a more realistic view of the acquired relationships we also performed
a posteriori evaluation, executed by three judges, which rendered 71%of the acquired
relationships as sound and 21% of them as “hidden”.
From these preliminary evaluations, we can conclude that the TermBlaster has
capabilities for acquiring valid term relationships. However, in comparison to the
LSG they are somewhat limited (70% versus 91% correctness). It also gains lesser
number of “hidden” relationships than the LSG.
4.5 Discussion
With the Little Search Game, we aimed to add to the domain model acquisition with
newSAG-based approach. We have deployed the game and in several experiments we
validated and examined the non-labeled and non-typed term relationships it produces.
We conclude them with these major findings:
•
The term relationships acquired by the game were semantically sound.
•
A significant portion of the acquired relationships were the valuable “hidden”
relationships that are hard to be explored by automated means.
•
The acquired relationships comprise non-taxonomic types (for which there is still
a need in domain models), and represent an interesting corpus for further enrich-
ment (labeling).
•
With limits, the Little Search Game is able to operate over specific domain
(as we demonstrated in preliminary experiments with the LSG's TermBlaster
modification).
Our efforts with TermBlaster (from the semantics acquisition point of view) can
be summarized as follows: The game is able to acquire valid term relationships
in the specific domain of software engineering education, although with limited
correctness. It also acquires some hidden relationships, despite the fact that players
are no more allowed to use any terms freely, but select them from given set instead.
In return for the limited correctness (which could perspectively be increased by
stricter consensus measures, e.g. more votes needed for a pass) and lower “hidden”
relationship gains, the game increased its dynamics and player understanding, as we
assessed through informal interviews with the players.
The limited correctness of acquired term relationships in TermBlaster (relative to
the LSG) could potentially be accounted to the changed interface. The combination
of set of choices and time stress could force players to act more hastily. At one side,
this creates a more dynamic game, on the other side, the player actions may not be
so precise.
Search WWH ::
Custom Search