The Manual Revision of the TERENCE Italian Smart Games - 2nd International Workshop on Evidence-Based Technology Enhanced Learning - page 30

Ta b l e 1 Example pre/post revision for GAM 75702, Story 1, Book 2

Solution Pre-revision Post-revision

AFTER to thank The inhabitants of the land of

“pı” thank Jasmine

BEFORE Louis leads the electrician the

wires

No change

WRONG All manage to split the fairly

rubbish without difficult calcu-
lations needs

All manage to divide garbage in
the right way

Each operator had the task of filling in a diary (in a spreadsheet format, see Tab.
2), made up of 33 fields, all changes made in every revised game and depending on
questions and solution types. This diary allowed the monitoring of all activities and
their analysis as reported below.

Ta b l e 2 Revision games log

Items Choices

Operator name
Book number
Story number
Game ID
Kind of game Who, what, before/after,

before/while, while/after,
before/while/after

Did you have to change the main question of the
game?

Ye s / n o

If yes, write the new main question text
Did you have to change the fixed event? Yes/no

If yes, write the new fixed event text
If yes, write the old fixed event text
Did you have to change the automatic solution?

Ye s / n o

Kind of solution Wrong, correct, before, af-

ter,
while, cause, effect

Did you have to change the event associated with
the automatic solution?

Ye s / n o

If yes, write the new fixed event as in the menu
If yes, write the new text as in the box solution
Notes

A total of 250 games were reviewed, with respect to 25 stories, with the highest
proportion of games of type before/after (30%). Based on data reported by operators,
the average review times were estimated. They were lower for reviewing a WHO-
game, and higher for temporal games. On average, for a set of games, i.e. one game