The 1st Release of the TERENCE Learner GUI: The User-Based Usability Evaluation - 2nd International Workshop on Evidence-Based Technology Enhanced Learning - page 4

private lodging in Avezzano; a summer school for deaf children in Ciampino,
nearby Rome;

- Hearing 12 (7-9 years old); 18 (9-11 years old);

- Deaf 1 (7-9 years old); 8 (9-11 years old).

• North of Italy: the Akademia summer school in Bolzano; the unity of audiology

and phonology of the Ca' Foncello Hospital in Treviso.

- Hearing 3 (7-9 years old); 3 (9-11 years old);

- Deaf 5 (7-9 years old); 7 (9-11 years old).

All the participants had used at least once a PC with mouse. In the North of Italy, 5%
of the children did not know the tablet and they had never used it. In each session,
there were 1 or 2 children per experimenter. Among the experimenters, there were
always an expert facilitator.

2.3 Experiment Tasks and Material

At the start of every session, each learner or their educators were asked some ques-
tions in order to know the learner's school class and age, and then the experimenter
inserted the appropriate login information on behalf of the child. The facilitator
informed all children about the goal of the evaluation, that is, to present them a sys-
tem for helping their peers to better understand a story. The facilitator then asked
the children to talk aloud their opinions: since the system was in its infancy, it was
important that the children would tell us their opinions while using the system, on
what was clear and what unclear, so as to help us improve the system with their
valuable feedback.
At this point, the evaluation session started. Every child could interact with the
learner GUI by using a 10” tablet or PC with mouse. During the session, the child
could perform the sequence of tasks in Table 1.
In particular,

• younger children, whose age is 7-9, read either “La Vacanza Comincia” (The

Holiday Begins) or “A Caccia di Delfini” (Dolphin Spotting),

• older children read either “La Mania della Competizione di Benedetto” (Ben's

Racing Problem) or “Sofia e il Nano dell'Isola” (Sophia and the Island Dwarf),

and played with the associated smart games that were, on average, 1 per game level:

• 1 who game and 1 what game;

• 4 games that required reasoning about time: 1 before-after, 1 before-while, 1

after-while, 1 before-while-after;

• 2 games that required reasoning about causality: 1 cause or 1 effect, 1 cause-

effect.

The learning material is reported and described in [2] and [4]; it is also available at
www.terenceproject.eu/demos .