Environmental Engineering Reference
In-Depth Information
4.4 Procedure
The experiment was conducted in two consecutive days. The first day, the participants were
familiarized with the MTS and the corresponding training condition. Later, they had to use
the training system to learn how to build the virtual helicopter model during four training
sessions (i.e. they built the virtual helicopter four times), with a short break between sessions
to allow the examiner to restart the system. The initial position of the bricks was generated
randomly, so it was different in each training session. The evaluator was in the room along
with the participant and information about the task performance (number of aids, number of
errors and the training time) was logged automatically by the system for further analysis of
the data. At the end of the training, the participants of both groups were requested to fill in a
questionnaire to evaluate their experience with the multimodal system.
The second day, the participants had to build the real LEGO™ helicopter model (the same one
as in the training sessions). Each participant was instructed to complete the model as rapidly
as possible, but without mistakes and missing bricks. At that time, the participants had the
instruction topic to consult it when they did not know how to continue with the task and this
action was recorded as one “aid”. The session was recorded in video for further analysis.
4.5 Performance measures
The measures to analyze the final performance of the trainee were: the training time, the
real task performance (execution time, number of aids with the instruction topic, number of
non-solved errors and type of errors) and the evolution of the performance of the trainee along
the training process and the transition from the training system to the real task. The subjective
evaluation of the platform was also analyzed.
4.6 Results and discussion
Before starting to analyze the results, it would be note that although the experimental task
is simple conceptually its correct execution is difficult due to the requested cognitive skills
component. The task required memorizing the exact position of 75 bricks, most of them
without any indication for functional or semantic meaning (just bricks with different size,
color and number of pins) and some of the assembly procedure was totally arbitrary (different
combinations of the bricks could generate the same shape of the helicopter model but only
one specific combination was considered valid and the rest of options were considered as
non-solved errors).
In general, there were not significant differences between the results obtained in both
groups. Statistical analysis was performed with independent-samples t-tests, equal variances
assumed.
Figure 10 shows that the mean real task execution time for the direct aids group, 18.6 minutes,
dd not have significant difference with the mean time for the indirect group, 17.9 minutes
(t
36
=0.36, p=0.721). Nevertheless, it is important to point out that during the building of the
real LEGO™model, the participants of the indirect aids group had some advantages respect to
the others because they were used to consult the instruction topic during the training period,
and in this way they could get the information quicker than the others.
In order to analyze the real task performance obtained in each group, each step of the task (75
steps in total) was classified according to the following criteria:
