Geography Reference
In-Depth Information
Table 1 Performance metrics (effectiveness and efficiency) for tested layouts
Successfully
completed
tasks
Time spent on a task on
average (95 % confidence
interval)
Normalized number of mouse clicks
needed for a task on average (95 %
confidence interval)
Stimuli
Layout 1
66 %
8.93 s (
0.68 s)
2.51 (
0.20)
Layout 2
72 %
9.21 s (
0.68 s)
3.31 (
0.24)
Layout 3
90 %
5.37 s (
0.51 s)
1.85 (
0.18)
Layout 4
93 %
4.52 s (
0.41 s)
1.87 (
0.21)
Layout 5
78 %
7.87 s (
0.47 s)
2.53 (
0.22)
Layout 4 was also most efficient in terms of required time (Table
1
). On average,
tasks could be solved twice as fast with Layout 4 as with Layout 2. Since some tasks
involved more consecutive actions than others, the number of clicks was normal-
ized for a meaningful comparison by the number of screen changes. For example,
students had to click at least twice to solve the first task in Layout 1, whereas in
Layout 5 one click would have been sufficient; so we divided the number of clicks
by 2 (i.e., the number of screen changes) for Layout 1. The normalized number of
clicks is considered as a measure for efficiency, as it indicates the participants
'
effort (Tamir et al.
2008
) to solve a task. A value of 1 would be optimal. However
participants needed for instance 2.5 times more clicks per task on average in Layout
1 and 5 before they managed to find the right GUI element (Table
1
).
A statistically significant difference in the average time spent on the tasks was
found between Layouts 3/4 and Layouts 1/2/5, because their confidence intervals at
a 95 % level do not overlap. A pairwise analysis of variances resulted in signifi-
cantly different average times between Layout 3 and 4 (p-value
0.01) as well as
¼
Layout 1 and 5 (p-value
0.03), however not between Layout 1 and 2
¼
(p-value
0.37). Regarding the normalized number of clicks needed to solve a
task on average, 95 % confidence intervals do not overlap for Layouts 3/4, Layouts
1/5 and Layout 2. A pairwise analysis of variances did not reveal a significant
difference concerning the normalized number of clicks between Layout 3 and
4 (p-value
¼
0.91).
A qualitative visual analysis of click maps suggested that the students were
strongly focused on the given tasks, as they were mainly clicking on the task-
relevant GUI items. Only a small number of clicks appeared on browser elements
like the URL bar, the refresh or the close button. To depict the students
0.88) as well as Layout 1 and 5 (p-value
¼
¼
first
intuition to solve the task, we separated their first clicks from subsequent clicks
(Fig.
2
, top). By this, we could identify whether students followed our intention or
whether they thought of alternative ways to solve a task. In most cases, first clicks
occurred at just a few functions. If the first click did not lead to the right result,
following clicks were more scattered. In the latter case, students lost time when
functions were not grouped.
'
