Civil Engineering Reference
In-Depth Information
THINK-ALOUD PROTOCOLS
Think-aloud is a method where a participant verbalizes what she is trying to accomplish,
as she is looking for information and trying to solve problems (Ericsson and Simon,
1993). Think-aloud is often done as part of usability testing. The participants are asked to
think out loud as they perform the task. Ericsson and Simon (1993) cited several authors
who were in favor of this method.
• “Try to think aloud. I guess you often do so when you are alone and working on a
problem” (Duncker, 1926).
• “Think, reason in a loud voice, tell me everything that passes through your head during
your work searching for the solution to the problem” (Claparede, 1934).
• “Don't plan what to say or speak after the thought, but rather let your thoughts speak, as
though you were really thinking out loud” (Silveria, 1972).
• “In order to follow your thoughts we ask you to think aloud, explaining each step as
thoroughly as you can” (Smith, 1971).
The purpose of think-aloud is to identify the user's intentions and actions for the purpose
of improving the interface design:
• Were the intentions correct? If they were not correct, how can the interface be improved
so that the user will have the correct intentions?
• Were the actions correct? Were they in agreement with the user's intentions? If the
actions were wrong, how can the interface be improved to avoid wrong actions?
One problem with the think-aloud method is that a secondary task is imposed, in addition
to the primary task of solving the problem presented on the screen. The secondary task—
speaking—will hence add to the overall workload, and users may therefore perform
worse than they would if they did not speak. Rosson and Carroll (2002) commented that
although it is not a natural behavior for most computer users to think aloud, it reveals
many unobservable cognitive sources of usability problems.
QUESTIONNAIRES
Questionnaires can be used to measure users' subjective evaluations of a product or a
software program. A questionnaire often includes a five-point or a seven-point scale,
which can be used to measure how good or bad the user finds the situation. The number
is useful because one can sort the good and poor parts of the interface. For example, one
may ask the user the question in Figure 7.5. The surprising result is that many users from
North American have difficulties with this particular icon, while people in Europe and
Asia recognize it as a warning sign, since it is commonly used on the roads.
A questionnaire must also be tested to see if it is usable. There are often many
opportunities to misunderstand questions. Note that for the question in Figure 7.5, the
wording is not correct: “Very Effective” is not the opposite of “Very Poor.” One should
either use poor-good or ineffective-effective. In addition, a seven-point scale provides
better resolution than a five-point scale. Respondents have a tendency of avoiding the
extreme ratings 1 and 5, which leaves only a 3-point scale.
Try to word the questions so that the same scale can be used throughout the
questionnaire. For example a question about aesthetics can be formulated as follows: “Do
