Methodology III: Empirical Evaluation - Foundations for Designing User-Centered Systems - page 347

We know a lot about users, and their capabilities and limitations. We have
covered much of this in the first part of this topic and considered the implications
for system design. Given what we know about the human visual system ( Chap. 4 ) ,
we can deduce that a 3-point font is likely to be too small to read. User testing is
not necessary here; the decision can be derived from first principles. User testing
may still be essential, however, for other system related questions—for example,
to provide evidence that the right design choices and assumptions have been made
when theories and assumptions make different suggestions, and that the system
will work in its intended context.
Comprehensive user testing can be expensive and time consuming, but that
does not mean that user testing should be ignored. As noted above, there are
several discount or lightweight methods that can be used (Monk 1998 ; Nielsen
1993 ) to identify potential problems more cheaply, and at an earlier stage of
development.
13.1.2 When Do We Carry Out User Testing?
Everybody makes mistakes—in the most general sense of the term—from time to
time. This includes developers who may base their design decisions on insufficient
or incorrect information at any stage of the development process—for example
during analysis, design, and/or implementation. The consequences of these mis-
takes may not always be immediately apparent, but may surface much later.
Carrying out evaluations helps to identify potential problems so that they can be
appropriately rectified before the system gets delivered.
The ultimate goal of user testing is to make sure that the users can get the
delivered system to do what they want it to do. In addition to helping to show that
a system meets its requirements and is acceptable to the users, testing can also help
to identify flaws in the development process. User testing is therefore often an
effective way of providing designers with timely feedback at a relatively early
stage during ongoing iterative development. Evaluating cheap, simple prototypes
early in the development cycle can help to avoid potentially costly mistakes at a
later stage in the lifecycle.
It is worth noting one of the practicalities of design that has implications for
evaluation. It may not always be possible for designers to consult users during all
stages of development. Instead, they work with a customer representative to come
up with a system that will be acceptable to the users. Sometimes, however, the
customer representative is not one of the target users, so they may not fully
understand the subtleties of how the users really carry out their work using the
system. It is therefore better to try to make sure that you can involve real users
during evaluation.
Many developers still neither really understand what their users do, nor why
they do what they do. Design and development are often carried out in isolation
from the end users—agile methods can help address this limitation—with the