Methodology III: Empirical Evaluation - Foundations for Designing User-Centered Systems - page 356

testing sessions a new page header was developed, and tests with the next four
participants demonstrated that the introduction of the new design had resolved the
problem. This rapid development methodology is used to test designs in an agile
way, which makes the most effective use of the testing time allocated to a project.
13.3.2 Field Studies and Field Experiments
Field studies, as the name implies, are evaluations that are carried out in the field,
that is, in real world settings. Field studies are often carried out to discover more
about the context of use of a technology that is to be designed or is being designed.
Studying activities in the ''real'' world can be challenging. If you just think about
the things that happen every day in an office environment you should start to get
the picture. In an office, users may be carrying out some task using the computer.
They can break off from their work at any point, though, such as when they are
interrupted by the telephone ringing, if a colleague stops by to discuss something
(work or otherwise), or if their boss calls them into a meeting. So carrying out the
task is not simply a matter of planning what to do, then just getting on and doing it
step by step in sequence, from start to finish: people will often have to juggle
several, possibly unrelated, and often unscheduled, activities at once. The big
advantage of field studies is that they show you how people really work. One
obvious design implication of the fractured nature of work is that you should make
it relatively easy for people to pick up where they left off after an interruption. The
main disadvantage of field studies is that it is often very difficult to exercise
experimental control over what happens, so it is harder to focus on the relation-
ships between some of the task variables, and to have results that are both general
and applicable to other settings or times.
Field experiments are trials of technologies in real world settings. This is often
when a fully functional prototype can be deployed into a real world setting and, as
designers and developers, we want to see how users will interact with the tech-
nology. Such field experiments tend to be for non-safety-critical systems, such as
recreational and social Internet sites. Often there is some latitude for changing the
technology, but most of the functionality is set. In this instance, the evaluation will
likely involve many different methods: collecting usage data, conducting obser-
vations of the technology in use, interviewing and surveying users, small con-
trolled experiments, and so on (for an example of a technology that was fielded and
evaluated see Churchill et al. 2003 ).
13.3.3 (Expert) Heuristic Evaluation
Heuristic evaluation (Nielsen and Molich 1990 ) is a relatively informal way of
analyzing the usability of an interface design. A small select number of people—
ideally interface design experts, and preferably domain experts too—are asked to