Methodology III: Empirical Evaluation - Foundations for Designing User-Centered Systems - page 345

Chapter 13
Methodology III: Empirical Evaluation
Abstract Evaluation is an essential part of development. There are several good
reasons for carrying out user testing in particular. A successful evaluation requires
careful planning. Here we describe the issues that you need to take into account
and discuss several effective methods that can be used to collect data. User testing
reduces the risk that you will deliver a system to your users that is unusable and is
therefore ineffective. We also touch briefly on the need make sure that any
evaluation that you carry out is conducted according to appropriate ethical
guidelines.
13.1 Introduction
Evaluation should be a routine and regular part of system development. Attitudes
to evaluation continue to vary widely, however, with many people still mistakenly
believing that testing only happens at the end of development. The main problem
with such an approach is that if development overruns while the delivery deadline
remains fixed, it is usually testing that gets cut (because it is the last thing to
happen). The net result is that the delivered system ends up being tested inade-
quately and, as a result, does not work as expected.
There are two statements that are worth remembering when it comes to eval-
uation. The first is ''Test early, test often.'' The second is ''Quick and dirty is better
than nothing.'' Any evaluation program should deliver something informative for
the system design and development. Testing can be expensive, but need not be.
Lightweight methods that offer a proportionately larger return on investment are
also available (Monk 1998 ; Nielsen 1993 ).
It is also worth remembering that when software engineers, in particular, talk
about evaluation they will often refer to V & V, or verification and validation.
Verification is about making sure that you are building the product right, and it is a
process that is often carried out within the developer company. In contrast, vali-
dation is about making sure that you are building the right product, and usually