Methodology III: Empirical Evaluation - Foundations for Designing User-Centered Systems - page 360

prototype is refined after each iteration of the development cycle: the prototype
evolves towards the deliverable version. This is the sort of approach that is used in
developing web sites, where they use wireframes to lay out the basic initial design,
which then gets filled in and refined as the design evolves. In revolutionary pro-
totyping, the current prototype is thrown away at the end of each iteration of the
development cycle, and a new prototype is developed.
In addition to helping identify design issues, prototypes can also be used to help
users to articulate requirements. People often find it much easier to talk about
something concrete, referring to the prototype, than to talk about something
abstract, where they have to imagine what the application or product should do.
Prototypes vary in cost, depending upon the sophistication of the prototype and
the length of the evaluation period (laboratory-based user testing vs field studies).
They do tend to give good results and are suitable for many stages of the design
process, for both formative and summative evaluations.
13.4.3 The Final System
Evaluations of the final system will often be performed in house first, possibly
using laboratory-based testing. If there is latitude for some redesign, systems may
be deployed and field experiments conducted, but this only tends to be the case for
systems that are not safety-critical, as noted above. For enterprise, and safety- and
security-critical systems, it is more usually the case that the system is evaluated in
full before it gets delivered to the customer. The final system will usually be
subjected to a formal acceptance test, normally on the customer's premises, where
the customer will sign to say that the system has successfully passed the agreed
tests. You should note that web sites are very rarely tested at the customer's site
(largely because they will normally be used from elsewhere).
Once a system is delivered and has been accepted by the customer, it is unlikely
that any further formal evaluation will take place. The picture is slightly different
for web sites, however, where the delivered system will normally reside in a
dynamic environment, so the iterative development may continue, albeit with
iterations that have a longer duration. In both cases, data on patterns of usage may
be collected, along with information about problems logged with customer sup-
port, as noted above, and this can be used to inform future development projects
and refinements to existing systems and products.
13.5 Measuring Usability
There are several dimensions to usability, so there are several measures, both
qualitative and quantitative, that can be used to indicate how usable a particular
artifact is. Most people will be familiar with task time as the de facto standard for