Database Reference
In-Depth Information
development cycle, not just during the initial stages of design."
The researchers also write: "Both techniques uncovered the same usability problems for the most
part, and at about the same level of sensitivity, as witnessed by the high positive correlations in
the two studies between the proportion of low- and high-fidelity subjects finding particular
problems. That is, when a problem was found by a high proportion of subjects in the high-fidelity
group, it also tended to be found by a high proportion in the low-fidelity group. However, some
problems that were found in the high-fidelity condition were not found in the low-fidelity condition,
and vice versa. This is true for both studies."
In addition to their published findings, Bob Virzi told me about some further exploratory analysis of their
data. They compared the high-fi prototype to the low-fi one in several ways: the number of successfully
completed tasks, the average time on a task, and the number of steps taken. These three measures
seemed to suggest few differences between the two kinds of prototypes, although this conclusion
requires additional scrutiny before being granted the same scientific weight as the published paper. In
other words, users were able to do the same tasks, in about the same number of steps, regardless of
the kind of prototype. There was a greater variability in task times, but the relative relationships were
preserved—if task 6 took longer than
task 1
in the hi-fi prototype, those two tasks showed the same
relationship in the low-fi prototype. From a practical perspective, these results imply that if something is
hard to do with the paper prototype, it's probably hard to do with the real interface as well.
Catani and Biers (1998)
Abstract:
"This study investigated the effect of prototype fidelity on the information obtained from
a usability test with potential users of a software product. Users engaged in a series of structured
tasks using one of three prototypes which emulated a Microsoft® Windows 95
™
based library
search computer system and which differed only in their fidelity (low = paper, medium = screen
shots, high = interactive Visual Basic prototype).
Results indicated there were no significant
differences as a function of prototype in the number and severity of problems
encountered nor in the subjective evaluation of the product
[emphasis added]. More
importantly, there was a high degree of commonality in the specific problems uncovered by users
using the three prototypes."
The authors also make an interesting conjecture: "However, in the real world, as the product moves
through the development cycle, prototypes change not only in fidelity but also in content and structure.
Perhaps it is this difference in content and structure which leads to the common belief that different
usability problems should emerge as a function of prototype fidelity." In other words, because later-
stage designs tend to be better than early-stage ones, some people might mis-attribute the interface
problems to the prototyping method rather than the completeness of the design.
Novick (2000)
Abstract:
"This paper introduces low-tech simulation as a technique for testing procedures and
their documentation. The key idea is to test the interface-procedure-documentation set in the
early stages of development, where changes can be still be made easily, using extremely low-
cost simulation materials. Using this low-tech method, developers can test and improve the
usability of documentation for user interfaces that are still being designed. An extended example,
involving a new aircraft cockpit interface for text-based air-traffic control communication, presents
a look at the low-tech approach in use. An evaluation of low-tech simulation shows that the
approach was practical. Qualitative analysis indicates that low-tech evaluation produces results
that are different from those obtained using the cognitive walkthrough for operating procedures
and similar to those obtained using traditional high-tech simulation."
Although this study focused on documentation, in practice the interface and its documentation are not
easily separable—Novick created a paper prototype of the interface as a means of testing its
documentation. Even though only one user tested the paper prototype, Novick found that the simulation
was good enough to "elicit unexpected [user] behaviors that shed light on ways to improve the
procedures and their documentation." The problems found with the paper prototype were later
confirmed by testing with six users on a high-tech simulator. Perhaps most interesting of all, both the
paper prototype and the high-tech tests found some problems that had not been predicted by the six
Search WWH ::
Custom Search