Errors: An Inherent Part of Human-System Performance - Foundations for Designing User-Centered Systems - page 284

Alternatively, we could design the system so that it would prevent the error
occurring, or at least mitigate the consequences of the error.
There is a standard set of questions that applies to the collection of human
performance data:
• Why gather data?
• What sort of data to gather?
• Where (and when) to gather data?
• How much data to gather?
• How to gather data?
These questions provide a basic framework for the discussion of the various
issues involved. The issues interact and overlap, so it is not possible to answer each
of the questions in isolation.
It is worth re-emphasizing at this point that erroneous behavior is an inherent
part of human performance, i.e., there is a close link between knowledge and error
(Mach 1905 ). The corollary of this is that knowledge or data relating to correct
performance are also needed to make an informed judgment regarding each
potential instance of state misinterpretation.
Here we only address the question of how to gather data, focusing on complex
domains where the system can change dynamically without human intervention
(such as aircraft, cars, and power plants). There are three basic data collection
methods that can be used: laboratory-based experiments; field-based observation;
and archive data. Each has its own set of strengths and weaknesses, as discussed
below. The final choice of method depends on the particular situation at hand.
10.2.1 Laboratory-Based Experiments
The first method is the standard behavioral science method of using laboratory-
based experiments. The main advantage of this method is that it allows for the
independent variables associated with a particular phenomenon to be experimen-
tally controlled. By varying one (or more) independent variables, the effect on the
dependent variable can be observed.
The main drawback is the lack of face validity between laboratory-based
experiments and the real world situation. This lack of validity makes it inappropriate
at best, and impossible at worst, to generalize from the results obtained in the
laboratory to the situation in the real world. The use of laboratory-based experiments
largely ignores the current consensus of opinion on the importance of context in
shaping human performance (Hollnagel 1993a ; Hutchins 1995 ; Nardi 1996 ).
It is a long and difficult task to develop and conduct laboratory experiments that
would meet all the requirements of a situation that would definitely lead to a
human error. This is partly because of the problems of availability and selection of
appropriate experimental subjects. The subjects would need lengthy experience of