Evaluating Information Visualizations - Information Visualization: Human-Centered Issues and Perspectives - page 23

Precision: a result is precise to the degree to which one can be definite about the
measurements that were taken and about the control of the factors that were not
intended to be studied.

•

Realism: a result is considered realistic to the extent to which the context in
which it was studied is like the context in which it will be used.

•

Figure 1 (adapted and simplified from McGrath [50]) shows the span of common
methodologies currently in practice in the social sciences. They are positioned around
the circle according to the labels: most precision, most generalizability and most real-
ism. The closer a methodology is placed to a particular label, the more that label ap-
plies to that methodology. Next, these methodologies are briefly described. For fuller
descriptions see McGrath 1995.

Fig. 1. Types of methodologies organized to show relationships to precision, generalizability
and realism. (adapted, simplified from McGrath 1995)

Field Study: A field study is typically conducted in the actual situation, and the ob-
server tries as much as possible to be unobtrusive. That is, the ideal is that the pres-
ence of the observer does not affect what is being observed. While one can put con-
siderable effort into minimizing the impact of the presence of an observer, this is not
completely possible [50]. Examples of this type of research include ethnographic
work in cultural anthropology, field studies in sociology, and case studies in industry.
In this type of study the realism is high but the results are not particularly precise and
likely not particularly generalizable. These studies typically generate a focused but
rich description of the situation being studied.

Field Experiment : A field experiment is usually also conducted in a realistic setting;
however, an experimenter trades some degree of unobtrusiveness in order to obtain
more precision in observations. For instance, the experimenter may ask the partici-
pants to perform a specific task while the experimenter is present. While realism is