Evaluating Information Visualizations - Information Visualization: Human-Centered Issues and Perspectives - page 38

ings, and opinions shed light on how they relate to existing support, and can effec-
tively guide the development of new support. This type of knowledge can be very
important at the early stage of determining what types of information visualizations
may be of value.
Summary of qualitative methods as primary: These four methods are just exam-
ples of a huge variety of possibilities. Other methods include action research [42],
focus groups [48], and many more. All these types of qualitative methods have the
potential to lessen the task and data comprehension divide between ourselves as visu-
alization experts and the domain experts for whom we are creating visualizations.
That is, while we can not become analysts, doctors, or linguists, we can gain a deeper
understanding of how they work and think. These methods can open up the design
space, revealing new possibilities for information visualizations, as well as additional
criteria on which to measure success.

5.3 Challenges for Qualitative Methods

A considerable challenge to qualitative methods is that they are particularly labour
intensive. Gathering data is a slow process and rich note taking is an intensive under-
taking, as are transcribing and subsequent analysis.

5.3.1 Sample Sizes
Sample sizes for qualitative research are determined differently than for quantitative
research. Since qualitative research is not concerned with making statistically signifi-
cant statements about a phenomenon, the sample sizes are often lower than required
for quantitative research. Often, sample sizes are determined during the study. For
instance, a qualitative inquiry may be continued until one no longer appears to be
gaining new data through observation [3]. There is no guideline to say when this
'saturation' may occur [70]. Sample sizes may vary greatly depending on the scope of
the research problem but also the experience of the investigator. An experienced in-
vestigator may reach a theoretical saturation earlier than a novice investigator. Also,
because each interview and/or observation can result in a large amount of data, some-
times compromises in sample size have to be made due to considerations about the
amount of data that can be effectively processed.

5.3.2 Subjectivity
Experimenter subjectivity can be seen as an asset because of the sensitivity that can
be brought to the observation process. The quality of the data gathering and analysis
is dependent on the experience of the investigator [56]. However, the process of gath-
ering any data must be concerned with obtaining representative data. The questions
circle about whether the observer has heard or understood fully and whether these
observations are reported accurately. Considerations include:

Is this a first person direct report? Otherwise normal common sense about 2 nd ,
3 rd , and 4 th hand reports needs to be considered.

•

• Does the spatial location of the observer provide an adequate vantage point from

which to observe, or might it have led to omissions?