Database Reference
In-Depth Information
a community it is important that we recognise that we are working towards a larger
understanding and that any given study will not present the bigger answer. It instead
will contribute to a gradual building of a bigger understanding. For this bigger under-
standing we need to encourage authors to openly discuss the limitations of their stud-
ies, because both the results and the limitations are important. This is also true for
negative results. It can be just as important to understand when there are no differ-
ences among techniques and when these differences exist.
5
Focus on Qualitative Evaluation
Qualitative inquiry works toward achieving a richer understanding by using a more
holistic approach that considers the interplay among factors that influence visualiza-
tions, their development, and their use [56]. Qualitative techniques lend themselves to
being more grounded in more realistic settings and can also be incorporated into all
types of studies. This includes qualitative studies conducted as part of the design
process [64, 73], in situ interviews [83], field studies [72], and use of observational
studies to create design and evaluative criteria that are derived from observed data
[71]. These types of studies offer potential for improved understanding of existing
practices, analysis environments, and cognitive task constraints as they occur in real
or realistic settings. In providing a brief overview of a variety of qualitative methods,
we hope to spark further research and application of qualitative methods in informa-
tion visualization; to expand our empirical approaches to include the application of
qualitative methods to design and evaluation; and to encourage a wider acceptance of
these types of research methodologies in our field.
5.1
Qualitative Methods
At the heart of qualitative methods is the skill and sensitivity with which data is gath-
ered. Whether the records of the data gathered are collected as field notes, artefacts,
video tapes, audio tapes, computer records and logs, or all of these, in qualitative
empirical approaches there are really only two primary methods for gathering data:
observations and interviews. Observation and interview records are usually kept con-
tinually as they occur, as field notes, as regular journal entries as well as often being
recorded as video or audio tapes. Artefacts are collected when appropriate. These can
be documents, drawings, sketches, diagrams, and other objects of use in the process
being observed. These artefacts are sometimes annotated as part of use practices or in
explanation. Also, since the communities we are observing are often technology us-
ers, technology-based records can also include logs, traces, screen captures, etc. Both
observation and interviewing are skills and as such develop with practice and can, at
least to some extent, be learnt. For full discussions on these skills there are many
useful topics such as Seidman [65] and Lofland and Lofland [45].
5.1.1 Observation Techniques
The following basic factors have been phrased in terms of developing observational
records but implicitly also offer advice on what to observe:
Search WWH ::
Custom Search