HEURISTIC EVALUATION: A GUIDE FOR USE IN INFORMATION VISUALIZATION - Information Visualisation: Techniques, Usability and Evaluation - page 184

means of interviews, questionnaires, observations, and think-aloud methods
[15][22]. This data is analysed by examining it and interpreting meaningful
patterns and themes to determine how they answer the actual research
question(s). Such a study is useful in itself. Qualitative research in
information visualization has become increasingly popular and recognized.
One reason is that quantitative studies often focus on perceptual or simple
cognitive tasks. Issues on a higher level, such as exploration, insight, and
decision-making can be hard to assess by objective and quantitative
approaches. Yet, it is possible, as evident by the work on how to measure
insight by Saraiya, North and Duca [23]. A qualitative study can also
complement a quantitative approach to obtain subjective views from
participants and to explain quantified data.
Another classification that answers questions about when to use a
particular method is formative evaluation. A formative evaluation provides
feedback that can be used to improve design during development. It
answers questions such as: what problems happen and why, and what can
be done to improve the visualization. A summative evaluation, on the
other hand, assesses the extent to which user and perhaps organizational
objectives have been achieved at the end of a development. It answers
questions about how effective and efficient a visualization is to users [22].
Formative and summative evaluation can be conducted by using a
quantitative or a qualitative method and at all levels of constraints. All
approaches to evaluation are scientific and useful (not only high constraint
ones) when used properly.
Heuristic Evaluation
Heuristic evaluation [2][4][24] is an analytical method that belongs to the
family of usability inspection methods. Usability inspection is the generic
name for methods where evaluators inspect the subject for evaluation, e.g.,
an interface [2]. The purpose is to identify usability problems. However,
heuristic evaluation may also address matters such as the severity of the
problems found, and how they can be overcome by redesign. The aim is to
find and explain usability issues so they can be addressed appropriately,
thus improving the usability. Heuristic evaluation can be used at all stages
of development, from early design ideas and prototypes, where the
purpose is formative and results are used for improvements during an
iterative design and development process, to evaluation of more mature
systems, with near complete design under realistic settings, where the
evaluation can be summative. Heuristic evaluation is then used to
determine whether the design meets some specific and predefined usability