Introduction - Modeling Users' Experiences with Interactive Systems - page 12

(Hornbæk, 2008). As Hassenzahl (2008) puts it “[designers] are able to build ideas
from anecdotal observations and loose associations”. In contrast, reductionist ap-
proaches to evaluation serve to assess the value of a design, to compare multiple
designs, or to develop theory and criteria to support evaluation.
Fig. 1.3 A simplified framework of the design process, derived from (Martens, 2009b, per-
sonal communication). According to this framework, the designer forms beliefs about the
potential consequences of use of a given product in a given context, grounded on empirical
knowledge. These beliefs are in turn externalized to artifacts and the artifacts are validated
on a number of criteria that have been derived from prior beliefs, or empirical insights from
the previous design iteration.
We argue against this distinction between evaluative and inspirational goals in
product evaluation. It is argued that current evaluation practices that embrace this
distinction lead to inefficacy in the evaluation of interactive products. This is re-
flected in figure 1.3 which depicts a simplification of a framework of the design pro-
cess initially proposed by (Martens, 2009b, personal communication). The frame-
work identifies three distinct activities in an iterative design process: a) extrapolat-
ing knowledge from context and forming beliefs about the potential consequences
of use of a given product within a given context, b) transforming (implicit) beliefs
to explicit knowledge through the design of artifacts, and c) validating the designed
artifacts in context.
According to this view, while holistic approaches serve to extrapolate knowledge
in forming or adapting the designer's beliefs, reductionist approaches serve to val-
idate the designed artifacts using a number of criteria that have been derived from
prior beliefs, or empirical insights from the previous design iteration. This leads to
a number of limitations in the evaluation process. First, reductionist approaches to
evaluation through pre-defined measurement scales may miss potentially relevant
concepts due to a failure of researchers to recognize their relevance, but may also
result in less meaningful information when participants cannot interpret the ques-
tions with respect to their own context. Second, holistic approaches such as the
experience narratives, provide rich information in all aspects that surround an expe-
rience but may create a risk of focusing on idiosyncratic experiences while failing