Conclusions - Modeling Users' Experiences with Interactive Systems - page 140

iScale is able to minimize retrospection biases when recalling one's past experiences
with a product.
Secondly, how can we aggregate the idiosyncratic experiences into generalized
knowledge? Chapter 4 presented a case where content-analysis was employed in
deriving key themes in the data, classifying narratives into a set of main categories
and identifying the distinct distributions over time across the different categories.
Two limitations were identified in this procedure. First, it is a labor intensive ac-
tivity which becomes increasingly a concern when qualitative data may be elicited
from a large amount of participants as in the case of iScale. Second, it is prone
to researcher bias as humans often rely on heuristics in forming judgments about
the relevance or similarity of two or more data instances (Kahneman et al., 1982).
Chapter 6 proposed a novel technique for the semi-automated analysis of experi-
ence narratives that combines traditional qualitative coding procedures (Strauss and
Corbin, 1998) with computational approaches for assessing the semantic similarity
between documents (Salton et al., 1975). It was argued that the proposed approach
supports the researcher through semi-automating the process of qualitative coding,
but also minimizes the risks of overemphasizing interesting, but rare experiences
that do not represent users' typical reactions to a product.

7.2 Implications for the Product Creation Process

We argued against the distinction between evaluative and inspirational purposes of
product evaluation which constitute the current practice (figure 7.1). Two methods
were proposed that aim at extrapolating rich qualitative information and informing
the validation process. This leads to two benefits in the evaluation of interactive
products. Firstly, it enhances the content validity of the validation process as the
measurement model reflects more accurately the dominant dimensions of users' ex-
periences with the product. Secondly, it enhances the validity of the extrapolation
process as it quantifies the significance of given experiences and as such it mini-
mizes the risks of overemphasizing interesting but rare insights.

7.2.1 Integrating Subjective and Behavioral Data

This work focused on methods for the subjective evaluation of interactive products.
How can such methods be combined with objective information regarding users'
behavior? Behavioral data may not only provide insights into the actual usage of
a product but may also inform the elicitation of users' experience. In Funk et al.
(2010) we provided a case study where usage information was used to augment
subjective feedback in two ways (figure 7.2). First, users were able to provide sub-
jective feedback at the times they liked as in traditional event-based diaries (Bolger
et al., 2003). Through process mining techniques (van der Aalst et al., 2007) we
were able to inquire into users' interactions that preceded the users' reports. Sec-
ond, through an observation specification language (Funk et al., 2008) we were able
to identify interaction patterns that we were interested in and probe for subjective