iScale: Studying Long-Term Experiences through Memory - Modeling Users' Experiences with Interactive Systems - page 84

community (Gerken et al., 2007; Barendregt et al., 2006; Fenko et al., 2009; Kara-
panos et al., 2008a; von Wilamowitz Moellendorff et al., 2006; Courage et al., 2009;
Vaughan et al., 2008; Kjeldskov et al., 2008).
From a methodological perspective, one may distinguish three approaches to un-
derstanding the development of usage and experience over time (von Wilamowitz
Moellendorff et al., 2006): cross-sectional, pre-post/longitudinal, and retrospective
reconstruction. Cross-sectional approaches are the most popular in the HCI domain
(Pr umper et al., 1992; Bednarik et al., 2005). Cross-sectional studies distinguish,
for example, user groups with different levels of expertise, for instance, novice and
expert users. Observed variation in experience or behavior is then attributed to ex-
pertise in the sense of a quasi-experimental variable. This approach is, however,
limited as it is prone to confounding variables, such as failing to control for external
variation and, more importantly, falsely attributing variation across the user groups
to expertise. Pr umper et al. (1992) already highlighted this problem, by showing that
different definitions of novice and expert users lead to different results.
Beyond the cross-sectional, one may further distinguish pre-post and true lon-
gitudinal approaches. Pre-post designs study the same participants at two points in
time. For instance, Kjeldskov et al. (2008) studied the same seven nurses, using a
healthcare system, right after the system was introduced and 15 months later. Kara-
panos et al. (2008a) explored how ten individuals formed overall evaluative judg-
ments of a novel pointing device, during the first week of use as well as after four
weeks of using the product. While these approaches study the same participants over
an extended period of time, they cannot tell much about the exact form of change,
due to the limited number of only two observations. True longitudinal designs take
more measurements and employ a number of statistical techniques to track change
in general and to estimate the impact of particular events on change. Because of
their laborious nature, however, they are only rarely used in practice and research.
Different granularities in longitudinal studies can be distinguished (see von Wil-
amowitz Moellendorff et al., 2006): a micro perspective (e.g. an hour), a meso
perspective (e.g. 5 weeks) and a macro perspective, with a scope of years of use.
Studies with a micro-perspective assess how users' experience changes through in-
creased exposure over the course of a single session of use. For instance, Minge
(2008) elicited judgments of perceived usability, innovativeness and the overall at-
tractiveness of computer-based simulations of a digital audio player at three distinct
points: a) after participants had seen but not interacted with the product, b) after
two minutes of interaction and c) after 15 minutes of interaction. An example of a
study with a meso-perspective is Karapanos et al. (2009c). They followed six indi-
viduals after the purchase of a product over the course of 5 weeks. One week before
the purchase of the product, participants started reporting their expectations. After
product purchase, participants were asked to narrate the three most influential expe-
riences of each day. Studies with a macro-perspective are 'nearly non-existent' (von
Wilamowitz Moellendorff et al., 2006).
A third approach is the retrospective reconstruction of personally meaningful ex-
periences from memory. Different variants of the Critical Incident Technique, pop-
ular in marketing and service management research (Edvardsson and Roos, 2001;