iScale: Studying Long-Term Experiences through Memory - Modeling Users' Experiences with Interactive Systems - page 104

.05, h p =0.15, but not for product qual-
ity, F(1,89) = 1.64, p=.2, h p =0.02, or for the interaction between mode of recall
and product quality, F(2,89) = 1.19, p=.7, h p =0.007. Post-hoc tests using the Bon-
ferroni correction revealed that participants in the constructive condition elicited a
significantly higher number of experience reports than in the value-account (p=.04,
Cohen's d=0.7) and the control condition (p

effect for mode of recall, F(2,89) = 7.74, p <

.001, d=1). No significant differences
were demonstrated between the value-account and the control condition (p=0.67,
d=0.3).

<

5.4.2.2 Richness of Elicited Experience Reports

To identify the different factors of richness, we submitted the experience reports to a
qualitative content analysis (Hsieh and Shannon, 2005). Open coding was performed
by the first author and resulted in three main types of information present in the re-
ports: discrete event information summarizes references to a concrete occurrence
that influenced the experience (e.g., “I found out that there was a mail application”),
temporal information which summarizes references to the exact time at which the
experience took place (e.g., “The first week I tried to practice the touch screen”),
and, expectations which summarize references to participants' expectations about
the reported experience (e.g., “As I bought this phone in Europe I expected that at
least all European languages are available for free online”). Each report was coded
by the first author for the presence or absence of temporal information, discrete
event information and expectations. Interrater agreement (Fleiss' Kappa, 2003) was
computed on a small random sub-set of the reports (10%) coded by the first author
and two additional researchers: temporal information (K=0.97), discrete event in-
formation (K=0.71), expectations (K=0.77). In all cases, interrater agreement was
satisfactory.
Significant differences were observed between the constructive and the no-
graphing (control) version of iScale with regard to references to discrete events
(p .05,

Fisher's one-tailed exact test), but not with regard to references to participants' ex-
pectations before the experienced event. 45 out of 146 (31%) reports elicited through
the constructive iScale contained at least one cue referring to a discrete event as op-
posed to 38 out of 192 (20%) in the control condition, and 20 out 146 (14%) con-
tained at least one cue referring to temporal information as opposed to 14 out of 192
(7%) in the control condition. No significant differences were observed between the
constructive and the value-account version as well as between value-account and
the no-graphing (control) version in any of the three dimensions or richness.

< .05, Fisher's one-tailed exact test), references to temporal information (p <

5.4.2.3 Test-Retest Consistency in Time Estimation

The two sessions of the study are expected to more or less result in the same ex-
perience reports. Thus, reports across these two sessions can be coupled. A total of
325 experience reports (71%) were coupled. We only coupled reports when we had
sufficient confidence that they reported the same experience. This, inevitably, left