User Experience Over Time - Modeling Users' Experiences with Interactive Systems - page 68

Ta b l e 4 . 2 Latent constructs measured through a single scale

Latent construct Measurement item

Usefulness useful - useless

Ease-of-use simple - complicated

Stimulation innovative - conservative

Identification positive message about me -

negative message about me

Goodness good - bad

Beauty beautiful - ugly

4.4.2 Data Analysis

A total of 482 experience narratives were collected during the four weeks of use.
These were submitted to a conventional qualitative Content Analysis (CA) (Hsieh
and Shannon, 2005; Krippendorff, 2004). Conventional CA is appropriate when
prior theory exists but the researcher wishes to stay open to unexpected themes
and only at a later stage relate findings to existing theory, whilst it shares a similar
analytical approach with Grounded Theory. Our approach consisted of three steps:
Open coding - A detailed coding aimed at identifying key themes in the data
without imposing pre-conceived categories. The process resulted in about 70 loosely
connected codes referring to about 700 instances in the data.
Axial coding - In the second step, the initial set of phenomena described by open
codes was categorized using axial coding. Open codes were grouped into categories
which were subsequently analyzed in terms of properties and dimensions. This re-
sulted in a set of 15 main categories reflecting aspects like the aesthetics of interac-
tion, learnability and long-term usability (see table 4.3).
Quantitative analysis - All experience narratives were classified as being primar-
ily related to one of the fifteen categories. This process was independently con-
ducted by the first author and an additional researcher (Interrater agreement K=.88).
Both researchers were already immersed in the data as they both participated in
the axial coding process. Narratives for which no agreement was attained were ex-
cluded from the subsequent analysis. We avoided clarifying disagreements to ensure
high uniformity within experience groups. The distribution of experience narratives
over the four weeks of the study was then identified for each of the 15 categories.
Based on the resulting temporal patterns and semantic information, the 15 categories
were then mapped into 3 broad themes reflecting distinct phases in the adoption of
the product: Orientation , Incorporation and Identification . An additional theme,
called Anticipation , was added to reflect users' a priori expectations that were cap-
tured during the first week of the study. Finally, separate regression analyses with
the two overall evaluative judgments, i.e. goodness and beauty , as dependent and
the four quality attributes, i.e. usefulness , ease-of-use , stimulation and identifica-
tion , as independent variables, were run for the three main groups of experiences,