Methodology III: Empirical Evaluation - Foundations for Designing User-Centered Systems - page 368

13.5.10 User Experience
Finally, there is the concept of user experience (Tullis and Albert 2008 ). In
addition to these mostly quantitative measures, there is the qualitative experience
of using the system that is being tested. This is an important concept, and is related
to several concepts that can sometimes be hard to define, and even harder to
measure. Many organizations now rate a high level of user experience (explained
in Chap. 2 ) as being a major determinant in the success of a system or product.
One of the factors that will influence user experience is task importance. If
the task is important to the user and the system gets the task done, then, it will
be a successful system. Early systems, e.g., TVs, phones, portable phones, PDAs,
Blackberries, were all hard to use and the times taken to use them were rela-
tively high compared to today's standards. However, they were successful
because they provided a better experience or supported a task that was not
supported before. Over time and extended use, other measures and aspects
became important.
13.6 The Ethics of Evaluation
Studies that involve users interacting with technological products now routinely
need to be vetted to ensure that participants are treated appropriately. This means
that ethical clearance (or approval) is required from the appropriate authoritative
body. In most countries this will normally be done by an ethics committee, whilst
in the US it will be carried out by an institutional review board (IRB). They will
review the study to determine that the relevant guidelines are being followed. The
main things they check are whether vulnerable people will be involved, whether
participants are aware of what they are committing to in the study, and that any
collected data is stored appropriately. Usually the latter involves anonymizing data
so that they cannot be linked to the participant. These requirements vary based on
funding, use, publication, and teaching, so take advice if you have not done this
before.
As a matter of routine you should produce information sheets for participants,
describing the study and explaining that they can withdraw from the study at any
point. You should also take informed written consent, having them sign a consent
form that says that they have read and understood the information sheet, that they
are willing to take part, and that they understand that they can withdraw at any
point. You should also think about how you will debrief at the end of a testing
session: you could either give them a debrief sheet, explaining the purpose of the
study in more detail, or simply verbally debrief them. You may also want to ask
them not to discuss the study with others, because it could influence their behavior
if they were subsequently in the study. Further details on this process are available
(Ritter et al. 2013 ; Ritter et al. 2009 ).