Performance Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 76

• TheObserverXTfromNoldusInformationTechnology( http://www.noldus.com/

human-behavior-research/products/the-observer-xt )

• OvoLoggerfromOvoStudios( http://www.ovostudios.com/ovologger.asp )

• MoraefromTechSmith( http://www.techsmith.com/morae.html )

• UsabilityTestingEnvironment(UTE)fromMindDesignSystems( http://utetool.

com/ )

• UsabilityTestDataLoggerfromUserFocus( http://www.userfocus.co.uk/resources/

datalogger.html )

Ourwebsite,MeasuringUX.com,alsoincludesasimplemacroforuseinMicrosoft
Word for logging start and finish times. An automated method of logging has several
advantages. Not only is it less error-prone but it's also much less obtrusive. The last thing
you want is a participant in a usability test to feel nervous from watching you press the
start and stop button on your stopwatch or smartphone.

TURNING ON AND OFF THE CLOCK
Not only do you need a way to measure time, but you also need some rules
about how to measure time. Perhaps the most important rule is when to turn the
clock on and off. Turning on the clock is fairly straightforward: If you have the
participants read the task aloud, you start the clock as soon as they finish read-
ing the task.

Turning off the clock is a more complicated issue. Automated time-keeping
tools typically have an “answer” button. Users are required to hit the “answer”
button, at which point the timing ends, and they are asked to provide an answer
and perhaps a few additional questions. If you are not using an automated
method, you can have users report the answer verbally or perhaps even write
it down. However, there are many situations in which you may not be sure if
they have found the answer. In this situation, it's important for participants to
indicate their answer as quickly as possible. In any case, you want to stop tim-
ing when the participant states the answer or otherwise indicates that she has
completed the task.

TABULATING TIME DATA
The first thing you need to do is arrange the data in a table, as shown in
Table 4.2 . Typically, you will want a list of all the participants in the first column,
followed by the time data for each task in the remaining columns (expressed in
seconds, or minutes if the tasks are long). Table 4.2 also shows summary data,
including the average, median, geometric mean, and confidence intervals for
each task.