Performance Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 79

WHAT'S THE RIGHT PRECISION FOR TIME DATA?
How accurate do you need to be with your time data? Of course, it depends on what
you're measuring, but the majority of the times we deal with it in the user experience
world are either in seconds or minutes. It's very rare that we need to record subsecond
times. Similarly, if you're dealing with times that are more than an hour, it's probably
not necessary to be more accurate than whole minutes.
Sometimes it's more appropriate to summarize time-on-task data using the
median rather than the mean. The median is the middle point in an ordered list
of all the times: Half of the times are below the median and half are above the
median. Similarly, the geometric mean is potentially less biased than the mean.
Time data are typically skewed, in which case the median or geometric mean
may be more appropriate. In practice, we find that using these other methods of
summarizing time data may change the overall level of the times, but the kinds
of patterns you're interested in (e.g., comparisons across tasks) usually stay the
same; the same tasks still took the longest or shortest times overall.
EXCEL TIP
The median can be calculated in Excel using the = MEDIAN function. The geometric
mean can be calculated using the = GEOMEAN function.
WHAT'S A GEOMETRIC MEAN?
While the mean (or arithmetic average) is based on the sum of a set of numbers, the
geometric mean is based on their product . For example, the mean of 2 and 8 is (2 + 8)/2,
or 10/2, which is 5. The geometric mean of 2 and 8 is sqrt(2*8), or sqrt(16), which is 4.
The geometric mean will usually be smaller than the mean.
RANGES
A variation on calculating average completion time by task is to create ranges, or
discrete time intervals, and report the frequency of users who fall into each time
interval. This is a useful way to visualize the spread of completion times by all
users. In addition, this might be a helpful approach to look for any patterns in
the type of users who fall within certain segments. For example, you may want
to focus on those users who had particularly long completion times to see if they
share any common characteristics.
THRESHOLDS
Another useful way to analyze task time data is by using a threshold. In many sit-
uations, the only thing that matters is whether users can complete certain tasks