Background - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 22

Variance tells you how spread out the data are relative to the average or mean.
The formula for calculating variance measures the difference between each indi-
vidual data point and the mean, squares that value, sums all of those squares,
and then divides the result by the sample size minus 1. For the data in Table 2.1 ,
the variance is 126.4.

Once you know the variance, you can calculate the standard deviation easily,
which is the most commonly used measure of variability. The standard devia-
tion is simply the square root of the variance. The standard deviation of the data
shown in Table 2.1 is 11.2 seconds. Interpreting the standard deviation is a little
easier than interpreting the variance, as the unit of the standard deviation is the
same as the original data (seconds, in this example).

EXCEL TIP: DESCRIPTIVE STATISTICS TOOL
An experienced Excel user might be wondering why we didn't just suggest using the
“Descriptive Statistics” tool in the Excel Data Analysis ToolPak. (You can add the Data
Analysis ToolPak using “Excel Options”>“Add-Ins”.) This tool will calculate the mean,
median, range, standard deviation, variance, and other statistics for any set of data you
specify. It's a very handy tool. However, it has what we consider a significant limita-
tion: the values it calculates are static. If you go back and update the original data, the
statistics don't update. We like to set up our spreadsheets for analyzing the data from a
usability study before we actually collect the data. Then we update the spreadsheet as
we're collecting the data. This means we need to use formulas that update automati-
cally, such as MEAN, MEDIAN, and STDEV, instead of the “Descriptive Statistics” tool.
Butitcanbeausefultoolforcalculatingawholebatchofthesestatisticsatonce.Just
be aware that it won't update if you change the data.

2.3.3 Confidence Intervals
A confidence interval is an estimate of a range of values that includes the true popula-
tion value for a statistic, such as a mean. For example, assume that you need to esti-
mate the true population mean for a task time whose sample times are shown in Table
2.1 . You could construct a confidence interval around that mean to show the range
of values that you are reasonably certain will include the true population mean. The
phrase “reasonably certain” indicates that you will need to choose how certain you
want to be or, put another way, how willing you are to be wrong in your assessment.
This is what's called the confidence level that you choose or, conversely, the alpha level
for the error that you're willing to accept. For example, a confidence level of 95%, or
an alpha level of 5%, means that you want to be 95% certain, or that you're willing to
be wrong 5% of the time.

There are three variables that determine the confidence interval for a mean:

• Thesamplesize,orthenumberofvaluesinthesample.Forthedatain

Table 2.1 , the sample size is 12, as we have data from 12 participants.

• Thestandarddeviationofthesampledata.Forourexample,thatis11.2

seconds.