Information Technology Reference
In-Depth Information
Confidence intervals are incredibly useful. We think you should calculate
and display them routinely for just about any means that you report from a
usability study. When displayed as error bars on a graph of means, they make it
visually obvious how accurate the measures actually are.
WHAT CONFIDENCE LEVEL SHOULD YOU USE?
How should you decide what confidence level to use? Traditionally, the three commonly
used confidence levels are 99, 95, and 90% (or their corresponding alpha levels of 1, 5,
and 10%). The history behind the use of these three levels goes back to the days before
computers and calculators, when you had to look up values for confidence levels in
printed tables. The people printing these tables didn't want to print a bunch of different
versions, so they made just these three. Today, of course, all of these calculations are
doneforussowecouldchooseanyconfidencelevelwewant.Butbecauseoftheir
longstanding use, most people choose one of these three.
The level you choose really does depend on how certain you need to be that the
confidence interval contains the true mean. If you're trying to estimate how long it will
take someone to administer a life-saving shock using an automated external defibrillator,
you probably want to be very certain of your answer, and would likely choose at least
99%.Butifyou'resimplyestimatinghowlongitwilltakesomeonetouploadanew
photo to their Facebook page, you probably would be satisfied with a 90% confidence
level. In our day-to-day use of confidence intervals, we find that we use a 90%
confidence level most of the time, sometimes a 95% level, and rarely a 99% level.
2.3.4 Displaying Confidence Intervals as Error Bars
Let's now consider the data in
Figure 2.2
, which shows the checkout times for
two different designs of a prototype website. In this study, 10 participants per-
formed the checkout task using Design A and another 10 participants performed
thecheckouttaskusingDesignB.Participantswereassignedrandomlytoone
group or the other. The means and 90% confidence interval for both groups
have been calculated using the AVERAGE and CONFIDENCE functions. The
means have been plotted as a bar graph, and the confidence intervals are shown
as error bars on the graph. Even just a quick glance at this bar graph shows that
the error bars for these two means don't overlap with each other. When that is
the case, you can safely assume that the two means are significantly different
from each other.
EXCEL TIP
Once you've created a bar graph showing the means, such as
Figure 2.2
, you then want
to add error bars to represent the confidence intervals. First, click on the chart to select
it. Then, in the Excel button bar, choose the “Layout” tab under “Chart Tools.” On the
“Layout” tab, choose “Error Bars>More Error Bars Options.” In the resulting dialog
box, select the “Custom” option near the bottom of the dialog box. Then click on the
Search WWH ::
Custom Search