Introduction to a variety of useful statistical ideas and techniques - Improving the User Experience through Practical Data Analytics - page 12

from a typical post-usability test Likert-scale with a sample size of 5-8 without a con-
idence interval can be viewed as unprofessional at best and extremely misleading at
worst. The typical small sample sizes often yield wide conidence intervals (we discuss
this further later in this section), and we'd argue that it's your professional duty to report
them. Without them, you're just not telling the full story.

SIDEBAR: PROBABILITY VERSUS STATISTICS
The topic of conidence intervals is often considered a “sub heading” of what we call “interval
estimation.” In turn, interval estimation is one of two topics under a major heading of what we call
“statistical inference.” This refers to the basic idea of being able to infer what we can about the true
mean (or other true quantity, such as the true standard deviation), based on the data.
When we predict what is likely to happen if we assume a known value for the mean (and
perhaps the standard deviation), indeed, this is the ield of probability —for example, when we
assess the probability that X comes out above 160, or below 180, etc. When we “turn it around,”
and assume the true (population) values are the unknowns (which is the heart of data analysis and
predictive analytics), and use data to make inference about the true values, we enter the world of
statistics .

SIDEBAR: POINT ESTIMATES AND THE PITFALLS OF USING THEM
ALONE
A point estimate is just a fancy name for a single value as an estimate; for estimating the true
mean, μ, the best point estimate (based on various criteria to deine the word, “best”) is the
sample mean, X-bar. There are times when you need to go forward with your best estimate (i.e.,
a single value), even though you know it is not the exact true value. However, most of the time,
you should not stop with knowing only the sample mean. Using only a point estimate is rarely the
best thing to do.
After all, it would be like the consultant (or “best qualiied statistics person” at the company)
saying to Bill, the boss, “OK, Bill, our best estimate of the mean time it takes to complete the
task is 37.5 seconds, but, of course, even though it's our best estimate, it's not the true mean!
Have a nice day!”
Not wanting to subscribe to such silliness is why we need to determine not only a point estimate
(remember—for us, a fancy word for X-bar), but also an interval estimate, which is a conidence
interval. This usually amounts to providing a range of values (an interval) that has a probability of
95% of containing the true population mean, μ.

1.3.1 THE LOGIC AND MEANING OF A CONFIDENCE INTERVAL
To determine the conidence interval, we start by reporting the sample mean, X-bar,
but we also report an interval:

X−e to X+e

where “e” can be (for the moment) called “error” and

( )

P X−e<μ<X+e =1−α .