Introduction to a variety of useful statistical ideas and techniques - Improving the User Experience through Practical Data Analytics - page 11

SIDEBAR: THE NOT-SO-MAGIC NUMBER 30
Statisticians, being on average (pun intended) a conservative lot, suggest that if the sample size is
at least 30, then there is no need for much worry about applying the central limit theorem. In fact,
there is a common belief that it's impossible to obtain meaningful results with a sample size of less
than 30. Indeed, many statistics instructors have used 30 as the “magic number”—the number you
need to obtain any meaningful result about anything.
In truth, for most of the probability distributions/curves that you will encounter in a UX research
setting (say, using data from Likert scales), the curve of the sample mean will converge fairly
quickly to what is very close to a normal-shaped curve, even for sample sizes less than 15. So, even
with a sample size of 10, or even 5, you should apply the central limit theorem and assume that the
probability curve for the mean, X-bar, is bell-shaped, or close enough to it for any practical purpose.
By the way, if anybody insists you need a sample size of 30, ask him/her if n = 29 is OK! After
all, nobody would ever say that you get great normal convergence at n = 30, but not that good at
n = 29. Although we all want larger sample sizes to increase the accuracy of our predictions, there is
really nothing magic about the number 30.
So, since we so often deal with means—as we shall see in subsequent chapters—
the normal distribution/curve is the basis of nearly everything we do in this topic,
even if the connection is not directly visible.
1.3 CONFIDENCE INTERVALS
One very important statistical technique is the calculation of a “conidence interval”
for an unknown quantity. As background to the concept, consider the simple idea of
an “estimate.” We think you'd agree that our best “estimate” is much more useful—
and believable—if accompanied by a measure of the uncertainty in the estimate! This
measure of uncertainty is best (and most often) conveyed by a conidence interval.
Put simply, a conidence interval is an interval, which contains a population value,
such as the population mean, with some speciied probability, usually, 0.95 or 95%.
SIDEBAR:
In the ield of data analysis, we never use the ive-letter “dirty word,” guess . We never guess!! We
estimate!!
In practice, conidence intervals are extremely useful—and even critical—to any UX
researcher. Here's why. You're often dealing with small sample sizes in UX research.
For example, in usability testing, you're typically testing with 5-8 participants. (You
can usually have conidence that you're inding the major usability issues with that
sample size. See Chapter 4 for a deeper dive into this very important assertion.) But
when these same 5-8 participants complete the typical posttest Likert-scale ratings,
you're entering a land mine of potentially weak conidence. Simply reporting a mean