Introduction to a variety of useful statistical ideas and techniques - Improving the User Experience through Practical Data Analytics - page 32

If 4.15 is not enough above 4.10 to be convincing, is 4.20 or 4.30? The answer
depends on the sample size, the variability in the data, and how much chance you
are willing to take when stating that the new design has a higher mean satisfac-
tion rating when in truth, it doesn't! This “risk” has a formal name of “signii-
cance level,” always denoted by “alpha,” and is traditionally chosen to be 0.05-1
chance out of 20.
OK, this is where the hypotheses formally come back into the discussion.
Our goal is to choose between two hypotheses. Their formal names are the null
hypothesis and the alternate hypothesis. Simply put, the null hypothesis typi-
cally states that there is no relationship between one or more things, or that the
status quo has not changed.
Let's assume the latter for now. Put another way, you assume that the status quo is
still the case, until you can prove that something, often a “μ,” has changed. As such, the
null hypothesis provides a benchmark against which we can propose another hypoth-
esis that indeed a change has taken place, or indeed, something is now different: an
“alternate” hypothesis. In our case, our null hypothesis is that the true mean satisfaction
of the new design is no higher than the 4.1 true mean of the current home page design.
Conversely, the alternate hypothesis is that the new design has a higher mean rat-
ing than the 4.1 mean of the current design (and, hence, we should go with the new
design!) The “μ” in the hypotheses below stands for the true mean of the new design. 4
Speciically, we are to choose between:
•

H0: μ ≤ 4.1 (mean satisfaction rating of new design is no higher than 4.1, the
mean of the current design; stay with the current design!)

• H1: μ > 4.1 (mean satisfaction rating of new design is higher than 4.1, the mean

of the current design; go with a smile with the new design!!)

And, you phrase your conclusion by deciding whether

You ACCEPT H0

or

You REJECT H0

From a practical point of view, the two hypotheses should be such that one of
them must be true, but both of them cannot be true . The above two hypotheses satisfy
this condition. So, whatever we conclude about H0, we automatically conclude the
opposite about H1.
So, how are you going to decide between the two hypotheses? First you will
collect some data using the new design, and compute the sample mean satisfaction ,
X-bar. Then, as a function of X-bar, the sample size, n , and the variability in the data
(discussed earlier in the conidence interval section), a conclusion will be reached.

4 We know the true mean of the current design, 4.1. We do not know the true mean of the new design.
We always hypothesize about quantities we don't know—never about quantities we already know!!