Information Technology Reference
In-Depth Information
Chapter 5
Testing Hypotheses:
Choosing a Test Statistic
“Forget 'large-sample' methods. In the real world of experiments
samples are so nearly always 'small' that it is not worth making
any distinction, and small-sample methods are no harder to
apply.” George Dyke [1997].
E
VERY STATISTICAL PROCEDURE RELIES ON CERTAIN ASSUMPTIONS FOR
correctness. Errors in testing hypotheses come about either because the
assumptions underlying the chosen test are not satisfied or because the
chosen test is less powerful than other competing procedures. We shall
study each of these lapses in turn.
First, virtually all statistical procedures rely on the assumption that the
observations are independent.
Second, virtually all statistical procedures require at least one of the
following successively weaker assumptions be satisfied under the null
hypothesis:
1. The observations are identically distributed.
2. The observations are exchangeable; that is, their joint distribution
is the same for any relabeling.
3. The observations are drawn from populations in which a specific
parameter is the same across the populations.
The first assumption is the strongest assumption. If it is true, the follow-
ing two assumptions are also true. The first assumption must be true for a
parametric test to provide an exact significance level. If the second assump-
tion is true, the third assumption is also true. The second assumption must
be true for a permutation test to provide an exact significance level.
The third assumption is the weakest assumption. It must be true for a
bootstrap test to provide an exact significance level asymptotcally.
