Information Technology Reference
In-Depth Information
TABLE 5.1 Types of Statistical Tests of Hypotheses
Test Type
Definition
Example
Exact
Stated significance level is exact,
t
test when observations
not approximate.
are i.i.d. normal;
permutation test when
observations are
exchangeable.
Parametric
Obtains cutoff points from
t
test
specific parametric
distribution.
Nonparametric
Obtains cutoff points from
bootstrap
percentiles of bootstrap
distribution of parameter.
Parametric
Obtains cutoff points from
bootstrap
percentiles of parameterized
bootstrap distribution of
parameter.
Permutation
Obtains cutoff points from
Tests may be based upon
distribution of test statistic
the original observations,
obtained by rearranging
on ranks, on normal or
labels.
Savage scores, or on
U
statistics.
An immediate consequence of the first two assumptions is that if obser-
vations come from a multiparameter distribution, then all parameters, not
just the one under test, must be the same for all observations under the
null hypothesis. For example, a
t
test comparing the means of two popula-
tions requires that the variation of the two populations be the same.
For nonparametric and parametric bootstrap tests, under the null
hypothesis, the observations must all come from a distribution of a specific
form.
Let us now explore the implications of these assumptions in a variety of
practical testing situations including comparing the means of two popula-
tions, comparing the variances of two populations, comparing the means
of three or more populations, and testing for significance in two-factor
and higher-order experimental designs.
In each instance, before we choose
1
a statistic, we check which assump-
tions are satisfied, which procedures are most robust to violation of these
assumptions, and which are most powerful for a given significance level
and sample size. To find the most powerful test, we determine which pro-
cedure requires the smallest sample size for given levels of Type I and
Type II error.
Whether Republican or Democrat, Liberal or Conservative, male or female, we have the
right to choose and need not be limited by what textbook, half-remembered teacher pro-
nouncements, or software dictate.
1
