Testing Hypotheses: Choosing a Test Statistic - Common Errors in Statistics - page 67

Independent Tests
Normally distributed random variables (as in Figure 7.1) have some
remarkable properties:

• The sum (or difference) of two independent normally distributed

random variables is a normally distributed random variable.

• The square of a normally distributed random variable has the chi-

square distribution (to within a multiplicative constant); the sum
of two variables with the chi-square distribution also has a chi-
square distribution (with additional degrees of freedom).

• A variable with the chi-square distribution can be decomposed

into the sum of several independent chi-square variables.

As a consequence of these properties, the variance of a sum of indepen-
dent normally distributed random variables can be decomposed into the
sum of a series of independent chi-square variables. We use these indepen-
dent variables in the analysis of variance (ANOVA) to construct a series of
independent tests of the model parameters.
Unfortunately, even slight deviations from normality negate these prop-
erties; not only are ANOVA p values in error because they are taken from
the wrong distribution, but they are in error because the various tests are
interdependent.
When constructing a permutation test for multifactor designs, we must
also proceed with great caution for fear that the resulting tests will be
interdependent.
The residuals in a two-way complete experimental design are not
exchangeable even if the design is balanced as they are both correlated
and functions of all the data (Lehmann and D'Abrera, 1988). To
see this, suppose our model is X ijk = m + a i + b j + g ij + e ijk , where
Sa i =Sb j =S i g ij =S j g ij = 0.
Eliminatin g t he ma in ef fe cts in the traditional manner, that is, setting
X ¢ ijk = X ijk -

X

i.. -

X

.j. +

X

... , one obtains the test statistic

( )

  Â

2

I = X ijk ¢

i j k

first derived by Still and White [1981]. A permutation test based on the
statistic I will not be exact because even if th e err or terms {e ijk } are
exchangeable, the residuals X ¢ ijk = e ijk - i.. - .j. + ... are weakly correlated,
with the correlation depending on the subscripts.
Nonetheless, the literature is filled with references to permutation tests
for the two-way and higher-order designs that produce misleading values.
Included in this category are those permutation tests based on the
ranks of the observations that may be found in many statistics software
packages.

e e e