Information Technology Reference
In-Depth Information
recommended for use in determining the precision of percentiles in the
tails (
P
20
,
P
10
,
P
90
, and so forth).
p
VALUES
Before interpreting and commenting on
p
values, it's well to remember
that in contrast to the significance level, the
p
value is a random variable
that varies from sample to sample. There may be highly significant differ-
ences between two populations and yet the samples taken from those pop-
ulations and the resulting
p
value may not reveal that difference.
Consequently, it is not appropriate for us to compare the
p
values from
two distinct experiments, or from tests on two variables measured in the
same experiment, and declare that one is more significant than the other.
If in advance of examining the data we agree that we will reject the
hypothesis if the
p
value is less than 5%, then our significance level is 5%.
Whether our
p
value proves to be 4.9% or 1% or 0.001%, we will come
to the same conclusion. One set of results is not more significant than
another; it is only that the difference we uncovered was measurably more
extreme in one set of samples than in another.
p
values need not reflect the strength of a relationship. Duggan and
Dean [1968] reviewed 45 articles that had appeared in sociology journals
between 1955 and 1965 in which the chi-square statistic and distribution
had been employed in the analysis of 3 ¥ 3 contingency tables and com-
pared the resulting
p
values with association as measured by Goodman and
Kruskal's gamma. Table 7.1 summarizes their findings.
p
values derived from tables are often crude approximations, particularly
for small samples and tests based on a specific distribution. They and the
stated significance level of our test may well be in error.
The vast majority of
p
values produced by parametric tests based on the
normal distribution are approximations. If the data are “almost” normal,
the associated
p
values will be almost correct. As noted in Chapter 6, the
stated significance values for Student's
t
are very close to exact. Of course
a stated
p
value of 4.9% might really prove to be 5.1% in practice. The sig-
nificance values associated with the
F
statistic can be completely inaccurate
for non-normal data (1% rather than 10%). And the
p
values derived from
TABLE 7.1 p-Value and Association
p-value
gamma
<
.30
.30-.70
>
.70
<
.01
8
11
5
.05
7
0
0
>
.10
8
0
0
