Biomedical Engineering Reference
In-Depth Information
females will. In general, statistical tests are designed to test
the null hypothesis that the independent variable did not
produce the changes predicted in the dependent variable.
Since statistical tests state only the probability that the
null hypothesis is true, the question arises of what level of
confidence (the reciprocal of the probability that the null
hypothesis is true) is required to reject the null hypothesis
and conclude that the data do not indicate the predicted
relationship between the independent and dependent vari-
ables. By convention, many people work at the 95%
confidence limit, rejecting the null hypothesis if there is
a less than 95% chance that it is false. However, there is no
definitive limit, and confidence limits should be set after
evaluating the consequences of failing to reject the null
hypothesis or falsely rejecting the null hypothesis. For
example, few people would fly if the chance of arriving at
their destination safely was only 19 in 20.
Once the confidence limit has been selected, it
becomes a decision point. If a 95% confidence level must
be reached to reject the null hypothesis, then the null
hypothesis is actually rejected only if a statistical test
indicates that the probability of obtaining a given set of
data by chance was less than 0.05 is. Probabilities are not
correlations, and a probability value of 0.001 does not
mean that the influence of the independent variable on the
dependent variable is “stronger” than if the probability
value were 0.05. If the influences of the independent
variable are constant, the p value of a test decreases simply
as a function of sample size. Thus, a test result of 7000 out
of sample of 10 000 would produce a p value much less
than a test result of 7 out of 10, even though the bias is still
70% for both.
Similarly, if 0.05 is the maximum probability value for
rejecting the null hypothesis, then a p value of 0.51 does not
meet the preselected criteria and the null hypothesis cannot
be rejected. Claiming that the value found “almost made it”
does not change the fact that the probability value is
nonsignificant. Similarly, since probabilities are not corre-
lations, there is no “trend” toward significance; the decision
point is an arbitrary one. Thus, observing two heads and
one tail in three coin tosses does not mean that one will
observe 200 out of 300 heads if one continues to toss the
same coin.
To return to our imagined study of chasing behavior in
male and female primates, if the difference between the
amount of chasing in two groups is greater than could be
attributed to chance (random variation in measures,
sampling of subjects, etc.), that result may still be for
reasons other than that males are inherently more aggres-
sive than females are. Perhaps older animals are more
aggressive than younger ones and all the males in the study
were older than the females were. In this example, age is
a confounding variable that may account for obtaining the
data that were predicted.
The hypothesis may have predicted correctly but for the
wrong reason. Controlling the selection of subjects to rule
out the possibility that results were due to age differences
can be done in two ways. One way is to fix age so that all
subjects in both groups are the same age. The other tech-
nique is to randomize age so that subjects for the two
groups are chosen randomly from a population containing
all ages. Randomizing assumes that the groups will not be
accidentally biased by age. Fixing age may provide a level
of comfort, but the data and conclusions may be limited to
a subset of males and females of a particular age. A good
compromise may be matching ages in the male and female
groups so that each subject in one group is matched with
a subject in the other group according to all variables except
the one of interest (in this case, sex). Certain unavoidable
confounding variables may persist; for example, males may
by virtue of their sex be larger than females are or have
different hormonal levels. However, these confounds are so
intimately related to sex that one might argue that male
versus female implies differences in hormones and size and
that these variables are part of the sex variable and therefore
need not be controlled.
Controlling for age might lead colleagues to ask
whether subjects' health, time in the laboratory, birth order,
birth month, or even time of day of birth should be
controlled. Indeed, the number of alternative confounding
variables is limited only by the imagination. Investigators
need not control for all possible confounds but only for
those at least as likely as the independent variable of
interest to influence the outcome of interest. However, this
raises the question of what alternative explanations are as
plausible as the hypothesis being tested. Identifying these is
a subjective decision, and it is plausible that a valid alter-
native may be rejected as totally implausible in one labo-
ratory but be the basis of a large research effort in another.
No hypothesis can ever be truly proven. Evidence is
amassed by repeatedly making and verifying predictions. If
the theory makes more accurate predictions than any
competing theory, it may be extremely useful, but it would
be virtually impossible to definitively prove it is true.
Similarly, failure to demonstrate that male primates chase
more than do females does not prove the null hypothesis.
Perhaps the theory is correct but chasing (the measure of
the dependent variable) was poorly measured. Perhaps sex
was misidentified in some of the subjects. Any experiment
done badly enough can fail to reject the null hypothesis.
Failure to reject the null hypothesis, or negative
evidence, can be useful if the hypothesis is restated to say
that the bias is less than 90% (or any preselected number)
and the null hypothesis, therefore, is that the bias is greater
than or equal to 90%. If the data do not support the null of
a bias greater than or equal to 90%, that hypothesis is
rejected in favor of the hypothesis that the bias is less than
90%. This kind of formulation never proves that there is no
