Biomedical Engineering Reference
In-Depth Information
null hypothesis is correct) - seems too high a level to set the bar; it
would make it hard to prompt any changes in practice, given the
statistical realities of clinical trials.
Hint
Results that are far from “significant” at the 95% confidence level
may nevertheless form the basis for major scientific breakthroughs.
Every scientist knows that the faintest clues can stimulate highly
productive research. In my opinion, statistically insignificant results
can be very important and should not be ignored. The outlier data
point, the signal that barely peeks above the background, must always
be seriously considered by an investigator; to ignore them is to risk
missing clues from nature. Somewhat arbitrarily, I use the term
“hint” to describe results between the one and one-and-a-half standard
deviation level. Of course, hints occur all the time and, if always
pursued, would leave no time for the central research. Progress
comes from the combination of noticing such hints and formulating
fruitful hypotheses based on the hint
and
a feel for possible
underlying mechanisms.
Trend
The term “trend” is commonly used to describe results that, while
stronger than hints, do not reach the 95% confidence threshold. The
precise meaning of the term is vague; I myself think of a trend as any
result that lies between the one-and-a-half and two standard deviation
level of significance, which is roughly between a P value of 0.15 and
0.05. When a trend is discovered, one is faced with a vexing clinical
problem. Suppose some clinical data indicate that a new therapy
cures twice as many patients as the conventional therapy and the
statistical analysis rejects the null hypothesis with 85% confidence.
This means that there is only a 1 in 7 chance that the new therapy is
the same as the conventional therapy - and the most likely difference
is the measured difference of a factor of two. Can this be ignored?
Many factors go into deciding on the best therapy for a patient.
Certainly the known and, even more so, the unexpected side-effects
of treatment and many other issues may qualify the interpretation
of the central result. There is a good case for a strong degree of
conservatism in clinical practice. It makes sense to require new
therapies, as well as diagnostic techniques and other procedures, to
stand the test of time, and to be only slowly and carefully instituted.
This conservatism lies in part behind the choice of 95% as a threshold
