Biomedical Engineering Reference
In-Depth Information
confidence level, as has already been mentioned. Nevertheless, I do
not think that doctors can ignore trends in advising their patients
about treatments. I address this point in the section below entitled
“Randomized Clinical Trials.”
A trend becomes more compelling when there is a mechanistic reason
to believe it. We often undervalue our understanding of biology. If,
for example, there were a widely obeyed dose-effect relationship, and
a new technique permitted higher doses to be administered without
what was judged to be an appreciable likelihood of additional
morbidity, a trend in the results that favored the new technique might
well be sufficient to allow its adoption.
One should not forget that trends are also heavy hints.
Conviction
Even a well-designed experiment with a result having statistically
clear and unambiguous “significance”, say at the P<0.005 (three
standard deviation) level or more, may be wrong or misleading.
Every scientist can tell stories of experiments that gave statistically
impeccable results, but that were nevertheless not reproducible.
Human error, systematic bias, faulty assumptions, an unrepresentative
patient population, multiple comparisons, any one of these can
overwhelm the statistics of a clinical trial.
2
2
I learnt this lesson very early in my professional career. While still a
graduate student engaged in an experiment in elementary particle
physics, another group in my laboratory made measurements to test the
very successful theory of quantum electrodynamics. They observed an
apparent violation of the theory and, as soon as their finding became
known, some theoreticians proposed that their experiment could be
explained if a hitherto unsuspected heavy electron existed. My group
was in a position to test this hypothesis immediately, so we were given
top priority to use the facility to look for such a high mass electron. To
do this we looked at the energy spectrum of particles scattered off a
liquid hydrogen target. If there were an excited electron, it would
appear as a peak in the energy distribution. And, sure enough, on the
very first night, we saw such a peak - standing well above the
background. We were of course, thrilled and began to discuss what to
name the new particle - and the proper attire to wear in Stockholm.
Cooler heads prevailed and we repeated the experiment under different
conditions - and the peak disappeared, never to be seen again no matter
how hard we tried or under what conditions (including the original one)
we measured. Almost certainly our “statistically significant” peak was
[continued on next page
