Methods for Deriving Pesticide Aquatic Life Criteria - Reviews of Environmental Contamination and Toxicology - page 51

2. Hypothesis tests are sensitive to how carefully a test was conducted (i.e., a well-
conducted test typically produces low variability within treatments), and how
many replicates were used. In other words, the minimum detectable significant
difference between treatments decreases with increased replication and with
decreasing variability between replicates. In regression analysis, the point
estimate is not affected by the number of replicates or the reproducibility among
replicates; only the size of the confidence limits is affected.
3. In hypothesis testing, the selection of Α (type I error rate), which is usually
arbitrarily chosen at 0.05, can completely change the resulting NOEC value.
With regression analysis, the confidence limits will change according to Α , but
the point estimate will not change.
4. The effect value obtained from a hypothesis test is completely dependent on what
toxicant concentrations were actually tested. Regression analysis allows for
estimation of a concentration that falls between those actually tested. Consequently,
regression analysis provides a way to predict an effect level for any given concen-
tration, which cannot be done with the results of hypothesis tests.
5. Changes in statistical procedure (such as use of data transformations) can have
large effects on results of hypothesis tests because of the discontinuous nature
of the data. For example, if the results of a hypothesis test are changed by a data
transformation, the change in the resulting effect level will probably be at least
a factor of 2, which is the reciprocal of the typical dilution factor used in toxicity
tests. However, in a regression analysis, the concentration-response curve is
assumed to be a smooth continuous function and results are affected very little
by small changes in statistical procedures.
6. Hypothesis testing does not properly interpret data inversions. That is, if a par-
ticular toxicant concentration caused a significant effect, but a higher concentra-
tion in the same test did not, then interpretation of hypothesis test results is
difficult. The same result, when analyzed by the regression approach, would just
widen the confidence limits of the point estimate.
7. Hypothesis tests require averaging of experimental units across replicates. For
example, if measured concentrations for a particular treatment vary, then the
concentrations must be averaged before the hypothesis test can be conducted.
With regression analysis, each experimental unit can be treated independently.
If concentrations vary within intended replicates, the results can be used without
averaging.
The most important conceptual point made by Stephan and Rogers (1985) is that
hypothesis tests give results that are statistically significant, but have nothing to do
with the biological significance of effects. Hypothesis tests are typically performed
with the Type I error rate ( Α ) defined, but without proper definition of an acceptable
Type II error rate ( B ), and without specifying an acceptable minimum significant
difference. Thus, there is no linkage of statistics to biology. Bruce and Versteeg
(1992) observe another shortcoming of hypothesis testing. Namely, when results
are reported only as an NOEC value, information on the concentration-response
curve and variability in the data is lost.