Experimental Design in Aquatic Ecology - Freshwater Ecology: Concepts and Environmental Applications - page 482

TABLE A.2 Description of Common Statistical Methods

Method Description Comments

t test Comparison of one mean and a

number or of two means

Regression Fits the best line to the data Can be linear or nonlinear, with one or

(minimizes variance) more independent variables

Correlation Fits to a simple linear relationship Does not denote causation; normally used

for data exploration

Analysis of variance Tests the effects of multiple treatments Commonly used in replicated

(ANOVA) manipulative experiments

Chi-square Compares expected versus observed

occurrences in categories

Meta-analysis Compares results from many studies Arnqvist and Wooster (1995);

Gurevitch and Hedges (1993)

These tests result in a probability value that can be used to express cer-
tainty. The convention is that a result is not significant until the investiga-
tor is 95% certain. However, this is purely an arbitrary value. When hu-
man lives are at stake, 95% is probably not good enough. When publication
of an ecological paper is at stake, 95% is probably sufficient.
Experimental design is presented here in a simplistic fashion. Replica-
tion is often a problem for the environmental scientist. On the one hand,
many do not understand replication; on the other hand, some experiments
cannot be replicated as discussed previously.
When scientists do not understand replication correctly, pseudorepli-
cation can be a problem because it can lead to undue faith in results.
Pseudoreplication occurs when “replicate” samples are taken from one
treatment (Hurlbert, 1984). For example, if we are testing the effects of ni-
trogen on water quality in two watersheds, and fertilize one watershed
with nitrogen and leave the second untreated, it is not appropriate to take
replicate water samples from each watershed and apply experimental sta-
tistics to them. We may be able to strongly infer results, but assigning sta-
tistical significance to the results is incorrect.

SUMMARY

There are several general types of experiments: natural experiments,
simulation models, and manipulative experiments. Each has its own
strengths and weaknesses. Natural experiments may be the most realistic
but often are not replicated, and assigning statistical certainty to results is
difficult. Simulation models may offer a way to approach intractably large
or complex systems and provide insights into key factors in these systems.
Manipulative experiments can be replicated and the results subject to sta-
tistical determination of certainty of outcome. However, such experiments
require replication, and this may lead to small-scale treatments or artificial
conditions that make relevance of the results to the system of interest ques-
tionable. All of the methods are useful to aquatic ecologists. The trick is in
asking the important questions and determining the best method to use to
obtain satisfactory answers.