Information Technology Reference
In-Depth Information
A hierarchical Bayesian analysis avoids the necessity of a prior decision as
to whether or not the trials can be combined; the extent of the combina-
tion is determined purely by the data. This does not come for free; in con-
trast to the meta-analyses discussed above, all the original data (or at least
the sufficient statistics) must be available for inclusion in the hierarchical
model. The Bayesian method is also vulnerable to all the selection bias
issues discussed above.
Guidelines For a Meta-Analysis
•
A detailed research protocol for the meta-analysis should be pre-
pared in advance. Criteria for inclusion and statistical method
employed should be documented in the materials and methods
section of the subsequent report.
•
Meta-analysis should be restricted to randomized controlled trials.
•
Heterogeneity in the trial results should be documented and
explained.
•
Do not attempt to compare treatments investigated in unrelated
trials. (Suppose, by way of a counterexample, that Old were given
as always to low-risk patients in one set of trials, while New was
given to high-risk patients in another.)
•
Individual patient data, rather than published summary statistics,
often are required for meaningful subgroup analyses. This is a
major reason why we favor the modern trend of journals to insist
that all data reported on within their pages be made available by
website to all investigators.
Kepler was able to formulate his laws only because (1) Tycho Brahe had
made over 30 years of precise (for the time) astronomical observations and
(2) Kepler married Brahe's daughter and, thus, gained access to his data.
PERMUTATION TESTS
Permutation tests are often lauded erroneously in the literature as
“assumption-free” “panaceas.” Nothing could be further from the truth.
Permutation tests only yield exact significance levels if the labels on the
observations are weakly exchangeable under the null hypothesis. Thus,
they cannot be successfully applied to the coefficients in a multivariate
regression.
On the other hand, if the observations are weakly exchangeable under
the null hypothesis, then permutation tests are the method of choice for
k
-
sample comparisons, multi-factor experimental designs, and contingency
tables, whenever there are 12 or less observations in each subsample.
Moreover, permutation methods can be used both to test hypotheses and
to obtain interval estimates of parameters.
