Information Technology Reference
In-Depth Information
the log-likelihoods of two competing models (i.e., the ratio of their
maximum likelihoods).”
The minimum Bayes factor does not involve a specific prior probability
distribution, rather, it is a global minimum over all prior distributions.
Bayarri and Berger [1998] and Berger and Sellke [1987]] provide a
simple formula for the minimum Bayes factor in the situation where the
prior probability distribution is symmetric and descending around the
null value. This is -exp p ln( p ), where p is the fixed-sample-size p
value.
As Goodman [2001] notes, “even the strongest evidence against the
null hypothesis does not lower its odds as much as the p -value magnitude
might lead people to believe. More importantly, the minimum Bayes
factor makes it clear that we cannot estimate the credibility of the null
hypothesis without considering evidence outside the study.”
For example, while a p value of 0.01 is usually termed “highly signifi-
cant,” it actually represents evidence for the primary hypothesis of some-
where between 1/25 and 1/8. 29 Put another way, the relative odds of the
primary hypothesis versus any alternative given a p value of 0.01 are at
most 8-25 times lower than they were before the study. If one is going to
claim that a hypothesis is highly unlikely (e.g., less than 5%), one must
already have evidence outside the study that the prior probability of the
hypothesis is no greater than 60%. Conversely, even weak evidence in
support of a highly plausible relationship may be enough for an author to
make a convincing case.
Two Caveats
1. Bayesian methods cannot be used in support of after-the-fact-
hypotheses because, by definition, an after-the-fact hypothesis has
zero a priori probability and, thus, by Bayes' rule, zero a posteriori
probability.
2. One hypothesis proving of greater predictive value than another in
a given instance may be suggestive but is far from definitive in the
absence of collateral evidence and proof of causal mechanisms. See,
for example, Hodges [1987].
When Using Bayesian Methods
Do not use an arbitrary prior.
Never report a p value.
Incorporate potential losses in the decision.
Report the Bayes factor.
29
See Table B.1, Goodman [2001].
Search WWH ::




Custom Search