Information Technology Reference
In-Depth Information
a consistent set of rules for reasoning about beliefs. First, he had to decide how
to rank degrees of belief, such as whether the coin we talked about earlier was
a fair coin with a 50 percent probability of coming up heads or a biased coin
with an 80 percent probability of coming up heads. He proposed ranking how
much we believe these possibilities by assigning a real number to each proposi-
tion such that the larger the number, the more we believe the proposition. He
put forward two
axioms
(established rules) as being necessary for logical consis-
tency. The first was that if we specify how much we believe something is true,
we are also implicitly specifying how much we believe it is false. Using a scale
of real numbers from 0 to 1 to specify beliefs, this axiom says that the belief
that something is true plus the belief that the same thing is false must add up
to one. This is the same as the usual
sum rule
for probabilities, which holds that
the probabilities for all possible outcomes must add up to one.
Cox's second axiom is more complicated. If we specify how much we
believe proposition Y is true, and then state how much we believe proposi-
tion X is true given that Y is true, then we must implicitly have specified how
much we believe that both X and Y are true. Assuming some initial background
information that we denote by B, we can write this belief relationship as an
equation as follows:
Prob (X and Y | B) = Prob (X | Y and B) × Prob (Y | B)
In words, this equation says that the probability that both X and Y are true,
given background information B, is equal to the probability that X is true given
that Y and B are true, times the probability that Y is true given B, regardless of
proposition X. The vertical bar “|” separates the different propositions in these
probabilities. This equation is the usual
product rule
for probabilities, which
states that the probability of two independent events occurring simultaneously
is the result of multiplying the individual probabilities together. The product
rule is easy to derive from a frequentist approach. Note that all probabilities are
conditional on the same background information B.
We can now derive the mathematical formula representing Bayes' Rule for
probabilities. It is obvious that the probability that X and Y are both true does
not depend on the ordering of X and Y on the left-hand side of our equation.
We therefore have:
Prob (X and Y | B) = Prob (Y and X | B)
By expanding each side and doing a little rearrangement, we arrive at
Bayes rule:
Prob (X | Y and B) = Prob (Y | X and B) × Prob (X | B) / Prob (Y | B)
Put into words, Bayes' Rule states that the probability of your initial estimate
X, given the original data B and some new evidence Y, is proportional to the
probability of the new evidence, given the original data B and the assumption
X, and to the probability of the estimate X, based only on the original data B.
For example, the probability of drawing an ace from a deck of cards is 0.077
Search WWH ::
Custom Search