Machine learning and natural language processing - The Computing Universe: A Journey through a Revolution - page 280

14 Machine learning and natural
language processing
… one naturally wonders if the problem of
translation could conceivably be treated as a
problem in cryptography. When I look at an
article in Russian, I say “This is really written in
English, but it has been coded in some strange
symbols.”
Warren Weaver 1
Ideas of probability: The frequentists and the
Bayesians
We are all familiar with the idea that a fair coin has an equal chance of
coming down as heads or tails when tossed. Mathematicians say that the coin
has a probability of 0.5 to be heads and 0.5 to be tails. Because heads or tails
are the only possible outcomes, the probability for either heads or tails must
add up to one. A coin toss is an example of physical probability , probability that
occurs in a physical process, such as rolling a pair of dice or the decay of a radio-
active atom. Physical probability means that in such systems, any given event,
such as the dice landing on snake eyes, tends to occur at a persistent rate or
relative frequency in a long run of trials. We are also familiar with the idea of
probabilities as a result of repeated experiments or measurements. When we
make repeated measurements of some quantity, we do not get the same answer
each time because there may be small random errors for each measurement.
Given a set of measurements, classical or frequentist statisticians have developed
a powerful collection of statistical tools to estimate the most probable value of
the variable and to give an indication of its likely error.
An alternative view of probability reflects the strength of our belief that the
coin is fair and not what statisticians call biased , tending to give one result more
frequently than the other. For example, perhaps we have reason to think that
the coin being fair, with a 50 percent probability of coming up heads, is only
one possibility. Maybe we think there is an equal chance that the coin is biased
and will come up as heads 80 percent of the time. Before tossing the coin, we
suppose that each of these two options is equally likely. But after tossing the
coin ten times and observing eight heads, we will want to modify our beliefs. It
now makes sense for us to believe that there is a greater than fifty-fifty chance
280