Information Technology Reference
In-Depth Information
and, since the variation in the probability density
is arbitrary, the term within the
coefficient brackets must vanish. Consequently, we obtain the probability density
δ
p
(
q
)
q
2
p
(
q
)
=
exp
[−
1
−
α
−
β
];
(1.36)
imposing the normalization integral relates the two parameters by
β
π
,
e
−
1
−
α
=
(1.37)
which, when inserted back into (
1.36
), yields
β
π
e
−
β
q
2
p
(
q
)
=
.
(1.38)
The second constraint, that of the second moment, then yields the parameter value
1
β
=
2
,
(1.39)
2
σ
so the distribution which maximizes the entropy subject to a finite variance is normal
with the same parameter values as given in (
1.25
). Using kinetic theory, it is readily
shown that the parameter
is the inverse of the thermodynamic temperature.
Obtaining the normal distribution from the maximum-entropy formalism is a remark-
able result. This distribution, so commonly found in the analyses of physical, social and
life phenomena, can now be traced to the idea that normal statistics are a consequence
of maximum disorder in the universe that is consistent with measurements made in the
experiment under consideration. This is similar to Laplace's notion of unbiased esti-
mates of probabilities. Marquis Pierre-Simon de Laplace (1740-1827) believed that in
the absence of any information to the contrary one should consider alternative possible
outcomes as equally probable, which is what we do when we assert that heads or tails is
equally likely in a fair coin toss. Of course, Laplace did not provide a prescription for the
general probability consistent with a given set of statistically independent experimental
data points. He did, however, offer the probability calculus as the proper foundation for
social science.
But this is not the end of the story. The existence of these multiple derivations of the
normal distribution is reassuring since normal statistics permeates the statistics taught in
every elementary data-processing course on every college campus throughout the world.
The general impression is that the normal distribution is ubiquitous because everyone
says that it is appropriate for the discipline in which they happen to be working. But
what is the empirical evidence for such claims? In Figure
1.1
the distribution of heights
was fit with the normal distribution, that is true. But one swallow does not make a
summer.
Every college student in the United States is introduced to the bell-shaped curve of
Gauss when they hear that the large class in sociology (economics, psychology and, yes,
even physics and biology) in which they have enrolled is graded on a curve. That curve
invariably involves the normal distribution, where, as shown in Figure
1.2
between
β
+
1
and
1 lie the grades of most of the students. This is the C range, between plus and
minus one standard deviation of the class average, which includes 68% of the students.
−
