Image Processing Reference

In-Depth Information

mechanics, statistics and information theory. He expressed his research as a prob-

lem involving the specification of probabilities when little information is available.

He examined the two objectivist and subjectivist methods, and chose the latter. By

representing a state of knowledge, this makes it possible to express possible conclu-

sions if not enough information is available to obtain conclusions that are certain. It is

therefore more general, and Jaynes adopted it for statistical mechanics.

The intuitive criteria that Jaynes expects from a measure of uncertainty are the

following:

1) the measure has to be positive and continuous;

2) it has to increase when the uncertainty increases;

3) it has to be additive if the sources are independent.

He thus obtains a unique measure of uncertainty represented by a discrete proba-

bility distribution that corresponds to these intuitive criteria:

n

H
p
1
,...,p
n
=

−

K

p
i
log
p
i
.

[A.9]

i
=1

In the same way Cox rediscovered probabilistic relations from his postulates,

Jaynes thus rediscovered, based on his criteria, the entropy in statistical mechanics and

simultaneously Shannon's entropy [SHA 59].

The maximum entropy principle can then be considered analogous to Laplace's

principle of insufficient reason. The essential difference is that Laplace's principle is

arbitrary in nature and can lead to paradoxes (the concept of equally probable cases

changes if the variable is changed), whereas the maximum entropy principle makes it

possible to make inferences on the basis of partial information in an unbiased fashion.

It can be chosen for the good reason that entropy is determined in a unique way as the

value that “implicates itself the least” with respect to the missing information and not

for the negative reason that there is no reason to think otherwise. However, it can be

criticized because the results it provides depend on how the problem is stated. This

criticism also applies to the principle of insufficient reason as we have seen in section

6.10.

A.3.6.
De Finetti and betting theory

The approach suggested by de Finetti, which predates that of Cox, also relies on

simple and intuitive axioms that lead to the properties of probabilities [FIN 37]. We

have of course the same axioms of increasingness and universal comparison, and most

importantly a consistency axiom which serves as the basis of the demonstration.

Search WWH ::

Custom Search