Image Processing Reference
In-Depth Information
The approach in principal component analysis, which projects each source of infor-
mation on the eigenvectors of the covariance matrix, is often used, in order to obtain
l new decorrelated sources, arranged in decreasing order of energy. Truncating the set
to keep only the first l ( l>l ) sources can often be done while preserving most of the
original set's energy.
However, in practice, this method quickly shows its limitations in image process-
ing, for example, because it cannot take into account complex dependences between
images or the spatial variations of dependences.
In order to express the information contributed by adding a new source of infor-
mation I l +1
, the preferred approach is that sug-
gested by Shannon, which relies on the concepts of information and entropy [KUL 59,
MAI 96]. Based on the joint probability of the first l sources p ( I 1 ,...,I l ) (estimated
most of the time by using frequencies of occurrence, for example, based on the multi-
dimensional histogram of gray levels in an image), the entropy (or mean information
per pixel in the case of images) of the first l sources is defined by:
H I 1 ,...,I l =
to an already known set
I 1 ,...,I l }
p I 1 ,...,I l log p I 1 ,...,I l ,
and the entropy contributed by the ( l +1) th source is expressed, either depending on
the entropies, or depending on the probabilities, as:
H I l +1 |
I 1 ,...I l = H I 1 ,...,I l +1
H I 1 ,...,I l
p I 1 ,...I l +1 log p I l +1 |
I 1 ,...,I l .
For two sources, we thus define redundancy 1 between them as:
R ( I 1 ,I 2 )= H ( I 1 )+ H ( I 2 )
H ( I 1 ,I 2 ) ,
and the complementarity of the source I 2 with respect to I 1 , i.e. the mean quantity of
information that has to be added to I 2 in order to have I 1 :
C I 1 |
I 2 = H I 1 |
I 2 ,
which leads us to the following relation:
H I 1 = R I 1 ,I 2 + C I 1 |
I 2 .
Analogous methods could be considered in a non-probabilistic framework, by rely-
ing, for example, on fuzzy entropy [LUC 72]. For the moment, the formalism is less
well developed in this direction.
1. This redundancy unfortunately cannot be generalized to more than two sources without
potentially losing its positivity property.
Search WWH ::

Custom Search