Fusion in Image Processing - Information Fusion in Signal and Image Processing - page 39

Several images from the same sensor. This consists, for example, of several chan-
nels on the same satellite, or multi-echo images in MRI, or also of image sequences
for scenes in motion. The data in those cases is relatively homogenous because it cor-
responds to similar physical measurements.
Several images from different sensors. This is the most common case, in which
the different physical principles of each sensor allow the user to have complementary
perspectives of the scene. They can consist of ERS and SPOT images, MRI or ultra-
sound images, etc. The heterogenity is then much greater, since the various sensors do
not deal with the same aspects of the phenomenon. Each image gives a partial image
with no information on the characteristics they are not meant to observe (for example,
an anatomical MRI yields no functional information and the resolution of a PET scan
is too low for a precise view of the anatomy).
Several elements of information extracted from a same image. In this situation, dif-
ferent types of information are extracted from an image using several sensors, oper-
ators, classifiers, etc., that rely on different characteristics of the data and attempt to
extract different objects, often leading to very heterogenous elements of information
to fuse. The extracted information can involve the same object (fusion of contour
detectors, for example) or different objects and the goal is then to find an overall inter-
pretation of the scene and consistency between the objects. The elements of informa-
tion can be on different levels (very local, or more structural when studying spatial
relations between objects).
Images and another source of information. By another source of information, we
mean, for example, a model, which may be particular like a map, or generic like an
anatomical atlas, a knowledge base, rules, information provided by experts, etc. The
elements of information are again in very different forms, both in nature and in their
initial representation (images in the case of a map or a digital atlas, but also linguistic
descriptions, databases, etc.).
3.3. Data characteristics in image fusion
The specifics of fusion in image processing make it difficult to take advantage
of the progress made in other fields of information fusion. One of the reasons is the
complexity of the data and knowledge, which make it impossible to attempt to find a
comprehensive system to combine in a single relation all of the image's components.
The complexity is partly due to the volume of information to process (for example,
a single MRI image of the brain takes up 8 to 16 megabytes). These large volumes of
data, on which statistical learning is often possible, are one of the reasons behind the
widespread use of probabilistic and statistical methods in image fusion.