Learning Iterative Image Reconstruction - Hierarchical Neural Networks for Image Interpretation - page 173

9. Learning Iterative Image Reconstruction
Successful image reconstruction requires the recognition of a scene and the gen-
eration of a clean image of that scene. In this chapter, I show how to use Neural
Abstraction Pyramid networks for both analysis and synthesis of images. The net-
works have a hierarchical architecture which represents images in multiple scales
with different degrees of abstraction. The mapping between these representations is
mediated by a local recurrent connection structure.
Degraded images are shown to the networks which are trained to reconstruct the
originals iteratively. Through iterative reconstruction, partial results provide context
information that eliminates ambiguities.
The performance of this approach is demonstrated in this chapter by applying it
to four tasks: super-resolution, filling-in of occluded parts, noise removal / contrast
enhancement, and reconstruction from sequences of degraded images.
9.1 Introduction to Image Reconstruction
Frequently, digital images of real-world scenes do not have enough quality for the
application at hand since the images have been degraded in some way. These degra-
dations arise in the image formation process (e.g. from occlusions) and from the
capturing device (e.g. due to low resolution and sensor noise).
The goal of the image reconstruction process is to improve the quality of the cap-
tured images, e.g. by suppressing the noise. To separate noise from objects, models
of the noise and the objects present in the images are needed. A scene can then be
recognized, and a clean image of it can be generated.
Freeman and Pasztor [72] recently proposed a learning approach to low-level
vision that they termed VISTA. It models images and scenes using Markov random
fields. The parameters of their graphical models can be trained, e.g. for a super-
resolution task. The demonstrated performance of the system is impressive. How-
ever, the models have no hidden variables, and the inference via belief propagation
is only approximate.
A common problem with image reconstruction is that it is difficult to decide the
right interpretation of an image part locally. For example, it might be impossible to
decide in a digit binarization task whether or not a pixel belongs to the foreground
by looking only at the pixel's intensity. If contrast is low and noise is present, it