Learning Iterative Image Reconstruction - Hierarchical Neural Networks for Image Interpretation - page 174

could be necessary to bias this decision with the output of a line-detector for that
location.
In general, to eliminate such local ambiguities, more context is needed. Feed-
forward models that cover such large context have many free parameters. They are
therefore expensive to compute and difficult to train.
Here, I propose to iteratively transform the image into a hierarchical representa-
tion. The image is interpreted first at locations where little ambiguity exists. These
partial results are used as context to bias the interpretation of more ambiguous re-
gions. The reconstruction problem is described using examples of degraded images
and desired output images, and then a recurrent neural network of suitable structure
is trained to solve the problem.
In order to investigate the performance of the proposed approach for iterative
image reconstruction, a series of experiments was conducted with images of hand-
written digits. The reasons for choosing digits were that large datasets are publicly
available, and that the images contain multiscale structure which can be exploited
by the learning algorithm. Clearly, if there were no structure to learn, training would
not help. The digits were degraded by subsampling, occluding parts, or adding noise,
and Neural Abstraction Pyramid networks were trained to reconstruct the originals.

9.2 Super-Resolution

Super-resolution is the process of inferring high-resolution detail from low-resolu-
tion images. It is a typical image reconstruction problem that has been investigated
by many researchers since it is needed for applications, such as enlarging consumer
photographs or converting regular TV signals to HDTV. Different complementary
approaches exist to increase the perceptual resolution of an image, as illustrated in
Figure 9.1.
The first idea is to sharpen the images, by amplifying existing high-frequency
image content. This is a dangerous operation since noise will be amplified as well.
The next approach is to fuse multiple low-resolution images that have been cap-
tured at slightly different positions. Fusion is based on the constraint that the super-
resolution image, when appropriately warped and down-sampled (to model the im-
age formation process), should yield the low-resolution inputs. This is feasible if

spatial frequency
Fig. 9.1. Complementary approaches to increase the perceptual resolution of an image: (a)
amplifying existing high frequencies; (b) combining multiple displaced low-resolution im-
ages; (c) estimating image details.

(a)

spatial frequency

(b) (c)