Image Analysis for Automatically-Driven Bionic Eye (Bioengineering in Neurological Disorders) Part 4

Focus of attention

As a matter of fact, the role played by cones in diurnal vision is preponderant. Cones are much less numerous than rods in most parts of the retina, but greatly outnumber rods in the fovea. Furthermore cones are arranged in a concentric way inside the human retina [Marr (1982)]. In this way focus of attention may be modelized by representing cones in the fovea area and its surroundings. The general principle is the following. Firstly a focusing point is chosen as the fovea center (gaze center) and a foveal radius is defined as the radius of the central cell. Secondly an isotropic progression of concentric circles determines the blurring factor according to the distance to the focusing point. Thirdly integration sets are defined to represent cones and an integration method is selected in order to gather data over the integration set to obtain a single value. Integration methods can be chosen amongst averaging, median filtering, morphological filtering such as dilation, erosion, closing, opening, and so on. Then re-sampled data are stored in a rectangular image in polar coordinates. This gives the encoded image. This image is a compressed version of the original image, but the compression ratio varies according to the distance to the focusing point. The following step can be the reconstruction of the image from the encoded image. This step is not systematically achieved as there is no need of duplicating data to process them [Robert-Inacio (2010)]. When necessary it works by determining for each point of the reconstructed image the integration sets it belongs to. Then the dual method of the integration process is used to obtain the reconstructed value. When using directly the encoded image instead of the original or the reconstructed images, customized processing algorithms must be set up in order to take into account that data are arranged in a polar way. In this case a full pavement of the image is defined with hexagonal cells [Robert (1999)]. The hexagons are chosen so that they do not overlap each others and so that they are as regular as possible. A radius sequence is also defined as follows:


This hexagonal pavement is as close as possible to the biological cone distribution in the fovea. Furthermore data are taken into account only once in the encoded image because of non-overlapping.

Fig. 21 illustrates the type of results provided by previous methods on an image of the Kodak database12 (Fig. 21). Firstly Fig. 21 shows the encoded image (on the right) for a foveal radius of 25 pixels and with hexagonal cells. The focusing point is chosen at (414,228), ie: at the central flower heart. Secondly the reconstructed image is given after re-sampling of the original image. In the following, the hexagonal pavement is chosen to define foveated images as it is the closest one to the cone distribution in the fovea.

Detection of points of interest

The detection of points of interest is achieved by using the Harris detector [Harris (1988)]. Fig. 22 shows the images with the detected points of interest. Points of interest detected as corners are highlighted in red whereas those detected as edges are in green. Fig. 22 illustrates the Harris method when using a regular image (a), in other words, an image sampled in a rectangular way, and a foveated image (b).

Hexagonal cell distribution.

Fig. 20. Hexagonal cell distribution.

Sequence of points of interest

Short sequences of points of interest are studied: the first one has been computed and the second one is the result observed on a set of 7 people. Fig. 23 shows the sequences of points of interest on the original image of Fig. 21a and Table 1 gives the point coordinates. Sequences are made of points numbered from 1 to 4. The observer sequence in white goes from the pink flower heart to the bottom left plant, whereas the computed sequence in cyan goes from the pink flower heart to the end of the branch. Another difference concerns the point in the red flower. The observers chose to look at the flower heart whereas the detector focused at the border between the petal and the leaf. This is explained by the visual cortex behavior. Actually the detector is attracted by color differences whereas the human visual system is also sensitive to geometrical features such as symmetry. In this case the petals around the heart are quite arranged in a symmetrical way aroud the flower heart. That is why the observers chose to gaze at this point. In this example the computed sequence is determined without computing again a new foveated image for each point of interest, but by considering each significant point from the foveated image with the central point as focusing point. Furthermore for equivalent points of interest the distance between two consecutive points is chosen as great as possible in order to cover a maximal area of the scene with a minimal number of eye movements.

Table 1 gives the distance between two equivalent points from the two sequences. This distance varies from 8 to 32.249 with an average value of 18.214. This means that computed points are not so far from those of the observers. But the algorithm determining the sequence must be refined in order to prevent errors on point order.

 Focus of attention on a particular image: from the original image to the reconstructed image, passing by the encoded image (foveated image)

Fig. 21. Focus of attention on a particular image: from the original image to the reconstructed image, passing by the encoded image (foveated image)

Point

Regular

Point

Foveated

Number

Detection

Number

Detection

Distance

1

(191,106)

1

(194,114)

8.544

2

(279,196)

2

(275,164)

32.249

3

(99,118)

4

(109,103)

18.028

4

(24,214)

3

(38,215)

14.036

Table 1. Distances between points of interest.

Applications

There exist two great families of applications: on the one hand, applications in the biomedical and health field, and on the other hand, applications in robotics.

In the biomedical field, a system such as the bionic eye can be very helpful at different tasks:

• light perception,

• color perception,

Detection of points of interest

Fig. 22. Detection of points of interest

Sequences of points of interest

Fig. 23. Sequences of points of interest

• contextual environment perception,

• reading,

• pattern recognition,

• face recognition,

• autonomous moving,

• etc.

These different tasks are achieved very easily for sighted people, but they can be impossible for visually impaired people. For example, color perception cannot be operated by touch, by hearing, by the taste or smell. It is a pure visual sensation, unreachable to blind people. That is why the bionic eye must be able to replace the human visual system for such tasks.

In robotics, such a system able to explore an unknown scene by itself can be of a great help for autonomous robots. For example AUV (Autonomous Underwater Vehicles) can be even more autonomous by being able to decide by themselves what path to follow. Actually, by mimicking detection of points of interest, the bionic eye can determine obstacle position and then it can compute a path avoiding them. Furthermore, application fields are numerous:

• in archaeology and exploration in environments inaccessible to humans,

• in environmental protection and monitoring,

• in ship hull and infrastructure inspection,

• in infrastructure inspection of nuclear power plants,

• in military applications,

• etc.

Each time it is impossible for humans to reach a place, the bionic eye can be used to make decision or help making decision in order to drive a robot.

Conclusion

In this topic, the bionic eye principle has been presented in order to demonstrate how powerful such a system is. Different approaches can be considered to stimulate either the retina or the primary visual cortex, but all the presented systems use a separate system for image acquisition. Images are then processed and data are turned into electrical pulses stimulating either retinal cells or cortical neurons.

The originality of our system lays in the fact that images are not only processed but analyzed in order to determine a sequence of focusing points. This sequence allows to explore automatically a complex scene. This principle is directly inspired by the human visual system behavior. Furthermore foveated images are used instead of classical images (sampled at a constant step in two orthogonal directions). In this way, every image processing algorithm even basic has to be redefined to fit to foveated images.

In particular, an algorithm for detection of points of interest on foveated images has been set up in order to determine sequences of points of interest. These sequences are compared to those obtained from a human observer by eye-tracking in order to validate the computational process. A comparison between detection of points of interest on regular images and foveated images has also been made. Results show that detection on foveated images is more efficient because it suppresses noise that is far enough from the focusing point while detecting as well significant points of interest. This is particularly interesting as the amount of data to process is greatly decreased by the radial re-sampling step.

In future works the two sequences of points of interest must be compared more accurately and their differences analyzed. Furthermore the computed sequence is the basis for the animation of the bionic eye in order to discover dynamically the new scene. Such a process assumes that the bionic eye is servo-controlled in several directions.

Next post:

Previous post: