Image Analysis for Automatically-Driven Bionic Eye (Bioengineering in Neurological Disorders) Part 2

State of the art: Overview

Supplying visual information to blind people is a goal that can be reached in several ways by more or less efficient means. Classically blind people can use a white cane, a guide-dog or more sophisticated means. The white cane is perceived as a symbol that warns other people and make them more careful to blind people. It is also very useful in obstacle detection. A guide-dog is also of a great help, as it interprets at a dog level the context scene. The dog is trained to guide the person in an outdoor environment. It can inform the blind person and advise of danger through its reactions. In the very last decades, electronics has come to reinforce the environment perception. On the one hand, several non-invasive systems have been set up such as GPS for visually impaired [Hub (2006)] that can assist blind people with orientation and navigation, talking equipment that provides an audio description in a basic way for thermometers, clocks or calcultors or in a more accurate way for audio-description that gives a narration of visual aspects of television movies or theater plays, electronic white canes [Faria (2010)], etc. On the other hand, biomedical devices can be implanted in an invasive way, that requires surgery and clinical trials. As presented in Fig. 4, such devices can be plugged at different spots along the visual data processing path. In a general way the principle is the same for retinal and cerebral implants. Two subsystems are linked, achieving data acquisition and processing for the first one and electrostimulation for the second one. A camera (or two for stereovision) is used to acquire visual data. These data are processed by the acquisition processing box in order to obtain data that are transmitted to the image processing box via a wired or wireless connection (Fig. 6). Then impulses stimulate cells where the implant is connected.

General principle of an implant

Fig. 6. General principle of an implant

Retina implant

For retinal implants, there exist two different ways to connect the electronic device: directly to the retina (epiretinal implant) or behind the retina (subretinal implant). Several research teams work on this subject worldwide. The target diseases mainly are:

• retinitis pigmentosa, which is the leading cause of inherited blindness in the world,

• age-related macular degeneration, which is the leading cause of blindness in the industrialized world.

Epiretinal implants

The development of an epiretinal prosthesis (Argus Retinal Prosthesis) has been initiated in the early 1990s at the Doheny Eye Institute and the University of California (USA)[Horsager (2010)Parikh (2010)]. This prosthesis was implanted in patients at John Hopkins University in order to demonstrate proof of principle. The company Second Sight1 was then created in the late 1990s to develop this prosthesis. The first generation (Argus I) has 16 electrodes and was implanted in 6 patients between 2002 and 2004. The second generation (Argus II) has 60 electrodes and clinical trials have been planned since 2007. Argus III is still in process and will have 240 electrodes.

VisionCare Ophtalmic Technologies and the CentralSight Treatment Program [Chun (2005)Lane (2004)Lane (2006)] has created an implantable miniature telescope in order to provide central vision to people having degenerated macula diseases. This telescope is implanted inside the eye behind the iris and projects magnified images on healthy areas of the central retina.

Subretinal implants

At University of Louvain, a subretinal implant (MIVIP: Microsystem-based Visual Prosthesis) made of a single electrode has been developped [Archambeau (2004)]. The optic nerve is directly stimulated by this electrode from electric signals received from an external camera.

In the late 1980s, Dr. Joseph Rizzo and Professor John Wyatt performed a number of proof-of-concept epiretinal stimulation trials on blind volunteers before developing a subretinal stimulator. They co-founded the Boston Retinal Implant Project (BRIP). The collaboration was initiated between the Massachusetts Eye and Ear Infirmary, Harvard Medical School and the Massachusetts Institute of Technology. The mission of the Boston Retinal Implant Project is to develop novel engineering solutions to restore vision and improve the quality-of-life for patients who are blind from degenerative disease of the retina, for which there is currently no cure. Early results are actually a reference for this solution. The core strategy of the Boston Retinal Implant Project 2 is to create novel engineering solutions to treat blinding diseases that elude other forms of treatment. The specific goal of this study is to develop an implantable microelectronic prosthesis to restore vision to patients with certain forms of retinal blindness. The proposed solution provides a special opportunity for visual rehabilitation with a prosthesis, which can deliver direct electrical stimulation to those cells that carry visual information.

The Artificial Silicon Retina (ASR)3 is a microchip containing 3500 photodiodes, developed by Alan and Vincent Chow. Each photodiode detects light and transforms it into electrical impulses stimulating retinal ganglion cells (Fig. 8).

In France, at the Institut de la Vision, the team of Pr Picaud has developed a subretinal implant [Djilas (2011)]. They have also set up clinical trials.

As well, in Germany [Zrenner (2008)], a subretinal prosthesis has been developed. A microphotodiode array (MPDA) acquires incident light information and send it to the chip located behind the retina. The chip transforms data into electrical signal stimulating the retinal ganglion cells.

In Japan [Yagi (2005)], a subretinal implant has been designed at Yagi Laboratory4. Experiments are mainly directed to obtain new biohybrid micro-electrode arrays.

BRIP Solution

Fig. 7. BRIP Solution

ASR device implanted in the retina

Fig. 8. ASR device implanted in the retina

At Stanford University, a visual prosthesis5 (Fig. 9) has been developed [Loudin (2007)]. It includes an optoelectronic system composed of a subretinal photodiode array and an infrared image projection system. A video camera acquires visual data that are processed and displayed on video goggles as IR images. Photodiodes in the subretinal implant are activated when the IR image arrives on retina through natural eye optics. Electric pulses stimulate the retina cells.

In Australia, the Bionic Vision system6 consists of a camera, attached to a pair of glasses, which transmits high-frequency radio signals to a microchip implanted in the retina. Electrical impulses stimulate retinal cells connected to the optic nerve. Such an implant improves the perception of light.

Cortex implant

William H. Dobelle initiated a project to develop a cortical implant [Dobelle (2000)], in order to return partially the vision to volunteer blind people [Ings (2007)]. His experiments began in the early 1970s with cortical stimulation on 37 sighted volunteers. Then four blind volunteers were implanted with permanent electrode arrays.A second microcontroller is also included in the belt pack and it is dedicated to brain stimulation. The stimulus generator is connected to the electrodes implanted on the visual cortex through a percutaneous pedestral. With this system a vision-impaired person is able to count his fingers and recognize basic symbols.

Stanford University visual prosthesis

Fig. 9. Stanford University visual prosthesis

In Canada, the research team of Pr Sawan [Sawan (2008)] at Polystim Neurotechnologies Laboratory7 has begun clinical trials for an electrode array providing images of 256 pixels (Fig. 10). Such images are not very accurate but they allow the patient to guess shapes. Furthermore clinical trials have proved that it was possible to directly stimulate neurons in the primary visual cortex.

Principle of Polystim Laboratory visual prosthesis

Fig. 10. Principle of Polystim Laboratory visual prosthesis

Bionic eye

Such a system has to mimick several abilities of the human visual system in order to make visual information available for blind people. The system is made of a camera acquiring images, an electronical device processing data and a mechanical system that drives the camera. Outputs can be provided on cerebral implants, in other words, electrodes matrices plugged to the primary visual cortex. When discovering a new scene the human eye processes by saccades and the gaze is successively focused at different points of interest. The sequence of focusing points enables to scan the scene in an optimized way according to the interest degree. The interest degree is a very complex criterion to estimate because it depends on the context and on the nature of elements included in the scene. Geometrical features of objects as well as color or structure are important in the interest estimation (Fig. 11). For example, a tree (b) is of a great interest in a urban landscape whereas a bench (a) is a salient information in a contryside scene. In the first case, the lack of geometrical particularities and the color difference make the tree interesting. In the second case the structure and the geometrical features of the bench make it interesting in comparison to trees or meadows.

Several steps are carried out successively or in parallel to process data and drive the camera. First of all a detection of points of interest is achieved on a regular image, in other words, on an image usually provided by a camera. One of the points best-scoring with the detector is chosen as the first focusing point. Then the image is re- sampled in a radial way in order to obtain a foveated image. The resulting image is blurred according to the distance to the focusing point [Larson (2009)]. Then a detection of points of interest is achieved on the foveated image in order to determine the second focusing point. These two steps are repeated as many times as necessary to discover the whole scene (Fig. 12). This gives the computed sequence of points of interest. In parallel a human observer faces the primary image while an eye- tracker follows his eye movements in order to determine the observer sequence of points of interest, when exploring the scene by saccades [Hernandez (2008)]. Afterwards the two sequences will have to be compared in order to quantify and qualify the computer vision process, in terms of position and order.

Next post:

Previous post: