An update on retinal prostheses

Abstract

Retinal prostheses are designed to restore a basic sense of sight to people with profound vision loss. They require a relatively intact posterior visual pathway (optic nerve, lateral geniculate nucleus and visual cortex). Retinal implants are options for people with severe stages of retinal degenerative disease such as retinitis pigmentosa and age-related macular degeneration.

There have now been three regulatory-approved retinal prostheses. Over five hundred patients have been implanted globally over the past 15 years. Devices generally provide an improved ability to localize high-contrast objects, navigate, and perform basic orientation tasks. Adverse events have included conjunctival erosion, retinal detachment, loss of light perception, and the need for revision surgery, but are rare. There are also specific device risks, including overstimulation (which could cause damage to the retina) or delamination of implanted components, but these are very unlikely.

Current challenges include how to improve visual acuity, enlarge the field-of-view, and reduce a complex visual scene to its most salient components through image processing. This review encompasses the work of over 40 individual research groups who have built devices, developed stimulation strategies, or investigated the basic physiology underpinning retinal prostheses. Current technologies are summarized, along with future challenges that face the field.

1. Introduction

Blindness and vision loss are among the most feared sensory disabilities (Chader et al., 2009). Unfortunately, despite the advances of modern medicine, millions of people around the world are experiencing the challenges that can come with severe vision loss. The field of vision restoration research is working to help these people, through the development of interventions specific to each indication, such as gene therapy, stem cells, optogenetics, vision restoration training, non-invasive stimulation, and vision prostheses. The latter is indicated for severe vision loss arising from degenerative retinal disease and is the subject of this review.

Over the past 30 years, technology has advanced to the point where it is now possible to implant electronic devices into the visual pathway to restore some version of sight. These devices, known as vision prostheses, can be implanted anywhere along the visual pathway; retina, optic nerve, lateral geniculate nucleus or visual cortex (Fig. 1). Whilst there are advantages and disadvantages of all locations, many devices to date have been implanted in or near the retina, or within the confines of the ocular globe. Devices in these positions may benefit from the geometric representation of the world at the retinal level and from residual retinal processing, although this is still to be proven conclusively. Retinal prostheses are also relatively easy to access surgically.

Fig. 1.

Potential implantation sites for vision prostheses. This review will focus on retinal implants, which are the most commercially advanced to date. Image courtesy of Bionic Vision Technologies, Australia.

To date, retinal prostheses have been implanted in patients with either inherited retinal degenerations (such as retinitis pigmentosa, RP) or age-related degenerative disease (such as atrophic age-related macular degeneration, AMD). In these conditions, the photoreceptors in the outer layers of the retina are damaged or lost, but the inner retinal neurons (bipolar and ganglion cells) remain relatively intact (Santos et al., 1997). This means that the devices can stimulate residual elements of the visual pathway, to restore basic vision to participants.

From early crude experiments of electrical stimulation of the visual pathway (Foerster, 1929), we now are at a time in history when retinal prostheses have been approved by regulatory authorities for people with profound vision loss. However, many challenges of the field have not been surmounted. The aim of this review is to provide an overview of the relevant anatomy and physiology involved with retinal prostheses, and the retinal remodeling that occurs during retinal disease (and how that can complicate the visual outcomes of prostheses). We will provide a history of retinal prosthesis development since the 1920s, and report on the current technical capabilities and limitations. Finally, we will offer a glimpse of what the future for retinal prostheses may hold, and the manner in which multi-disciplinary groups around the world are working together to improve sight for millions of people with profound vision loss.

2. Basic retinal anatomy and physiology

Retinal prostheses aim to replace the function of photoreceptors in a degenerate eye, and hence restore vision. However, there are challenges in this aim. In part, this is due to the complexity of retinal processing which the devices aim to replicate.

Vision begins when the optics of the eye project spatiotemporal patterns of light into the deepest layer of a thin sheet of neural tissue called the retina, lining the back of the eye. There, light-sensitive molecules called photopigments, located in the outer segment membranes of our photoreceptors undergo a conformational change as they absorb the incident photons of light. This triggers a cascade of finely tuned physiological responses that eventually result in the transmission of visual signals to the brain, through one or several of the many parallel pathways of our visual system.

The sensory photoreceptors are the first neural cells involved in vision. They line the back of the eye, forming the photoreceptor layer of the retina, and are located in close contact with the retinal pigment epithelium (RPE). RPE cells support visual function by regenerating photopigments and digesting shed photoreceptor outer segments. Without this key support from RPE cells, photoreceptors progressively atrophy and die - as can be observed in retinal degenerative disease. Vertebrate retinas contain at least two distinct types of photoreceptors, rods and cones. Rods form a homogeneous population and mediate night-time vision with their exquisite sensitivity, while cones support vision in bright environments in the daytime. In total, the human retina contains approximately 120 million rod and 6 million cone photoreceptors.

Rod and cone photoreceptors are graded-response neurons that do not produce action potentials. Instead, when they absorb light, they modulate the rate at which they release neurotransmitters to the two classes of retinal neurons that they synapse onto, the horizontal and bipolar cells. This synaptic connection is made in the outer plexiform layer of the retina, and the somas of both bipolar and horizontal cells are located in the inner nuclear layer. Parallel visual processing pathways appear to initiate in over 12 distinct types of bipolar cells (Masland, 2012). The majority of bipolar cells are cone driven, with each cone frequently driving multiple bipolar cells at the same time (Kling et al., 2019). Bipolar cells relay information to the output cells of the retina, called the retinal ganglion cells (RGCs), and amacrine cells in the inner nuclear layer interact with both RGCs and bipolar cells to shape the neural signals created in the RGCs. Finally, RGC axons form bundles on the innermost surface of the retina and which converge at the optic disk, where they end up forming the optic nerve.

The complex physiology of the retina provides significant challenges for retinal prostheses. At this time, electrical or photovoltaic stimulation of the retina is provided in a relatively unspecified manner, where multiple cell types will be activated at once. As discussed later in this review, the challenge now arises whereby more selective stimulation is required to optimize device resolution and patient outcomes.

3. Retinal remodelling in degenerative disease

Another significant challenge in the development of retinal prostheses is the fact that the retina does not maintain its layered structure and clearly defined functions during disease processes. As such, it is important to have an understanding of the remodeling that can occur in retinal degeneration, and what that can mean for retinal prosthesis design.

The earliest clinical manifestations of retinal degenerative disease depend upon the initiating mechanism(s) and the retinal cell types involved in the disease progression (retinal pigment epithelium, photoreceptors, etc). In the case of retinitis pigmentosa (RP), symptoms may appear in the late teens to early 20s and involve difficulty seeing and navigating in low light levels or adapting to rapidly changing light levels. As the disease progresses, all light perception at low light levels is lost and visual perception in daytime environments begins to be compromised, then is often lost later in life. This loss of visual perception corresponds to loss of rod photoreceptors commonly impacted by RP, followed by cone photoreceptors. What is less appreciated is that the circuits between secondary neurons, mediating the transfer and processing of signals are also impacted, forming an active and dynamic impediment to vision rescue strategies.

In outer retinal diseases such as RP and age-related macular degeneration (AMD) the precise circuit topologies of retinal neurons and glia are altered, along with the ability of the retina to appropriately process information. This rewiring or plasticity of neurons and glial cells in adult differentiated retina has been termed retinal remodeling and involves changes in the expression profiles of genes, proteins and neurotransmitter receptors, de novo neuritogenesis from all classes of neurons in the retina, and the formation of new synaptic processes within collections of these neurites called microneuromas (Fletcher and Kalloniatis, 1996; Jones et al., 2016a, 2016b; Marc et al., 2003; Strettoi et al., 2002).

Retinal remodeling occurs in phases, and progresses to a sustained, progressive neural degenerative disease that mimics central nervous system diseases such as Alzheimer’s and Parkinson’s. This phased neural degeneration begins with cell stress and then subsequently the deafferentation of the neural retina.

The retinal deafferentation has direct relevance to rescue of vision via bionic or biological methods, including gene therapy interventions and retinal prostheses. If we are to have successful outcomes for vision rescue therapies, we need to understand the fundamental biological mechanisms at play in retinal degenerative disease, to arrest or at least slow down the aberrant plasticity that the retina undergoes in response to photoreceptor cell stress and loss. Retinal plasticity itself may in fact be showing us the path forward, and might be amenable to intervention, particularly early in the disease process. Perhaps responding to retinal vision loss earlier in the disease process would be more successful than later interventions? To find answers, the cellular and molecular mechanisms involved in metabolic revision of neurons and glia as well as the phenotypic and pharmacological reprogramming that neurons appear to undergo in retinal degenerations will need to be investigated.

4. History of retinal prostheses

Before delving into the current state of play in this field, it is useful to have an understanding of the history. The first experiments using electricity to stimulate the visual pathway, and hence restore some degree of vision, were completed using direct cortical stimulation. In the 1700s, the French chemist and physician Charles Le Roy attempted to use crude transcranial electrical stimulation to cure a patient’s blindness (Uhlig et al., 2001). With repeated stimulation, the patient briefly perceived bright flashes of light called phosphenes, but remained otherwise blind.

Early studies in the 1920s by German ophthalmologist Foester confirmed that direct electrical stimulation of the visual cortex allowed a completely blind man to perceive spots of light (Foerster, 1929). Subsequent work by Krause and Schum then showed that it was possible to generate such phosphenes in people who had long-standing vision loss, by stimulating the brain of a man who had been blind from a gunshot wound for 8 years (Krause and Schum, 1931). Importantly, Krause showed that the phosphenes from a fixed point on the cortex were localized to a corresponding point in visual space and that even in a blind patient, the phosphenes could be elicited (Lewis et al., 1998).

Several decades later, interest re-emerged in the concept of vision restoration using electrical stimulation, and preclinical cortical studies were completed by Brindley and Lewin (1968) and Dobelle et al. (1976). Indeed, the work of Dobelle continued into clinical trials of a cortical implant, which was implanted in over 10 people in the late 1990s (Dobelle, 2000). However, these early cortical implants had poor resolution, significant surgical challenges, and often resulted in medical or psychological complications (Margalit et al., 2002; Lane, 2012).

The first report in the literature of a retinal prosthesis was by an Australian engineer called Graham Tassicker, who reported on the implantation of a photovoltaic array in the suprachoroidal space of a blind volunteer who perceived post-operatively a “uniform white light” (Tassicker, 1956). Advances in micro-engineering and retinal surgery in the late 20th century permitted the proliferation of the field, with devices now being manufactured in a smaller size, and of better materials, and the surgeons able to implant into retinal locations that were previously too challenging.

Early pioneers in the field of retinal prostheses include Alan and Vincent Chow (Optobionics), Eugene de Juan, Mark Humayun, Robert Greenberg and Jim Weiland (Second Sight Medical Products), Joe Rizzo and John Wyatt (Boston Retinal Implant Project), Eberhart Zrenner (University of Tuebingen) and Rolf Eckmiller (University of Bonn). Their successes led to the proliferation of retinal prosthesis development efforts, and the regulatory approval of three devices: Argus II (Second Sight, USA), Alpha AMS (Retina Implant, Germany) and IRIS II (Pixium, France), which will be described in further sections.

5. Basic mechanism of action of retinal prostheses

Retinal prostheses utilise engineered devices to replace the function of the damaged or missing photoreceptors in the retina. They can do this either via direct electrical stimulation, where a current is directed to electrodes implanted in or near the retina (Humayun et al., 2003; Ayton et al., 2014; Fujikado et al., 2016), or via photodiodes (Zrenner et al., 2011; Lorach et al., 2015), which utilise incident light to trigger the electrical stimulation.

In direct electrical stimulation, the device is generally composed of both an external and internal system. The former is generally a camera-based system, which captures an image of the outside world. This information can be transformed using image processing algorithms, to emphasise points of interest or decrease background interference. The image is then sent from the processing unit to an internal system, terminating in electrodes implanted in the eye. Direct power can then be transferred either through wires or through wireless transmission. Once the signal reaches the internal implant, it is decoded to an analogue signal and electrical currents are sent to each individual electrode.

In photovoltaic implants, the image is projected onto the retina using incident light where photodiodes transduce the signal into electrical stimulation. The light source can either be natural light (i.e. the image that a person would normally see falls onto their retina), or can be manipulated to be projected onto the retina using infrared wavelengths (Lorach et al., 2015). In the case of natural light, direct power is required to transduce the photodiode output signal into a stimulation pulse. The photodiode system allows natural eye movements, so that, within the limits of the projected field-of-view, a person’s eyes are always lined up with the image that is being stimulated on the retina. On the other hand, a camera-based system can have significant mismatch, as the eye may be pointing in a different direction than the head-mounted camera. This can lead to misdirection, where patients will reach to incorrect positions to find an object. This is of particular importance, as one potential risk of prosthesis implantation is limitation in the eye movements of the recipient (either due to the presence of electronic components, or due to extraocular muscle restrictions from the surgery. These considerations must be taken into account when training a person how to use their retinal prosthesis.

6. Image perception with a retinal prosthesis

Patients implanted with retinal prostheses can see light patterns of monochromatic dots, or “phosphenes”. The quality of the perceived image depends on a number of factors, including the number of electrodes/photodiodes on the implant, the stimulation strategies implemented, and the levels of greyscale that can be identified by the patient.

In short, if a patient looks at a cup with a camera-based (e.g., epiretinal) device, the camera first captures an image of the cup. The gain of the image is automatically adjusted in accordance with the brightness of the background, and grey tones adjusted to optimise the image presentation. The image is then pixelized according to the number and layout of electrodes on the device, and separate signals are sent to each electrode to provide the patient with the pixelated image. Advanced image processing algorithms (as discussed later in this review) can assist in this process by performing tasks such as edge detection to highlight borders of objects.

The vision provided by retinal prostheses is certainly distinct from natural perception and requires rehabilitative training and extensive practice to optimise outcomes.

7. Types of retinal prosthesis

Distinguished by surgical approach, there are four types of retinal prostheses; epiretinal prostheses (in which electrodes are placed on the retina), subretinal prostheses (in which electrodes are placed beneath the retina), suprachoroidal prostheses (in which electrodes are placed in the suprachoroidal space) and intrascleral prostheses (where the electrodes are placed within a pocket in the sclera); Fig. 2.

Fig. 2.

Locations of retinal prostheses. Image from Ayton et al. (2014). First-inhuman Trial of a Novel Suprachoroidal Retinal Prosthesis. PLoS One; 9(12): e115239.

7.1. Epiretinal prostheses

In an epiretinal prosthesis, the electrode array is placed on the inner surface of the retinal nerve fiber layer. These devices provide stimulation at the closest location to the target retinal ganglion cells, but can experience arcuate distortion of the phosphenes due to the direct stimulation of ganglion cell axons. The Argus II System (Second Sight Medical Products, USA) has a 60-channel stimulating epiretinal microelectrode array (Humayun et al., 2012), and is the most widely used device to date. It has both FDA and CE mark approval and has been implanted in over 350 patients globally. In May 2019, Second Sight announced that they were no longer manufacturing the Argus II device, and were reallocating those resources to their cortical implant (called the Orion). The IRIS II (Pixium Vision) and Epiret (Univ of Aachen) devices are other examples of the epiretinal approach. Whilst IRIS II was granted CE mark approval, it is no longer available commercially (since 2018), and the Epiret device is also in experimental phases only.

7.2. Subretinal prostheses

Subretinal prostheses are placed between the photoreceptor layer and the retinal pigment epithelium/choroid. This makes the implant location the closest physical space to the damaged or lost photoreceptors which they are replacing. An example of a subretinal prosthesis is the Alpha AMS implant (Retina Implant AG, Germany), which has 1600 photodiodes (Zrenner et al., 2011). In the Alpha AMS device, the photodiodes are not used for energy, but to measure the local light intensity. The power supply for the operation of the chip and for the stimulation current takes place inductively from the outside and is conducted via trans-scleral cables to the subretinal chip. The Alpha AMS (and its predecessor, the Alpha IMS) received CE mark approval, but unfortunately the company dissolved in March 2019. The work on the Alpha AMS will continue within its academic partners (including the University of Tübingen). Currently, a clinical trial of another subretinal device called the PRIMA implant (Pixium Vision, France) is underway in Europe and the USA in patients with age-related macular degeneration. This device utilises technology from Stanford University, and consists of photodiodes that respond to projected infrared light (Lorach et al., 2015).

7.3. Suprachoroidal prostheses

An Australian group (Bionic Vision Technologies) (Ayton et al., 2014) and a Japanese group (Osaka University) (Fujikado et al., 2016) have each developed novel approaches of implanting a retinal prosthesis further back in the eye. The Australian surgery involves implanting the electrodes into the suprachoroidal space, between the choroid (posterior blood supply) and sclera (outer white) of the eye (Saunders et al., 2014). The Japanese approach involves inserting the electrode array into a pocket within the scleral tissue. Lead-wires for both these devices are curved around the temporal process of the zygomatic bone, travelling posteriorly under the scalp to a junction behind the ear. At this junction is a sub-scalp stimulator or, in the case of the Australian prototype (Ayton et al., 2014), a percutaneous connector linking to an external stimulator. Both groups chose suprachoroidal approaches in order to simplify the surgical procedures and reduce associated risks. The possible disadvantage of this approach is that the electrodes are relatively farther from the retinal ganglion cells, so higher currents are required, and the current spread could be greater. However, clinical trial results of both devices have been promising (Ayton et al., 2014; Fujikado et al., 2016), with similar outcomes to those measured by other devices.

8. Engineering considerations in retinal prostheses

To achieve measurable vision outcomes from retinal prostheses, as described in the previous sections, there are a number of engineering, retinal stimulation and image processing aspects that need to be considered in device design and system implementation.

An electronic retinal prosthesis must perform several basic functions in order to replace the sense of vision. First, it must detect light in the nearby environment of the implant patient. Secondly, the light must be converted to an electrical stimulus, typically a biphasic pulse. Finally, the electrical pulse must be applied to the retina via a microelectrode array. Specific challenges related to retinal prostheses are image/video processing, the retinal electrode array, and packaging. Electronic design, that is wireless power/data transfer, and microstimulator circuits, are also part of the system design, but will not be discussed further here.

Light detection can be accomplished via an external camera and/or microphotodiodes integrated onto the implant. Incorporating the photosensitive elements in an implanted circuit (i.e. in the eye) offers the clear advantage of placing the light detection function under control of eye movements. To date, this has been accomplished by custom-designed chips with microphotodiodes integrated on chip and the chip placed in the subretinal space (Stingl et al., 2015a; Ho et al., 2017). Implanted cameras have been proposed, but have not yet been implemented in human devices (Zhou et al., 2010; Stiles et al., 2011). The optics of the eye focus natural light (Alpha-AMS; Retina Implant AG, Germany) or infrared light (PRIMA; Pixium Vision, France) onto a subretinal chip. In the Alpha-AMS device, the microphotodiode produces an electrical signal that is then amplified by circuitry on-chip. The PRIMA device uses high-intensity pulsed infrared light to drive infraredsensitive microphotodiodes and the resulting photovoltaic signal stimulates the retina.

Most other devices, including the Argus II epiretinal implant (Second Sight, USA), the Osaka University STS implant (Japan) and the Australian suprachoroidal device (Bionic Vision Technologies, Australia) use an external camera system wirelessly coupled to an implantable stimulator. This allows the use of image processing algorithms to maximise the usefulness of the image output from the camera. These algorithms will be discussed in the following section “Image processing”.

8.1. Retinal stimulating electrode arrays

Implementation of a stable electrode-retina interface has a number of challenges. Many of these challenges stem from the fact that the retina is curved and microelectrode technology typically produces planar structures (Opie et al., 2014). Even if an array with the right curvature can be produced, the eye curvature varies from person to person and even a separation of 100 mm can be significant (de Balthasar et al., 2008). If the curvature mismatch causes a device to mechanically compress the retina, then the retina can be damaged. If the curvature mismatch results in separation of the electrodes from the retina, then more current may be needed to activate the retina and the current may spread over a larger retinal area. Therefore, the ideal retinal stimulating electrode would have the flexibility to match the curvature of the retina without placing significant mechanical pressure on the retina. Platinum is the electrode material most frequently used for neural stimulation, but retinal stimulators to date have used alternative materials or novel versions of platinum, including platinum grey (high surface area platinum) (Zhou, 2005) and iridium oxide (Lorach et al., 2015; Daschner et al, 2018), which have better charge injection properties than smooth platinum.

8.2. Packaging

If an active electronic circuit is placed in the body, then hermetic packaging is required to protect the circuit from water and ions (Vanhoestenberghe and Donaldson, 2013). Even simple electrodes with wire leads require that the leads remain well insulated. Two types of hermetic packaging are utilized in retinal prostheses: enclosures and encapsulation. An enclosure is a traditional sealed package (e.g. close-fitting titanium) with decades of usage in other implanted electronics. Argus II uses an enclosure and has been functioning in some users for over 10 years. Encapsulation; the use of a conformal coating to form a protective layer around the electronics, has not been used as extensively as a hermetic barrier for a number of reasons, but two retinal prostheses, Alpha-AMS and PRIMA, rely on encapsulants. One major issue for the first-generation Alpha-IMS was failure of the encapsulation; the design of the Alpha AMS greatly improved this shortcoming (Daschner et al., 2018). PRIMA’s long-term reliability is currently being tested in clinical trials. The benefit of using an encapsulant is size, since a thin, protective layer on a silicon chip is significantly smaller than a metal case. In addition, this is also required to allow light to reach the photodiodes that capture the image on a photosensitive chip. A subretinal chip must use an encapsulant, while a retinal stimulating array, connected to an electronics module housed in an enclosure, can be either suprachoroidal, subretinal, or epiretinal. Use of an enclosure, while providing long-term reliability, does require routing of conducting lines (‘feed-throughs’) through the packaging to each electrode site on the array, which limits the number of electrode contacts as each feed-through is a potential source of hermetic failure. An encapsulated subretinal chip does not have this issue since removal of the encapsulant at each electrode site can be done to achieve a large number of pixels. Simply put, the choice of packaging is a trade-off between electrode number and density versus reliability. An enclosure will last longer, but an encapsulant allows a design with more electrodes and possibly better visual acuity. Reported device failures have provided significant impetus to improve upon hardware design and surgical procedures. Aside from encapsulation challenges, Retina Implant AG have also reported failure mechanisms relating to corrosion of the CMOS chip, degradation of the power supply cable, and procedural errors during implantation and subsequent revision surgery (Daschner et al., 2017). A recent retrospective analysis of 274 Argus II (Second Sight) implantations revealed that conjunctival erosion could occur over the suture tabs, prompting suggested modifications to the surgical procedure (Rizzo et al., 2019). Moreover, possible revisions to the radio-frequency (RF) link were indicated following reports of a gradual loss of RF signal at 4 years post-implant (da Cruz et al., 2016). Engineering challenges arising from lead-wires breakages are common also; reports of shortened device lifespan in IRIS II (PIXIUM) clinical trial patients may be attributable to microfractures in the lead-wire and possible moisture ingress (Hemami and Jacobs, 2017). Accelerated aging of the implant components, combined with pre-clinical implantation studies, forms an essential part of the test-and-revise design process.

These engineering considerations and challenges have led to the development of retinal prostheses of varying design. It is possible that different device styles may be beneficial for patients with different disease subtypes; this remains to be seen, as the field progresses.

9. Retinal stimulation strategies

No matter the engineering design, all retinal prostheses need to activate retinal cells in order to generate phosphenes, the visual percepts that make up prosthetic vision.

It has been the historical custom to contrast epiretinal with subretinal electrodes as stimulating either retinal ganglion cells (RGCs) or bipolar cells (BCs), respectively. This oversimplification, however, has given rise to the mistaken impression that subretinal electrodes cannot stimulate RGCs directly and that epiretinal electrodes can only activate RGCs. In practice, most retinal prosthesis electrodes have the potential to stimulate any cell of the retina. They function by creating a voltage gradient across the thickness of the retina that influences all neurons. This voltage gradient induces the flow of ionic currents and changes the local transmembrane potential of neurons, thus engaging voltage-gated ion channels and giving rise to the release of neurotransmitters from nonspiking neurons and the generation of action potentials (spikes) in RGCs that project to the rest of the brain. The classic strategy for creating a voltage gradient has been to use cathodic (negativegoing) stimuli for epiretinal stimulation (da Cruz et al., 2013) and anodic (positive-going) stimuli for subretinal stimulation (Zrenner et al., 2011). These configurations exploit the topography of ion-channels within the center-surround organization of the ganglion cell soma and have been shown to have lower excitation thresholds and better selectivity of RGCs (Eickenscheidt et al., 2012; Boinagrov et al., 2014).

When spikes are elicited by the direct depolarization of RGCs, this is termed direct RGC stimulation. If, however, electrical stimulation of the retina causes neurotransmitter release from neurons presynaptic to RGCs, which then leads to RGC spike generation through synaptic mechanisms, this is termed indirect RGC stimulation. Any electrode configuration that passes current through the thickness of the retina (epiretinal, subretinal, suprachoroidal, etc.) can elicit both direct and indirect stimulation (Weiland et al., 2016). Which mode (direct or indirect) is dominant depends not only on the relative proximity of electrodes to RGCs versus other retinal neurons, but also on the time-varying characteristics of the electrical stimulus. Due to the kinetics of the voltage-gated sodium and potassium channels underlying spikes, RGCs are most responsive to fast voltage changes on the order of 1 millisecond in duration (1000 Hz). As a loose rule of thumb, bipolar cells are more responsive to 100 Hz stimulation, whereas photoreceptors are more responsive to 10 Hz (Freeman et al., 2010; Twyford and Fried, 2016). Thus, both electrode placement and electrical wave form can be varied to optimally target either RGCs or the remaining retinal network, potentially including residual photoreceptors that lack functional outer segments.

Beyond targeting specific cell classes of the retina (RGC, BC, photoreceptor, amacrine, horizontal), it has recently been proposed that electrical waveforms could be optimized for selective stimulation of targeted RGC types (Rathbun et al., 2018), of which over 30 have been classified in the mouse retina (Baden et al., 2016). Each of these types conveys a different feature of the visual world to the brain, therefore, selective activation of RGC types could refine the image perceived by retinal implant recipients. Such refinements could reasonably include better spatial and temporal contrast and full color vision, as well as more abstract visual properties.

These stimulation strategies are vital to the development of higher resolution retinal prostheses in the future. In addition to specific electrical properties of the stimulation, next-generation devices will almost certainly require the use of much smaller electrodes in the vicinity of target cells, which will have the additional advantage of limiting the cross-talk that dominates activity in the degenerated retina when using today’s retinal implants.

10. Image processing

Use of an external camera, worn on glasses or goggles, and a processing unit makes possible the implementation of image processing algorithms that can enhance and/or simplify an image, to account for the low-resolution of retinal protheses. Image processing has been defined as hardware or software operations that transform visual data from the sensor to perceptual parameters that are coded as stimulation (Barnes et al., 2016).

Real-time image processing is necessary, since the subjects will be correlating camera direction with the location of the perception and the stimulus must update fast enough to create the perception of where the camera is pointed. The system must be portable. Rapid advancements in augmented reality headsets (like HoloLens, Microsoft Inc., and Magic Leap One, Magic Leap Inc.) suggest that use of off-the-shelf hardware is a feasible option for processing video in real time. Basic functions the image processing will perform include decimation and averaging of the pixels to match the density of the electrode array. In addition to these basic processing methods, image processing methods can be categorized by function into two broad groups: (1) methods for general processing for all activities and scenes; and (2) methods that are specialized to support a more narrow group of activities.

General image processing focuses on downsampling from a high-resolution input image to a low-resolution display of a retinal prosthesis. Early work in this area, performed in a significant number of studies with normally-sighted participants viewing a retinal implant simulator, have demonstrated the benefits of image filtering. Hayes et al. incorporated regional averaging (calculating the average value over a surrounding neighbourhood of pixels) and Gaussian filtering (where a Gaussian profiled weighted average is applied) which was demonstrated to result in a sharper downsampled image and hence improved visual performance in tasks such as visual pursuit (Hayes et al., 2003; Hallum et al., 2006). An early report of image processing in Argus II recipients stated that several image filtering methods were available to users, including edge detection, contrast enhancement, and difference of Gaussian (Humayun et al., 2009). The Argus II device has also had a software upgrade called “Acuboost™, in which user-controlled zooming and image processing to extract global features in the image could be used to improve visual acuity (Sahel et al., 2013). Following early human implant trials (Rizzo et al., 2003), the Boston Retinal Implant Project also described a number of image processing algorithms that would be used with their device, including contrast enhancement, edge extraction, blurring and thresholding (Shire et al., 2012). Finally, an Australian study using the Lanczos2 filter, a high performing image downsampling filter, showed improvements in performance on a localization of light task over using a minimal image processing approach (Barnes et al., 2016)

Prostheses based on implantable microphotodiodes or any form of eye-resident visual sensor present challenges for incorporating image processing due to the restrictions of fully implantable processing (e.g., of heat dissipation). However, Zrenner et al proposed incorporating a head-mounted display that takes an image stream from a head worn camera, and uses image processing to convert it to a processed display that can be viewed by the eye resident sensing (Lasker/IRRF Initiative, 2014). Indeed, this method is incorporated in PRIMA(Pixium, France), which utilizes a head worn camera system to translate incident light to infrared pulses to stimulate the subretinal electrodes. This will allow processing of the camera image to occur prior to projecting infrared light onto the photodiodes.

Rather than aiming to improve general visual performance, many methods have been proposed to boost performance for specific tasks that are seen as important to improving quality of life. For example, face detection may be aided by using a standard computer vision face detection algorithm, which highlights areas in the image in which a face appears (Stanga et al., 2013). So, although the participant cannot see the detail of the face, they are aware it is there by the activation of phosphenes in that region.

Other methods have been developed to help identify obstacles on the ground plane, including the use of depth algorithms. One approach used a wearable depth camera, and then separated the obstacles and ground plane using computer vision algorithms (McCarthy et al., 2015). In this study of normally sighted participants with a wearable prosthetic vision simulator, the depth method reduced the number of collisions with ground-based obstacles compared to standard image processing, when the obstacles were poorly contrasted from their background (McCarthy et al., 2015). This method was also evaluated in participants that were implanted with the Australian suprachoroidal prosthesis (Bionic Vision Technologies, Australia), showing similar results with two participants (Barnes et al., 2015).

A newer modality is to use thermal cameras for aiding object localization, particularly of people and hot or cold food items at a table setting. A recent study showed improvements in detection accuracy compared with using standard approaches for mobility, orientation and object localization tasks (Bajaj et al., 2019). In a similar setup, a stereoscopic camera was used to select a depth range that will be displayed to the Argus II implant, while information from closer or farther distance is filtered out (Sadeghi et al., 2019).

In summary, there are a number of image processing algorithms that may be of benefit to users of retinal prostheses to maximize the utility of the presently limited device acuity. Whilst many have been investigated using simulations, current work is focusing on implementing them with users of implanted retinal prostheses, to determine the efficacy in real-world settings.

11. Retinal implant surgery

This section will provide an overview of the surgical techniques required for retinal prostheses. Of course, different devices have specific surgical methods (which would be too lengthy to detail in full); instead, we will provide a high-level discussion of the surgical requirements.

To date, chronic clinical implantations have been completed on ten devices, as detailed in Table 1.

Table 1.

Retinal prosthesis devices that have been tested in chronic human implantation studies (to September 2019).

Device Name	Company/Research Consortium	Electrode configuration	Array material, size and surgical location	Stimulation parameters	Number of electrodes	Patients implanted	Best reported visual acuity (if available)	Current status
Artificial Silicon Retina (ASR) (Chow, 2013)	Optobionics, USA (company)	Microphotodiodes (5000 pixels). Each pixel contained a 9 μm × 9 μm iridium oxide electrode and a 20 μm × 20 μm MPD.	Subretinal silicone array, 2 mm diameter	200 μA, 5 Hz	5000	>40		Discontinued – did not meet efficacy endpoints. Company closed
Argus I (Humayun et al., 2003)	Second Sight Medical Products, USA (company)	Platinum electrodes (8 × 200 μm and 8 × 500 μm diameter)	Epiretinal silicone array, 3 mm diameter		16	6		Discontinued (1st generation prototype)
Argus II (Humayun et al., 2012)	Second Sight Medical Products, USA (company)	Platinum electrodes (200 μm diameter)	Epiretinal Polyimide array 3 mm diameter	4-677μA, 20 Hz	60	>350	20/1260	FDA and CE mark approval. Available commercially. Suspended manufacture in May 2019 (resources diverted to a cortical prosthesis, the Orion)
Tuebingen Prototype (Wilke et al., 2011)	University of Tuebingen, Germany (university) – led to Retina Implant AG devices (below)	Microphotodiodes (5000 pixels) and 16 × 50 μm diameter titanium nitride electrodes	Subretinal silicone array	N/A	5000 photodiodes and 16 electrodes	11		Prototype device, was intended for short term experiments only
Alpha-IMS (Stingl et al., 2015a)	Retina Implant AG, Germany (company)	Microphotodiodes (1500 pixels). Each pixel contained a 50 μm diameter iridium oxide electrode	Subretinal silicone array, 3 mm × 3 mm	N/A	1500	30		CE mark approval. Discontinued (1st generation)
Alpha-AMS (Stingl et al., 2017)	Retina Implant AG, Germany (company)	Microphotodiodes (1600 pixels) with circular iridium oxide electrodes (30 μm diameter)	Subretinal silicone array, 3 mm × 3 mm	Max. 50 μA, pulse duration 0,2 – 2 ms, up to 20 Hz	1600	28	20/546	CE mark approval. Company ceased trading March 2019
Learning Retinal Implant System (Hornig et al., 2007)	Intelligent Medical Implants (IMI)	Platinum electrodes (250 μm diameter)	Epiretinal polyimide array, 2.4 mm2 area	0.5-380 μA, 60 Hz	50	4		20 patients were implanted in a temporary, surgical study. 4 patients had the implant for > 30 months. Company sold to Pixium Vision (France)
IRIS II (Muqit et al., 2019)	Pixium Vision, France (company)	Platinum electrodes (approx. 250 μm diameter)	Epiretinal polyimide array, 2.4 mm2 area	Not published	150	10		Discontinued (2nd generation prototype)
PRIMA (Lorach et al., 2015)	Pixium Vision, France (company)	Photovoltaic pixels (100 μm in width)	Subretinal array, 2 mm × 2 mm	N/A	378	5		In clinical trial for age-related macular degeneration (France and USA)
Gen 1 suprachoroidal device (Ayton et al., 2014)	Bionic Vision Australia (research consortium)	Platinum electrodes (30 × 600 μm diameter, 3 × 400 μm diameter)	Suprachoroidal silicone array, 19 mm × 8 mm	75 μA, 200 Hz	24 active (33 in total, but the outer ring were ganged and functioned as a single return electrode)	3	20/8397	Discontinued (1st generation prototype)
Gen 2 suprachoroidal device (Abbott et al., 2018)	Bionic Vision Technologies, Australia (company)	Platinum electrodes (44 × 1 mm diameter, 2 × 2 mm diameter for returns)	Suprachoroidal silicone array (17 mm × 8.5 mm)	Not yet reported	44 active, 2 return	4		In clinical trial for retinitis pigmentosa (Australia)
STS device (Fujikado et al., 2016)	Osaka University, Japan (research consortium)	Raised platinum electrodes (500 μm diameter and 500 μm height)	Intrascleral polyimide array, 5.7 mm × 4.6 mm	≤1 mA, 20 Hz	49	5		In clinical trial for retinitis pigmentosa (Japan)
EPIRET3 (Roessler et al.,2009)	RWTH Aachen	25 iridium oxide electrodes (100 μm diameter)	Epiretinal polyimide array, 2 mm × 3 mm	3.2–100 μA, 20 or 100 Hz	25	6		6 patients were implanted for 28 days. No active clinical trials at this time

All retinal implants require a vitreoretinal surgeon to implant the device. An operation known as a “3-port pars plana vitrectomy” is mandatory for implantation of the epiretinal (Roessler et al., 2009; Humayun et al., 2012; Muqit et al., 2019) and subretinal (Lorach et al., 2015; Stingl et al., 2017) prostheses, but not required for suprachorioidal (Saunders et al., 2014) or intrascleral (Fujikado et al., 2016) devices. Implantation of retinal prostheses is preferably undertaken with general anesthesia due to the complexity of the surgical steps.

11.1. Argus II epiretinal prosthesis, Second Sight Medical Products, USA

The Argus II device (Humayun et al., 2012) has an external Application-Specific-Integrated-Circuit (ASIC) contained within hermetic casing that is attached to an internal coil, and these components are supported on an encircling silicone band, with an intraocular cable connector to the epiretinal 60-electrode array (Fig. 3B). The silicone band is surgically sutured onto the sclera of the eyeball to hold the implant securely onto the eyeball surface. A standard vitrectomy surgery is performed, and then the electrode array is inserted into the vitreous cavity and positioned with the correct orientation on the macula. A titanium tack is then used to secure the heel of the implant to the retinal surface and hold the electrode array in position across the macula. Typical surgery times for Argus II are up to four hours, depending on surgeon experience.

Fig. 3.

(A) The external components; glasses (left) and video processing unit (right) of the Argus II implant. (B) The ocular components of the Argus II implant, including a scleral band, internal coil (for power and data transfer) and the electrode array which is attached to the epiretinal surface with a tack. Images courtesy of Second Sight Medical Products, USA.

11.2. IRIS II epiretinal prosthesis, Pixium Vision, France

The IRIS II epiretinal implant (Muqit et al., 2019) is a 150-electrode stimulating array attached to a flexible intraocular foil connected to the electronics for wireless energy reception (Fig. 4). In clinic, transpupillary long-pulse laser photocoagulation burns are applied to the intended location of the fixation tack for surgery. This is performed a minimum three weeks before the implantation, guided by fundus photographs with implant landmarks and location of the fovea center. The laser photocoagulation creates a chorioretinal adhesion at the retinal tack site prior to surgical implantation, the aim being to decrease the potential risks of retinal detachment proximal to the tack.

Fig. 4.

The glasses and video processing unit (left) and ocular components (right) of the Iris II epiretinal implant. Images courtesy of Pixium Vision, France.

The retinal implant surgery involves exposure of the scleral wall and fixation of three rectus eye muscles. A partial-thickness rectangular surgical pocket is prepared within the sclera superiortemporal to the area of the projected implant position. After complete vitrectomy, the retinal tack is placed at the predefined lasermarked position through a scleral opening. The intraocular part of the implant is then attached to the inner retinal surface, “docking” via a retinal tack. Average surgery times for IRIS II implantation are 2.5-4 h.

11.3. Alpha AMS subretinal Implants, retina implant AG, Germany

Three photovoltaic subretinal implants have been developed by Retina Implant AG (Germany); a protype, the first generation Alpha IMS and the second generation Alpha AMS device (Fig. 5). Alpha IMS and AMS differ mainly in improved fabrications methods (Daschner et al., 2018). An ENT surgeon implants the external components, which are attached the the skull bone behind the ear. The intraocular part consists of the implant attached to a foil that are both positioned in the subretinal space. A vitrectomy surgery is performed, a surgical pocket is created in the sclera, and the subretinal device is passed into the eye over a surgical glide. A subcutaneous silicone cable under the temporal muscle connects the intraocular and retroauricular components. The entire implantation process usually took between 6 and 8 h, and in individual cases up to 10 h.

Fig. 5.

The external coil and power supply unit (A), and the subretinal array implanted in the retina (B) with a closeup of the photovoltaic diodes of the Alpha AMS implant. Note that as the device is photovoltaic, it does not use an external camera embedded on goggles, as the other devices do. Images courtesy of Retina Implant AG, Germany.

11.4. Suprachoroidal prosthesis, bionic vision technologies, Australia

In Australia, two suprachoroidal retinal implants have been trialled in patients; the prototype Gen 1 and the Gen 2 device (Fig. 6). There are two main differences between the devices. The former was connected to a percutaneous connector for stimulation, and hence could only be used in laboratory settings (Ayton et al., 2014). This connector needed to be surgically inserted behind the ear by an ENT surgeon. The second generation device is fully implantable, and can be used at home (Abbott et al., 2018). In addition, the number of electrodes has almost doubled on the second generation device. In both devices, the connecting helical lead wire between the stimulator or percutaneous plug and the intraocular array is loaded into a trochar and this is tunneled into the orbit. The lateral rectus eye muscle is disinserted to allow access. The intraocular implant is then passed into the eye within the suprachoroidal space. Surgical time is usually less than four hours, including extraocular procedures.

Fig. 6.

The intraocular components of the Gen 2 suprachoroidal implant, currently being tested in clinical trials in Australia. Image courtesy of Bionic Vision Technologies.

11.5. Prima subretinal prosthesis, pixium vision, France

The PRIMA System consists of a miniaturized and wireless subretinal microchip, which is activated using projected infrared light from external goggles (Fig. 7) (Lorach et al., 2015). A retinal surgeon performs the vitrectomy surgery. The PRIMA microchip is implanted into the subretinal space through a 2–3 mm opening in the retina. The small size of the PRIMA implant and the less invasive nature of its implantation surgery are particular features that have been designed to preserve the residual peripheral vision. This device is currently being trialled in patients with age-related macular degeneration, using a prototype 378-pixel device with 100 μm photovoltaic pixels. Using the current manufacturing process used for Prima, it may be possible to reduce the individual electrode size down to 50 or 70 microns. Once the PRIMA is positioned under the central fovea, either a bubble of gas or silicone oil is injected into the vitreous cavity. The surgical incision sites are then all closed. The PRIMA implantation surgery time is between 1 and 2 h, and can be performed under local anesthesia.

Fig. 7.

The PRIMA subretinal device, showing details of the preclinical photodiode array (top left) and the clinical external controller and glasses (bottom right). Images courtesy of Pixium Vision, France.

11.6. Suprachoroidal-transretinal stimulation device, Osaka University, Japan

The Suprachoroidal-Transretinal Stimulation (STS) device (Osaka University, Japan; Fig. 8) (Fujikado et al., 2016) is implanted within a pocket in the sclera of the eye. At first, the patient is placed under local anesthesia, and the lateral rectus muscle is dissected at its insertion. Transscleral electrical stimulation is applied to the exposed sclera to elicit visual responses and identify the best location for the implant, after which the patient is placed under general anesthesia. The subcutaneous part of the device (electronics package) is implanted behind the temporal bone, and the cable is affixed on the zygomatic bone. The electrode array is then implanted into the scleral pocket. On average, the surgery takes 6 h (Sakaguchi et al., 2011).

Fig. 8.

The Suprachoroidal-Transretinal Stimulation (STS) device from Osaka University; showing the power and data coil which is implanted subcutaneously (top) and the electrode array which is placed inside the scleral pocket (bottom). Image courtesy of Prof. Takashi Fujikado, Osaka University, Japan.

Following surgery for all these devices, a standard application of steroid and antibiotic eyedrops is used for a minimum of four weeks. As part of standard of care, all implanted patients are reviewed at regular intervals to assess intraocular pressure, surgical wound healing, inflammation, implant position, and retinal status. Following a period of healing, the device is activated, and psychophysics, functional vision testing, and patient reported outcomes can be assessed, as outlined in the following sections.

As with any ocular surgery, there are a number of risks that need to be considered before a patient chooses to receive an implant. Firstly, there are surgical risks, such as retinal tears, retinal detachments, damage to the eye or orbit, or ocular infection (endophthalmitis). There is a risk that patients may lose their residual natural vision, but this has not been noted to date. Adverse events that have been reported in previous clinical trials include raised intraocular pressure, conjunctival damage, endophthalmitis, and retinal haemorrhage. The majority of these events resolve well with treatment.

One of the key attributes of device development is ensuring that potential risks can be managed appropriately. Hence, retinal prostheses can usually be explanted if required (although this can be more challenging with some devices, i.e. those with tacks, than others). It is also possible to “turn off” most devices if the patient no longer wishes to use the prosthesis. Long term safety has been demonstrated in the Argus I device implants, which have now been in patients for almost 15 years (Yue, 2015). Full details on the safety profiles of each device can be found in their clinical trial publications, or via the www.clinicaltrials.gov website.

12. Clinical psychophysics

Following a period of recovery post-surgery, the electrical device parameters must be personalized to the recipient. Although retinal implants may specify tens or hundreds of electrodes, the proximity of the electrodes to the target neurons, as well as density and interconnectedness of remaining viable inner retinal neurons will greatly affect the efficacy of stimulation and the quality of the visual perception. Variation between recipients can include duration of blindness, retinal health, surgical placement, axial globe length, and adherence or conformability of the device to the globe curvature.

It is important to note that the perception observed by implantees does not necessarily represent synthetic images or form vision. In many cases the perception is spatially distorted and temporally complex, owing to the diversity of stimulated cell types in the retina and the unwanted stimulation of axon fibres (Fine and Boynton, 2015). It is the practical goal of clinical psychophysics to relate stimulus parameters to patient perception, so as to provide optimal device parameters for each recipient.

The most basic parameter for a retinal implant is the threshold charge at which a visual percept is elicited, and this will vary across electrodes and patients (de Balthasar et al., 2008; Ahuja et al., 2013; Shivdasani et al., 2014). Not all electrodes will produce a percept; large electrode-retina distances or smaller ganglion cell densities adjacent to the electrode are both factors that will increase the activation threshold (Ahuja et al., 2013) and may require electrodes to be stimulated as groups of two or more before a threshold is obtained (Tran and Wolfensberger, 2017).

For devices with no direct access to single electrodes (e.g. photodiode devices), whole-array psychophysics may be conducted using a calibrated full-field stimulus source, such as a ganzfeld flash stimulator (Dagnelie, 2008). This approach has been shown to be sufficient to determine global parameters for photodiode devices such as activation threshold and amplifier gain (Zrenner et al., 2011; Stingl et al., 2016), and may also inform patient selection for retinal prostheses at large (Ahuja et al., 2013). To confine activity to a subset of electrodes, focused light can be projected directly onto photodiodes by automated tracking of the fundus or an adaptive optics scanning laser ophthalmoscope (Zrenner et al., 2011).

Threshold values may vary over time, or even within a single test session (de Balthasar et al., 2008; Velikay-Parel et al., 2013). A clinical recommendation is to determine threshold for a subset of electrodes on a regular basis, to examine the stability of these measurements over time.

Retinal implants can convey a range of brightness or luminance if the stimulation is modulated by amplitude or frequency. Colorcoding has been described (Stanga et al., 2012) but is not commonly reported. Amplitude-modulation of brightness is more practical to implement, but it has been reported that the size of the percept increases with amplitude whereas the effect on size is smaller with increases in stimulation frequency (Nanduri et al., 2012; Sinclair et al., 2016).

The steepness of the brightness-growth function will vary across electrodes, and therefore the goal of brightness matching tasks is to find the point of subjective equality (PSE) between two stimuli. For example, the PSE might represent the stimulus intensity at which the same pulse train on two spatially separate electrodes (Greenwald et al., 2009), two different pulse trains on the same electrode (Horsager et al., 2009), or different pulse trains of different electrodes (Horsager et al., 2010) appear equally bright. With photovoltaic devices it is possible to capture electrode artifact on standard electroretinographic equipment to assess and adjust the brightness-growth functions in response to varying luminances of full-field illumination (Stingl et al., 2016). Perception of up to 9 grey levels has been reported for users of the Alpha IMS subretinal device (Zrenner et al., 2011) and 6–10 stimulus levels in the Argus II epiretinal device (Greenwald et al., 2009).

The complexity of retinal stimulation becomes apparent when activating multiple electrodes to form an ‘image’. Responses from neighboring electrodes will interact both visually and temporally, resulting in both facilitatory and suppressive effects (Horsager et al., 2010; Horsager et al., 2011), concurrent stimulation of On and Off pathways, and possible desensitization whereby phosphene brightness gradually fades over time (Horsager et al., 2009; Freeman and Fried, 2011; Fornos et al., 2012). Mitigating these effects requires judicious coordination of inter-electrode timing and pulse parameters. Future stimulation strategies may eventually mimic the neural code of the retina to be more selective of the many retinal cell types (Jepson Lauren et al., 2014).

Interactions between electrodes can be assessed using two point discrimination tasks; can the participant discriminate one electrode from another? Often the case is that immediately adjacent electrodes are difficult to discriminate but the task becomes easier with increasing distance between electrode pairs (Lauritzen et al., 2011). It is useful to get feedback from the participant about the size, shape, and overlap of phosphenes to interpret discrimination results. Phosphene shapes have been characterized using a variety of manual and electronic mapping methods, and reported to be round dots, dark-centered rings, complex arcs, blobs, or ellipses (Fujikado et al., 2011; Nanduri, 2011; Luo et al., 2016; Sinclair et al., 2016).

Phosphene sizes are often much larger than the area of retina covered by each electrode. For example, a single 200 mm electrode of the Argus II subtends a visual angle of 0.7 degrees (Yue et al., 2016) but can produce phosphene lengths >10 degrees of visual arc (Nanduri et al., 2008). Distortions in phosphene shape and size can be explained by the effects of unwanted axonal stimulation (Horsager et al., 2011; Nanduri, 2011; Fine and Boynton, 2015; Beyeler et al., 2017). Improvements to electrode design and stimulation parameters may avoid axonal stimulation (Weitz et al., 2015).

One last consideration is the entangled relationship between phosphene location and gaze angle. Stimulation of electrodes at fixed positions on the retina will generate phosphenes that move with the position of the eye in the orbit (Sabbah et al., 2014; Caspi et al., 2017b). This is not a confound for subretinal photodiode devices, but for devices utilizing a fixed video camera the recipient must constrain their eye gaze to avoid disassociation between real-world and perceptual frames of reference. Using feedback from an eye-tracker may allow the prosthesis to compensate for gaze shifts and restore naturalistic gaze behaviour (Caspi et al., 2017a; Titchener et al., 2018).

13. Visual function and functional vision testing

Once the basic psychophysical tests have been completed with the retinal implant, and the optimal device parameters have been set, it is possible to measure visual function outcomes for the patient.

Visual function is a broad term, as natural human vision has many aspects and all of them contribute to the functionality of vision. The most important ones are visual acuity and visual field, but contrast discrimination, dark adaptation and color vision as well as movement perception are essential for daily life. They can be evaluated with standardized examinations in ophthalmology. Apart from that, there are a wide range of daily-life visual experiences, based on pattern and shape recognition and commonly described as orientation and mobility (O&M) or activities of daily living (ADL). These are harder to evaluate or quantify due to their subjective and non-standardized nature. They are often referred to as “functional vision” tasks.

If a retinal prosthesis can mediate light perception, it allows further evaluation of vision. Due to the variety of retinal prosthetic systems available nowadays either as approved devices or systems in clinical tests (Table 1), it is difficult to define universal tests using retinal implants. Several reviews summarizing the outcomes of previous clinical trials to date have been published (Stingl and Zrenner, 2013; Goetz and Palanker, 2016; Cheng et al., 2017; Mills et al., 2017), and a number of papers have proposed methodologies for the assessment of vision outcomes in these trials (Stingl et al., 2013a; Finger et al., 2014a,b; Geruschat et al., 2015; Finger et al., 2016).

The first aspect of vision tested in all clinical trials with prosthetic vision is visual acuity (either as grating acuity or optotype acuity) to define the spatial resolution of the perception, known as the minimal angle of resolution (MAR). These tests are performed as forced choice tests, similarly to the clinical routine in natural vision. Computer-based visual function tests such as the Freiburg Acuity and Contrast Tests (FrACT) (Bach, 1996), the Basic Assessment of Light and Motion (BaLM) (Bach et al., 2010) and the Basic Assessment of Grating Acuity (BaGA) (Wilke et al., 2007) are often used to give quantitative measures of visual ability with a retinal prosthesis.

As in natural vision, grating acuity does not correspond exactly to the optotype acuity (Graf, 2004), and so the results from these tests cannot always be compared directly. Clinical experience from previous trials has shown that people wearing a retinal implant can more easily perform a grating acuity test than optotype reading such as Landolt C rings or letters (Stingl et al., 2015b; Stingl et al., 2017). The highest visual acuity reported in the literature to date, as measured with Landolt C-rings, was 20/546 (logMAR 1.43) with the Retina Implant AG Alpha AMS (Stingl et al., 2017). A suprachoroidal prosthesis from the Bionic Vision Australia Group enabled a measurable acuity of 20/8397 or logMAR = 2.62 (Ayton et al., 2014) using an automated grating acuity test (Bach, 1996). Reported grating acuities have ranged from 20/1260 using the epiretinal Argus II (Humayun et al., 2012) to 20/364 in Retinal Implant AG Alpha AMS (Edwards et al., 2018).

Reading letters (or even words) is still a challenge for artificial vision. However, in all clinical trials to date some of the participants have been able to read large letters, as long as the lighting and contrast were appropriate (Zrenner et al., 2011; da Cruz et al., 2013; Shivdasani et al., 2017).

The other commonly reported laboratory or clinically assessed attributes of visual function are visual fields (perimetry) and motion detection. Due to a number of technical limitations and considerations, it is not easy to perform perimetric testing on a participant with a retinal implant. However, one group did show that the retinal sensitivity for the perimetric stimulus increased in a single patient after Argus II implantation (Rizzo et al., 2015). Motion detection is possible with the Argus II epiretinal implant (Humayun et al., 2012) as well as with the Alpha IMS and AMS subretinal devices (Stingl et al., 2015b; Stingl et al., 2017). A single patient using the Alpha IMS device could correctly recognize a speed of 35 degrees per second, which corresponds to a car moving at 22 km/h at a distance of 10 m (Stingl et al., 2015b).

In addition to measures of “visual function” such as acuity, fields and motion detection, measures of “functional vision” are also required for an accurate depiction of a person’s abilities with a retinal prosthesis. The most common of these are measurements of orientation and mobility (O&M) and activities of daily living (ADLs). Reports of O&M and ADL with retinal implants range from object localization in a laboratory setup to qualitative documentation of real-world visual experiences in daily life (Humayun et al., 2012; da Cruz et al., 2016; Zrenner et al., 2017). With the Argus II epiretinal implant, participants are able to localize doors, follow a line along the floor, follow a “shoreline” along the pavement and sort grey/white/black clothing (Humayun et al., 2012; Dagnelie et al., 2017a). A standardized tool for functional vision assessment has been developed, known as the Functional Low-Vision Observer Rated Assessment (FLORA), which includes a self-report section, a list of functional vision tasks for observation and a case narrative summary (Geruschat et al., 2015). Use of the FLORA in participants with the Argus II system showed common functional benefits including locating lights and windows and avoiding obstacles (Geruschat et al., 2016). The subretinal Alpha IMS and AMS devices also enabled participants to discriminate grey levels, localize, count, and to some extent discriminate the shape of various objects in both laboratory setups and in their daily lives. Additional self-reported benefits included participants describing seeing the outlines of houses, the shape of another person’s head or mouth, moving vehicles on the street, or the outline of their own hand (Stingl et al., 2015a).

As is evident from these descriptions, the measurement of vision outcomes with retinal prostheses to date has included a variety of test and reporting methodologies, with most observations of patient performance being from laboratory-based tasks.

A vital aspect to the assessment of patient outcomes in retinal prostheses is qualitative data capture on patients’ lived experience. For example, some patients will not be able to complete standardized laboratory-based tasks but may report being able to use the device at near-vision range to see mouth shapes (smiles), the presence or absence of spectacles on a person’s face, or being able to tell the difference between a glass of red wine and white wine (Stingl et al., 2013b). Others may report only a modest benefit in day-to-day visual function, such as identifying the location of light sources or edges of high-contrast doorways. Whatever the level of realized benefit, experiential learning should be encouraged and can often be assisted through the use of visual rehabilitation utilizing multidisciplinary teams.

14. Patient reported outcome (PRO) questionnaires

Clinical outcome measures, and even standardized functional performance measures (as described in the previous section) do not comprehensively capture the range of a person’s activities of daily living (ADLs). The prosthesis wearer’s subjective experience of benefit may not be captured by such measures, and is difficult to capture in a quantifiable way that allows comparison among wearers of a single retinal implant, let alone different prosthesis systems, or vision restoration attempts brought about through different approaches.

To articulate the prosthesis wearer’s real-world abilities, the use of standardized visual functioning questionnaires (VFQs) has been widely adopted in clinical trials, although typically as a supporting outcome measure. This supporting status has advanced to some extent since the implementation of Rasch analysis, a powerful analytical tool originally founded on item response theory in educational testing and later adopted by patient-reported outcomes (PROs) in rehabilitation medicine (Massof, 2002).

A common assumption when administering a PRO questionnaire is that all items fall along a common dimension of (visual) difficulty, and all respondents along the complementary dimension of (visual) ability. When Rasch analysis is applied to difficulty ratings of a single set of items by a large group of respondents, difficulty estimates – known as item measures – are obtained for the items and ability estimates – known as person measures – for the respondents, with confidence intervals for the estimates. In addition, Rasch analysis tests the validity of the unidimensionality assumption, and can provide estimates for items and respondents along several dimensions (e.g., central resolution and peripheral visual field extent) simultaneously (Massof et al., 2005).

Items in a PRO typically only cover a certain range of visual difficulty, e.g., from normal visual acuity (VA = 20/20) to profound visual impairment (VA = 20/500), and items are chosen so they cover that range in roughly equal intervals: A sparser item distribution reduces the precision and accuracy with which the person measure can be estimated. A PRO can only measure person abilities that fall in the range covered by the items: A respondent with 20/15 VA will rate all items as very easy, while one with bare movement perception will rate all items as impossible; the ability of either respondent falls outside of the range of the instrument, so a different instrument should be chosen.

Which PRO should be chosen to estimate functional vision in a particular clinical trial or study population depends on the level of visual functioning at baseline and the highest expected level during follow-up. For visual prosthesis studies, baseline vision may be as low as light perception, in the case of end-stage RP, or around the legal blindness limit, in the case of age-related macular degeneration; note however, that retinal prostheses in AMD seek to restore vision in the central scotoma to at best the level of native peripheral vision, in an area that had no functional vision at baseline. Using an instrument that can only measure functional vision down to the equivalent of VA = 20/400 would not be wise, since at best a few items in the instrument will become possible to do visually. Choosing an instrument that probes perception of crude shapes, movement, light projection, or the bare presence of light is much more appropriate in this case, since there will be many items that are facilitated by the function of the visual prosthesis. If the functionality of the prosthesis is such that some wearers reach the ceiling of the range, a more demanding PRO can be used on follow-up, to differentiate among participants’ performance levels.

A “pure” VFQ only measures visual ability, i.e., it frames all items in terms of “How difficult is it for you to…?” Yet many vision-related PROs also contain items exploring quality of life, or impact of vision loss. This is not a problem for the use of Rasch analysis, as long as it takes into account that items will scale along multiple dimensions, which may include eye health, general health, well-being, or anxiety. Often such instruments will categorize the items, and thus the responses, into so-called domains that do not necessarily coincide with the dimensions that emerge from Rasch analysis. In other words, rankings along such different domains may be highly correlated, and Rasch analysis can help to elucidate the underlying structure of so-called “latent variables.” (Massof and Ahmadian, 2007)

Table 2 provides examples of well-calibrated PROs that can and have been used in retinal prosthesis trials.

Table 2.

Patient-reported outcome measures (PROs) that have previously been used in retinal prosthesis clinical trials.

Patient-reported outcome (PRO)	Details
Ultra-low vision visual functioning questionnaire (ULV-VFQ) (Dagnelie et al. 2017b; Jeter et al.,2017)	This instrument was developed on the basis of extensive focus group interviews with patients whose bestcorrected vision did not allow them to see the largest letters on the chart at 0.5 m distance (VA < 20/1600), but who could at least tell the difference between day and night (Adeyemo et al., 2017). The instrument has demonstrated robust performance for both native ULV and prosthetic vision, and even for substitute vision in users of the Brainport tongue sensor (Dagnelie, unpublished observation). In addition to the original 150-item version of this questionnaire, shorter versions with 50 and 23 items are available
Forty-eight item Veterans Administration low vision visual functioning questionnaire (VALVVFQ-48) (Stelmack et al. 2004)	This instrument was developed for a population with moderate to profound vision loss, but with preserved form vision (20/70 – 20/800) and has been extensively used in low vision rehabilitation (Stelmack and Massof, 2007). While this instrument may be out of range for most visual prosthesis users, it is a good complement to the ULV-VFQ for those whose improvement puts them beyond the range of the ULV-VFQ
Impact of vision impairment questionnaire (IVI) (Lamoureux et al. 2006)	This instrument was designed to measure emotional and healthrelated effects of visual impairment as well as functional ability, and it has been used extensively in populations with mild to severe lo vision (20/70 – 20/400)
Impact of vision impairment – very low vision (IVI-VLV) (Finger et al. 2014b)	This adaptation of the IVI to populations with more profound visual impairment was developed specifically for use in a population that might participate in the Australian visual prosthesis trials. The range of this instrument may not be adequate for this purpose, since all but a few items rely on form vision, at levels that many current visual prosthesis users do not reach

In conclusion, the only instrument that has been demonstrated to be sensitive in the functional vision range of current visual prosthesis wearers is the ULV-VFQ. However, as the functionality of visual prosthesis systems improves, other PRO instruments may prove useful. The use of instruments that have been calibrated through Rasch analysis is highly recommended.

15. Conclusion

The field of retinal prostheses has developed rapidly over the past three decades, and advances have been significant. Within this time frame, we have moved from theoretical musings to the commercial availability of three retinal prosthesis devices, albeit not without commercial challenges. The two manufacturers with the largest patient base, Retina Implant AG and Second Sight Medical Products, have recently withdrawn their current offerings from the market, with the latter now conducting consumer testing of a cortical visual prosthesis to address a wider market. Despite the difficult road to commercial viability, over 500 people worldwide have benefited from the technology and retinal implants continue to be refined and commercialised. In addition, a number of additional benefits have come from the field of retinal prostheses, including the development of new patient-reported outcome measures and a better understanding of retinal anatomy and physiology, both in normal tissue and in degenerative disease

There are challenges ahead; next generation devices will need to provide improved visual outcomes, so that they may be used in people with more residual vision and/or higher functional requirements from retinal prostheses. This aim is being addressed in a number of ways. Firstly, hardware development is focusing on making smaller electrodes of materials that allow reduced perceptual thresholds. This includes work being completed on using biomimetic materials, so that the electronics can become physically and physiologically embedded within the retina. Software improvements are focusing on using advanced vision processing algorithms, and the use of depth and thermal cameras to better delineate the areas of interest in a visual scene. Simplifying the visual scene will provide better outcomes with the current resolution limitations of retinal prostheses.

Another area of investigation currently underway with retinal prostheses is the use of the technology as an earlier, neuroprotective treatment option. Previous work by scientists affiliated with Retina Implant AG (Germany) led to the development of transcorneal electrical stimulation (TES) as a potential mechanism for upregulating growth factors and protecting retinal cells in retinitis pigmentosa (Schatz et al., 2011). This resulted in a commercial product, known as OkuStim. Whilst a recent observational trial showed no significant improvement in visual function over six months with the treatment, the authors concluded that a longer duration trial may be needed to show possible benefits (Wagner et al., 2017). In parallel, other groups are investigating using implanted electrode arrays for the same neuroprotective purpose. For example, Abbott and colleagues showed that chronic low-level electrical stimulation using an implanted array preserved photoreceptor function in p23H-3 rats (Abbott et al., 2019).

A third future pathway for retinal prostheses will be utilising the electronic devices in combination with other treatment modalities, particularly cell therapies. As mentioned previously, there is evidence that chronic low-level electrical stimulation can increase growth factors in the retina, which could be beneficial for the survival and growth of cell therapies provided to the eye.

The commercial climate is tough – retinal prostheses need to prove both efficacy and commercial viability, which is hard for emerging technologies. As the field works together to better explore the biological mechanisms of vision loss and the optimizations we can provide through engineering, future devices will be better placed to help patients.

This review has provided discussion on many of the vital aspects of a retinal prosthesis; engineering, retinal physiology and degenerative remodeling, surgical techniques, measurement of psychophysics, visual function and patient reported outcomes, stimulation strategies and image processing. However, it would be remiss of us not to conclude this overview with acknowledgement of one of the most important indicators of device success – participant motivation and well-being. All clinical trials have been blessed with the participation of altruistic, motivated and strong individuals who freely volunteer their time for the studies and development of this new technology. To them, we give our sincere thanks.

HIGHLIGHTS.

• Retinal prostheses have now been implanted in over 500 people with profound vision loss.

• They are suitable for people with vision loss from outer retinal degenerative diseases, like retinitis pigmentosa.

• Current devices can allow users to localize high-contrast objects and assist orientation skills.