Reading with a Simulated 60-Channel Implant

Abstract

First generation retinal prostheses containing 50–60 electrodes are currently in clinical trials. The purpose of this study was to evaluate the theoretical upper limit (best possible) reading performance attainable with a state-of-the-art 60-channel retinal implant and to find the optimum viewing conditions for the task. Four normal volunteers performed full-page text reading tasks with a low-resolution, 60-pixel viewing window that was stabilized in the central visual field. Two parameters were systematically varied: (1) spatial resolution (image magnification) and (2) the orientation of the rectangular viewing window. Performance was measured in terms of reading accuracy (% of correctly read words) and reading rates (words/min). Maximum reading performances were reached at spatial resolutions between 3.6 and 6 pixels/char. Performance declined outside this range for all subjects. In optimum viewing conditions (4.5 pixels/char), subjects achieved almost perfect reading accuracy and mean reading rates of 26 words/min for the vertical viewing window and of 34 words/min for the horizontal viewing window. These results suggest that, theoretically, some reading abilities can be restored with actual state-of-the-art retinal implant prototypes if “image magnification” is within an “optimum range.” Future retinal implants providing higher pixel resolutions, thus allowing for a wider visual span might allow faster reading rates.

Introduction

Extensive efforts have been undertaken to develop artificial vision devices aimed at restoring some vision to blind individuals (Zrenner, 2002; Merabet et al., 2005; Degenaar et al., 2009). Most of these devices use electrical currents to stimulate the visual pathway, bypassing damaged structures. Briefly, these devices incorporate an image capture module (e.g., a camera), a processor that transforms the captured image into a pattern of stimulation currents, and an electrode array through which electrical currents are delivered to the target stimulation site (i.e., retina, optic nerve, or visual cortex). Among these, retinal prostheses are probably the most advanced approach as first clinical trials are currently underway (for an up-to-date-review refer to Chader et al., 2009).

The “artificial vision” elicited by electronic retinal prostheses will be fundamentally limited by two main factors: the inherent properties of the electrode-nerve interface and surviving neural tissue and the technical characteristics of the device. The first are the matter of extensive research efforts, both in animal studies and human clinical trials (see, e.g., Jensen and Rizzo, 2007, 2008; de Balthasar et al., 2008; Greenwald et al., 2009; Horsager et al., 2009, 2010; Tsai et al., 2009), but remain poorly understood. In contrast, the technical constraints associated with the concept of retinal prostheses might be readily identified: (1) low-resolution limited by the discrete number of stimulation contacts in the device and (2) the fact that all percepts will be located in a fixed and restricted portion of the visual field corresponding to the area of the retina stimulated by the electrode array. Our group has conducted a series of studies aimed at determining the visual information required to perform a variety of tasks under such restricted viewing conditions. They were based on computer simulations carried out on subjects with normal vision (Sommerhalder et al., 2003, 2004; Perez Fornos et al., 2005, 2008; Pelizzone et al., 2006; Perez Fornos, 2006). These experiments demonstrated that about 500 distinct phosphenes, distributed on a 10° × 7° central retinal area could restore significant reading abilities to blind patients (i.e., reading accuracy >95% and reading rates of 60–75 words/min). Another study using simulations of artificial vision on normally sighted volunteers (Dagnelie et al., 2006) showed that accurate paragraph text reading (>90%) could be achieved with a 16 × 16 retinal implant with 30% phosphene dropout, at reading rates around 30 words/min.

Today, the first prototypes of retinal prostheses have become available for human testing and clinical trials have been launched. For example, the Argus II™ implant of Second Sight^® Medical Products, Inc¹. (Sylmar, California, USA) contains 60 electrodes. The Iris™ implant of IMI Intelligent Medical Implants, GmbH² (Bonn, Germany) contains 49 electrodes. Since these numbers are 5–10 times lower than the number of contacts (pixels) used in the psychophysical studies cited above, we decided to re-examine reading capabilities while simulating the real technical constraints of present prototypes. Our main purpose was to determine the upper limit (best possible) reading performance attainable with such a device, by assuming that all the information delivered by the implant reaches the brain (i.e., that there is no loss and/or distortion of the “electric image” at the electrode-nerve interface). In addition, our previous studies examining eye–hand coordination and mobility tasks (Perez Fornos, 2006; Perez Fornos et al., 2008) revealed that each task has different requirements in terms of spatial resolution or magnification (“zoom”). In other words, not only the actual number of points or pixels available to decode the image but also the content of the image (effective visual field size) represented by these pixels is crucial to achieve the best possible performance for each task. We therefore also explored which would be the optimum viewing conditions for the reading task given the technical constraints imposed by a 60-channel retinal implant.

Materials and Methods

Subjects

Four subjects (S1, female, 31 years old; S2, male, 45 years old; S3, male, 29 years old; S4, male, 38 years old), part of the research staff of the Geneva University Hospitals and familiar with the purpose of the study, participated in the experiments. They were fluent French speakers and readers, had corrected visual acuity of 20/20 or better, normal or corrected to normal vision, and normal ophthalmic status.

Subjects signed appropriate consent forms. Experiments were conducted according to the ethical recommendations of the Declaration of Helsinki, and were approved by local ethical authorities³.

Experimental setup and procedure

We measured reading performance of normal subjects on a setup simulating conditions of prosthetic vision (Figure 1). Subjects were seated in front of the stimulation screen (22′′ computer monitor, ELSA Ecomo 22H99), at a distance of 57 cm, set to a resolution of 800 × 600 pixels (the stimulation screen subtended 40° × 30° of visual field, and 1° on the screen corresponded to 20 screen pixels). They had to navigate and read full-screen pages of text (Figure 1A) using a low-resolution (60 pixels⁴, 256 gray levels) restricted viewing window. The remaining background was gray (average luminosity of the text page, see Figure 1B). The viewing window was stabilized in central vision, its position, and content being updated according to the subject's gaze position (sampled at 250 Hz with a high-speed video eye and head tracking system; Eyelink I; SensoMotor Instruments GmbH, Berlin, Germany). A simple block-averaging algorithm (square, real-time pixelization) was used to produce the low-resolution text images. Briefly, this algorithm merges a group of N × N original screen pixels into a larger stimulation pixel with a uniform gray level equivalent to the average of the original screen pixels. This image-processing algorithm is described in detail in Perez Fornos et al. (2005). A detailed description of the experimental setup can be found in previous publications (Sommerhalder et al., 2003, 2004).

Figure 1.

Experimental setup and procedure. Subjects had to read pages of text visible only through a low-resolution, restricted viewing window. The viewing window moved following the subject's center of gaze, and its content was updated accordingly. (A) Example of one of the text pages read by the subjects. (B) Illustration of the stimulation screen as viewed by the subject during the experiments.

All experiments were carried out monocularly (dominant eye) and in central vision. For each experimental trial, subjects had to read aloud four pages of a text extracted from the Swiss newspaper Le Temps. This newspaper is written in common language; its contents are of general interest and average difficulty. A single page of text contained seven lines and approximately 25 words. The font type used was Arial and font size was fixed, so that the height of the lowercase letter “x” subtended 1.8° of visual angle. During the experiment, the examiner noted the total number of errors per trial. Total time required for each trial was also recorded.

The spatial resolution seen by the subject was systematically varied to mimic the use of an adjustable magnification factor in which more or less characters were visible at once (as subjects do naturally by moving their head toward or away from a visual target). Stimulation pixel sizes of 0.7°, 0.6°, 0.5°, 0.4°, 0.3°, and 0.2° were tested. This resulted in spatial resolutions of 2.6, 3, 3.6, 4.5, 6, and 9 pixels/char. Experiments were performed using two different viewing window orientations: horizontal (10 × 6 pixels) and vertical (6 × 10 pixels). Figure 2 shows samples of pixelized portions of text in all tested conditions. Note that varying spatial resolution and keeping font size fixed resulted in viewing windows of different sizes in each condition. This is obviously not fully representative of a retinal prosthesis where device size is fixed and character size can vary. However, it had to be implemented that way because of technical limitations (e.g., to present several lines of text on the screen at very large character sizes would have been impossible; see Discussion).

Figure 2.

Examples of text portions presented in the viewing window at different spatial resolutions. All viewing windows contain 60 pixels: (A) 10 × 6 pixels (horizontal viewing window), (B) 6 × 10 pixels (vertical viewing window). The spatial resolutions are indicated below each stimulus. The corresponding portion of non-pixelized (screen pixels) text is presented above each pixelized image. Note that varying spatial resolution resulted in more or less characters being represented at once in the 60-pixel viewing window.

Each subject performed three trials (four text pages each) per experimental condition. A new text was selected out of a pool of 100 for each trial (none of the subjects read the same text twice). To minimize order effects, the testing order was determined using a Latin Square⁵ (see Table 1). At the end of the main experiment, in order to assess reading performance including possible learning effects, subjects performed three additional trials at the optimum spatial resolution determined in the main experiment, using both viewing window orientations.

Table 1.

Order of testing conditions for each subject.

Subject	1st orientation tested	2nd orientation tested	Spatial resolution sequence
S1	V	H	Increasing
S2	H	V	Decreasing
S3	H	V	Increasing
S4	V	H	Decreasing

Subjects started either with a horizontal (S2, S3) or vertical (S1, S4) viewing window orientation. Within each orientation, the order in which spatial resolutions were presented was either increasing (2.6–9 pixels/char) or decreasing (9–2.6 pixels/char).

Data analysis and statistics

Performance was measured in terms of reading accuracy (% of correctly read words transformed to rationalized arcsine units (RAU) for statistical analyses; see Studebaker, 1985) and reading rates (words/min).

Results for each experimental condition are expressed as values ± SEM. Statistically significant effects of spatial resolution and viewing window orientation were determined using two-factor, repeated measures analysis of variance (ANOVA). Post hoc comparisons were done using the Tukey HSD test. Paired Student's t-tests were used to determine statistically significant differences in final reading performance between viewing window orientations.

Results

Figure 3 presents mean reading performance versus spatial resolution for each subject. Maximum reading performances were reached at spatial resolutions between 3.6 and 6 pixels/char. Performance declined drastically for all subjects at low spatial resolutions (<3.6 pixels/char). For most subjects performance (mainly reading rates) also declined at higher spatial resolutions (>6 pixels/char). Within the optimum spatial resolution range, reading accuracy was perfect or close to perfect for all subjects (except S2 for the horizontal viewing window). While reading rates also peaked in this range, they showed more individual variability.

Figure 3.

Individual reading performance versus spatial resolution (pixels/char) for four subjects (three trials per condition). Left panels: reading accuracy expressed in RAU ± SEM (left axes) and in % (right axes). Right panels: reading rates (words/min) ± SEM.

Figure 4 presents mean reading performance across all subjects. Maximum reading accuracy is reached and reading rates peak at a spatial resolution of 4.5 pixels/char for both viewing window orientations. Mean reading accuracy and mean reading rates dropped at the two lowest and at the highest spatial resolutions tested. Results were quite similar for both vertical and horizontal viewing window orientations, except that reading rates (Figure 4, right) tended to be higher for the horizontal viewing window.

Figure 4.

Mean reading performance for all subjects versus spatial resolution (pixels/char). Results were calculated across all the three trials performed in each condition. The gray bars correspond to the optimum resolution range found in previous studies (Sommerhalder et al., 2003, 2004; Perez Fornos et al., 2005; Dagnelie et al., 2006). Left panel: Reading accuracy expressed in RAU ± SEM (left axes) and in % (right axes). Right panel: Reading rates (words/min) ± SEM.

There was a statistically significant effect of spatial resolution on reading accuracy (ANOVA: F_5,36 = 20.97, p < 0.0005). Neither the orientation of the viewing window nor the interaction effect were statistically significant. Reading rates were significantly affected by both parameters (ANOVA: spatial resolution F_5,36 = 10.84, p < 0.0005; viewing window orientation F_1,36 = 5.29, p = 0.02) but not by their interaction. Results of post hoc comparisons of spatial resolution are summarized in Table 2.

Table 2.

Results of post hoc multiple comparisons (Tukey HSD test) of the effects of spatial resolution on reading performance.

			P
Spatial resolution	Mean	SEM	2.6	3	3.6	4.5	6	9
READING ACCURACY (RAU)
2.6	41.09	5.39		0.029	<0.0005	<0.0005	<0.0005	<0.0005
3	69.87	9.85	0.29		0.0003	<0.0005	<0.0005	0.038
3.6	106.44	4.22	<0.0005	0.003		0.976	0.989	0.919
4.5	112.99	3.41	<0.0005	<0.0005	0.976		1.000	0.524
6	111.96	3.68	<0.0005	<0.0005	0.989	1.000		0.598
9	97.66	6.68	<0.0005	0.038	0.919	0.524	0.598
READING RATES (WORDS/MIN)
2.6	5.58	1.13		0.555	<0.0005	<0.0005	0.001	0.525
3	10.57	2.55	0.555		0.022	0.002	0.062	1.000
3.6	20.48	2.12	<0.0005	0.022		0.953	0.998	0.025
4.5	23.05	2.60	<0.0005	0.002	0.953		0.791	0.002
6	19.23	2.44	0.001	0.062	0.998	0.791		0.070
9	10.71	1.38	0.525	1.000	0.025	0.002	0.070

Significant p-values (<0.05) are highlighted in bold.

Reading under such experimental conditions was unnatural for normal subjects. Consequently, individual results in a given condition might have been biased by adaptation to the task (i.e., subjects are likely to perform better toward the end of the study). Indeed, such an effect could be observed on the results of the main experiment (see Table 1; Figure 2): for the first measured viewing window orientation subject S2 reached lower reading accuracy, subjects S1 and S4 lower reading rates. Thus, at the end of the main experiment we replicated measurements at the “optimum” spatial resolution of 4.5 pixels/char for both viewing window orientations. These results are presented in Figure 5. All subjects achieved perfect or close to perfect reading accuracy with both viewing window orientations (100% in most trials), with no significant difference between conditions. Final reading rates were significantly higher with the horizontal than with the vertical viewing window (paired t-test: t₃ = 7.7, p = 0.005; horizontal mean = 33.9, SEM = 3.6; vertical mean = 26.5, SEM = 3.1).

Figure 5.

Reading performance measured at the end of the experiment using the “optimum” spatial resolution (4.5 pixels/char). Mean individual results for four subjects were calculated on the basis of three trials per condition. Results obtained with vertical (red bars) and horizontal (blue bars) viewing window orientations are compared. Mean initial performance for each subject in the same condition is also presented for comparison (gray dots; see Figure 3). Left panel: Reading accuracy expressed in RAU ± SEM (left axis) and in % (right axis). Right panel: Reading rates in word/min ± SEM.

Discussion

Reading is strongly associated with vision-related estimates of quality of life, and it represents one of the main goals of low vision patients seeking rehabilitation (Wolffsohn and Cochrane, 1998; Hazel et al., 2000; McClure et al., 2000). It is therefore important to evaluate potential reading capabilities for present prototypes of retinal prostheses.

Our results show that, theoretically, some reading abilities could be restored with a 60-electrode retinal prosthesis, similar to devices currently used in clinical trials (Richard et al., 2007; Humayun et al., 2010). If the brain can grasp all the information transmitted by the implant, subjects could be able to read at rates between 20 and 35 words/min. This is much slower than the average 250 words/min achieved by subjects with normal vision (Legge et al., 1985). It is also slower than Braille reading, which allows reading rates of about 100 words/min (Mousty and Bertelson, 1985; Knowlton and Wetzel, 1996; Lorimer, 1996). Yet, the maximum reading abilities predicted in this study might prove to be very useful to patients when alternatives are not available (e.g., reading wall signs). Our results are encouraging in this respect.

This study also demonstrates that reading performance is “optimum” at a given spatial resolution range. For reading, it will be crucial to adapt the text size (magnification) to obtain approximately 4.5 pixels/char (i.e., 1–2 characters on the whole surface of a 60-contact array). These results agree with those of our previous studies (gray bars in Figure 4; Sommerhalder et al., 2003, 2004; Perez Fornos et al., 2005), where best reading performance was obtained down to spatial resolutions of about 4.8 pixel/char. They are also in good accordance with studies of Dagnelie et al. (2006), which demonstrated a significant effect of character size on reading performance and suggested a spatial resolution of 4 pixels/char. A simple way for retinal prosthesis wearers to optimize the spatial resolution of the text to be read would be to move the image capture device (i.e., the camera) closer to the text. Additionally, it might also be useful to incorporate user-controlled magnification (“zoom”) to the devices for the cases when approaching the camera close enough to the target is not possible (e.g., wall signs).

If performance peaks in a narrow range of spatial resolution, this also means that it is impaired outside this range. Why? Either resolution is too low for adequate character recognition or characters become too big to be entirely seen in the small visual field provided by the implant (see Figure 2). For a 60-channel implant, only 1–2 characters can be seen at once at the optimum spatial resolution. This means that the effective visual span of the device needs to be very restricted for characters to be accurately recognized. It is well established that reading rates are considerably impaired with small visual spans, and that they improve with wider visual spans up to about 8–10 characters (Beckmann and Legge, 1996; Fine et al., 1996). Therefore, one possibility to improve reading rates in future retinal prostheses would be to provide larger visual spans, so that more characters could be simultaneously represented at the spatial resolution of 4–5 pixels/char. This could be achieved by increasing the electrode density on the device and/or by increasing the implant size. The fact that horizontal implant orientations yielded significantly faster reading rates in this study is consistent with this analysis.

An additional interesting outcome of these results is that they raise caution about using automated standard clinical tests to evaluate the “vision” of actual retinal implant wearers. Most clinical tests are designed assuming a “step-like” (i.e., psychometric) behavior of performance when varying a given parameter. For example, in the standard visual acuity tests used clinically (e.g., ETDRS), patients are presented with shapes of decreasing size until recognition accuracy drops below a certain criterion. The test is stopped at that point and the last shape size accurately recognized is used to calculate visual acuity. Yet, the results of this study suggest that, for retinal prostheses wearers, the testing procedure should detect an “optimum” acuity or an “optimum range” of acuity levels instead of the “threshold level” used in clinical tests with “natural vision” observers. In conclusion, before using standardized automated testing algorithms with retinal implant wearers, the assumptions on which they have been developed need to be questioned in order to avoid erroneous results because of the competing technical constraints previously mentioned (limited resolution and restricted visual field).

The use of simulations of artificial vision to evaluate the potential rehabilitation prospects of retinal prostheses has been a source of debate and our experimental setup has some limitations. Both deserve some discussion.

The advantages of simulation studies such as the one presented here have been recognized by others and used to address specific questions (Pelli, 1987; Cornelissen and Van den Dobbelsteen, 1999). On one hand, by using normal observers with a simulated impairment one can look at the effect of a single parameter without biasing its effect with that of others. On the other hand, the use of simulations makes it easy to repeat experiments within a single subject and a given parameter can be varied over a full range. Similar simulation work in the field of hearing via cochlear implants (see, e.g., Friesen et al., 2001) was extremely profitable.

The simulations presented here do not attempt to exactly mimic percepts elicited by retinal implants. Our approach allows the establishment of a theoretical upper limit of performance by assuming that all the information provided by the device reaches the brain (i.e., that there is no additional loss of information at further stages, such as the electrode-nerve interface). In addition, they rely on certain simplifications.

First, the 60-pixel images used as stimuli in these experiments were generated using an image-processing algorithm which decomposes the original image into a smaller subset of square uniform pixels. This squared form is certainly not what retinal prosthesis wearers might experience, so in a previous study (Perez Fornos et al., 2005) we compared performance between images processed using this simplified algorithm with a more physiologically plausible model (dots with Gaussian brightness distribution). We found no difference in performance when real-time processing algorithms were used.

Second, electrode spacings of current epiretinal prototypes are substantially larger than pixel sizes used in our simulations; for example, for a 6 pixels/char resolution, each stimulation pixel subtended 0.3° and correspondingly the entire viewing window subtended a visual field of 3° × 1.8°. Furthermore, the different spatial resolutions used in these experiments were generated by varying pixel size, and keeping font size of the texts fixed (see Figure 2). This is the inverse of what happens in retinal prostheses, where size is fixed. It was technically easier to implement it this way in our simulations, and this simplification is not likely to have had a significant effect on reading performance since the information content of the stimulus is the same and tests were performed in central vision on subjects with normal vision. If spatial resolution was enough for characters to be identified, reading scores were perfect or close to perfect. As long as stimulation pixel sizes remain largely above the visual acuity level of subjects, reading scores are unlikely to be influenced by pixel size. In addition, previous studies in normally sighted subjects have shown that reading rates decline only slightly for large character sizes beyond the optimum range of 0.3°–2° (Legge et al., 1985). Therefore, at most, reading rates with a more realistic, larger stimulation pixel size would have been slightly lower.

Third, epiretinal devices currently in clinical trials use head-mounted cameras to capture the visual scene, which means that, for reading, subjects would have to scan pages of text via head movements, and not eye movements as used in our simulations⁶. Even if there could be some advantage of eye-controlled page navigation in normal reading, studies with low vision patients have demonstrated that the severe visual factors to which they are subject hamper reading performance to a degree such that the advantage/disadvantage of a given page navigation method is reduced (Beckmann and Legge, 1996). Given the stringent technical limitations depicted in our simulations (very low-resolution and very limited visual span), eye movement scanning strategy was probably far from optimal and any possible advantage compared to head movements would be diminished.

We have further experimental evidence to supporting the two above mentioned assumptions. We conducted an additional experiment with two subjects (S1 and S2, the ones with the best and worst performance in the experiments) using a stimulation pixel size of 2° and a very large font size, in which the height of the lowercase letter “x” measured 9.5° (corresponding to the “optimum” 4.5 pixels/char resolution). We used mouse-controlled page navigation⁷, where subjects had to navigate across the pages of text with the computer mouse (the viewing window being still stabilized in a fixed region of the retina). Both subjects achieved perfect reading. Reading rates were only slightly lower than in the “optimum” condition from this study (0.4° pixel size, horizontal orientation, eye-controlled navigation): 33(±1.8) words/min for S1 and 22(±2.6) words/min for S2, but these differences were not statistically significant. Altogether, this evidence supports the idea that, in our results, the main factors limiting reading performance were the reduced resolution and limited visual field depicted in our simulations. Pixel size and page navigation method could only have had a small, non-significant influence on performance.

In summary, despite the methodological limitations mentioned above and although the transmission of visual information by a real retinal prosthesis will certainly not be perfect, this study helps establish the theoretical upper limit of reading performance that could be achieved with a 60-channel retinal implant given its technical constraints. We believe that simulation experiments such as the one presented here provide knowledge of primary importance in the design and understanding of prosthetic devices.

Conclusion

The investigation presented here shows that a 60-channel retinal implant might be able to restore some reading abilities to blind patients. Indeed, early results from clinical trials suggest that a few blind patients wearing a 60-channel retinal implant were able to recognize letters and even short words (da Cruz et al., 2010; Stanga et al., 2010). However, the optics of the system will have to be customizable, to adapt to the task at hand. In the case of reading, the optimum visual span for this device seems to be very small: 1–2 characters. To achieve faster reading rates, future devices have to provide wider visual spans while containing enough electrodes to maintain the optimum spatial resolution required for this task. Finally, an interesting outcome of this study is that the competing technical constraints of retinal prostheses result in an “optimum range” for a given parameter (in our case character size) to achieve best performance, instead of the “threshold levels” measured in many clinical psychophysical evaluations.

Conflict of Interest Statement

This work was presented at the Association for Research in Vision and Ophthalmology (ARVO) congress (Sommerhalder et al., 2008). The authors take part in the multicenter clinical trial of the Argus II™ implant of Second Sight^® Medical Products, Inc. (www.clinicaltrials.gov NCT00407602). However, the study presented here was carried out before the beginning of this collaboration.