Frequency and Amplitude Modulation Have Different Effects on the Percepts Elicited by Retinal Stimulation

To restore functional form vision, epiretinal prostheses have been implanted in blind human subjects to electrically elicit percepts. These findings suggest that frequency modulation may be the best way to produce percepts that range widely in brightness while minimizing loss of spatial resolution.

Abstract

Purpose.

In an effort to restore functional form vision, epiretinal prostheses that elicit percepts by directly stimulating remaining retinal circuitry were implanted in human subjects with advanced retinitis pigmentosa RP). In this study, manipulating pulse train frequency and amplitude had different effects on the size and brightness of phosphene appearance.

Methods.

Experiments were performed on a single subject with severe RP (implanted with a 16-channel epiretinal prosthesis in 2004) on nine individual electrodes. Psychophysical techniques were used to measure both the brightness and size of phosphenes when the biphasic pulse train was varied by either modulating the current amplitude (with constant frequency) or the stimulating frequency (with constant current amplitude).

Results.

Increasing stimulation frequency always increased brightness, while having a smaller effect on the size of elicited phosphenes. In contrast, increasing stimulation amplitude generally increased both the size and brightness of phosphenes. These experimental findings can be explained by using a simple computational model based on previous psychophysical work and the expected spatial spread of current from a disc electrode.

Conclusions.

Given that amplitude and frequency have separable effects on percept size, these findings suggest that frequency modulation improves the encoding of a wide range of brightness levels without a loss of spatial resolution. Future retinal prosthesis designs could benefit from having the flexibility to manipulate pulse train amplitude and frequency independently (clinicaltrials.gov number, NCT00279500).

Retinitis pigmentosa (RP) and age-related macular degeneration (AMD) are photoreceptor diseases that cause vision loss in more than 15 million people worldwide.¹ Both disorders are initially characterized by gradual photoreceptor loss, but in later stages of the disease, this photoreceptor loss is accompanied by significant remodeling of the inner retina.^2–4 Within bipolar and amacrine cells, there is a significant loss of cells that is accompanied by significant changes in connectivity.^2,5–7 Within ganglion cell layers, there is also significant loss of cells; however, their morphologic structure and the connections to the optic nerve seem to be relatively well maintained.^6–9 The lack of a proven clinical treatment for RP and AMD has given rise to several experimental vision therapy technologies. Proposed solutions include optogenetic modification that transforms remaining retinal cell into light-sensitive cells,^10–12 a variety of gene replacement therapies,¹³ and epiretinal^14–16 and subretinal^17–19 prostheses.

Since 2000, two separate clinical trials of a chronic retinal prosthesis have been conducted.^14,20 These devices electrically stimulate the inner retinal ganglion cell layer with a microelectrode array implanted in proximity to the retina, and visual phosphenes are elicited via electrical stimulation.¹⁵ Obviously, a critical factor in determining the quality of the vision produced by prostheses is likely to be the ability to control the appearance of the individual phosphenes. Ideally, retinal stimulation would have the capacity to target individual ganglion cells (including specific subtypes of the approximately 20 different types of ganglion cell²¹) and would be capable of producing activation patterns within these cells that match the spatiotemporal activity of the normal retina. However, such cellular resolution is far beyond the abilities of current technology. Current chronic and semichronic devices contain square (50 × 50 μm) electrodes of or quadruple square (120 × 120 μm) electrodes; a semichronic array recently implanted by Retina Implant AG, Reutlingen, Germany), 220-μm diameter disc electrodes (the Argus II device; Second Sight Medical Products, Inc., Sylmar, CA) and 260- to 520-μm-diameter disc electrodes (Argus I; Second Medical Products, Inc.) described in this article. Thus, in all these devices, a single electrode simultaneously activates hundreds to thousands of cells with a wide variation of structure²¹ and function.²²

To create form vision, multiple electrodes must be stimulated in concert. Present technology uses light sensors, either in an external camera or as part of the implant, to capture an image. Each electrode will stimulate the retina in accordance with the amount of light detected by the corresponding area of the image sensor. Currently, the most common approach toward representing the visual world through phosphenes is to assume that any given electrode produces a phosphene that represents a relatively constant region of visual space and that only the apparent brightness of that phosphene varies as a function of the stimulation pattern. This approach can be thought of as having the goal of creating an image much like a gray-scale digital scoreboard, where each electrode can be thought of as a pixel that varies only in brightness.

Over the past several years, our group has performed a series of experiments quantifying the relationship between stimulation and percept for a single electrode or pair of electrodes. This work has shown that perceptual thresholds are influenced by the proximity of the electrodes to the retina surface^23,24 and that both threshold and brightness for a single electrode or pairs of electrodes can be predicted across a variety of parameters, such as frequency, pulse duration, and amplitude.²⁵ In particular, it has been shown that increases in both current amplitude and stimulation frequency result in an increase in percept brightness.^25,26 A cortical visual prosthesis group has also reported an increase in percept brightness with an increase in either stimulation frequency or current amplitude.²⁷

However, these studies relied primarily on either threshold or brightness-matching judgments and did little to examine how the shape of elicited percepts varies as a function of stimulus amplitude and frequency. Previous clinical visual prosthesis studies reporting on phosphene shape were largely anecdotal, did not systematically repeat multiple trials for a given stimulus, and did not compare the shapes of percepts produced across a variety of pulse trains.^28–32 A preliminary study by our group showed that the size of the phosphenes increased nonuniformly across subjects with current amplitude.³³

In the present study, changes in stimulus amplitude and frequency had separable effects on the shape of elicited percepts. As a consequence, it may be possible to develop stimulation protocols for encoding visual images that use a combination of frequency and amplitude coding to independently manipulate the size and brightness of phosphenes, thereby providing an increased flexibility that may improve the ability of these prostheses to represent the visual world.

Methods

Subjects

Data reported in this study were collected on a single subject who had a chronically implanted 16-electrode retinal prosthesis (Second Sight Medical Products, Inc.). The subject (tested at the University of Southern California), along with five other patients, was implanted with a first-generation, 16-channel device (A16 system) in 2004 as part of a phase 1 safety trial that began in 2002. The other five patients were excluded from this study for a variety of reasons, as described previously.³⁴ The subject in this study had been blind for 10.5 years before implantation at age 55, with no light perception (NLP) in the implanted eye. Subjects implanted with the second-generation 60-channel device (Argus II) were not available for extensive testing during the time of this study.

The study protocol was approved by Keck School of Medicine at the University of Southern California Institutional Review Board after the subject's informed consent was obtained, and the protocol complied with the Declaration of Helsinki.

System

The Second Sight Medical Products epiretinal prosthesis contains both intraocular (electrode array) and extraocular (e.g., glasses, video processing unit) components. As described previously,^24,25 the intraocular array implanted epiretinally in the macular region of the retina consists of 16 platinum disc electrodes in a 4 × 4 arrangement contained within a clear silicone rubber platform (Fig. 1A). The array is held in place with a custom retinal tack. The 16 electrodes implanted in this subject are 260 or 520 μm in diameter (subtending 0.9° and 1.8° of visual angle, respectively) and are arranged in an alternating checkerboard pattern spaced 800 μm center to center. Custom software on a laptop computer is used to program the external video processing unit (VPU), which in turn sends stimulus commands to the implant. Power and signal information are sent from the VPU through a wire to an external transmitter coil that is attached and aligned magnetically and coupled inductively (i.e., wirelessly) to a secondary coil that is implanted subdermally in the subject's temporal skull behind the ear. The secondary coil provides power and signal information to an implanted pulse generator (IPG), which decodes the signal and produces the commanded stimulus pulses. The IPG transmits pulses to the array of electrodes via a multiwire cable that traverses the sclera (Fig. 1B).

Figure 1.

The prosthesis system. (A) Overview of the implant, (B) 4 × 4 electrode array, and (C) biphasic pulse train. Pulse trains were varied by changing either pulse amplitude (dashed arrows) or pulse frequency (solid arrows).

Psychophysical Methods

Control Task: Tactile Drawing.

Our experiments relied on our subject's ability to draw percepts accurately and consistently across trials. However, our blind subject had been without tactile–visual feedback for more than 15 years; and, as a result, he was likely to show more variation in drawing than a blindfolded sighted subject would show. We therefore began by performing a control experiment with tactile targets to compare his error in drawing tactile shapes with those of the control subjects. In these control drawing experiments, the test stimuli consisted of a set of 11 tactile shapes made of felt with a cardboard background. Subjects were asked to feel the felt shapes and then draw them on a board. Head movement was minimized with a chin rest.

After each stimulus presentation, the subject outlined the shape on a grid screen (containing 6-in. horizontal and vertical gridlines) with a center location aligned horizontally and vertically with the subject's head. Drawing was performed with a pen that had a cap of a different color from the pen. A head-mounted camera (CMOS S588-3T; Misumi, Tokyo, Japan), located on the subject's glasses, was used to record the trials to a digital video recorder (DVR). Video files were analyzed off-line to extract shape data using custom built tracking software. In the first stage of processing, the entire image was rotated appropriately using the grid screen background as a reference. In the second stage, vertical and horizontal gridlines and the distance from the subject to screen were used to set a new coordinate system in visual angle co-ordinates (since the subject was 16 in. from the screen, 4 gridlines = 24 in. corresponded to 73.8° of visual angle). In the third stage, the location of the pen cap was tracked (based on its color) across each frame of the video file. Finally, a binary-shaped data file was built from pen cap coordinate locations across all frames.

Binary images from each drawing trial were described using four shape descriptors: area, major and minor axis lengths, and orientation. The area of each shape was obtained from the 0th geometric moment (number of non-0 pixels in the image), whereas the orientation and the lengths of major and minor axes were calculated from the eigenvalues and eigenvectors of the centralized moments (fitting an ellipse to the shape and measuring the length of the longest and shortest axis to this ellipse). From these, we were also able to obtain the minor-to-major axis ratio, indicating degree of shape elongation. The orientation of the shape was simply the angle of the longest eigenvector.

Trial-by-trial variability across these four descriptor values was calculated as the standard deviation across trials. For area and major and minor axis lengths, the standard deviation was calculated as a percentage, by normalizing to the mean value of that descriptor. A large standard deviation implied that shapes varied widely on a trial-by-trial basis for a particular descriptor, whereas a small standard deviation indicated that there was little variability across trials.

Retinal Stimulation.

In the retinal stimulation experiments, stimuli were charge-balanced, 0.45 ms/phase cathodic-first biphasic pulse trains that were always 500 ms in duration. Pulses were charge balanced across cathodic and anodic pulses for safety reasons. Each pulse train was presented on a single electrode, and nine electrodes were tested in total. Chosen electrodes had the greatest dynamic range. All data were recorded under photopic conditions.

Phosphene shape and brightness were manipulated in two ways. In the modulated-amplitude condition, we modulated current amplitude between 1.2× and 6× threshold (threshold was defined in a separate experimental run with a 20-Hz pulse train and using a method of adjustment procedure described previously²⁴), while holding the frequency constant at 20 Hz. In the modulated-frequency condition, the frequency of the pulse train was varied between 13 and 120 Hz, while current amplitude was held constant at 1.25× threshold (Fig. 1C).

Phosphene shape for both the modulated-amplitude and -frequency conditions was measured by using methods analogous to those described above for the tactile shapes. Phosphene brightness was measured with brightness rating procedure I, in which the subject compared the brightness of the phosphene to a reference stimulus.³⁵ The subject was explicitly instructed to rate the apparent brightness independently from the apparent size of the phosphene. For a given electrode, the reference stimulus was the same for both modulated-amplitude and -frequency conditions.

Shape and brightness judgments were conducted in separate runs. Within a run, each frequency/amplitude was presented 5 to 10 times in random order among other test stimuli that varied in either amplitude or frequency. For each electrode, we measured responses for 10 different pulse trains (six frequencies at 1.25× threshold, and five amplitudes at 20 Hz). Modulated-amplitude and -frequency conditions therefore contained a single-pulse train (1.25× threshold at 20 Hz) that was common to both conditions. Data were collected from 970 trials. In a small percentage of the trials (1.33% for brightness runs, 5.38% for shape runs), when stimulation was near threshold, no phosphene was seen. On brightness runs, we recorded a brightness of 0. On shape runs these trials were excluded. In total, we collected 450 trials of brightness data (since all trials were included) and 520 trials of shape data (from which 28 trials were excluded).

Results

Control Tactile Drawing Experiment

Visual inspections of the drawings suggested that our subject differed in drawing capability between compact (minor axis length >50% major axis length) and elongated shapes (minor axis length <50% major axis length). Thus, we subdivided our tactile data into these two shape groups. As described above, the shapes were classified in terms of their area, major and minor axis lengths, and orientation. For all shape descriptors, we calculated both drawing bias—the difference between the tactile target and the mean shape drawings of that tactile target—and variability—the differences across repeated drawing trials for a given tactile target. Data for area and orientation are described in the text; data for all descriptors are shown in Table 1.

Table 1.

Drawing Variability for the Tactile Control Experiment across the Descriptors

Shape Descriptor	Area (%)	Major Axis (%)	Minor Axis (%)	Orientation (°)
Tactile data
Compact shapes	17.5 ± 1.5	8.9 ± 1.0	10.8 ± 1.2	22.2 ± 3.6
Elongated shapes	33.6 ± 1.9	13.6 ± 1.1	31.0 ± 6.1	7.7 ± 1.8
Phosphene Data	16.1 ± 1.4*	9.9 ± 0.5	18.9 ± 1.7	6.9 ± 0.5

Data are expressed as the percentage or degree of variability. Tactile variability was separated into two categories: compact shapes (minor axis length, >50% of the major axis length) and elongated shapes (minor axis length, <50% of the major axis length). Phosphene drawing variability (row 3) was equal to (orientation, major and minor axes) or significantly less than (area) tactile drawing variability.

^*P < 0.05, Student's two-tailed t-test; significantly better performance for phosphenes than for elongated tactile shapes.

Area.

Both compact and elongated shapes were drawn larger (compact = ∼1.5 ± 0.15 times larger than the tactile target; elongated = 1.8 ± 0.3) than the actual size of the shape. For compact shapes, this difference in area between the compact shape drawings and actual tactile target was evenly distributed across the major and minor axes (∼1.3 ± 0.07 and 1.2 ± 0.07 times larger, respectively). For elongated shapes, the larger area bias was due to differences in the minor axis (∼1.6 ± 0.10 times larger) rather than the major axis (∼0.9 ± 0.07 times larger). As far as area variability was concerned, there was less variability across trials for compact (∼17% ± 2% error) than for elongated shapes (∼34% ± 2% error).

Orientation.

Our subject tended to draw elongated shapes biased by 9 ± 1.85° counterclockwise. Measuring angular bias was not possible for compact shapes since, being almost circular, they did not have a definitive major axis orientation. Drawings were less variable for elongated (∼8 ± 2° error) than for compact (∼22 ± 4° error) shapes (also, see Table 1).

Retinal Stimulation Experiment.

Nearly all phosphenes appeared as elongated ellipses (∼93%), with their minor axis length being less than 50% of the major axis length. We therefore compared phosphene variability to control experiment results for elongated shapes. Phosphene drawing was surprisingly consistent compared with performance for tactile shapes.

Area.

For phosphenes, the mean (across electrodes) variability in phosphene area across trials was 16% ± 1.4% (compared with ∼34% ± 2% for elongated tactile shapes). This difference in trial-by-trial variability between drawings of phosphenes and drawings of elongated tactile shapes was significant (P < 0.05, Student's two-tailed t-test).

Orientation.

For phosphenes, the mean orientation variability was ∼6.9 ± 0.5° (compared with ∼8 ± 2° for elongated tactile shapes; Table 1), and there was no significant difference in drawing variability between phosphenes and tactile shapes (P = 0.799, Student's two-tailed t-test).

These results suggest that our subject was able to accurately and consistently report the size and orientation of phosphenes elicited by retinal stimulation. Indeed, our results suggest that a major part of the variability across drawing of phosphenes may in fact be due to a drawing error rather than variability in the elicited percept.

Phosphene Descriptions

Percepts on individual trials were drawn as curved and straight lines, wedges, or relatively round spots. Phosphenes subtended ∼5 to 24° along their major axes and 1 to 8° along their minor axes, and ranged in area from ∼7- to 50° squared. The subject reported that percepts appeared light gray, white, or yellow. As either the stimulation amplitude or frequency increased, the subject reported that phosphenes were perceived as brighter with sharper contours. Examples of individual drawings of phosphenes for three different electrodes are shown in Figures 2B–D.

Figure 2.

Phosphene drawings from three different electrodes (a single trial for each electrode). (A) The array, showing the example electrodes in B–D. For all three electrodes, stimulation was 0.45 ms biphasic and 20-Hz pulse train, for a duration of 500 ms. (B, C) For electrodes D2 and C4 the pulse train was at 1.25× threshold. (D) For electrode B3 the pulse train was at 3.00× threshold.

Phosphene Size and Brightness

Figure 3 shows, for electrode, D2, how phosphene shape changed with an increase in either stimulation amplitude or frequency. The left column represents the individual trials (each trial shown in a different color) aligned based on their position on the reference grid. The right column shows the average drawing across five trials, plotted in a gray-scaled image. Because there was some variability in the positioning of the drawing of phosphenes across individual trials, we aligned the drawings from each trial shown in the first column along their mean centroid. The first row (Figs. 3A, 3B) represents phosphene drawings for stimulation using the baseline parameters of 1.25× threshold and 20 Hz, the second row (Figs, 3C, 3D) shows drawings when the amplitude was increased to 4× threshold, but frequency was kept constant at 20 Hz. The third row represents phosphene drawings when stimulation frequency was increased to 80 Hz, but amplitude was kept constant at 1.25× threshold. Comparison of the different rows demonstrates that phosphene size increases with amplitude (comparing rows 1 and 2), but size/shape does not increase substantially with frequency (comparing rows 1 and 3).

Figure 3.

The effects of current amplitude and frequency for electrode D2. Left: individual drawings for a set of trials (each drawn in a different color) aligned using the reference grid. Right: the average of these trials, aligned at the mean centroid of each individual drawing. (A, B) Ten trials with a pulse train of current amplitude 1.25× threshold and frequency of 20 Hz (one of the trials is a replication of the data shown in Fig. 2B). (C, D) Five trials at a current amplitude of 4.0× threshold and a frequency of 20 Hz. (E, F) Five trials at a current amplitude of 1.5× threshold and a frequency of 80 Hz.

As described above, as well as measuring the size of the elicited percepts, we also measured their brightness using a rating procedure. For this electrode, phosphene brightness did not increase significantly (P = 0.37, one-tailed Student's t-test) with increases in stimulation amplitude, but did increase significantly (P < 0.01, one-tailed Student's t-test) with an increase in stimulation frequency (Table 2, row 7).

Table 2.

Slope Values for Best Fit of Brightness and Size versus Amplitude and Frequency Plots in Figure 4

Electrode	Amplitude Coding: Brightness Slope (Rating/X Th)	Frequency Coding: Brightness Slope (Rating/Hz)	Amplitude Coding: Size Slope (Increase/X Th)	Frequency Coding: Size Slope (Increase/Hz)
B1	2.91*	0.105*	0.73*	0.042*
B3	0.40*	0.108*	1.19*	0.000 (0.64)
A2	1.39*	0.096*	1.51*	0.002 (0.13)
A4	4.93*	0.150*	1.28*	0.029*
D4	0.27 (0.45)	0.350*	0.68*	0.001 (0.19)
D3	1.45*	0.202*	1.52*	0.003*
D2	0.44 (0.37)	0.430*	1.92*	0.004 (0.08)
C4	0.98*	0.099*	0.98*	0.001 (0.35)
C1	2.48*	0.206*	0.73*	−0.001 (0.16)
Mean (all electrodes)†	1.69*	0.194*	1.17*	0.009 (0.17)

^*P < 0.01; slope is significantly greater than 0.

^†Mean slopes across all nine electrodes.

Figure 4 shows analogous brightness (Figs. 4A, 4B) and size (Figs. 4C, 4D) data for all nine electrodes (each in a different color) for both modulated-amplitude (Figs. 4A, 4C) and modulated-frequency (Figs. 4B, 4D) conditions. In amplitude plots, the x-axes are normalized with respect to threshold. In size plots, the y-axes are normalized with respect to the apparent size of a standard stimulus of 1.25× threshold and a frequency of 20 Hz. The straight lines on each plot are linear regression best-fit lines of the datasets.

Figure 4.

The effects of amplitude and frequency on apparent brightness and size. (A, C) Brightness and apparent size as a function of normalized (relative to threshold) amplitude for nine electrodes. (B, D) Brightness and apparent size as a function frequency for the same nine electrodes. Each electrode's data are fit with the best-fit linear regression.

Brightness increased as a function of both amplitude and frequency. Figure 4A shows brightness ratings as a function of increasing amplitude in the modulated-amplitude condition: brightness increased with amplitude, as indicated by slope of the best-fit line being significantly larger than 0 in seven of nine electrodes (P < 0.01, one-tailed Student's t-test for seven electrodes, P > 0.05 in the remaining 2). Figure 4B shows brightness ratings as a function of increasing frequency in the modulated-frequency condition: apparent brightness increased as a function of frequency for all nine electrodes (P < 0.01, one-tailed Student's t-test).

Apparent size always increased with increasing amplitude, but generally did not increase with frequency. Figure 4C shows drawing size (mean area) as a function of amplitude in the modulated-amplitude condition: For all nine electrodes, the size of the phosphenes increased as a function of amplitude (P < 0.01, one-tailed Student's t-test). Figure 4D shows size as a function of frequency in the modulated-frequency condition; in six of nine electrodes, size did not vary with frequency (P > 0.05); and in the remaining three electrodes slopes were significantly larger than 0 (P < 0.01, one-tailed Student's t-test). Although phosphene size significantly increased in three electrodes, in all cases the slopes (s) were relatively shallow (s = 0.3%–4.2% increase/ Hz). In other words, a doubling of the amplitude resulted in a 1.7 to 2.9× (mean, 2.2×) increase in size, whereas a doubling of the frequency resulted in only a 1 to 1.8× (mean, 1.2×) increase in size (normalized by frequency). Slope values and corresponding statistics for Figure 4 plots are also shown in Table 2.

Figure 5 shows data for normalized brightness and normalized size averaged across all electrodes. Best-fit lines are again based on a linear regression model. As for the individual electrode data, phosphene brightness increases with either amplitude or frequency, as indicated by a best-fitting slope that is significantly greater than 0 (Figs. 5A, 5B; P < 0.01, one-tailed Student's t-test). Percept size increases with increasing amplitude (Fig. 5C, P < 0.01, one-tailed Student's t-test) but does not change with increasing frequency (Fig. 5D, P > 0.05, one-tailed Student's t-test). It is also worth noting that larger increases in apparent brightness can be elicited by changes in frequency than changes in current amplitude—doubling the frequency results in a 1.4× increase in apparent brightness, whereas doubling amplitude only results in a 1.2× increase in apparent brightness. These data are summarized in the last row of Table 2.

Figure 5.

Normalized size and brightness as a function of amplitude and frequency averaged across nine electrodes. Data are fit using linear regression.

A previously published study of perceptual brightness versus amplitude used single pulses²⁶ (instead of pulse trains as performed here). In that study, it was found that brightness rating data were best fit with a power function with an exponent close to 0.4. Our results were comparable to the single-pulse published results within the range of amplitudes that we tested—similar to the single-pulse data, we found that brightness roughly doubles across a five times increase in amplitude.

Computational Modeling

These results can be qualitatively described using a simple model based on estimates of the spread of current from a metal disc in a semi-infinite medium (based on electrophysiological spatial threshold data)³⁶ and the perceptual sensitivity model by Horsager et al.,²⁵ previously used by our group to predict the perceptual sensitivity of the retina to electrical stimulation in human subjects. As described previously, this model bears a strong resemblance to models used to describe both temporal sensitivity for normal vision and retinal sensitivity to electrical stimulation, as measured neurophysiologically,^37–40 and can predict both absolute thresholds and suprathreshold brightness matching across a wide variety of pulse trains. A schematic of the model is shown in Figure 6.

Figure 6.

The study model. BOX 1: the time stimulus, f(t), was transformed into a spatiotemporal representation, on the basis of the measured electrophysiological thresholds from a disc electrode. BOXES 2–5: the output was then passed through a modified version of the perceptual sensitivity model incorporating threshold and suprathreshold parameters. The resulting output corresponds to a spatial brightness response, B(r).

We began by applying a spatial attenuation function to the temporal input stimulus pulse train to produce b₁(r,t), a spatiotemporal stimulus profile:

where f(t) is the electrical stimulation input pattern, t is the time (in milliseconds), r is the distance from the center of the stimulating electrode (in micrometers), and I(r) is the current attenuation from a disc electrode. The function used to model the spatial attenuation of current is given by

where r is the distance from the center of the stimulating disc electrode, and a is the radius of the electrode (Fig. 6, BOX 1). The I(r) function was obtained by inverting the relationship between threshold and distance from the edge of a 200-μm diameter platinum disc electrode (previously reported in Ahuja et al.³⁶). In that paper, thresholds from salamander retina were shown to increase with distance r from the stimulating electrode. We assumed that current attenuation at distance r was inversely proportional to the increase in threshold at r.

We then passed this spatiotemporal stimulus through the perceptual sensitivity model. In brief, the stimulus was convolved with a temporal low-pass filter that had a one-stage gamma function with a time constant τ₁ = 0.42 ms as its impulse response (Fig. 6, BOX 2). We then assumed that the system became less sensitive as a function of accumulated charge and calculated the amount of accumulated cathodic charge over time, and convolving this accumulation with a second one-stage gamma function with time constant τ₂ = 45.25 ms (Fig. 6, BOX 3). The output of this convolution was scaled (by a factor ε = 8.73) and subtracted from the output of the first convolution. The resulting time course was half rectified.

The previous instantiation of this model determined the amplitude or frequency required to reach threshold or a fixed brightness level. To do this, the half-rectified output, b₃(r,t), was passed through a power nonlinearity of β = 3.4 at threshold and β = 0.8 at suprathreshold. For our purposes a continuous mapping of amplitude/frequency to brightness was required. We therefore replaced β with a continuous function that nonlinearly rescaled b₃(r,t), across space and time, based on a sigmoidal function dependent on the maximum value of b₃(r,t; Fig. 6, BOX 4):

The parameter values of a (asymptote) = 14, s (slope) = 3, and i (shift) = 16 were chosen to match the observed psychophysical data. Interestingly, for parameter values that reproduced the observed behavior, we found that the sigmoidal function had an accelerating slope near threshold and a compressive slope when amplitude values were at suprathreshold levels: properties very similar to those demonstrated by parameter β in the original Horsager model. These were the only free parameters used to develop this model. All other parameters were based on those of the original Horsager data and model.

Finally, as in the Horsager model, the output, b₄(r,t) was convolved with a low-pass filter described using a three-stage gamma function with time constant τ₃ = 26.25 ms (Fig. 6, BOX 5). The maximum value of the output from this slow integration over time was used to represent the brightness response for each location in space, B(r). For a given stimulus, the brightness of a phosphene was assumed to be linearly related to the maximum brightness of the B(r) plot. We estimated the size of the phosphene by calculating the area where B(r) > ϴ, where ϴ was fixed as the maximum brightness elicited by a threshold stimulus.

Thus, most of the parameters in this model (τ₁, τ₂, τ₃, and ε) were fixed on the basis of previous work or separate measurements of threshold. Only the parameters of the sigmoid function (a, s, and i) were varied to match our psychophysical results.

Phosphene predictions generated with this model are shown in Figure 7. The top row shows the effect of increasing amplitude and the bottom row shows the effect of increasing frequency. Note that the 1.25× threshold with a 20-Hz stimulus (outlined images) is common to both the amplitude modulation and the frequency modulation conditions. The maximum value of B(r), representing the brightness of the phosphene, is reported below each simulated phosphene. Analogous to our psychophysical data, the model replicates the finding that increasing current amplitude results in increases in both brightness and size, whereas increasing frequency results in an increase in apparent brightness, but little increase in size. The model also replicates the finding that larger increases in apparent brightness can be elicited by changes in frequency than by changes in current amplitude.

Figure 7.

Predicted percepts with increasing amplitude (top row) and increasing frequency (bottom).

Because our model assumes uniform current spread from a disc electrode and equal sensitivity across the retina, all predictions are of a round, symmetric percept. Although our subject did occasionally draw circular percepts, the majority of his percepts resembled elongated ellipses, such as those shown in Figure 3. This result is probably due to unequal sensitivity across the retinal surface. It is possible that retinal stimulation activates not only the neural tissue directly below the electrode but also the passing axon fibers tracts.^28,41–44

Comparing Experimental Data with Modeling Predictions

Figure 8 replots the data from Figure 6 so as to compare size and brightness ratings for the same stimuli. As mentioned above, both the modulated-amplitude and -frequency conditions contained a stimulus at 1.25× threshold at 20 Hz. This stimulus can therefore be used as a standard reference—that is, both brightness ratings and size ratings were normalized so that this standard reference had a brightness rating of 10 and a size of 1. For each stimulus, we plotted the brightness rating assigned to that stimulus normalized by the standard reference along the x-axis and the apparent size of that stimulus normalized by the standard reference along the y-axis. The stimuli presented as part of the amplitude-modulation condition are shown in light gray, and the stimuli presented as part of the frequency-modulation condition are shown in dark gray. Dotted and solid lines show model predictions for amplitude and frequency modulation, respectively.

Figure 8.

Comparing brightness rating and apparent size psychophysical data to model predictions for the effects of changing amplitude and frequency. Experimental data from amplitude modulation are shown in dark gray, and data from frequency modulation are in light gray. Dotted and solid lines: model predictions.

As described above, the sixfold increase in amplitude tested in our experiment resulted in large changes (∼sixfold increase) in area but only moderate changes in brightness (∼twofold increase). In contrast, the sixfold frequency range tested in our experiment resulted in relatively little (∼2-fold) change in area but in a larger change in brightness (∼3.5-fold). Our model produced the same qualitative behavior. Note that, despite its good performance, this should be considered a descriptive rather than a predictive model, given that it was not used to predict an independent data set.

Discussion

Our experiments had two goals. The first was to determine whether the apparent shape of phosphenes was consistent across different trials. Our second goal was to see whether amplitude and frequency had separable effects on phosphene size and brightness.

Phosphene Reproducibility

To date, little work has examined whether consistent phosphenes are elicited across repeated stimulation. In a previous study by our group,²⁶ measuring apparent brightness for single electrode stimulation, we found that brightness consistently increased as a function of current amplitude. This study reported that for all electrodes, there was consistency in brightness ratings across different trials in the same session, and for the subject tested, there was also consistency in brightness ratings across different days. In a second study, we found that subjects could learn to correctly identify two different patterns of stimulation based on percept appearance.⁴⁵ Optic nerve stimulation studies have reported phosphene positional variability of ∼5 to 10° visual angle,⁴⁶ but did not examine the shapes of these phosphene. An acute epiretinal implant group semiquantitatively reported that repeated paired stimulation trials resulted in similar phosphenes on 66% of trials.²⁸

We directly examined percept appearance by asking subjects to draw phosphenes as well as rate their brightness. Our subject drew shapes based on tactile targets despite being blind for over 15 years. The ability to perform this control experiment with tactile targets with reasonable consistency and accuracy suggests that our drawing task provides a reliable measure of phosphene appearance. This assumption that the drawing task provides a reasonably good measure of phosphene appearance is further supported by measures of drawing variance across repeated stimulations. Variance in drawing perceived phosphenes was similar to that demonstrated for drawing tactile shapes in the case of orientation; and, in the case of area and major and minor axes, the variance for phosphenes was smaller than for tactile shapes. These results suggest that much of the variance we observed in our subject's drawings may be attributable to drawing error rather than variance in perceptions from trial to trial. The ability to generate electrically elicited percepts that are stable in appearance is most likely consistent with electrophysiology data showing that short electrical pulses elicit ganglion cell firing with remarkable reliability.^44,47

Phosphene Brightness

Manipulating current amplitude and frequency have different effects on phosphene size and brightness. Changing current pulse amplitude effects both phosphene area and, to a lesser degree, phosphene brightness. In contrast, changing current pulse frequency results in relatively little change in area but a much larger change in brightness.

Our finding that increasing either frequency or amplitude results in an increase in brightness is consistent with the previous literature for both epiretinal^25,26 and cortical²⁷ prostheses, as well as neurophysiological studies examining the responses of the retina to electrical stimulation.^22,48

Ganglion cell spike rates increase with contrast. Perceptually, brightness is a relative measure, so something considered bright by a typical observer has high contrast with its surroundings.⁴⁹ Previous in vitro work examining retinal responses to electrical stimulation has shown that, for pulses less than 0.1 ms, every current pulse can generate a spike for pulse frequencies up to 250 Hz, which is near the natural maximum for retinal firing.^47,50–53 These findings were based on relatively small conical electrodes with a length of 125 μm and a base diameter of 30 μm or disc electrodes 9 to 15 μm in diameter, whereas our findings are based on large disc electrodes. However, it is likely that, ganglion cell spiking within the population of cells underneath our electrodes follows the electrical stimulus pulse train pattern with reasonable precision across the range of temporal frequencies that we used.^22,36,47,50

We also found that increasing amplitude has a limited effect on the maximum brightness. This effect of saturation of brightness at higher amplitudes has been noted in previous experiments by our group. For example, we found in a previous study²⁶ that brightness ratings elicited with single-pulse (0.975 ms) stimulation are best fit with a power function. In our model the saturation of brightness with current amplitude was modeled by a sigmoidal nonlinearity that assumes asymptotic responses as a function of current amplitude at higher amplitudes. A possible explanation for this observation is that increasing amplitude increases brightness by producing multiple spikes per pulse,⁵⁴ which is interpreted by the visual system as increased brightness. However, cell refractory properties will eventually impose an upper limit on the number of spikes elicited by a single pulse, and at that point, further increases in amplitude beyond that amplitude will not increase the rate of neural response. Our data support this explanation, since brightness increases with amplitude at near threshold, but at four times above threshold, the increase in brightness approaches asymptote.

Phosphene Size

Manipulating amplitude results in large changes in area but only moderate changes in brightness. In contrast, manipulating frequency results in relatively little change in area and a larger change in brightness. Once again, this can be easily understood based on current understanding of retinal integration of electrical stimuli and basic electrostatics. When amplitudes are increased, suprathreshold current will be applied to a wider area of retina.³⁶

In frequency coding, the current spread remains the same across the range of frequencies. The electrical current dissipates virtually immediately at the end of the current pulse. Although cell depolarization may decay at a slower rate, the use of biphasic pulses reduces summation across pulses. Neurons lying at a distance from the electrode at which current is well below the spiking threshold will not fire, regardless of increasing frequency. This behavior is captured by the early part of the sigmoidal nonlinearity in our model, whereby low amplitudes on the retina result in no response. Although increasing frequency does reduce perceptual threshold,²⁵ this attenuation in threshold is small compared with the decrease in current with distance from the stimulating electrode. Thus, threshold attenuation with increasing frequency is likely to result in only a small expansion of phosphene size.

Caveats

Our model was able to successfully replicate our general finding that increasing frequency has greater effects on phosphene brightness than on phosphene size, whereas increasing current amplitude has greater effects on phosphene size than phosphene brightness. However, this model should be considered as a descriptive rather than predictive model, given the number of free parameters and the fact that it was not used to predict an independent data set. Moreover, the model is a major simplification of the complex retinal processes involved.

First, our model was not designed to successfully predict percept shape. The model assumed stimulation of cells below the electrode with a uniform current spread and equal sensitivity of the retina in all directions. As a consequence, our model predicts perfectly circular phosphenes. In contrast, our experimental data generally found that phosphenes were elongated and had angled lines or arcs. This finding suggests activation of axon fiber tracts as well as cell bodies. Activation of fiber tracts would be expected to produce a phosphene elongated in the direction of the axon tract, since the percept would include the receptive field locations of all (or a proportion of) the ganglion cells whose axon fibers lie under the electrode.^28,41–43 Consistent with this hypothesis, increases in phosphene size with increasing amplitude tended to be mainly along the minor axis dimension. Phosphene size does not increase along the major axis, since the spread in stimulation area along an axon tract simply continues to stimulate the same collection of fiber tracts (although points along the tract length are stimulated). Along the minor axis the spread in stimulation area would recruit additional axon tracts. Incorporating biophysical models of activation may allow us to predict shape as well as brightness and size in the future.

Second, these data are from a limited number of electrodes from a single subject, in whom the array was implanted in the macular region with electrodes in contact with the retina. Results may vary in other prosthesis subjects with different implant locations and threshold values.

Implications for Future Retinal Prostheses

A variety of engineering constraints are likely to limit the capabilities of retinal prostheses, such as electrode size, charge density limit and electrode-to-retina distance.²⁴ Working within these limitations, a successful retinal prosthesis will use a stimulation paradigm that optimizes available resolution and contrast to present the visual world to patients. The work described herein makes several advances toward this goal. First, these data are the first to systematically show consistency in percept shape across a variety of stimulation parameters. Second, we find that phosphene size and brightness can to a certain degree be dissociated using amplitude or frequency coding. Based on these findings, we believe that frequency coding may offer a reasonable solution to maintaining high resolution, while producing percepts that range widely in brightness. Finally, these results can be plausibly explained with a simple model based on electric field spread from a disc electrode and a previous model describing perceptual sensitivity to retinal stimulation. Together, these findings demonstrate further progress toward optimizing stimulation patterns for a retinal prosthesis with form perception capabilities.