Percepts evoked by multi-electrode stimulation of human visual cortex

Abstract

Background:

Direct electrical stimulation of early visual cortex evokes the perception of small spots of light known as phosphenes. Previous studies have examined the location, size, and brightness of phosphenes evoked by stimulation of single electrodes. While it has been envisioned that concurrent stimulation of many electrodes could be used as the basis for a visual cortical prosthesis, the percepts resulting from multi-electrode stimulation have not been fully characterized.

Objective:

To understand the rules governing perception of phosphenes evoked by multi-electrode stimulation of visual cortex.

Methods:

Multi-electrode stimulation was conducted in human epilepsy patients. We examined the number and spatial arrangement of phosphenes evoked by stimulation of individual multi-electrode groups (n = 8), and the ability of subjects to discriminate between the pattern of phosphenes generated by stimulation of different multi-electrode groups (n = 7).

Results:

Simultaneous stimulation of pairs of electrodes separated by greater than 4 mm tended to produce perception of two distinct phosphenes. Simultaneous stimulation of three electrodes gave rise to a consistent spatial pattern of phosphenes, but with significant variation in the absolute location, size, and orientation of that pattern perceived on each trial. Although multi-electrode stimulation did not produce perception of recognizable forms, subjects could use the pattern of phosphenes evoked by stimulation to perform simple discriminations.

Conclusions:

The number of phosphenes produced by multi-electrode stimulation can be predicted using a model for spread of activity in early visual cortex, but there are additional subtle effects that must be accounted for.

1. Introduction

Electrical stimulation of visual cortex leads to the perception of a flash of light known as a phosphene [1,2]. It has been recognized for many years that this could be harnessed to create a visual cortical prosthesis (VCP), a device that could restore some visual function in patients who are totally blind due to damage to the retina, or early visual pathways, but with an intact visual cortex. In fact, several prototype VCPs have already been tested [3-9], are currently in a clinical trial [10] or are in development [11-17]. The potential effectiveness of such a device relies on two basic assumptions: 1) that the visual cortex of blind subjects retains a structured map of visual space, and 2) that we can use patterned stimulation within this map to convey visual forms to the subject.

Several studies have demonstrated that patients retain an orderly map of visual space in early visual cortex years after the onset of acquired blindness. This has been demonstrated by mapping phosphene locations in blind subjects enrolled in testing of prototype VCPs [4,7,8,10,18,19] and observation of normal resting state patterns of activity in visual cortex as measured by human neuroimaging [20].

Most experiments using electrical stimulation of visual cortex have focused on the attributes of the phosphenes evoked when single electrodes are used for electrical stimulation. However, the assumption has been that selective activation of an array of many electrodes could be used to create a set of phosphenes which would fuse or blend to evoke the perception of a unified or coherent form such as a line, character, or a simple object. Some experiments in both non-human primates [21] and blind human subjects [7,22] have provided hints that simultaneous stimulation of sets of implanted electrodes can be used to convey visual patterns. However, the ability to reliably generate visual forms using multi-electrode stimulation has been elusive, and neither the visual percepts obtained with concurrent multi-electrode stimulation, nor their correspondence to the pattern of phosphenes expected based on stimulation of single electrodes, have been well characterized.

Here we present the first systematic investigation of the rules governing perception of multiple phosphenes when groups of electrodes over visual cortex are stimulated simultaneously. We found that simultaneous stimulation of surface electrodes separated by distances of less than 4 mm tended to produce perception of a single phosphene while stimulation of those separated by greater distances tended to result in perception of two phosphenes. In addition, simultaneous stimulation of three or more electrodes typically resulted in a consistent spatial pattern of phosphenes. However, significant shifts, rotation, and scaling of this pattern are possible on each trial. Furthermore, subjects sometimes failed to report at least one of the phosphenes associated with a particular pattern, and in general had difficulty accurately reporting on the pattern of phosphenes perceived with stimulation of more than three electrodes on single trials. These results in sighted subjects validate the idea that subsets of the map of visual space can be directly stimulated to robustly convey a visual pattern, but also point out the challenges in assessing the results of multi-electrode stimulation in sighted subjects and indicate the complexity of trying to convey patterns to blind subjects using a VCP.

2. Materials and methods

2.1. Subjects

Electrical stimulation was conducted in patients (n = 11) with medically intractable epilepsy who had subdural electrodes implanted for clinical monitoring. Informed consent was obtained from all subjects, and the Baylor College of Medicine Institutional Review Board approved all procedures. Patients remained in the epilepsy-monitoring unit for 4–14 days. Clinical monitoring continued uninterrupted during experimental sessions.

2.2. Electrodes

The custom surface electrode strips used in these experiments contained research electrodes (platinum, 0.5 mm diameter, 2–6 mm spacing) positioned in between the standard clinical electrodes (platinum, 2 or 3 mm diameter, 1 cm spacing), embedded in silastic (PMT Corporation, Chanhassen, MN). The electrodes used in this study were located on the surface of the occipital cortex near the calcarine fissure and the occipital pole. This area is known to correspond to the primary visual cortex (V1), and other early visual cortical areas (V2, V3). Up to 16 electrodes were tested in each hemisphere. Three different hybrid clinical/research electrode strips were used with a variable number of research electrodes (8, 12, or 16).

2.3. Receptive fields

Receptive field (RF) mapping was conductied using procedures that were identical to those used in previous reports [23,24]. Briefly, checkerboard stimuli were flashed in various screen locations while the subjects conducted a task that required central fixation. The center of the RF was determined by fitting a 2D-Gaussian curve to the response data for each visual field location.

2.4. Electrical stimulation general

During all experiments, the patients remained seated comfortably in their hospital bed. A ground pad was adhered to the patient's thigh. All electrical stimulation was monopolar. Electrical stimulation currents were generated using a 16-channel system (AlphaLab SnR, Alpha Omega, Alpharetta, GA) controlled by custom code written in MATLAB (Version 2013b, The MathWorks Inc., Natick, MA).

We first screened all electrodes to determine which ones were capable of producing phosphenes and to determine a rough estimate of the threshold current required for phosphene perception. This was done by manually increasing the current amplitude by small increments (0.1–0.5 mA) on successive trials through the range of 0.3–4.0 mA until the subject first reported a phosphene. Rigorous quantitative methods for determination of threshold currents as we have used in the past [24,25] were not used so that we could quickly move on to testing of multi-electrode patterns.

During each stimulation trial, an auditory warning tone cued the patients to fix their gaze on a small cross on the touchscreen (Fig. 1E). This was followed by a second tone that indicated the beginning of the electrical stimulation period. Electrical stimulation consisting of a train of biphasic pulses (−/+), cathodal leading, with 0.1 ms pulse duration per phase, was then delivered at a frequency of 200 Hz, with an overall stimulus train duration of 200 or 300 ms. Currents tested ranged from 0.3 to 4.0 mA resulting in a total charge delivered of 1.2–24 μC per trial.

Fig. 1. Methods.

A) Surface model of the occipital lobe of one subject showing typical electrode placement for one of our hybrid electrode strips containing both clinical electrodes (large circles, 2 or 3 mm) and research electrodes (small circles, 0.5 mm). Colored regions here and in all figures indicate predicted location of V1 (white) and V2 (light green) based on standard atlas. Dashed line indicates location of calcarine fissure. B) Cross section through occipital cortex near the calcarine fissure illustrating determination of electrode separations. For pairs of electrodes pairs lying within one gyrus (green and blue pair), distance was calculated as the nominal separation on the electrode strip, while for electrodes lying on opposite sides of sulcus (blue and red pair) distance along the cortical surface was calculated in AFNI/SUMA. C) Receptive field mapping. Subjects performed a letter detection task at a central fixation point while checkerboard patterns were flashed in various locations on the screen. D) Phosphene mapping. The subject maintained fixation while electrical stimulation was delivered to of one or more electrodes and then drew the location of the perceived phosphenes on the touchscreen. E) Timing of phosphene drawing task. Subjects received a warning tone 1 s prior to stimulation and another tone at the onset of stimulation. Biphasic pulse trains at 200 Hz were delivered for 200–300 ms. After stimulation the subject was allowed as much time as necessary to draw the perceived phosphene.

2.5. Phosphene reporting and mapping

Our phosphene mapping technique is illustrated in Fig. 1D. Subjects viewed an LCD touchscreen that was typically located 57 cm in front of them. Screen distance was sometimes adjusted to allow the receptive fields and phosphenes associated with the implanted electrodes to appear within the confines of the touchscreen. On each trial, subjects indicated whether they saw a phosphene by verbal report, then drew the outline of the phosphene using a stylus on the touchscreen. In rare cases where stimulation of a single electrode generated two completely distinct phosphenes, those electrodes were excluded from further testing. In some early cases, phosphene outlines were obtained by pencil drawings on paper rather than using the touchscreen.

A variable sampling strategy was used with different subjects and with different blocks of testing in order to optimize the testing we were able to perform in the limited time available with each subject. In subjects YAB, YAF, YAH, YAN, and YAO, the subjects drew each of the phosphenes they perceived, and we typically conducted multiple trials for each individual electrode (2–8 trials, median 3), pair (3–7 trials, median 3), or triplet (1–5 trials, median 3) tested. In subjects YAU and YAY, we tested more multi-electrode groups, the subjects drew each of the phosphenes they perceived, and we typically conducted only 1 trial for each individual electrode or group of electrodes tested. In subject YBH, we tested every possible pair available based on the implanted electrodes but had the subject report only the number of phosphenes perceived and only 1 trial was used for each pair tested. In subject YAO, we also tested additional triplets where the subject reported only the number of phosphenes perceived and only one trial was performed. Whenever multiple trials were used, we report the mean phosphene location across trials.

Each single electrode or multi-electrode group was tested in a separate block. For example, all trials with testing of the group electrode 1 – electrode 6 – electrode 10 were typically tested in one block with no intervening trials with other multi-electrode groups. For pair and triplet stimulation, the subjects could usually draw all the perceived phosphenes following one stimulation trial. With quadruplet or quintuplet stimulation, however, the subjects sometimes required multiple trials to determine how many phosphenes they perceived and to indicate the location of each one.

2.6. Analysis of phosphene maps

All phosphene drawings were fit with ellipses. The center of the best-fit ellipse was taken as the center of the phosphene. We used (major diameter + minor diameter)/2 as the measure of phosphene size. Phosphene size in degrees of visual space was calculated by using the formula: V = 2 arctan(S/2D) where V = visual angle in degrees, S = size of the object or stimulus in question, and D = the viewing distance.

2.7. Electrode localization

Electrodes were localized relative to structural model of the subjects’ brains using Analysis of Functional NeuroImages software (AFNI) [26], FreeSurfer [27,28], and SUMA [29], as previously described in Ref. [24].

2.8. V1-V2-V3 flat map model

We used a previously published set of transforms [30] to model the V1-V2-V3 complex, and all cortical activation modeling was performed after projecting electrode locations onto this map. To project the electrode location onto the map we used the center of the RF for all electrodes for which we had RF mapping data. For those electrodes where we did not have accurate RF information, for example when the RF location was not included in the visual field range sampled during the RF mapping experiments, we instead used the center of the individual phosphene map. The scaling of the V1-V2-V3 complex was adjusted for each subject by adjusting the spacing between known landmarks, such as two of the clinical recording electrodes, to be equal to the nominal distance known to exist for those landmarks.

2.9. Calculation of predicted cortical activation

For each electrode stimulated, the predicted diameter of cortical activation was obtained using a modification of our previously published model [24]. Here we extend that model by assuming a 2D Gaussian activation profile in visual cortex. The sigma of the Gaussian profile was obtained by setting 4.292 sigma to be equal to the diameter computed by our previous equation. By doing this we are picking a Gaussian activation profile of a width such that the top 90% of the activation profile should correspond to the predicted diameter of cortical activation we obtained from our previous experiments. Activation from multiple electrodes was calculated by linear summation of the individual activation patterns from each electrode.

2.10. Calculation of predicted phosphenes

Size of predicted phosphenes were calculated based on the location of the electrodes in the map of visual space, and the current used for stimulation. Location in the map of visual space was based on RF location if known, or on the location of the phosphene in visual space when each electrode was stimulated individually. In our previous work [24], we had calculated predicted phosphene diameter by multiplying the diameter of activated cortex by the inverse magnification factor based on electrode eccentricity. In this report, we now model the phosphene as a 2D Gaussian in visual space with 4.292 sigma set to be equal to the diameter predicted by the previous equations. For simultaneous electrode stimulation, the phosphenes predicted for stimulation of each electrode are combined by linear summation.

2.11. Calculation of distance on the cortical surface

We used a hybrid method to obtain the best estimate of the distance separating each pair of electrodes tested. For pairs of electrodes pairs lying within one gyrus, distance was calculated as the nominal separation on the electrode strip, while for electrodes lying on opposite sides of sulcus, the SurfDist function in AFNI was used to calculate the distance between the cortical surface model nodes that were closest to the two electrodes. The cortical surface model was not used to calculate the inter-electrode distance for nearby pairs, that were not separated by a sulcus, because we suspected in those cases this would inject errors due to both inaccuracies in the generation of the surface model and the distance from the electrodes to the closest nodes on the surface model.

2.12. Alignment of multi-electrode stimulation data across multiple trials

We performed three manipulations to adjust for the trial-to-trial variability in phosphene locations associated with multi-electrode stimulation. First, data from each simultaneous multi-electrode stimulation trial were adjusted to have the same center of mass that was observed for the phosphenes associated with individual stimulation of the same set of electrodes. For pair stimulation, this was the only alignment operation that was utilized. Second, phosphene locations obtained on individual trials of simultaneous multi-electrodes stimulation were rotated about the center of mass. For each multi-electrode stimulation pattern tested, we calculated the angle in visual space of the line segments connecting each pair of phosphenes. These angles were calculated for independent stimulation and concurrent stimulation of the same set of electrodes. We performed a rigid rotation of the data from single trials of concurrent stimulation to minimize the average error between the angles measured for concurrent stimulation vs. those measured for independent stimulation of the same electrodes. Finally, data from individual trials of multi-electrode stimulation were scaled to have the same average separation between phosphenes pairs as that measured for individual stimulation of the same electrodes.

2.13. Pattern discrimination experiments

Subjects were asked to discriminate between two or three patterns of phosphenes generated by multi-electrode stimulation. These experiments were 2AFC or 3AFC with a single stimulus interval. Trials with different patterns were presented in random order. On each trial, a warning tone was played and then the subject was required to fixate during the presentation of the electrical stimulus. After the electrical stimulation the subject gave a verbal report of which of the patterns they perceived on that trial. No extended training was performed prior to the block used to assess discrimination performance. Only a few trials were used to establish the pattern of phosphenes observed for each multi-electrode group.

2.14. Statistical methods

Assessment of the significance in the difference in means between two groups were made using unpaired or paired t-tests. An unpaired test (Matlab function ttest2) was used for the assessment of differences in electrode separation between pairs that produced one phosphene or two phosphenes as these two groups contained independent samples and different numbers of pairs (Fig. 3A). Paired tests (Matlab function ttest) were used to compare differences in the properties of observed phosphenes that were measured under the independent or concurrent stimulation conditions. This included assessment of differences in the separation, size and average eccentricity of the phosphenes that were measured under the two stimulation conditions (Fig. 4D, F-H). For all t-tests we provide the degrees of freedom, the test statistic, and whether the p value was above 0.05, below 0.05, or below 0.001.

Fig. 3. Summary of overall pair results.

Summary of overall pair results. A) Overall data for separation on cortical surface versus number of phosphenes perceived for all pairs tested which produced one or two phosphenes (n = 230). Estimated separation between electrodes on the cortical surface for pairs of electrodes that produced single phosphenes (left) and those that produced two phosphenes (right). Red line indicates median, black circle indicates mean. Boxed area indicates 25 to 75 percentile region of data. Notches indicate 95% confidence interval on median. Whiskers indicate limits of data not considered outliers. Plus symbols indicate outliers. B) Data for separation on cortex versus number of phosphenes perceived for pairs where we had a reliable determination of pair type. Boxplot conventions as in panel A. Schematics below each column indicate typical phosphenes observed for pairs of each type for independent (grey) and simultaneous (red) stimulation.

Fig. 4. Changes in phosphene perception obtained with simultaneous pair stimulation.

A) Location of the surface recording strip placed in subject YAU. Of the 16 mini-electrodes implanted 10 were used for pair sampling (dark grey filled circles). B) Example Type 1A pair from this case. Simultaneous pair stimulation led to perception of a single large phosphene (red ellipse) that overlapped the location of both of the phosphenes produced by individual electrode stimulation (grey ellipses). C) Example Type 1B pair from this case. Simultaneous stimulation led to perception of a phosphene (red ellipse) that overlapped only one of the two phosphenes produced by individual electrode stimulation (grey ellipses). D) Size of phosphene perceived for individual stimulation and concurrent stimulation for pairs that produced only one phosphene. For Type 1A pairs, simultaneous stimulation consistently produced phosphenes that were larger than those obtained by independent stimulation. E) Example Type 2 pair from this case. Phosphenes obtained with simultaneous stimulation (red ellipses) were similar in size or slightly smaller than those obtained with individual stimulation (grey ellipses) and were located slightly further apart in visual space. F) Separation between phosphenes produced by individual stimulation vs. simultaneous stimulation for pairs that produced two phosphenes. G) Average size of phosphenes obtained by individual stimulation vs. simultaneous stimulation for pairs that produced two phosphenes. H) Average eccentricity of phosphenes produced by individual stimulation vs. simultaneous stimulation for pairs that produced two phosphenes.

For assessment of the differences between phosphene separation, phosphene size, and phosphene eccentricity between the independent and simultaneous stimulation conditions we also describe the results of linear regression (Fig. 4F-H). For linear regressions, we report the value of r, whether the p value was below 0.05 or 0.001, and the slope.

For assessment of differences in the mean between three or more groups we used ANOVA (Matlab function anova1) to establish whether there was an overall difference between groups (Fig. 3B). This was followed by post-hoc pairwise testing of the differences between particular groups (Matlab function multcompare). For the post-hoc comparisons we report whether the p value was below 0.05 or 0.001.

To assess the differences in the average trial-to-trial error in phosphene location following different alignment processes utilizing data from the same triplets (Fig. 8A) we used repeated measures ANOVA (Matlab function ranova). This was followed by post-hoc pairwise comparisons between the groups (Matlab function multcompare). For the post-hoc comparisons we report whether the p value was below 0.05 or 0.001.

Fig. 8. Summary of trial-to-trial variation for triplet stimulation.

A) Average trial-to-trial error in location of phosphenes produced by simultaneous stimulation relative to the location of phosphenes produced by individual stimulation of the same electrodes. Average error is shown for raw data (R) and following removal of translation (T), translation and rotation (TR), and translation, rotation, and scaling (TRS) errors. Error bars indicate standard error of mean. B) Histogram of translation factors required to align trials. C) Histogram of rotation factors required to align trials. D) Histogram of scale factors required to align trials.

Confidence intervals for mean performance in the 2AFC and 3AFC pattern discrimination tasks were determined using binomial statistics (Fig. 11B). These can be compared to the expected level of performance for random guessing (50% and 33.3%) as shown in the figure.

Fig. 11.

Behavioral discrimination between multi-electrode stimulation patterns. A) Illustration of the 2AFC task. Subjects fixated a touchscreen while multiple-electrode stimulation was delivered. After the stimulation, the subject viewed two or three possible visual patterns and selected the visual pattern that most closely corresponded to the percept elicited by the electrical stimulation. B) Summary of results from all behavioral experiments testing discrimination between three (left column) or two (right columns) patterns of multiple electrode stimulation. Error bars indicate 95% confidence intervals calculated using binomial statistics. Dashed lines indicate chance level of performance for the 3AFC (33.3%) and 2AFC (50%) tasks. Number of trials for 3AFC (YFH:180) and for 2AFC (YAF1:60, YAF2: 141, YAF3: 150, YAF4: 141, YAH: 30, YAJ1: 14, YAJ2: 66, YAO:79).

3. Results

3.1. Overview

We studied the perception of phosphenes observed with concurrent multi-electrode stimulation of 313 different groups of electrodes in human visual cortex (Table 1). Subjects (n = 11) were sighted epilepsy patients hospitalized for invasive monitoring. We examined both the resulting number and spatial configuration of phosphenes obtained with individual patterns of multi-electrode stimulation (n = 8), and the ability of subjects to discriminate between two or more patterns generated by stimulation of different multi-electrode groups (n = 7).

Table 1.

Summary of multi-electrode groups tested. The table provides the number of groups tested, and the number of times different numbers of phosphenes were perceived for groups ranging in size from simultaneous stimulation of 2 electrodes to simultaneous stimulation of 6 electrodes.

Number of electrodes in pattern	Number of phosphenes
	1	2	3	4	5	6	Number of patterns tested
2	74	156	2	0	0	0	232
3	4	34	30	1	0	0	69
4	0	0	4	3	0	0	7
5	0	0	1	2	1	0	4
6	0	0	0	1	0	0	1

3.2. Localization of electrodes

Each subject was implanted with custom surface electrode recording strips that featured research electrodes (0.5 mm) located in between standard clinical recording electrodes (2 or 3 mm) in one of three configurations (Fig. 1A; PMT) [24]. We obtained a surface model of the cerebral cortex using images obtained in preoperative MRI, and overlaid a standard probabilistic atlas of visual areas V1 and V2 [31] (White and light green shading, Fig. 1A). We determined the location of implanted electrodes relative to the cortical surface model and visual areas, and we include the results of multi-electrode stimulation only from sites that appeared to be localized to early visual cortex, areas V1-V3. The overall distribution of electrodes used for phosphene reporting and mapping experiments included 45 electrodes in V1, 20 in V2, and 2 in V3. We used a hybrid method to determine distance between pairs of electrodes (Fig. 1B; methods).

3.3. Initial screening

For each subject, we first conducted RF mapping to determine which electrodes were visually responsive and the location of the RFs in visual space (Fig. 1C) [23]. We next conducted a screening procedure to determine which electrodes evoked a phosphene when electrical stimulation was delivered at low currents (Fig. 1E; methods). In many cases, we also had the subject draw the location of the phosphene perceived (Fig. 1D). In further sessions, we then conducted concurrent multi-electrode stimulation with small groups of electrodes. The current used for each electrode during multi-electrode stimulation testing was set slightly above the threshold for phosphene production for that electrode when it was stimulated in isolation. The exact amount above threshold selected was variable, but was typically about 20%, and was in all cases at a level that allowed to the subject to easily see and localize the phosphene on every trial.

3.4. Examples from concurrent pair stimulation

We use two examples from one case (YAB) to indicate typical results from concurrent electrical stimulation of pairs of electrodes (Fig. 2). First, we illustrate results from a pair of electrodes located close together on the cortical surface near the V1-V2 border (Fig. 2A-D). Based on our previous research [24](methods), we can model the cortical activation expected from electrical stimulation of each of the two electrodes. Here we show the expected activation of each electrode on a flat map model of the V1-V3 complex (Fig. 2B) [30]. We use the flat map model for illustration of the expected activation pattern because our guiding hypothesis is that it is spread of activation within the cortical sheet that will predict the results of multi-electrode stimulation. In this case, we expected the cortical activation from the two electrodes would be highly overlapping and that therefore the subject would report perception of a single phosphene (Fig. 2C). When the two electrodes were stimulated concurrently, the subject did report perception of only one phosphene in a location in visual space (red circle Fig. 2D) that overlapped with the reported location of the phosphenes observed with individual stimulation of each electrode (grey circles Fig. 2D).

Fig. 2. Simultaneous stimulation of pairs of electrodes.

Results from one subject (YAB) for stimulation of one pair of electrodes separated by 2.8 mm (top row) and another pair separated by 10 mm (bottom row). A) Location of the electrodes used in pair 1 (red), and research and clinical electrodes not used in the current pair (white). B) Predicted cortical activation for simultaneous stimulation of the two electrodes in pair 1 depicted on a flat map of V1 (white), V2 (light green) and V3 (grey), including iso-azimuth and iso-eccentricity lines (black). Representations of the vertical meridian (VM) and horizontal meridian (HM) are indicated. Thicker line within V1 indicates region of V1 typically found buried within the calcarine fissure. Red regions indicate top 10% of activation, orange indicate top 50%, and yellow indicate top 90%. For this pair, activation resulting from stimulation of the two electrodes is predicted to overlap on the cortical sheet. C) Predicted phosphenes for simultaneous stimulation of pair 1. Red indicates top 10% in brightness, orange indicates top 50%, and yellow indicates top 90%. D) Actual reported locations for the phosphenes for pair 1 associated with independent (grey) and simultaneous (red) stimulation of the two electrodes. E) Location of the electrodes used for pair 2 (red). F) Cortical activation predicted for simultaneous activation of the electrodes in pair 2. In this case, two discrete peaks of cortical activation are predicted. G) Phosphenes predicted for pair 2. H) Location of actual phosphenes for pair 2 reported when the two electrodes were stimulated individually (grey) and simultaneously (red).

In the second example from the same case (Fig. 2E-H), a pair of electrodes was located 10 mm apart, with one electrode near the V1-V2 border, and the other firmly in V1 near the calcarine fissure. In this case, we predicted that concurrent activation of the two electrodes would lead to two independent peaks of activity in early visual cortex (Fig. 2F) and to perception of two phosphenes (Fig. 2G). When concurrent stimulation of this pair was tested the subject reported perception of two phosphenes (red circles Fig. 2H) with one phosphene located near each of the locations of perceived phosphenes when the electrodes were stimulated independently (grey circles Fig. 2H).

3.5. Average results for number of phosphenes versus distance

Overall, we tested 232 pairs from 7 subjects. There were 90 pairs with both electrodes in V1, 29 with both electrodes in V2, 89 with one electrode in V1 and one in V2, 13 with one electrode in V1 and one in V3, and 11 with one electrode in V2 and one in V3. Two pairs produced perception of three phosphenes and were excluded from further analysis. As expected, we found that subjects tended to perceive one phosphene when the electrodes were located close together on the cortical surface and two phosphenes when they were separated by greater distances (Fig. 3A). The difference in means between the one phosphene (n = 74; m = 9.57 mm; sd = 6.56 mm) and two phosphene (n = 156; m = 26.36 mm; sd = 16.31 mm) groups was significant by unpaired t-test (t(228) = 8.5301; p < .001).

However, we found that there were two distinct types of results obtained when the subject perceived only one phosphene. In one type (Type 1A), the subject perceived one phosphene in a location that overlapped with the locations of both phosphenes observed with individual stimulation. In the other type (Type 1B), the subject perceived one phosphene in a location in visual space that was overlapping or near the location of only one of the two phosphenes obtained with independent stimulation.

Next, we show the average data for cases where we could clearly identify each pair as Type 1A, Type 1B, or Type 2 (subject perceived two phosphenes; Fig. 3B). For Type 1A pairs, a single phosphene, resulting presumably resulting from overlapping activity in early visual cortex, was reported only when the two electrodes were located within a short distance (n = 15; m = 4.11 mm; sd = 1.55 mm). Type 1B pairs were found in cases where the electrodes were separated by much larger distances (n = 9; m = 13.47 mm; sd = 6.62 mm), similar to pairs for which two phosphenes were reported (n = 39; m = 14.14 mm; sd = 6.82 mm). Differences between groups were found to be significant by one-way ANOVA (F(2) = 15.6412; p <. 001). Pairwise comparisons indicate that the mean electrode separations for the Type 1A group is significantly different than the Type1B group (p < .05) and the Type 2 group (p < .001), and that the electrode separation for the Type1B group is not significantly different than for the Type 2 group (p > .05).

3.6. Changes in phosphene perception with pair stimulation

Although the basic results for pair stimulation were simple to interpret based on the separation of the electrodes on the cortical surface, there were several observations that suggest additional interactions. We examined these more subtle effects for a large set of pairs obtained from a single case (subject YAU; Fig. 4). First, we illustrate example Type 1A (Fig. 4B), Type 1B (Fig. 4C), and Type 2 pairs (Fig. 4E) from this subject. Note that for the Type 1A example, the single phosphene observed with concurrent stimulation (red) overlaps with both phosphenes observed with independent stimulation (grey) and is much larger in size. For the Type 1B pair, the single phosphene observed with concurrent stimulation (red) overlaps only one of the two phosphenes observed with independent stimulation (grey) and is slightly smaller in size. Finally, for the example where two phosphenes were observed with concurrent stimulation, the two phosphenes were each near the location of the phosphenes observed with independent stimulation, but they were located slightly further apart in visual space and were slightly smaller in size.

These results were typical when we examined all pairs from this case. For Type 1A pairs (n = 8; Fig. 4D), phosphenes observed with concurrent electrical stimulation (m = 6.25°; sd = 1.53°) were significantly larger than those obtained with independent stimulation (m = 3.07°; sd = 0.43°; t(7) = 6.8516; p < .001). For Type 1B pairs (n = 2; Fig. 4D) phosphenes obtained with concurrent stimulation (3.74°, 0.96°) were similar in size or smaller than the corresponding phosphene obtained with independent stimulation (4.87°, 2.38° respectively).

For Type 2 pairs (n = 35), the separation between phosphenes observed with concurrent stimulation (m = 7.80°; sd = 3.55°) was significantly larger than the separation obtained with independent stimulation (m = 5.46°; sd = 2.85°; t(34) = 6.5832; p < .001; Fig. 4F). Other than this average increase in separation of 2.34°, linear regression reveals that the phosphene separations measured for the two conditions are well correlated (r = 0.8055; p < .001; slope = 1.0016).

In addition, for Type 2 pairs, phosphene sizes obtained with concurrent stimulation (m = 2.49°; sd = 0.55°) were consistently smaller than those obtained with independent stimulation (m = 3.59°; sd = 0.96°; t(34) = 8.1370; p < .001; Fig. 4G). In this case, the linear regression reveals that phosphene sizes measured in the two conditions exhibit significant correlation, but with a slope much less than one (r = 0.5521; p < .001; slope = 0.3135). This may indicate that the subject tended to regularize reporting of phosphene size in the context of the simultaneous stimulation experiment.

To understand these changes in phosphene reporting in the simultaneous condition, it is also important to evaluate whether there was a change in the average eccentricity of the two phosphenes between the independent and simultaneous conditions (Fig. 4H). We found the average eccentricity of phosphenes obtained with concurrent stimulation (m = 9.30°; sd = 2.45°) was significantly different than the average eccentricity obtained for independent stimulation of the same pairs (m = 8.47°; sd = 2.48°; t(34) = 5.7097; p < .001). However, average eccentricity was well correlated between the independent and simultaneous conditions (r = 0.9388; p < .001; slope = 0.9281) and the mean magnitude of the difference in eccentricity between simultaneous and independent conditions (0.83°) is not large enough to explain the average differences in phosphene size and separation that we observed.

3.7. Examples from concurrent triplet stimulation

Results from concurrent triplet stimulation were in general consistent with the results obtained with pair stimulation. We use two example triplets to illustrate some of the basic findings (Fig. 5). In the first example (Fig. 5A-D), we used concurrent stimulation of two electrodes that were in close proximity near the V1-V2 border, and a third electrode that was located at a greater distance away in V1 (Fig. 5A red circles). Based on these locations, we predicted concurrent stimulation from this triplet to result in two peaks of activity in early visual cortex (Fig. 5B) and the perception of two phosphenes (Fig. 5C). The subject did perceive two phosphenes with concurrent stimulation of this triplet (red circles Fig. 5D). This result with concurrent triplet stimulation is analogous to Type 1A pair results.

Fig. 5. Simultaneous stimulation of triplets.

Results from two subjects for concurrent stimulation of three electrodes. All color and naming conventions are as in Fig. 2. A) Location of the electrodes used in triplet 1 (red circles) from subject YAB. B) Predicted cortical activation for simultaneous stimulation of triplet 1 depicted on a flat map of the V1-V3 complex. For this triplet, two peaks of cortical activation are predicted based on combination of activation from two electrodes and a separate peak from the more distant electrode. C) Phosphenes predicted for simultaneous stimulation of triplet 1. D) Actual location of the phosphenes reported for triplet 1 with independent (grey) or simultaneous (red) stimulation of the three electrodes. E) Location of electrodes used in triplet 2 (red circles) from subject YAF. F) Cortical activation predicted for simultaneous stimulation of triplet 2. In this case, again two discrete peaks of cortical activation are predicted. G) Predicted phosphenes for triplet 2. H) Actual location of reported phosphenes for triplet 2 for individual (grey) and simultaneous (red) stimulation of the three electrodes. The subject perceived only one phosphene for this triplet, apparently failing to report the more eccentric phosphene.

As with concurrent pair stimulation, concurrent triplet stimulation sometimes led to failure of the subject to perceive one of the expected phosphenes (Fig. 5E-H). In this example, again two of the electrodes used were located close together on the cortical surface, and one was located at a greater separation on the cortical surface (red circles Fig. 5E). Again, we would expect two peaks of activation in early visual cortex (Fig. 5F) and perception of two phosphenes (Fig. 5G). However, in this case the subject reported perception of only one phosphene, in a location in visual space that was overlapping with the two phosphenes produced by individual stimulation of the two nearby electrodes (Fig. 5H). It appears that no phosphene was perceived associated with the third electrode, and this can be seen as analogous to Type 1B pair results.

3.8. Summary of concurrent triplet stimulation

Overall, we tested concurrent electrical stimulation of 69 triplets from 7 subjects (Table 1). Limitations imposed by the clinical environment precluded us from being able to sample all possible triplet combinations, so we selectively biased our sampling towards attempting triplet stimulation with groups of electrodes we thought were likely to produce perception of multiple phosphenes. For the 69 triplets tested, 8 had all electrodes in V1, 4 had all electrodes in V2, 36 had electrodes in V1 and V2, 3 had electrodes in V1 and V3, 6 had electrodes in V2 and V3, and 12 had electrodes including all three areas.

3.9. Stability of spatial configurations: example case

To examine the stability or robustness of the phosphene locations reported under different stimulation conditions, we first show an example from a case (YAN) in which the same three electrodes were stimulated individually, in simultaneous pairs, or as a simultaneous triplet (Fig. 6). The location of the phosphenes obtained from pair stimulation (Fig. 6C-E) was well predicted by those obtained from independent stimulation (Fig. 6B), with some small shifts in position apparent on individual trials. The location of the phosphenes obtained with simultaneous triplet stimulation (Fig. 6F) was consistent with locations obtained from the independent stimulation.

Fig. 6. Relationship of single electrode stimulation, pair stimulation, and triplet stimulation.

A) Location of the three electrodes on the occipital cortex (red, blue, and green filled circles) of subject YAN. B) Location of perceived phosphenes when the three electrodes were stimulated individually on multiple (3–5) trials. Ellipses are color coded to correspond to the electrodes in panel A. C-E) Location of perceived phosphenes when different pairs of electrodes were stimulated on multiple (3–5) trials. F) Location of perceived phosphenes when all three electrodes were stimulated simultaneously on multiple (5) trials.

3.10. Spatial configuration of phosphene patterns obtained with triplet stimulation

While the number and spatial configuration of phosphenes obtained with concurrent triplet stimulation was generally in accord with results from pair stimulation, and consistent with the cortical activation model, we found that there could be considerable variability in the absolute location of the pattern of phosphenes reported on individual trials. In addition, there were sometimes substantial differences in measured phosphene locations between the concurrent stimulation and independent stimulation conditions.

We first show two examples from one subject illustrating the type of trial-to-trial variations that were observed (Subject YAO; Fig. 7). Example 1 from this case used electrodes 1 (blue), 10 (red), and 11 (green) (electrode locations Fig. 7A; data with corresponding colors in Fig. 7B-D). In this case the electrode strip runs roughly orthogonal to the calcarine sulcus on the medial wall of the occipital cortex, and the three electrodes lie in different visual areas (V1, V2, V3). Multiple trials were conducted to map the phosphenes that were produced by individual stimulation of the three electrodes (Fig. 7B) and by simultaneous stimulation of all three electrodes (Fig. 7C). The phosphene locations obtained with triplet stimulation (ellipses, Fig. 7C) are in roughly the locations expected from the mean of the individual stimulation trials (asterisk symbols, Fig. 7C). There is moderate variability in the exact locations of the phosphenes perceived on individual trials. However, if we align each trial by transformations including translation, rotation, and scaling (Fig. 7D; methods), we can see that the overall spatial pattern or configuration of phosphenes obtained on each trial was actually very similar.

Fig. 7. Stability of spatial patterns obtained by triplet stimulation.

A) Location of electrode strip on the cortical surface (subject YAO). B-D) Example results from triplet containing electrodes (1-10-11). B) Location of the phosphenes associated with independent stimulation of electrode 1 (blue, 5 trials), electrode 10 (red, 3 trials) and electrode 11 (green, 3 trials). Asterisks indicate mean locations. C) Raw locations of the phosphenes associated with each electrode when the triplet was stimulated simultaneously (4 trials). Dashed lines connect phosphenes associated with each electrode obtained on the single trials, and are used to allow better visualization of the spatial configuration of phosphenes obtained on each trial. Asterisks indicate mean locations of phosphenes from independent stimulation trials. D) Location of the phosphenes associated with each electrode following removal of translation, rotation, and scaling errors on each trial. E-G) Example results from triplet containing electrodes (1-6-12). E) Location of the phosphenes associated with independent stimulation of electrode 1 (blue, 5 trials), electrode 6 (purple, 3 trials) and electrode 12 (orange, trials) when stimulated individually on multiple trials. F) Raw location of the phosphenes associated with each electrode when the triplet was stimulated simultaneously (4 trials). G) Location of the phosphenes associated with each electrode following removal of translation, rotation, and scaling errors on each trial.

In the second example from this case, we used a triplet composed of electrodes 1 (blue), 6 (purple), and 12 (orange) (electrode locations Fig. 7A; data Fig. 7E-G). These electrodes lie in two different visual areas (V1, V2). For this triplet, there is both a larger difference in location of the phosphenes perceived between independent (Fig. 7E) and simultaneous (Fig. 7F) stimulation conditions, and a greater trial-to-trial variability obtained within the simultaneous stimulation condition (Fig. 7F). In general, the three phosphenes were separated by larger distances in visual space with simultaneous stimulation. In addition, on one of the trials there was a substantial change both in the reported separation of the phosphenes, and in the orientation of the overall pattern in visual space. However, when we align each trial allowing translation, rotation, and scaling, we can again see that a very consistent spatial pattern of phosphenes was observed on each trial (Fig. 7G).

This pattern of results was observed across all of the triplets for which we had phosphene drawings (n = 37 triplets from four cases, 1–5 trials). To quantify the variability in reporting of phosphene locations obtained with multi-electrode stimulation, we calculated the scatter in reporting of phosphene location across trials in the simultaneous condition with respect to the mean location reported for the corresponding phosphene in the independent condition. This was quantified for the raw data (R), and then following transformations which allowed only translation (T), translation and rotation (TR), or translation, rotation, and scaling (TRS) (Fig. 8A). Overall, using a repeated measures ANOVA we found significant differences between the different alignment methods (F(3) = 7.928; p < .001). Post-hoc pairwise comparisons reveal significant differences between the R and T (p < .05), R and TR (p < .001) and R and TRS (p < .001) alignment methods. The difference between the T and TR methods was not significant (P > .05), while the differences between T and TRS methods and between the TR and TRS methods were significant (p < .001).

The amount of translational correction required to align individual trials was typically fairly small (0–2°), as would be expected for small errors in establishing a consistent fixation location on each trial, but on some trials was as large as 3–5° (Fig. 8B). The amount of rotational correction required was also usually small (0–10°), as might be expected for small errors in establishing the cardinal axes and a reporting framework, but on some trials could be as large as 15–45° (Fig. 8C). We found a large range of scaling factors were required to best align the different triplet stimulation trials (Fig. 8D). The scaling factors did not cluster at one value for all trials of all triplets, which indicates that our results cannot be accounted for by a simple change in monitor distance or by a consistent shift in the depth plane at which the phosphenes were perceived by the subject. The reported phosphenes were separated by a larger distance in the simultaneous condition for many of the triplets that were tested (scaling factors greater than 1), and that this is analogous to the larger separation for phosphene pairs in the simultaneous condition which was illustrated earlier (Fig. 4F).

3.11. Overall robustness of phosphene locations when stimulated electrodes are included in different patterns

In the analysis of triplet data just presented, we demonstrated the relative stability of the spatial pattern of phosphenes obtained with electrical stimulation of a single set of electrodes on multiple trials. A separate but related question is the extent to which phosphenes associated with stimulation of a particular electrode are reported in a consistent location in visual space as the electrode is included in different multi-electrode groups. To illustrate this, we show data from one case where many triplets were stimulated (Subject YAO; Fig. 9). The location of phosphenes associated with a particular electrode are shown in the same color across the full set of triplets. Examination of the raw data (Fig. 9A) indicates that phosphenes associated with a particular electrode did tend to remain in the same region of visual space no matter which group of electrodes was tested. However, there was considerably more trial-to-trial variability in phosphene location when testing pairs or triplets compared to individual phosphenes. However, after aligning the raw data from each triplet using translation, rotation, and scaling (Fig. 9B), we found a more consistent location of the phosphenes associated with each electrode.

Fig. 9. Overall robustness of phosphene location across testing with different multi-electrode groups.

Data are taken from all triplets tested for subject YAO. A) Raw data from all triplets. The centers of phosphenes associated with a stimulation of a particular electrode (circle symbols) are presented in the same color across all the triplets in which that electrode was tested. For example, phosphenes associated with stimulation of electrode 1 are shown in blue for all triplets which included electrode 1. Square symbols indicate the location of the phosphene associated with each electrode when that electrode was stimulated in isolation. B) Data from each triplet after alignments including translation, rotation, and scaling relative to the phosphenes obtained from independent testing of each electrode. Circle symbols show the data from each electrode in the same colors as in panel A and square symbols indicate the centers of phosphenes obtained with independent stimulation of each electrode.

3.12. Results from stimulation of more than three electrodes

In a limited number of cases, we tested simultaneous stimulation of 4–6 electrodes. We show an example in which four electrodes located in V1 were stimulated simultaneously (Subject YAU; Fig. 10A-D). For this multi-electrode group, we predicted three peaks of cortical activation (Fig. 10B), and perception of three phosphenes (Fig. 10C), however the subject perceived four phosphenes (Fig. 10D). The phosphenes reported by the subject were smaller in size and subtended a larger region of visual space than predicted by receptive field locations and predicted cortical activation, in a manner similar to the effects reported for pair and triplet stimulation. When an additional electrode was added so that simultaneous stimulation now included five electrodes (Fig. 10E-H), the subject then reported perception of five phosphenes.

Fig. 10. Simultaneous stimulation of greater than three electrodes.

Results from one subject (YAU) for simultaneous stimulation of groups of 4 (top row) and 5 (bottom row) electrodes. A) Location of the electrodes used in the quadruplet. B) Predicted cortical activation for simultaneous stimulation of the quadruplet depicted on flat map. C) Predicted phosphenes for simultaneous stimulation of this quadruplet. D) Actual phosphenes obtained with independent (grey) and simultaneous (red) stimulation of the four electrodes. E) Location of electrodes used in the quintuplet tested. F) Cortical activation predicted for simultaneous activation of the quintuplet. G) Predicted phosphenes for simultaneous stimulation of the quintuplet. H). Actual phosphenes obtained with individual stimulation (grey) and simultaneous stimulation (red) of the five electrodes.

While subjects could perceive as many as 4–5 phosphenes with simultaneous stimulation of 4–6 surface electrodes, we found that as the number of electrodes was increased, they had increasing difficulty in accurately report the number and location of phosphenes observed on a single trial. Accurately recovering the location of all phosphenes perceived was often a serial process requiring multiple trials.

3.13. Behavioral testing

Discussions with our subjects revealed that they did not perceive coherent shapes or easily identifiable forms with multi-electrode stimulation. Instead, they reported the perception of independent phosphenes or patches of light that combined or remained independent depending on the spacing of the tested electrodes. For example, the subjects did not perceive a triangle with simultaneous stimulation of three electrodes, but instead one, two, or three independent phosphenes depending on the separation of the electrodes on the surface. Nevertheless, we examined whether they could reliably discriminate between the pattern of phosphenes obtained with simultaneous stimulation of two different sets of three electrodes (Fig. 11). These were either two alternative forced choice (2AFC) or three alternative forced choice (3AFC) experiments in which the subject was stimulated with one pattern of electrodes, and then asked to report which pattern they perceived by selecting from two or three visual stimuli. The visual stimuli were created by combining the phosphenes drawn by the subject for each of the individual phosphenes for the triplets being tested. Subject performance was above chance (33% in the 3AFC, 50% in the 2AFC) in all but one experiment.

4. Discussion

Multiple aspects of our results are consistent with a framework in which the number of and pattern of phosphenes produced by multi-electrode stimulation can be predicted by a model for cortical activation and the location of the electrodes within the map of visual space in early visual cortical areas. This includes: 1) the observed phosphenes were in the expected region of visual space, 2) the number of phosphenes perceived could be predicted by the separation of the electrodes on the cortical surface, 3) the spatial configuration of the phosphenes observed could be predicted based on data from individual electrodes and was stable from trial-to-trial, and 4) subjects perceived stable patterns of phosphenes based on concurrent stimulation of electrodes in different visual areas. However, other elements of the results reveal greater complexity, including: 1) trial-to-trial variability in the exact location, rotation, and size of the spatial pattern perceived, 2) apparent changes in the separation between phosphenes when using concurrent vs. independent stimulation of the same electrodes, and 3) the subject failing to notice or report phosphenes associated with particular electrodes.

4.1. Predicting the number of phosphenes perceived

We found that the cortical separation required to evoke two distinct phosphenes with simultaneous stimulation of two electrodes with milliamp currents was ~4 mm. This estimate for a cortical two-point discrimination distance is similar to those from previous investigations that used large surface electrodes to estimate this metric [4,8,24,32]. Experiments utilizing penetrating micro-electrodes in humans [9,33] and non-human primates [25,34-37] have typically found lower thresholds for generation of phosphenes (single to tens of microamps), a smaller radius of cortical activation, and a smaller cortical two-point discrimination distance (hundreds of microns). However, data from a recent clinical trial of a VCP that utilized penetrating electrodes provided a somewhat more variable estimate of cortical activation and two-point discrimination distance [22,38].

We found one important exception to this framework. Stimulation of some pairs of electrodes that were separated by considerable distances on the cortical surface evoked perception of only one phosphene. When this happened, the location of the phosphene obtained with simultaneous stimulation was invariably near the location of only one of the two phosphenes obtained with independent stimulation. This implies the subject failed to perceive or report one of the two phosphenes in the simultaneous condition. Stimulation of three or more electrodes also led to cases where subjects failed to report the phosphene associated with a particular electrode.

One possible reason for this is that the phosphenes associated with different electrodes can differ substantially in brightness or other attributes. In experiments with blind subjects, we have found that careful balancing of currents delivered to each electrode to compensate for variability in phosphene brightness can aid in multi-electrode mapping [18,19]. Another possibility is that subjects did not uniformly distribute their spatial attention throughout the task.

4.2. Changes in phosphene perception with multi-electrode stimulation

While the number and spatial pattern of phosphenes perceived with multi-electrode stimulation was generally consistent with our expectations from single electrode stimulation, there were several observations that indicate more subtle interactions.

First, we found considerable trial-to-trial variability in the absolute location of the pattern of phosphenes reported with multi-electrode stimulation. We expected that the primary source of this variability would be small changes in eye position at the beginning of each stimulation trial, and we found that alignment of individual trials using the center of mass of the reported phosphenes significantly improved consistency but did not remove all variability.

Second, there were changes in the apparent size and separation between phosphenes, both for pairs, and for larger multi-electrode groups. This result could not be accounted for by simple changes in monitor position, or the depth plane at which the phosphenes were perceived, between testing sessions. These changes could reflect long distance excitatory and inhibitory interactions within early visual cortex, or changes in the way that activity patterns in early visual cortex are readout by downstream areas. Further testing will be required to resolve these possibilities, ideally including direct measurement of activity patterns in early visual areas during multi-electrode stimulation.

Finally, we found variability in the orientation in visual space of the perceived pattern of phosphenes. This could result from small changes in the subjects’ assignment of a reference frame on each trial, and we have reason to believe that such variability may be even larger in blind subjects [19]. It is also possible that there were accuracy trade-offs induced by having to report the location of multiple phosphenes. For example, it could be difficult to provide a report that is both very accurate in terms of the relative spatial arrangement of the phosphenes and very accurate in terms of the absolute location and rotation angle of the pattern in visual space.

4.3. Robustness of phosphene locations across tested conditions

We found that the phosphene associated with stimulation of a particular electrode remained in the same region of visual space as that electrode was included in different electrode groups, and that precision of phosphene location improved as we allowed for small variations in the location, rotation, and scaling of patterns that were observed on single trials. In separate experiments with blind subjects, we found that the final estimate of the phosphene locations for the full set of implanted electrodes can be further improved by fitting a standard map of visual space to the aligned multi-electrode stimulation data [19].

4.4. Pattern perception and discrimination with multi-electrode stimulation

Our subjects perceived reliable spatial configurations of phosphenes with multi-electrode stimulation and could discriminate between two or three different multi-electrode patterns at a level that was significantly greater than chance. These results are consistent with those from an early prototype VCP [7], although here we provide better documentation of the phosphene patterns that were perceived, the variability across trials, and the amount of training that was necessary. Similar levels of pattern discrimination performance have been obtained in recent experiments using penetrating electrodes [21,22].

The moderate level of performance that we observed in the discrimination tasks could be related to the fact that our subjects did not report the perception of coherent or easily recognizable visual forms with concurrent multi-electrode stimulation, and that they often required a serial reconstruction process distributed over multiple trials to be able to report the location of all the phosphenes perceived, or even to be sure how many phosphenes were perceived. These observations led us to look for ways to improve the coherence of patterns perceived on single trials. We found that rapid dynamic stimulation of a sequence of electrodes produced visual percepts that subjects could much more easily identify and discriminate [18], and similar results have been obtained with retinal stimulation [39,40]. It is possible that subjects could also improve discrimination performance with structured training, or with alternative stimulation strategies such as use of stimuli that are repetitively flashed.

It is not surprising that concurrent multi-electrode stimulation of a few surface electrodes does not evoke perception of coherent forms. This could be due to many reasons including gaps between the phosphenes that are associated with each electrode, differences in the size, shape, color, and texture of the phosphenes produced by each electrode, or that the full boundary of visual space defined by the set of phosphenes did not correspond to a recognizable object. In addition, the retina and early visual pathways normally provide input to early visual cortex that is structured very differently in terms of the layers activated, the cell types and functional columns activated, and the timing or spatial coherence of the activation. It has been assumed that VCPs based on penetrating electrodes would allow for better access to early visual cortex, and an enhanced ability to produce perception of visual forms. Recent studies in non-human primate [21] and human [22] subjects have demonstrated that VCPs based on implantation of 100–1000 penetrating electrodes are possible, and the subjects in these studies could make some simple discriminations based on multi-electrode stimulation. However, it remains unclear whether concurrent stimulation of large numbers of penetrating electrodes can be used to reliably generate perception of arbitrary visual forms, or whether additional measures such as dynamic stimulation, current steering and shaping, or use of biomimetic stimulation timing may be required.

5. Conclusions

Our results provide support for the idea that stimulation of subsets of implanted electrodes could be used to reliably convey spatial pattern information in future VCP recipients, but also highlight the complexity of this challenge. In blind subjects with implanted VCPs, it will be crucial to develop stable testing and reporting conditions to evaluate the results of multi-electrode stimulation. In addition, the ability to use VCPs to evoke the perception of coherent visual forms will require continued improvements to both the implanted hardware (size, spacing, and visual field coverage of the implanted electrode arrays) and the electrical stimulation paradigms used to convey information to the subject.

The current results carry implications not only for the development of future VCPs and other BCI applications, but also for understanding the relationship between cortical activity and perception. Electrical stimulation in our experiments likely resulted in activation of a region 2–6 mm in diameter around each electrode. Our results confirm that direct activation of a region this large V1, V2, or V3 results in a visual percept [41], and that the size and location of the visual percept is correlated with the size and location of the activation within the map of visual space in early visual areas [24,32]. In addition, we now provide evidence that the number of phosphenes perceived is correlated with the number of discrete regions of activity in early visual cortex, and that stable patterns of phosphenes can be obtained by combing electrical stimulation in different visual areas.

While concurrent multi-electrode stimulation that produces large regions of activity within early visual field maps is sufficient to produce course visual percepts, it has so far been ineffective as a mechanism to reliably evoke perception of recognizable visual forms. It may be necessary to obtain fine scale activation of early visual cortex at the level of the appropriate functional columns to activate the normal visual pathways which lead to form perception. Even with development of interfaces that have adequate spatial resolution, and adequate coverage of both the map of visual space and functional maps within early visual cortex, it may still be necessary to use dynamic stimulation or other alternative electrical stimulation strategies to obtain better form perception. Overall, it will be important to combine electrical stimulation with measurement of activity across multiple areas of visual cortex to further determine the full spatial temporal pattern of activity that leads to perception of simple spots of light or to coherent contours.