Multi-electrode stimulation evokes consistent spatial patterns of phosphenes and improves phosphene mapping in blind subjects

Abstract

Background:

Visual cortical prostheses (VCPs) have the potential to restore visual function to patients with acquired blindness. Successful implementation of VCPs requires the ability to reliably map the location of the phosphene produced by stimulation of each implanted electrode.

Objective:

To evaluate the efficacy of different approaches to phosphene mapping and propose simple improvements to mapping strategy.

Methods:

We stimulated electrodes implanted in the visual cortex of five blind and fifteen sighted patients. We tested two fixation strategies, unimanual fixation, where subjects placed a single index finger on a tactile fixation point and bimanual fixation, where subjects overlaid their right index finger over their left on the tactile point. In addition, we compared absolute mapping in which a single electrode was stimulated on each trial, and relative mapping with sequences containing stimulation of three to five phosphenes on each trial. Trial-to-trial variability present in relative mapping sequences was quantified.

Results:

Phosphene mapping was less precise in blind subjects than in sighted subjects (2DRMS, 16±2.9° vs. 1.9±0.93°; t(18) = 18 , p = <0.001). Within blind subjects, bimanual fixation resulted in more consistent phosphene localization than unimanual fixation (BS1: 4.0±2.6° vs. 19±4.7°, t(79) = 24, p <0.001; BS2 4.1±2.0° vs. 12±2.7°, t(65) = 19, p <0.001). Multi-point relative mapping had similar baseline precision to absolute mapping (BS1: 4.7±2.6° vs. 3.9±2.0°; BS2: 4.1±2.0° vs. 3.2±1.1°) but improved significantly when trial-to-trial translational variability was removed. Although multi-point mapping methods did reveal more of the functional organization expected in early visual cortex, subjects tended to artificially regularize the spacing between phosphenes. We attempt to address this issue by fitting a standard logarithmic map to relative multi-point sequences.

Conclusions:

Relative mapping methods, combined with bimanual fixation, resulted in the most precise estimates of phosphene organization. These techniques, combined with use of a standard logarithmic model of visual cortex, may provide a practical way to improve the implementation of a VCP.

Introduction

Technological advances have revived interest in prosthetic approaches to treat acquired blindness. In recent years, several groups have brought visual cortical prostheses (VCPs) to the late development stage or to actual clinical trials [1]–[3], as reviewed by [4]–[7]. VCPs consist of a camera that captures images of the world, a processing module that translates images into stimulation patterns, and a set of electrodes implanted in or on visual cortex. This approach bypasses damage to early visual structures such as the retina or optic nerve, to deliver information directly to the brain. The basis for VCPs is built on two observations: firstly, electrical stimulation of single electrodes in early visual cortex produces a visual percept, or phosphene, in a discrete part of visual space [8]–[12], and secondly, visual cortex contains a retinotopic map of the world [13]–[18]. In theory, multiple electrodes in early visual cortex could be stimulated in precise spatiotemporal patterns to evoke the perception of specific visual forms or the entire visual scene.

There are several requirements that need to be met for VCPs to be able to provide useful information to users. Firstly, electrical stimulation of visual cortex must still produce visual sensations after loss of sight. Evidence suggests that, while there may be some changes to cortical excitability, even patients with long-standing acquired blindness can still perceive phosphenes [9], [11], [19], [12], [20], [21] as long as their visual cortex is undamaged. Secondly, a structured map of visual space also needs to remain intact. While cortical plasticity following long-term deafferentation may change the size of cortical areas due to functional repurposing [22]–[27], previous studies indicate that the map of visual space persists in cortex years after loss of sight [11], [12], [21]. In addition to these biological requirements, to successfully deploy a VCP requires methods that are reliable and efficient at phosphene mapping [9], [28], [19], the process of determining the location of the phosphene generated by electrical stimulation of the implanted electrodes.

In practice, phosphene mapping has been reproached in two ways, absolute and relative [9], [29]. For absolute mapping, the subject reports the location of a single phosphene relative to a central fixation point. For relative mapping, the subject reports the spatial relationship between phosphenes (relative angle and distance). In sighted subjects, phosphene mapping is typically conducted with an absolute approach and is a straightforward process. Subjects visually fixate on a monitor placed in front of them, and after electrical stimulation is delivered to an electrode, they make visually guided movements to point to or draw the location of the perceived phosphene on the monitor, with respect to the fixation point. This type of phosphene mapping has been conducted in sighted epilepsy subjects, and in general has resulted in determination of phosphene locations for each electrode that closely match the receptive field (RF) location measured for the same electrodes [5], [30], [31].

There is good reason to expect mapping phosphene locations would be more difficult in blind subjects. Without visual inputs, blind subjects are unable to take advantage of either visual fixation or visually guided pointing. Rather than visual fixation, blind subjects are instructed to use one hand to maintain contact with a tactile fixation point, and to report the location of an electrically evoked visual sensation with their other hand [11], [12], [32], using a reaching movement, for which their only feedback is proprioception of the hand fixating and the arm used for pointing. The limited tactile cues yield an impoverished framework for the reporting environment and provide less feedback that could be used to compensate for subtle errors that occur on each trial, which may be exaggerated in absolute mapping. Both absolute and relative methods have been attempted in blind subjects. The investigators that created the first prototype VCPs used both methods and reported on some of the differences between the two methods [9], [11], [32], [33]. However, no systematic and quantitative investigation of the reliability of the two methods has been conducted in blind subjects.

The main rationale behind use of relative mapping methods is that most of the error in phosphene mapping results from small errors in gaze direction at the start of each trial. It is assumed that by stimulating two or more electrodes on a single trial the relative arrangement between the phosphenes associated with each electrode can be measured. However, for relative mapping to be successful it is necessary that the spatial pattern or configuration of phosphenes perceived by the subject be stable across trials. There are reasons to believe that spatial configurations are stable across trials and can be used to guide discriminations [21], [34] but this observation and its relationship to the improvement in phosphene mapping reliability have not yet been carefully examined.

In this report, we utilize data from two rare populations, sighted patients undergoing monitoring for medically refractory epilepsy and blind patients implanted with an early generation VCP. We found blind subjects to have significantly impoverished performance in phosphene localization relative to sighted subjects. This comparison was used to demonstrate the need for mapping methods tailored specifically to the blind subjects who will be the recipients of the next generation of VCPs. We approached this goal in two ways. The first technique was to improve the quality of the tactile fixation with a bimanual approach, using the index finger from each hand on the tactile fixation point. The second technique was to use relative mapping methods (stimulating two or more electrodes per trial in sequence). Finally, we show how relative mapping methods could be more effective at revealing some important features of the map of visual space and propose a combination of relative mapping and a standardized logarithmic map of visual space to adequately capture the organization and structure specific to each subject.

Methods

Subjects

All research and protocols were approved by Institutional Review Boards at Baylor College of Medicine (BCM) and University of California, Los Angeles (UCLA), and all subjects gave written informed consent. Data were collected from fifteen sighted and six blind subjects. Sighted subjects were patients with medically refractory epilepsy treated at BCM. Subjects were male and female, aged 22-61, with a mean age of 35. Sighted subjects were hospitalized in the epilepsy monitoring unit for 4-14 days following surgical implantation of subdural electrode grids and strips. Fifteen sighted subjects were included in this report. The original case identifiers for these subjects were LF, MR, YAA, YAB, YAC, YAE, YAF, YAH, YAI, YAM, YAN, YAO, YAU, YAV, YAX, and can be used to compare with earlier and future reports using these subjects. For ease of referral, the remainder of this report will refer to each sighted subject by a designated local identifier, SS1 through SS15, respectively.

All blind subjects are participants in an ongoing early feasibility study (NCT03344848) for the Orion visual cortical prosthesis (Second Sight Medical Inc.) being conducted at BCM and the University of California, Los Angeles. Subjects were male and female, aged 29-64, mean age 49. All subjects had usable vision in early life, and late onset blindness. Causes of blindness for each subject are summarized below in Table 1. For ease of referral in the remainder of this report each subject has been given local identifiers in addition to their clinical trial identifiers as indicated in the table below.

Table 1.

Blind subject information. Table indicates subject global and local identifiers, study site, age at time of implant, age of onset of blindness, and cause of blindness.

Site	Global ID	Local ID	Age of Onset	Age at Implant	Gender	Etiology	Bare Light Perception
UCLA	02-659	BS1	22	29	Male	Head trauma	None
BCM	03-281	BS2	45	57	Male	Head Trauma	None
UCLA	02-334	BS3	53	56	Male	Optic neuropathy secondary to burn/trauma	Minimal in contralateral eye
UCLA	02-941	BS4	63	64	Female	Retina damage secondary to liver abscess	None
UCLA	02-182	BS5	20	52	Male	Congenital Glaucoma	None

Electrodes and Electrical Stimulation

Sighted subjects were implanted with subdural grids and strips with standard clinical electrodes (3mm diameter) for monitoring of epileptogenic activity. Placement of electrodes in these subjects was guided by clinical criteria. In most sighted subjects, select implanted subdural strips also contained additional research mini electrodes (0.5 mm diameter) imbedded in between the larger clinical contacts (SS3 – SS15), two subjects (SS1 and SS2) had grids containing only the standard clinical electrodes implanted. Research strips were one of three configurations. Schematics of each array configuration implanted in each sighted subject are available in Table S1.

Clinical and research grids and strips were manufactured by PMT (Chanhassen, MN). Electrical stimulation was performed with a 16-channel AlphaLab SnR (Alpha Omega, Alpharetta, GA) and controlled by a custom user interface developed in MATLAB (Version 2013b, The MathWorks Inc, Natick, MA). All stimulation was monopolar, grounded to a return pad placed on the subject’s thigh. Stimulation was comprised of pulse trains 200 ms in duration composed of biphasic 0.1 ms per phase, square, symmetric pulses, delivered at 200 Hz.

Blind subjects were each implanted with the Orion Visual Cortical Prosthesis System (Second Sight Medical Products Inc, Sylmar CA). The implanted system consists of an array of 60 electrodes, 2 mm in diameter, spaced 3 mm apart diagonally and 4.2 mm apart within rows, center-to-center (Figure 1C), and an internal processing control module imbedded in the skull. Each 2 mm electrode consisted of an electrically contiguous group of thirty-seven 0.2 mm diameter circular contacts, made from sputtered platinum gray on silicone. The implanted processing module delivers electrical stimulation and acts as the return for monopolar stimulation. Electrical stimulation was controlled via a software interface developed by Second Sight Medical Products. Stimulation consisted of 100-250 ms duration pulse trains composed of biphasic 0.2 ms per phase, square, symmetric pulses, delivered at 20, 60, or 120 Hz. Pulse duration used was the standardized value used across the clinical trial. Pulse frequency increments and maximum value were limited by the hardware capabilities of the Orion Visual Processing Unit (VPU). Stimulation implemented with BS1 was conducted at 60 Hz at the request of the clinical trial sponsor for the safety of this subject. Stimulation carried out with BS2 was conducted at 120 Hz. A summary of stimulation parameters specific to each task is available below in Table 2.

Figure 1.

Discrepancy in reported phosphene precision between sighted and blind subjects. A. Task flow. Subjects were instructed to fixate on a point on a touchscreen monitor placed in front of them at eye level, while a pulse train of electrical stimulation was delivered to a single electrode. An auditory tone indicated the end of stimulation and cued subjects to report the phosphene location. B. Sighted subjects conducted tasks seated in their hospital bed and directed their gaze toward a centrally located fixation cross. Stimulation was delivered by an Alpha Omega neural stimulator. Subjects reported location of phosphenes on touchscreen display. C. Electrode array implanted in blind subjects. Contact numbers are indicated from 1 (upper right) to 60 (lower left). D. Blind subjects were seated in a chair in a laboratory testing room and fixated by placing their left index finger on a tactile point on the monitor. Electrical stimulation was delivered by the Visual Processing Unit (VPU) of their Orion VCP. Subjects indicated the location of the perceived phosphene with their right index finger. E. Precision in reported phosphene location, quantified as 2DRMS, for individual sighted (SS1 – SS15) and blind subjects (BS1 – BS5). Each data point reflects 2DRMS for an individual electrode. Error bars indicate mean and one standard deviation. F. Group precision (sighted vs. blind) data. Each point reflects the average precision (2DRMS) for a subject, across all electrodes evaluated for that subject. Error bars indicate group average and standard deviation.

Table 2.

Stimulation parameters by subject and task. Table presents stimulation parameters used during each task and each subject.

Subject	Task	Fixation	Frequency (Hz)	Pulse Width (ms)	Duration (ms)
SS1	Thresholding	None	200	0.1	200
SS2	Thresholding	None	200	0.1	200
SS3	Thresholding	None	200	0.1	200
SS4	Thresholding	None	200	0.1	200
SS5	Thresholding	None	200	0.1	200
SS6	Thresholding	None	200	0.1	200
SS7	Thresholding	None	200	0.1	200
SS8	Thresholding	None	200	0.1	200
SS9	Thresholding	None	200	0.1	200
SS10	Thresholding	None	200	0.1	200
SS11	Thresholding	None	200	0.1	200
SS12	Thresholding	None	200	0.1	200
SS13	Thresholding	None	200	0.1	200
SS14	Thresholding	None	200	0.1	200
SS15	Thresholding	None	200	0.1	200
BS1	Staircase threshold	None	20	0.2	250
BS2	Staircase threshold	None	20	0.2	250
BS3	Staircase threshold	None	20	0.2	250
BS4	Staircase threshold	None	20	0.2	250
BS5	Staircase threshold	None	20	0.2	250
BS1	Current Selection	None	60	0.2	100
BS2	Current Selection	None	120	0.2	100
SS1	Absolute Mapping	Visual	200	0.1	200
SS2	Absolute Mapping	Visual	200	0.1	200
SS3	Absolute Mapping	Visual	200	0.1	200
SS4	Absolute Mapping	Visual	200	0.1	200
SS5	Absolute Mapping	Visual	200	0.1	200
SS6	Absolute Mapping	Visual	200	0.1	200
SS7	Absolute Mapping	Visual	200	0.1	200
SS8	Absolute Mapping	Visual	200	0.1	200
SS9	Absolute Mapping	Visual	200	0.1	200
SS10	Absolute Mapping	Visual	200	0.1	200
SS11	Absolute Mapping	Visual	200	0.1	200
SS12	Absolute Mapping	Visual	200	0.1	200
SS13	Absolute Mapping	Visual	200	0.1	200
SS14	Absolute Mapping	Visual	200	0.1	200
SS15	Absolute Mapping	Visual	200	0.1	200
BS1	Absolute Mapping	Unimanual	60	0.2	100
BS2	Absolute Mapping	Unimanual	120	0.2	100
BS3	Absolute Mapping	Unimanual	20	0.2	250
BS4	Absolute Mapping	Unimanual	20	0.2	250
BS5	Absolute Mapping	Unimanual	20	0.2	250
BS1	Absolute Mapping	Bimanual	60	0.2	100
BS2	Absolute Mapping	Bimanual	120	0.2	100
BS1	Relative Mapping	Bimanual	60	0.2	100
BS2	Relative Mapping	Bimanual	120	0.2	100

Electrode localization

Pre- and post-surgical imaging was used to determine electrode locations for each subject. Prior to surgery, subjects underwent T1-weighted structural MRI in a 3T scanner. These scans were used to create cortical surface models using FreeSurfer [35], [36]. Whole-head CT was conducted post-implant and alighted to pre-surgical imaging using Analysis of Functional Neuroimaging (AFNI) software [37]. Electrode locations were manually determined using a combination of AFNI and SUMA [38] and projected to the nearest node of the cortical surface model using custom methods developed in MATLAB (Version 2019a, The MathWorks Inc, Natick, MA).

Screening and threshold determination

Screening sessions were conducted to determine which electrodes reliably produced phosphenes when electrically stimulated and to determine the current amplitude at which each subject could reliably perceive and localize a phosphene. For sighted subjects, screening trials consisted of an auditory cue followed by electrical stimulation and a verbal report by the subject of whether stimulation evoked a percept. The current amplitude was stepped from 0.3-4.0 mA, until the subject reported a phosphene. Electrodes that did not evoke a phosphene at 4.0 mA were excluded. For the phosphene mapping experiments reported here, the current used for testing was selected to be above threshold such that stimulation always produced a phosphene that was easy to perceive and localize on every trial.

For blind subjects, electrodes with impedance above 18 kΩ were disabled. Initial viability and thresholds for each electrode was determined using a staircase threshold procedure set by the clinical trial sponsor. Electrical stimulation was applied at currents incremented from 0 to 8 mA, with three repeats at each increment. Threshold was determined as the lowest current for which three consecutive trials produced a phosphene. Currents used during comparative mapping tasks were secondarily adjusted by incrementing the current amplitude delivered to each electrode until the subject could easily perceive and localize each phosphene. Qualitative adjustments were made to equalize the subjective brightness and size of each phosphene perceived.

Fixation techniques

Sighted subjects were instructed to visually fixate on a 0.5° cross presented on the monitor by training their gaze and focusing their attention on this point. On each trial, subjects were asked to maintain fixation from before the onset of electrical stimulation until they had completed their report of phosphene location on the touchscreen.

Blind subjects were instructed to fixate on a tactile point placed on the touchscreen monitor. They were asked to focus their attention on the tactile point at the tip of their finger(s) and to imagine looking toward this point. If they retained an eye(s), they were asked to keep their eyes still and pointed toward the tactile point to the best of their abilities. Two forms of tactile fixation were evaluated in blind subjects, unimanual and bimanual. For unimanual fixation subjects were instructed to place their left index finger on a tactile point approximately 0.5° in diameter that was placed on the monitor, and to direct their attention toward that digit (Figure 2A, upper). Following stimulation, the subject used their right index finger to indicate phosphene location, while maintaining fixation on their left index finger. This fixation protocol was used for the initial comparison of phosphene variability between sighted and blind subjects. For bimanual fixation, subjects were instructed to place their left index finger on the tactile point and to overlay their right index atop their left, and then focus their attention on both fingertips (Figure 2A, lower). The subject reported phosphene location with their right index finger, while maintaining attention on the fixation point. This protocol was used for the comparison between absolute and multi-point relative mapping strategies.

Figure 2.

Precision of reported phosphene location for unimanual and bimanual fixation. A. Task descriptions. Unimanual fixation (top), for which subjects fixated with their left index finger and report with their right index finger. For bimanual fixation (bottom), subjects placed both left and right index fingers on the fixation point, then reported phosphene location with their right index finger, while leaving their left index finger on the tactile fixation point. B. Precision for bimanual and unimanual fixation for BS1 (red) and BS2 (blue). Each data point reflects precision (2DRMS) for a single electrode. Error bars indicate mean and standard deviation.

Phosphene mapping strategies

We compared two phosphene mapping techniques, absolute and multi-point relative. Absolute mapping was conducted in all sighted and blind participants, multi-point mapping was evaluated only in blind subjects BS1 and BS2. The stimulation parameters used with each subject and each task are summarized later in Table 2.

Absolute mapping

Sighted subjects were seated comfortably in front of a touchscreen monitor (Wacom, Toyonodai, Kazoshi, Saitama, Japan) placed 28.6 to 57.3 cm in front of them and instructed to visually fixate (Figure 1B). Monitor distance was adjusted to the allow receptive fields of the electrodes tested to fit on the monitor screen. On each trial, electrical stimulation was delivered to a single electrode, followed by an audible cue, and then subject response (Figure 1A). Size, shape, and location of the phosphene were reported by the subject drawing the percept on the monitor.

Blind subjects were seated 30.5 cm in front of a touchscreen monitor and instructed to fixate on a tactile point (Figure 1D). At this distance, the monitor encompasses a range of 50° by 45°. This distance was set by the clinical trial because it provided touch screen dimension which fully encompassed the expected range of visual field coverage of the implanted array and was found to be a comfortable distance for subjects to report. Electrical stimulation was delivered to one electrode per trial, after which subjects were instructed to report with their right index finger the location of the center of the perceived phosphene, or to draw the outline of the perceived phosphene in the location where it was perceived. Out of the 60 electrodes implanted, 50 – 59 electrodes were mapped per subject. Bimanual absolute mapping was conducted at 120 Hz for BS2 (03-281), while 60 Hz was used for BS1 (02-659), at the request of the clinical trial sponsor. Two blind subjects performed bimanual absolute mapping; BS1 mapped 25 electrodes in this setup, and BS2 mapped 46 in this setup. The comparison to unimanual fixation only used the electrodes mapped with both fixation techniques.

Relative mapping

Multi-point mapping consisted of sequential stimulation of 3-5 electrodes. Sequential, rather than concurrent, stimulation was employed for two primary reasons. Firstly, the current threshold for perception was relatively large, and there was a safety concern to stimulate more than two electrodes at once. Secondly, with the blind subjects evaluated, stimulation delivered simultaneously on two electrodes often resulted in one phosphene in a location spatially distinct from that produced by either electrode stimulated individually. Electrode sequences stimulated a series of adjacent electrodes on the array. Generally, electrode sequences were selected so that they did not cross a major sulcus, such as the calcarine fissure, and were composed of electrodes that were all in the same cortical area (all in V1 or V2). For instance, in BS2, one sequence consisted of the first through fourth electrodes in the top row of the array (electrodes 1 – 4). Another sequence consisted of the first electrode in row 5, the second electrode in row 6, and the third electrode in row 7. In total, BS conducted 22 sequences, mapping 48 electrodes across 2 sessions, and BS2 conducted 26 sequences, mapping 50 electrodes across 7 sessions,

On each trial, a 100 ms duration pulse train was delivered to each electrode, with a 250 ms gap between each electrode in the sequence (Figure 3A). Stimulation conducted with BS2 was delivered at 120 Hz; stimulation conducted with BS1 used 60 Hz at the request of the trial sponsor. Sequence timing was selected be short enough to reduce eye movements during the trial, but long enough that each phosphene was perceived separately. A tone presented at the end of the stimulation sequence cued subjects to respond. Subjects indicated on a touchscreen monitor the center of each phosphene in the order and location perceived (Figure 3B).

Figure 3.

Improvement in precision of phosphene reporting with relative mapping and types of variations present in individual trials. A. Stimulation sequence timing used for multi-point relative mapping for BS2. B. Illustration of multi-point relative mapping task. Multiple electrodes were stimulated on each trial and then the subjects reported the location of each phosphene perceived, in the order experienced. C. Precision of phosphene reporting with relative mapping with multi-point sequences. Each point represents precision (2DRMS) for phosphenes evoked by stimulation of a single electrode, black error bars indicate mean +/− standard deviation. The first column, labeled “Raw”, indicates data with no trial-to-trial alignment. Subsequent columns reflect precision following removal of translational, rotational, and scaling variations. D. Frequency histogram indicating distribution of magnitude of translational variation for each subject for each trial. Averages for each parameter are indicated with a dashed line. E. Frequency histogram of rotational variation for each trial. F. Frequency histogram of scaling variation.

Trial-to-trial precision

Precision of localized phosphenes was quantified with a two times distance root mean square metric (2DRMS). This metric was calculated within-session, for each electrode mapped.

$$2\text{DRMS}=2\sqrt{{\sigma }_{\text{x}}^2+{\sigma }_{\text{y}}^2}$$

where σ_x is the x component of the standard deviation of the point cloud, and σ_y is the y component.

Data alignment

No alignment was applied to absolute mapping data. Final phosphene locations for each mapped electrode were determined by averaging phosphene locations across all trials for a given electrode.

Linear transforms (translation, rotation, and scaling) were used to align relative mapping trials. Trials for each sequence were first aligned to the center of mass across all trials. The set of phosphene locations from each trial was then rotated around its center of mass until equal to the average angle across all trials. Next, length of each pattern in degrees was determined by summing the length between each node, the average value was determined, and each trial was scaled such that its total length matched the group average. Precision following alignment was evaluated for all trials within a single session, as well as evaluated across sessions for any sequences presented in multiple sessions. The magnitude of each type of observed variation (translation, rotation, scaling) was evaluated within each session and across sessions.

Contribution of each type of trial variation on imprecision

A generalized linear model was fit to evaluate the contribution of each type of observed trial variation (translation, rotation, and scale) on 2DRMS. Models were fit in MATLAB using the native function fitglm(). A linear fit, with a normal distribution and reciprocal link was used. Input values were the average translation, rotation, and scale factor for each sequence during each session, and the output parameter was the average 2DRMS of each mapped phosphene in each sequence. Model fits with interactions were evaluated, with no significant interactions identified between each variation.

Cortical magnification factor

Cortical magnification factor (CMF) was used to further analyze map structure. This was calculated by the ratio of distance on the surface of the brain in mm to the distance in visual space for phosphenes evoked by electrical stimulation on neighboring electrodes:

$$\text{CMF}=\frac{{\text{d}}_{\text{e}1-\text{e}2}}{{\text{d}}_{\text{p}1-\text{p}2}}$$

where d_e1-e2 is the center-to-center distance of electrodes 1 and 2 on the array in mm and d_p1-p2 is the distance in visual space between phosphenes evoked by electrode 1 and 2 in degrees. This was calculated for each trial of multi-point mapping for pairs of phosphenes evoked by neighboring electrodes. Electrode pairs evaluated were restricted to be located on the same gyrus and to lie in V1.

A standard mapping equation was used to estimate the expected relationship between CMF and eccentricity for each subject (see solid lines in Figure 8) [15], [17], [31]. The value for the scaling factor A was determined for each subject. This was done by evaluating a range of scaling factors between 15-45 and using phosphene data from all sequences collected, to determine which scale factor provided the best agreement between expected and actual separation on cortex. Scaling factors were restricted to this range based on MRI evaluations of many normally sighted subjects [17].

Figure 8.

Cortical magnification factor measured for phosphenes evoked by neighboring electrodes within multi-point sequence. Analysis was restricted to electrodes in V1. A. CMF across all multi-point sequence trials for BS1. Each point is CMF calculated for a pair of phosphenes on a single trial. Black lines indicate expected pattern of CMF across eccentricity based on the visual mapping function described in methods. B. CMF across all multi-point sequence trials for BS2.

$${\text{CMF}}_{\text{mdl}}=\frac{\text{A}}{\text{ecc}+3.67}$$

where CMF_mdl is the predicted CMF, A is the area scaling factor, and ecc is eccentricity.

Array placement on a logarithmic map of the cortical sheet

A flat map model of the V1-V3 complex known as the Banded Double-Sech model [39] was created for BS1 and BS2 (Figures 7 and S5). Using modified code from [39], a scale factor based on data from multi-sequence mapped phosphenes was used to adjust the model for each subject. Once the scaling parameter was determined, phosphenes obtained from each multi-electrode sequence tested were projected on to the flat map based on their location in visual space. The electrode array was assumed to be rigid and to lie flat on the cortical surface. The location and rotation of the electrode array on the flat map model was determined for each subject by implementing a cost function to minimize the sum of a weighted distance between the projected cortical location of each phosphene from each sequence and the electrode that evoked it. Projection of phosphene locations to cortical space from visual space was conducted using functions provided by [39]. The cost function applied was:

$$D\left(x,y,\theta \right)={\sum }_1^p{W}_p\ast \sqrt{{\left(x_p-x_e\right)}^2+{\left(y_p-y_e\right)}^2}$$

$$W_p=\frac{n_t}{2{\text{DRMS}}_{\text{p}}}$$

where D is the value to minimize, x and y are cartesian coordinates in brain space, θ is the angle of rotation applied to the grid segment, x_p and y_p are the coordinates of each individual phosphene evoked during multi-point mapping that have been projected into cortical space, x_e and y_e are the coordinates of the electrode which evoked that phosphene, and W_p is a weight parameter determined by the number of trials (n_t) a certain sequence was repeated across all sessions during which that sequence was mapped, divided by the precision of that phosphene as measured by 2DRMS.

Figure 7.

Relative mapping with multi-point sequences captures key attributes of functional organization of early visual cortex. A. BS2 electrode overlay on brain. A set of multi-point sequences were selected that correspond to rows on the electrode array (colored rows). Arrows show progress across different visual field representations. The light gray arrow points towards the calcarine, in an area representing upper visual field of V1. The median gray arrow points towards the superior rows of the array, away from the calcarine in an area of cortex representing lower visual field of V1. The dark gray arrow continues the trajectory away from the calcarine fissure and towards the top row of the array. It approximately indicates progression into the lower visual field representation of V2. B. Location of phosphenes obtained with stimulation of each of the rows indicated in A in the same color. The arrows on the visual field map indicate the progression in the visual field for phosphenes evoked by the corresponding electrode sequences in the V1 upper field representation (light gray), V1 lower field (median gray), and V2 lower field (dark gray).

Fitting was conducted separately for contiguous groups of electrodes that lay on either side of the calcarine. Groups were additionally divided into separate groups per visual area (V1 and V2) on either side of the calcarine fissure. This resulted in three individually placed segments per subject (BS1: V1 upper field, V1 lower field, V2 upper field; BS2: V1 upper field, V1 lower field, V2 lower field). Once the best location for each portion of the electrode array was determined on the flat map model, the model was used to project the electrode coordinates from cortical space to a phosphene prediction in visual space.

Results

Reliability of phosphene reporting in sighted and blind subjects

We first conducted a direct comparison of trial-to-trial precision of reported phosphene location with absolute mapping between sighted and blind participants. Sighted subjects used a visual fixation point (Figure 1B) during electrical stimulation, and the blind subjects used unimanual tactile fixation (Figure 1D). Precision of phosphene location was quantified by 2DRMS and calculated for each electrode mapped. Precision of reported phosphene locations among blind participants were substantially poorer than among sighted participants (Figure 1E–F; 2DRMS mean ± standard deviation, 16±2.9° vs. 1.9±0.93°; t(18) = 18 , p <0.001).

This comparison between the precision of phosphene mapping in sighted and blind subjects, using standard techniques used in the two populations, was made to illustrate the magnitude of the challenge faced in conducting future phosphene mapping in blind VCP recipients. The rest of our report will focus on how to improve the precision of phosphene mapping specifically within the blind population.

Improvement in reliability based on fixation method

Next, we evaluated whether a different fixation method could improve precision of phosphene location among blind participants. In this fixation technique, which we refer to as bimanual fixation, subjects were instructed to use both left and right index fingers on the tactile fixation point during stimulation, and then report phosphene location using their right index finger while maintaining left index finger contact with the fixation point (Figure 2A). The precision of phosphene reporting with bimanual fixation was significantly better than with unimanual fixation (Figure 2B; BS1: 4.7±2.6° vs. 19±4.4°, t(79) = 7, p <0.001; BS2 4.1±2.0° vs. 12±2.4°, t(65) = 7, p <0.001).

Improvements in precision based on mapping method

Having established that bimanual fixation led to improved precision in phosphene reporting, we next assessed whether relative mapping (Figure 3A–B) could further enhance reliability. Provided that a stable spatial pattern or configuration of phosphenes results from each trial of electrical stimulation, relative mapping using multiple electrodes could lead to better estimates of the location of the phosphenes for each electrode in the sequence after small errors in absolute location, size, and angle of the perceived pattern are subtracted out. We found this to be the case in our blind subjects.

Without removing the trial-to-trial errors in overall location, angle, and size of perceived patterns, phosphene reporting precision as measured by 2DRMS for each electrode was similar across absolute (Figure 2B), and multi-point relative (Figure 3C raw) mapping methods (BS1: 4.7±2.6° vs. 3.9±2.0°; BS2: 4.1±2.0° vs. 3.2±1.1°).

Next, we examined the precision of phosphene reporting following removal of the three most prominent types of trial-to-trial variation observed, global changes in translational, rotational, and scaling (Figure 3C). Removal of translational deviations significantly improved the precision of phosphene reporting with multi-point sequences compared to the raw trials (BS1: 1.6±1.3° vs. 3.9±2.0° (p<0.001); BS2: 1.3±0.84° vs. 3.2±1.1° (p<0.001) and represented the largest contribution to imprecision across the three types of errors described. Removing rotational variation further improved precision (smaller 2DRMS) for phosphene location for both BS1 and BS2 for multi-point sequences (BS1: 1.5±0.27°; BS2: 1.1±0.50°). The small improvement over removal of translational shifts was significant for BS2 (p<0.001). Additionally, removing scaling variation from multi-point sequences resulted in a small, but significant improvement in precision for BS2 (0.84±0.41°, p<0.001), but not BS1 (0.99±0.95°, p<0.1).

Quantification of variability in relative mapping

We next quantified the magnitude and full range of the types of trial-to-trial variability observed in multi-point mapping. The mean displacement of the reported location of a phosphene sequence was less than 2° from the average reported location of all trials for a given multi-point sequence (Figure 3D, dashed lines; BS1: 1.5±0.69°; BS2: 1.2±0.34°). Across all trials of all sequences collected, the maximum amount of shift (Figure 3D) of a single sequence’s reported location from the average location was several degrees in magnitude (BS1: 7.1°; BS2: 5.2 °). The mean rotation across all trials and sequences (Figure 3E, dashed lines) was similar for both subjects (BS1: 5.1±6.5°; BS2: 9.1±13°). Although mean perceived and reported rotation of a sequence was low, the maximum rotation of a sequence from its mean orientation observed was much larger for both subjects (BS1: 68°; BS2: 89°). Average changes in the scaling or size of the perceived patterns (Figure 3F, dashed lines) obtained with multi-point stimulation were generally less notable than either translational or rotational variations (BS1: 1.0±0.14; BS2: 1.0±0.15), with similar range of deviations for both BS1 and BS2 (BS1: 0.62 – 1.8; BS2: 0.49 – 1.8).

A generalized linear model was used to formally evaluate the impact of each source of trial variation identified on the precision metric (2DRMS). A linear model with normal distribution and reciprocal linkage was used. In the case of each subject, translation was found to be the most significant (BS1: p = 4.0e-8; BS2: p = 0.0058), followed by rotation (BS1: p = 0.029; BS2: p = 0.035). Scale was not significantly represented in the model for either subject. Interactions were not found to be significant between any of the three parameters.

Examples of trial-to-trial variation present in specific multi-point sequences

Two examples from BS2 are presented to demonstrate the type and range of errors that occur across trials when a phosphene sequence is presented to a blind subject via electrical stimulation of early visual cortex (Figure 4). The first example sequence (Figure 4A–D) was presented eleven times during a single session, where trials were presented intermixed with other multi-point sequences. The subject indicated each phosphene was spatially and temporally distinct, with each individual phosphene clearly visible and each appearing at a similar brightness. Raw trials (Figure 4A) indicate a substantial variability in the absolute location, angular orientation, and scale of the perceived pattern of phosphenes. Translational deviation accounts for a large portion of the trial-to-trial variability, and once removed, a more consistent pattern emerges (Figure 4B). Removing angular variation further reveals a consistent shape (Figure 4C). Finally, after removing trial-to-trial variation in scaling, the spatial configuration observed across trials was robustly repeated and internally consistent (Figure 4D). A second example of trial-trial variation in reporting of patterns, shows a sequence that evokes phosphenes in the pattern of a simple character (Figure 4E–H). As with the first example, removing trial-to-trial errors in translation (Figure 4F), rotation (Figure 4G), and scaling (Figure 4H) reveals that the subject very reliably perceived a consistent spatial pattern of phosphenes.

Figure 4.

Maintenance of spatial configuration of phosphenes within sessions. A. A five-electrode sequence presented to BS2. Location and orientation of raw trials. B. Same trials after removal of translational deviations by aligning trials to the center of mass across all trials. C. Same set of trials following removal of rotational variation. D. Same set of trials after removal of scaling variation. The internal structure of the pattern is robustly maintained and clearly visible once shifts and rotations in space are removed. Array inset indicates electrodes used in this sequence. E-H. A second example from the same subject illustrating results with a multi-electrode pattern in the shape of a simple character. E. Raw trials are shown in this panel. F. Trials with translational variations removed. G. Trials with rotational variation removed. H. Trials with scaling variation removed.

Examples of variability in pattern across sessions

Having documented the range of trial-to-trial variation in phosphene reporting observed within single reporting sessions, we now present two examples illustrating the variability in perceived phosphene patterns across sessions (Figure 5). The first example (Figure 5A–C) shows the phosphene pattern resulting from electrical stimulation on the same exact sequence of electrodes presented during six testing sessions conducted on different days. The full set of trials from all sessions show the center of the phosphene pattern varies in location from a minimum elevation of −0.75° to a maximum of 6.8°, and a minimum azimuth of 3.4° to a maximum of 9.0°, with an average total length of 7.3° (Figure 5A). The range of variation in rotation of the perceived patterns across all trials is +/−5.7° from the average orientation. When trials were aligned within sessions (Figure 5B), trials from different sessions tend to cluster in slightly displaced parts of visual space, with some variation in orientation and scaling of the perceived pattern. Trials aligned across sessions (Figure 5C) reflect a clear and robust spatial pattern, with consistent relative angles and spacing between phosphenes. A second example shows phosphenes resulting from partially overlapping sequences of electrodes from five different sessions with a minimum of three overlapping electrodes (Figure 5D–F). When we examine the full set of trials across all sessions, (Figure 5D), the electrodes common to the tested sequences (38, 39, and 40), evoked phosphenes centered across a wide range of visual space locations (AZ: −0.10° – 5.9°; EL: −18° – −7.2°), and generally separated into distinct spatial regions when aligned within sessions (Figure 5E). Following alignment across sessions using the common electrodes mapped, the spatial configuration of the phosphenes obtained on each trial was again revealed to be consistent across sessions (Figure 5E).

Figure 5.

Maintenance of spatial configuration of phosphenes across multiple sessions. A – C. Stimulation delivered to electrode sequence 56-57-58-59-60 presented during each of the 6 different sessions. A. All raw trials of this sequence collected across all sessions. B. Trials aligned by the session in which they were collected. C. All trials aligned across all sessions. Inset indicates the location of the electrodes on the array. D – F. Electrode sequences encompassing electrodes 35-36-37-38-39-40 with a subset of the electrodes (38-39-40) repeated during five separate sessions. D. All raw trials across all sessions. E. Trials aligned by the session in which they were collected. F. Trials aligned across all sessions using the electrodes that were common to all sequences (38-39-40).

When presenting the same sequence of electrodes (38-39-40) as part of a larger sequence, the subject reported a consistent configuration among these electrodes but incongruous spatial relationships among the remaining portions of the sequence (Figure 5F). each series resulted in the same configuration of the final three phosphenes, despite having different starting points. The electrodes that were not in common across all sessions, however, had more variability in their reported location. Sequences presented in session 1 (S1) and session 2 (S2), despite presenting phosphenes evoked from overlapping electrodes, result in different reported configurations when one electrode was omitted from the series, which can be seen in the fanning-out of sequence outside of the phosphenes evoked by electrodes 38, 39, and 40.

Quantification of pattern variability across sessions

The above examples show that spatial configurations of phosphenes obtained with multi-point relative mapping remain stable within and across sessions. We next compared the precision of phosphene mapping and magnitude of trial variation within and across sessions (Figure 6). In general, precision of reported phosphene location measured across multiple sessions was lower for absolute mapping than multi-point relative mapping (Figure 6B, first and third datasets presented, BS2: 6.5±1.2° vs. 4.7±1.1). When examining all multi-point trials across sessions without alignment, as compared to only examining trials within single sessions, there was a higher magnitude of translational, rotational, and scaling errors (Figure 6, BS2: F–H) and lower precision for trials (Figure 6B, middle two columns, BS2: 4.7±1.1° vs. 3.6±1.3°). However, once trials were aligned, the precision of phosphene locations is similar for trials aligned within a single session compared to trials aligned across multiple sessions (Figure 6B, right two columns, BS2: 0.80±0.32° vs. 0.91±0.25°). This provides strong evidence that perceived patterns evoked by multi-electrode stimulation are maintained both within and across sessions. Similar results were found for BS1, but in that case we could not make a within vs. across session comparison for absolute mapping data (Figure 6A, C – E).

Figure 6.

Precision of phosphene mapping within and across sessions and dependence on trial-to-trial variation in translation, rotation, and scaling. A. Precision within and across sessions for multi-point mapping for BS1. Left two segments show precision prior to any alignment (across vs. within session, 6.5±3.3° vs. 3.6±1.3°). The right two segments indicate precision after removal of trial-to-trial translational, rotational, and scaling variations (across vs. within session, 0.49±0.27° vs. 0.61±0.34°). B. Precision within and across session for absolute and multi-point mapping for BS2. The two segments on the left indicate precision for reported phosphene locations determine through absolute mapping (across vs. within session, 6.5±1.2° vs. 3.6±1.4°). The middle two segments show precision for reported phosphene locations determined with multi-point relative mapping (across vs. within session, 4.7±1.1° vs. 3.0±0.94°). The last two segments indicate precision for multi-point mapped phosphenes after removal of translational, rotational, and scaling variation (across vs. within session, 0.80±0.32° vs. 0.91±0.25°). C – E. Frequency histograms showing the magnitude of each type of alignment deviation for multi-point sequences across (light gray) and within (dark gray) sessions for BS1. Dashed lines indicate means for each distribution. C. Translational variation (across vs. within session, mean 3.0° vs. 1.3°). D. Rotational variation (across vs. within session, mean 6.9° vs. 3.4°). E. Scale variation (across vs. within session, std 0.19 vs. 0.14). F – H. Frequency histograms showing the magnitude of each type of alignment variation across and within sessions for BS2. F. Translation variation (across vs. within session, mean 1.9° vs. 1.2°). G. Rotation variation (across vs. within session, mean 19° vs 15°). H. Scale variations for BS2 (across vs. within session, std 0.19 vs. 0.16).

Structure captured by multi-point sequence mapping

Multi-point relative mapping with sampling of specific rows on the electrode array captured some key expected features of functional organization of early visual cortex, based on work in sighted subjects [4,33,34]. This is illustrated with a set of sequences sampled in subject BS2 (Figure 7). The two electrode rows sampled that lie below the calcarine fissure on the brain (red and orange) produce phosphenes in the upper visual field as expected, with posterior electrodes in both rows evoking more foveal percepts and more anterior electrodes producing phosphenes in increasing eccentricity. The rows of electrodes just superior to the calcarine (yellow-orange and light green) similarly produced phosphenes that lie along iso-angle lines in visual space in the lower visual field, with more anterior electrodes producing more eccentric phosphenes. Movement from the row closest to the calcarine fissure (yellow-orange) to the next row further superior (light green) results in phosphenes found closer to the vertical meridian (VM) as is expected for superior movement within area V1 above the calcarine fissure. As rows are examined that lie further superior to the calcarine (dark green, light blue and dark blue), moving into area V2, the progression of the phosphenes reversed in visual space, with progressive movement away from the VM.

Although structured sampling using rows on the electrode array provided some useful information, there was additional complexity that was revealed when we examined the full set of sequences from each subject (Figure S1). Spatial relationships for a given set of electrodes determined by sampling with one sequence may conflict with those determined by sampling with other sequences, despite being internally consistent within and across sessions. Furthermore, there was a tendency for subjects to regularize the reported space between phosphenes obtained with multi-electrode stimulation. For example, when presenting a sequence of electrodes that produce phosphenes at increasing eccentricities, such as electrodes 56 – 60 in BS2, the subject tended to report equal spacing between each perceived phosphene, rather than reporting more space between the more eccentric phosphenes. This pattern of regular spacing was consistently observed across sequences at all eccentricities evaluated (BS1: 4.3 – 31°; BS2: 2.4 – 26°). The trend towards regularization of distance between phosphenes was found for sequences whether they generated straight lines of phosphenes in visual space or produced curved trajectories.

To quantify the observation that subjects tended to report regularized distance between phosphenes, cortical magnification factor (CMF) was calculated for neighboring electrodes from multi-electrode sequences, on each trial. The CMF values observed were consistent across different eccentricities (Figure 8, data points) and different from those predicted by a standard mapping in sighted subjects (Figure 8, solid lines). This regularization of reported phosphene spacing was additionally demonstrated by presenting sequences with the same end points on a row of electrodes, but varying the intermediary electrodes presented (Figure 9). For example, if stimulation was delivered sequentially to electrodes 56, 57, and 58 (sequences 1 and 2, Figure 9A–B ) the distance between phosphenes evoked by electrodes 56 and 58 (3.6°) was larger than if stimulation was delivered sequentially to only electrodes 56 and 58 (sequence 4, Figure 9D) (1.8°). When individual electrodes along this row were dropped from the sequence, the subject nevertheless reported a consistently spaced set of phosphenes (Figures 9E–F).

Figure 9.

Examples indicating regularization of reported phosphene locations when using multi-point sequences. A. Phosphene locations evoked on all trials of sequence 1(56, 57, 58, 59, and 60). B – D. Three patterns using the same row of electrodes tested in A, but with different electrodes dropped form the sequence. B. Phosphene locations evoked on all trials of sequence 2 (56, 57, 58, and 60). C. Phosphenes evoked by all trials of sequence 3 (56, 57, 59, and 60). D. Phosphenes evoked by all trials of sequence 4 (56, 58, 59, and 60). E. Average pattern of phosphenes evoked for each sequence tested. Patterns have been vertically displaced to allow easier comparison. Scale bar indicates 1°. F. Separation in visual space between phosphenes associated with particular electrode pairs as they appear in different sequences. The colored circles represent the pair of phosphenes for which the distance measurements were taken. For example, the first set of bars indicates the separation in visual space for phosphenes evoked by electrodes 56 and 58. This pair was measured in sequences 1, 2, and 4. In the case of patterns 1 and 2, there was an intermediary electrode (57) presented between 56 and 58, the distance in both cases was 3.8°, whereas when stimulation was delivered to 56 and 58 consecutively, the distance between the resulting phosphenes was 1.6°.

Limited structure apparent in absolute maps

Maps of visual space based on absolute mapping were constructed by averaging the location of individually mapped phosphenes. For both BS1 (Figure 10B) and BS2 (Figure 10E), maps constructed in this manner do not reflect the highly structured representation of visual space expected for early visual cortex. The only clear feature evident in these maps was that electrodes above the calcarine generally produced phosphenes below the horizon and vice versa.

Figure 10.

Comparison of overall map of visual space obtained by absolute mapping and visual space map fit to relative mapping with multi-point data. A. Array placement on the medial wall for BS1. Electrode 1 is indicated in the superior - anterior position, and electrode 60 in the inferior posterior direction. B. BS1 phosphene map generated by absolute mapping. C. BS1 visual filed map made by fitting a model of the V1-V3 complex to phosphene locations determined through relative multi-point mapping. D. Array placement for BS2. E. BS2 absolute phosphene map. F. BS2 visual filed map generated by fitting a model of the V1-V3 complex to multi-point sequence data.

Fitting a logarithmic map of visual space

As described above, structured sampling of the electrode array using multi-electrode sequences was partially effective at revealing the structure of visual field maps in early visual cortex, but could not account for all complexity. We hypothesized a better estimate of the overall map in each subject could be obtained by fitting a model of the V1-V3 complex to the data from multi-electrode sequences (Figure S2). The Banded Double-Sech model [39] fit to the data assumes a logarithmic mapping of each visual area, and was adjusted for each subject by a scaling factor (BS1: 20.4; BS2: 24.5). The placement of the array structure on the flat map of cortex was optimized by minimizing the average weighted displacement between the cortical projection of phosphenes and the associated electrodes within the array structure (BS1: V1_UF = 2.8 mm, V1_LF = 1.4 mm, V2_UF = 2.9 mm; BS2: V1_UF = 0.40 mm, V1_LF = 1.9 mm, V2_LF = 1.5 mm). The visual field maps based on the V1-V3 model fit to multipoint data (Figure 10C and F) had clear internal structure reflective of the organization found with multi-point sequences, but additionally provided logarithmic spacing between phosphenes that was not well captured by multi-point sequences.

Discussion

Sighted vs blind subjects

We evaluated the reliability of phosphene reporting in both sighted and blind subjects. We found absolute mapping yields consistent, reliable results in sighted participants, but significantly more variability with blind participants. Blind subjects had over 8 times the variability in reporting phosphene location compared to their sighted counterparts. Possible reasons for this include differences in testing setup, changes to functional organization of visual cortex in blind subjects, and differences in the ability of blind subjects to compensate for shifts in body, head, or eye position.

There were notable differences in the exact framework used for phosphene mapping and in the set of parameters used for electrical stimulation in the two groups tested. This was due to both the timeframe in which each set of experiments occurred, and limitations of the stimulation system used to test in the blind subjects. The small differences in electrical stimulation trains, however, were unlikely to impact these experiments. Differences in stimulation frequency, pulse width, were likely to create changes in the exact current required for perception of a phosphene and could potentially impact the size or brightness of the perceived phosphenes. In all testing, current amplitudes were adjusted so that each subject could clearly perceive and locate the phosphene. We have no reason to believe small changes in stimulation parameters would have impacted the subjects’ ability to precisely locate phosphenes. A much larger difference between the two experiments was the way in which subjects fixated. Sighted subjects used visual fixation whereas blind subjects used tactile fixation. This was a necessary change, and very likely to impact performance.

The disparity in precision between blind and sighted subjects may partly be explained by blind participants being more impacted by positional errors associated with absolute mapping. Positional errors – which include small shifts between the subject and the reporting monitor, gaze angle, fixation strategy, and pointing response – have been described as a central weakness of absolute mapping [40]. Sighted subjects had minimal scatter between subsequent trials for a given phosphene, indicating whatever positional errors they faced were easily overcome, likely compensated for with subtle shifts in body position or gaze angle. Positional errors conversely seem to compound in blind subjects, resulting in shifts of several degree across reported location for a single phosphene.

Blind subjects may be more susceptible to positional errors for several reasons. Blind participants cannot use the same visual cues to align themselves in front of the monitor and fixation point or to compensate for the subtle shifts in body and head position that occur during testing. There are also likely unaccounted errors from eye position, which is an issue of considerable importance to both the process of mapping and the continued development of cortical prostheses [41]. Blind individuals typically have more difficulty maintaining a steady gaze angle or may have a nystagmus that causes unpredictable shifts in eye position [42], [43], making consistent fixation a challenge and contributing to positional jitter. For these reasons, regularizing the setup and introducing elements to help the subject self-center were imperative.

Positional errors also affect the reporting phase of the task. Reporting phosphene location by pointing to it on a monitor relies heavily on proprioception. Visual feedback normally updates this internal representation of body position and plays a central role in planning trajectory and kinematics of reaching movements [44]. Without visual feedback, positional drift occurs [45]. Blind subjects lack this visual feedback and cannot correct placement by visually aligning their pointing finger with the phosphene location. With this in mind, we focused on implementing a fixation strategy that could provide an alternative means to update proprioception by reinforcing the location of the fixation point with the reporting hand.

Improvements to fixation strategy or reference frame

Previous literature, with sighted participants and simulated phosphenes, indicated that improving tactile feedback increased the reliability and accuracy of phosphene localization [46]. By instructing the strategic use of both hands and employing a tactile board on which the subjects were to respond reduced error by a factor of two to three. Stronks and Dagnelie [40] employed a similar fixation strategy in their simulated studies, having subjects place both index fingers side-by-side. Here we are able to build on this earlier work by directly comparing two forms of tactile fixation in blind subjects with phosphenes evoked by electrical stimulation of cortex.

Bimanual fixation improved precision by 2 to 3 times compared to unimanual fixation. This supports our hypothesis that a providing additional proprioceptive feedback can indeed improve reporting precision for blind subjects. In the context of mapping, making consistent physical connection between the hand used for fixation and the hand used for reporting phosphene location provides a simple way to update proprioception [44]. Further improvements to the framework for reporting phosphenes are possible and should be the subject of future investigations. It may be advantageous, for example, to use tactile markers to establish vertical and horizontal axes or to incorporate a tactile grid [40].

Improvements to mapping strategy

Although the bimanual tactile fixation strategy did improve the reliability of phosphene mapping in both blind subjects tested, it did not increase precision to the level achieved by sighted subjects. Relative mapping utilizing stimulation of multiple electrodes on each trial allows the assessment of the location of one phosphene relative to another, rather than relating each individual phosphene to a central, body-external, tactile fixation point. Without alignment across trials, subjects contend with the same difficulties in localizing phosphenes experienced during absolute mapping, such as an inability to make subtle corrections to their gaze angle or body positions. However, once variations in absolute placement are removed, precision was significantly improved. The precision for a single phosphene mapped with this relative approach in blind subjects was similar to the same range of values as sighted persons performing absolute mapping.

Trial-to-trial variability

Most variability across trials was explained by three main components: translation, rotation, and scale. These observed trial variants may have different origins and likely have different impacts on VCP functionality that vary by task. Translation, or shifts in the absolute location of the pattern, was the largest component of trial variability. This type of variability would mostly impact localization tasks in daily activities, especially when holding a steady gaze or camera angle. Small changes in gaze direction or body position at the start of each trial are likely to explain a large portion of the observed variation, and it may be adequately addressed in future devices with integrated eye tracking. Perceived rotation of a phosphene pattern was the next largest trial variation observed. Rotational deviations were typically minimal but, on occasion, presented as a pattern nearly orthogonal to its typical orientation. In a low context environment, this could mean certain simple shapes or characters could be easily confused. The reason for pattern rotation is unclear, but we theorize it is related to subjects’ ability to form a stable framework for phosphene reporting and that improving the reporting framework during testing or adding more visual context to a presented scene in free-viewing may reduce the likelihood of large rotational variations. Changes in scale were the smallest contributor to imprecision, and presumably will have less of an impact on operational use than translational or rotational deviations. Scaling variations may relate to the distance or plane at which phosphenes are perceived. Importantly, when any of these variations occured, there was no internal distortion to the phosphene pattern. The whole form was rotated, shifted, or scaled, and the internal structure of the pattern remained intact.

Maintenance of pattern

Despite trial-to-trial variability in the exact location, orientation, or scale, the internal spatial relationships among phosphenes in a sequence were robustly maintained across multiple sessions. Previous experiments have implied that spatial patterns of phosphenes were maintained and could be used to make simple discriminations [21], [34], [47], but had not quantitatively examined this observation in detail. Here we present strong evidence that simple spatial patterns are consistently perceived over time (Figure 6A–C), and that the perceived spatial configuration of phosphenes evoked by a sequence of electrodes is similar whether presented as its own sequence or as part of a larger pattern (Figure 6D–F).

Robust maintenance of phosphene patterns is important for multiple reasons. First, it demonstrates that structured mapping is likely retained in early visual cortex several years after late onset of blindness. Second, it is the key feature that makes multi-electrode sequences useful in obtaining precise locations of phosphenes associated with each electrode. Because spatial relationships are robustly maintained, relative mapping is useful in parsing fine spatial details between phosphenes. Third, it has important implications for the ability of future VCP users to perform specific daily tasks. Stable phosphene patterns suggests VCP users will be able to reliably use patterned sequences of phosphenes to recognize simple shapes or forms. In support of this premise, previous work showed dynamic stimulation of multi-electrode groups could reliably convey several simple characters to blind subjects [21].

Impact of mapping strategy on the measured structure of the map of visual space

Maps of visual space determined by absolute mapping did not yield a structured representation of visual space. The only clear feature captured by absolute mapping was the distribution of phosphenes evoked by electrodes on either side of the calcarine fissure; electrodes above the calcarine generally, but not always, produced phosphenes the subjects reported below the horizon and vice versa. The imprecision of the method was not adequately offset by conducting many trials for each electrode. In this way, absolute mapping was found to be both comparatively uninformative and inefficient at mapping visual space.

Relative mapping with multi-point sequences approached the task with structured sampling under an assumption that visual cortex has a highly organized structure. This method was able to capture key features similar to the known functional organization of early visual cortex as measured in sighted subjects [14], [48], [17]. Beyond the general structure of phosphenes below the calcarine fissure producing percepts in the upper visual field and vice versa, use of multi-point sequences yielded reports of sets of phosphenes in visual space that lay in increasing eccentricities along iso-angle lines, and which exhibited progression towards the VM with movement in the superior direction in area V1. Additionally, sampling with rows of the electrode array located further superior to the calcarine, presumed in area V2, then resulted in sets of phosphenes which progressed away from the VM as expected. However, structured sampling along rows had other added complexity and some clear limitations.

Spatial relationships for a given set of electrodes determined by sampling with one sequence may conflict with those determined in by sampling with other sequences, despite being internally consistent within and across sessions. This may be related to the subjects’ tendency to report regularized space between phosphenes that were presented sequentially from stimulation on nearby electrodes. The reasons for these observations are unclear but are unlikely to suggest the lack of a single robust map of early visual cortex. While motor errors in the reporting phase of the tasks may contribute to the regularized intervals between phosphenes, conversations with BS2 indicated he also perceived the phosphenes to occur at regularly spaced intervals. We theorize the perceptual contribution to this effect may result from either blind subjects implementing a different framework for performing visual tasks in low context environments, or the small irregularities in the spatial shifts between phosphenes could be masked by the regular temporal interval of their presentation. It will be important for future research to parse out the source of this regularization to appropriately mitigate the effects.

We fit a model of the V1-V3 complex [39] to multi-point sequence data (Figure S1) to devise maps of visual space that represented the structure found in these data while also incorporating expected changes in CMF with eccentricity. Fitting a logarithmically spaced map to the scale and location of multi-point sequences is similar to an atlas-based approach. Standard retinotopic atlases use average anatomical landmarks or functional organization to fit the logarithmic structure of visual cortex to the conformal topology of a subject’s visual cortex, and have been shown to predict cortical retinotopy of out-group sighted subjects with high accuracy [49]–[51]. An atlas approach may provide useful insights or a useful basis for predicting phosphene location in blind subjects. However, the available atlases have not yet been validated with blind subjects and unpredictable changes to visual cortex may occur after loss of sight [22]–[26], that may result in a need to alter the scaling or landmarks used in a standard atlas to better accommodate these variations. Because these atlases were developed on a sighted population and we are naive to the ways plastic changes may have impacted visual cortex physiology after full vision loss, it is important to validate the atlases on a blind population. For these reasons, our current implementation used a model that assumes basic logarithmic mapping within each visual area but does not assume the location or size of each visual area in relation to topological landmarks as is done when using standard atlases based on fMRI data. Moving forward it would be advantageous to test phosphene maps extracted from our hybrid approach and an atlas-based approach, as well as conduct a comprehensive validation of retinotopic atlases on a blind subject group.

Determining veridical structure of visual field maps in blind subject

In addition to being reliable, phosphene maps should ideally be an accurate representation of the visual space subtended by the electrode array. We are presently unable to claim which map is the most accurate representation of each subject’s visual space maps in early visual cortex. Earlier work in the field functionally validated their phosphene maps by generating simple visual patterns from the map [34]. Similarly, we used the information derived from multi-point sequences to plan stimulation patterns in the shape of letters [21]. In this experiment, stimulation was delivered sequentially to several electrodes to dynamically trace a pattern through visual space. With no prior training, our subject, BS2, correctly identified four different letter shapes (“W”, “N”, “M”, “U”) at 93% accuracy. Drawings produce by this subject during this task show an alignment between the letter endpoints and the mapped phosphine locations. This provides some confirmation that the multi-point method can provide accurate enough information to plan and deliver simple, useful visual patterns to blind users of a VCP. Moving forward, validity of different maps can be evaluated by planning simple character or shape sequences based on differently generated maps and compare the perceptual experience reported by the subject.

Conclusions

Having a reliable and efficient way to obtain phosphene maps in individual blind subjects will likely be important for the successful implementation of a new generation of VCPs. Our results demonstrate that obtaining accurate phosphene maps in blind subjects is fraught with challenges, and results may heavily depend on the exact techniques that are employed. Using the described methods to reinforce proprioceptive feedback and focusing on mapping techniques that prioritize relative spatial relationships can improve the confidence that the phosphene mapping data collected is reflective of the underlying spatial maps in the visual cortex. Finally, standardized maps still provide utility and can be fit to experimental data to provide a highly structured map, reflective of functional organization while retaining nuanced details associated with each subject that may otherwise be lost. Ultimately, we recommend a hybrid approach, fitting structured maps to the experimentally obtained location and scale of underlying cortex.

• Blind participants have difficulty reliably localizing phosphenes evoked by electrical stimulation of early visual cortex

• Bimanual fixation improves precision of reported phosphene location

• Relative mapping with multi-electrode sequences improves precision of reported phosphene location

• The spatial configurations of phosphenes observed during electrical stimulation of multi-electrode sequences is stable across trials

• Fitting a map model of the V1 – V3 complex to multi-point sequence data can be used to make an overall estimate of the visual field map of early visual cortex in blind subjects