Technical considerations for generating somatosensation via cortical stimulation in a closed-loop sensory/motor brain-computer interface system in humans

Abstract

Somatosensory feedback is the next step in brain computer interface (BCI). Here, we compare three cortical stimulating array modalities for generating somatosensory percepts in BCI. We compared human subjects with either a 64-channel “mini”-electrocorticography grid (mECoG; 1.2-mm diameter exposed contacts with 3-mm spacing, N = 1) over the hand area of primary somatosensory cortex (S1), or a standard grid (sECoG; 1.5-mm diameter exposed contacts with 1-cm spacing, N = 1), to generate artificial somatosensation through direct electrical cortical stimulation. Finally, we reference data in the literature from a patient implanted with microelectrode arrays (MEA) placed in the S1 hand area. We compare stimulation results to assess coverage and specificity of the artificial percepts in the hand. Using the mECoG array, hand mapping revealed coverage of 41.7% of the hand area versus 100% for the sECoG array, and 18.8% for the MEA. On average, stimulation of a single electrode corresponded to sensation reported in 4.42 boxes (range 1–11 boxes) for the mECoG array, 19.11 boxes (range 4–48 boxes) for the sECoG grid, and 2.3 boxes (range 1–5 boxes) for the MEA. Sensation in any box, on average, corresponded to stimulation from 2.65 electrodes (range 1–5 electrodes) for the mECoG grid, 3.58 electrodes for the sECoG grid (range 2–4 electrodes), and 11.22 electrodes (range 2–17 electrodes) for the MEA. Based on these findings, we conclude that mECoG grids provide an excellent balance between spatial cortical coverage of the hand area of S1 and high-density resolution.

1. Introduction

As the branch of neuro-restoration called “brain-computer interface” (BCI, also called a brain-machine interface or BMI) continue to improve, with impressive motor control [1,3,9,12,14,19], the absence of tactile feedback during dexterous manipulations has become increasingly apparent. Somatosensation is an integral component of behavior, as evidenced by studies in which sensory impairment has led to degraded grasp or poorer movement performance relative to healthy counterparts [8,16,24]. Similarly, restoring sensation in nonhuman primate (NHP) motor BCI systems has demonstrated that sensory feedback improves both motor signals and motor control [26].

NHP studies have successfully induced artificial somatosensation by delivering charge-balanced biphasic pulses of electric current through microelectrode arrays (MEA) embedded in the primary somatosensory cortex (S1). With direct electrical stimulation to S1 in a vibrational “flutter” discrimination task, NHPs learned to use cortico-electrical or physical stimuli nearly interchangeably, with comparable accuracies [22,23]. Artificial sensation based on the stimulation of S1 in trained monkeys has also been demonstrated in a true closed-loop BCI. Operating a virtual effector, NHPs successfully discriminated between targets using “textural” clues from cortical stimulation [12,17]. In a separate human BCI experiment, a Blackrock MEA array was placed over the hand area of S1 and showed reliable, safe stimulation. The area of the hand covered was localized to the ventral surface, just below and partially on the proximal phalanges [7]. A similar study with an MEA implanted over the arm area in a tetraplegic patient with some motor and sensory ability still present in that arm, also showed safe generation of somatosensation, including both cutaneous and proprioceptive responses [3].

These research studies have largely been performed using MEAs meant to deliver small amounts of electric charge through each electrode. However, just as neural activity may be recorded at a wide range of spatiotemporal scales by different electrode designs, electrical stimulation may also be delivered over a wide range of parameters via electrodes of different scales. This study evaluates three electrode styles, using an example of each to highlight the differences. Each has precedent for use in a BCI, regarding their somatotopical coverage and specificity for stimulation of the hand area of S1: standard electrocorticography grids (sECoG), “mini”-ECoG grids (mECoG), and MEAs. For recording, EcoG-style electrodes (sECoG and mECoG) lie on the surface of the brain, without penetrating the cortex, capturing population-scale neural activity (see Fig. 1), whereas microelectrode arrays penetrate the cortical surface to record the activity of individual neurons [15]. Because of these different purposes, the two styles of electrodes have very different physical designs. The MEAs used in this study were two 6 × 10 grid patterned arrays of 1.5-mm length iridium oxide coated tips, at 0.4-mm pitch. Conversely, an 8 × 8 sECoG grid has 4.75 mm diameter disk electrodes with 1.5-mm exposed surface, a 1-cm center-to-center pitch. A mECoG is laid out similarly to the sECoG, but proportionally smaller: 3-mm pitch with 2-mm diameter contacts, 1.2-mm exposed surface (see Fig. 1). Both sECoG and mECoG stimulation produces unnatural sensations [14] whereas MEA stimulation produces natural or quasi-natural sensations [3,7]. We suggest that mECoG provides an attractive spatial scale for stimulating somatosensory cortex for applications that do not require natural percepts.

Fig. 1.

Grid comparison. A. Standard 8 × 8 electrocorticography (sECoG) grid (left) next to a “mini” ECoG (mECoG) grid (right). Both the contacts and the spacing are smaller in the mECoG grid. B. To-scale comparison of grid size overlaid on a 3-dimensional reconstruction of the brain from a magnetic resonance image. The primary motor cortex is outlined by the dashed blue line, and the primary somatosensory cortex is outlined by the dashed orange line. To scale drawings of the microelectrode array (MEA; red), the mECoG grid (purple), and the standard-ECoG (yellow) with representative electrode sizes and spacing placed in the corner (the electrodes in the MEA are too small to see). They are centered on the primary somatosensory hand area, across from the primary motor hand area. The MEA covers a small portion of the area compared to the mECoG and sECoG. MEA = Microelectrode array; mECoG = Mini-electrocorticography grid; sECoG = standard-electrocorticography grid. MEA results were derived from Flesher et al. [7].

2. Methods

2.1. Patient selection, implants, and recordings

2.1.1. Standard and mini-electrocorticography grids

Two patients with epilepsy who underwent implantation of ECoG for seizure localization were enrolled in a pilot study for generating artificial somatosensation. Consent was obtained for this study, which was approved by the institutional review board. These patients underwent a standard craniotomy to access the frontotemporoparietal regions for placement of grid and strip electrodes. The somatosensory cortex, including the hand area, was accessible from this craniotomy exposure. Subject P1, a 55-yearold male, and subject P2, a 30-year-old male, both underwent this procedure. P1 had an 8 × 8 mECoG grid (2-mm contacts with 1.2mm diameter exposed surface, embedded in silastic sheeting, spaced 3-mm from center-to-center; FG64C-MP03, Ad-Tech Medical Instrument Corporation, Wisconsin, USA) placed onto the lefthemisphere hand area of S1, with neuronavigational guidance. The mECoG is FDA-approved for recording and stimulation in humans. Subject P2 had an 8 × 8 sECoG grid (4.75-mm contacts with 1.5-mm exposed surface, in silastic sheeting, spaced 1-cm center-to-center, AU8X8P4, Integra Life Sciences Corporation, New Jersey, USA) placed over the S1 hand area on the right hemisphere, again using neuronavigation guidance. The dura was closed over sECoG and mECoG, with sutures anchoring the exiting wires to prevent unwanted movement. The bone was replaced, and the scalp and skin were closed in a standard fashion. The leads were tunneled out of the scalp and sutured in place on the scalp to prevent migration.

The patients were placed in the epilepsy monitoring unit for seizure activity. ECoG recordings from the mECoG and the sECoG were extracted for analysis. The data were acquired at 2000 samples/second with a reference electrode placed on the scalp. Recordings were made using an Xltek NeuroWorks data acquisition system (Natus Medical Incorporated, Wisconsin, USA) with an Xltek EEG32U amplifier. The Grass Technologies S12X Cortical Stimulator (Natus Neurology Incorporated, Warwick, RI) was connected to an EEG machine. During the clinical stimulation mapping sessions, the contacts of the mECoG and sECoG were stimulated, and subjective assessments of sensation were recorded. The epileptologist utilized stimulation parameters commonly used for ECoG mapping of eloquent cortex and for seizure localization [2,21,25,28]. Stimulation was applied between adjacent pairs of electrodes with alternating current, frequency of 50 Hz, pulsewidth 300 ls, duration of 1 s, and amplitude ranging from 1 mA to 10 mA. Amplitude was increased sequentially until sensation, involuntary movement, or nothing, occurred. The location on the hand and the verbatim descriptions of sensation were recorded. The bipolar pairs were explored sequentially throughout the grid. In areas with robust sensations, stimulation was repeated to assess the stability of sensation.

2.1.2. Coverage and specificity

To capture the utility of the different modalities, we used the concepts of “coverage” and “specificity”. Coverage was defined as the dermatomal areas on the hand that had percepts felt by the subjects during cortical stimulation. The mapping of somatosensation from electrical stimulation was based on the subjects’ descriptions. Subjects pointed to the area on their body and verbally described, with anatomic detail, where they felt the percept. Percent coverage was then estimated based upon this description, with partitioning of the hand and fingers into anatomic areas (16 divisions for the palm, eight for digit one, and six each for the other digits) to replicate the divisions set out by previous human sensation mapping [7]. If sensation occurred anywhere in an area, the entire area was included. Specificity was broken into two mirrored concepts, “redundancy”, the number of electrodes that stimulated the same box, and “resolution”, the number of boxes stimulated by each electrode. For redundancy, we calculated the number of these dermatomal divisions (“boxes”) stimulated by each electrode, and for resolution, the number of electrodes that stimulated each box. These metrics were employed to capture the utility of electrodes for creating distinct and separate percepts across different stimulations.

For MEA patterns of stimulation in the hand area, data were taken from Flesher et al., 2016, describing artificial sensation with electrical microstimulation through a MEA [7]. In that report, a 38-year-old male tetraplegic patient (M1) was implanted with two 6×10-electrode MEAs (32 functional electrode tips each, coated with sputtered iridium oxide film, 1.5-mm shanks, 2.0-mm × 4.0-mm total area) over the left-hemisphere, S1 hand area. Stimulation occurred over 6 months and used one-second, biphasic asymmetrical pulse trains at 100 Hz and 60–100 μA current amplitude. The subject reported sensations by verbally indicating which areas of the hand had percepts based on reporting the dermatomal divisions [7].

3. Results

3.1. Hand coverage and specificity

For subject P1, 26/64 (40.6%) mECoG electrodes produced somatosensory percepts in the hand, covering 41.7% of the hand area (Fig. 2). Sensation was limited to the fingers, with no coverage in the palm. Stimulation through mECoG electrodes corresponded to sensation, on average, in 4.42/48 (9.2%) boxes (range 1–11 boxes; “resolution”), whereas each box was stimulated by an average of 2.65/26 (10.2%) mECoG electrodes (range 1–5 electrodes; “redundancy”) as summarized in Fig. 2. The sECoG grid covered 100% of the hand area with 14/48 (29.2%) electrodes, and another five electrodes produced percepts in the face and tongue. The average number of boxes affected by stimulation on a single electrode was 19.11/48 (39.8%; range 4–48 boxes) and, on average, 3.58/14 (25.6%) electrodes (range 2–4 electrodes) produced percepts in the same dermatomal box. The reported coverage for the implanted MEA by Flesher et al. was 18.8% with 44/64 (68.8%) electrodes exhibiting distinct areas of sensation (2 additional electrodes exhibited complex sensation involving the whole hand) [7]. Sensation was elicited in an average of 2.3/48 (4.8%) boxes per electrode (range 1–5 boxes) and each box was stimulated by an average of 11.22/44 (25.5%) electrodes (range 2–17) as illustrated in Fig. 3.

Fig. 2.

Coverage, redundancy, and resolution of stimulation by electrode array type. A. The surface area of the hand included in stimulation by electrode array type and the percentage of electrodes that resulted in somatosensory stimulation by electrode array type. The surface area covered was based on dermatomal divisions (boxes); if stimulation occurred anywhere within the box, it was included. The sECoG covered 100% of the hand, but only 29.2% of the electrodes resulted in hand sensation; whereas the MEA had 68.8% of electrodes resulting in somatosensation, but only covered 18.9% of the hand. The mECoG array was the most balanced, covering 41.7% of the hand and using 40.6% of the electrodes. B. A comparison of the average boxes stimulated per electrode (resolution) and electrodes per box (redundancy) between the electrode array types. The mECoG array again showed the best balance of resolution where each electrode stimulated a contained area (as opposed to the large somatosensory area covered by each electrode in the sECoG) and had relatively low redundancy, where the electrodes were not all stimulating the same area (as in the MEA). MEA = Microelectrode array; mECoG = Mini-electrocorticography grid; sECoG = standard-electrocorticography grid. MEA results were derived from Flesher et al. [7].

Fig. 3.

Representations of the stimulation across the grid types, the surface area, and location of the sensations. A. Each grid type exhibiting where on the grid, and how much of the grid, was involved in somatosensory percepts. B. The areas covered by the sensory percepts. The MEA was concentrated in the upper palm area, whereas the mECoG showed excellent representation in the fingers. The sECoG covered the whole hand broadly. C. The central location of each sensation from stimulation. Each dot represents an electrode stimulation, highlighting the concentration of electrodes in the same area for the MEA (redundancy) and the large spread of the electrodes stimulation percepts (resolution), of the sECoG.

3.2. Safety and reliability

Electrical stimulation of the mECoG and sECoG resulted in reliable percepts of sensation. Electrodes with somatosensation were tested twice, with replication of the qualitative feeling associated with the stimulation and the location on the hand. Described sensations included “tingling” or “electricity” in subject P1 with mECoG, and “sharpness”, “tingling”, or “heaviness” in subject P2 with the sECoG. No adverse events occurred, and no cranial sensations were noted. For subject P1, with the mECoG, one area was chosen for multiple stimulations, over 100 times, without adverse events, pain, or alteration in the percept. The dermatomal location of the sensation was stable throughout the experimental session for both mECoG and sECoG electrodes.

4. Discussion

We compared three electrode types—mECoG, sECoG, and MEA— by evaluating the somatotopical coverage and specificity of electrical stimulation in the hand area of somatosensory cortex. Data for the two ECoG-style electrode types were collected from patients undergoing surgical treatment for epilepsy, while data for the MEA were taken from the literature, referring to a study of a bidirectional BMI with a tetraplegic subject [7]. To facilitate comparison, methods for evaluating coverage and specificity of the mECoG and sECoG grids were adapted to those described in Flesher et al. [7].

4.1. Matching the scales of the electrode, stimulus, and cortex

We posit that the electrode size and geometry of the mECoG electrodes are well matched to the structure of the underlying cortical networks, so the density of electric charge in the cortical tissue during stimulation is sufficient to activate local populations of neurons and produce relatively focused somatosensory percepts. Overall, the sECoG grids provided significant coverage of the entire hand, but with low resolution, and the MEA provided less coverage, over about 19% of the hand, with relatively high redundancy. The mECoG grid provided more balanced coverage over around 41% of the hand. Moreover, the resolution, the percentage of effective electrodes causing percepts in any given dermatomal box was equivalent (approximately 25%) for both the MEA and sECoG, but with much less redundancy for the mECoG electrodes, exhibiting 10% of effective electrodes stimulating the same box. With strong perceptual coverage over the hand and reasonable redundancy in the somatotopical mapping, these results suggest that the mECoG grids are an appealing candidate for the stimulating element of a bidirectional BCI that does not require natural percepts.

Sutherling et al. found that the anterior-posterior length of cortical representation was 7-mm (+/−0.9-mm) and 5.7-mm (+/−1.2-mm) for the thumb and index fingers respectively, with a total hand representation of 2 cm [4,27], whereas Flesher et al., estimated the S1 hand area to be 4 cm based on magnetoencephalography [7]. These sizes suggest why the MEA might be inadequate in terms of absolute coverage, but excellent for resolution: with a total planar area of 8 mm² for the MEA, the study reported coverage of just 18.8% of the surface area of the hand [7]. However, with the dense coverage within that area, many electrodes produced somatosensory percepts in overlapping areas, resulting in high redundancy. At the other extreme, sECoG electrodes covered the entire hand area with just a few electrodes, each of which led to percepts over large swaths of the hand (low resolution). However, the 64-contact mECoG electrodes array provided comparable resolution to the MEA, with low redundancy, while covering nearly half of the total hand somatotopy.

4.2. Electrode placement, stability, and other considerations

This study found that the mECoG electrodes covered the fingers almost entirely, but did not include the palm and some portions of the finger. Patients involved in these experiments underwent no form of pre-planning to evaluate hand sensory areas, presumably, a permanent implant for a bidirectional BCI system would include extensive preoperative planning for optimal placement. An ideal workflow would include preoperative, task-specific fMRI or magnetic encephalography imaging, vessel imaging (to ensure the planned location is not limited by vasculature), neuronavigation, and an awake surgery for optimal array placement.

Conceptually, stimulation-based somatosensory BCIs operate under different principles than motor BCIs. Whereas motor control-oriented systems require neural signals if they are to differentiate movements, i.e. “degrees of freedom”, somatosensory stimulation must address neurons already allocated for sensation over given areas, i.e., “degrees of perception”, since interpreting the signal is performed by the brain, not the computer. Altered topographic mapping of somatosensation in those with amputations [18], or in the blind [20], suggests that plasticity is prevalent in somatosensation, and is likely to allow for improved representation. Recent work with cortical stimulation and a prosthetic limb exhibited this sort of phenomenon, showing ownership of prosthesis with timed cortical stimulation and the visual information of touch [5].

The MEA, mECoG, and sECoG are the most widely adopted, FDAapproved options for human recording and stimulation. However, other types of recording modalities are in use, or may be in the future. First, a larger sized mECoG (same electrode density) would combine the coverage of sECoG with the non-invasive implantation and high-resolution of the mECoG. The micro-ECoG, with scales closer to MEA, showing an electrode diameter of around 100-μm and pitch of around 4-mm, have shown recording parameters similar to MEA [11]. Presumably, the stimulation profile would be similar to that of MEA as well, and have the advantage of minimally invading the cortex. Other novel approaches are also in the works including optogenetic prostheses [13], high-electrode count picocurrent arrays with inter-electrode spacing of 30-μm [10], and larger-scale high-density micro-ECoG arrays, which have all been successfully used in animal models [6]. Finally, strategies of combining modalities may be necessary. A MEA in the thumb and index finger areas, with an mECoG for the rest of the hand area, or even combined with a specifically designed m- or sECoG spanning the rest of S1 may prove well-suited for a more complete restoration of somatosensation.

This work is limited by sample size and experimentallyuncontrolled clinical environment for collecting data. Participants described in this study suffer from epilepsy or tetraplegia, which may alter the signals, sensations, and mapping otherwise achievable. Current spread is altered significantly based on the size and shape of the different types of electrodes and thus limits the generalizability of the study. However, we based our analysis on perception of the stimulus, rather than focus on the specific stimulation parameters in order to concentrate on the practical utility of each modality. Additionally, the metric of dermatomal boxes used to quantify coverage, where any stimulation within the box results in the whole box being included, could degrade some of the finer details of the sensation and might overestimate the percentage of hand area covered. However, this work serves as a starting point for exploring how to engineer artificial sensation for use in a closed-loop motor/sensory BCI. mECoG electrodes might be useful for a closed-loop BCI system, but measurements of long-term stability and reliability are still necessary. Future work will require more patients to better elucidate the somatotopic coverage and specificity available with mECoG electrodes, while the effect of stimulation parameters, such as frequency, pulse width, and amplitude, will also need to be explored in each of the different electrode modalities. This is necessary if we are to identify any limitations in the range of percepts evoked through stimulation.

5. Conclusion

Restoring somatosensation to those with a functional loss is the next step in BCI evolution. Providing pain sensations to reduce pressure ulcers, stretch sensors to improve bladder function, and integrated motor/sensory closed-loop BCIs to produce dexterous movements are realistic early goals. However, methods for implementation are not well explored, differ from motor BCI systems, and require careful consideration going forward to maximize progress. This exploration of different modalities suggests that the mECoG exhibits potential for the successful delivery of somatosensation into the brain. To provide somatosensory percepts through cortical stimulation to the entire hand, the coverage needed would be too large for an MEA, and the spacing of a sECoG would not provide fine enough detail. A mECoG might provide the optimal balance.