A modular high-density μECoG system on macaque vlPFC for auditory cognitive decoding

Abstract

Objective.

A fundamental goal of the auditory system is to parse the auditory environment into distinct perceptual representations. Auditory perception is mediated by the ventral auditory pathway, which includes the ventrolateral prefrontal cortex (vlPFC). Because large-scale recordings of auditory signals are quite rare, the spatiotemporal resolution of the neuronal code that underlies vlPFC’s contribution to auditory perception has not been fully elucidated. Therefore, we developed a modular, chronic, high-resolution, multi-electrode array system with long-term viability in order to identify the information that could be decoded from μECoG vlPFC signals.

Approach.

We molded three separate μECoG arrays into one and implanted this system in a non-human primate. A custom 3D-printed titanium chamber was mounted on the left hemisphere. The molded 294-contact μECoG array was implanted subdurally over the vlPFC. μECoG activity was recorded while the monkey participated in a ‘hearing-in-noise’ task in which they reported hearing a ‘target’ vocalization from a background ‘chorus’ of vocalizations. We titrated task difficulty by varying the sound level of the target vocalization, relative to the chorus (target-to-chorus ratio, TCr).

Main results.

We decoded the TCr and the monkey’s behavioral choices from the μECoG signal. We analyzed decoding accuracy as a function of number of electrodes, spatial resolution, and time from implantation. Over a one-year period, we found significant decoding with individual electrodes that increased significantly as we decoded simultaneously more electrodes. Further, we found that the decoding for behavioral choice was better than the decoding of TCr. Finally, because the decoding accuracy of individual electrodes varied on a day-by-day basis, electrode arrays with high channel counts ensure robust decoding in the long term.

Significance.

Our results demonstrate the utility of high-resolution and high-channel-count, chronic μECoG recording. We developed a surface electrode array that can be scaled to cover larger cortical areas without increasing the chamber footprint.

1. Introduction

Real-world hearing is a complex computational problem for two main reasons: (1) auditory stimuli can simultaneously change along many dimensions, such as loudness and location, and (2) stimuli of interest (e.g. the voice of a friend) are often mixed together with other environmental sounds (e.g. the sound of frothing milk) [1, 2]. Thus, for a listener to interact efficiently with their environment, they must be able to selectively attend to stimuli of interest and importance (e.g. your friend’s voice), while simultaneously ignoring extraneous stimuli (i.e. the frothing milk) [2–7].

A classic test of real-world hearing is the detection of an auditory stimulus that is embedded in a noisy background [8–12]. For example, normal-hearing listeners can readily detect and hear a friend’s voice at a party, even when the background is noisy due to the voices of other speakers, music, and clinking glasses. This is often referred to as the ‘cocktail-party problem’ or ‘hearing in noise’ and is one of the fundamental challenges of the auditory system [13].

Although the neuronal correlates of hearing in noise have been studied acutely in a variety of model species and using many different techniques and stimulus paradigms [8, 9, 14–24], these prior works focused on early cortical centers and neglected the contribution of later regions, like the ventrolateral prefrontal cortex (vlPFC). The vlPFC is part of the of the ventral auditory pathway, which is widely considered to play a substantial role in auditory perception and cognition [25].

In addition to not understanding vlPFC’s contribution to fundamental tasks like hearing-in-noise, we only have a rudimentary understanding of how information is encoded in populations of vlPFC neurons [21, 24, 26–42]. Because of this, we do not have a good understanding of the spatiotemporal resolution of the neuronal code that underlies vlPFC’s contribution to perception. The topographic and temporal scale of these neuronal populations has not been tested because, until recently, it was not possible to record from large numbers of brain sites simultaneously [43]. Even today, such large-scale recordings in the cortical auditory system are still very rare.

In this project, we developed a microelectrocorticographic (μECoG) recording array in order to identify the task-related information that can be decoded from μECoG vlPFC signals. To overcome some of the challenges of recording from neuronal populations and to do so chronically [44–47], we demonstrate the feasibility of a μECoG array that provides, to our knowledge, the highest electrode density and channel count in any non-human primate study (table 1). The advantage of this technique is that the array is assembled out of smaller sub-arrays that are combined into one larger one. Because of this technique, it is possible to tailor the size, shape, and electrode density for the particular brain target. This design also permits the recording chamber to house electronics that scale vertically as channel count increases, giving more room for electrode-interface connections. A water-tight recording chamber design enables electronics to be permanently implanted on the animal, to allow for future channel-count scaling. We show that μECoG signals from this array contain multiple sources of information that relate to a monkey’s performance on a hearing-in-noise task. We further show how decoding improved as we increased the number of simultaneously decoded channels and how the array remained stable after 1 year of implantation.

Table 1.

μECoG comparison table. Comparison between this work and other μECoG studies.

Studies	Year	Subject	Number of Electrodes	Spacing (mm)	Coverage (mm2)	Density (Sites/mm2)
Hollenberge et al [48]	2006	Rat	64	0.75	36	1.78
Benison et al [49]	2007	Rat	256	0.5	64	4.00
Molina-Luna et al [50]	2007	Rat	72	0.64	27	2.67
Kim et al [51]	2007	NHP	56	1	36	1.56
Rubehn et al [52]	2009	NHP	252	2	2100	0.12
Ledochowitsch et al [53]	2011	Rat	256	0.5	64	4.00
Besle et al [54]	2011	Human	127	10	12 700	0.01
Viventi et al [55]	2011	Cat	360	0.5	90	3.60
Pasley et al [56]	2012	Human	64	4	1000	0.06
Khodagholy et al [57]	2015	Rat	64	0.1	1	64.00
Escabi et al [58]	2014	Rat	196	0.25	12.25	16.00
Hotson et al [59]	2016	Human	128	3	1152	0.11
Kellis et al [60]	2016	Human	16	1	16	1.00
Khodagholy et al [61]	2016	Human	240	0.23	840	0.29
Kaiju et al [62]	2017	NHP	96	0.7	4704	0.02
This work	2020	NHP	294	0.61	114.4	2.57

2. Materials and methods

2.1. Electrode

Our modular 294-channel μECoG electrode array was built from three separate 98-channel arrays. These arrays were fabricated using microfabrication methods in a cleanroom environment at the Shared Materials Instrumentation Facility at Duke University. The full array was composed of 294 electrode contacts (229-μm diameter) with an inter-electrode pitch of 610 μm (center-to-center).

The electrode contact size was chosen to match our previously published results [47, 63, 64]. The electrode pitch was designed to be as dense as possible, while fully covering vIPFC with the number of available channels. A local reference electrode (38.1-mm long × 200-μm wide) surrounded the contacts (figure 1(a)). This yielded a total sensing area of 10.4 × 11 mm² with a density ~2.6 sites/mm². The sensing and reference electrode contacts were Cr/Au (20/250 nm).

Figure 1.

Electrode assembly and fabrication. (a) Molded electrode array with a high-resolution sensing area (10.4 × 11 mm²) including 294 electrode channels. The red arrow indicates the location of the local reference ring electrode. (b) A 3D cross sectional schematic of the electrode array. The electrodes were fabricated on top of a 25-μm Kapton Polyimide sheet. 20 nm of chromium and 250 nm of gold were E-Beam deposited to form the electrode contacts and traces. 6 μm of polyimide (PI 2611) was spin coated onto the device to serve as the top encapsulation layer. The electrode contacts and connector openings were etched open using RIE. (c) Illustrations of the electrode molding process. 1. PVA glue was applied to a glass slide to secure PVA tape. Electrode arrays were then laminated onto the PVA tape in order to temporarily hold the electrode sub-arrays for alignment. The PVA tape also prevented silicone from accidently covering up the electrode contacts during the molding process. 2. A stencil was placed onto the assembly to check the electrode alignment and to set the thickness of the silicone molding. 3. Well-mixed and de-gassed silicone was poured onto the mold. 4. A wiper removed the excessive silicone. 5. The silicone cured at room temperature for 24 h. 6. The assembly was rinsed under tap water to remove the PVA tape and release the molded array.

2.1.1. Fabrication.

The electrode stack-up is shown in figure 1(b). Similar to our previous work [65], the base electrode substrate consisted of a 25 μm Kapton^® Polyimide sheet (Fralock, Inc, Valencia, CA), which was manually laminated onto a glass slide coated with cured polydimethylsiloxane (PDMS) (Sylgard 184, Dow Corning, Midland, MI). Using an e-beam metal evaporator (CHA Industries E-Beam), 20 nm of chrome (Cr) and 250 nm of gold (Au) were deposited onto the Kapton sheet. S1813 positive photoresist (Shipley Microposit) was used as a positive mask to wet etch the Au and Cr layers (Gold Etch TFA, Cr Etchant 9057; Transene, Danvers, MA). Next, a 6-μm thick layer of polyimide (PI 2611, HD Microsystems, Parlin, NJ) was spun onto the surface of the array and cured. The top polyimide layer was selectively etched using a Trion Phantom II reactive ion etcher (RIE). The electrodes were then removed from the glass substrate. Finally, a stiffener, which was composed of a 50-μm thick layer of pressure sensitive adhesive (3M467-PSA) and 153-μm thick layer of polyimide, was applied to the back of each 51-pin zero-insertion-force (ZIF) connector, two for each 98-channel array. We measured the impedance (@ 1 kHz) of each electrode in saline solution using a NanoZ system (White Matter LLC, Mercer Island, WA).

2.1.2. Assembly (molding)

We used biocompatible medical-grade silicone molding (MDX4–4210, Dow Corning, Midland, MI), which is similar to the silicone that we used previously as artificial dura [66, 67]. Using this silicone, we combined three separate electrode sub-arrays together to construct a single uniform high-density electrode array. This design resulted in a narrow cable entry point into the recording chamber (figure 1(a)). Because the silicone also covered the sharp electrode edge of the thin polyimide, it reduced the possibility of brain damage during array placement.

The molding process is shown in figure 1(c). We manually aligned the electrodes on a glass slide with water-soluble (Polyvinyl alcohol, PVA) tape (3M 5414). The tape prevented the electrode contacts from being covered by silicone and maintained alignment during molding. We designed and fabricated custom molding and alignment stencils to ensure that the silicone-molded 294-channel μECoG array had a uniform shape and thickness. Once the silicone cured, the molded electrode array was released by rinsing it under water. With only a 100-μm thick coating of silicone molding, the molded array remained flexible. The functionality of the array was bend tested to a radius of 2.5 mm using a glass rod (supplement figure 1 (available online at stacks.iop.org/JNE/17/046008/mmedia)).

2.2. Recording chamber

Traditionally, recording chambers have been obtained from commercial vendors. Although these commercial chambers have many benefits, they also have substantial disadvantages. For example, because these commercial chambers have a universal design, they do not fit tightly on the skull—even if manually adjusted during surgery—and consequently, do not fully integrate with the bone [68, 69]. When a chamber fails to integrate with the skull bone, it can come loose or break off, ending the experiment and endangering the animal [70]. Further, because these manual adjustments take a substantial amount of time during surgery, it increases the length of the procedure, which increases risk to the animal.

More recently, custom chambers, which fit tightly on each animal’s skull, are created with 3D-printing or computer-numerical-control-machining techniques [70–72]. However, because these chambers are typically designed for penetrating microdrives rather than surface electrodes, they are not watertight. A watertight chamber was essential for this study because we permanently attached the adapter PCBs to the chamber.

To address these challenges, we developed a new type of recording chamber that met our design criteria. This chamber had three major components: (1) a 3D-printed custom titanium chamber whose base curvature sat seamlessly on the monkey’s skull, (2) a molded silicone rubber chamber wall (Sugru, FormFormForm Ltd., London, United Kingdom), and (3) a molded silicone gasket. The wall and the gasket worked together to form a watertight chamber.

2.2.1. Identification of vlPFC.

vlPFC was initially identified through structural MRI scans of the monkey’s brain [73–76]. vlPFC is dorsal of the inferior ramus of the arcuate sulcus. vlPFC neurons are functionally identified by their task-related auditory responses [41, 77, 78]. The left hemisphere was targeted to facilitate comparisons with previous findings [7, 41, 79–85].

2.2.2. Skull mapping.

To create a custom base that fit the contour of the monkey’s skull, we first extracted the portion of the skull MRI data that was located under and near the potential chamber location. We then performed a re-mesh step (MeshLab, Visual Computing Lab ISTI-CNR) that reduced the number of triangles in the 3D skull model from ~1 million to ~3000. This step balanced the costs associated with long processing times with the detail needed to produce a realistic model of skull curvature. Finally, we exported this simplified skull surface model to Autodesk Inventor (Autodesk Inc. San Rafael, CA) to complete the chamber design.

2.2.3. Design and fabrication.

With the simplified skull model, we were able to quickly iterate designs to optimize the structure and mounting strategy (figure 2). We 3D printed low-cost plastic test samples, which allowed our surgeon to optimize the locations of the bone screws in advance. These low-cost test samples were made out of Polylactic Acid (PLA) via the common fused filament fabrication (FFF) method. The test samples also allowed test fitting and fine tweaks of the design before we submitted the design for titanium printing. We used finite element analysis software simulation tools (Autodesk Nastran, Autodesk Inc, San Rafael, CA) to optimize our design parameters (e.g. shape and wall thickness), which improved the chamber’s impact resistance.

Figure 2.

General procedure for creating the custom chamber base. 1. Structural MRI data determined the target location and the corresponding skull location in which to mount the chamber. 2. The target skull surface was extracted and simplified to reduce the curvature details and processing time. 3. The chamber was then iterated to optimize the design and mounting strategy. 4. The final chamber design was re-imported back into the targeting software environment to validate placement.

To simulate a worst-case scenario, instead of calculating the average impact force, we calculated the dynamic energy of a free-falling monkey at the moment when the chamber hits the ground. From the work-energy principle, the maximum impact force is: ${\text{F}}_{\text{max}}=2\frac{\text{mgh}}{\text{d}}$, in which m is the mass of the falling object in kilograms, g is the gravitational acceleration constant (9.8 m s⁻²), h is the falling distance in meters, and d is the impact distance in meters.

The impact force on a monkey’s chamber can be difficult to determine because it depends on how the chamber hits the ground, (e.g. which part of it hits the ground, the angle of impact, and if it was somehow protected). We chose an impact distance of 6 cm, along with a 1.2-m falling distance, and a monkey weight of 11.0 kg. This yields an impact force of ~4300 N. Our simulation found that the maximum yield strength was ~700 MPa (figure 3), when the impact force occurred on the top of the chamber, which is below the yield strength limit of ~900 MPa for titanium. Additional simulation results for impacts on the front, side, and corner of the chamber are shown in supplement figure 2.

Figure 3.

Mechanical simulation of free-fall impact force. We optimized the chamber structure to withstand the impact force of a free-falling NHP. The most vulnerable point of the chamber was the safety stopper tab (red dash square) which prevented the cap from being over-tightened. When an impact force was applied from the top surface (yellow arrow, ~4300 N), a maximum pressure of 718.4 MPa (maximum point identified by the simulation software) occurred in the tabs, which is below the yield strength limit of ~900 MPa for titanium. The simulation results from impact forces in different directions are shown in supplement figure 2. The impact force distribution in the chamber body over time is shown in movie M1.

We made our chamber watertight by constructing: (1) a custom-molded silicone rubber wall with a reinforcement stiffener and (2) a silicone-molded gasket between the titanium chamber base and cap. The chamber stack-up is shown in figure 4(b).

Figure 4.

Chamber system assembly steps. (a) Left. Molding steps for the silicone rubber wall. (i) Demolding agent was applied to the aluminum mold. (ii) The mold was assembled, and the stiffener piece was placed at the center of the mold. (iii) The mold was filled with silicone rubber (Sugru) and excess material was removed using a squeegee. (iv) After curing the mold in an oven (60 °C, 4 + hours), the molded wall was removed from the aluminum mold and the alignment portion of the stiffener was broken off. Right. A 3D rendering of the aluminum mold that was used to mold the silicone rubber wall and the silicone gasket. The molding steps of the silicone gasket were similar to the silicone rubber wall but without the stiffener. Silicone (MDX4–4210) was used for the gasket. (b) Exploded 3D view of the chamber system. (c) The final chamber on top of a model skull. Detailed assembly steps are shown in movie M2.

The silicone rubber wall compressed and deformed around the electrode to form a watertight seal. A 3D-printed stiffener (DuraForm PA, 3D System) reinforced the silicone rubber wall and helped it to retain its shape under pressure. The reinforcement stiffener was printed using selective laser sintering. This printing method increased the surface roughness of the stiffener, which, in turn, improved the adhesion of the stiffener to the silicon rubber.

To seal the gap between the chamber base and cap, we designed a custom silicone (DuPont MDX4–4210) gasket. The gasket was compressed by the chamber cap to seal the chamber. We could replace the gasket if there was any sign of wear or damage.

To make the silicone gasket, we first applied a thin layer of a medical-grade biocompatible mold-release agent (Duraglide MCC-DGF14A, MicroCare Corp., New Britain, CT) to the surface of the stencil mold. This facilitated removal of the molded gasket after it cured. We next placed well-mixed silicone in a desiccator for 30 min to remove the air bubbles trapped during mixing and then poured it into the mold; we heated the mold to accelerate the curing process (60 °C for 6 + hours). The silicone rubber wall was constructed in an analogous manner, using the same aluminum stencil mold. Detailed molding steps and the corresponding stencil molds for both parts are shown in figure 4(a).

We tested the water resistance of the chamber by placing a water-contact indicator strip (3M 5559) within a fully assembled chamber and soaking it in phosphate buffered saline (PBS; pH = 7.4; Quality Biological, MD) at room temperature. For visual confirmation, we also 3D printed (Transparent Resin [3D Hubs, Amsterdam, Netherlands] via the Polyjet method) a clear chamber (supplement figure 3). We found that, for both the titanium and clear chambers, the indicator strip remained dry after several days of soaking.

After validation, the final design (resolution: ± 100 μm) was 3D-printed in titanium (TiAl₆V₄) via direct metal laser sintering (3D Hubs). A picture of the final chamber is shown in figure 4(c).

2.3. Data acquisition system and in vivo recording

Typically, chamber size increases as the number of electrodes implanted increases. In this work, although having hundreds of electrodes, we minimized the chamber footprint by exploiting a modular stacking design that used off-the-shelf components. The final chamber size was 3.5 × 2.7 × 1.8 cm³, including mounting wings and weighed only 76 g including the PCBs, cap, gasket, screws, and all other hardware.

Within the chamber, we developed two custom PCB interface boards that connected the electrode array to the data-acquisition system. Four 51-pin ZIF (504070 series, Molex, IL) connectors connected the level-1 interface board with two of the 98-channel arrays. Two additional ZIF connectors connected the level-2 interface board with the third 98-channel array. The level-1 board was connected to the level-2 board through two 100-pin high-density stacking connectors (55909 series, Molex, IL), which passed 196 electrode signals along with multiple reference signals from the level-1 board to the level-2 board. The top of the level-2 board included eight Nanostrip connectors (NSD-36-VV-GS, Omnetics Connectors Crop.) that interfaced with four 64-channel amplifier headstages (RHD 2164, Intan Technologies).

With a maximum of 256 channels recorded from the four 64-channel amplifiers, we shorted the remaining outer contacts (supplement figure 4) on the adapter boards to create another local reference ring that could be used in data analysis. Future designs could incorporate the amplifier integrated circuits into each level PCB, enabling all 294 recording electrodes.

Digital data from the Intan amplifiers were transferred via thin and flexible SPI tethers, and logged with the OpenEphys system [86] at 20 kS s⁻¹. The programmable hardware filter settings in the Intan headstages were set to 0.1 Hz–7.5 kHz. The system diagram is shown in figure 5.

Figure 5.

μECoG recording system. (a) A 3D rendering of the full system during recording. (b) System block diagram. Two of the electrode sub-arrays were connected to interface board level-1, and the third sub-array was connected to the interface board level-2. The two interface boards were connected via two 100-pin stacking connectors that passed 196 electrode channels and the local references. Eight Nanostrip connectors were used to connect to the Intan headstages during recording. Side views of interface board levels-1 & − 2 are also shown at the bottom of the panel. An OpenEphys controller recorded from the four Intan headstages.

2.4. Chamber implantation

Prior to implantation, the electrodes were connected to the interface PCBs, and the level-1 and level-2 PCBs were connected together. After this assembly step, the PCBs were conformally coated (Bondic, Laser Bonding Technology Inc, Ontario, Canada) to protect the circuits from potential water ingress and improve reliability. The electrode and PCB assemblies were gas sterilized (ethylene oxide; Duke University Medical Center).

The University of Pennsylvania Institutional Animal Care and Use Committee approved all of the surgical and experimental protocols. All surgical procedures were conducted using aseptic surgical techniques, during which the monkey was under general anesthesia.

In brief and following from our previous work [70], the scalp and the periosteum were incised along the midline. After stereotactically identifying the vlPFC (figure 6(a)) [75], we used a piezoelectric drill (Synthes) with a small round cutting burr to perform a craniectomy. Next, after the dura was incised and reflected, the μECoG array (which was attached to the titanium chamber) was placed on the brain surface (figure 6(b)). The dura was then overlain and the calvarium was replaced. The recording chamber was attached to the skull via Ti bone screws; the screws were inserted through holes in the legs of chamber. The legs and edges of the craniectomy were protected with Geristore (DenMat). Finally, the cover was placed and secured on the recording chamber, and the skin edges were relaxed and sutured over the chamber.

Figure 6.

Targeted location of vlPFC and image of the implanted μECoG array. (a) Schematic of a rhesus macaque brain. Based on MRI imaging, the grey square shows the region of the prefrontal cortex that we targeted as the vlPFC [77, 78, 82, 83]. D: dorsal; P: posterior. (b) Image of the implanted μECoG array at the time of surgical implantation. The chamber with the electronics is shown at the top of the picture. At the bottom of the image, the craniectomy and the electrode array that is lying on the surface of the vlPFC is visible. This image was obtained prior to overlaying the dura on the array and brain surface.

2.5. Behavioral task

The hearing-in-noise task tested a monkey’s ability to detect a target vocalization that was embedded in a background chorus of vocalizations. 400–700 ms after the monkey grasped a touch-sensitive lever, we presented a target vocalization that was embedded in a background chorus. Following onset of the target vocalization, the monkey had 600 ms to move the lever to report hearing the vocalization.

The target vocalization was a single exemplar of a coo (duration: 400 ms) that was recorded from an unknown conspecific [87, 88]. The background chorus was created by superimposing 30–40 different vocalizations, which were also from unknown con-specifics, at random times to create a 6600-ms stimulus; on a day-by-day basis, we created new tokens of this background chorus. Because we minimized the amplitude troughs of this stimulus mixture, we reduced the possibility that the monkey could detect the target vocalization if it occurred within an amplitude trough of the background chorus [21, 89]. The sound level of the target vocalization was varied relative to the level of the background chorus (65 dB SPL): the target-to-chorus ratio (TCr) was nominally between −5 and + 15 dB.

The onset of the target vocalization was drawn from an exponential distribution [90] (min: 1200 ms; mean: 4200 ms; max: 6100 ms). This strategy encouraged the monkey not to anticipate target onset [90]. On some trials, however, the vocalization chorus terminated without a target vocalization. These were catch trials.

The hearing-in-noise task was a detection task. A hit occurred when the target vocalization was presented and the monkey released the lever within 600 ms of target onset (i.e. the response-time window). A miss occurred when the vocalization was presented but the monkey did not release the lever within the response-time window. A false alarm occurred when the monkey released the lever when the target vocalization was not presented. A correct rejection occurred when the monkey held onto the lever throughout a catch trial. Monkeys were rewarded on hit and correct-rejection trials.

2.6. Data analysis

To determine whether vlPFC activity contained information that correlated with different TCr values (−5, 0, 5, 10, and 15 dB) and the monkey’s choices, we conducted two classification analyses on the μECoG signal. In one classifier, we tested the degree to which we could decode different TCr values. In this analysis, we used μECoG signals generated from hit trials only of the hearing-in-noise task to ensure that we did not conflate behavioral choice with TCr. In the second classifier, we tested the degree to which we could decode the monkeys’ behavioral choices (hits, misses, false alarms, and, in a subset of sessions, correct rejections), independent of TCr. We conducted these classification analyses on the μECoG broadband signal, which is the 0.1 Hz–7.5 kHz signal that was recorded on the OpenEphys system via the Intan headstages.

For each classification analysis [24, 91–94], we constructed an N-dimensional population response vector that constituted the μECoG signals from a population of N different electrodes to R repetitions of S conditions (i.e. each TCr value or behavioral choice). We used a Support Vector Machine (SVM) to classify these data, and each classification underwent a k-fold cross-validation procedure. This procedure divided the training set of μECoG data into k smaller subset (i.e. folds) and, in an iterative fashion, one subset was tested by a model trained on the remaining k-1 subsets.

Because different numbers of trials might have occurred for different conditions (e.g. each TCr value), we subsampled our data to ensure that we had the same number of trials for each condition [24, 95]. For each electrode, we calculated the mean μECoG amplitude and its variance that was generated over the entire duration of the target vocalization or the equivalent period prior to lever release for false alarm trials. To control for potential bias due to electrode-by-electrode differences in the μECoG signals, we z-scored the μECoG signal, relative to a 750-ms baseline period preceding chorus onset.

To test how well the μECoG signals discriminated between the different TCr values or the monkey’s choices, we implemented a linear-readout procedure. Because both the TCr and the choice classifications had more than two conditions (e.g. we had 5 different TCr values), we implemented a ‘one-versus-all’ classification. In this method, we built a classifier for each TCr value and trained each of them to discriminate between one particular TCr value versus all of the remaining TCr values. Using the test data, we identified the classifier that maximized performance (i.e. percent correctly decoded) and report average performance of this ‘one-versus-all’ classification over 10 iterations with randomly picked channels for each decoding group (number of channels). An analogous procedure was conducted to analyze choice. All classifiers were constructed using a SVM procedure that was implemented in the MATLAB programming environment and used the LibSVM library [96] with a coarse-grained search that optimized the LibSVM parameters (e.g. cost function) for each classifier.

Because single-electrode connectivity varied on a day-by-day basis, we removed electrodes with high impedances (i.e. > 1 MΩ at 1 kHz). Those electrodes with impedances < 1 MΩ were used to construct the N ≥ 1 dimensional population response vector; we simultaneously examined 1, 2, 4, 8, 16, 32, 64–127, and ≥ 128 channels. Because these decoding analyses were independently conducted on a session-by-session basis, differences in the number of electrodes per session did not affect the outcome of our decoding analyses. We plot each decoding session independently to maximize transparency regarding potential session-by-session differences.

3. Results

The μECoG electrode array was implanted in June 2017. The recordings for this study began in November 2017 and continued for ~6 months. As of early 2019, the electrode was still implanted and was still viable.

Monkey A (N = 9 behavioral and recording sessions reported in this study) reliably reported the presence of the target stimulus (i.e. a vocalization) that was presented simultaneously in a background ‘chorus’ of vocalizations (figure 7, top). Because we focused on TCr values above threshold, the monkey’s performance was essentially constant across values of TCr. The monkey’s mean hit rate at TCr = − 5 was 74% (s.e.m. = 6%), and at TCr = 15, it was 72% (s.e.m. = 7%). Their false-alarm rate was also fairly constant across TCr values: 17% (s.e.m. = 7%) at TCr = − 5 and 20% (s.e.m. = 7%) at TCr = 15.

Figure 7.

Psychophysical performance on the hearing-in-noise task. (Top) Average hit rates and false-alarm rates averaged across all recording sessions as a function of TCr. The data are replotted as a function of d’ in the middle panel. The bottom panel plots the average response time across all recording sessions as a function of TCr. Response time is the time from stimulus onset until lever movement.

The middle panel of figure 7 replots the monkey’s performance as a function of d’. d’ is a non-parametric measure of the monkey’s sensitivity in detecting the target stimulus by incorporating both the hit rate and the false-alarm rate [97]. A d’ value of 0 implies that the monkey could not discriminate between the target and the background chorus. At the lowest TCr value of −5, the mean d’ value was 2.4 (s.e.m. = 0.7). At the highest TCr value of 15, the mean d’ value was 1.7 (s.e.m. = 0.4).

The mean response time (i.e. the time between stimulus onset and the onset of lever movement; figure 7, bottom) was also fairly consistent across different values of TCr: 209 ms (s.e.m. = 23) at TCr = − 5 and 231 ms (s.e.m. = 22) at TCr = 15. Taken together, these behavioral values suggest that the monkey could report the presence of the target vocalization in the background chorus of vocalizations.

Figure 8 shows the evoked response potential (ERP) that was averaged across all of the μECoG electrode channels (with impedances < 1 MΩ at 1 kHz) and across all sessions. On hit trials (top row), we found that the target vocalization modulated the average ERP response. That is, we saw a deflection in the ERP following onset of the target vocalization (which occurred at time = 0). In contrast, on miss trials (second row), even though the target vocalization was presented, the average ERP response was flat. That is, the target vocalization did not modulate, on average, the μECoG signal. The average ERP was also modulated by false-alarm trials (third row), whereas the average ERP on correct-rejection trials (fourth row) was flat.

Figure 8.

Average auditory evoked-response potential (ERP). The normalized (z-scored) average auditory ERP across all recording channels and across all sessions is plotted as a function of behavioral choice (rows) and TCr (columns). The average ERP is plotted in red. The standard deviation is the grey shading. FA; false alarm. CR; correct rejection. Time is plotted relative to stimulus onset (t = 0). The dotted horizontal line indicates voltage = 0.

Because these data suggest that different behavioral choices (hits, misses, false alarms, and correct rejections) modulated the average ERP signal, in the follow sections, we use a classification analysis to quantify the degree to which we could decode behavioral choice as well as TCr values from the μECoG signals.

As noted in Materials and Methods, there was tremendous day-to-day variability in the decoding of individual electrodes, which we attribute to poor connections at the Nanostrip connectors that were plugged and unplugged each day (see Discussion for more details). This variability can be seen in figure 9, which depicts the single-channel decoding accuracy of each μECoG electrode after 5 months of implantation and after 11 months of implantation. As can be seen, this variability was not stationary: the decoding pattern from 5-months post-implantation was quite different than that seen at 11 months. Nonetheless, for a large proportion of these single electrodes, single-electrode decoding accuracy was above chance (i.e. 25% [4 behavioral choices: hit, miss, false alarm, and correct rejection]).

Figure 9.

Individual channel decoding accuracy. (a) The locations and spatial configuration of the electrodes used in the decoding analyses are highlighted by the red square. (b) Single-channel decoding (classification) accuracy of the μECoG array after (left) 5 months and (right) 11 months implantation. Each square represents the classification decoding accuracy of a single electrode; see scale legend next to each panel. Electrodes colored black had impedances > 1MΩ at 1 kHz and were removed from the analysis. In this figure, we show the accuracy of a classifier trained to decode four different behavioral choices; chance performance would then be 25%.

As an index of this day-to-day variability, we identified, for each recording session, those electrodes that were in the top 20th percentile of decoding accuracy. We then generated a heat map reflecting the number of times that each μECoG electrode was part of this top 20th percentile. Figure 10 shows this heat map.

Figure 10.

Heatmap of the channels that were in the top 20th percentile of decoding accuracy. The heat map, corresponding to the same electrodes in figure 9(a), shows the number of times (see color bar at right) that each μECoG recording channel, which is represented by a square, contributed to the 20th percentile. Values were calculated across all sessions and across both types of decoding (TCr and behavioral choice).

In our first decoding analysis (figure 11(a)), we decoded TCr as a function of the number of electrode channels. Although, the decoding of N = 1 channels was somewhat variable, we found that the median performance of each recording session was above chance levels (i.e. because there were 5 TCr values [−5, 0, 5, 10, and 15 dB], chance performance was at 20%). Across all sessions, for N = 1 channels, the median decoding accuracy for the broadband signal was 27% (interquartile range [IQR]: 25%–30%). This median broadband value was greater than chance performance (Wilcoxon sign-rank test; H₀: median value is equal to chance; p < 0.000001). As we decoded more channels, the decode accuracy increased from a median value of 27% for N = 1 channels to a median value of 36% (IQR: 33%–41%; p = 0.002) when we combined the N = 64 + and N = 128 + channels (we combined across these channel counts because we could not calculate N = 128 + decoding for all sessions; see figure 11). Overall, we found that as we increased the number of channels (N = 1,2,4,16,32,64,64+,128+), the median decoding accuracy increased with channel number (Kruskal-Wallis test; H0: median difference between decoding accuracy is the same p < 0.000001).

Figure 11.

Population decoding accuracy of the μECoG array. Each panel shows the decoding (classification) accuracy of the μECoG array for (a) TCr and (b) behavioral choice as a function of the number of simultaneously decoded channels. Data points show the mean decoding accuracy. Error bars represent the standard deviation of the mean. Different colors represent performance from different recording sessions, the color legend indicates the recording day post implantation. In each panel, the dashed horizontal line illustrates chance performance.

The decoding of behavioral choice is shown in figure 11(b). Across all sessions, the median decoding accuracy for the broadband signal was 59% (IQR: 50%–64%) for N = 1 channels. For a combination of N = 64 + and N = 128 + channels, the median decoding accuracy was 76% (IQR: 72%–78%). Both of these median values were above chance performance (Wilcoxon sign-rank test; H₀: median value is equal to chance; p = 0.002, both). We also observed that, as we increased the number of channels from N = 1 to N = 128 +, we found an increase in decoding accuracy (Kruskal-Wallis test; H₀: median difference between decoding accuracy is the same; p < 0.000001).

In general, the decoding accuracy was relatively constant over the ~1-year time period of study with some notable exceptions (e.g. session 321). Nonetheless, we tested whether we could identify systematic changes that occurred over this time period. One possibility is that the μECoG electrode array degraded over time or had some other idiosyncratic fluctuations in the function of the electrode array. To address this question, we replotted the decoding accuracy of the array for behavioral choice as a function of time from implantation (figure 12(a)) for the broadband data. Over an approximate 1-year period (relative to surgical implantation), we could not identify a reliable monotonic decline or improvement in decoding accuracy for N = 64 + channels (Spearman correlation; H₀: no monotonic relationship between decoding accuracy and date of recording session; ρ = 0.33, p = 0.09). A second possibility is that the impedance of the electrode changed over time. As seen in figure 12(b), we could not identify any appreciable decrease or increase in the average impedance of the μECoG channels. Another possibility is that this variability reflects the behavior of the monkey: that is, the session-by-session fluctuations in the monkey’s behavior was reflected in session-by-session fluctuations in decoding accuracy. However, we could not find evidence for this possibility: the broadband decoding accuracy of behavioral choice (TCr) did not correlate with the monkey’s hit rate, false-alarm rate, or d’ values (Spearman correlation; H₀: no monotonic relationship between session-by-session decoding accuracy and behavior; all p values >0.05).

Figure 12.

Population decoding accuracy and impedance of the μECoG array as a function of time. (a) Decoding accuracy of behavioral-choice data from 64 + simultaneously decoded channels. Data are organized as a function of time. (b) Impedance of the electrode contacts over the duration of the implant. The mean impedance is plotted in red and the shaded area represents the standard deviation of the impedances. Electrodes with impedances >1MΩ at 1 kHz were excluded from the analysis. Electrode impedances were measured more frequently than behavior.

In our final analysis, we asked whether the spatial organization of the μECoG recording channels on the array affected decoding accuracy. This question was motivated by the observation that with as few as N = 1 randomly selected electrodes, we could decode TCr and behavioral choice at better than chance levels.

To explore this question, we conducted another series of analyses in which we sampled over different spatial extents of the array (figure 13). We first, like above, used all electrodes with impedances <1MΩ at 1 kHz. We tested the decoding accuracy of ‘low’, ‘mid’, and ‘high’ density configurations of the electrode array. ‘Low’ density refers to simultaneously decoding using 4 electrode channels with a nearest neighbor of ≥6. This effectively means that electrodes are separated ≥3.05 mm; as a reminder, the inter-electrode pitch is 610 μm. ‘Mid’ density refers to simultaneously decoding using 12 electrodes with a nearest neighbor of ≥ 3. ‘High’ density refers to simultaneously decoding using 24 electrodes with a nearest neighbor of 1. Each decoding group (spatial density) was iterated 20 times with a random selection of given number of channels and density. Under these testing methods and behavioral task, we found that sample density did not affect decoding accuracy. That is, as the sampling density increased, the median decoding accuracy was the same (Kruskal-Wallis test; H₀: median difference between decoding accuracy is the same; p = 0.9). However, the high-density array could be used to improve decoding accuracy through optimized channel selection. We found that, under the same testing methods with figure 11, if we only considered individual channels with decoding performance in the top 20th percentile, decoding accuracy improved dramatically (figure 14). For N = 16 + channels, the median decoding accuracy for TCr was 67% (IQR: 61%–72%) (figure 14(a)) in comparison to 36% for N = 64 + channels that were chosen randomly (figure 11(a)). Similarly, the median decoding accuracy for choice was 88% (figure 14(b)) in comparison to the 76% for N = 64 + channels (figure 11(b)).

Figure 13.

Decoding (classification) accuracy of behavioral choices as a function of electrode density. The thin lines represent performance from different recording sessions. Different colors indicate the different sessions; see legend. The thick black line represents mean performance across all sessions. Error bars represent the standard deviation of the mean. ‘Low’ density refers to simultaneously decoding 4 electrode channels with a nearest neighbor of ≥6. This effectively means that electrodes were separated ≥3.05 mm; as a reminder, the inter-electrode pitch was 610 μm. ‘Mid’ density refers to simultaneously decoding 12 electrodes with a nearest neighbor of ≥3. ‘High’ density refers to simultaneously decoding 24 electrodes with a nearest neighbor of 1.

Figure 14.

Population decoding accuracy of the μECoG array when only using channels in the top 20th percentile of individual channel decoding. Each panel shows the decoding (classification) accuracy of the μECoG array for (a) TCr and (b) behavioral choice as a function of the number of simultaneously decoded channels. Solid lines show the mean decoding accuracy. Error bars represent the standard deviation of the mean. Different colors represent performance from different recording sessions, the color legend indicates the recording day post implantation. In each panel, the dashed horizontal line illustrates chance performance.

4. Discussion

We developed a modular recording system that can scale to record larger channel counts without increasing the footprint of the implant. To our knowledge, this is also the first demonstration of long-term recording from a non-human primate from a μECoG array (electrode pitch <1 mm) with hundreds of channels. Utilizing silicone molding, we combined the sensing area of three thin-film sub-arrays to form a uniform high-resolution μECoG array. This array contains 294 channels and covered 10.4 × 11 mm². The modular approach not only allowed customizations of electrode size, shape, and density for the targeted brain area but also allowed the acquisition interface to expand vertically instead of horizontally, which minimizes the implant footprint. In addition, this modular approach could expand and scale to record over a thousand channels with larger brain coverage by stacking more PCBs and molding more electrodes together. Although we found that having a large number of electrodes in the array was more important than the density (see figures 11 and 13), the two are related: a large number of electrodes essentially requires high density.

We also developed a design approach for custom 3D-printed titanium recording chambers that are watertight, optimized for surface arrays, and are individually mated to the skull of each animal and recording target. Unlike penetrating microdrives that rely on penetrating through a combination of silicone grease, silicone sealant, and Silastic membrane to prevent fluid ingress [98], surface electrodes typically have a flexible cable, which is thin but wide, which cannot penetrate through these sealant layers. To achieve a watertight seal, we developed a molding technique that generates a highly customizable silicone rubber wall that compresses and deforms around the electrode cable to seal the electrode entry point. We also created a silicone gasket to seal the gaps between the chamber cap and base. This design approach can be easily applied to all surface array implants and allows the PCBs within the chamber to be upgradable to add additional capabilities such as recording and stimulation.

When we used the μECoG array to decode different task parameters, we had two major findings. First, there was substantial day-to-day variability in the decoding accuracy of each electrode. We attribute this variability to poor connections in the Nanostrip connectors between the implant PCBs and the recording headstages (Intan Technologies, Inc). We have not observed this variability in prior studies that did not utilize Nanostrip connectors [47, 63, 64]. Nonetheless, when we examined multiple electrodes simultaneously, we found that we achieved significant decoding accuracy that was maintained for over 300 d. The degree to which these differences reflect the information being decoded, the μECoG array itself, and/or the brain area under investigation is unclear. Further work is needed to clarify this issue. Second, with our behavioral task, we found that, whereas channel number did significantly improve decoding accuracy, the specific spatial distribution of the channels (figure 13) did not, in general, affect decoding accuracy, which was not consistent with our original expectation. Nonetheless, we found that when we decoded TCr and behavioral choice, a high-density array could improve the decoding accuracy through optimized channel selection (figure 14(a)). The effect of increasing the number of channels (compare N = 1 and N = 16 +, figure 14(a)) was more dramatic with optimized channel selection. This suggests that, at least in the vlPFC, there is a benefit from using a high-resolution electrode array.

From a neuroscience perspective, this study contributes substantially to our knowledge of the vlPFC. First, we do not have a full understanding of the vlPFC’s contribution to perception and behavior. Is it low-level stimulus information or higher-level perceptual information [78, 99–102]? Here, we found that the μECoG signals, which are surface electrical brain potentials, coded both types of information: sensory (TCr) and perceptual (choice). The basis for the encoding of this information can be seen in the averaged ERP traces that are shown in figure 8. In this figure, it is clear that there is a distinct relationship between behavioral choice and the ERP signature. Importantly, for both false-alarm and correct-rejection trials, because the target vocalization was not presented, the differential modulation on these two trials types cannot be attributed trivially to different stimulus conditions. The relationship between different TCr values was less obvious in the ERP traces than it was for behavioral choice. Consistent with this observation, the decoding accuracy for TCr values was generally less than the decoding accuracy for behavioral choice (figure 11). Interestingly, when we randomly select 128 + channels of electrodes (figure 11(a)), the decoding accuracy of TCr was less than if only choosing the best performing 16 + electrodes (figure 14(a)). This seems to suggest that sensory information in vlPFC is rather focal and would be hard to find with a low-density array. This rich and diverse quality of information has tremendous potential for future studies that would utilize vlPFC, or more generally PFC, μECoG signals, as part of a brain-machine interface (BMI).

Unlike traditional BMIs that typically subsample channels for the purpose of reducing power usage of an implanted device, we found that optimizing the channel selection could also improve decoding accuracy (figures 11 and 14). With a high-density array, it might make sense to subsample channels based on the current task needs, just like the brain may not readout all neurons (channels) equivalently [24]. From a neuroscience perspective, channel selection may very well be similar to a downstream population of neurons that reads out only the most informative neurons [26, 103]. For example, those neurons whose firing rate was modulated to the greatest extent (and/or most reliably) by different values of choice, TCr, or only those neurons whose tuning properties matched the frequency content of the target vocalization [39, 104–106]. As shown in figures 11 and 14, when we only select the top 20th percentile channels for decoding, the decoding accuracy was boosted in both TCr and choice. This finding supports future work aimed at exploiting high-channel number arrays: implanting a large number of electrodes and then selectively choosing the best electrodes to use for decoding would enable higher performance than a lower-density array.

Finally, another point to consider when optimizing decoding accuracy is the time interval under consideration. Here, we used the mean value of the μECoG amplitude, which was generated over the entire duration of the target vocalization. Instead, we could have broken this duration up into smaller intervals and tested whether this ‘temporal’ information could have improved decoding accuracy and utilized a broader swath of channels. Indeed, in the auditory cortex, the timing of extracellular spiking activity contains significant information regarding the location of a sound source [35–38].

5. Conclusion

We developed a new high-resolution and high channel-count surface electrode array that can be scaled to cover larger cortical areas without increasing the chamber footprint. Further, we demonstrated the utility and robustness of this high channel-count, high-resolution chronic μECoG array by finding that both sensory and perceptual information can be decoded from vlPFC μECoG signals.