Towards Large-Scale Human-Based Mesoscale Neurotechnologies

Abstract

Direct human brain recordings have transformed the scope of neuroscience in the past decade. Progress has relied upon currently available neurophysiological approaches in the context of patients undergoing neurosurgical procedures for medical treatment. While this setting has provided precious opportunities for scientific research, it also has presented significant constraints on the development of new neurotechnologies. A major challenge now is how to achieve high-resolution spatiotemporal neural recordings at a large-scale. By narrowing the gap between current approaches, new directions tailored to the mesoscopic (intermediate) scale of resolution may overcome the barriers towards safe and reliable human-based neurotechnology development, with major implications for advancing both basic research and clinical translation.

Graphical abstract

INTRODUCTION

Intracranial recordings have provided an unprecedented opportunity to study the basic neural processes underlying human behavior. Major advances in human neuroscience have been carried out at multiple scales, ranging from single neurons to field potentials, to address diverse human behaviors across perception, action, and thought (Engel et al., 2005; Jacobs and Kahana, 2010; Mukamel et al., 2005). However, it is increasingly evident that there are significant gaps in current technologies. We largely fail to capture local and near-local sub-network activity, for example within and among cortical columns, that is likely central to emergent properties giving rise to behavior. This issue refers not only to the density of electrodes, but also, and just as critically, to the extent of the brain being covered and sampled.

Devices currently in use for human intracranial recordings include microelectrodes to record from single neurons and macro-scale to record local field potentials. At one end of the spectrum, single cell recordings have provided remarkable demonstrations of neural correlates to cognition, such as the selective responses of temporal lobe neurons to different images of a celebrity or an object as evidence of higher-order visual conceptual processing (Quiroga et al., 2005). Other recent examples include the discovery of cells in the medial temporal lobe that encode spatial location during virtual navigation (Ekstrom et al., 2003; Jacobs et al., 2013) and lateral temporal cortex responses to speech sounds (Creutzfeldt et al., 1989). What is so powerful about these findings is that they are seemingly at the basic unit of neural computation—and also that they have been obtained using the same microelectrode approaches that have been the mainstay of electrophysiology in animal studies for decades, allowing for common dialogue and interpretability in the broader context of neuroscience.

While studies using single cell recordings in humans are growing steadily, at the other end of the spectrum, the number of studies utilizing macroelectrodes to record local field potentials from the cortical surface (known as electrocorticography, ECoG) has exploded. These recordings typically overlay an estimated 10⁵ neurons, representing heterogeneous cellular and synaptic inputs, and including sources that are near- and far-field (Miller et al., 2009). Despite the relatively coarse nature of these recordings compared to single unit data, the achievements have been equally tremendous as those obtained with single cell recordings, and have added significant knowledge to our understanding of human brain function in such critical areas as movement, language, and memory (Engel et al., 2005; Jacobs and Kahana, 2010; Lachaux et al., 2012). Outside of their contribution to basic neuroscience research, both approaches have provided unexpected, but important information about the basic pathophysiological processes underlying human neurological diseases that could not have been observed from recordings carried out outside of the cranium (de Hemptinne et al., 2013; Schevon et al., 2012; Truccolo et al., 2014; Worrell et al., 2012).

While these advances have been important, future progress will be heavily constrained by significant technological limitations that apply to the clinical context of human brain recordings. This is an issue because a technological revolution in systems neuroscience is currently underway that will make the divide between experimental animal work and human physiology greater than ever before.

In the realm of animal models, advances in photonics and imaging in conjunction with genetically encoded calcium and voltage sensors now allow hundreds to thousands of neurons to be monitored simultaneously at cellular and single action-potential resolution (Ahrens et al., 2013; Deisseroth and Schnitzer, 2013). This represents a major paradigm shift because there is a realization that current small-scale recordings are missing out on critical information that can only be interpreted from the analysis of large-scale multi-neuron activity. The goal of these new technologies is to maintain the cellular resolution of single unit studies but to dramatically increase the number of units simultaneously monitored, so that entire local circuits or whole brain areas can be studied at once (Alivisatos et al., 2012).

This has been described as the ‘meso-scopic’ scale of neural circuits and populations, because this scale of resolution is intermediate between that of single cells studied with microelectrodes and entire brain regions mapped using indirect methods such as fMRI, MEG and EEG (Figure 1)(Freeman, 2005; Sejnowski et al., 2014). Most of these new methodologies in animals cannot be applied in humans, at least in the short term. Accordingly, a major challenge facing human neuroscience is how to achieve a similar transformative scientific goal of mechanistic specificity.

Figure 1. Dimensions of spatial and temporal resolution in human neurophysiology.

Axes not drawn to scale. Large-scale ‘micro-ECoG’ may play an important role in advancing intermediate meso-scale, multi-scale neurophysiological recordings. Note that coverage (spatial extent) is just as important as spatial resolution. Adapted from Sejnowski et al., 2014.

In animal models, large-scale recordings with local precision will result in a complexity and richness of data that will be required for a mechanistic understanding of brain function. Such recordings promise to advance our knowledge beyond single-cell neural correlates of behavior, towards the ability to address how collective processing of populations of neurons give rise to emergent properties underlying complex behavior and function. It is paramount that the same perspective be applied towards human neuroscience.

The neurological underpinnings of many human behaviors are not amenable to direct investigation in animal models. This is especially true for neurological disorders, the majority of which have unique expression in humans. Because emerging genetic-based approaches to neural recording will not be feasible in humans in the foreseeable future, and single cell resolution at the scale of the whole human brain, let alone a single gyrus, is currently unfathomable, the question of how best to move forward is important and worthy of careful consideration. That is, to date, no available methods exist for recording neural activity in humans that scales the gap between microelectrode and current ECoG resolution and coverage. Achieving this goal has the potential to powerfully link mechanistic data from animal models with map-level data in humans.

In this perspective, I will suggest that to advance human-based neuroscience at the mesoscale, new neurotechnologies will be required that are appropriate for use in humans, are applicable in clinical settings, and that target the scales of resolution lying between those obtainable with current microelectrode and ECoG approaches. To clarify the practical context, I will first briefly describe the obstacles that arise from the clinical settings of epilepsy surgery in which intracranial recordings are carried out. I will in turn suggest new practical approaches to large-scale recordings, primarily focused at the cortical surface, which may lead us to substantial improvements in both spatial and temporal resolution.

The window of opportunity for intracranial neurophysiology

In clinical settings, intracranial neurophysiology is commonly applied for diagnostic purposes and/or for pre-interventional brain mapping related to surgery for neurological disorders. The application of neurophysiological mapping, in which anatomical correlates of function are empirically identified in individual patients, is critical for the effectiveness and safety of these procedures. Intracranial neurophysiology is routinely carried out in the surgical treatment of epilepsy, movement disorders, psychiatric conditions, brain tumors, and pain. Here, I will focus on the role of neurophysiologic mapping in epilepsy surgery because, as described below, its application in this setting arguably holds the most important implications and greatest potential for the future of mesoscale recordings.

Epilepsy surgery is an important and heavily underutilized treatment option for patients who suffer from refractory seizures that are not controlled by medications. The rates of complete seizure control after surgery can be as high as 80%, and follow-up studies have documented the long-term success of surgical treatments (Englot and Chang, 2014; Englot et al., 2012; Jobst and Cascino, 2015). However, the outcome of epilepsy surgery critically depends upon whether the seizure source can be well localized. If not, the success rate drops considerably. As a result, most patients undergo extensive preoperative workup to localize the seizure focus. Noninvasive imaging modalities sometimes fail to isolate the seizure focus or there is discrepancy between different preoperative tests. In these cases, patients will undergo a surgical procedure to implant intracranial electrodes. The implantation serves two purposes: 1) to help pinpoint the location of the seizure focus to be removed surgically, and 2) to facilitate electrical stimulation-based brain mapping of critical brain regions that need to be protected during surgical resection. In many cases, neither of these objectives can be definitively met with non-invasive imaging or recordings.

Depending on the clinical situation, the surgical implantation is usually carried out following craniotomy by placing thin electrode subdural arrays, called ‘grids’, on the exposed brain surface below the dura, and often supplemented by gently sliding additional electrode strips under the dura to unexposed cortical surfaces(Fountas and Smith, 2007; Van Gompel et al., 2008). The conventional grid is often configured as an 8 × 8 array, with 1 cm inter-electrode spacing. The typical macro electrode contact diameter is 2 – 3mm. This relatively large electrode surface area contributes to low impedance characteristics (around 100 ohms), which are highly favorable in the electrical noise-rich environment of a hospital room. A grid can cover a substantial part of the lateral hemisphere, but typically samples sparsely, from only a few electrodes on a given gyrus. Thin penetrating cylindrical ‘depth’ electrodes are also commonly used, either in combination with surface grids or alone (called stereoencephalography, sEEG) (McGonigal et al., 2007) to access medial or deeper aspects of the cerebral hemispheres. The wires from the implanted electrodes are usually tunneled through the skin and connected to an amplifier. Patients are then monitored in a hospital ward until several electrographic seizures are captured, after which electrodes are surgically removed. An example of implanted intracranial electrodes superimposed on 3D reconstruction of the cerebrum (Figure 2).

Figure 2. Implanted intracranial electrodes superimposed on 3D reconstruction of the cerebrum.

A. Electrode position and relative size in red. Standard 1 cm spaced array over frontal lobe (20 electrode), and ‘high-density’ 4mm array over the lateral cortex covering peri-Sylvian regions (256 electrodes). Subtemporal strip electrodes. Exposed area of electrodes are to scale. B. Speech sound stimulus acoustic waveform. C. Example neural response spectrograms from two neighboring electrodes (z-score) on the superior temporal gyrus. D. Single-trial, high gamma rasters at individual electrodes.

The placement of electrodes is governed by clinical indication and need. Because it is not possible or practical to safely cover the entire brain, electrode coverage is usually guided by clues from the preoperative workup. ‘Eloquent’ brain areas, such as the sensorimotor or language cortex, are often covered by surface electrodes to facilitate electrical stimulation mapping. Penetrating depth electrodes are rarely placed in eloquent areas unless there is compelling evidence that the seizure focus is overlapping, because of the risks of direct injury and local hemorrhage. Overall, intracranial monitoring is safe but has known associated risks of hematoma, cerebrospinal leak, pain, and infection. A recent large series showed complication rate as high as 7%, but there was no permanent morbidity or deaths (Hedegärd et al., 2013).

It is during the seizure monitoring phase, which typically lasts from 7–10 days, that patients voluntarily participate in research studies. Notably, this corresponds to a relatively stressful time for patients. It being soon after surgery, patients often have headaches and are understandably anxious. Remarkably, given these conditions, a large proportion of patients typically volunteer to participate in research — usually motivated by the desire to contribute to knowledge that might help others.

Indeed, the potential for discovery in such studies is extraordinary, and has already been demonstrated on a number of critical fronts. For example, because the temporal lobe is the most common location for seizures, it is a frequent site for electrode implantation. As a consequence, there have been numerous revealing studies of human memory in mesial temporal structures such as the hippocampus and entorhinal cortex (Jacobs and Kahana, 2010). Similarly, a range of studies on auditory, speech, and sensorimotor processing have been made possible by virtue of the fact that electrodes employed for clinical stimulation mapping often cover these areas.

Given the opportunity afforded by human intracranial recordings, an assessment of current technologies, and their potential for advancement, is warranted.

Microelectrode recordings

Microelectrode recordings are not a standard part of the clinical evaluation for the localization of seizures, but there exists a long, productive history of microelectrode recordings for research purposes in epilepsy surgery. Single unit recordings were first done in humans about 60 years ago, investigating the neural changes that are associated with seizures (Calvin et al., 1973). Soon afterwards, microelectrode recordings were applied in several brain regions, including the thalamus, medial temporal lobe(Halgren et al., 1978), and lateral temporal cortex(Ojemann et al., 1988). Early studies were carried out acutely in the operating room during awake craniotomies, and subsequently performed using implanted electrodes.

Several investigators have designed modifications of standard clinical electrodes to facilitate clinical research needs. For example, microwires can be can inserted as a bundle through the tip of depth electrodes (Fried et al., 1999), or embedded between adjacent macroelectrodes (Howard et al., 1996). The yield of such approaches is can be highly variable, subject to electrode failure and often providing recording from only 1–2 units per bundle of 9 microwire electrodes (Misra et al., 2014). At present, these are technically difficult procedures from which to obtain consistent and reliable responses.

Fully configured surface-penetrating silicon microelectrode arrays have also been used in clinical settings (Chan et al., 2013; Truccolo et al., 2014). However, scaling up microelectrode recordings to cover more than a relatively small area, currently about 1 cm², poses a real challenge. Owing to safety considerations associated with the potential for tissue damage, penetrating microelectrode arrays are not typically placed into critical brain areas such as the intact, functioning sensorimotor or language cortex. Indeed, for this reason most research studies employing microelectrode recordings have been carried out in brain areas destined for removal (Ojemann, 2013), however in most cases of seizure focus localization, such regions are not clearly defined at the time of implantation.

Electrocorticography (ECoG): surface recording technologies are an important step forward

ECoG refers to neural recordings that are carried out directly from the cortical surface. A critical advantage of ECoG-based surface recordings is that they are inherently safer than alternatives for direct recording because they do not penetrate the pial surface of the brain. This attribute has significant implications for device safety, in addition to facilitating neural recordings from critical brain regions that would otherwise be inaccessible, for ethical reasons, using penetrating electrodes (such electrodes are typically only used in brain areas destined for excision or permanent lead implantation).

Popularized in acute intraoperative clinical use by Herbert Jasper and Wilder Penfield in the 1950s, ECoG electrodes were later configured into non-penetrating implantable arrays. The impetus for implantable ECoG arrays was the limited spatial resolution of noninvasive recordings from the scalp, electroencephalograhy (EEG). However, beyond simply improving the spatial selectivity of neural recordings, direct recording of neural signals at the brain surface has led to the discovery of a critical high frequency component of the neural signal that was previously underappreciated. Until relatively recently, the majority of ECoG studies have relied on analysis methods commonly used with EEG such as averaged evoked potentials and spectral analysis with emphasis on traditional oscillatory brain frequencies (i.e. delta, theta, alpha, beta, and gamma). Indeed, until the last decade, it was commonly assumed that ECoG signals were similar to those of EEG recorded from the scalp, and as a result, investigators rarely examined signals above the 50Hz cutoff applied in most EEG recordings. Furthermore, scalp EEG is usually clinically examined as a raw signal in the time domain, and owing to power-law scaling of the bioelectrical potential (Miller et al., 2009), high-frequency activity is far lower magnitude and not easily visible compared to low-frequency components.

Primarily driven by increasing the sampling rate capacity of digital amplifiers, our view of the high frequency ECoG signal has significantly expanded over the past 15 years (Lachaux et al., 2012). In the human sensorimotor cortex, Crone found that there is an evoked high frequency component of the spectral power between about 70–150 Hz, which was 1) reliably stimulus-locked, 2) very spatially focal, and 3) temporally precise (Crone et al., 1998). While high frequency responses in experimental animal work had previously been observed, Crone’s was a sentinel finding because it was one of the first demonstrations in humans that these signals could be safely and practically detected from the cortical surface—thereby catalyzing widespread interest in human intracranial recordings.

What is the significance of the high frequency signal for interpreting cortical surface recordings? The high frequency component was initially interpreted as related to the oscillatory gamma band (around 40Hz), and therefore termed ‘high gamma’. However, accumulating evidence suggests that the origin of the high-frequency component is not oscillatory, like most other lower EEG/LFPs frequency bands, but rather is part of a larger ‘broadband’ spectral source generated by local non-rhythmic synaptic activity and action potentials, and is not directly related to oscillatory gamma (Manning et al., 2009; Miller et al., 2014; Ojemann et al., 2013). Indeed, the value of ‘high gamma’ as an index of population spiking activity has a rich history in experimental animals (usually exploited as multiple unit activity, MUA), and the relationship between LFP, EEG, ECoG, high gamma and unit firing has been comprehensively described in two recent reviews (Buzsáki et al., 2012; Einevoll et al., 2013).

These observations have been transformative for the application of ECoG to the interrogation of human cortical circuits. For example, high gamma signal in the human auditory cortex is evoked robustly by speech sounds. A depiction of human temporal lobe cortical response to a speech sound (/sa/) is provided in Figure 2. The spectrogram of the neural response is shown in Figure 2C. The high gamma portion is primarily above 70Hz. Despite the relatively low magnitude, the signal-to-noise ratio of the high gamma response is very high and can be observed on single trials. As a result, high gamma field potential responses can be shown as single trial rasters, as is often done with single-unit recordings (Figure 2D). Not surprisingly, the high gamma power has been strongly correlated with neuronal firing rate (Manning et al., 2009; Ray and Maunsell, 2011; Steinschneider et al., 2008), and, interestingly, also with the fMRI BOLD signal (Conner et al., 2011; Mukamel et al., 2005; Ojemann et al., 2013). High gamma amplitude has also been shown to couple with the phase of lower frequency signals (e.g. theta or beta) during behavior (Canolty et al., 2006) and abnormally in disease states (de Hemptinne et al., 2013). Complementing the lower frequency signals commonly measured from scalp EEG, intracranial detection of cross-frequency interactions may provide a critical approach to understanding how local neuronal processing is coordinated in broad, distributed networks. Overall, the high gamma signal in ECoG has already provided significant indications for the value of intermediate-scale neural recordings – for understanding systems-level behavior as well as achieving multi-scale integration across microelectrode and noninvasive modalities in humans.

The rationale for improving recording density and coverage for clinical and research purposes

There are substantial limitations to the clinical effectiveness of intracranial recordings as it is currently applied, and any advance in the state-of-the-art for human recordings must help overcome these shortcomings. Despite the invasive monitoring in the epilepsy setting and extensive evaluations that lead up to it, a significant proportion of patients do not become seizure free after surgery (this particularly true for patients with a normal appearing MRI). The reasons are multifactorial, but one of the primary causes is from significantly under-sampling the locations of potential seizure foci. In effect, it is easy to miss the actual seizure onset zone (Noe et al., 2013; Wetjen et al., 2009). With our typical recordings, the fraction of total brain volume that that we actually record from is very small—it has been estimated to be on the order of 1% (Halgren et al., 1998; Lachaux et al., 2012). The complex three-dimensional structure of the human brain makes it nearly impossible to record comprehensively from within sulci and fissures, or deep structures such as the insula and hippocampus. In addition, the relatively large electrode contact sizes employed with ECoG obscure heterogeneous tuning of local neuronal processes and it is more difficult to pick up higher frequencies of the neural signal, which may aid in identification of seizure foci. For example, abnormal high frequency oscillations (HFOs), which are better detected with small electrode contact sizes, have been suggested as a potential biomarker of epileptic seizure foci, (Worrell et al., 2012).

Clinical outcomes may also be improved by the development of improved methods for mapping brain function, especially in the areas that are being considered for possible resection. Electrical stimulation mapping is a fairly coarse technique and has many shortcomings. The stimulation itself may trigger seizures during the mapping procedure (Tate et al., 2013), and is inefficient because only one location can be tested at a given time. Most importantly, when mapping areas outside of the sensorimotor cortex, electrical stimulation is prone to false negative results (e.g. appears to be silent/non-eloquent, but is not). Stimulation can disrupt ongoing behavior in a task— for example, one’s ability to name visual objects or count numbers during language mapping. However, stimulation at subthreshold levels or the absence of an appropriately sensitive behavioral assay may lead to an underestimation of important functional localization (Hamberger et al., 2014). In contrast, brain mapping based on ECoG recording during behavioral tasks without stimulation has been directly compared with electrical stimulation mapping, and appears to be a promising alternative (Cheung and Chang, 2012; Lachaux et al., 2007; Miller et al., 2011; Schalk et al., 2008; Sinai et al., 2005).

To summarize, while human intracranial recordings are a precious opportunity, they are also constrained by significant limitations. The ethical considerations are paramount, and as a result, neurotechology development needs to be carried out hand-in-hand with clinical needs. Penetrating microelectrode recordings are feasible in specific contexts, but it is not realistic to be scale up such approaches to the extent required for meaningful, distributed coverage without inducing significant brain damage. Likewise, key brain areas of interest will be inaccessible because of the potential for brain injury. Meanwhile, standard clinical macroelectrodes can be placed safely over eloquent areas, but provide unsatisfactorily low resolution, and typical applications only yield a few electrodes over a given gyrus. From a clinical standpoint, improving our ability to cover more areas with local resolution will allow us to better map the onset of seizures and understand seizure propogation patterns, while also facilitating more detailed mapping of critical brain areas defined physiologically.

To address this, several years ago we started employing ‘high-density’ 4mm spaced ECoG grids, in a 16×16 array, for 256 total electrodes (Figure 2). There was no additional risk from this modification, but it resulted in a greater than 4-fold increase in electrode density that powerfully facilitated studies addressing the functional organization of the human speech cortices (Bouchard et al., 2013; Chang et al., 2010; Mesgarani and Chang, 2012; Mesgarani et al., 2014). These discoveries were made possible by leveraging the spatiotemporal specificity of high gamma ECoG responses recorded from distinct local tuning at single electrodes, while also covering the entire cortical area of interest and sampling many response types.

In Mesgarani et al., for example, on average 40–130 electrodes in individual patients were found to be speech-responsive in the superior temporal gyrus while they listened to continuous speech (Mesgarani et al., 2014), compared to the handful of responses typically recorded from conventional arrays (Chang et al., 2011). Further, continuous speech has phonetic segments that last on the order of 50–100 milliseconds. The spatial and temporal resolution offered by these high density recordings demonstrated that selectivity differed between adjacent electrodes, and that by recording from a more densely sampled portion of the superior temporal gyrus, the diversity of response selectivity in adjacent patches of cortex could be comprehensively captured. This global view was critical because it allowed us to address the structure of acoustic speech information processing by comparing and contrasting response selectivity across the population, as opposed to focusing only on the encoding at single electrode (Figure 3). The study also revealed that the distributed response selectivity exhibited hierarchical organization, structured along important acoustic-phonetic featural distinctions. This was not a technological breakthrough by any means, but it clearly shows why the current alternatives could not have addressed these scientific questions and how even incremental improvements can have very meaningful impacts. Despite these major advances, we know that this merely represents the tip of the iceberg in terms of understanding how local neural ensembles are coordinated as a network within a given gyrus. Even higher density recordings that approach the level of the cortical column may allow us to examine network-level organization for the first time.

Figure 3. Population cortical responses to speech listening obtained with complete, higher-density coverage of human superior temporal gyrus.

Hierarchical clustering of single-electrode and population responses to phonemes in continuous speech. Rows correspond to individual phonemes, and columns correspond to single electrodes. Clustering across both axes demonstrates phonetic feature selectivity and hierarchical organization. From Mesgarani et al., 2014.

Defining the right scale

The optimal spacing of electrodes for ECoG in humans is currently unknown. While there is no question that the 10 mm spacing on most conventional arrays heavily under-samples cortical areas (Menon et al., 1996), we have very little information about the actual density and coverage that is ideal for guiding new neurotechnologies. As described above, individual electrodes demonstrate significantly different responses at 4mm spacing (Flinker et al., 2011), suggesting that there is considerably more room for increasing density. While we know that higher density is generally needed, the point at which signals from closely spaced electrodes become redundant has not been systematically determined in humans. Furthermore, many physical and practical considerations become relevant when scaling up the number of channels and shrinking electrode contact size.

The anatomic and physiologic scale of response selectivity in the cortex may reveal some clues to ideal electrode spacing. Anatomically, cortical ocular dominance columns in the human visual cortex are about 1mm wide, whereas in macaques they are about 0.4mm wide (Adams and Horton, 2009). The presence of cortical columns and their roles in other cortical regions is unclear, so it is important to also define spacing functionally. Previous studies have attempted to define the optimal spacing of ECoG electrodes using a combination of modeling approaches and existing data. For example, the biophysical correlation predicted by volume conduction (finite element modeling) and spatial spectral analysis have suggested minimum spacing of 1.7–1.8 mm for subdural recordings (Slutzky et al., 2010). Several studies have applied 16 channel microwire array densities at 1 mm spacing and have found overlapping signals as well as meaningful differences between adjacent electrodes (Kellis et al., 2010; Leuthardt et al., 2009).

A major consideration is how the neural correlation between adjacent electrodes is heavily affected by which frequency band is being examined. High-frequency band neural activity is spatially more localized as compared to low-frequency band neural activity, which his more distributed (Menon et al., 1996; Schalk et al., 2007). Therefore, optimal electrode spacing is a frequency dependent parameter. For a 4 mm spaced electrode grid, there appears to be a systematic relationship between spatial correlation and frequency. Using actual data from those recordings, we found significant differences in functional spatial resolution depending upon the frequency band of interest (Figure 4). This general relationship is not novel, but systematic quantification of these parameters is still lacking. More importantly, it shows that current ECoG grids can capture the spatial resolution of lower frequency bands, but the resolution falls short of the resolution limits for high gamma, and likely, even higher frequency responses.

Figure 4. Relationship between spatial distance and signal correlation across electrode pairs, stratified by frequency band.

Derived from actual data (unpublished), human cortical recordings obtained on a 4mm spaced ECoG grid. Significant differences in spatial resolution, especially at less than 1 cm, can be observed depending upon the frequency band of interest. Note that correlations less than 4mm are extrapolated, as 4mm is the shortest distance assessed.

As density increases, the electrode size will necessarily decrease. Using a volume conduction model, the effect of electrode size on spatial resolution can be estimated. An electrode that is too small will not provide the necessary sensitivity to deeper cortical sources, whereas one that is too large will record unwanted deep sources and a larger volume of tissue overall. A recent analysis found that decreasing the cortical surface of an electrode beyond a 1mm provides little benefit to spatial resolution for relevant laminar sources in the cortex (Wodlinger et al., 2011). These estimations were done on the neural signals in the time domain under rest conditions. As a result, the functional independence of neural signals may be significantly greater in the context of task-based information processing, especially in the high frequency domain where smaller electrodes may gain sensitivity.

Decreasing electrode size has several practical tradeoffs that may affect the ability to carry out reliable and sensitive recordings. The signal levels are lower as fewer sources are being recorded from, and prevalence of noise grows dramatically with the increased electrode impedance. These factors may contribute to the low yield of embedded microwires in currently available commercial ECoG options, as previously mentioned (Misra et al., 2014). Small electrode contacts will require head-stage pre-amplification, which is not typically necessary with larger conventional ECoG contact sizes. Micro-scale electrodes will be more susceptible to several other important factors such as the effects of cerebrospinal fluid on electrical conductance and shunting, the movements of the brain within the skull from hemodynamic pulsations and head movement, and increased impedance from tissue reactions at pial surface. These are important practical limitations that will directly affect the reliability of micro-scale electrodes and may render them unacceptable for clinical purposes.

Achieving the full potential of ECoG at mesoscale resolution will require newer microfabrication approaches that have more direct control and flexibility over the miniaturization of electrodes. It is necessary to optimize the electrode size, material, roughness, and geometry for the specific application. Most current commercial methods still rely on bulk metal electrodes and discrete wires to form electrical cables. Due to the manual manufacturing methods, these devices are limited in their ability to scale to higher number of electrodes or improve electrode spacing. New microfabrication approaches, originally designed for semiconductor device manufacturing, are a promising direction (Fukushima et al., 2014; Hollenberg et al., 2006; Minev et al., 2015; Rubehn et al., 2009; Viventi et al., 2011). Microfabrication consists of three fundamental steps: 1) layer by layer deposition of materials such as metals, ceramics, or polymers, 2) photolithography to transfer a pattern from a mask to the underlying layer so that certain areas are protected while others are exposed, and 3) selective etching of the underlying exposed layer while keeping intact the areas protected by photolithography. These three process steps are repeated with various materials, geometries, and layers to build three dimensional structures, and form the basis for all electronic devices and many sensor applications (Tooker et al., 2012). An example of a microfabricated micro-ECoG array for human cortical neurophysiology is shown in Figure 5.

Figure 5. Examples of micro-fabricated ECoG array for human application.

A microfabricated ECoG grid consisting of platinum metal conductors (silver) between two insulating polymer layers (translucent). The circular pads are exposed platinum electrodes designed to make contact with the brain. The platinum lines are insulated and form the routing wires that connect the electrodes to the recording instrumentation. Manufactured by Lawrence Livermore National Laboratories.

Biocompatible electrode materials such as platinum can be deposited and lithographically patterned in polymer substrates (such as polyimide, parylene, silicone) to make extremely thin, conformal recording sensor arrays and routing wires. By taking advantage of scaling in microfabrication technologies, features less than a hundredth of the size of the traditional bulk manufacturing methods are possible. Further, by stacking multiple layers of polymers and conductive routing layers, a significantly higher density and number of electrodes can be achieved. Prototype micro-ECoG arrays have been tested successfully in animal models and have provided examples of safety and reliability in chronic settings, as evidence that these approaches can be reverse engineered for experimental work in animal models (Bastos et al., 2014; Fukushima et al., 2014).

A recent notable breakthrough using microfabrication approaches was the demonstration that action potentials can be recorded from the cortical surface in both rodents and humans (Khodagholy et al., 2014). The ‘neuron-sized-density’ (30 micron spacing) enabled microscopic sampling of field potential and spiking activity. This observation is important because it suggests that optimal electrode spacing is relatively arbitrary, with one far end capable of exquisite spatial and temporal resolution at the level of spikes. Possibly, the high frequency response scales all the way to the level of action potentials depending upon electrode parameters.

With these new approaches, empirically defining the relationship between electrode spacing, contact size, and frequency band is an important priority. Analytically, optimal spacing can be addressed by several approaches that have been defined for emerging neurotechnologies. One important factor is defining a useful framework for quantifying the informatic limits for a given approach (Cybulski et al., 2014; Marblestone et al., 2013). These informatic limits can help guide technology design by quantifying the information content of an electrode array as a function of the spacing between electrodes.

Increasing electrode density has a direct tradeoff with coverage. This occurs because current state-of-the-art neurophysiology acquisition systems cannot process more than several hundred channels simultaneously. While the technology for sensor microfabrication is evolving rapidly and can be scaled with relative ease, the solutions for transmitting the signal at that scale are far behind. It is not possible to have hundreds or thousands of wires routing percutaneously through the scalp in clinical setting. Large channel counts have major implications for every downstream component, including connectors, routing, amplification, signal processing, and storage. Multiplexing the signals will be required to get all the signals in a single or few wires.

New human-based technologies would ideally be built to transmit such signals wirelessly through the skin from an implanted telemetry system for safe chronic applications. Generally, performance features such as robustness, power consumption, and efficiency are especially critical in human applications. Limited recordings can be achieved with off-the-shelf technologies, but exceeding a thousand channels will require dedicated, novel algorithm and hardware development for processing massive data bandwidths. While solutions will certainly come from related fields in materials and wireless engineering, a major challenge is that the specifications of currently available medical grade components lag far behind those of advanced components developed for consumer applications.

With these above considerations, it is possible to propose a feasible near-term plan to achieve mesoscale intracranial recordings. The optimal spacing is still unknown but most evidence suggests that a practical target range is around 500 microns to 1 mm given the potential tradeoffs. This should be refined empirically with task-based studies addressing the relationship between spacing and contact size, towards resolving more local, higher frequency neural signals (e.g. beyond high gamma). Microfabrication approaches are a very promising avenue to the development of scalable, customized micro-ECoG arrays. Scaling channel counts may pose a bigger challenge as current data transfer and amplification will not scale easily, and represent a potential bottleneck to massive channel count recordings that should be addressed early on.

The applications of mesoscale human recordings are myriad. Previous efforts have concentrated on brain-machine interface purposes, which are completely synergistic with the goals described here and summarized in several excellent previous reviews (Hatsopoulos and Donoghue, 2009; Moran, 2010). I have focused on the context of epilepsy surgery, because, as in the past, this setting will continue to play a major role in intracranial neurophysiology as we enter the era of large-scale recordings justified both by clinical and research needs.

For research purposes, a realistic and highly productive scenario would be to functionally map out a human gyrus in high detail. At 1 mm spacing, for example, this task would require an array with spatial coverage of 600 electrodes (about 1.2 cm wide and 5 cm long). This would represent a 100-fold increase in density over current conventional ECoG arrays, but could be performed with currently available neurophysiology data acquisition systems. A more ambitious goal might be to map out the entire lateral temporal lobe at the same density, which would require on the order of 5–9×10⁴ electrodes. At this scale, more customized electronics would be required.

While surface recordings hold tremendous promise, there are other important neurotechnology challenges for human recordings. For example, surface arrays placed over the convolutions of the brain surface can typically only access the gyral surface, lacking access to the two-thirds of total cortical surface hidden in sulci. These spaces are typically lined with arachnoid adhesions, which can only be accessed by delicate manual dissection – but thin, flexible microfabricated arrays have the potential work there (Matsuo et al., 2011). Another major challenge is how to achieve similar mesoscopic resolution from deep subcortical nuclei, such as the basal ganglia, which are defined by 3-dimensional structural organization. It is still unclear how to develop a multi-channel sensor configuration that can capture this spatial geometry while also being minimally invasive.

The development of new human-based neurotechnologies can be addressed by multi-disciplinary teams, integrating expertise from neuroscientists, physicians, and engineers. It will also require significant input from biomedical device industry partners and regulatory oversight from the FDA. The technical and personnel challenges of such endeavors seem daunting, but real efforts are now underway. Importantly, there are also significant spin-off opportunities for ‘reverse translation’ of human-based technologies, such as ECoG, to animal models. Using shared technologies in humans and animal models has profound implications for meaningful study of basic mechanisms and development. By complementing other approaches and scales of study (e.g. optogenetics (Ledochowitsch et al., 2011), more invasive recordings, and disease models), a common technological approach holds the potential to identify robust principles and paradigms shared across neural systems.

CONCLUSION

In sum, despite the constraints of clinical settings, there is extraordinary opportunity in the coming decade for human-based neurotechnology development. Mesoscale recordings accomplished by leveraging high-density surface recordings have tremendous potential for addressing how local neural processing is carried out in the context of more broadly distributed networks. Modern electrode microfabrication and signal processing can make some of these goals realizable in the next few years. The application of new technology must meet the highest ethical and safety standards, and when it does so, should be used whenever possible, as the implications for human health and knowledge are significant.