Listening to Brain Microcircuits for Interfacing With External World—Progress in Wireless Implantable Microelectronic Neuroengineering Devices

Abstract

Acquiring neural signals at high spatial and temporal resolution directly from brain microcircuits and decoding their activity to interpret commands and/or prior planning activity, such as motion of an arm or a leg, is a prime goal of modern neurotechnology. Its practical aims include assistive devices for subjects whose normal neural information pathways are not functioning due to physical damage or disease. On the fundamental side, researchers are striving to decipher the code of multiple neural microcircuits which collectively make up nature’s amazing computing machine, the brain. By implanting biocompatible neural sensor probes directly into the brain, in the form of microelectrode arrays, it is now possible to extract information from interacting populations of neural cells with spatial and temporal resolution at the single cell level. With parallel advances in application of statistical and mathematical techniques tools for deciphering the neural code, extracted populations or correlated neurons, significant understanding has been achieved of those brain commands that control, e.g., the motion of an arm in a primate (monkey or a human subject). These developments are accelerating the work on neural prosthetics where brain derived signals may be employed to bypass, e.g., an injured spinal cord. One key element in achieving the goals for practical and versatile neural prostheses is the development of fully implantable wireless microelectronic “brain-interfaces” within the body, a point of special emphasis of this paper.

I. INTRODUCTION

New generations of supercomputers are breaking milestones beyond the petaflop barrier, with some (well earned) fanfare [1]. While these computing behemoths represent the pinnacle of contemporary computing and information processing abilities of man-made electronic machines, at the highest levels of any technologies, it is useful and even humbling to consider the biological supercomputer endowed to us by evolution—namely the human brain. This approximately 1.5 liter volume of squishy soft matter, weighing somewhere between 1.3 and 1.4 kg, and having a perhaps not-so-impressive visual appearance to an engineer, nonetheless easily outperforms any man-made machine across a huge spectrum of computational characteristics, even without accounting for its most unique and advanced aspects such as sophisticated cognitive abilities.

To give a rough perspective, the human brain, a highly distributed, parallel, and hierarchal biological computer, is composed from about 10¹¹ neural cells, nature’s own “transistors.” While the details of its local microcircuitry vary a great deal from one region of the brain to another [2], one universal feature of this 3-dimensional “ultra-VLSI” design is the large number of interconnects between individual neurons (up to 10³ and beyond for each). This richly networked circuitry enables approximately 10¹⁵ interconnections (synapses that also feature remarkable plasticity) to operate on the neural code, a uniquely complex “software” only partially understood today, and arithmetically equivalent to well over petaflop computational rates—even if the individual neurons switch ionic currents at sluggish msec speeds. And all of this with total power consumption of about 20 watts, less than a laptop computer.

The quest of modern neuroscience, in close partnership with engineering, physics, microbiology, etc., bifurcates research into: i) unraveling of the neural code and circuit operation at increasing levels of complexity to decipher brain functions such as sensing, planning, and specific commands and ii) the application of such learning to new neurotechnologies in clinical and rehabilitative context. The former defines an enormously rich domain especially in the exploration of the primate brain (monkeys and human subjects). The latter refers to the efforts to build a new generation of biomedical engineering concepts to meet and deal with a spectrum of severe neurological and other impairment in human subjects, among which restoration of or substitution for lost neural function is central to work reviewed in this paper. Accordingly, the principal aim of the paper is to offer a general reader snapshots of recent progress in the development of techniques for “listening” to the brain, specifically in primates, within those surgically accessible regions where our understanding of the neural code is beginning to take shape. For example, decoded “motor” command signals issued by and extracted from the brain are now employed in the very first human trials to enable paralyzed individuals (spinal cord injury, stroke) to operate external assistive devices such as a computer keyboard.

This paper is organized as follows. Sections II and III provide a short introduction to the principles of acquiring signals by electrical recording from within the brain, while Sections IV and V highlight cutting edge research in primates. Specifically, in Section II we describe neural recording from live brain circuits (in vivo) by cortex penetrating microelectrode arrays which record local electrical activity from targeted populations of neural cells (neurons)—at a single neuron spatial and temporal resolution. In Section III, information from such multichannel recording devices (up to one hundred individual “intra-cortical” microelectrodes at present) is shown to yield surprisingly rich information of specific functions of the neural code, exemplified by “thought-to-motor action” command signals issued for movement (e.g., of arms, legs, etc.). We review recent work, where such neuroengineering techniques have yielded a rich template of information in nonhuman primates (monkeys)—and culminated in the first human pilot trials with severely neurologically disabled subjects. In Section IV we describe current state-of-art from the viewpoint of compacting and integrating the bulky neural signal acquisition, processing and transmission electronics, typically residing outside a subject, towards wireless “portable” or “wearable” systems. Aided by modern microelectronics, we show, e.g., how the cumbersome tethering of the wiring cabled from a subject’s head to external electronics can now be replaced by short range wireless broadcasting units which are head-mounted atop the skull of a monkey. Using this new generation of miniaturized units, though requiring a wiring path from the intracortical microelectrode arrays (residing below the skull) through a subject’s skin, enables us to highlight the importance of specialized high performance microelectronic chips, including the need for very low power, ultralow-noise analog integrated circuits. A culmination of this paper is Section V, where we focus on ongoing efforts to integrate the cortical micro-electrode recording array plus its integrated “system-on-chip” signal acquisition and telemetry system, as a single unit embedded entirely within the body (head), without any skin penetrating wiring. We show examples of research from the authors’ group where the proof-of-concept for such fully implantable wireless “microsystems” have now been initially demonstrated in monkeys. The ability to employ fully wireless, implantable neural interfaces at multiple locations in the brain, that are enveloped by the body’s most potent protection against infections, etc., namely the skin, opens up potentially entirely new vistas for next generations of neurotechnologies, with opportunities and challenges summarized in the concluding Section VI.

Note to the reader: Advanced microelectronic on-chip systems are beginning to play increasingly important role for future electronic interfaces with the brain and neural circuits. As an example, a recent special issue of IEEE Transactions on Neural Systems and Rehabilitation Engineering (IEEE TNSRE) shows several examples of sophisticated approaches to miniaturized wireless low-power microsystems for neural signal capturing, processing, and wireless telemetry [3]. It is the authors’ view, however, that for any chronic in vivo and clinical applications in primates, it is imperative that all such designs and device fabrication platforms are closely matched, up front, with the numerous challenges faced by any implantable biomedical engineering devices—including physiological, anatomical, surgical, and, above all, safety considerations. As a research modality, this requires a rather seamless chain of intimate cross-disciplinary teaming across a broad swath of physical, life, and medical sciences expertise, executed as a continuous and coherent effort for any new device system concept from its inception to final clinical transitioning.

II. MICROELECTRODE ARRAYS FOR EXTRACELLULAR RECORDING FROM NEURAL MICROCIRCUITS

For in vitro preparations (such as cultured neurons in a dish, or slices of rodent brain), the time-dependent nonlinear (current-voltage) characteristics of a single neural cell can be quantitatively measured and modeled with high degree of precision [4]. This is accomplished by physically piercing the cellular membrane by submicrometer tipped capillaries that provide a conductive path to the interior of the neuron, without breaking the elastic membrane though which the key ionic constituents (K⁺, Na⁺, Cl⁻, etc.) flow to form a brain’s transistor. These so-called patch-clamp methods are extraordinarily difficult to extend to in vivo work in living subjects, especially for multiple neurons. However, there is a long history of using bundles of “extracellular” microwires inserted into brain tissue, where inert metal needles such as Pt or Ir record the “fringing” electrical fields and potentials at their unisulated tips near a neuron. Since the potential from the neurons outside their membrane drops rapidly with distance akin to that of a electrostatic current dipole, a guide of thumb is that any given exposed microwire “antenna” tip should reside within about 30–50 μm from the neuron’s cell body within the background of conductive brain tissue to acquire a usefully measurable signal [5]. We note that all present noninvasive (i.e., external to the skull) “long-distance” brain “imaging” techniques, from EEG to MEG to fMRI, lack spatial resolution anywhere near the single neural cell level.

Invasive insertion of microelectrodes as two-dimensional or quasi-three-dimensional arrays (MEAs) into brain tissue at a given cortical location enables the capture across a population of neurons of the ~msec duration biphasic action potentials (or “spikes”), at single cell resolution, whose repetition or “firing” rate informs much of the neural code. Quantitative modeling of the circuit physics for the neuron-induced electrical signals is complex, influenced by the role of the electrolytic bilayer at the electrode tip/brain tissue, tip shapes, tissue reaction, etc. [6]. Metals such as Pt and Ir are chemically stable in the tissue’s seawater-like environment and have reasonable work functions to match the electrolyte. In addition to the action potentials, attention is increasingly paid also to lower frequency potential contributions (so-called LFPs), which likely contain important complementary information about the local neural network dynamics. Thus the equivalent circuit of the single microelectrode/tissue recording interface should typically cover a bandwidth of ≈1–10 KHz.

Microelectrode arrays composed of bundles of wires have been now largely replaced by “monolithic” arrays for work in primates (monkeys, and now in first human trials). Of these, we mention the Si-based arrays where lithographic and electrochemical techniques are combined to fabricate tapered microscale “beds of needles” [7]–[9]. For example, in the so-called “Utah” MEA (inset of Fig. 1), each approximately pyramidal-shaped 1–2 mm long electrode, with its p-Si shank insulated by biocompatible parylene, is coated at its tip by Pt or Pt/Ir. The heavily doped Si provides a low-loss conducting path onto the planar support substrate wherefrom a wirebonded bundle (say, 100 insulated 1 mil Au wires) transports the recorded neural signals through the skull and the skin to exterior electronics (Fig. 1). Operationally, specialized neurosurgical techniques are employed to access cortical areas of interest with the arrays inserted to the brain typically using calibrated pneumatic single-tap impactors. The fabrication of this type of “intracortical” microelectrode structure has now been advanced to wafer scale processing [10] so that the physical length of the microelectrodes can be graded across the array, thereby enabling access with spatially controlled depth across multiple layers of the brain’s cortical circuit structure. The recording performance depends on many factors including a given surgical approach so that quantitative comparison of performance between geometrically different MEA types is often ambiguous, e.g., in monkeys [11].

Fig. 1.

A silicon-based cortical microelectrode array (inset); implanted for intracortical neural microcircuit recording via a percutaneous connection to a skull mounted pedestal connector (main figure schematic).

In case of a typical intracortical array designed to access the motor cortex in a monkey or a human primate (subject/patient), the adjacent electrode spacing is a few hundred μm, laid out as a square lattice in part to optimize recording from individual neurons, though finite “interference” from adjacent neurons is a practical challenge for signal processing. The occasional pickup from two nearby neurons can generally be discriminated and separated from their time-amplitude stamps by downstream “spike sorting” algorithms. Given the cortical anatomy (density of neurons), the arrays may in a successful case of a monkey pick up useful signals at vast majority (up to 90%) of the electrode sites, even as the underlying neuron organization is spatially rather random. A single action potential spike might generate recorded amplitudes from few tens up to > 100 μV (lower left panel of Fig. 2), registered by an electrode of ~500 kΩ in impedance, thereby framing the design requirements for subsequent analog preamplifiers (gain, noise figure, etc.), digitizing circuits, and the downstream signal processing and data management electronics. Note that apart from fundamental sources of noise common to all electrical system (Johnson noise etc), the brain operates against its own inherently “noisy” background (since its circuits are continuously “alive”), thereby adding to the challenges of acquiring consistent neural signals that correlate with specific action and behavior. At the other end of the spectrum of microelectrode performance, so to speak, is the question how reaction by the body tissue on implantable electrodes and related device structures affects their truly long term chronic performance (≫ 1 year)—as well as impact a subject’s safety [12], [13].

Fig. 2.

(Upper left): Location of the arm area in the cortex; (lower left): typical action potentials of neural spikes; (right panel): local neural landscape with a single, needle-like recording microelectrode in the vicinity of a neuron (cell body size ~20 μm).

III. INTRACORTICAL MICROELECTRODE ARRAYS IN ACTION—ACCESSING THE NEURAL CODE OF THE MOTOR CORTEX FOR “THOUGHT-TO-ACTION” IN NEURAL PROSTHETICS

Within the past decade, experiments on nonhuman primates (monkeys) have become progressively more sophisticated in enabling recording from brain microcircuits by intracortical MEAs with sufficient fidelity so that “cracking the neural” code for specific functions by the motor cortex has become possible. The permanently (chronically) implanted arrays yield signals from regions of the brain that can now be directly related, e.g., to intended arm/hand movements [14]–[19], [60], enabling long term (> 1 year) exploration of motor cortex space in task performing monkeys. Within the primary motor cortex, the location of the prime source for commands to the arm is known with an approximate spatial map indicated in Fig. 2, which also sketches an image of the local neuronal architecture (“pyramidal cells”). With recent development of decoding techniques towards “real-time” algorithms, based on probabilistic analysis (a rich and critical subject we are not able to cover here; but see, e.g., [20]–[22], [61]), good correlation has been achieved for the arm movement of a monkey between the signals recorded directly from the brain (“thought-for-action”) and the physical action by the animal in 3-dimensions. The inputs for decoding movement kinematics are correlations in the rates/phases of spiking activity across a microelectrode array, in conjunction with tracking the motion of the arm of the monkey, e.g., using a joystick or finger touch to move a cursor on computer screen. As a recent example, the neural commands thus decoded have enabled a monkey to operate both real and virtual robotic arms [23]. The driving long-term goal of this work is to develop a neurotechnology that might significantly restore or replace lost functions in neurologically or physically impaired humans.

One highlight of research in recent human pilot trials has shown how the recorded intention-driven neuronal ensemble activity can be converted into a control signal that enables a tetraplegic patient to perform useful tasks [24]–[26]; for a broad overview, see, e.g., [62]. We note that earlier Kennedy and colleagues using a glass cone electrode [27] demonstrated a type of “communication prosthesis” that used only one or two neurons from the motor cortex of locked-in human patients to slowly move a cursor across a virtual keyboard to type out messages. Fig. 3 shows a photograph of the current “Braingate” version of the 96 channel brain recording platform in a human trial setting, a multi-institutional effort with team led by the authors (JPD and LRH) and their collaborators. The system is based on chronically implanted silicon MEA, as described above, with the neural signals guided via a 100 wire bundle through the skull and skin to a specialized head-mounted titanium-based connector. The connector enables cabled access to external electronics which combine low noise preamplifiers for each channel with their signal multiplexing, perform analog-to-digital conversion, execute spike analysis and other signal processing tasks (the preamplifier module is directly head mounted in the current FDA approved trial version). Finally, the decoded and processed neural signals are interfaced with external devices such as an electronic mouse to enable direct cortical control by the brain of a cursor on a computer screen or an artificial prosthetic device (e.g., a wheelchair).

Fig. 3.

Photo of a subject patient in a human clinical pilot trial, operating a cursor on a display screen by direct cortical “thought-to-action” control (after [24], [62] and courtesy of BrainGate2.org). The implanted multielectrode array is connected via a skin penetrating wirebundle to a head-mounted stage for analog signal amplification. This stage is tethered to other signal processing (digital) and neural signal decoding electronics in the subject’s vicinity, and the operation is supervised by trained technical personnel.

The early clinical trials have been conducted with several other disabled subjects suffering from major impairment of motor functions. The BrainGate system has demonstrated its enabling capability, e.g., for a patient whose spinal cord is severed at the neck, to control a cursor on a computer screen for communication activities such as reading e-mail, typing messages, drawing elementary free form shapes, and operating a open-close prosthetic hand. Another subject, after suffering a brain-stem stroke nine years earlier, is at this writing employing such tools after more than three years after her MEA implant [28]. Other assistive devices under such direct “brain control” which are being tested and under development include connecting the cortical recording system to a wheelchair and a robotic arm. Further, the use of cortical signals is being pursued as a possible means to enable, e.g., amputees to operate such arms. The striking results from the first human trials are also motivating work to microminiaturize and enhance the performance of these types of neural interface systems by turning to advanced microelectronics, as discussed in the next sections. We also note that much effort is being invested in developing better algorithms for improved accuracy for the “brain control.” Recent results indicate, e.g., that the correlations between spiking activity and the brain’s intended movement that are present in tetraplegic humans in the primary motor cortex (specifically arm velocity and position) share many kinematic tuning features whether movement is imagined by these subjects, or is performed by able-bodied monkeys. Thus ongoing work involves study of design choices for improved neuroprostheses that include kinematic representation and decoding methods that translate neuronal ensemble spiking activity into an ever more reliable control signal.

IV. COMPACTING “NEUROELECTRONICS”—TOWARDS PORTABLE AND WEARABLE WIRELESS SYSTEMS

The advantages of compacting the neural signal extraction and process electronics to wearable or, ultimately, implantable wireless modular systems offer many benefits in untethering the subjects from bulky external hardware. For example, fundamental brain science in nonhuman primates can be advanced through uninterrupted recording from freely moving monkeys. For neural prosthetics and related future neurotechnologies, untethering a human subject via wireless neural communication links would leapfrog many present limitations in performance and quality of care. In this section we show illustrations of recent research towards wireless systems where compact external head-mounted units underscore the motivation and importance to integrate “wearable” microelectronic devices and wireless telemetry physically near/at the cortical sensors. Depending on the application, there are many different ways to group and/or distribute the active electronics on the subject’s body, including their packaging, once the neural signals have been percutaneously extracted from an implanted (electronically “passive”) microelectrode sensor array by cabling through the subject’s skin. In this section we show examples of research at the frontier of such head-mounted external wireless modules for monkeys, while addressing the ultimate goal of fully implanted systems in the next Section V, where we show state-of-the-art work towards fully implantable sensor array/microelectronics platforms where active microelectronic circuits reside within a subject’s body.

Many groups worldwide are now working towards wireless active microelectronic neural interfaces, especially focusing on very low power, system-on-chip integrated circuits which incorporate the analog, digital, and telemetric components, respectively. Several sophisticated designs have been demonstrated at the benchtop level including approaches for circuit integration on a single wafer [29]–[35]. To date, however, much of such integrated ASIC-based engineering work is still awaiting for successful transition to in vivo use, especially for primate research. Here we review one successful transition through the work of Shenoy, Harrison, and colleagues who have developed wireless, atop-head-mounted modules for freely moving monkeys through several stages on miniaturization [36]–[39]. We note that related exterior, headmounted systems, albeit at more modest level of performance, have been developed in recent years for freely moving rodents [31], [35], [41], [42].

The Stanford/Utah group has deployed a system for recording and wirelessly transmitting neural data from electrode arrays implanted in rhesus macaque monkeys freely moving in their cages [38]. As in Section III, neural data are obtained through a 96-channel cortical MEA implanted in macaque motor and premotor cortex, but now with the skin-penetrating percutaneous cable connecting the array to a small printed circuit board (PCB) which houses the active electronics as well as a lithium battery pack (Fig. 4). The PCB resides in an aluminum enclosure attached to the skull by titanium mounting hardware with prior added protection by a thick poly-methacrylate layer (dental cement grade PMMA). The active electronics of this system exploit selected features of a monolithic integrated neural interface (INI) micro-chip design which amplifies, digitizes, and, in one implementation, transmits neural data across a ~900 MHz wireless channel [43]. In the context of a freely moving primate system, the chip was augmented by subunits on a common PCB that distributes the microelectronics and enables connections to a battery power supply, clock, and access to initial programming, all within the headstage enclosure (51 × 38 × 38 mm³). In one version, a stub antenna protrudes 8 mm through a hole in the lid as shown in Fig. 4. These type of systems have been used to record data from rhesus macaques performing many unconstrained regular activities. For a targeted wireless transmission range of ~4 m in free space (consistent with animal’s cage) its total power budget is about 60 mW. (For reference we note that, apart from the module’s size, such power would be prohibitive for implanting a device below the skin.) On a single 2 A-hr battery pack, this system runs contiguously for several days. The wireless, head-mounted system has been used to record and telemeter one channel of broadband neural data at 15.7 kilosamples per second (kSps) from a monkey performing routine daily activities in the home cage (“broad-band” refers to the ability to capture both the action potentials as well as low frequency potentials of importance). The total weight of this system including the batteries is 114 g, suitable for a headmount but considerably heavier than human-wearable devices such as earpiece-mounted personal communication packages. An illustration of the functional performance is shown in Fig. 5 where a photographic image of a monkey reaching for food coincides (and is preceded) by robust increase in the “firing rate” of the action potential spikes from the premotor cortex, wirelessly detected outside the cage. The Stanford group has been able to provide nearly continuous recordings from monkeys so far up to 40 days, demonstrating the paradigm of “always on” neural signal acquisition [39]. The system is being further miniaturized, with recent advances involving a particular approach to on-board multichannel performance (rapidly sweeping across individual channels at broadband performance) and employing RF telemetry near 4 GHz carrier frequency [40].

Fig. 4.

Schematic and geometry of a wireless head-mounted modular system for recording neural signals from monkeys. The constituent elements that are housed in an aluminum can, attached to the animal’s skull, are show in the right photographic inserts (from bottom to top: MEA with its wire bundle, mounting hardware, the microelectronic circuitry on a PCB, and the aluminum enclosure, respectively). After [38], with permission.

Fig. 5.

Upper traces: Video snapshots of a monkey reaching for food outside its home cage. Lower traces: bursts of neural cell firing activity from one channel (microelectrode) recorded synchronously via a wireless link outside the cage (from [39] with permission).

A somewhat related device has been constructed by the authors’ group and demonstrated with a headfixed monkey performing a task, with analog and digital microcircuits capturing full bandwidth neural signals on 16 channels and mounted on a compact PCB within a headmounted enclosure, but choosing the modality for wide bandwidth telemetry via a microcrystal infrared low power (~1 mW) semiconductor laser [44] (see also next section).

V. FULLY IMPLANTABLE WIRELESS NEURAL MICROSYSTEMS—AIMING FOR THE “ULTIMATE” ELECTRONIC INTERFACE WITH THE BRAIN

While advances such as those described above in compacting the analog, digital, and telemetric elements into relatively small head mountable modules represents a significant step in miniaturization of neural interfaces, the ultimate goal in neural prosthetics as well as fundamental brain science in primates envisions a fully implanted wireless system for future high-performance chronic brain-communication interfaces. By this we mean truly body embedded brain-interfaced “microsystems” where any number of neural sensor arrays plus all the active microelectronic circuits are sealed within the (nonhuman or human) primate’s ultimate protective “envelope,” the skin. The neural signals are broadcast only transcutaneously— i.e., without any skin-penetrating (percutaneous) wires or feedthrough connectors, whatsoever.

A fully implantable, wireless system presents formidable biomedical engineering device challenges. Its benefits include elimination of the infection risk which is inherently present with any percutanous connections, reduction in the mechanical vulnerability of the skull-mounted modules to accidental impact (e.g., moving animal or epileptic patient), and a host of clinical and health care benefits that are especially applicable to human subjects and patients. Fully implantable neural microsystems offer the added possibilities of “scaling up” for access to several distinct brain locations for capturing, e.g., neural signals from two or more interacting sites in the cortex that are involved in coordinated cortical activity that integrates sensing, planning and action. Furthermore, such systems can be more flexibly designed for bidirectional communication with the brain.

Here we summarize recent progress in the authors’ group which has produced to our knowledge the first proof-of-concept demonstration of a fully implanted prototype wireless system employed in an awake behaving monkey [45], [46], [63]. In so doing, we will point out key challenges that raise the technical bar for fully implantable neural interface systems in comparison with those that employ electronics outside the body of the subject. The targeted longevity of chronic implants for primates likewise imposes much higher demands of reliability and safety than, say in cases of laboratory animals such as rats or mice.

Given the success in using the specific types of “passive” multielectrode arrays (MEAs) described in the above sections, including their approval by FDA for human trials [47], a reasonable starting point is to employ these structures as the physical platform in adding and integrating active microcircuits for fully implantable microsystems. The most immediate design “grand challenge” then involves the question of the specifics of dedicated microelectronic integrated circuits (ASICs) and implanted telemetry/power units, both in terms of their requisite performance and spatial layout. The convergence to a particular design must include many anatomical, physiological, and surgical considerations, which may force an engineer to compromise in unaccustomed ways to demands of form factors, physical flexibility, device processability, and biocompatibility.

The implantable concept advanced at Brown University over recent years is shown schematically in Fig. 6 [48]–[50]. A single device “microsystem” is constructed on a flexible and durable polymer substrate (Kapton or liquid crystal polymer) onto which constituent components are integrated and assembled in electrically interconnected two-island geometry as described next [50]. Embedded in the flexible Kapton substrate are five 25 μm wide planar wires (for signal and power) that connect the “cortical front end” to the “cranial back end” though a narrow tether. The front end resides below the skull while the backend is designed to reside between the skull and the skin of a primate so that the flexible tether threads through the subject’s skull. The rationale for the dual-platform microsystem, with its spatial division into the two separate microelectronics islands, is based on numerous considerations including reliability, access, signal extraction, and power delivery—and the cumulative learning acquired by physicians and engineers to date with the passive MEA implants in primates (Section III). This spatial division of the active microelectronic component is shown as a block diagram in Fig. 7. We note that integrating all the active microelectronics immediately atop the intracortical electrode array (hence placing the entire microsystem below the skull) was our first purely engineering design choice, but the approach described here appears to resolve a number of very challenging issues of a single intracortical “button.” Such a truly monolithic, compact system is being presently pursued by sophisticated engineering packaging approaches at the University of Utah by Solzbacher, Harrison et al. [37], [51], including early benchtop demonstrations (at this writing) of high performance with only 10 mW total power. This construct takes advantage of the INI chip design mentioned in Section IV, and integrates a receiving coil for conventional wireless inductive power transfer by proximity RF link (power coil external to head within ~cm from the receiving coil).

Fig. 6.

The design concept for a fully implantable, transcutaneous, wireless brain communication interface for primates, composed of the cortical module (with ASIC analog amplifiers and multiplexing integrated onto the multielectrode array) and the cranial module (integrating A/D converter, command/control, and telemetry ICs and components). The images show the placement of the microsystem within a subject’s head. The entire microsystem is mounted on a single flexible polymer substrate which embeds five planar conducting microwires for electrically interconnecting the cortical and cranial units. An external receiver and inductive power supply unit is shown as a compact head proximity unit (see Fig. 9 for details). (Courtesy of Braingate2.org and from [49], [50] with permission).

Fig. 7.

Upper trace: Block diagram of the active device distribution within the implantable microsystem. Lower trace: cross-sectional photographic view of a fully encapsulated device, showing the location of the key components within the cortical and cranial units.

Within the cortical front end of the Brown University design, the silicon MEA is directly flip-chip bonded to a ultralow-power analog ASIC chip which houses preamplifiers addressing each channel (microelectrode) across the entire neural broadband (0.5–5 KHz) plus a multiplexing circuitry for data serialization. The integration of the analog preamplifiers with the MEA is important in order to minimize the distance raw analog signals must travel before amplification. Some of the performance parameters of the chip include gain of 44 dB, bandwidth 2.3–7.3 kHz, noise (RTI) 4.8 μV rms, and power/amplifier channel of 52 μV, with the details of its design described elsewhere [52]. The experiments described below have been conducted with a 16-channel system for logistical reasons, though the scaling to a full 100 channels is readily accomplished and now being implemented by us (matching the present maximum size of 10 × 10 microelectrode arrays). We note that inspiration for these analog ASIC designs has been derived from prior and parallel work by Harrison et al. [53]. We impose as key design criteria that the power dissipation of the ASIC (100 channel) chip does not impart heat to the cortex that exceeds 0.1 °C in the tissue within the volume accessed by the neural probes.

The “back end” cranial unit is fabricated by assembling a dedicated A/D chip and a digital ASIC command/control chip on the same substrate plane that also houses a microcrystal semiconductor laser (VCSEL = vertical cavity surface emitting laser) for transmission of the digitized single composite broadband neural signal data stream through a subject’s skin near 850 nm in the infrared (IR). The data is sampled at 40 kSps—a rate more than adequate for neural signals. Among the advantages of the IR wireless transmission modality is the very large bandwidth which modern optical transceiver systems possess (≫ Gb/sec, if needed). Transmission through a primate’s skin (a rhesus macaque) causes scattering, but a conventional photodiode is still able to pick up the digital stream when placed within ~2 mm of the skin surface (with photon counting electronics the estimated transmission range can readily exceed 1 m). On the flip side of the substrate is a planar RF receiving coil for enabling inductively coupled (transcutaneous) receiving of power and clock to the microsystem. (We note that both power and clocking can also be configured to be delivered optically via an optical fiber using a high-efficiency photovoltaic energy converter [54].) The breakdown of the total system power budget at this writing is shown in Table 1.

Table 1.

Breakdown of Power Consumption of Implantable Microsystem by Components, Based on 3.3 VDC Delivered Wirelessly to the System

Component	Power	Unit
ADC	4.5	mW
Preamplifier	52	μW
Amplifier overall	1.3	mW
Controller	5	mW
VCSEL	2	mW
Total	12.32	mW

The entire microsystem of is presently encapsulated in polydimethylsiloxane (PDMS) for electrical isolation and mechanical flexibility. Surgical implant considerations require careful control of PDMS thickness to maintain flexibility in the tether and to prevent buildup over the electrode array. Fig. 8 shows a photographic image of the entire structure after encapsulation, immersion testing in saline and immediately prior to implant to a monkey (following sterilization), a well as an infrared (night vision camera) snapshot of the IR beam exiting a subject monkey. The main functions of the encapsulation are to ensure: i) that electrical leakage current to the adjacent tissue is less than 10 pA and ii) ionic leakage from the tissue to the electronic components is inhibited. For chronic implant applications, this presents a formidable challenge for all researchers in the field of implantable neural prosthetics. We view our initial approach, using PDMS (NuSil R-2188), as a useful starting pathway at least to subchronic or short-term (~months) in vivo animal testing. In addition, we have designed and implemented an encapsulation test unit (ETU), which simulates the topographical, thermal, and electrical stresses put on the encapsulant, including adhesion issues [56], [57] to test leakage current and component functionality over extended periods of soaking (presently up to six months in accelerated testing in hot saline solution at T = 52 °C. The testing is done using a small test circuit board that is a simplified version of the complete implantable neurosensor. The test structure enables continuously monitoring the resistance between interdigitated conductors on the substrate surface as well as leakage current through the encapsulation material. The leakage currents are a proxy for the presence of ions that might have leaked through the encapsulation material. The test structure includes elements with all the same morphological characteristics that are encountered on the real devices and includes a working ADC. In a test of ten sample devices, the leakage current between bath and circuit was found to typically vary between 1 and 10 pA at 3 VDC with no significant change over time (months). The ADCs provided appropriate data for the duration of the test. In spite of these results, it is clear that chronic implants will require a more reliably impermeable barrier. We are presently exploring combinations of soft organic polymers with inorganic thin film multilayer barriers or heterogeneous mixtures, solid solutions of inorganic molecules in polymers. Ongoing work includes inorganic materials such as SiO_x, SiC, and their mixtures.

Fig. 8.

Upper trace: Full photographic image of the implantable microsystem (displayed as though viewed from the direction of the skull/brain). The cortical front end’s flexibility is shown by gravity pull (for realistic animation of the flexibility of the cortical–cranial tether, see [57] at www.braingate2.org/sensors.asp). The ground reference wire is visible, but the receiving planar inductive coil on the backside of the cranial module is not. The polymer encapsulant shows holes which are presently used to fix the cranial module onto the skull via Ti-screws to prevent motion from impact by a monkey in its cage. Lower trace: Transmitted IR laser from the top of a monkey’s head (restrained), with an implanted microsystem under operation, imaged by a night vision camera.

Various configurations of our microsystem have been so far been surgically implanted into nonhuman primates (rhesus macaques) at Brown University. As elsewhere in this article, all animal procedures were conducted conforming to the National Research Council’s Guide for the Use and Care of Laboratory Animals (1996), and according to protocols approved by Institutional Animal Care and Use Committee (IACUC) at each institution. A recent case study involved a double microsystem implant to a monkey where a fully wireless unit was accompanied with a “wired” version, the latter acquiring the power/clock by a percutaneous connection (skin piercing). The cortical units of the 16 channel microsystems were implanted into the arm region of the motor cortex (primary and premotor areas, respectively) [45], [46], [63]. The purpose of the second “power-by-wire” unit was to compare the quality of the multichannel neural recordings in the presence and absence of the inductive RF coupling unit, and its possible electromagnetic interference effects on the implanted microelectronic circuits. The fully wireless system (i.e., no skin penetrating wiring whatsoever) was employed in conjunction with a compact external module, placed in near contact with the animal’s head, which housed a photodiode for optical detection of the digital neural digital stream, amplifier circuitry, as well as the primary RF coil for power delivery to the implanted system (Fig. 9).

Fig. 9.

Upper trace: Schematic of the external-to-head-unit in an exploded view, displaying the IR photoreceiver and RF inductive power coils, respectively. Lower trace: a photograph of the PCB layout of the unit, where the primary RF coil and the photodiode are “co-centric,” together with the latter’s preamplifier circuit [59].

In the first proof-of-concept experiments, we have found that the fully wireless implanted microsystem in an awake monkey can yield neural signals with good noise ratio as soon as two days after postoperative period, and that the RF inductive powering of the implant does not significantly degrade the quality and usefulness of the signals. The upper trace of Fig. 10 shows recordings from an animal from a completely wireless transcutaneous implant, respectively. The data emphasizes the outcome in terms sampling of the data for a single spike, with the digitized and reconstructed waveforms acquired under realistic testing conditions (i.e., in an awake monkey). It is noteworthy that a very high fraction of the microelectrodes yielded signals with unambiguous single neural cell “spiking” (say, about 12 out of possible 15 channels—one channel reserved for timing synchronization). In terms of the system signal-to-noise ratio, we have achieved quality of signal throughput on par with that acquired by the conventional wired system (such as described in Section III) which employs neural signal amplification and processing through the Cerebus acquisition system.

Fig. 10.

Upper trace: Recordings from a monkey implanted with a fully wireless neural recording microsystem, where neural signals exit transcutaneously by the infrared link, and power to the system is delivered via inductive coupling. Lower trace: Raster plots correlating spiking activity relative to timing of the hand grasp movement of another monkey trained to sit in a chair and grasp (in this “hybrid” device the power to the implant was delivered percutaneously).

In recent and ongoing experiments, specific microsystem configurations are being used to wirelessly transmit acquired neural signals from an animal which is actually performing a task. In the simplest version, once initial characterization was completed of the microsystem, the primate was trained to sit in a chair and grasp. During this exercise, we recorded neural activity over seven trials. The spiking activity correlated well with the timing of the movement—raster plots shown in lower trace of Fig. 10 (in this animal, the power was delivered to the implant via a wire). These results show clearly that: i) our system is able to transmit broadband data transcutaneously (wirelessly) out to a receiver and ii) that the data being transmitted is in fact neural in origin and movement dependant. We note that while the present system attains 16 channels of neural broadband data (one channel, #16, carrying the frame synchronization word to align the received data stream), it has been designed from the outset to be expandable without major redesign—specifically for a 100 channel MEA platform (work under way). A 4 × 4 MEA has allowed for simpler integration and faster turn-around time of devices. However, it is clear from previous work that more neural units, and thus more channels, are required to accurately decode intention in a high-dimensional space, such as in representing wrist and arm joint angles, individuated fingers, and so on. Beyond increasing the number of channels for a given MEA, the layout of our BIC system also allows, in principle, its extension to a number of cortical “front end” implants (various brain recording sites) which are connected to a common subcutaneous backend telemetric unit. As future systems will handle 100 channels of broadband data transmission, one could imagine placing perhaps up to six sensors around various points of interest on the cortex.

VI. SUMMARY

By selected illustrations, we have reviewed ongoing developments, through multiple tiers, of new device enablers aiming to advance our understanding of dynamics of brain microcircuits in nonhuman and human primates—and means to employ extracted neural signals for use in “thought-to-action” brain interfaces to directly interact with external devices. At a fundamental level, one hopes to learn about the operational aspects and language of the brain in ways that has not been possible previously. For future applications, the rapidly developing new neural technologies offer tantalizing prospects, especially for human healthcare in instances of injury or degenerative neurological diseases. The scientific ingredients embedded in brain-interface devices cut briskly across multidisciplinary lines—from specialized microelectronics to implantable biomedical devices and onto neurology and medical sciences, in a manner which requires in their developmental phases a unified and coordinated research modality across such a “rainbow” of technical areas.

Yet several major challenges must be met and await resolution prior to translation, especially of the first proof-of-concept wireless devices in monkeys, to high performance, chronic, and “individualized” brain-communication interfaces, especially for human use. One basic question concerns the long term performance of all the brain-invasive intracortical sensor arrays, including their material ingredients, structural design and geometrical form factors. The arrays must be proven to ensure only modest tissue damage over time spans of years. Even if medium-term tissue-electrode interactions (from immune reactions, etc.) appear presently to be moderate in terms of decreasing the recording performance of the array and impairment of neural circuitry, the longer term data available today in primates is still statistically being accumulated.

Another major challenge concerns the safety and reliability of any wearable or implantable wireless microelectronics packages, including those discussed above. In an wearable case (electronics mounted on the head or elsewhere on the body, with percutaneous wired connection to the brain implanted arrays), obvious criteria involve the built-in safety and security mechanisms to avoid any unintended leakage of currents to the implant in cases of electronics failures—or wirelessly transmitted information being subject to interference or unauthorized use whether via a microwave or infrared broadband link. All such the challenges converge metaphorically and literally in case of the fully implantable active systems. In particular, the demands on packaging materials and techniques are considerably more difficult as one must ensure hermetic sealing on a very long time scale, especially against the penetration of any ionic species across the impermeable encapsulating barriers. Unlike in cases of electrical stimulation implant devices such as pacemakers and cochlear implants (which for reasons of device constructs, their limited function, and anatomical locations can accommodate, e.g., “hard” hermetically sealed titanium packaging), the neural recording implants require the support of steady dc voltages on the order of 3–4 V across any encapsulate materials. Thus long term electric field induced ionic transport, electromigration, and stresses place extra challenges which have not yet been solved. Considerable materials engineering and quantitatively accurate testing will be required (and is under way) to develop the next generation encapsulate/packaging enclosure materials.

Work is also under way in several laboratories to configure (and miniaturize) the external transceiver systems and the hardware/software systems that will enable real-time decoding of the recorded and wirelessly transmitted neural signals, as well as provide commensurate amounts of information storage. In analog to the developments in personal communication devices (such as smart phones, etc.), it is not unreasonable to expect comparably portable/wearable devices being employed as “brain phones” in future neural prosthetic systems.

At the other end of the spectrum, and particular to the fully wireless implant development, is the need to develop surgical implant techniques which adapt to the form factor and layout of the implantable device systems. Specialized surgical approaches will, needless to say, require close interaction with the device scientists to ensure convergence to optimal implant designs from the viewpoint of safety and reliability of the implant process.

That said, the many ongoing technical explorations towards implementing wireless means of listening and capturing the brain’s language at detailed level of its neural microcircuitry represents the beginning stages of what is likely to be an exciting decade or two in both fundamental and applied brain science research. We can envision microminiaturized implantable wireless devices which offer simultaneous access to multiple areas of the brain, including neural stimulation by electrical or optical [57] means, with the aim towards the ultimate aspiration of fully implantable wireless neural interfaces for an eventual “two-way” communication with predetermined functional centers in the primate brain.

Biographies

Arto V. Nurmikko (Fellow, IEEE) was born in Finland. He received the B.S., M.S., and Ph.D. degrees in electrical engineering from the University of California at Berkeley.

He is the L. Herbert Ballou University Professor of Engineering and Physics at Brown University, Providence, RI. His current research involves basic physical and neuroscience, including application of device technology concepts to neuroengineering.

Prof. Nurmikko is a Fellow of the American Physical Society and the Optical Society of America, and Member of the American Academy of Arts and Sciences.

John P. Donoghue received the Ph.D. degree in neuroscience from Brown University, Providence, RI.

He is Henry Merritt Wriston Professor in of the Department of Neuroscience and Director of the Brain Science Institute at Brown. His research is aimed at understanding neural computations used to turn thought into movements and to produce brain machine interfaces that restore lost neurological functions.

Prof. Donoghue has received a Javits Award (NIH) and 2007 K.J. Zülch Prize (Max Planck/Reemstma Foundation).

Leigh R. Hochberg received the B.S. degree from Brown University, Providence, RI, and the Ph.D. degree from Emory University, Atlanta, GA.

He is a Neurologist who holds several appointments, including those at Brown University (Biomedical Engineering), Massachusetts General Hospital, Veterans Administration Hospital Providence, and elsewhere. His clinical Interests spans stroke and neurocritical care, while his translational neuroscience interests include brain–computer interfaces and neurotechnology. His research into neural prosthetics currently involves the first clinical pilot trials in human subjects.

William R. Patterson (Member, IEEE) received the B.Sc. degree in physics and the M.Sc. degree in electrical engineering from Brown University, Providence, RI.

He is currently a Senior Lecturer and Senior Research Engineer at Brown University. His current research interests include low-power analog circuit design for biomedical applications, circuits, and architectures for microphone array technology.

Yoon-Kyu Song received the B.S. and M.S. degrees in electrical engineering from Seoul National University, Seoul, Korea, and the Ph.D. degree in electrical engineering from Brown University, Providence, RI.

Following research faculty appointments at Brown, he is currently an Assistant Professor at Seoul National University. His research interests include basic and applied semiconductor optoelectronics and neuroengineering.

Christopher W. Bull received the B.Sc. degree in mechanical engineering and the Ph.D. degree in materials science from Brown University, Providence, RI.

He is a Senior Research Engineer and Senior Lecturer at Brown University. His research interests include the neural-electronic interface, and complex materials.

David A. Borton received the B.Sc. degree in biomedical engineering from Washington University at St. Louis, MO. He is currently working toward the Ph.D. degree in biomedical engineering at Brown University, Providence, RI.

Farah Laiwalla is currently working towards the M.D. Ph.D. degrees at Brown University, with expertise in integrated circuit design and experimental neuroscience, with interest in clinical applications of neurotechnology.

Sunmee Park received the B.S. degree in biomedial engineering at Seoul National University, Seoul, Korea. She is currently working toward the Ph.D. degree in biomedical engineering at Brown University, Providence, RI.

Yin Ming received the B.S. and M.S. degrees in electronics engineering from Tsinghua University, Beijing, China, and the Ph.D. degree in electrical engineering from North Carolina State University, Raleigh.

He is currently a Postdoctoral Fellow at Brown University, Providence, RI, working on a wireless neural recording system. His research interests include low noise, low power analog/mixed circuit design for wireless biomedical applications.

Juan Aceros (Member, IEEE) received the M.S. degree in mechanical engineering and the Ph.D. degree in electrical engineering from Northeastern University, Boston, MA.

He is currently a Senior Research Associate at Brown University, Providence, RI. His research interests are in the area of fabrication of implantable neuroengineering devices and topics related to micro/nano fabrication and MEMS.

Dr. Aceros is a member of the Materials Research Society (MRS) and the American Society of Mechanical Engineers (ASME).