Autonomous Head-mounted Electrophysiology Systems for Freely-Behaving Primates

Abstract

Recent technological advances have led to new lightweight battery-operated systems for electrophysiology. Such systems are head mounted, run for days without experimenter intervention, and can record and stimulate from single or multiple electrodes implanted in a freely-behaving primates. Here we discuss existing systems, studies that use them, and how they can augment traditional, physically restrained, “in-rig” electrophysiology. With existing technical capabilities these systems can acquire multiple signal classes, such as spikes, local field potential, and electromyography signals, and can stimulate based on real-time processing of recorded signals. Moving forward, this class of technologies, along with advances in neural signal processing and behavioral monitoring, have the potential to dramatically expand the scope and scale of electrophysiological studies.

1. Introduction

Electrophysiology methods pioneered by Evarts and Mountcastle have provided the ability to measure electrical activity from individual neurons in the non-human primate brain during sensory, motor, and cognitive tasks [1] [2]. Such measurements are essential for uncovering fundamental scientific principles and enabling next generation neurological treatments, including brain-machine interfaces (BMIs) (e.g. [3–9]). Unfortunately the current, widely available electrophysiology methods limit the range and complexity of studied behavioral tasks. Recent advances in miniature, low-power recording, processing, and stimulation technology are beginning to release these limitations. These technologies open the possibility for investigating an increasingly broad and naturalistic set of behaviors in non-human primates.

Recordings traditionally use one or more penetrating electrodes to observe changes in the extracellular potential caused by the firing of nearby neurons (e.g. [10–20]). Each individual electrode requires at least one wired connection to a rack-mounted amplifier system. The signal path is very sensitive to micro-motion of the wires and to sources of electrical noise. Therefore studies typically constrain head position, body posture, and range of motion; consequently, experiments are limited to a few hours. This effectively prevents the study of a broad range of natural, real-world behaviors such as locomotion, navigation, object manipulation, vocalization, and social interaction. Importantly, it also limits the ability to study relatively simple behaviors, such as arm movements, across a broad range of behavioral contexts, often considered essential for elucidating the true underpinnings of neural control. New technology and techniques loosen these constraints. Some examples include: studies of neurally or behaviorally contingent electrical stimulation that are typically limited to a few hours in a constrained setting [21] can be broadened to free behavior for days [22]; multi-day studies of the neural correlates of simple arm movements or of neural prosthetic systems become possible [23, 24]; measures of neural tuning stability can be extended from hours to days by tracking neuron identities [25–27].

The history of electrophysiology is punctuated with heroic experiments that broke contemporary conventions by using clever design and new technology to study ever wider ranges of behaviors. In smaller animals, tethered and cantilevered recording systems can be effectively applied if the range of motion is limited and the experimental field is carefully designed, such as a rat navigating a planar maze [28–31]. These experimental techniques have led to many breakthrough studies [32–35]. Such approaches have been applied in primates [36, 37], but monkeys possess a high degree of dexterity and inquisitiveness that limits the applicability of such techniques to short timescale tasks with carefully designed geometries [38]. As Sun and colleagues [39] note humorously: “failure to appreciate the frustration of an unattended monkey can lead to the destruction of expensive equipment.”

Wireless technologies have been applied to eliminate the restrictions of wired tethers. Delgado was an early pioneer of such technologies, most famous for his 1963 bullfighting demonstration. By switching on a wireless neural stimulator in caudate nucleus he stopped a bull charging towards him. Starting in the 1950s, Delgado used wireless neural stimulation to modify free behavior in cats, monkeys, apes, and humans [40]. In addition to stimulation, technology for transmitting single unit activity was developed as early as the 1960s [41–43]. These systems relied on analog signal transmission, in which even highly amplified signals can be contaminated with noise during transmission; they are sensitive to device orientation, occlusion, and interference. Such sensitivities can compromise data as posture and position of the animal can alter the properties of the received signal. More recently, digital systems with onboard data storage or digital wireless transmission with error checking have been employed across animal models, from insect [44, 45] to rodent [46, 47] to primate [24, 48–50].

2. New Tools and Technology

In the last five years, a variety of devices enabling electrophysiological from freely-behaving primates have been developed. Figure 1A is a concept sketch summarizing, and expanding upon, the composite functionality of these systems, providing signal input, processing, and output capabilities. Many of these systems use a head-mounted form factor as shown in Figure 1B. In Figure 1B, a head-mounted electrophysiology system continuously records and wirelessly telemeters neural activity from a macaque in his home cage. At the moment captured by the sketch, the macaque has decided to recover a treat placed on a foraging board.

Figure 1.

Autonomous head-mounted electrophysiology systems overview: (A) shows a system diagram summarizing potential autonomous head-mounted electrophysiology system designs. (B) shows a sketch of how a wireless telemetry recording system operates and provides a general sense of scale of these systems. This is the recording setup used in HermesC [49] studies, in which one broadband channels and multiple threshold channels are recorded and synchronized to recorded video.

On the input side, systems have been developed for intra-cortical (spikes and local field potential) electrodes [24, 48–50], electromyography [48], and behavioral inputs [24, 51]). Signals are processed with methods employed in conventional electrophysiological setups. For example, intra-cortical signals can be amplified, filtered, digitized, and processed with fidelity equivalent to commercial rack systems [24]. These signals are either retained in full broad-band form [24, 48–50], as waveform snippets [48], or by applying a voltage threshold to the spike band to isolate action potentials[49, 52]. Processed signals are stored to onboard memory [24, 48], telemetered via radio link[49, 50], or used to control electrical stimulation output[48]. Additional outputs, such as optical stimulation for use with optogenetics [35, 53–55] or electronically controlled drug delivery could be integrated. The power requirements for optical stimulation are well below the power consumption of current systems and could be enabled with board mountable super-luminescent LEDs or semiconductor laser diodes.

For primate research, systems must be durable, light-weight, and should run without requiring servicing, such as battery or memory changes, for long periods of time to minimize disruptions to free behavior. Researchers have addressed these constraints and developed fully enclosed wireless neural recording systems that store data to onboard memory [24, 48]. Systems with onboard memory have proven fruitful in early studies of electrode interface stability, properties of motor cortical neurons, and BMI applications [22– 24]. However, the data cannot be accessed until the onboard memory is retrieved. Thus, these systems lack real-time access to neural data, making synchronization with stimuli presentation or behavioral measurement, and the study of closed loop neural control difficult or impossible. Fortunately, recent technological advances allow wireless data telemetry [49, 50], enabling a broader class of studies.

Battery life tradeoffs can effect experimental design. For example, the HermesD [50] system is capable of continuous operation for up to two days, recording and digitally telemetering 32 channels of broadband neural data sampled with 12 bits of resolution at 30 kS/s. With the same battery system, HermesC [49] can telemeter one channel of broadband with 10 bits of resolution at 15.7 kS/s and 20 channels of 1 kS/s threshold crossings for 6.8 days. The HermesD system provides higher fidelity data, but limits experimental duration relative to HermesC. HermesD is a better solution if the experiment allows daily servicing of equipment; however, if the study requires longer uninterrupted recording, HermesC may be the better solution. The form factor of these recording systems is small enough to sit on top of an adult rhesus macaques head without any notable changes in behavior; complete systems (chassis, electronics, and batteries) weigh 56-250g (Neurochip 56 g[48], HermesC 114g[49], HermesB 250g[24]) and adult rhesus macaques typically weigh approximately 10kg.

Many other systems using application specific integrated circuits (ASICs) have been developed, and these could be used for freely moving monkey experiments, if appropriately modified, in the future [56–71]. ASICs offer potential benefits to head-mounted systems, including substantial reductions in size and power consumption with simultaneous increases in channel count and signal bandwidth. They could also introduce new modalities such as multi-channel stimulation [60, 67], onboard spike sorting [61, 72], and chemical sensing using voltammetry [69]. Advances in this field are largely motivated by the need for fully implantable systems that move towards the requirements for clinically viable neural prostheses [58, 59, 64, 70, 71]. While some requirements for these clinically oriented devices share common goals with head-mounted electrophysiology systems, fully implantable systems are subject to different design constraints, ultimately distinguishing them from head-mounted systems.

3. Studies to Date

Autonomous head-mounted electrophysiology has demonstrated utility in both the neural engineering and basic neuroscience domains. On the neural engineering side, the HermesB system [24], has documented neural recording stability, and instability, with a Utah electrode array [73] over multiple days. Understanding the properties of electrode arrays is critical to the development of robust neural prosthetic systems and could enable multi-day learning studies requiring the tracking of single neurons. Figure 2A shows action potential waveforms from a single channel across 29.5 hours. Note that the the amplitudes of the two sorted units, from the same electrode, are flexing relative to one another. This neural recording system can monitor head movement continuously with an onboard accelerometer. This study also showed large shifts in waveform amplitude coincident with >3g acceleration events, see 2B. By using head acceleration data as a proxy for overall level of movement, HermesB also demonstrates how a BMI can use local field potential energy to reliably detect periods of physical activity and inactivity. HermesC [49], shown in Figures 2C and D, can track more neural channels for longer period of time, by multiplexing its broadband channel; Figures 2E summarizes data from a nearly continuous 12.5 day dataset. We see that on the multi-day timescale recorded action potentials can be relatively stable (Figure 2F, left), but can also change dramatically (Figure 2F, right).

Figure 2.

The Hermes system and single unit stability across days. (A) Shows the 95% confidence intervals of sorted action potentials from a single channel recorded with HermesB [24] accross 29.5 hours, note that the action potentials of the units are flexing relative to one another. (B) Shows a large shift in peak-to-peak waveform amplitude coincident with a large accelerometer event. (C) Is a drawing of the HermesC system [49] from implant to electronics. (D) is two pictures of the actual system: on the left is the chassis base, Utah array connector, and board stack; on the right is the fully enclosed system with transmitting antenna. (E) shows the 95% peak-to-peak envelope of waveforms observed for 12.5 days for channels 2-20 with HermesC. The voltage is averaged across 3 hour bins, normalized to the average voltage on that channel. Variations fell within 65% - 215% of the mean, or 9.3 uV - 220 uV. As shown in (F), channels can remain static; although single units often appear and disappear.

Studies with the Neurochip system have allowed for novel systems neuroscience studies. In addition to neural recording capability, Neurochip [48] as shown in Figures 3A/B, can record from EMG electrodes and can provide neural stimulation. Jackson et al. [22] utilized this capability to demonstrate plasticity in the motor cortex. Figure 3C shows a schematic of the recording and stimulation paradigm. During physically constrained in-rig experiments, an animal was trained to perform a wrist flexion/extension task in which the tuning direction of an individual neuron could be measured. While the animal was freely moving in the home cage, spikes were recorded from one neuron, Nrec in figure, with characteristic tuning. When that neuron spiked, small stimulation currents were delivered at a second site, Nstim. Over a period of two days, the tuning direction at Nrec became more like that of Nstim, as shown in Figure 3D, suggesting that connections between the two areas had been strengthened in some way. The multi-day timescales required for this particular study underscore the importance of long term recording and stimulation using a wireless device. Previous plasticity experiments have been limited to shorter time scales, and perturbations that the animal can adapt to within minutes. Some mechanisms of learning may simply be inaccessible to the experimenter at traditional timescales. Also, a wide range of new neurological therapeutics might become possible with such a method of encouraging cortical plasticity.

Figure 3.

The Neurochip system and multi-day stimulation induced plasticity. (A) is a drawing of the neurochip system [48] and (B) is a picture of the neurochip board. (C) is a schematic of the paradigm employed by Jackson et. al. in [22], in which an electrode with a sortable unit, Nstim, was stimulated after a delay from when Nrec, a unit recorded on another electrode, spiked. The paradigm was run continuously for 2 days with the neurochip. After the two days, tuning curves for Nstim, Nrec, and a control unit were recorded with a wrist flexion task. (D) shows the tuning curves plotted relative to the axis labels shown. Note that the tuning of Nstim after stimulation is more similar to Nrec than before stimulation and that the tuning of the control cell is relatively unchanged.

A second study using Neurochip [23] measured the correlation between the activity of single neurons and EMG during an in-rig wrist flexion/extension task, as well as freely moving in the cage. They observe similar correlations across the day during free-behavior, but not during sleep. These correlations could vary greatly between free-behavior and a trained task, although the preferred direction of a neuron tended to be preserved. Also, neural recordings during sleep show evidence of the transition between slow wave and REM sleep.

Novel hippocampal studies are also enabled by autonomous head-mounted electrophysiology. Many studies of hippocampus rely on the ability of an animal to navigate an environment. By developing a wireless system [39], Sun and colleagues have successfully recorded from hippocampal neurons while monkeys were navigating a 15 × 15 room to obtain treats in remembered locations. Interestingly, they record place fields, summarizing the correlation between spatial location and neural firing, that change between different task definitions in the same environment. Using a monkey model for hippocampal studies may enable more complex memory and navigation tasks than are currently possible with commonly exercised rodent models.

4. Expanding the Scope of Study

The experiments above highlight a few specific applications of this new class of technology, but why focus on such techniques when complex questions can be studied with carefully controlled, constrained, and measured behaviors in a traditional experimental rig? In the case of motor control, we believe that the study of free behavior can augment traditional experiments in an important way.

As an example, consider the hypothetical progression of non-human primate motor control studies shown in Figure 4A. The traditional setup is depicted in sub-panel 1, where a monkey is engaged in a highly constrained arm movement task, with his head position and body posture fixed. While this task design allows the experimenter to test hypotheses about the motor system with a great deal of control and precision, including the ability to acquire tens to hundreds of repeated trials per behavioral condition (e.g. [74]), it is a small subset of the full capability of the motor system. Dominated by simple point-to-point arm movement tasks, these careful studies have seminally advanced the field over the past few decades. However, even small deviations from the restricted regimes of arm movements studied in these highly constrained tasks (e.g. isolated or enforced shoulder posture, arm orientation, movement speed, and force) can lead to a breakdown in prevailing models (e.g. [14–20, 75, 76]).

Figure 4.

Expanding the scope of motor system studies: (A) shows a progression of potential motor control studies enabled by head-mounted electrophysiology systems. 1 is the traditional wired in-rig setup in which the animal is engaged in a reaching task with a highly constrained posture. 2 utilizes a wireless telemetry system and the animal is able to move to many different postures as he plays with a sensorized manipulandam. 3 shows the possibility of studying complex movements, such as leaping. 4 is the ability to decode motor intentions. (B) is a sketch of a potential computer vision system for recording detailed body posture simultaneously with wireless neural data. The images from all cameras are used to construct a model of all joint positions at every moment in time.

Work by Aflalo and Graziano [77, 78] demonstrates the diversity of neural responses during unconstrained arm movement in rig. In this study, the monkey's head and ipsilateral arm are constrained, but the contralateral arm is recorded during free movement. Thus, the only difference from the traditional setup is the lack of a defined behavioral task. They show that existing models, developed with constrained behavioral task data, do not explain the majority of neural variability in their free movement data. This result exposes a weakness in the prevailing course of study: the limits of task design can lead to impoverished models of the observed neural system. Thus, we believe that data recorded during increasingly complex behavior are necessary to augment the understanding developed from physically constrained in-rig studies.

With head-mounted recording technologies, the field can begin to explore an increasingly complicated range of behaviors with non-human primates, as in Figure 4A sub-panel 2, in which a monkey is manipulating a toy from multiple postures. Such manipulanda can be sensorized to allow arbitrarily complicated behaviors to be precisely measured. Multiple infrared reflective fiducial markers, have long enabled researchers to study 3D position of the joints of the arm and/or hand (e.g. “Polaris”, “Optotrak”, and “Vicon” systems; also [79–81]). Unfortunately, the use of fiducial markers during free-behavior is difficult, requiring the use of “jackets” and even then are likely to be destroyed quickly due to innate monkey curiosity. Ideally, we need a system capable of tracking posture from markerless images. Fortunately, computer vision is meeting such demands for the entertainment industry [82]. These computer vision systems analyze markerless images from multiple cameras views, allowing for 3D reconstruction of the subject's pose from which limb joint angles can be inferred. Figure 4B provides a sketch of such a system in operation, with multiple camera views allowing estimation of all joint positions as shown to the right. Such systems, adapted for non-human primates, could enable the collection of 24 hour continuous datasets of simultaneous full-body posture and corresponding neural data. These potential datasets will test the limits of current models of motor control, by allowing the study of quite complex movements, as in sub-panel 3. As our understanding of the motor system advances, so should our ability to develop prosthetic systems that operate in complex environments and are capable of decoding the mere “thought” of jumping, climbing, or other naturalistic movements, as illustrated in sub-panel 4.

With the ability to record neural and behavioral data for 24 hours a day / 7 days a week from a freely-behaving monkey, a new class of experiments can emerge. By collecting extremely long and precise datasets, experimental questions can be answered by data mining as is commonly explored in bioinformatic studies of the genome. For example, an experimenter interested in studying the neural correlates of limb motor control can query long datasets of free behavior for relevant movements and corresponding neural data.

A growing number of proposed clinical treatments of neurological diseases, including epilepsy [83], Parkinsons [84], and dystonia[85], rely on electrical stimulation of neural tissue. Stimulation based treatments of depression and obsessive compulsive disorder are also currently under study [86]. Wireless recording and stimulation systems for primates can allow for rapid design and testing of such approaches. More generally, continuous neural recording from animal models of neurological disorders could help guide drug discovery and implantable medical device development.

5. Single Trial Analysis & Decoding with Parallel Recordings

In-rig experiments allow the experimenter to repeat trial conditions through training and behavioral control. This repetition allows the experimenter to focus on variability of interest by trial averaging to remove undesired variability, such as Poisson spiking noise [87]. As studies move increasingly toward free-behavior, such repetition becomes increasingly unlikely. We recognize that a major challenge is that every trial is effectively different, thus precluding traditional trial-averaging which is necessary when recording from one neuron at a time. All analysis essentially becomes single trial analysis; we, and others, have been working in recent years to produce turn-key methods for creating “single trial neural trajectories”, with millisecond time resolution, which is possible when simultaneously recording from tens to hundreds of neurons and applying advanced analytical methods (for relevant reviews see [88, 89]; see also [87, 90]).

In addition to using techniques to denoise simultaneously recorded neural data, we can limit the variables under study by filtering for a smaller subset of behaviors. Such an approach can be applied to long datasets; for example, in motor studies one can query for a limited range of movements and postures based upon a question of interest. Alternatively, the use of in cage “smart toys” such as button push panels, manipulandams, and treadmills, provide methods for creating more constrained sets of repeated behavior. These methods allow for a continuum of study complexity, bridging highly constrained traditional in-rig tasks and completely unconstrained free-behavior, as illustrated in Figure 4A.

Single trial analysis techniques are becoming increasingly sophisticated, employing a range of linear and nonlinear dimensionality-reduction techniques, including principal components analysis (PCA) [91–93], locally linear embedding (LLE) [94–96], and Gaussian process factor analysis (GPFA) [90]. These methods can be applied to wireless parallel recordings during free-behavior, such as those achieved with HermesC. HermesC currently records and wirelessly transmits 20 channels of threshold crossings and is readily expandable to 96 channels. In Figure 5B, we project firing rate onto the first two orthogonal dimensions found by the smooth and factor analysis technique [90]. Firing rates are from fourteen electrodes in motor/premotor cortex recorded with HermesC [49] during free reaches to acquire treats placed on a foraging board. Analysis was constrained to outward and inward reach periods, as identified by analyzing video frames synchronized to neural recording with the setup show in Figure 1A.

Figure 5.

An example of in cage single trial analysis and decoding: (A) shows the class of behavior marked in the traces below. Monkey D was engaged in a free-pace/free-posture reaching task to acquire treats placed on a platform in front of her cage. (B) shows the first two orthogonal neural dimensions found by smooth and factor analysis [90] during four minutes of intermittent reaching. Projections were calculated from 14 simultaneously recorded threshold channels collected with HermesC [49]. Color coding marks the period of time for outward and inward reaches consistent with the color of the arrows in (A)

A neural decoder can be developed for this class of data. Using the projected neural dimensions, a support vector machine classifier was trained with the first 17 reaches shown. The classifier found all seven reaches in the remaining period shown, with one false positive. This simple example demonstrates the possibility of decoding a wide range of complex behaviors. Furthermore, by developing neural decoding techniques that apply during free behavior, we may be able to increase the robustness of neural prosthetic systems.

More generally, decoding techniques can be used to test theories about the underlying system. For example, we can compare two models of neural firing by measuring how well they decode or predict behavior. Neurally contingent stimulation, as demonstrated by the Neurochip study [22], can provide evidence for learning and plasticity. This protocol can be generalized from stimulation based upon the firing of a single neuron to stimulation based upon the activity of a population of neurons as interpreted by a decoder. Decoding contingent stimulation could provide a method by which experimenters can attempt to selectively disrupt sensory, motor, or cognitive processing. For example, if we can successfully decode when a monkey is planning to reach, we could stimulate to alter the activity of downstream neurons or stimulate the recorded population to perturb this plan. Such protocols open up the powerful possibility for experimenters to introduce 24 hour a day causal interventions of the neural system. As another, potentially simpler example, it should be possible to electrically or optically (via optogenetics) stimulate cells or regions of the brain periodically (e.g., 20 or 40 Hz) for days and weeks to test theories regarding the causal relationship between rhythmicity and behavior [97–99]

6. Conclusions

Autonomous head-mounted electrophysiology systems have begun to make an impact on both basic neuroscience and neural engineering. As these systems advance and become more accessible to the electrophysiology community, they will augment existing studies with the potential to expand experimental context. Experimenters can begin to release the physical constraints necessary for using wired rack-mounted systems and can expand studies from a few hours to multiple days. The utility of these techniques will rest heavily upon advances in single trial neural signal processing and methods for behavioral measurement, such as markerless motion tracking.