Multi-scale imaging and recording of in-vivo neural activity

Abstract

Most in vivo neural technologies have been developed to enhance the resolution of subcellular neuronal activity or to expand the spatial range for tracking ensembles of neurons in the brains of live animals. While these approaches offer great promise for understanding cellular and circuit functions in behaving mammals, their cross-sectional observations are inherently limited in accounting for causal interactions of neural dynamics across various scales of brain architecture. Consequently, the simultaneous observation of multi-scale neural activity has emerged as a crucial strategy for achieving a more comprehensive understanding of brain function. These advancements facilitate the simultaneous detection of diverse signals, providing unprecedented insights into dynamic neurophysiological mechanisms within three-dimensional brain structures that remain poorly understood. Here, we review state-of-the-art technologies for the parallel observation of multiple neural targets in vivo. We highlight strategies for simultaneously observing brain signals at multiple resolutions, aiming to bridge the spatiotemporal gaps between microscopic and macroscopic domains of neurobiology. We also emphasize the technical integration of neural tools to concurrently acquire electrophysiological activity and optical imaging, leveraging their complementary strengths. Finally, we discuss the future challenges and potential prospects of multimodal neural techniques, paving the way for a deeper understanding of brain functions and disorders.

INTRODUCTION

The bottom-up architecture of the nervous system, where individual synapses and neurons create complex neural circuits throughout the brain, provides a crucial framework for understanding brain function (1, 2). Advances in neural technologies are continuously increasing the number of observable cells and the range of brain regions that can be studied (3). However, two primary methods for monitoring neural activity—electrophysiological recording and optical imaging—typically address different spatial and temporal scales, hindering a comprehensive understanding of large-scale neural dynamics. Electrophysiological recording offers exceptional temporal resolution, capturing spiking activity from individual neurons within sub-millisecond intervals. Recent developments in high-density multi-channel silicon probes, with up to 1,024 recordable channels, have made it possible to investigate population-level spiking and circuit dynamics by simultaneously acquiring data from numerous diverse neurons (4, 5). Additionally, multi-array silicon probes with long shanks are specifically designed to reach and record from multiple, interconnected deep brain regions, thereby considering the effects of long-range inputs and neural projections (6). Despite these advancements, electrophysiological recording has inherent limitations. One significant challenge is the relatively low yield of individually identified neurons following spike sorting, a complex process essential for extracting single-unit activity from extracellular data. Another limitation is the difficulty of accurately identifying the anatomical location of recorded neurons based solely on their spatial position, complicating the correlation of electrical signals with specific cell types or brain structures.

Optical imaging stands out for its high spatial resolution, allowing researchers to visualize neuronal activity from up to 10,000 neurons at once using either a 1-photon (1P) large field-of-view (FOV) microscope or a 2-photon (2P) scanning microscope (7, 8). The use of genetically encoded fluorescent sensors, delivered through transgenic lines and viral vectors, enhances large-scale calcium imaging. This approach enables the examination of individual neuron activity, identification of specific neurons, and determination of their population characteristics (9, 10). In optical imaging, spatial expansion typically occurs volumetrically or laterally across the cortical mantle through rapid resonant scanning and large FOVs (11). However, a significant limitation is that the objective's focus cannot penetrate deeper into subcortical regions due to tissue scattering. The use of implanted GRIN (Gradient Refractive Index) lens endoscopy helps address this challenge by facilitating imaging in deeper brain areas, though only within a few millimeters. Additionally, optical imaging has a notable drawback: its temporal resolution is limited by the sampling frame rate of the imaging detector and the decay dynamics of the calcium indicators. Most electrophysiology studies have demonstrated the ability to record cellular spiking activity with sub-millisecond temporal resolution, but they inherently lack spatial precision. While existing optical tools offer fine spatial alignment, their readout—particularly from calcium fluorescence imaging—cannot fully capture the temporal resolution of single-neuron action potentials. The slow kinetics of calcium indicators, which serve as proxies for neural signals, hinder the observation of cellular activity at the sub-millisecond level needed for spiking (2, 7). Although voltage indicators can monitor the membrane potential of dozens of neurons more quickly, they are limited by fluorescence decay, restricting observable activity to several minutes (12, 13). Remarkably, integrating electrophysiology with optical imaging creates a powerful synergy that addresses the limitations of each method while enhancing their individual strengths. This integration allows simultaneous observation of multiple signals, providing critical insights to tackle long-standing questions in neurobiology and bridging various scales of analysis. However, the technical challenge of simultaneously recording optical signals and electrophysiological data from the same neural populations remains significant.

This review provides a comprehensive examination of the technical integration of neural methods for the simultaneous and complementary acquisition of multimodal brain signals in vivo. While most previous reviews have concentrated on multifunctional neural technologies that merge signal acquisition with optogenetic neuromodulation (14, 15), this is the first review dedicated solely to the state-of-the-art in multimodal observation, an emerging trend in neuroscience. The main emphasis is on the mechanical and optical coupling of imaging systems aimed at capturing signals from various fluorescent sensors with differing spatiotemporal resolutions. We then discuss the monolithic assembly of dual readouts on electrical devices, emphasizing the synchronization of diverse neurophysiological properties from multiple targets. Additionally, we explore a range of methods for integrating electrophysiology with neural imaging, which is vital for bridging different scales in neurobiology. The synergistic effects of these multi-scale neural dynamics offer a systematic framework for understanding functional connectivity, spanning from the cellular scale of local circuits to the broader scale of multiple brain regions. This innovative approach is crucial for elucidating the functions and physiological mechanisms of circuits that have long been debated in neurobiology. Finally, we outline future perspectives on the development of neural interfaces for multi-targeting at various scales, presenting unprecedented opportunities to investigate unresolved questions regarding the brain's mechanisms in neurobiology.

MULTIMODAL NEURAL IMAGING

Fluorescence sensors have become essential tools for investigating neural mechanisms during behavior, as they provide real-time, target-specific readouts of neural responses through fluorescence fluctuations. These “plug-and-play” sensors can monitor calcium, voltage, and various biomolecules. Recent advances in genetic engineering have enabled the development of fluorescent sensors that target biomolecular specificities previously inaccessible through electrochemical methods. Numerous fluorescence sensors have been created, with the potential to design sensors for virtually any ligand or peptide (16). The occurrence of an action potential typically leads to an influx of Ca^2＋ into neurons via voltage-gated calcium channels and other pathways. Genetically encoded calcium indicators (GECIs), such as GCaMP and jGCaMP sensors—derived from green fluorescent protein (GFP) and the Ca^2＋ binding protein calmodulin (CaM)—are widely used to report calcium dynamics (9, 17, 18). Although GECIs exhibit a relatively slow kinetic response of several seconds, calcium imaging remains a dominant method for observing neural activity (19). In contrast, genetically encoded voltage indicators (GEVIs), such as the ArcLightb (20), ASAP (21), Archons, and Ace2N families (22), contain a voltage-sensing domain (VSD) that modulates fluorescence in response to voltage changes. These voltage-induced conformational changes in the VSD can alter the intensity of the fluorescent signal or affect fluorescence resonance energy transfer (FRET) between the attached fluorescent protein and rhodopsin, enabling rapid imaging of sub-second single action potentials (23, 24). However, a primary disadvantage of GEVIs is their significantly lower fluorescence compared to GECIs, leading to a suboptimal signal-to-noise ratio (SNR). In recent years, there has been rapid progress in developing fluorescent sensors for extracellular ligands and intracellular signaling molecules that were previously challenging to access in vivo. For instance, bacterial periplasmic binding proteins (PBPs) and G-protein coupled receptors (GPCRs) have been engineered as scaffolds for genetically encoded neuromodulator indicators (GENIs), including the dLight (25) and GRAB families (26), which bind to neuromodulators such as dopamine, serotonin, and acetylcholine (16, 27). These indicators provide unprecedented opportunities for real-time monitoring of physiological levels of neuromodulators across various temporal and spatial scales using microscopy or photometry.

Neuroimaging is a technique that visualizes changes in the fluorescence of calcium, voltage, or neuromodulator sensors. Widefield epifluorescence imaging, utilizing macroscopes and COSMOS, is an emerging method for simultaneously mapping mosaic calcium activity across multiple cortical areas on a cortex-wide scale (28, 29). A macroscope achieves a cortical FOV of 13.4 × 11.3 mm with a 1.24 magnification factor through a cranial window, where the skull is either thinned or replaced with transparent materials to maintain optical clarity (30, 31). Two-photon (2P) imaging, which employs resonant-galvo-based scanning with 800 nm excitation light, can record calcium or voltammetric activity with single-cell resolution from tens of thousands of neurons (e.g., Quadroscope (32), Diesel2p (33), 2p-RAM (11), FASHIO-2PM (34), FHIRM-TPM 2.0 (35), MINI2P (36)). 2P microscopy generally maintains a high frame rate across a volumetric FOV up to 600 μm in depth from the cortical surface or a mesoscopic FOV of up to 5 × 5 mm (37, 38). Three-photon (3P) microscopy, utilizing 1,300 nm excitation, can penetrate deeper to image specific localized regions, such as the rodent hippocampus, at depths of up to 1.3 mm (39, 40). Additionally, photometry is an optical technique that measures fluorescence intensity fluctuations from an implanted optical fiber in the brain. It has been widely used to monitor population-level changes in fluorescence from specific cell types or local brain regions (41).

Using multiple fluorescent sensors, each marked with a distinct color, is crucial for understanding the spatiotemporal connectivity and causality of target-specific neural responses across various scales (9). However, simultaneously monitoring two or more fluorescent labels presents a significant challenge due to spectral overlaps, where different fluorescent emissions are detected at the same wavelength (42). Technical solutions for observing multiple labels typically involve either capturing serial images with the same FOV through a time-division multiplexing approach or simultaneously taking parallel images of two fluorescent sensors using multiple photodetectors.

Multi-targeting in a FOV

Photometry is an optical technique that simultaneously measures signals from multiple fluorescent sensors within the same brain region. Its core principle involves spectrally separating the fluorescent responses gathered through an implanted optical fiber (43, 44). To isolate the mixed activities of various sensors, photometry systems employ wavelength-specific dichroic mirrors, emission filter sets, and multiple photodetectors. In time-division photometry, an isosbestic fluorescence of 405 nm is used for background signals, while excitation fluorescence at 470 nm and 560 nm is delivered to the brain as alternating pulses at 90 Hz. Neural responses then pass through 494-531 nm and 586-627 nm emission filters, where they are detected by photodetectors. A background signal is subsequently subtracted to compute the relative changes in fluorescence intensity of individual targets. To measure calcium activity and endocannabinoid (eCB) signals in the lateral amygdala (LA) related to stress-induced threat memory overgeneralization, GRAB_eCB2.0 (45), RCaMP1a were used GRAB_eCB2.0, RCaMP1a as fluorescence sensors (Fig. 1A). The delayed pattern of fluorescence cross-correlation, reflecting the regulation of LA excitatory activity by a multi-target validated endogenous cannabinoid substance, has enhanced our understanding of the mechanisms behind stress-induced memory impairment and suggested potential therapeutic targets for stress-related disorders (46). Frequency modulation is another photometry approach used to mitigate spectral crosstalk. The emitted fluorescence signals contain distinct frequency components that correspond to the assigned frequencies of each excitation light source. This frequency-selective demodulation enhances signal specificity, even amidst overlapping emission spectra, enabling the simultaneous recording of multiple fluorescent sensors. In a study, acetylcholine and dopamine signals in the ventrolateral striatum (VLS) of mice were measured simultaneously using GRAB_Ach3.0 and GRAB_rDAh sensors, with excitation at 470 nm at 167 Hz and 565 nm at 223 Hz, respectively. A dichroic mirror separated the emitted fluorescence responses, which were collected by distinct photodetectors. This technique effectively revealed a pattern in which dopamine signals inhibited acetylcholine signals with a time difference of approximately 100 ms, demonstrating an anti-correlation between the two neurotransmitters. The findings support the hypothesis that dopamine regulates acetylcholine activity with a delay during reward-based decision-making (43). Recently, to tackle the critical issue of spectral overlaps in multi-target fluorescence, spectrometer-based photometry has been introduced, allowing for the spectral unmixing of multiple fluorescence signals (47, 48).

Fig. 1.

Optical imaging of multi-target, multi-scale neural signals. (A) The optical sensors GRAB_eCB2.0 (for eCB) and RCaMP1a (for calcium) are co-expressed in the lateral amygdala. Time-division photometry simultaneously measured calcium activity and endocannabinoid (eCB) signals in the lateral amygdala. (B) Dual-color widefield imaging independently observes the cortical dynamics of norepinephrine and calcium signals. CaMKIIα::NECa mice exhibits average fluorescence responses of GRAB_NE2m and jRGECO across the cerebral cortex before, during, and after exercise using dual-color widefiled imaging. (C) Multiscale imaging integrates widefield and two-photon (2P) microscopy. A glass microprism placed on a small craniotomy in an intact skull enables cellular calcium imaging with 2P excitation during simultaneous widefield imaging. Regional calcium activity in the entire cortical mantle is monitored by the widefield image, while calcium activity of pyramidal neurons in the cortex is simultaneously monitored via the prism with 2P microscopy.

Widefield imaging allows for simultaneous multi-signal acquisition across a large FOV in the cortex (49, 50). The dynamics of norepinephrine and calcium activity were visualized using the GRAB_NE2m and jRGECO1a sensors, which were excited by 488 nm and 561 nm light sources, respectively. To correct for hemodynamic changes in the cortex, alternating illumination with 405 nm isosbestic light was employed. A 567 nm long-pass dichroic mirror, along with 525 ± 36 nm and 609 ± 34 nm emission filters, was used to isolate signals from multiple targets (Fig. 1B). This dual-color mesoscopic imaging, utilizing two sCMOS cameras, enabled independent observation of cortical dynamics related to norepinephrine and calcium signals at a rate of 10 Hz with a resolution of 512 × 512 pixels during sensory processing and locomotion (51). Additionally, another widefield imaging approach demonstrated dual-color observation of ACh3.0 and jRCaMP1b sensors, highlighting the role of acetylcholine in connecting behavioral fluctuations to the functional reorganization of cortical activity (52).

Multimodal imaging with a 2P microscope is a powerful technique that captures neural signals at both cellular and subcellular resolutions within the same FOV. This method provides high spatial resolution and deeper tissue penetration while ensuring precise optical sectioning (53, 54). A dual-color 2P imaging system was employed to monitor membrane voltage and calcium activity in layer 2/3 pyramidal neurons. For dual-color imaging, researchers utilized the Kv-ArcLight-ST and jRGECO1a sensors, which were selectively excited by lasers at 940 nm and 990 nm, respectively. The emitted fluorescence signals were then separated into their respective spectra using a 575 nm cutoff long-pass dichroic mirror, followed by a 525 ± 70 nm emission filter for ArcLight and a 607 ± 45 nm emission filter for jRGECO1a. Images were acquired at a frequency of 30 Hz with a spatial resolution of 512 × 512 pixels. This system revealed distinct network-level patterns of subthreshold and spiking activity during the progression of epileptic seizures, demonstrating its potential for spatiotemporal mapping of input-output transformations in neural circuits under both physiological and pathophysiological conditions (55). In another application of 2P imaging, simultaneous monitoring of intracellular messengers and calcium dynamics offers insights into neuromodulation over extended timescales, linking rapid electrical activity with slower biochemical signaling. In one study, two-color 2P imaging of cAMP and calcium demonstrated that calcium-dependent cAMP responses encode specific information, such as direction selectivity during visual processing and locomotion. The emitted fluorescence was separated using a 570 nm long-pass dichroic mirror, along with 495-540 nm and 575-645 nm emission filters, and detected independently by two photomultiplier tubes (PMTs). Signals were acquired at a resolution of 512 × 512 pixels across three separate planes, each spaced 30 μm apart. Each plane was captured at a rate of 3.4 Hz using a piezo objective scanner. Approximately 400 neurons in layer 2/3 of the primary visual cortex (V1) were imaged during voluntary running sessions, a condition known to enhance norepinephrine, cAMP, and PKA activity in the cortex. This simultaneous imaging of calcium and cAMP signals in responsive neurons revealed that action potentials trigger cAMP transients through calcium-dependent adenylyl cyclase. Notably, the cAMP transients persisted significantly longer than the calcium transients, indicating that cAMP acts as a mechanism for storing and integrating information from both calcium signaling and GPCR activation (56).

Neural observation at multi-scale

The combination of 2P and widefield imaging enables the simultaneous measurement of micro-scale single-neuron activity and meso-scale activity across the entire cortex (38, 57). This objective is achieved through a novel microscope design featuring an orthogonal axis configuration. In this setup, a mesoscopic objective is positioned above the cranial window to capture activity across the entire cortex, while a 2P objective is oriented horizontally to image a smaller area using a microprism. In a specific dual-imaging microscope, widefield imaging was employed to simultaneously monitor cortical calcium activity in a transgenic mouse expressing GCaMP6f in pyramidal neurons. A 395 nm LED was used to capture calcium-independent signals, while a 470 nm LED captured calcium-dependent signals, alternating between the two to correct for background and hemodynamic changes. Both excitation wavelengths passed through a 495 nm long-pass dichroic mirror to illuminate the cortex. The emitted GCaMP6f fluorescence was then reflected by the same mirror and filtered through a 525 ± 50 nm emission filter. This widefield calcium imaging achieved a resolution of 512 × 500 pixels with 4 × 4 binning, allowing visualization of large FOV cortical activity. Simultaneously, micro-scale imaging was conducted using a 920 nm laser to capture cellular activity in layers 2/3 of the somatosensory cortex (S1). The emitted fluorescence was divided into two spectral channels by a 565 nm dichroic mirror, followed by 525 ± 50 nm and 605 ± 70 nm emission filters, which were detected by two separate photomultiplier tubes (PMTs). This filter combination prevented contamination between the images produced by the 2P and widefield systems. The 2P system scanned approximately 50 neurons for Ca^2＋ imaging across a 200 × 200 μm FOV in layers 2/3 of S1. To avoid optical crosstalk, the widefield and 2P frames were interleaved with a short offset of 34-67 ms. This delay was carefully selected to be shorter than the decay time of the calcium indicators, thus preserving single-spike resolution. Consequently, this dual-imaging system provides a novel approach for investigating cortical networks at multiple spatial scales in situ (Fig. 1C) (38).

The integration of widefield imaging and photometry has significantly enhanced our understanding of the interactions between cortical and subcortical circuit activities. In one study, researchers used a 475 nm LED to observe regional fluorescence of calcium activity across the cortex of Thy1-GCaMP6f mice. They captured widefield images at a rate of 20 Hz with a sCMOS camera, employing a 535 ± 30 nm emission filter, at a resolution of 256 × 256 pixels. To monitor subcortical activity simultaneously, optical fibers were diagonally inserted into the CA1 region of the hippocampus, positioned carefully to avoid disrupting the widefield imaging field of view. Photometry was performed using frequency-modulated excitation, with 405 nm serving as an isosbestic wavelength for motion artifact correction and 465 nm for GCaMP6f excitation. This multi-scale optical approach revealed that, following unilateral hippocampal injury, contralateral spreading depolarization (SD) was not propagated through the cortex; rather, it was triggered by pathological network activity intrinsic to the hippocampus (58). Another approach for integrating widefield imaging and photometry involves capturing widefield images of the cortical surface while also accessing striatal networks. In this configuration, mesoscale cortical imaging of GCaMP6s was performed at 40 Hz with a 25 ms exposure time using a 470 nm LED and a 12-bit CCD camera. The images were processed with 8 × 8 binning to improve signal detection. Concurrently, photometry signals from axon terminals in the striatum were recorded using frequency-modulated excitation at 405 nm for isosbestic control and 465 nm for GCaMP6s excitation. This multi-scale observation confirmed a strong correlation between cortical and striatal activity during sensory stimulation and movement, revealing that specific cortical activity patterns influence presynaptic activity in the striatum (59).

The combined use of six-plane 2P and 3P microscopy has facilitated the simultaneous imaging of activity in both superficial and deep cortical neurons that express GCaMP6s. A system called DEEPscope employs dual excitation through an adaptive excitation polygon-scanning multiphoton microscope. This device uses 1,320 nm excitation for 3P imaging and 920 nm excitation for 2P imaging, allowing it to record neuronal activity from the same cortical region within a 3.23 mm × 3.23 mm field of view. To achieve this, the two excitation beams are temporally interleaved and combined into a single optical path using a 980 nm long-pass dichroic mirror, enabling them to share the same scanning pathway. A remote focusing module adjusts the optical path length for each beam, allowing for imaging of neuronal activity at varying cortical depths within a single frame. Using this technique, the polygon-scanning DEEPscope successfully captured calcium activity from 4,183 neurons at depths of 320 to 400 μm with 2P imaging and from 340 neurons at a depth of 600 μm with 3P imaging. Additionally, the DEEPscope simultaneously observed dendritic and cellular-level activity from 503 neurons across extensive cortical regions with a high resolution of 0.67 μm per pixel (60).

Multimodal neural imaging combines various fluorescent sensors and optical techniques to observe neural activity across different spatiotemporal scales in living animals. By employing fluorescence sensors like GECIs, GEVIs, and GENIs, these techniques can simultaneously track calcium, voltage, and neuromodulator dynamics. Imaging methods such as widefield, 2P, and 3P microscopy are integrated with photometry to capture neural activity, ranging from single cells to entire brain regions at multiple scales. This comprehensive approach is essential for uncovering complex functional connections and providing a fuller understanding of brain function.

MULTIMODAL PROBES WITH DUAL READOUT MODULES

Extracellular electrophysiology is a powerful tool for measuring the electrical patterns of neural activity, especially action potentials, with sub-millisecond resolution (6). Multi-channel silicon probes, such as UCLA probes (6) and Neuropixels (61) can contain thousands of electrodes, each measuring just 10 micrometers square. These probes can be inserted or implanted into deep brain regions of behaving animals, significantly enhancing our understanding of neurobiology by monitoring synchronous, large-scale spiking activity from intact neuronal ensembles. Recent advancements in microfabrication technology have led to the creation of more densely packed, miniaturized, and interconnected probes, facilitating the observation of neural populations at fine spatiotemporal scales (62). To fully understand intercellular communication, it is essential to observe chemical signals. Chemical neuromodulators released from one neuron can influence the population dynamics of others. These physiological messengers play a critical role in the pathogenesis and treatment of neurological and psychiatric disorders (63). A key unresolved question in neurobiology is how the timing and causality of neuromodulator release relate to the activity of individual neurons, local circuits, and larger brain regions. Traditional methods for monitoring neuromodulators, such as fast-scan cyclic voltammetry (FSCV) with carbon fibers (64) and microdialysis (65), have significant limitations. FSCV offers real-time detection but typically needs post-hoc calibration, whereas microdialysis can only assess chemical distribution, complicating the capture of precise spatiotemporal dynamics. Both methods are not well-suited for integration with electrophysiological recordings in behaving animals. However, recent advancements in electrochemical methods have made it possible to simultaneously monitor dual dynamics, effectively combining high-spatiotemporal-resolution electrical spiking activity with chemical neuromodulator detection (66).

Electrochemical modification of electrode

The detection of neurotransmitters and neuromodulators has been pursued using analytical methods such as enzyme coupling and organic semiconductors. Recently, researchers have focused on improving detection sensitivity by modifying electrode surfaces with nanocomposites, including platinum nanoparticles (67), graphene (68), and conductive polymers (69, 70). These modifications are essential for creating multifunctional probes with dual-readout capabilities. The 16-channel electrodes were enhanced with platinum nanoparticles (PtNPs) and conductive polymer (poly (3, 4-ethylenedioxythiophene):poly (4-styrenesulfonate)), enabling simultaneous electrophysiology and dopamine detection (Fig. 2A). The modified electrodes with PtNPs/PEDOT:PSS exhibited a low impedance of 18.26 ± 3.76 kΩ at 1 kHz, and the dopamine sensitivity per unit area was 0.23 ± 0.013  pA/μM/μμ² at a 0.35 V oxidation potential. A study utilizing this dual-mode electrode revealed that both spiking activity and dopamine concentration in the striatum decreased synchronously following a fear-inducing condition (70). Another probe features electrodes modified with a nanocomposite of reduced graphene oxide (rGO) and PEDOT:PSS, enhancing electrical stability and dopamine sensitivity, with a remarkable sensitivity of 15 pA/μM. The probe includes 32 channels, with each electrode measuring 15 × 25 μm, and is designed to accommodate 128 readouts across four shafts. In a specific configuration, 118 channels were used for acquiring electrophysiological signals, while 10 channels were designated for electrochemical sensing. After the injection of nomifensine, a dopamine reuptake inhibitor, the probe successfully recorded an increase in neuronal firing in the motor and deeper cortical layers, accompanied by a gradual rise in dopamine levels to 192 ± 29 nM. This multimodal probe demonstrated stable, high-sensitivity neural monitoring for up to six weeks (71).

Fig. 2.

Multimodal probes for electrical and chemical detection. (A) A flexible electrode with dual readout modules consists of a 128-channel array distributed across four shanks, containing 118 electrophysiological recording electrodes and 10 dopamine detection sites. (B) This dual-mode probe uses enzymatic modification on an electrode. It consists of sixteen electrodes distributed across four shanks, which are 7 and 9 mm in length and have electrode diameters of 12 and 17 μm. The larger electrodes were modified with nanomaterial-enzyme compounds.

Enzymatic modification of electrode

The immobilization of molecules, particularly enzymes, on electrode surfaces enables the selective detection of specific neurochemicals. Enzymes function as bioreceptors, facilitating the detection of non-electroactive neuromodulators (72, 73).

The electrode array architecture was designed with four shanks, measuring 7 mm and 9 mm in length. A 2 mm gap between the shanks allowed for the simultaneous measurement of neural activity in both the cortex and hippocampus of rats. The tips of the probes were equipped with electrodes that had diameters of 12 μm and 17 μm. The larger electrodes were modified directionally with a nanocomposite of platinum nanoparticles (PtNPs) and reduced graphene oxide (rGO), followed by an enzymatic layer of glutamate oxidase (GluOx) and m-phenylenediamine (mPD) (Fig. 2B). Glutamate was monitored at a voltage of 0.1 V, with a sensitivity of 141.00 ± 5.66 nA μM^–1 mm^–2 and a limit of detection of 0.3 μM. The dual-mode microelectrode simultaneously measured glutamate concentrations from six channels and electrophysiological activity from 21 channels in the cortex and hippocampus of epileptic rats under RuBi-GABA perturbation. The results demonstrated that the increase in glutamate concentration occurred before the surge in electrophysiological signals, and that these changes in the hippocampus preceded those in the cerebral cortex. This finding suggests that alterations in glutamate levels within the hippocampus may serve as a crucial early warning indicator for epilepsy (74).

Recent advancements in multifunctional electrodes have aimed at improving selectivity and signal amplification. This progress has resulted in the creation of flexible neural probes equipped with organic electrochemical transistor (OECT)-based amplifiers, which offer stable detection of amplified signals while reducing noise. The use of differential sensing—where one of two electrodes serves as a reference—shows significant potential for selectively detecting brain metabolites like lactate and glucose. This method allows for the simultaneous measurement of both electrical and chemical signals with enhanced precision (75).

Thus, multimodal probes have become a powerful tool in neurobiology, allowing for the simultaneous measurement of electrical and chemical neural signals. By addressing the limitations of traditional single-readout methods, these probes offer a more comprehensive understanding of intercellular communication. Their effectiveness stems from advanced techniques used to modify electrode surfaces, such as electrochemical methods with nanocomposites or enzymatic modifications. These innovations enable the selective and sensitive detection of various neurochemicals alongside neural spiking activity, offering critical insights into the complex interplay between electrical and chemical signaling in the brain.

MULTIMODAL INTERFACE FOR SIMULTANEOUS ELECTRICAL ACTIVITY AND OPTICAL IMAGING

The study of neural dynamics involves examining functional connectivity at both the local circuit level and across multiple brain regions. Cortical neurons typically receive and transmit inputs from nearby and distant sources simultaneously. The complexity of the corticocortical network makes it state-dependent, requiring analysis of both the spiking activity of individual neurons and the patterns of neuronal populations in different regions at the same time (76, 77). Combining electrophysiological recording with optical imaging promises high spatiotemporal resolution across all scales of neural activity. However, the physical implementation of these techniques together presents significant challenges, especially outside the brain (63, 78). Electrophysiological recording requires adequate workspace for the coordinated insertion of silicon probes with micromanipulators (79). Similarly, high-resolution optical imaging requires a working distance of several millimeters between the microscope objective lens and the sample, which can disrupt other engineering collaborations (80). Therefore, a spatial compromise is essential for successfully integrating these two mechanical systems.

Integration of electrophysiology and widefield imaging

Widefield fluorescence imaging, utilizing a long working distance objective lens and a large FOV, facilitates a unique experimental approach, particularly when combined with transcranially inserted probes. By angling the probe's insertion, the workspace is effectively separated from the optical imaging area. In one study, widefield calcium activity was captured from the GCaMP6s-expressing dorsal cortex with an 8 × 8 mm FOV at a rate of 40 Hz. Concurrently, a 64-channel silicon probe was inserted into the V1 and retrosplenial (RSP) cortices at a 45° angle, recording the spiking activity of 7 to 55 single units per session. Affiliation maps were subsequently created by correlating the spike timing of individual neurons with calcium signals across the entire cortical surface. This multi-scale analysis revealed that neurons within the same cortical area exhibited distinct spatial correlation patterns, suggesting functional differentiation across large-scale networks (81).

A more protracted probe, such as Neuropixels, allows access to deeper brain regions within the subcortex. In a separate study, cortical calcium imaging was conducted on transgenic mice expressing GCaMP6s at 35 Hz using an sCMOS camera. Simultaneously, Neuropixels probes were inserted into the striatum at a 45° angle to a depth of 6 mm to record electrical activity during a visually guided behavior. To reduce interference from LED illumination, a common average referencing technique was applied within the action potential frequency band. This integrated system demonstrated that three segmented striatal domains are topographically connected to cortical regions, forming a continuous and cooperative long-range circuit during sensory and motor processing (Fig. 3A) (82). In another study, widefield calcium imaging recorded cortical GCaMP6 signals at 40 Hz, while Neuropixels probes were inserted at a 45° angle into the cerebellum to capture single-unit activity. Among the 6,699 recorded cerebellar neurons, 1,976 exhibited stable cortical connectivity in both anesthetized and awake conditions. This multimodal system demonstrated that individual cerebellar neurons connect with various cortical regions, displaying dynamic, state-dependent reorganization (83).

Fig. 3.

Multimodal observation of electrical activity and optical imaging. (A) The angular insertion of a silicon probe into an intact craniotomy allowed for widefield imaging while simultaneously acquiring electrical activity. This multimodal observation captured regional activity patterns of the cortical mantle and large-scale neural activity of individual neurons in the striatum. (B) The optical transparency of the electrode array facilitated the integration of electrophysiology and widefield calcium imaging without optical obstruction or shadows in the intact cortex. A flexible, transparent, 64-channel Neuro-FIT electrode recorded spiking activity in the CA1 region of the hippocampus while simultaneously performing widefield imaging of the cerebral cortex. (C) A multimodal device that integrates electrodes and optics features dual readout modules for diverse physiological samples, enhancing our understanding of functional brain connectivity. Opto-microprobes, which consist of a 256-channel electrode array and a low-autofluorescence fiber optic, demonstrated simultaneous electrophysiological recordings of spiking activity and photometric detection of dopamine sensors.

The E-Scope is an integrated onboard ultra-compact microscope combined with a silicon probe, offering a solution for simultaneous calcium imaging and electrophysiology in freely moving mice. This multimodal approach measures extracellular electrical activity in the cerebellum alongside calcium imaging in the anterior cingulate cortex (ACC). It has been used to uncover long-range correlations between these two brain regions during social interactions (84).

A significant challenge in integrating optical imaging with electrophysiology is that opaque silicon probes obstruct the FOV. To address this issue, flexible and transparent probes like the neuro-FITM have been developed using Parylene-C. These probes ensure a clear FOV and eliminate optical shadows. The neuro-FITM features 64 channels and has a low insertion force of just 1 mN, allowing for successful insertion into the hippocampal CA1 region without buckling. This system enables simultaneous monitoring of cortical activity through widefield imaging while recording spiking activity from the hippocampus (Fig. 3B). It has shown that sharp-wave ripples (SWRs) selectively activate cortical activity, highlighting functional differences among hippocampal neural subpopulations during cognitive processing (85). Recently, transparent electrode materials have become essential for the simultaneous integration of electronics and optics. An ethylene glycol (FPE)-treated PEDOT:PSS electrode exhibited a low electrochemical impedance of 45.8 kΩ on a 20 × 20 μμ² electrode and it maintained transparency in a 100 μm z-stack, showing no obstruction under 2P microscope (86). And bilayer-nanomesh microelectrode arrays, with an impedance of 130 kΩ, were developed by reliably stacking a metal layer within a transparent nanomesh over a low-impedance faradaic layer. This transparent, 32-channel microelectrode enabled two-photon imaging of individual neurons in layers 2/3 of the visual cortex in vivo, alongside high-fidelity electrophysiological recordings of visual-evoked activity (87).

Integration of electrophysiology and photometry

While photometry is a well-known method for monitoring population-level fluorescence from calcium activity, the precise correlation between photometric calcium and electrophysiological activity remains unclear. To address this, the opto-microprobe was developed, combining silicon probes with optical fibers to enable both electrophysiology for extracellular spiking activity and photometry for population-level calcium changes. This configuration allows for the simultaneous collection of substantial neural spiking activity and fluorescence responses from the local secondary motor cortex (M2) region during spontaneous operant licking behavior. The dual data sets—spiking activity from 204 neurons and changes in fluorescence—revealed a negative correlation between the physical distance of individual neurons and their spiking activity relative to the slope of fluorescence (88). In a separate study, the opto-microprobe was employed to simultaneously investigate dopamine dynamics using the dLight1.2 sensor and local spiking activity in the ventral striatum. During the optogenetic manipulation of dopaminergic neurons in the ventral tegmental area (VTA), naturally occurring reward-level dopamine had a minimal effect on striatal firing. In contrast, artificially elevated dopamine levels induced a robust electrophysiological response on fast timescales. This work highlights that multimodal techniques, such as the opto-microprobe, are promising for precisely quantifying the physiological correlation between neuromodulators and neural activity (Fig. 3C) (89).

Monolithic manufacturing, which integrates electrodes and optics into a single probe, helps reduce heat generation and electrical interference. One example is a multimodal opto-electric silicon probe that incorporates transparent gold nanogrid microelectrodes. This probe features a 462 nm micro-LED, a micro-photodetector, and a filter, all stacked on a 40 μm thick PET film. The gold nanogrid electrodes achieve high optical transparency of up to 81% and a low impedance of 6.3 Ω·cm² at 1 kHz, enabling both clear imaging and high-fidelity electrical recording. The micro-photodetectors exhibit high sensitivity to GCaMP fluorescence under LED illumination and a linear response to varying intensities (90). Another multimodal neural probe monolithically integrates photodiodes, 32-channel electrodes, and LEDs. A key feature of this design is the photodiode-electrode array pairs on each shank, which allow for the measurement of co-localized multimodal signals in the hippocampal CA1 region and enable precise matching of fluorescence signals with spiking activity (91).

Multimodal interfaces that combine electrophysiology with optical imaging represent a significant advancement in understanding neural dynamics across various scales, from single neurons to large brain regions. To overcome the physical and technical challenges of integrating these systems, researchers have developed innovative solutions. These solutions include strategically angling probes to avoid obstructing the optical FOV and creating specialized transparent or miniaturized probes. By merging electrical recording with diverse optical methods, these systems provide a more comprehensive understanding of how electrical activity relates to population-level neural signals, ultimately offering deeper insights into the complex relationships of neural signaling across different spatiotemporal scales.

OUTLOOK

This review introduces multimodal observations that facilitate the complementary collaboration of spatiotemporal resolution by synchronizing two heterogeneous brain signals. When configuring devices within a multimodal interface, it is essential to establish a trade-off between optical access for focus adjustment and physical access for probe insertion. This review discusses several strategies for instrumental coupling, including optical filtering for dual imaging (38, 52, 57), angle-adjustable probe insertion (83-85), and device transparency (86, 87). Furthermore, the development of multimodal observation techniques in neuroscience has primarily focused on reducing crosstalk and enhancing the signal-to-noise ratio (SNR) in dual readout modules. The advent of novel materials has enabled the acquisition of high-fidelity and high-resolution neural signals.

A significant challenge in optical imaging arises from the light sources used to initiate fluorescence fluctuations, which can generate photoelectric artifacts that interfere with electronic recordings. In simultaneous multimodal observation, these artifacts contaminate electrophysiological output, especially when the optical input irradiates the metal electrodes. The core issue is that this noise can appear as high-amplitude, short-duration spike-like signals during electrophysiological recordings (92).

Consequently, a critical challenge is to develop materials that can be deposited on electrodes to enhance signal quality without introducing photodetection artifacts. In this context, we primarily introduced PEDOT:PSS as a coating material for multimodal electrodes (74, 86, 93). PEDOT:PSS has gained popularity as an alternative to metals due to its effectiveness in preventing photoelectric artifacts. The conductivity of this polymer arises from the presence of conjugated double bonds along its insulating backbone. However, PEDOT:PSS displays suboptimal electroactive stability and mechanical properties, which limit its application in chronic brain implantation (92). An alternative approach involves heavily doping silicon substrates with boron to reduce photoelectric noise in metal electrodes, successfully lowering the artifact to below 50 μVpp, which is below the typical spike detection threshold (94).

Multimodal imaging using multicolor fluorescence is crucial for various neurobiology research areas. In particular, identifying neuron types and neural interactions at cellular or even subcellular resolution enhances our understanding of the architectural and spatial relationships within local neural circuits and across long-distance networks in the brain. While optical integration offers the potential for technical multimodal imaging in situ, the spectrally overlapping fluorescence from multiple targets complicates the ability to distinguish true signals from false positives due to bleed-through. Therefore, effective methodologies are needed to accurately separate the genuine signals from each fluorescence sensor. The detected fluorescence signal from each wavelength channel is demodulated to regenerate unmixed data through the linear correction of a mixing matrix. However, linear unmixing requires a reference spectrum to quantify the unique signals of each fluorescence sensor and involves additional time for analysis following signal acquisition (29, 48). Learning Unsupervised Means of Spectra (LUMoS) employs an algorithm for blind unmixing without a reference spectrum. It utilizes an unsupervised machine learning clustering method to identify inherent spectral signatures of individual fluorescence from mixed images and separates channels blindly, without restrictions on the number of fluorescence sensors (95).

Another non-reference-based unmixing technique, the Process of Ultra-Multiplexed Imaging of Biomolecules via the Unmixing of the Signals of Spectrally Overlapping Fluorophores (PICASSO), operates by iteratively minimizing the mutual information between mixed images (96). Using a demixing fluorescence approach, thin and short multimode fibers (MMFs) coupled to a miniscope captured images from multiple fluorescent sources. The non-negative matrix factorization algorithm then extracted spatio-temporal fluorescence signals directly from the raw video data by analyzing the scattering fingerprints of individual time traces (97).

Calcium imaging using GECI sensors, which serve as proxies for neural activity, displays slow kinetics and dynamics over several hundred milliseconds. This limitation prevents direct observation of cellular or subcellular spiking activity. However, recent advancements in dual imaging of voltage and calcium indicators have demonstrated a spatial correlation between electrical and calcium activity at the subcellular level within milliseconds. Combining 2P imaging of voltage and calcium, researchers were able to simultaneously detect the activity of the voltage-sensitive dye ANNINE-6plus and GCaMP6f from the distal spiny dendrites of Purkinje neurons in awake mice (98). The results showed that the number, timing, and spatial distribution of dendritic voltage and their calcium correlations were highly heterogeneous both within and between complex spikes. Additionally, the multimodal readout of GECIs and GEVIs, such as jRGECO1 and QuasAr, provides deeper insight into the mechanisms of disorders centered around calcium channels, including schizophrenia, epilepsy, and autism (99). However, GEVIs have the drawback of a short lifetime, lasting only several seconds to minutes due to photobleaching. Consequently, new GEVIs, such as JEDI-2P (100) and ASAP4e (101) have demonstrated advancements in genetic and technical improvements for the photostability of optical sensors.

To extend the limits of existing multimodal observation ranges and better understand the complex interactions between neurons and neuromodulators, it is essential to develop electrical or optical sensors specialized for new targets and spectral fluorescence. Calcium sensors that operate in the near-infrared (NIR) and mid-infrared (MIR) spectrum, with a wavelength range of 650 nm or longer (e.g., FR-GECO (102), WHaloCaMP (103), iGECI (104), jYCaMP (105)) and dopamine sensors (HaloDA 1.0 (106)) have been shown to be compatible with multicolor imaging of three targets in the brains of awake mice when used alongside conventional green and red sensors. In addition, aptamer- (107) and graphene-based (108) electrochemical electrodes offer an alternative to optical imaging of GENIs, allowing for greater compatibility with neural electrodes.

CONCLUSION

Multimodal neural technologies enable the simultaneous identification of various neural dynamics across different spatiotemporal scales in the brains of awake animals. This capability is crucial in several areas of neurobiology. First, multimodal observation of neurophysiological responses and signals from multiple targets or brain regions can help answer key questions about the causal functional roles of neural circuits and their input-output connectivity. Second, by facilitating multi-scale observation, these technologies allow us to appreciate both the overarching structure and the intricate details of the brain's architecture, enhancing our understanding of fundamental neurobiological principles. These combined effects enable the experimental testing of longstanding neuroscientific hypotheses that were previously hindered by technical limitations. Consequently, multimodal observation offers valuable insights into current challenges in neurobiology and introduces new strategies for uncovering the causes of neurological diseases.