Probing neural codes with two-photon holographic optogenetics

Abstract

Optogenetics ushered in a revolution in how neuroscientists interrogate brain function. Due to technical limitations, the majority of optogenetic studies have employed low spatial resolution activation schemes that limit the types of perturbations that could be made. Yet, neural activity manipulations at finer spatial scales are likely to be important to more fully understand neural computation. Spatially precise multiphoton holographic optogenetics promises to address this challenge and opens up many new classes of experiments that were not previously possible. More specifically, by offering the ability to recreate extremely specific neural activity patterns in both space and time in functionally defined ensembles of neurons, multiphoton holographic optogenetics could allow neuroscientists to reveal fundamental aspects of the neural codes for sensation, cognition, and behavior that have been beyond reach. This review summarizes recent advances in multiphoton holographic optogenetics that substantially expand its capabilities, highlights outstanding technical challenges, and provides an overview of the classes of experiments it can execute in order to test and validate key theoretical models of brain function. Multiphoton holographic optogenetics could significantly accelerate the pace of neuroscience discovery by helping to close the loop between experimental and theoretical neuroscience, leading to fundamental new insights into nervous system function and disorder.

Introduction

Causal perturbations in neuroscience are indispensable for understanding how neural circuits encode information and drive behavior. Optogenetics has revolutionized how neuroscientists causally test models of neural computation and function by offering cell-type precision coupled with rapid and reversible neural control. Although one-photon wide-field optogenetics offers immense possibilities to experimental neuroscience, the inability to recreate precise, user-defined spatiotemporal patterns of activity in highly scattering brain tissue makes it challenging to address the specific features of neural dynamics that drive computation and behavior. For example, theorists have long debated the relative importance of spike rate, spike time, neural synchronization, and neural ensemble structure for key aspects of sensation, cognition and actions(1)(2)(3). Despite 15 years of optogenetics experiments, such fundamental questions remain mostly unanswered.

Patterned illumination offers experimentalists the ability to recreate extremely specific, quasi-physiological patterns of activity in the intact brain with optogenetics(4). Patterned illumination with one-photon excitation (5)(6)(4)(7)(8)(9)(10) is highly advantageous: the required hardware is relatively simple to use and cost-effective, one can readily co-stimulate large numbers of neurons, and brain heating can be minimal. However, scattering by brain tissue severely compromises the effective resolution of optogenetic excitation with visible light. Combining sparse expression of the optogenetic protein and one-photon patterned illumination can activate defined ensembles of neurons, yet multiphoton optogenetics provides optical control of neurons with high spatial and temporal precision in scattering tissue and with relatively dense expression of the opsin protein (11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26). Patterned multiphoton optogenetics, such as with computer generated holography (Figure 1), provides control over ensembles of neurons. Coupled with simultaneous two-photon functional imaging, neuroscientists can thereby both ‘read’ and ‘write’ neural activity patterns with remarkable precision.

Figure 1: Two-photon holographic optogenetics.

a) Left: schematic of multi-neuron photostimulation with holographic optogenetics. Right: Schematic of the main modes of multiphoton optogenetic stimulation; instantaneous illumination of the entire soma with a low numerical aperture (NA) light spot the size of the soma (top), instantaneous illumination of the entire soma with a temporally-focused light spot the size of the soma with improved axial resolution (middle), scanning a near-diffraction limited spot in a spiral pattern on the soma of the target neuron (bottom). b) Principles of computer-generated holography (CGH). A Fourier-transform based iterative algorithm calculates the phase hologram to be loaded on to the spatial light modulator (SLM) in order to generate at the objective focal plane a user-defined 2D or 3D light pattern (54). (c) Optical implementation of spiral scanning CGH (14)(21)(11). The spatial light modulator (SLM) is optically conjugated to a pair of galvanometer scanners (XY) which are in turn conjugated to the objective pupil plane. The SLM generates a 3D distribution of spots which are then simultaneously scanned in a spiral scanning fashion. Note that spiral-scanning does not require temporal-focusing and no diffraction grating is used. d) Optical implementation of scanless 2D-CGH with temporal focusing (TF) (25). The SLM generates a 2D illumination pattern onto a diffraction grating (G). The 2D pattern is then imaged onto the objective focal plane. The femtosecond laser pulses are only spectrally recombined in the objective focal plane which generates an axial optical sectioning mechanism known as temporal focusing: 2P absorption only occurs at the focal plane since peak intensities above and below the focal plane are too weak to generate significant 2P absorption (37)(38) . This implementation is restricted to 2D since only a 2D SLM-generated pattern can be imaged on the 2D grating plane. (e-f) Examples of optical implementations of scanless 3D-CGH with temporal focusing. In 3D-SHOT (e) (13), a laser spot of pre-defined size hits the grating (G). The spot is temporally focused on a rotating diffuser and then subsequently replicated in 3D space in the Fourier plane of a spatial light modulator conjugated to the objective pupil plane. The rotating diffuser is used to both extend the beam in the orthogonal dimension and remove the speckle from the 3D distributed discs of light. The MT-CGH (f) (57) approaches generalizes the 3D-SHOT technique and provides additional flexibility by using an additional SLM to generate a custom shape or size light pattern on the diffraction grating (G). This custom light pattern is then replicated in 3D space with the second SLM. Dashed magenta lines indicate the imaging plane.

This review summarizes the state of the art in multiphoton optogenetics, highlights remaining technical challenges to be overcome, and describes how neuroscientists can use this emerging technology to address neural codes for sensation, cognition and action. For a more in-depth technical review see, for example, references (4)(27)(28)(29). Optogenetics via visible illumination has also been reviewed elsewhere(4)(30)(31); the focus here will be on two-photon optogenetics.

Technical background on multiphoton optogenetics and recent advances

Controlling neural activity with light offers high temporal versatility and is readily targeted to genetic subtypes. However, to control many single neurons within a subtype, one must spatially restrict the light to only the chosen neurons (or restrict opsin expression to only those neurons) (32). Achieving this involves two key ingredients: the ability to focus light onto single neurons in scattering tissue, and the ability to illuminate many neurons simultaneously or nearly simultaneously (e.g., within a few milliseconds of each other). Multiphoton excitation addresses the first challenge, and holography addresses the second.

Multiphoton optogenetics:

Scanning a near diffraction-limited laser spot capable of two-photon excitation over the soma of a neuron with opsin expression, for instance ChR2 (a microbial light-activated channel derived from algae that is sensitive to two-photon excitation), can be used to drive neuronal spiking (Figure 1a) (33) (34). This ‘spiral scanning’ approach is straightforward and can be advantageous, yet, depending on the kinetics and potency of the opsin used and its expression level, can in some cases require continuous scanning of the targeted neurons for milliseconds to drive action potentials (several milliseconds for some opsins (35), but <1 millisecond for highly potent opsins (11)). Alternative optical approaches have refocused the laser beam into a spot of a size on the order of the neuronal soma to obviate the need for scanning(16)(36)(12)(24)(17). These ‘single shot’ schemes simultaneously activate many opsin molecules on the soma and proximal dendrites (Figure 1a,d–f). Focusing a gaussian femtosecond laser beam (typically used for two-photon excitation) to a large lateral spot inherently degrades axial resolution. To address this, investigators have used ‘temporal focusing’, which reinstates high axial precision by deliberately spreading the femtosecond laser pulses in time with a diffraction grating, such that they only reach peak pulse power at a specific focal plane (25)(37)(38)(29). Importantly, temporally-focused holographic beams have been shown to be relatively robust to scattering, with preserved axial sectioning across a few scattering lengths in brain tissue (39)(12), Spiral scanning also requires lower energies (21), so it is able to activate more neurons with a limited laser budget, at the potential relative expense of temporal control. Conversely, temporally-focused excitation, with its soma-sized spots, requires higher instantaneous energies and thus activates fewer neurons for the same average power (40), but affords simultaneous excitation of many more opsin molecules. High speed temporal control with low jitter (<1 msec) has been achieved with temporally-focused excitation(41)(13)(42), and with spiral scanning using sub-msec spirals and potent opsins (11). Since jitter has typically been measured in brain slices, it remains unclear how well this translates to the in vivo condition, and whether temporal precision depends on the desired induced spike rate. As shown originally for the fast opsin ChETA, fast off rate kinetics of opsins can be important for driving precisely timed spike trains(43). The effective spatial resolution of spiral scanning or temporally focused single shot schemes, as so far reported in several studies, are comparable (5-10 μm lateral, ~20-40 μm axial)(15)(12)(16)(44)(13).

Multiphoton holography:

Simultaneous activation of multiple single neurons requires the generation of multiple two-photon excitation foci at the same time, which can be done with holographic technology (Figure 1). Computer generated holography (CGH) enables the dynamic shaping of optical wavefronts to generate 3D objects in image space(45)(46). In the most common versions of holographic photo-activation, a computer-generated phase mask displayed on a spatial light modulator (SLM) spatially shapes a coherent optical wavefront into a user-defined 2D or 3D pattern of spots (Figure 1b)(47)(48)(49)(17)(25)(50), used to activate specific sets of neurons either optogenetically or through uncaging of glutamate (31)(12)(22)(21)(20)(51)(11)(14)(32)(15)(52)(53). The optical phase mask to be displayed on the SLM for a given pattern at the objective focus is usually computed using the iterative Fourier transform-based Gerchberg Saxton algorithm (54), and new algorithms are being developed to improve computation speed and/or optimize intensity distribution (55)(56). The generated illumination spots can either be near diffraction-limited spots that are then all spiral scanned on the target neurons, or they can be soma-sized spots without scanning (Figure 1a). Of note, the initial implementations of CGH with temporally-focused soma-sized spots were restricted to a 2D spatial distribution since the generated holograms had to be imaged onto the diffraction grating plane (Fig 1d). Recently, optical advances solved this problem and now allow temporal focusing of different spots at different focal planes, providing full three-dimensional temporally focused optogenetics (Figures 1e–f)(12) (17)(57)(58). Finally, while this review focuses on computer-generated holography, one can also achieve custom light shaping with generalized phase contrast (GPC), an interferometric technique which does not require an iterative computation of the phase mask (59)(16)(60). The combination of GPC with holography has recently been shown to allow for 3D GPC and greater flexibility of spot shapes (61)(57).

Recent engineering advances afford high pixel counts SLMs with high refresh rates (300–600 Hz) (11)(62)(63) that provide optical control over nearly the entire field of view in conventional two-photon microscopes, and spatial and temporal multiplexing with multiple SLMs can afford even higher pattern rates (11). Concurrent advances in high pulse energy fiber lasers (up to 60–100 μJ/pulse) permit the simultaneous activation (i.e., simultaneous rather than fast sequential illumination) of tens to hundreds of neurons. With these tools, holography can allow experimenters to recreate complex spatiotemporal sequences of neural activity in large populations of neurons.

Opsins for multiphoton optogenetics:

The biophysical characteristics of opsin molecules are critical for the design and success of multiphoton holographic experiments. The relatively low single-channel conductance of some microbial opsins can necessitate that high levels of opsin expression be used in order to achieve photocurrents sufficient to drive action potentials at non-toxic levels of multiphoton excitation(33). Because of these issues, many cation opsins, including ChR2, are sub-optimal for multiphoton excitation. A key advance in recent years has thus been the rational engineering or discovery of new microbial opsins that are more light sensitive and provide much larger maximum photocurrents. The ultra-fast opsin ChroME, a point mutant of Chronos(64) that affords nearly four-fold greater photocurrents than its parent opsin, is far more effective at activating neurons compared to previously used opsins, including C1V1(44) and Chrimson (13). Recent advances in engineering ChroME have resulted in even more potent versions (‘ChroME2f’ and ‘ChroME2s’) that still retain relatively fast kinetics(65). CoChR and ChRmine are naturally occurring ultra-potent opsins that are also highly advantageous for multi-photon excitation(11)(41). The more potent of these opsins (ChroME, ChroME2f/s, ChRmine, and CoChR) all permit the reliable optical control with short illumination times and simultaneously enable the targeting of many more neurons with less total laser energy. Both CoChR and ChRmine are relatively slow opsins (in their closing kinetics), while ChroME closes nearly as fast as the opsin ChETA (43), and only slightly slower than its parent Chronos. Slow opsin kinetics are advantageous because they provide substantially greater charge transfer for each illumination photon, thus requiring lower laser energies to drive action potentials, at the relative expense of tight temporal control. Fast kinetic opsins, such as Chronos, instead require higher pulse energies but can provide submillisecond temporal control at physiological rates of spiking (up to 100 Hz)(13)(18). With regard to their two-photon absorbance spectra, CoChR absorbs optimally in the bluer end of the spectrum (peak ~900 nm)(41), ChRmine between 1035–1080(11), while ChroME absorbs best at ~1000 nm (13). The ChroME2.0 variants absorb similarly to ChroME, while other point mutations in ChroME can significantly blue-shift its excitation spectrum which could be useful in some circumstances(65).

Since unwanted activation of opsin molecules by the laser used for imaging neuronal activity is a major concern, their absorbance at the typical wavelengths for calcium imaging (~920 nm) must be considered, and the more potent or sensitive the opsin, the more undesirable activation one should expect. All of these opsins can be highly enriched at the soma and minimized in other neuronal compartments by the addition of targeting sequences from naturally soma-targeted channels (e.g., Kv2.1)(66)(67)(68)(41)(11). This ‘soma-targeting’ restricts off target photo-excitation of the dendrites or axons of other neurons (66)(13), and may reduce unwanted activation of opsin molecules by reducing the net time that the scan laser for calcium imaging illuminates opsin molecules on each neuron. Based on these features, researchers can choose the appropriate opsin and targeting motifs that are optimally suited to the demands of their experiment.

As a counterpoint to two-photon optogenetic excitation, employing opsins that hyperpolarize neurons can be used to suppress the activity of specific ensembles of neurons(44)(13)(22). The more recently discovered anion channels, such as the GtACRs(69), can be particularly advantageous since they have extremely high conductivity, and should shunt the membrane in addition to hyperpolarizing it, unlike ion pumps (e.g, eNpHR or Arch) which only hyperpolarize the membrane. Indeed, two recent studies have demonstrated ensemble and cell-type specific two-photon optogenetic suppression(13)(22). The power requirements for optogenetic suppression of large ensembles may be much higher than for excitation; this is because, typically, one does not know when exactly a spike will occur, and so the cells must be continuously illuminated. This constrains the total number of neurons that can be simultaneously suppressed owing to laser power and brain heating constraints (13). Employing step-function inhibitory opsins(70)(71) can conceivably overcome this issue. Recently, a new optogenetic harboring a fusion of GtACR2 to ChrimsonR enables potent bidirectional control with two photon excitation(72).

Optical crosstalk with the calcium imaging laser is another key point of consideration for all-optical experiments (73)(14)(13), particularly when using potent opsins with relatively slow off-kinetics All known microbial opsins are likely to absorb at 920 nm (the wavelength typically used for imaging GCaMP6). Thus, simultaneous calcium imaging of GCaMPs will cause some level of undesired direct electrical effect on the opsin-expressing neurons in the imaging volume. Most studies mitigate cross-talk by using low laser powers for imaging (at the expense of signal to noise ratio), by imaging at effectively slower frame rates per plane (~3–10 Hz, resonant scanning with optical or mechanical fast axial focusing), and by imaging across large fields of view to minimize the effective dwell time on each neuron’s soma per frame(65)(14)(13)(11). Alternatively, one can use blue opsins for photo-stimulation and image calcium dynamics at the red end of the two-photon spectrum (>1040 nm) (22)(51). However, commercially available lasers for optogenetic activation at the blue end of the two-photon spectrum are highly power limited. Standards for measuring optical cross-talk in all-optical experiments are critical for any future study; this is particularly critical when studying biophysically compact or highly excitable neurons for which the cross-talk could well lead to substantial depolarization and even spiking.

Currently, at least in mammals, opsins are typically delivered via adeno-associated viruses (AAVs). Though these can be acutely and precisely delivered through stereotaxic injection, the development of transgenic reporter lines would substantially facilitate experimentation by reducing the inherent variability associated with intra-cranial viral delivery(74). Alternatively, AAVs with appropriate capsids can be delivered intravenously in neonates or even adults, providing widespread expression analogous to that achieved with transgenic reporters(75). Regardless of the delivery method, without careful titration, viral expression can lead to high transgene levels that are toxic to neurons, as has been found with activity sensors like GCaMP6 and with opsins like ChR2(14)(76). Thus, stable genetic reporter lines in mice, flies, zebrafish(77), and ultimately in other species, such as non-human primates, would be highly advantageous for the widespread adoption of holographic optogenetics.

Outstanding challenges for multiphoton optogenetics

Although multiphoton optogenetics offers unparalleled opportunities for precisely perturbing neural activity, there are several key challenges that must still be overcome to broaden its utility and increase its precision.

Achieving ‘true’ single-cell resolution

Although multiphoton excitation can achieve high optical resolution in the brain, empirical measurements from numerous technical studies indicate that the effective resolution of multiphoton optogenetic-driven spiking, in some cases, is larger than the size of the typical neuron, particularly in the axial direction (15)(33)(22)(44)(12)(11)(78)(23)(66)(14)(21)(12)(13) which should always be considered when designing or evaluating experiments that may benefit from cellular resolution . We can define ‘single cell resolution’ as the ability to photo-activate only the desired target neurons while having no significant direct impact on any other neurons in the volume. This can be difficult to assess in practice when neurons are recurrently connected and nearby neurons may excite each other synaptically. One potential way to address this (without pharmacology) would be to ensure that some neurons do not express opsin (for example, by using a Cre-off viral vector to drive opsin expression(79)(80), and sparse Cre-on expression of a fluorophore). Any changes in these non-opsin-expressing neurons must be through the network. Since the cell density of cortical neurons(81) and other brain structures ensures that in many cases more than one neuron will be illuminated by an effective excitation focus, many, if not most, photo-excitation regimes cannot avoid incidentally activating additional neurons, typically directly above or below the intended desired target. Although it’s unclear whether achieving single-cell resolution even matters for certain classes of experiments, in cases where the experimental design depends on activating single neurons and inferring their impact (such as inferring functional or monosynaptic connectivity)(82)(83)(66), off-target effects can be highly problematic. In cases where larger ensembles are co-activated, the impact of the off-target neurons will likely depend on the ratio of off-targets to on-targets. However, to a certain extent, it is difficult to address the importance of off-target effects without achieving a system that minimizes or entirely avoids them. Thus, in some circumstances, non-single cell resolution may complicate the interpretability of holographic experiments. Positive effects could be due to a set of unintended targets, whereas negative results could arise if the impact of the off-targets cancel out the effects of the targeted ensemble. In cases where it is possible to measure some or all of the off targets, analysis can be made contingent on the entire ‘directly photo-stimulated’ ensemble, rather than just the ‘targeted’ ensemble(84). Note, however, that many directly-photo-stimulated cells will be in planes that are not necessarily sampled by the imaging system.

The difficultly in achieving absolute single-cell precision stems from several key issues: the need to activate sufficient opsin molecules on a neuron’s soma to drive it to action potential threshold(33), the incomplete restriction of opsin molecules from the dendrites(66), the packing density of neurons in brain tissue, the heterogeneity in opsin expression level between cells, and the heterogeneity in intrinsic excitability between neurons even of similar subtypes. In the initial study characterizing multiphoton optogenetics by scanning a focused spot, the beam location that drove maximal photocurrents was surprisingly above the targeted cell(33). This is because defocused multiphoton light can activate many more opsin molecules on the cell than a small spot can, owing to its larger cross-sectional area, and because opsins molecules have a high two-photon absorption cross-section (33) such that the defocused light is sufficient for two-photon excitation of opsin molecules on the plasma membrane. Thus, despite the optical sectioning ability of multiphoton excitation, opsin molecules on nearby neurons will almost always also be gated, leading to off-target depolarization. It should be clear that out-of-focus excitation is a problem not only for scanning, but also for parallel two-photon excitation techniques(42). Perhaps most importantly, the effective spatial resolution of two-photon excitation is highly power dependent. Higher powers generate more out-of-focus excitation, and when the laser power needed to excite a neuron approaches the saturation of its somatic opsin molecules, the optical sectioning capabilities of two-photon excitation are severely compromised. In other words, if the laser power on a target neuron exceeds the power range where the photocurrent has a quadratic dependence on excitation power, the axial resolution will deteriorate, whether using focused or parallel illumination; in such cases, there could be a substantial increase in the likelihood of off-target activation(42).

Despite these challenges, there are several approaches that can improve the spatial fidelity of the holographic perturbation (Figure 2). First, increasing the single-channel conductance of the opsin, such that lower powers can be used to drive neurons to threshold, should decrease off-target effects, since light levels farther from opsin saturation could be employed. Second, targeting the opsin to the soma will reduce activation of other neurons through their dendrites(66)(13)(41)(78). Third, sparsening opsin expression (85) such that optically excitable neurons are farther from each other will also help reduce off-target effects. Fourth, delivering just the right amount of light to each neuron – and not more – to drive it to spike could help maximize the effective spatial fidelity. Excess energy on a neuron (i.e., more light than needed to drive a spike) increases off-target effects, while too little light leads to activation failures. This balance can be achieved by measuring the effectiveness of photo-activation with simultaneous calcium imaging and adjusting the power per neuron during the experiment. Prior closed-loop paradigms have demonstrated the ability to dynamically adjust the magnitude of target activation in real time (86). Extending this approach to large ensembles for the explicit purpose of maximizing the effective resolution of photo-stimulation could be highly advantageous. Additionally, deliberately avoiding targeting neurons that require power levels that saturate the opsin to drive them to threshold will also increase the effective resolution, at the expense of flexibility in deciding which neurons to stimulate.

Figure 2: Improving the effective spatial fidelity of multiphoton holographic optogenetics.

In the ‘naïve’ approach, any group of neurons is stimulated with equal light power sufficient to activate any of the target neurons, which results in off-target activation. To mitigate these undesired effects, opsin expression can be sparsened to reduce the number of off-targets (top right), only neurons strongly expressing opsins can be stimulated with lower light powers (middle right), or each neuron can receive the optimal light energy to maximize fidelity and reduce off-target stimulation (bottom right).

Strategies for increasing the size of the controllable ensemble

Volumetric two-photon laser scanning microscopes can image the activity of thousands of neurons in the same experimental session, and in specialized cases, this can capture the activity of a substantial fraction of the local population that may be relevant to a specific computation or behavior(87)(88). With the recent advent of two-photon mesoscopes, tens of thousands of neurons could be sampled within the same experimental session. Given that some behaviors in animals may require the co-activation of large numbers of neurons, it could prove highly advantageous to obtain optical control of all of these neurons in the same experimental session using two-photon holographic optogenetics. There are two principal constraints on the number of neurons that can be simultaneously controlled (i.e., co-stimulated using one hologram): the potency of the opsin and the power of the laser. One can combine multiple high energy lasers into one microscope (11); however, too much energy can cause brain heating (13)(40) or tissue damage depending on the density of the holographic targets. Therefore, one way to increase the size of the controllable neural ensemble is through opsin optimization. The ideal opsin would have a high-single channel conductance, absorb well at 1040 nm given currently available commercial high-energy femtosecond lasers, and show little adaptation to repeated light exposure, as opsin desensitization can compromise the fidelity of the evoked spike trains(89)(90). It would also minimally absorb at ~920 nm to reduce cross-talk with the imaging laser for GCaMP. Recently, the crystal structure of multiple opsins, including anion opsins, have been solved(91)(92)(93)(94), which has facilitated the rational engineering of the opsin pore to maximize conductance, sensitivity, and spectral absorbance (e.g., in the case ChroME)(13). Further structural analysis of new opsins (e.g., ChRmine and ChroME) and additional mutagenesis could yield opsins with substantially improved optical characteristics, which has recently been achieved with the ChroME2.0 variants(65). In addition to opsin optimization, when exact simultaneity of co-activation of a neuronal ensemble is not required, a larger group of neurons can be ‘co-activated’ in a potentially behaviorally relevant epoch of time by interleaving multiple holograms on a single SLM that target different neuron subsets, and also by using multiple SLMs and alternating between them(11).

Mesoscale multiphoton holographic optogenetics.

Nearly all computations and behaviors depend on the coordinated interactions between multiple brain regions. Even in the mouse and for simple sensory-guided actions, activity may be distributed across multiple cortical regions that are millimeters apart. Understanding how patterns of activity in one brain area leads to specific patterns of activity in downstream regions will require precise optical control and readout across these areas simultaneously. By the same token, the role and influence of feedback from higher to lower areas will likewise require a similarly flexible and large field of view microscope.

The introduction of mesoscale multiphoton microscopes, i.e., systems that achieve simultaneous imaging across >5 mm fields of view, has opened the door to measuring neuronal activity across multiple relevant brain areas at the same time(95)(96)(97)(98). Adapting holographic optogenetic systems to these mesoscale microscopes will open up enormous opportunities for understanding how information propagates between brain areas and leads to behavior (Figure 3a). Although existing spatial light modulators do not have the pixel size and number to simultaneously illuminate across these very large volumes, combining SLMs with fast galvo-galvo positioning systems (11)(99)(100) could approximate full volumetric control even over ~5×5 mm regions.

Figure 3: Approaches to extend multiphoton holographic optogenetics.

a) Two-photon mesoscopes could be outfitted with holographic pathways to obtain high fidelity optogenetic control over very large brain volumes. b) GRIN lenses can be used to relay holographic patterns from conventional holographic systems into deep brain circuits. c) Removal of overlying tissue can allow direct access to deeper structures, such as the hippocampus. d) The two-photon microscopes could be directly coupled to an optic fiber or miniatured and head-mounted.

Multiphoton optogenetic control deep in the brain

In mammals, most brain structures are well beyond the reach of conventional two-photon imaging or optogenetics delivered from the brain surface due to the scattering length of brain tissue. To overcome this challenge, many groups have either removed the overlying brain tissue (e.g., the cortex and white matter to image the hippocampus) (101)(102) or implanted relay lenses (e.g., GRIN lenses)(103)(104)(105)(106)(107) (Figure 3b,c) or other optical elements(108)(109). These approaches have been adapted to two-photon optogenetics (104)(107)(26) with the same caveats. Although GRIN lenses significantly limit the field of view, degrade optical resolution and suffer from various forms of optical aberrations (which may be correctable)(110), they can provide optogenetic control anywhere in the brain that can be accessed by lens implantation. This will probably be particularly crucial in the brains of larger animals, such as primates, in which many brain structures are far beyond the reach of multiphoton excitation and would require too much tissue removal to be accessed directly from the surface. A third approach is to employ three-photon excitation, which can reach nearly 1.5 mm into the brain(111)(112). This has worked well for imaging moderately deep structures but requires much higher pulse energies, which can lead to brain heating or tissue damage. Three-photon excitation might also work for optogenetics(113), although the energy requirements might limit three-photon activation to stimulating relatively small populations of neurons in deeper structures. So far, two-photon optogenetics from the brain surface has been shown to be effective up to cortical layer 5 in mouse primary visual cortex(11), but going much deeper, non-invasively, may require three-photon excitation. A rigorous measurement of the depth penetration of two-photon optogenetic excitation in different species and brain structures is still needed.

Multiphoton optogenetics in freely behaving animals.

Understanding natural behaviors almost obligatorily requires freely behaving animals. Nearly all existing multiphoton holographic optogenetics studies to date have required head-fixation under the microscope. However, remote ‘single-cell’ resolution multiphoton imaging and optogenetics in freely behaving animals is possible by using flexible optical fiber-based systems (Figure 3d). A first approach consists in coupling the microscope to a fiber bundle-thousands of single core fibers assembled together, typically further attached from the output facet to a micro-objective or a GRIN lens. The desired intensity pattern is produced at the microscope objective focus, transmitted through the fiber bundle with each single core acting as an individual ‘pixel’, and refocused back onto the sample. This approach has so far been successfully used to achieve one-photon simultaneous patterned activation and functional imaging in freely moving animals (114), and has been shown to be compatible with two-photon imaging (115), including during free behavior (116). A promising alternative for even deeper and less invasive access is to replace the fiber bundle by a hair-thin multimode fiber directly inserted in the animal’s brain, without the use of an attached imaging lens (117)(118)(119). The fiber transmits the image through a superposition of optical modes travelling at different speeds, resulting in a scrambled image at its output. However, although complex, light propagation through multimode fibers is deterministic. Therefore, principles of digital holography can be used to compute the optical field at the fiber input required to generate the desired intensity pattern at the fiber output. Compatibility with multiphoton optogenetics and freely moving animals is however yet to be demonstrated and not free of challenges, since this technique is more sensitive to fiber bending (although it might be correctable to some extent (120)(121)), and pulse dispersion management through long distances of fiber remains challenging. Finally, another alternative approach would be to use a single mode fiber to deliver excitation pulses to a fully miniaturized two-photon microscope directly implanted on the animal’s skull; this has been applied to achieve two-photon functional imaging in freely moving animals (122)(123)(124). Augmenting these devices with two-photon patterned stimulation capabilities has so far mainly been challenged by the technical difficulty to miniaturize SLMs. Light-shaping techniques based on digital micromirror devices instead of SLMs may be more amenable to these miniaturization efforts(4)(125).

Optimally designing holographic neuronal perturbations

Perhaps the most fruitful and least developed aspect of holographic optogenetics is the design of the neural ensemble perturbation. The precision of multiphoton holographic optogenetics offers the ability to co-activate arbitrary combinations of neurons within the field of view, enabling a truly massive number of possible stimulation patterns. Yet experimental time is limited, and ample noise in the brain and in measurement requires averaging across repetitions of the same perturbation. Therefore, in any given experimental session, one can only test a small subset of possible perturbations. The simplest approach is to stimulate one neuron at a time and estimate its influence on the network, which doesn’t require holography(82)(24). This can be useful for mapping unitary functional interactions within the network, but because the net effects of most single neurons are quite small, many trial repetitions are needed, and thus only a small subset of single neurons can be probed per session (Figure 4a). More importantly, most single neurons are unlikely to generate the rich dynamics that might impact behavior or brain areas downstream.

Figure 4: Multiple approaches to use multiphoton optogenetics to reveal neural codes underlying behavior.

a) Single-cell stimulation can reveal unitary functional interactions in neural circuits. b) Co-stimulation of neural ensembles that share functional properties measured with calcium imaging. c) Targeted photo-stimulation of neurons predicted to act as pattern completion neurons. d) Identifying the minimal subset of neurons that can alter behavior through serial removal of unnecessary neurons.

Thus, many multiphoton optogenetics experiments will involve the photo-stimulation of ensembles of neurons. But how does one choose the members of each ensemble, the desired activity level of neuron in the group, and the temporal structure of the stimulated pattern? These will all impact, potentially profoundly, the outcome and interpretability of the photo-stimulation(126). Perhaps even more basically, how many neurons should one stimulate at one time? If one wishes to influence behavior it might make sense to stimulate as many of the neurons at a time that show functional relevant responses during the specific behavior(11). In this case, the constraint is the number of neurons that the two-photon optogenetic system can concurrently stimulate. But this may not be necessary in many cases to observe interpretable changes in behavior. Electrical microstimulation or one-photon optogenetic stimulation in primates, rodents and zebrafish that were estimated to stimulate very small numbers of neurons could modify perceptions or drive operant responses (127)(128)(129)(130)(131). Importantly, in some cases electrical or one-photon optogenetic stimulation can trigger physiological-like patterns of activity and complex perceptions or behaviors, suggesting that certain brain circuits can in fact be triggered with fairly non-physiological stimulation paradigms(132)(133)(134)(135)(136). However, the ability to re-create precise patterns of activity directly rather than relying on the circuit of interest to generate such patterns on its own has advantages for probing key attributes of the underlying neural codes that drive perception and behavior.

Importantly, several recent studies using multiphoton holographic optogenetics have put forth evidence that targeting very small numbers of neurons (2–20) could influence behavior(11)(137)(51) (138)(80)(26)(84). The crucial factor in most of these recent holographic optogenetic studies was that the targeted neurons were chosen based on their physiological responses – something not possible with one-photon optogenetics or electrical microstimulation, except in rare circumstances. In some of these studies, the authors targeted neurons based on their shared response to a sensory or cognitive feature (e.g., the orientation of a grating or direction of a cued action)(138)(11)(84)(84) (Figure 4b). In one of these, computational analysis of the physiological data identified specific neurons that appeared to have a very high degree of functional coupling with many other neurons (putative ‘hub like’ or ‘pattern completion’ neurons)(137) (Figure 4c). In another study, the authors used a process of elimination strategy to identify the minimal ensemble that could still elicit behavior (in this case, tail bending in zebrafish)(51) (Figure 4d).

In studies that focused on mammalian neocortex(137)(11)(84), activation of the targeted neurons appeared to amplify activity within functionally coupled sub-networks, possibly through shared, recurrent connectivity. These studies posited that this amplification or completion-like effect was crucial for ultimately impacting behavior. Thus, at least in highly recurrent networks like the neocortex, activating small groups of neurons appears to be sufficient to modify behavior provided the neurons are carefully chosen, although this could depend on the design of the behavioral task. In many other networks that have far sparser recurrent connections or are predominantly GABAergic (e.g., the cerebellum or the basal ganglia) such sparse activation schemes might fail to generate measurable changes in behavior. It is also important to consider that in all of these holographic optogenetic studies, additional off-target neurons may have been incidentally activated. Such off-target activation, when measured, can be addressed and potentially ruled out in neural data analysis as some studies have done (11)(84), but its role in induced behavior may be difficult to determine.

While designing photo-activation of ensembles based on shared physiological activity in a sensory or motor task is unquestionably logical and informative, it also limits the ability of experiments to identify unexpected and novel aspects of the neural code. Neural networks (e.g., in the cortex) can encode extremely diverse and high-dimensional stimuli(139), yet due to limited experimental time one can only probe a small part of this large space. As a complement to the ‘rational’ choice of neural targets based on their physiological responses, one could instead use random ensemble stimulation with simultaneous imaging to first construct a predictive model of the network under study(140). If the generated model were sufficiently accurate (validated by predicting the network response to held out random photo-stimulation ensembles), an experimenter could use the model to design and test photo-activation schemes that could maximally push the network along specific dimensions, potentially revealing novel stimulation patterns that drive large, predictable changes in behavior.

All these foregoing strategies rely on designing photo-activation ensembles in advance; alternatively, one could employ fast closed-looped holographic optogenetics to update the photo-stimulus pattern millisecond by millisecond, which has been achieved with both one- and two-photon optogenetics(141)(142)(86)(143)(142)(144). Closed-loop strategies can leverage the natural dynamics of the system to ‘select’ the specific holographic perturbation and inform a network’s internal dynamics. By analogy to the voltage clamp of single neurons, with closed-loop optogenetics one can ‘clamp’ the activity of a specific ensemble to a particular activity level (86) and read out the laser energy targeted to the illuminated ensembled needed to keep it stable, as a metric of its functional input, in different sensory or behavioral conditions. Alternatively, one could implant ‘artificial’ connections between specific neurons (or classes of neurons) that do not normally interact, by artificially yoking the activity of certain neurons to the ongoing activity of another(86). Such experiments could causally test hypotheses for how connections between neurons in a network lead to population dynamics or behavior. In a different strategy, one could yoke the activity of a specific ensemble not to neural events, but to behavioral events.

Optimally designing the behavioral task in tandem with the neural perturbation is just as critical. Multiphoton holographic optogenetics provides many novel opportunities but also comes with key constraints – perhaps the most important being the limited size of the controllable ensemble. Because of this constraint, the first consideration for task design is that perturbations of even small numbers of neurons should yield a detectable change in behavior. In sensory tasks, for example, one should thus probe how the holographic perturbation influences perception close to detection or discrimination threshold where the added activity might have the greatest impact on behavior. Relatedly, one can take into account the potential ability for certain networks to exhibit attractor dynamics or multi-stability; one can deliver stimuli or use operant behaviors that put the network at an unstable point (e.g. at a putative bifurcation) such that a small optogenetic perturbation could trigger a large change in network dynamics. By the same token, one might deliberately study networks characterized by extremely sparse spatiotemporal representations (such as observed for natural visual stimuli in V1)(145)(146) which would make it technically easier to reproduce the endogenous dynamics with sparse holographic optogenetics.

Probing the neural codes of sensation, cognition and action

The vast majority of prior experimental and theoretical work on neural coding has relied on correlational analysis of neural activity. Multiphoton holographic optogenetics can provide the experimental means to probe the fundamental logic and syntax of the neural code. Below is a small sampling of the possible questions that holographic optogenetics could help resolve.

Synchrony and spike timing in visual perception:

The long-standing debate over the contribution of neural synchronization in the visual cortex to sensory perception is one of the best known in sensory neuroscience. Ample correlational evidence exists to support both temporal and rate coding schemes(147)(148)(149). Standard correlational analysis or even simple circuit perturbations cannot resolve this question; instead, one could use multiphoton holographic optogenetics. First, one could drive a fixed number of action potentials in a relevant group of visual cortical neurons that is sufficient to drive the appropriate operant response on a visual discrimination task. Then one could re-organize the exact same number of spikes in the temporal domain so that the population exhibited varying degrees of synchrony, yet each neuron still exhibited the exact same firing rate (Figure 5a). If changing the temporal structure has no influence on visual discrimination, one could reasonably conclude that, at least in this task, neural synchronization is not critical for perception. Alternatively, one might find that increasing neural synchrony facilitates perception – and then one could even compare the efficacy of synchrony in different frequency bands. More generally, spike timing – even without global synchronization, may be critical for neural coding. Recording physiological spike times (e.g., with voltage imaging, see below) and then recapitulating those spikes but shuffling their times while preserving rates could address this question.

Figure 5: Examples of using multiphoton holographic genetics to address neural codes and plasticity rules.

a) Probing the role of neural synchrony. Four example possible single-trial raster plots of holographically induced ensemble activity of increasing synchrony. Note that the absolute spike rate per trial is fixed, while the temporal pattern is altered. b) Schematic of two types of multiphoton holographic perturbations, one that obeys the intrinsic low dimensional architecture of the neural activity patterns a group of neurons exhibits (‘on-manifold’) and one that does not (‘off-manifold’). c) Schematic of using multiphoton holographic optogenetics to artificially induce Hebbian spike timing dependent plasticity between two artificially chosen neural ensembles (indicated by the two colors). Top right: Schematic of the conventional STDP learning rule plotting change in synaptic strength vs the time delay between pre- and postsynaptic activation. Top right: Holographic stimulus pattern to induce STDP. Bottom: schematic of fast interleaved photo-stimulation of two ensembles to drive STDP between them.

Impact of noise and noise correlations on sensory perception:

Cortical neurons show a high degree of trial-to-trial fluctuations in their firing rates to the identical stimulus. This response ‘noise’ is not independent for each neuron, but is in many cases correlated, albeit often weakly. Although these ‘noise correlations’ have been used extensively to infer ensemble architecture in neural circuits, the functional impact of these correlations remains unclear. Theoretical considerations or computational analysis of neural data imply that these correlations might either facilitate encoding, degrade it, or have no effect depending on the conditions(150)(151). Holographic optogenetics can causally test these ideas. Again, in a paradigm where one can generate artificial percepts with holographic optogenetics, one can arbitrarily control the across-trial correlations in the firing rates of photo-stimulated ensembles(13)(86). One could reduce the noise correlations to zero or increase them to a maximum, and probe the impact on behavior as well as the population dynamics of the rest of the network.

Probing the functional impact of physiological patterns of neural activity:

Activity in neural populations is inherently restricted to a small subset of all possible patterns of the system by its intrinsic synaptic connectivity, and the biophysics of its component neurons and synapses(126)(152). Multiphoton holographic optogenetics, with careful calibration, can drive activity patterns that obey the low-dimensional architecture of a neural network and simultaneously track the impacts on downstream activity and behavior (Figure 5b). However, a key constraint for multiphoton optogenetics is the field of view: since there will be many neurons that are still part of the network but outside of the field of view, even the most precise perturbations will lack control over key components of the relevant neural network. In this respect, one-photon optogenetics has a clear advantage over the volume and number of neurons it can address, albeit with substantially lower spatial resolution. A further consideration for holographic experiments that attempt to recreate physiological patterns of activity is off-target effects and incomplete control over the induced spike rates, as discussed above. Any amount of off-target effects may divert the perturbation from its intended course. Just as problematically, activation failures of targeted neurons (or hyperactivation) could also impact the results on the network.

Probing the rules and impact of synaptic plasticity on neural networks during learning:

Fast, dynamic changes in synaptic weights are likely to be crucial for learning and memory formation. Spike timing dependent plasticity (STDP) is perhaps the leading candidate for how physiological patterns of neural activity during experience lead to changes in the neural circuit architecture to encode memories(153). Synaptic learning rules have been probed extensively in brain slices, and occasionally in vivo, but only on the small scale and in non-physiological conditions. Holographic optogenetics coupled with calcium imaging – particularly in platforms with millisecond optogenetic timing – affords experimenters the ability to probe these learning rules all-optically, across hundreds if not thousands of neurons in individual animals, and across days. One prior study could ‘imprint’ an ensemble holographically (154), although subsequent showed this plasticity was non-Hebbian(155). Using closed-loop stimulation another study could generate long-term plasticity between user-defined trigger and follower neurons (86), although it remains to be determined if this plasticity follows well-established learning rules identified in brain slices(153)(156). Notably, these studies did not leverage precise, millisecond timing which should be critical for the STDP process as previously described. With appropriate timing, one might test the standard STDP learning rules in specific neural circuits in specific behavioral contexts, before or after training, during the administration of a drug, or in a diseased state. Perhaps even more intriguingly, one might ‘write-in’ patterns of synaptic plasticity entirely holographically by optogenetically pairing action potentials in two ensembles to generate an artificial memory (Figure 5c). Although this has been achieved with one-photon optogenetics via opsin expression from an activity-dependent promoter (e.g., c-fos)(157), holographic optogenetics is not constrained by the restrictions imposed by an activity-dependent promoter.

In a similar vein, one might use holographic optogenetics to take temporal ‘snapshots’ of functional network connectivity before, during and after learning. Networks can be mapped by stimulating one or a few neurons at a time, and the network architecture inferred from the changes in activity of all the other neurons in the network. By doing so, one might identify the key plastic connections in a circuit that are critical for learning, and even attempt to reverse their plasticity with holographically induced spike-timing dependent long-term depression, and thus reverse the learning.

Combining Multiphoton holographic optogenetics with in vivo voltage imaging

To re-create precise neural activity patterns, one must measure them accurately in the first place. Calcium indicators, however, indirectly report spike rates, and cannot provide millisecond-precise spike times; this is because calcium kinetics and calcium indicators are relatively slow, and conventional two-photon imaging microscopes image at low frame rates. Additionally, the variable transform between spike rate and calcium indicators across neurons (even of the same cell type) make it similarly difficult to infer exact spike rates(158)(159)(160). Remarkably, in the current state of the art, the temporal bandwidth of two-photon optogenetic stimulation exceeds that of two-photon calcium imaging owing to the fast update rates of SLMs for holography and the brief illumination times needed to drive neurons to spike (sometimes < 1 msec).

One promising alternative is population voltage imaging, specifically in the two-photon regime(161)(162)(163)(164)(165). While much progress has been made with one-photon voltage sensing, some of the best voltage sensors have yet to be shown to be effective with two-photon excitation. Recent work with the ASAP class of voltage sensors(161)(165) as well as chemical voltage-sensitive dyes(163) has demonstrated that specialized high-speed two-photon imaging systems can sample neuronal action potentials in vivo, albeit from very restricted fields of view or regions of interest. These proof-of-concept studies will enable researchers to accurately measure and then replay extremely precise spatiotemporal neural sequences high speed two-photon optogenetics. Expanding the scale of two-photon voltage imaging is still necessary, but undoubtedly a major goal of modern neurotechnology.

Conclusions and future outlook

Although optogenetics has revolutionized experimental neuroscience in the last 15 years, numerous classes of experiments aimed at fundamental questions in brain function remain beyond the reach of conventional (one-photon) optogenetic approaches. Perhaps most importantly, in brain regions like the neocortex, two-photon optogenetics can afford the requisite spatial resolution to recreate the precise patterns of neural activity needed to probe certain major outstanding questions of neural coding. Yet there are still critical technical and conceptual challenges to overcome in multiphoton holographic optogenetics, including expanding the scale and improving the spatial fidelity of its optical control.

Multiphoton holographic optogenetics may also someday form the basis of therapeutic optical-brain interfaces. Currently, brain prostheses typically rely on electrical micro-stimulation(166), whether it be in peripheral structures like the cochlea, or central structures like the cerebral cortex(167). Although electrical stimulation is technically straightforward compared to optogenetic manipulation and requires no introduction of exogenous genetic elements, it has relatively poor spatial resolution, will often activate fibers of passage(168), and may cause harm to the underlying tissue. Optical interfaces – at least in superficial structures, need not penetrate the brain, and when coupled with multiphoton excitation, should achieve cellular resolution. Using the precision of holographic optogenetics, precise perturbations should be far more effective at generating the neural dynamics needed to generate artificial percepts for effective cortical sensory prostheses for those with vision or auditory impairments. Perhaps even more conjecturally, holographic prostheses in higher cortical areas might be used to treat cognitive disorders through closed-loop spatially precise interventions.