Brain-wide organization of neuronal activity and convergent sensorimotor transformations in larval zebrafish

Summary

Simultaneous recordings of large populations of neurons in behaving animals allow detailed observation of high-dimensional, complex brain activity. However, experimental approaches often focus on singular behavioral paradigms or brain areas. Here, we recorded whole-brain neuronal activity of larval zebrafish presented with a battery of visual stimuli while recording fictive motor output. We identified neurons tuned to each stimulus type and motor output, and discovered groups of neurons in the anterior hindbrain that respond to different stimuli that elicit similar behavioral responses. These convergent sensorimotor representations were only weakly correlated to instantaneous motor activity, suggesting that they critically inform, but do not directly generate, behavioral choices. To catalog brain-wide activity beyond explicit sensorimotor processing, we developed an unsupervised clustering technique that organizes neurons into functional groups. These analyses enabled a broad overview of the functional organization of the brain, and revealed numerous brain nuclei whose neurons exhibit concerted activity patterns.

Introduction

Until recently, our circuit-level understanding of the brain has been built up from targeted studies of individual brain regions. While such approaches have enabled a detailed understanding of a plethora of specific brain functions, they have difficulty revealing emergent brain-wide functional activity that results from interactions between brain areas. Moreover, they tend to bias observations towards activity closely related to known behavioral paradigms, leaving a broad range of salient activity patterns undiscovered. Recent advances in functional imaging technology have begun to release these restrictions, making simultaneous recording across large brain areas possible (Lemon et al., 2015; Peron et al., 2015; Prevedel et al., 2014). In the case of the larval zebrafish, the small size and transparency of the animal make it possible to record activity from the whole brain at single-cell resolution in a behaving animal (Ahrens et al., 2013a; Cong et al., 2017; Kim et al., 2017; Portugues et al., 2014; Vladimirov et al., 2014).

Whole-brain imaging has enabled comprehensive analysis of activity underlying two innate sensorimotor responses, phototaxis (Wolf et al., 2017) and the optomotor response (Naumann et al., 2016), as well as spontaneous swimming behavior (Dunn et al., 2016a), resulting in the discovery of specific brain regions involved in these sensorimotor pathways. However, the relationship among these disparate sensorimotor pathways has not been investigated in depth. Here we ask, how do brain-wide populations of neurons represent these different stimuli and associated behaviors, and are there intermediate sensory/motor representations where different sensorimotor pathways converge? To extend the scope of whole-brain investigations, and to enable such a study of the relationships between different sensorimotor pathways, we presented a battery of different visual stimuli to each fish (Fig. 1c), allowing us to examine the same neurons in different behaviorally relevant contexts, and obtain a more comprehensive view of their converging or diverging contributions across different sensorimotor pathways.

Figure 1. Whole-brain recordings of neuronal activity.

(a) Schematic of experimental setup for fictive swimming combined with light-sheet imaging (see Methods). (b) Illustration of functional dataset format. Whole-brain volumes were imaged at ~2 volumes/sec for ~50 min (2.11 ± 0.21 volumes/second; 6,800 ± 470 time frames, n = 18 fish). The activity traces of individual neurons were automatically extracted for 8.0×10⁴ ± 1.6×10⁴ cellular ROI’s per animal (n = 18). Scale bar, 50 μm. (c) Illustration of visual stimuli presented during functional imaging. Four stimulus paradigms from left to right: phototactic stimulus (phT), moving stripes (Optomotor response or OMR), expanding dot (looming or visual escape response), and dark flashes. (d) Example neuronal activity of single neuron ROI’s within an imaging plane (right inset) in the tectal region in the midbrain (cell activity is z-score normalized). The stimulus (phototactic stimuli alternating with white background) is illustrated with a bar above the calcium traces, and the recorded fictive behavior is plotted at the bottom of the panel, with three plots indicating left turns, forward swims and right turns from top to bottom. (e) Image stacks were registered to the Z-brain reference brain atlas (Randlett et al., 2015) containing ~300 labels of anatomical regions. Scale bar, 50 μm.

Whole-brain imaging can also capture many salient activity patterns that are unrelated to the explicit processing of experimental stimuli, and presents an opportunity to systematically catalog such activity patterns. However, inspecting and analyzing the activity of ~100,000 neurons poses a significant computational challenge, and requires the right tools. Many analysis approaches have been developed to interpret neural population activity data based on projections to low dimensional spaces (Cunningham and Yu, 2014; Freeman et al., 2014; Lopes-dos-Santos et al., 2013), regression (Feierstein et al., 2015; Miri et al., 2011; Portugues et al., 2014; Wolf et al., 2017), noise correlation (Averbeck et al., 2006; Cohen and Kohn, 2011), and clustering (Bianco and Engert, 2015; Romano et al., 2015, 2017). Here, we developed a density-based agglomerative clustering method for discovering groups of neurons with similar dynamics. Although cells were grouped solely based on functional activity, many of the resulting clusters were anatomically compact, revealing known as well as previously uncharacterized brain structures. We implemented our analysis tools with a custom graphical user interface (GUI) for interactive and flexible data exploration. The code, and all the data, is made publicly available, to enable community efforts for mining these rich datasets to discover deeper structure in whole-brain neuronal activity and its relation to visual stimuli and behavior.

Results

Whole-brain recordings of neuronal activity

Using a light-sheet imaging system reported previously (Vladimirov et al., 2014) (Fig. 1a), we recorded the activity of the majority of neurons in the larval zebrafish brain simultaneously, while presenting a battery of visual stimuli and recording fictive swimming behavior. For each fish, calcium imaging data was recorded at ~2 volumes/sec for ~50 min (2.11 ± 0.21 volumes/second; 6,800 ± 470 time frames, n = 18 fish) (Fig. 1b, Fig. S1e, Suppl. Movie 1-2). The pan-neuronally expressed calcium indicator GCaMP6f (Chen et al., 2013) was localized to the cell nuclei by fusing it to the histone H2B protein (Vladimirov et al., 2014), which facilitates automatic cell segmentation performed via a template matching algorithm (Kawashima et al., 2016). We thus obtained the activity traces of ~80,000 cellular ROIs per fish (a total of 1.4 million ROI’s, 8.0×10⁴ ± 1.6×10⁴ ROIs for 18 animals; example traces in Fig. 1d), accounting for the vast majority of neurons in the brain with the exception of the most ventral part (Fig. S1a).

Blocks of visual stimulus patterns associated with the following behavioral paradigms were projected onto a screen below the fish during imaging (Fig. 1c). (1) Phototaxis (phT). Larval zebrafish are attracted by light and are averse to darkness, and use spatial differences in luminance to guide their navigation (Brockerhoff et al., 1995; Burgess et al., 2010). We presented a half-field dark stimulus on either the left or the right side, separated in time by a whole-field white baseline. (2) The optomotor response (OMR). This is a position-stabilizing reflex to whole-field visual motion, in which fish turn and swim in the direction of perceived motion (Orger and Baier, 2005; Orger et al., 2000). The stimuli used were whole-field stripes moving in different directions. (3) The visual escape (looming) response. In free-swimming fish, expanding discs elicit an avoidance response; although in tethered preparations fish often exhibit freezing behavior, there is still a strong sensory-related response in large areas of the brain (Dunn et al., 2016a; Heap et al., 2018; Temizer et al., 2015; Yao et al., 2016). The looming stimuli were presented from either the left or right side of the fish. (4) The dark-flash response (DF). This reflex is characterized by long-latency, large-angle turns in response to sudden darkening of the environment (Burgess and Granato, 2007; Chen and Engert, 2014). Alternating whole-field dark and bright stimuli were used for this stimulus block. (5) Spontaneous behavior. In the absence of visual stimuli, fish spontaneously swim in alternating sequences of repeated turns to the left and to the right (Dunn, Mu et al. 2016). For this block, fish were imaged under homogeneous background illumination.

During imaging sessions, fictive swim signals were detected by extracellular recordings of the descending axial motor neurons on each side of the tail (Ahrens et al., 2012; Masino and Fetcho, 2005). These recordings were reconstructed into fictive swim bouts, which were used to decode turns (Fig. 1d, bottom bar; Methods) (Ahrens et al., 2013b; Dunn et al., 2016b). We then used the fictive swim bouts to determine if the stimuli reliably elicit directional swims during our experiments. We found that for all stimuli, the vast majority of recordings showed significant stimulus-driven behavior, while there is also clear variability in the robustness of the behavior response within the population (Fig. S1g).

In order to make anatomical comparisons across experiments with different fish, we registered the functional imaging stacks to each other (Fig. S1b,c; Methods), and also to the Z-Brain atlas for larval zebrafish (Randlett et al., 2015) that contains molecular labels and definitions for known anatomical regions (Fig. 1e, Fig. S1d). To enable intuitive data analysis, we developed a custom interactive platform in MATLAB (Fig. S1f, code and data available online) to efficiently explore this data in both functional and anatomical contexts.

Identification of sensory-related neurons

A first step towards understanding the brain’s sensorimotor transformations is identifying cells whose neural activity closely tracks the stimulus input. To do this, we simulated stimulus-responsive activity traces for each stimulus type (stimulus regressors) and quantified how similar each cell’s activity was to each stimulus regressor (regression coefficients) (Fig. 2a, Fig. S2a,b, see Methods). To visualize these results, we plotted whole-brain stimulus tuning maps, showing the anatomical location of all cells with high regression coefficients for each stimulus type (Fig. 2b-e).

Figure 2. Identification of sensory related neurons.

(a) Stimulus regression, using phototactic stimulus as example. Fish were shown a periodic stimulus during imaging that consists of leftwards and rightwards phototactic stimuli separated by a whole-field bright background. The stimulus regressors (black) are constructed by convolving a binary step function with an impulse kernel of GCaMP6. The colored traces show the ΔF/F (mean±SD for all ROI’s with Pearson’s correlation coefficient r>0.5) for a single fish. (b-e) Average stimulus response maps across multiple fish for (b) phototactic stimuli (phT), (c) moving stripes (optomotor response, or OMR), (d) expanding dots (looming or visual escape response), and (e) whole-field dark versus bright (dark-flash response). (f) Extraction of stimulus-locked component of neural activity. The stimulus presentation (top) is periodic. The neural activity (middle) can be averaged over the stimulus period to extract the stimulus-locked, periodic component (bottom). (g) Cells can be ranked by the magnitude (percent of variance explained) of their stimulus-locked component. Three example cell activity traces are shown in order of decreasing periodicity during OMR stimulus presentations (h-k) Functional cell types for cells tuned to different stimulus types (see Fig. S2 legend for detailed descriptions. h: phT, i: OMR, j: Looming, k: Dark Flash. The cells with the most highly periodic activity for each stimulus type were selected and sorted into clusters using k-means clustering (Methods). Left panels show average activity for each cluster. Right panels show anatomical location of cells within clusters. Scale bars, 50 μm. OTec: optic tectum. aHB: anterior hindbrain. Cb: cerebellum. pTec: pretectum. PO: preoptic area. Dien: Diencephalon. Pa: pallium. Hb: habenula.

The phototaxis (phT) tuning map (Fig. 2b, Fig. S2c) shows lateralized and symmetric tuning dispersed throughout the midbrain optic tectum, as well as active populations in the cerebellum and anterior hindbrain. The lobes of the optic tectum, which receive input from the contralateral eye, display more abundant responses to the dark half of the stimulus than the bright half. The moving stripes (OMR) stimulus elicits strong and robust responses in the pretectum and anterior hindbrain (Fig. 2c, see also Fig. S2d for OMR forward vs backward), consistent with previously reported tuning maps (Kubo et al., 2014; Naumann et al., 2016). The looming stimulus evokes widespread, bilaterally symmetric activity patterns in large areas of the tectum and in stereotypical locations within the forebrain, diencephalon (thalamus, habenula), and hindbrain (Fig. 2d). Previous studies have also identified looming-responsive cells in the optic tectum and demonstrated that looming-evoked escapes are mediated via the Mauthner system in the hindbrain (Dunn et al., 2016a). The activity maps for dark and bright whole-field stimuli (Fig. 2e) are also bilaterally symmetrical and quite distinct from the other maps. The dark field map overlaps partially with the phototaxis map, but also exhibits unique activation in tectum (the homologue of the mammalian superior colliculus), pretectum, habenula, pallium (related to mammalian areas including cortex, hippocampus and striatum), and cerebellum, whereas the activity related to bright field is located close to the midline in the midbrain area.

The preceding regression analysis is powerful enough to identify distinct stimulus tuning maps, but is implicitly biased towards cells that show sustained responses to stimuli, and can miss cells that have different response patterns (for example transiently active at the onset of the stimulus). To detect these more complex responses, we can take advantage of the fact that even complicated stimulus responses will still be periodic with the same frequency as the experimental stimulus blocks. Therefore, the cells whose activity is most periodic can be interpreted as the most stimulus-locked (sensory-related), without bias towards specific temporal activity patterns. To catalog major patterns of sensory-related cell activity, we extracted the periodic (trial-averaged) component of activity for each cell (Fig. 2f) and ranked cells based on the magnitude (variance explained) of the periodic component (Fig. 2g). We then selected highly periodic cells for each stimulus type and sorted them into clusters using k-means clustering (see Methods), revealing a range of representative activity patterns with diverse temporal response types (Fig. 2h-k, Fig. S2e, see Fig. S2 legends for detailed descriptions of clusters). Responses to phototaxis (Fig. 2h, Fig. S2e) included sustained activity in the midbrain during phT, whole-field bright, or half-field bright stimuli (see also Fig. S2f-i), ramping activation in the superior (dorsal) raphe nucleus and dorsal posterior hindbrain, and cells in the midbrain transiently activated by the onset of phT (Fig. S2j). Responses to OMR (Fig. 2i) included various combinations of direction-selective tuning in the pretectum, optic tectum, anterior hindbrain, and dorsal hindbrain, as well as cells that preferentially respond to the bright stimulus between OMR stimulus presentations (Fig. S2k). Clusters responsive to looming (Fig. 2j) included populations of direction-selective cells in the optic tectum and anterior hindbrain, along with bilaterally responsive cells in the diencephalon and medial hindbrain. Dark-field tuned cells (Fig. 2k) were broadly situated throughout the brain, but densely populated in anterior hindbrain (left side in this example fish; laterization varies), while light-tuned cells were concentrated in the opposite-side anterior hindbrain and other locations consistent with bright-selective clusters from the phT and OMR maps. Because this approach utilizes whole-brain imaging data and is sensitive to any stimulus-locked activity patterns, it has the potential to produce a comprehensive list of stimulus-driven functional cell types.

Identification of motor-related neurons

We next identified cells whose activity was closely related to motor output, near the ‘endpoint’ of the sensorimotor transform. We utilized the measured fictive behavior to construct left and right motor regressors, and plotted motor tuning maps showing the anatomical location of all cells with high regression coefficients for left and right turns (Fig. 3a-c, data for OMR stimulus blocks shown as example, see Fig. S3e-h for other stimuli, see also Methods; Fig. S3a-d). These maps reveal a dense, lateralized and highly concerted neural population in the hindbrain, which is consistent with previous studies of spontaneous fictive swimming (Dunn et al., 2016b), and also highlight known motor-related anatomical landmarks including the spinal projection neurons (e.g. the RoV3, MiV1 and MiV2 reticulospinal neurons that control turning, see Fig. 3c, upper right inset) (Orger et al., 2008; Randlett et al., 2015). However, these direct motor regression maps also show considerable activity in the optic tectum, overlapping with the corresponding sensory maps (Fig. 2c,i). This is because stimulus-evoked behaviors produce a strong correlation between the stimulus and motor regressors, causing a simple regression analysis to mix up stimulus and motor-related neuronal activity.

Figure 3. Identification of motor-related neurons.

(a) Left: anatomical map of all cellular ROI’s with r>0.5 to either leftwards (red) or rightwards (cyan) motor regressors during OMR stimulus blocks (example fish). (b) Average motor regression maps for OMR stimulus blocks (n=11 fish). Analogous maps for other stimuli shown in Fig. S3e-h. Inset: activity in the proximity of the hindbrain spinal projection neurons that control turning (RoV3, MiV1 and MiV2 neurons); masks from Z-Brain Atlas. (c) Motor regression: motor regressors (for left, right turns, respectively) were constructed by convolving the processed fictive swimming traces with an impulse kernel. Brain activity from all cellular ROI’s is regressed against these regressors. The dF/F traces show the mean±SD for all ROI’s with r>0.5 for a single fish. (d) Example decomposition of the motor output into stimulus-driven (motor avg.) and stimulus-independent (motor res.) components. (e) Dissection of brain-wide motor-related activity. Lower left panel: Scatter plot of regression coefficients for all cells with respect to motor avg. (y axis) and motor res. (x axis) regressors, for an example fish in relation to the left-side motor output. Maps were calculated using all stimulus blocks together (similar analysis for individual stimulus blocks shown in Fig. S3m). Top left, purple box: stimulus-driven motor map showing cells ranking in the top 2% for motor avg. regression only. Bottom right, green box: independent motor map showing cells ranking in the top 2% for motor res. regression only. Top right, blue box: map showing cells ranking in the top 2% for both motor avg. and motor res. regressions. (f) Cells ranking in the top 2% for forward swimming (bilateral) regression only (calculated using all stimulus blocks together). Note the bilateral and dense cluster of cells at the red arrow locations in rhombomere 5, close to the MiD2 reticulospinal neurons (according to ZBrain). The bilateral swimming regressor is the average of left and right motor regressors (Fig. S3k). (g) Cells ranking in the top 2% for turning (unilateral) regressions only (calculated using all stimulus blocks together). The lateral swimming regressors are the right and left motor regressors minus the bilateral regressor (Fig. S3k). Dotted yellow lines: boundary between rhombomeres 2 and 3. Note the contralateral activity (arrowheads). Scale bars, 50 μm. pTec: pretectum. nMLF: nucleus of the medial longitudinal fasciculus. aHB: anterior hindbrain. pHB: posterior hindbrain. vSPN: ventral spinal projection neurons. rh: rhombomere. aHB(1-2): anterior hindbrain rhombomeres 1,2. ARTR: anterior rhombencephalic turning region. IO: inferior olive.

To disentangle stimulus-driven motor activity from independent motor activity, we again took advantage of the periodic nature of the experimental stimulus presentations to decompose the motor outputs into trial average (motor avg.) and residual (motor res.) components, which represent stimulus-driven motor output and independent motor output, respectively (Fig. 3d, Fig. S3i-j, Methods). Regression analysis was then used to quantify separately how strongly each cell’s activity was tuned to stimulus-driven or stimulus-independent motor output. We visualized this analysis by plotting the regression coefficients for each cell in a two-dimensional space (Fig. 3e, lower left). To understand the anatomical organization of different types of motor activity, we plotted tuning maps for cells that were highly tuned (top 2% rank) to stimulus-driven motor activity, stimulus-independent motor activity, or both (Fig 3e, magenta, green, and blue boxes, respectively; OMR stimulus shown as example, for other for stimuli see Fig. S3m). The stimulus-independent motor map (Fig. 3e, green box) contains neurons almost exclusively posterior to the rhombomere (rh.) 2/3 boundary in the hindbrain (Fig 3e, dotted line), while the stimulus-driven motor map (Fig. 3e, magenta box) contains cells mostly anterior to the rh. 2/3 boundary, including large populations of neurons in the pretectum and anterior hindbrain (rh. 1,2) that overlap with the OMR stimulus map (Fig. 2c). The intersection map (Fig. 3e, blue box) is more similar to the independent motor map (for the thresholds chosen here). The same analysis performed for other stimulus blocks (phototaxis, looming, and dark flash, Fig. S3m) reveals that stimulus-independent motor maps are similar across all types of stimuli, while stimulus-driven motor maps show distinct tuning patterns. This suggests that the motor-tuned cells posterior to the rh. 2/3 boundary are dedicated motor centers that are employed similarly regardless of stimulus type.

We applied a similar analysis to disentangle activity related to swimming to the right or left (unilateral) from forward swimming (bilateral), which are convolved in motor output. Specifically, we averaged the left and right motor outputs to form a bilateral swimming component, and subtracted this bilateral component from the left and right motor outputs to yield unilateral swimming components (Fig. S3k). We then plotted tuning maps for bilateral (Fig. 3f) and unilateral (unilateral, Fig. 3g) swimming, which uncover more anatomical subtleties. Compared to previous methods for separating right, left, and forward swimming (Fig. S3b) (Ahrens et al., 2013b), this analysis accentuates more clearly the differences in tuning between turns and forward swimming. The bilateral swimming map features a tight pair of clusters in Rhombomere 5 near the MiD2 spinal projection neurons that to our knowledge has not been previously characterized. The unilateral maps highlight the anterior rhombencephalic turning region (ARTR, also know as the hindbrain oscillator, HBO, which has shown to mediate turning biases (Dunn et al., 2016b)), along with correlated cells in ventrally extending columns in rh.5/6 and the inferior olive. These maps also reveal a smaller population of cells in rh. 1/2 correlated to the contralateral ARTR clusters (see also Fig. S3l).

Multi-stimulus convergence

Having mapped cells active at the ‘beginning’ and ‘end’ of the sensorimotor transformation, we turn our attention to the transformation itself, specifically the question of how distinct sensory pathways eventually converge onto a common motor outputs. First, we identified convergent cells that respond to multiple stimuli. For example, to produce a convergence map between phT and OMR stimuli, we selected cells that ranked highly on both phT and OMR stimulus tuning maps (Fig. 4a,b; see Methods and Fig. S4a-b). We found that phT-OMR convergent cells were concentrated in the anterior hindbrain (aHB), specifically in rhombomere 1 and the medial area of rhombomere 2. Interestingly, a small number of cells were tuned to phT and OMR in opposite directions, so we classified cells as responsive to either to phT-only, OMR-only, both phT and OMR in the same directions (“congruent”), or both in opposite directions (“incongruent”) (Fig. 4b, Fig. S4d). To identify convergent cells across all stimulus types, we performed a similar analysis for every pairwise combination of stimuli (Fig. 4c, Fig. S4e, see legend for definition of congruent/incongruent in each case) and calculated the proportion of convergent cells (Fig. 4d, see also Fig. S4c). We observed that phT and OMR share the most convergent cells, while looming shares the fewest convergent cells with all other pathways. This may be reflective of the fact that phototactic (and potentially also OMR) stimuli elicit navigational or exploratory swimming (Chen and Engert, 2014), while looming stimuli evoke escape responses. For all simulus pairs, convergent cells were found mostly in the midbrain (mostly pretectum and optic tectum) and anterior hindbrain, and congruent convergent cells significantly outnumber incongruent (Fig. S4c). Within the pretectum and optic tectum, areas known to process visual stimuli, it is not surprising that many cells exhibit convergent tuning to multiple stimuli, especially because the stimuli themselves share overlapping visual features. However, the high density of convergent cells in the anterior hindbrain (Fig. 4f), specifically restricted to Rhombomeres 1 and 2 (henceforth abbreviated aHB(1-2)), is suggestive of a specific role for this brain area in transforming sensory input into behavior. This area is likely homologous to a subregion of the mammalian pons based on broad divisions of the embryonic brain (Kandel et al., 2013).

Figure 4. Multi-stimulus integration.

(a) Average map across fish for the intersection of phototactic responsive cells and OMR responsive cells (top 2% of cells by rank). Multi-stimulus responsive cells are concentrated in rh. 1 and the medial stripes of rh.2. (b) Whole-brain regressions were performed to a set of regressors that include phT only, OMR only, phT&OMR in the same direction (congruent), and phT&OMR in opposite directions (incongruent). Cells were classified by their best regressor. Average functional activity of cells associated with each regressor is plotted (ΔF/F, mean±SD). (c) Multi-stimulus convergence maps for all stimulus pairs (top 5% of cells by rank). Cells are colored based on their direction tuning and congruence. For directional stimuli (phT, OMR, loom), congruent cells are tuned to both stimuli in the same direction, while incongruent are tuned to opposite directions. Incongruent cells were labeled according to the direction of the stimulus listed first. For pairs involving the dark flash stimulus, congruent cells were defined as tuned to whole-field dark (which elicit large angle turns), while incongruent cells were tuned to whole-field bright. (d) Quantification of the number of convergent cells for each stimulus pair. Number and shading indicate proportion of cells that are highly tuned (top 5%) to either stimulus. (e) Illustration of two alternative hypotheses for convergence of multiple stimulus pathways. Left - direct motor convergence: information from non-overlapping visual representations (e.g. phT, OMR) directly feeds into premotor systems, which then compete to produce different behaviors. Right – sensory convergence: different visual representations first feed into a behavior-centric visual representation before affecting motor circuits. The present results support the second model, with the anterior hindbrain containing the behavior-centric, convergent visual representations. (f) Distribution of convergence cells for different brain regions along the anterior-posterior axis. Percentages are normalized to all convergence cells, shown for all stimulus pairs. Significant numbers of convergent cells are found in the diencephalon, midbrain, and anterior hindbrain rhomobomeres 1&2 (aH(1-2)). For phT/OMR and OMR/DF stimulus pairs, the average density of convergent cells is highest in rhombomere 1. Telen: telencephalon. (g) Scatter plot of convergent activity in a 2-dimensional sensory-motor space (Methods). Red points: convergent cells, defined as the intersection of the top 5% of cells ranked by phT and OMR regression. Blue points: top cells ranked by sensory component (same number of points as red). Green points: top cells ranked by motor component (same number of cells as red). Top and right: histograms of motor and sensory components, comparing distribution of convergent cells (red) to the most motor or sensory-related cells, respectively. (h) Cellular activity for 3 example neurons shown in (g) during phT and OMR stimulus blocks. (i) Average anatomical maps (n = 11 fish) showing location of the cells represented in (e): top-ranking convergent cells (left map, red), sensory-related cells (middle map, blue) and motor-related cells (middle map, green), and merge (right map, overlap between sensory-related and convergent appears purple. Note significant overlap in the anterior hindbrain (appears purple). Boundaries for rhombomeres 1,2, and 3 are overlaid. Scale bars, 50 μm.

In addition to where different sensory pathways merge into similar behaviors, we are also interested in how such integration is achieved. Specifically, we tested two different models of multi-stimulus convergence: a direct motor convergence model, in which the first convergent cells are part of the motor system (Fig. 4e, left), and a sensory convergence model, in which the first convergent cells are distinct from and upstream of the motor system (Fig. 4e, right). Although the sensory convergence model is more complex, it may allow for more flexible computations - for example, convergent cells may be inhibited by more urgent motor commands or otherwise modulated without affecting core motor circuits. A key difference between these models is that the sensory convergence model predicts a population of convergent neurons whose activity does not closely follow the motor output. To test which model of convergence more accurately describes sensorimotor transformations in the larval zebrafish brain, we plotted the activity of highly convergent cells (top 5%) in a two-dimensional sensory-motor space (Fig. 4g, red dots, phT/OMR convergence shown as example, see Methods), and compared them to the most sensory-related (blue) and most motor-related (green) cells. We observed that the convergent cells have activity that is highly sensory-related but only weakly motor-related (Fig. 4h). Indeed, this observation was consistent across multiple fish, for both left and rightward stimuli (Fig. S4f,g,i), suggesting that most convergent cells convey sensory information. We also compared the anatomical distribution of convergent cells to the most sensory and motor-related cells (Fig. 4i, see also Fig. S4h), which revealed prominent overlap between convergent cells and the most sensory-related cells in aHB(1-2).

Taken together, these results demonstrate the presence of sensory convergence neurons concentrated in the aHB(1-2) brain region. We hypothesize that sensory convergence neurons in aHB(1-2) play a critical role in sensorimotor transformations, processing sensory signals from the midbrain into turning biases that instruct dedicated motor regions in the more posterior hindbrain segments. The anatomical location of the sensory convergence cells, bordering the tectum, cerebellum and motor nuclei such as the ARTR (Dunn et al., 2016b), makes it well-suited to form synaptic connections with both dedicated sensory and motor regions. As a preliminary test of the causal role that these sensory convergence neurons play in biasing behavioral output, we performed optogenetic stimulation of cells in the aHB(1-2) while measuring fictive output in fish expressing the opsin CoChR (Klapoetke et al., 2014) under the elavl3 promoter (Fig. S4j-m). Consistent with the sensory convergence model, we found that stimulating one side of the aHB(1-2) increased turning to the ipsilateral direction, but did not increase overall swim frequency. While additional experiments targeting functionally-identified subpopulations will be required to understand in more detail the role played by this sensory convergence area, our whole-brain imaging experiments have constrained possible circuit models and can precisely guide future investigations.

Whole-brain functional clustering

Although sensorimotor systems are central for understanding the functional organization of the brain, many neurons in the brain exhibit coherent activity that is not strongly related to either the experimental stimuli or measured motor outputs; furthermore, meaningful activity correlations between neurons are present even in the absence of stimuli (spontaneous activity). To address these limitations of our methodology and catalog broader brain-wide activity patterns, we developed an unsupervised, density-based, agglomerative clustering algorithm that groups neurons into functional clusters based on their activity alone (regardless of their anatomical location) (Fig. 5a,b see also Methods and Fig. S5a). Applying the clustering algorithm the full datasets (all stimulus blocks) generated ~100-150 functional clusters for each fish, and each major brain area contained at least a few clusters (Fig. 5b). Identified clusters not only included cells tuned to stimulus or motor activity, but also cells uncorrelated to both (Fig. 5c,d, Fig. S5j,k; Methods). We found that the number of cells included in each cluster spanned three orders of magnitude, but activity within each cluster was nevertheless highly concerted (Fig. 5e,f; examples in Fig. 5a). Many of the clusters also exhibited substantial correlations with other clusters (Fig. 5g), so we performed hierarchical ordering of the cluster centers to visualize the relationships between clusters (Fig. S5b,f). The blocks or branches of this hierarchy can reveal additional functional organization on broader scales; for example a prominent group of related clusters all consist of neurons in the olfactory bulb (Fig. 5g, red block, see also Fig. S6f).

Figure 5. Whole-brain functional clustering.

(a) A diverse collection of automatically identified functional clusters from one example fish. Clustering was performed on activity data from all stimulus blocks together. After whole-brain clustering, the most dissimilar clusters were selected based on their hierarchical ranking for visualization. a1: functional activity profiles: normalized ΔF/F for each cell is plotted along the horizontal axis. a2: corresponding anatomical map. (b) Full set of individual clusters as z-projections (from the same fish). All clusters are shown in batches of 6 on duplicate z-projections for clarity. (c) Clusters for an example fish ranked from stimulus-related (periodic, as in Fig. 3c, red) to not stimulus-related (aperiodic, purple). (d) Clusters for an example fish ranked from motor-related (high regression coefficient to motor res., as in Fig. 3f, red) to not motor-related (low regression coefficient, purple). (e) Histogram of size distribution of all clusters, pooled across fish (mean±SEM). (f) Histogram of average correlation between cells within clusters, pooled across fish (mean±SEM). (g) Correlation matrix of all clusters from a single fish, ordered so that most similar clusters are adjacent. Submatrix indicated by red bars highlights putative olfactory bulb neurons (see Fig. S6f). (h) Visualization of all clusters within one fish with t-SNE. For color assignment, the total of 139 clusters (6,499 cells) were ordered by hierarchical clustering, and adjacent shades of hsv colors are assigned based on the resulting leaf order. Gray points represent cells that did not pass the clustering criterion i.e. not assigned to any cluster (1:10 down-sampled for clarity). (i) Two-fold cross-validation (as in Fig. S5e) between multiple sets of stimuli within each fish (n = 6 fish), with scores indicating the fraction of cells in matched clusters over the total number of cells. Each fish in the analysis has been presented with all 5 different stimuli. The “all” category uses the combined data from all 5 stimulus periods. (j) Results of the automatic clustering algorithm applied to only the 'spontaneous' condition (no stimulus presented), for an example fish. Only cells that passed the two-fold cross-validation test are shown. (k) Histogram of average within-cluster anatomical distance, pooled across fish (mean±SEM). (l) Examples of clusters that are anatomically isolated. (m) Examples of anatomically dispersed clusters that are difficult to identify based on anatomical location, as they are intermixed with other clusters. (n) Average map of clusters conserved in anatomical space. Each of these clusters are selected for having anatomically corresponding clusters in at least 6 other fish (out of 18 fish, see Methods). Clusters are ranked and colored within fish as in (k). (o) Putative abducens nucleus (ABD) network for the control of eye-movements. Arrows: anterior and posterior clusters of the ABD map to rhombomeres 5 and 6, respectively. Arrowheads: oculomotor nucleus (OCM) clusters. (p) Illustration of the eye-control circuit. Red/blue indicates control of rightward/leftward eye movement. (q-r) Two-dimensional sensory/eye-movement mapping (similar to Figure 3f, except the average response of the ABD were used as eye-movement regressors instead of the tail-movement motor output). q: Analysis of the sensory and eye-movement activity of one example fish. Horizontal axis: stimulus component (square root of variance explained by periodic component of activity); vertical axis: motor component (regression coefficient with left eye-movement res., as in Fig. 3d); Top 1000 motor-ranked cells are plotted in colors, corresponding to (r). r: Anatomical map of cells shown in (q). See Fig. S6g for corresponding figures for right motor res. Scale bars, 50 μm. Dien: Diencephalon. OTec: optic tectum. aHB: anterior hindbrain. ABN: abducens nucleus. CN-X: cranial nerve X. ARTR: anterior rhombencephalic turning region. pTec: pretectum.

Unsupervised clustering faces the challenge of the lack of ground truth for validating the derived clustering structure. To address this, we first examined our clustering result using t-SNE (Van Der Maaten and Hinton, 2008), a visualization method for high-dimensional data in which the functional activity of each cell is represented by a point in 2 dimensions, and local proximity of points reflects functional similarity between cells (Fig. 5h). In the t-SNE plots, functional clusters identified by our clustering method appeared as isolated islands in the periphery (dots colored according to cluster membership), while unclustered cells (i.e., with noisy or idiosyncratic activity below the density threshold) congregated in the center (grey dots). This suggests that our clustering results and the choice of clustering threshold are generally appropriate. To more systematically evaluate the clustering results, we performed a two-fold cross-validation analysis, and found that 76% of neurons are consistently clustered from both halves of the time-series data (Fig. 5i, top left; for details see Methods and Fig. S5e,n). Interestingly, the ARTR, which has been shown to modulate exploratory swimming in the absence of stimuli (Dunn et al., 2016b), is more prominentently clustered during spontaneous conditions than under stimulus driven conditions (Fig. 5j compared to Fig. S5o). We utilized a similar cross-validation approach to investigate how clustering varies across stimulus types, quantifying what percent of neurons retain the same cluster membership for different stimulus blocks (Fig. 5i, see also Fig. S5p). We found that phT and OMR produce the most similar clusterings, while looming clusters are the most distinct from other stimuli, consistent with the earlier observations (see Fig. 4d). Intruigingly, clustering obtained from the spontaneous periods (no stimulus presented) matched significantly (≥50%) with the phT, OMR, and DF conditions. This suggests that about half of the functional clusters can be reliably detected in diverse stimulus contexts (including no stimulus), and perhaps reflects the role that underlying anatomical connectivity plays in shaping functional activity.

Since the functional clustering uses no anatomical knowledge as input, emergent anatomical patterns in the clusters can reveal relationships between brain-wide structure and function. We observed that most clusters are anatomically compact rather than dispersed (Fig. 5k and Fig. S5c), suggesting that functionally related neurons are often organized into small brain nuclei (Fig. 5l). However, the spatial extent of clusters varies significantly, including prominent examples of dispersed clusters that are spatially intermingled with other clusters (Fig. 5m). One might expect that many clusters would appear in similar anatomical locations across different individual animals, but it is also possible that some functional clusters are absent or appear at different locations in different animals. To investigate this, we screened for clusters that have a similar anatomical counterpart in several individual fish, and found that about half of all clusters are anatomically conserved (Fig. S5h; see Methods and Fig. S5g for criteria). An anatomical map of these conserved clusters (Fig. 5n) shows that conserved activity is found most densely in the anterior hindbrain and forebrain, but is conspicuously absent in the optic tectum, consistent with previous observations that stereotypy in the tectum is low (Portugues et al., 2014; Randlett et al., 2015; Romano et al., 2015). The functional clusters identified here can, in some cases, be confidently mapped to homologous mammalian brain areas (e.g. the clusters in the abducens, inferior olive); in other cases (e.g. aHB, ARTR) no direct homologues are known, but can be roughly mapped to larger brain regions based on anatomical landmarks (such as rhombomeric position). More detailed homology may soon be obtained via brain atlases, which continue to be developed and updated (Marquart et al., 2015; Randlett et al., 2015; Ronneberger et al., 2012).

Taken together, the functional and anatomical features of the clusters can give insight into the underlying neural circuits. To further investigate circuits of interest, regression and clustering approaches can be flexibly and interactively combined with anatomical knowledge to produce circuit hypotheses (Fig. S6a). Indeed, we identified a rich variety of circuits, which are discussed in the supplemental text accompanying Fig. S6, including a jaw and gill movement control circuit (Fig. S6b), mesencephalic locomotion-related region (Fig. S6c), the raphe nucleus and the vagus cranial system (Fig. S6d,e), the olfactory bulb (Fig. S6f), and an eye-movement control circuit (Fig. 5o,q,r, Fig. S6g-k). These circuits are included as interesting examples but are by no means an exhaustive list of what can be identified in our data. To comprehensively mine this data for functional circuits, we encourage readers to continue to explore the data further on their own, utilizing our open-source code (Data and Software Availability).

As a verification of this exploratory approach, here we briefly describe characterization of an eye movement control circuit. Two pairs of anatomically compact clusters in the ventral hindbrain in rhombomeres 5 and 6 consistently appear in the whole-brain clustering (Fig. 5o, arrows), standing out because their within-cluster activity is highly correlated (many correlations exceed 0.8, see Fig. S6k), but not similar to either the stimulus or the fictive swimming behavior (Fig. S6h,i,j). The anatomical location of these clusters matches the abducens nuclei (ABD), which contains motor neurons responsible for generating coordinated eye movements (Cabrera et al., 1992; Gestrin and Sterling, 1977). We also observe a pair of small clusters near the anterior border of the hindbrain that are tightly correlated to contralateral ABD clusters, which we identified as the oculomotor nuclei (OCM) (Fig. 5o, arrowheads) (Higashijima et al., 2000). To characterize the functional organization of this circuit in more detail, we mapped cells with OCM-related activity in a sensory/eye-movement space, along with their anatomical locations (Fig. 5q,r, see also Fig. S6g, Methods). In addition to the OCM and ABD clusters, we observe clusters of cells in the more dorsal hindbrain with more sensory-related activity (Fig. 4r, circled). We speculate that these neurons could be responsible for the integration of sensory inputs into the eye-movement control circuit (Fig. 5p). Using a combination of regression and clustering analyses on whole-brain activity data, we have been able to verify a circuit for which we have neither the motor output (eye-tracking) nor a robust stimulus drive (stimuli used here do not strongly drive eye-movement). Used in this way, whole-brain clustering can serve as a cellular-resolution screen to direct further investigations into specific neural circuits.

Discussion

In this study, we generated a multi-animal, whole-brain, cellular-resolution dataset incorporating multiple stimulus paradigms and fictive motor output, and developed extensive analyses aimed at understanding the functional organization of the larval zebrafish brain. Previous studies have characterized functional activity related to specific stimuli, e.g. phototaxis, OMR, looming, dark flash, and spontaneous behavior, but have not examined the relationships between different stimulus processing pathways. Simply comparing across different experiments cannot distinguish convergent cells (single neurons that respond to multiple stimuli) from a mixture of unimodal cells in close proximity, but by combining multiple stimuli in the same experiments, we have been able to unambiguously identify convergent cells. We discovered that rhomomeres 1-2 of the anterior hindbrain (aHB(1-2)) contains cells with convergent activity for a large variety of stimuli, suggesting that this anatomical region plays a key role in diverse sensorimotor contexts. Furthermore, we found that many of these convergent cells were primarily tuned to behaviorally relevant sensory signals, rather than motor activity. For example, right-preferring sensory convergence cells were active during rightwards phT and OMR stimuli even when the fish did not actually turn right. We interpret this to mean that these sensory convergence cells generalize stimuli into categories based on whether the stimulus biases the fish to turn left or right, to inform, but not directly generate, turning behavior.

To avoid oversimplication of the role sensory convergence cells play in sensorimotor transformations, we should also consider several caveats. Conceptually, we have drawn a distinction between the direct motor convergence and convergent sensory models, but it is important to realize that cellular responses in the hindbrain form a continuum between purely sensory and purely motor-related activity. By identifying sensory convergence cells, we have not ruled out parallel, direct motor convergence pathways. In fact, it is likely that such pathways exist, especially for looming escape responses, which have been shown to activate the Mauthner system (Dunn et al., 2016a). There are also subtle differences in the anatomical organization of convergence across stimulus types, indicating that the aHB(1-2) may be composed of several subcircuits performing overlapping but distinct functions. Most notably, looming responses tend to recruit cells in the more lateral stripes, which are likely mediated by different neurotransmitter types from the medial stripes (Kinkhabwala et al., 2010), and cells mediating bright and dark-field responses appear to form distinguishable subclusters along the anterior/posterior axis.

We have generally interpreted the sensorimotor transformation to follow a broad anterior-to-posterior organization, with sensory signals processed in the forebrain and midbrain, sensory convergence taking place in the anterior hindbrain, and motor output encoded in the posterior hindbrain. This is of course a simplification and it is likely that signals flow in the opposite direction as well, especially because sensorimotor pathways are known to depend on feedback pathways related to corollary discharge (Crapse and Sommer, 2008). However, in our data we cannot easily determine the direction of information flow because the temporal resolution of our data is limited by sampling rate (2-3 Hz), and we do not know the underlying structural connectivity. Nevertheless, by utilizing optogenetics (Fig. S4j-m), we have shown that aHB(1-2) causally biases turning directions. Further investigations of these sensorimotor transformations on the circuit level would benefit from finely controlled (e.g. functionally defined single cells) perturbation experiments, high-framerate neural activity recordings using voltage indicators (Piatkevich et al., 2018), and large-scale electron-microscopy analysis of structural connectivity (Hildebrand et al., 2017; Vishwanathan et al., 2017).

While our fictive motor recordings have provided a critical handle on behavioral output, they cannot fully capture the behavioral repertoire of the animals, because (1) fictive recordings are low-dimensional representations of behavior, (2) fish may behave differently in tethered and free-swimming contexts, and (3) behavioral responses are often variable from trial to trial and across animals. While we are able to decode turning directions, it is difficult in fictive preparations to reliably elicit or detect sophisticated tail kinematics. Furthermore, in motor regression analyses, we often obtained better results if we defined the motor outputs (left and right) as the activity of selected motor neurons (“motor seed”) rather than using the fictive swims themselves (Methods; Fig. S3c-d). The difference may reflect physiological differences between calcium measurements of cellular activity of motor neurons and extracellular voltages in the tail, or could be due to imperfect recording or decoding of fictive swims. Although all stimuli we analyzed elicited motor responses, the behavioral output was in general quite variable, which is consistent with previous studies in both fictive and free-swimming contexts (Bianco et al., 2011; Trivedi and Bollmann, 2013) (although behavioral responses in tethered preparations tend to be less robust). In our analysis, behavioral variability was actually a key asset, allowing us to differentiate independent motor activity from sensory-driven motor activity (Renart and Machens, 2014). Although we observed cells that were tuned only to sensory signals, we did not observe any particular brain regions exclusively correlated to motor variability (motor res., as in Fig. 3d,e). This suggests that motor variability is not generated in well-defined anatomical nuclei, but may instead arise in a distributed fashion, perhaps originating as small perturbations that are amplified by distributed neural circuitry.

The sensory convergence cells in aHB(1-2) likely play a different role than the ARTR, which has also been implicated to contain an intermediate representation for behavior, but represents behavior more directly (Dunn et al., 2016b; Wolf et al., 2017). Our interpretation is that the ARTR is an internal generator of turning biases for future swims, and is most influential in the absence of stimuli, while the aHB(1-2) specifically processes stimuli to generate turn biases. It is likely that the aHB(1-2) and ARTR are interconnected, given their anatomical proximity and overlapping roles in sensorimotor processing. Based on our observations in aHB(1-2), it is reasonable to hypothesize that sensory convergence cells also form a crucial site for integrating multiple simultaneously presented stimuli. In cases where competing sensory stimuli are presented, convergent sensory activity may critically inform behavioral choice. Although our results based on serially presented stimuli do not directly address this hypothesis, they nevertheless suggest that the sensory convergent cells in the aHB(1-2) may play a critical role in transforming competing stimuli into behavioral choice.

To extend our analysis beyond sensorimotor processing, we used functional clustering to identify concerted activity that is not strongly correlated to either sensory or motor events. Although clustering cells based on functional activity has previously employed in many different contexts, we found that tailoring the algorithm to the particular characteristics of whole-brain calcium imaging data was necessary to achieve satisfactory results. Our clustering algorithm is a variation of agglomerative clustering with additional specific features that promote robustness to noise and high computational efficiency. The algorithm has a tunable threshold regulating how correlated the activity of cells within the same cluster should be (Methods). The value of the threshold affects the clustering as follows: as the threshold is increased, the activity of cells within clusters becomes more similar, and the clusters become more robust to a cross-validation test (Fig. S5d,e), but fewer cells are included in clusters, and the number of clusters is reduced. As the threshold is decreased, more cells are included in clusters, but some anatomically distinct clusters merge together into larger, more loosely associated groups of cells. For the presented analysis, we have empirically chosen a threshold (0.7) that balances this tradeoff; however, this choice of threshold is not obviously superior to other values. Moreover, we believe that the clustering algorithm is most powerful when applied in an exploratory, interactive manner. Rather than presenting exhaustive results by varying clustering threshold and other parameters, we instead encourage interested readers to explore the data themselves using our open-sourced code and datasets (Data and Software Availability).

In addition to identifying specific clusters, we observe three broad characteristics of the functional organization of the larval zebrafish brain in the whole-brain clustering results. First, a significant percentage of neurons do not correlate strongly to any clusters (those are the unclustered neurons in Fig. 5h, see also Fig. S5e), as has also been observed in other neural systems (Okun et al., 2015). On the other hand, many thousands of cells in the motor system exhibit hightly correlated activity, and it is not clear why some clusters contain so many cells while other cells have very unique activity patterns. Some of the functionally isolated neurons may be still be developmentally immature (Boulanger-Weill et al., 2017), as they are especially common in areas where neurons are added to the rapidly developing brain (Fig. S2l). Others may be computationally important despite remaining unidentified by clustering, and understanding their functional roles may require more complex analysis or perturbation experiments. Highly correlated activity may also be a feature of a developmentally young brain, and motor activity may differentiate further in adult fish to allow for more nuanced motor control. Of course it is likely that uncorrelated activity is present on timescales that are not discernable with calcium imaging.

Second, functional activity among different clusters varies gradually, and most clusters exhibit significant correlations with other clusters (Fig. 5g). Moreover, most characteristics of the clustering, including number of clusters, cross-validation score, and total number of cells included in clusters vary continuously with the clustering threshold (Fig. S5d). This suggests that the functional relationship between clusters depends on the chosen correlation scale, and that there is no clear correlation scale where brain-wide organization is most evident. As a result, the relationships between functional clusters are complex and likely best described hierarchically (Fig. S5f).

Third, the extent to which functional clusters are anatomically conserved varies across brain regions (Fig. S5h); this is consistent with previous observations of the positioning of transgenic labels (Randlett et al., 2015). It still remains to be determined what factors are responsible for this observed stereotypy/variability of the functional organization. A possible hypothesis is that some, but not all, regions of the brain are self-organized through experience-dependent plasticity, which may lead to across-animal variability on smaller anatomical scales.

We have shown that in-depth explorations of functional whole-brain activity data can generate strong hypotheses about sensorimotor processing and the broader functional organization of the brain. It is worth noting that the fast, near-simultaneous nature of light-sheet imaging is crucial for both dissection of sensorimotor processing and brain-wide functional clustering. Indeed, the residual (non-trial-averaged) activity of cells can only meaningfully be compared if activity of all cells is recorded nearly simultaneously, and functional clustering requires concurrent observations of neuron groups in faraway brain regions. The analytical methods presented here are also more generally applicable to other large-scale, high-resolution functional datasets. We have made our software platform and data openly available to facilitate further analysis and adaptation of this platform for other experimental studies. The analysis of the functional organization of brain-wide circuits is complementary to techniques of molecular phenotyping (Lovett-Barron et al., 2017), optical manipulation, electrophysiology, viral tracing, and connectomics, and in combination with these techniques, offers the exciting potential to promote our mechanistic understanding of brain-wide circuits.

STAR Methods

Contact for Reagent and Resource Sharing

Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Xiuye Chen (xiuyechen@gmail.com).

Experimental Model and Subject Details

Transgenic zebrafish, panneuronally expressing calcium indicator GCaMP6 under the elavl3 promoter and nucleus-targeted as Tg(elavl3:H2B-GCaMP6f), was used for imaging (Vladimirov et al., 2014). Transgenic larval zebrafish (5 - 7 dpf) expressing CoChR-eGFP (Klapoetke et al., 2014) under the elavl3 promoter were used for optogenetic stimulation. Zebrafish larvae (5 dpf – 7 dpf), were paralyzed with 1mg/ml alpha-bungarotoxin (Sigma-Aldrich), and embedded with 2% low melting point agarose. [Sex cannot be determined before ~ 3 weeks post fertilization (http://jeb.biologists.org/content/205/6/711)]. Zebrafish ≥ 7dpf are cared for in accordance with an automatic fish housing system at 28°C under Janelia Research Campus Policy “Guidelines for the Use of Zebrafish in Research.” Zebrafish ≤ 7dpf are kept in an incubator at 28°C. Zebrafish between 5 and 7 dpf are fed Rotifers. All experiments presented in this study were conducted in accordance with the animal research guidelines from the National Institutes of Health and were approved by the Institutional Animal Care and Use Committee and Institutional Biosafety Committee of Janelia Research Campus.

Method Details

Light-sheet imaging

Light-sheet imaging experiments were performed with an experimental setup previously described (Vladimirov et al., 2014), which achieves almost whole-brain imaging with concurrent presentation of visual stimuli and electrical recordings of fictive swimming. The imaging rate was about 2 brain volumes/s (2.11 ± 0.26 Hz), with the variability due to differences in the size of the brain for different animals (mainly because of the thickness difference). Each experiment lasted between 30 and 120 min.

Cell detection

After imaging acquisition, the time series of each plane was registered by custom-written C/CUDA software for the XY plane translation. Z-drift was corrected by first comparing patches of images with the nearby planes from the first 2 min of the recording, and then calculated by linear fitting the distance across the Z-planes. Only the parts of experiment when the XY- or Z-drift smaller than 1 um was used for further analysis.

Cells were detected from the time-averaged image. In the Tg(elavl3:H2B-GCaMP6f) line, calcium indicator was mainly localized in the nucleus and forms a bright disk on each plane, a property that facilitates neuron detection. First, GCaMP expression area was extracted by binary thresholding based on pixel intensity and local contrast. Second, each pixel was normalized locally by assigning a relative rank of intensity within a disk patch (radius = 4 μm), then further smoothed by a circular patch with radius of 1.6 μm. The center of a cell body was identified as being a local maximum point, and the calcium trace for the cell was calculated as the average over circular patch with radius 2.8 μm (Kawashima et al., 2016). The DF/F traces of identified cells are then standardized as z-scores to have zero mean and unit variance (and thus have arbitrary unit).

Fictive behavior recording

The fictive behavior setup has been previously described (Dunn et al., 2016b). Two suction glass pipettes (~45 μm inner diameter) were attached on the skin from each side of the tail. Gentle suction was applied to help the electrical contact with the motor neuron axons. These electrodes record spiking from multiple motor neuron axons, providing readout of intended locomotion (Ahrens et al., 2012; Dunn et al., 2016b; Masino and Fetcho, 2005). Extracellular signals were amplified (Molecular Devices, Axon Multiclamp 700B), fed into a computer using a National Instruments data acquisition card, and recorded by custom written C# software.

The fictive swim bouts were first detected as previously described (Ahrens et al., 2012, 2013a), then used to decode fictive turns (Dunn et al., 2016b). The extracellular signal from left and right side of the tail were recorded from two independent channels of the amplifier. The fictive swim signals were calculated as the smoothed power of the deviation from baseline. Individual swim bouts were detected automatically, and then a weighted average swim bout amplitude was used to normalize and balance the signal from left and right channel. Each swim bout was weighted by a normalized rising exponential function, to take into account the fact that turns affect the start of swim bouts more heavily than the end of swim bouts. For the same reason, to determine the fictive turn amplitude and distance, filtered and normalized fictive signals during swim bouts were weighted with a decaying exponential function (tau = [bout duration]/3) to emphasize the initial burst that determines overall turn direction. Then the turn amplitude was calculated by the difference of weighted power divided by the sum of the weighted power from both sides: (Power_Left - Power_Right) / (Power_Left + Power_Right). The sum of the weighted power was used to measure the swim distance.

Visual stimulation

Visual stimulation patterns were generated by custom-written C# software, delivered by a projector with homogenous red light, and projected to a diffuser that was stuck to the bottom of the imaging chamber. The visual stimulation consisted of serial repetitions of different sets of patterns: For phototaxis, two patterns were presented: Left-Dark / Right-Bright, and Left-Bright / Right-Dark. After each pattern a period of whole field dark was presented. The dark area slightly crossed the midline in order to motivate the fish to swim towards the bright side. For moving grating stimulation, three or four directional drifting gratings were presented: forward, backward, left-ward, and right-ward. Forward and backward gratings moved along the body axis towards the head or the tail direction. Left- and right-ward gratings were oriented 120 degrees away from the head direction, either to the left or the right side. Grating speeds were as follows: Leftwards, rightwards, backwards: 0.4mm/sec. 6 degree/sec. Forwards: 1 mm/sec, 15 degree/sec. For spontaneous stimulation, the whole field was dark and no other visual feature was presented. For dark flash stimulation, whole field dark and whole field light were presented alternatingly for 20 sec each. For looming stimulation, a bright background was presented, and a dot was expanded on either left or right side of the fish. Expansion lasted for 5 seconds, and was followed by 25 seconds of whole field bright.

Image Registration

Nonrigid image registration was done with CMTK (http://www.nitrc.org/projects/cmtk/), similar to registration done for the Z-Brain atlas (Randlett et al. 2015). Additionally, one “bridge brain” was created by imaging the same fish both with the light-sheet microscope (used for the functional dataset in this study) and a confocal microscope (used for the creation of the Z-Brain atlas). Functional light-sheet datasets were first registered to the bridge brain, and the transformations from light-sheet to confocal for this bridge brain were subsequently applied to complete the registration. To obtain the anatomical location of individual cell-ROI’s in the transformed coordinates, we used the CMTK built-in command “streamxform”.

MATLAB GUI

A graphical user interface (GUI) was written in MATLAB® for interactive data visualization and analysis. For each fish, the functional data loaded consisted of one calcium trace per each segmented cell, calculated as change in fluorescence ΔF/F). Also loaded were the annotated stimuli, fictive behavior, anatomical location of segmented cells (both raw and registered), anatomy stacks, and annotated Z-Brain masks. The GUI integrates most analyses used in this study, including but not limited to manual cluster selection, selection based on anatomy, set operations, regression methods, unsupervised clustering methods, storage of clusters, integration with the Z-Brain atlas, and various visualization and export options. The software was written and tested with MATLAB 2016a and running on Windows 7.

Automated Functional Clustering algorithm

This algorithm was custom developed to suit this dataset, and the code is available as part of the GUI. We outline the algorithm below:

1. Divide all cells into “functional voxels” (~10 cells each)

   1. Perform k-means clustering on all cells (k=20)
   2. Perform k-means clustering on outputs of (a) (k = ~400)
   3. Discard any cells whose correlation with the voxel average activity is less than $THRESH
   4. Discard any voxels with fewer than 5 cells
2. Merge voxels into clusters based on density in functional space

   1. for each pair of voxels ij (starting from most correlated): if the correlation between voxel i and j is greater than $THRESH, and the correlation between the the voxel j and the centroid (average) of the cluster containing voxel i is greater than $THRESH: then group voxel j in the same cluster as i.
   2. discard any clusters with fewer than 10 cells
3. Clean up clusters using regression to cluster centroids

   1. for each cell k: if the correlation to the closest cluster’s centroid is greater than $THRESH: include cell k in that cluster
   2. Discard any clusters with fewer than 10 cells
4. Iterate merge and cleanup steps

   1. Perform step 2 and 3 once more, using clusters as input voxels.

This clustering algorithm can either be applied to all cells in the brain or a chosen subset of interest, and the correlation threshold determining clustering stringency ($THRESH) can be adjusted to trade-off completeness and accuracy (see Results and Fig. S5d). For most analysis in the text, the value of $THRESH was 0.7.A whole-brain single-cell resolution data (~100,000 cells, ~5000 time frames) can be clustered on a standard desktop computer on the timescale of minutes. The algorithm is sensitive enough to detect even very weak functional clustering patterns in the data, including artefactual signals resulting from the scanning laser itself (Fig. S5i, these artefactual clusters were thereafter excluded from analysis).

Clustering Cross-Validation

We divided the data into two halves along the time dimension; where multiple stimulus repetitions were present, we used an equal number of repetitions for the two halves. We then use the Hungarian method (Munkres assignment algorithm) to match clusters produced from the two halves of the data based on the fraction of common cells contained in a pair of clusters. This matching index was also visualized as the total “mass” distributed along the diagonal entries (Fig. S5e). The cross-validation coefficient (Fig. 5i) is the fraction of cells that were assigned membership to the same cluster for both halves of the data.

Screen for anatomically conserved clusters

We used the following criteria to determine whether two clusters were considered occupying comparable locations in anatomical space. First, given a pair of clusters, we calculated the distance of all pairs of cells within each cluster (sets d₁ and d₂), and the distance of all pairs of cells between the two clusters (d₁₂). If the relative distance mean(d₁₂) / min(mean(d₁), mean(d₂)) was less than 2, we considered the clusters conserved (with some restrictions on cluster sizes to exclude outliers). For each cluster in a given fish, we compared it with all clusters in all other fish. If there were at least 6 other fish that contained at least 1 matched cluster based on the criteria above (multiple matching clusters were possible), this cluster was marked as a conserved cluster over the population (Fig. 5n).

Stimulus and motor regressors

The set of stimulus regressors that represent different stimulus features (Fig. 2d-h) were generated by convolving box-car regressors with a single exponential kernel for the calcium indicator GCaMP6f (half-time 0.4 sec, peak delay 0.08 sec).

Motor regressors were generated from recorded fictive behavior (forward, leftward and rightward swimming) convolved with the temporal filter of the calcium indicator (exponential function, half-time 0.4 sec, peak delay 0.08 sec). Though sufficient for some analyses, this simple convolution may not fully account for the relation between the electrophysiological recording from the tail and the nuclear-labeled calcium trace. It is also possible that in some individual fish, the recording was not stable for the entire duration of the experiement. For some regression maps, we found that the results can be improved by using “motor seed” traces as the behavior regressor. The motor seeds are a small number of neurons manually selected in each fish based on two criteria: (i) the neurons have the highest correlations to the fictive behavior and (ii) they are located in the region of the hindbrain that is known to send output signals to drive swimming behavior (Fig. S3c-d). Motor seeds were calculated for all stimulus blocks. The average activity traces of the motor seed cells were denoted “motor outputs” in the manuscript (as opposed to “motor regressors”). In analysis for Fig. 3a,b, the fictive motor regressors were used, whereas in Fig. 3e-g and Fig. 4, the motor outputs taken from motor seeds were used.

2D sensory – motor regression

In several analysis (Fig. 3e, Fig. 4g, Fig. 5q), functional activity of cells is plotted in a 2D activity space. In these plots, each dot represents a cell, and it’s location in x and y are determined by various quantifications of the cell’s activity.

Fig. 3e: x-axis: stimulus-independent motor - regression coefficients with respect to left motor res. regressor. Plot was calculated using all stimulus blocks together. y-axis: stimulus-driven motor - regression coefficients with respect to motor avg. regressor.

Fig. 4g (also Fig. S4f,g): x-axis: stimulus-independent motor - regression coefficient of cellular activity with left motor res. The maximum value from phT or OMR blocks was used. y-axis: sensory component - square root of the variance explained by periodic component of activity. The minimum value from phT or OMR blocks was used.

Fig. 5q (also Fig. S6g): x-axis: sensory component - square root of variance explained by periodic component of activity. y-axis: eye-movement component - regression coefficient with left motor res. of ABD activity. Similar to Fig. 4g except ABD activity is used as the motor regressor instead of fictive traces from the tail.

Decomposing activity into trial averages and trial residuals

For any trace, we index its value at trial k and time t (relative to the beginning of the trial) as x_t,k. Its trial average component is defined as $x_{t,k}^{Avg}=x_t^{Avg}=\frac{1}{n}{\sum }_{j=1}^n{x}_{t,j}$ (n is the number of trials; we are using both subscripts t, k to indicate that $x_{t,k}^{Avg}$ is a vector of the same length as the original x_t,k, although its value does not depend on k.), and its trial residual component is $x_{t,k}^{Res}=x_{t,k}-x_t^{Avg}$. Importantly, x^Avg and x^Res are orthogonal, because ${\sum }_{t,k}{x}_{t,k}^{Res}{x}_{t,k}^{Avg}={\sum }_{t=1}^T{\sum }_{k=1}^n\left(x_{t,k}-x_t^{Avg}\right)x_t^{Avg}=0$ (T is the period of the stimulus). In fact, x^Res is orthogonal to any trace that is periodic to the stimulus, according to the same argument. In particular, let S represent the subspace of all periodic traces locked to stimulus trials, then x^Avg is the orthogonal projection of x in S.

The sensory periodicity (as in Fig. 3e and Fig. 4g) is calculated as the square root of the fraction of the variance of x^Avg relative to x. The square root is taken so that the resulting index is comparable in scale to correlation coefficients (e.g. consider the correlation between the trial-average and the original trace) that we use in other analyses such as the correlation to motor-residuals and motor-averages.

When applying the decomposition into trial-average and trial-residual to motor traces, the orthogonality of the decomposition removes the correlation between motor and stimulus, and regressing to motor-res hence improves the identification of motor related neurons as used in Fig. 3 and 4.

As shown above, the motor-res is orthogonal to S. This means that (correlation to motor-res)² + (sensory periodicity)² = the fraction of variance of x within the joint subspace spanned by {motor-res, S} ≤ 1 . Note that the subspace spanned by {motor-res, S} is the same as that spanned by {motor, S}. So the amount of remaining activity $\sqrt{1-(\text{correlation to motor-res}{)}^2-(\text{sensory periodicity}{)}^2}$ represents activity beyond sensory and recorded motor behavior.

Selection of stimulus-locked functional clusters

This analysis is performed in the GUI for single fish, for each stimulus condition, and the curated selection of clusters are presented in Fig. 2h-k. The analysis sequence is as follows. We (1) rank all cells in the brain by their “periodicity” (average standard deviation between stimulus repetitions, for z-scored traces) to select the top periodic cells (a good range is top 5%-10%); (2) divide into a relatively large number of clusters (k-means with k=20); (3) rank by motor regression (correlation) coefficient (highest coefficient of either left/right motor output), and discard the clusters that have relatively high correlation to motor (threshold varies slightly between animals); (4) rank these k-means clusters based on average correlation of cells within a cluster, and discard (outlier) clusters with significantly less concerted activity patterns; (5) rank the remaining clusters hierarchically, and merge (or discard) highly similar clusters. Similar to the whole-brain clustering algorithm, the above steps are not designed to find the globally optimal clusters, but are meant to provide qualitative insight into the characteristic types of functional profiles present in the brain.

Optogenetic Stimulation Experiments

Transgenic larval zebrafish (5 - 7 dpf) expressing CoChR-eGFP (Klapoetke et al., 2014) under the elavl3 promoter were used for optogenetic stimulation. The fish line was created using the Tol2 system (Urasaki et al., 2008). The fish were placed into a fictive behavior recording setup similar to that described above, but underneath a two-photon microscope instead of a light-sheet microscope. To evoke consistent swimming, a slow forward grating was presented at all times with closed-loop visual feedback. Neurons expressing CoChR-eGFP were visualized through 2-photon imaging (920 nm excitation) of the eGFP tag, and the anterior hindbrain (aHB) was identified using visual inspection and the functional maps. A region of the aHB(1-2) (a 50 μm × 50 μm plane) inside the left/right responsive multi-stimulus clusters (Fig. 4) either left or right of the midline were stimulated by line-scanning with a Ti:S laser tuned to 1030 nm (Coherent Chameleon Ultra II). A dwell-time of 5 ms per neuron at a stimulation rate of 2 Hz was used. Turn direction (see Fictive behavior recording) and swim frequency during periods of forward grating and optogenetic stimulation was compared with turn direction and swim frequency during periods of forward grating without optogenetic stimulation. Recordings lasted 5-6 minutes per animal.

Quantification and Statistical Analysis

Fig. 2a, bottom: mean±SD dF/F plotted for cells with regression coefficient > 0.5

Fig. 2b-e: cells with regression coefficient > 0.5 plotted. Multiple fish superimposed.

Fig. 2h-k: left: mean±SD dF/F plotted for cells with regression coefficient > 0.5. right: cells with regression coefficient > 0.5 plotted. Single fish examples

Fig. 3a: cells with regression coefficient > 0.5 plotted, single fish example

Fig. 3b: cells with regression coefficient > 0.5 plotted, n=11 fish superimposed

Fig. 3c, bottom: mean±SD dF/F plotted for cells with regression coefficient > 0.5

Fig. 3e, maps: top 2% of cells by regression rankings plotted for each fish, n=17 fish superimposed

Fig. 3e, scatter plot: each cell represented by a point. x value is regression coefficient for motor res. for that cell. y value is regression coefficient for motor avg. for that cell. Data for a single example fish is shown, with motor regressors derived from the left-side motor output. See methods for definition of motor output, motor avg. and motor res.

Fig. 3f,g: top 2% of cells by regression rankings plotted for each fish, n=17 fish superimposed

Fig. 4a: phT and OMR maps show top 3% of cells by regression coefficient ranking. Convergence map shows cells included in both phT and OMR maps.

Fig 4b: Cells that have a correlation coefficient > 0.4 to at least one of the regressors shown are classified by their best regressor. mean±SD dF/F plotted for cells all taken from the same fish. n refers to the number of cells included in each category.

Fig. 4c: Convergence maps show cells included in top 5% rank for both stimuli.

Fig. 4d: Number of convergent cells divided by number of cells that are highly tuned (top 5%) to either stimulus.

Fig. 4f: Number of convergent cells (defined in Fig. 4c) found in different brain areas for n=18 fish. mean±SEM shown in red.

Fig. 4g: scatter plot: each cell represented by a point. x value: regression coefficient for motor res. for that cell. y value: square root of the variance explained by the periodic component of the cell’s activity. Data for a single example fish is shown, with motor regressors derived from the left-side motor output. Red points: convergent cells, defined as intersection of top 5% phT and OMR regression maps. Blue points: top cells ranked by y value, same number as red cells. Green points: top cells ranked by x value, same number as red cells. Top: histogram of x values for red and green cells. Right: histogram of y values for red and blue cells.

Fig. 4i: Maps of red, green and blue cells (defined as in Fig. 4g), superimposed for n = 11 fish.

Fig. 5e: Histogram of number of cells included in each cluster, mean±SEM across n=18 fish plotted for each bin

Fig. 5f: Histogram for average correlation between cells for each cluster, mean±SEM across n=18 fish plotted for each bin

Fig. 5g: Matrix of Pearson correlation coefficients between pairs of clusters.

Fig. 5h: t-SNE for one example fish.

Fig. 5i: Two-fold cross-validation showing fraction of clusters that pass a cross-validation test. See Methods for details of cross-validation. Average fraction of clusters shown for n=6 fish.

Fig. 5k: Histogram of average anatomical distance between cells for each cluster, mean±SEM across n=18 fish plotted for each bin

Fig. 5n: Map of clusters that are conserved for at least n=6 fish. Conserved clusters are superimposed for a pool of n=18 fish. See methods for details of definition of “conserved”.

Fig. 5o: Map of ABD clusters, n=17 fish superimposed.

Fig. 5q: Each cell represented by a point. x value: square root of the variance explained by the periodic component of the cell’s activity. y value: regression coefficient for motor res. for that cell.

Data and Software Availability

Software and instructions for downloading data are available at www.github.com/xiuyechen/fishexplorer.

KEY RESOURCES TABLE

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Chemicals, Peptides, and Recombinant Proteins
Alpha-Bungarotoxin	Sigma-Aldrich	B137
Deposited Data
Functional data (calcium traces of all ROI’s with metadata) from all n=18 fish, formatted for the GUI	This study	www.github.com/xiuyechen/fishexplorer
Experimental Models: Organisms/Strains
Tg(elavl3:H2B-GCaMP6f)	(Vladimirov et al., 2014)	N/A
Tg(elavl3:CoChR-eGFP)	(Klapoetke et al., 2014)	N/A
Software and Algorithms
MATLAB (Behavioral and imaging analysis, modeling)	MathWorks	https://www.mathworks.com/products/matlab.html
C# (.NET Framework 3.5) (stimulus/behavior)	Microsoft	https://docs.microsoft.com/enus/dotnet/csharp/
Computational Morphometry Toolkit (CMTK)	NITRC	http://www.nitrc.org/projects/cmtk
Custom GUI “FishExplorer”	This study	www.github.com/xiuyechen/fishexplorer
Cell detection	(Kawashima et al., 2016)	N/A

Highlights:

• Sensory input drives behavior via distributed circuits in larval zebrafish

• Activity from nearly all neurons in the brain were recorded in behaving animals

• Convergent representations of diverse visual stimuli inform behavioral choices

• Unsupervised clustering reveals patterns of brain-wide functional organization

Chen et al. examine brainwide functional organization in larval zebrafish under diverse visual stimulus conditions. They systematically characterize neurons related to convergent sensorimotor processing as well as extract concerted brain-wide activity patterns beyond sensorimotor contexts.