Sparse Component Analysis: A method that uncovers separable computations within neural population activity

Abstract

In many neural populations, the computationally relevant signals are posited to be a set of ‘latent factors’ – signals shared across many individual neurons. A given brain area may perform many computations, each associated with distinct factors, that can together compose an overall action. Methods for uncovering such structure typically require supervision, which can limit discovery of novel aspects of activity. Here, we introduce Sparse Component Analysis (SCA), an unsupervised approach. SCA facilitated surprisingly clear parcellations of neural activity across a range of behaviors, when seeking both linear and nonlinear embeddings. We applied SCA to motor cortex activity from reaching and cycling monkeys, single-trial imaging data from C. elegans, and activity from a multitask artificial network. SCA revealed both simple and unexpected instances where the overall population response was built compositionally from sets of factors with distinct computational roles.

Graphical Abstact

eTOC Blurb

Zimnik et al provide a new, interpretable, unsupervised dimensionality approach for neural population activity. Their approach, Sparse Component Analysis (SCA) reveals latent factors corresponding to distinct computations without users needing to provide behavioral labels or known timing of computations.

Introduction

The study of computation by neural populations has experienced a conceptual shift. Traditionally, one first characterized single-neuron responses, then extrapolated to determine how the population would behave. This worked well in situations where population-level function (e.g. saccade specification) was a straightforward extension of single-neuron responses (e.g. tuning for target location)^1,2. Yet it became appreciated that this strategy must often be inverted. In many areas, single-neuron responses are complex, heterogeneous, and make sense only in the context of population-level computation.

In many biological and artificial networks, computations involve ‘latent factors’^3–10: signals that are shared across the population and consistent across trials, despite variable single-neuron spiking¹¹. Factor co-evolution can yield explanations regarding how a computation is accomplished^11–20. Estimating factors from data poses a challenge: of the many statistically valid estimates, not all provide equally interpretable views of the underlying computations. This challenge is particularly acute if a network uses multiple sets of factors to instantiate multiple sub-computations, deploying them flexibly and/or compositionally. Such strategies appear likely to lie at the heart of intelligent flexible behavior. There is thus a need for a method that can identify, in mixed single-neuron responses, factors corresponding to distinct computations.

Motor-cortex activity during delayed-reach tasks provides a canonical example (Fig. 1A). Most neurons are active during both delay and movement, with no clear relationship between these activity patterns^21–24. That relationship becomes clear only after identifying two distinct factor-sets: ‘preparatory factors’ (active during the delay; Fig. 1A, left) and ‘execution-related factors’ (active while reaching; Fig. 1A, right)^25,26. This observation enabled experiments addressing how movements are prepared²⁶, executed¹⁵, and corrected²⁷. Reaching also provides an example of compositionality: reach sequences reuse these same factors in sensible, but also sometimes surprising, ways^28,31. More sophisticated forms of computational flexibility likely also leverage compositional reuse of computations subserved by distinct factors^13,32–34.

Figure 1. SCA seeks non-synchronous factors.

A. Different factors are active during reach preparation and execution, allowing different dynamics. B. Timecourse of two idealized neural processes with flexibility resembling that of preparation and execution. Process A always precedes process B, and they always overlap, but are non-synchronous and have non-identical durations across conditions. A 50-neuron population was simulated by multiplying such ground-truth factors (one per process) by an orthonormal matrix, then adding Gaussian noise (two examples shown at right). SCA, but not PCA, recovered ground-truth factors. C. During cycling, left-arm factors (left subspace) are active when the task is performed with the left arm (purple trajectory) but not the right arm (orange dot). Right-arm factors have the complementary property. Both factor types are co-active during bimanual cycling (not shown). D. Time course of two idealized neural processes with flexibility resembling that during cycling. Neurons were simulated from these processes, and dimensionality reduction applied, as in panel B. E. SCA architecture. F. Left: Two idealized factors, active at non-overlapping times. Right: These same factors rotated by −45°. The L1 norm (SCA cost function’s second term) is shown for original and rotated factors. G. State-space view, for two timepoints, of factors from panel F. Solid/dashed lines: original/rotated factors. Black/red colors: earlier/later timepoints. H. Exploration of how the sparsity penalty interacts with factor overlap. Two factors overlap for duration $\tau$ (total duration $T=100$). For moderate overlap ($\tau =50$; top), no rotation is sparser than the ground-truth factors themselves, which can thus be recovered by seeking sparsity. For high overlap, ($\tau =80$; bottom), there does exist a rotation that increases sparsity. I. L1 norm, versus rotation of factors in H, for $\tau =0$ (no overlap) to $\tau =100$ (complete overlap). Seeking sparsity recovers ground-truth factors so long as the norm is smallest when rotation $\theta =0$. For large overlap this ceases being true: the trace crosses below the dashed line. J. Summary of results in panel I. The sparsity-maximizing rotation is zero (identifying ground-truth factors) for temporal overlap <58%.

Standard unsupervised methods (e.g. principal component analysis; PCA) are typically insufficient to discover factors with computationally distinct roles; PCA-estimated factors are frequently as ‘mixed’ as the individual-neuron responses. The field has thus developed multiple supervised approaches. One approach (used in the example above) is to divide data into temporal epochs, based on prior knowledge or assumptions, before applying an otherwise-unsupervised method^17,19,25. Explicitly supervised methods, such as demixed PCA (dPCA), leverage labels³⁵. Targeted dimensionality reduction (TDR and model-based TDR³⁶) identifies factors that covary with task-related variables¹². Methods such as jPCA¹⁵ seek hypothesized structure (e.g. factors obeying rotational¹⁵, or unconstrained³⁷ linear dynamics). These approaches cannot be applied unless one can infer the correct labels, temporal divisions, or potential forms of population-level structure.

Here, we present an unsupervised approach, Sparse Component Analysis (SCA). SCA leverages a basic premise of compositionality: distinct computations are not necessarily coactive and may exhibit temporal flexibility relative to one another, including different timings or orderings. SCA seeks factors that 1) are sparse in time, facilitating discovery of temporally flexible factors, and 2) evolve within orthogonal dimensions, facilitating discovery of factors subserving potentially distinct computations. SCA’s simple cost function reflects very general assumptions, can be applied to a wide range of datasets, and can assume linear or nonlinear factor embeddings.

We applied SCA to datasets from monkeys performing motor tasks, C. elegans during mating, and a multi-task artificial recurrent neural network (RNN). During reaching, SCA identified distinct motor-cortex factors related to reach preparation, execution, and postural maintenance. SCA identified surprisingly analogous factors during unimanual cycling, plus factors related to stopping an ongoing movement. These factors were reused compositionally across conditions. During bimanual cycling, SCA identified arm-selective factors, also used compositionally. For C. elegans, SCA identified factors, and individual neurons, related to specific behavioral motifs. In the RNN, SCA recovered factors subserving compositional computations^13,38. Thus, SCA both recovered known findings without supervision and uncovered previously unknown factor categories.

Results

Discovering compositional, non-synchronous factors with sparsity and orthogonality

We designed SCA to reflect the assumption that behavioral flexibility arises from compositionality: the combination of distinct computations (or sub-computations) in specific ways. Combinatorial flexibility may take many forms^{16,17,19,35,39–43} including order, duration, and when or whether computations overlap. By contrast, two computations with locked timecourses cannot produce compositional flexibility – indeed one probably wouldn’t consider them separable. SCA leverages the fact that ‘non-synchronous’ factors are typically more sparsely active in their ‘true’ ground-truth basis than in a mixed basis.

Consider preparatory and execution-related computations during reaching. These evolve in orthogonal dimensions with unique dynamics (Fig. 1A). Preparation must precede execution, yet exhibits temporal flexibility: preparation can be lengthy or brief²⁶, can be updated²⁷ or canceled⁴⁴ prior to execution, and can occur alone or overlap with execution of a previous reach²⁸. Figure 1B illustrates a simplified form of such flexibility: process A always begins before process B but their relative timecourses are flexible. Bimanual movements provide a complementary example (Fig. 1C). Distinct factor sets (employing orthogonal dimensions) are active when the left or right arm moves^45,46, and become co-active when both move simultaneously⁴⁷. Figure 1D illustrates a simplified version: processes A and B can occur simultaneously or alone.

We desired a method for discovering factors displaying one or more such forms of flexibility, without requiring prior knowledge regarding what factor types are present or when they are active. As discussed above, such discovery is likely facilitated by encouraging sparsity and orthogonality. Current methods can be agnostic to factor sparsity, can implicitly discourage it, or can implicitly encourage it. Methods also differ regarding whether orthogonality is enforced / encouraged. The SCA cost function directly and explicitly encourages sparsity and orthogonality:

$$\text{arg}\underset{\mathbf{U},\mathbf{V}}{\text{min}}\left(|\mathbf{W}(\mathbf{X}-\mathbf{\text{XUV}}){|}_F^2+{\lambda }_{\text{sparse}}|\mathbf{\text{XU}}{|}_1+{\lambda }_{\text{orth}}{\left|\mathbf{V}{\mathbf{V}}^{\top }-\mathbf{I}\right|}_F^2\right)$$

The first term encourages latent factors to capture response variance (minimizing reconstruction error of individual-neuron activity). Unlike PCA, SCA does not seek to concentrate variance within its initial components. The second term encourages factors to be sparse in time, within or across conditions. Identified factors may not actually be sparse, but should be sparser than if they mixed non-synchronous subcomputations. The final term encourages orthogonal neural-latent mappings (loadings), respecting the assumption that subcomputations employ orthogonal dimensions to avoid interference (this assumption can be relaxed by reducing ${\lambda }_{\text{orth}}$).

Concretely, let ${\mathbf{X}}^{T\times N}$ be a data matrix. Each column contains one neuron’s response, possibly concatenated across conditions, with $T$ total timepoints. The factors, $\mathbf{Z}$, are estimated via a mapping, $\mathbf{Z}=\mathbf{X}\mathbf{U}$, from $N$-dimensional activity space to $K$-dimensional factor space (Fig. 1E). The matrix ${\mathbf{V}}^{K\times N}$, whose rows are constrained to be unit norm, reconstructs neural activity from the factors with the goal that $\mathbf{X}\approx \mathbf{Z}\mathbf{V}$. SCA’s cost function is agnostic regarding the ‘meaning’ of different times; no labels are assigned nor are data divided into epochs. Offset terms in both mappings are omitted for brevity, see Methods. Neural activity in $\mathbf{X}$ is (optionally) weighted, via the diagonal matrix ${\mathbf{W}}^{T\times T}$, such that reconstruction when activity is strong (e.g. during execution) is not given precedence over reconstruction when activity is less strong (e.g., during preparation).

The fundamental intuition is that temporal flexibility, amongst computations, implies that estimated factors will be more sparse if each pertains to only one computation. Consider two idealized ‘ground-truth’ latent factors, active separately (Fig. 1F: times with black and red backgrounds). Consider factor-activity, at a given moment, in state space (Fig. 1G). Activity lies on the x-axis, at [1,0], when factor 1 is active. Activity lies on the y-axis, at [0,1], when factor 2 is active. If we rotate our factor-estimate, $\mathbf{Z}$, along the unit circle (dashed vectors), we can equivalently explain neural activity by counter-rotating $\mathbf{V}$. However, rotation mixes ground-truth factors, rendering them less sparse: both are active during both epochs (Fig. 1F, right). Consequently, the sparsest solution is the ground truth: each timepoint has $\left|\mathbf{Z}\right|=1$, instead of $\left|\mathbf{Z}\right|=\sqrt{2}$.

This continues to be true when ground-truth factors overlap partially, until overlap exceeds ~58% (Fig. 1I,J; Methods). This highlights a potential limitation of SCA. Penalizing sparsity cannot demix highly overlapping factors; the sparsest solution will, interestingly, be a 45° rotation where one dimension captures their commonality and another captures their differences (Fig. 1H). In practice, this limitation is acceptable and possibly desirable: when factors are mostly co-active, it may be unclear scientifically whether they are part of the same process. Answering that question will depend not on analysis, but on additional experiments testing their capacity for temporal flexibility.

Unsurprisingly then, SCA can reveal temporal flexibility, amongst factors, only when flexibility is sufficiently recruited by the experiment. When experimental design is sufficient, SCA can successfully recover ground-truth factors without supervision (Figure 1B,D; Fig. S1) even when individual-neuron responses (last column) are noisy and mixed. In contrast, PCA (third column) does not recover the ground truth; PCA is agnostic to sparsity and can tend to mix sparsely active factors (doing so can concentrate variance in the top principal components). Several other existing unsupervised approaches, including Independent Component Analysis and Nonnegative Matrix Factorization, also fail to consistently recover the ground-truth factors (Fig. S1).

SCA factors aren’t necessarily more interpretable than PCA factors, but they can be when distinct subcomputations are used flexibly or compositionally. As we will show, if such factors are not present, SCA is unlikely to ‘invent’ them; for consistently co-active factors, it is typically not possible to reduce the second cost-function term without substantially increasing the first term. A failure to find sparse factors can thus argue for rejecting the motivating hypothesis. Conversely, if SCA finds such factors, their nature (when and how they are active) may suggest more specific hypotheses that can be tested. We illustrate these points below.

Center-out reaching - SCA factors reflect distinct neural computations

Because of the established presence of distinct computations^{14,15,26,48,49} in motor cortex during center-out reaching (Fig. 2A,B), we thoroughly consider how SCA compares with other approaches in this context, then proceed more swiftly for additional datasets. Monkeys held their hand at a touchpoint and were shown one of eight targets. After a delay period, monkeys swiftly reached, holding their new posture until reward delivery. They then returned to the starting position to begin the next trial. Return reaches were self-initiated; there was no requirement to do so within a certain time.

Figure 2. SCA identifies preparatory, execution-related, and posture-related factors in motor-cortex reaching data.

A. Trial timing. B. Example trial-averaged behavior for one condition. C. Firing rates of six example neurons (one trace per reach direction). Shaded regions: median reach duration. D. Top eight PCA factors, ordered by the moment when maximum occupancy (cross-condition variance) occurred. E. Eight SCA factors (when requesting eight) ordered as in panel D. Each factor’s magnitude directly reflects its impact on neural activity, due to the unit-norm constraint on SCA’s $\mathbf{V}$ matrix. F. Outward-reach activity within three SCA dimensions. G. Return-reach activity in the same space. H. Left: For each SCA factor (rows), the fraction of total across-condition variance accounted for by each epoch (columns). Right: Mean and standard deviation of occupancy concentration (Methods; p<0.01, n=100 resampled neural populations). I. SCA and wPCA account for virtually identical fractions of neural variance for a given number of requested factors. Inset. Generalization performance (reconstructing neural activity) for held-out neurons and timepoints is very similar for SCA and wPCA. Shading indicates standard error, and overlaps heavily for SCA and wPCA.

It is established that preparatory factors become active shortly after target onset and remain relatively static throughout the delay^26,50. After the go-cue, preparatory-factor activity collapses and execution-related factors become active^25,26. It has recently become appreciated that additional factors, related to postural-maintenance^51,52, are active while holding a target³¹. A straightforward hypothesis is thus that a full trial will be composed of multiple distinct computations: preparing then executing an outward reach, maintaining posture, then preparing and executing a return reach. We asked whether SCA and/or PCA could identify this hypothesized progression without supervision. To focus on factors where activity varies with direction, we employed the common preprocessing step of subtracting the time-varying cross-condition mean from each neuron’s response^25,28.

The presence of distinct processes was apparent neither at the single-neuron level (Fig. 2C) nor in PCA-derived factors (Fig. 2D). For both, activity relating to preparation, execution, and posture were mixed. For example, the neuron shown in the top row of Fig. 2C was active during the delay period, outward reach, hold period, and return reach. Many PCA factors were also continuously active throughout most of the trial. Neural responses and PCA factors sometimes hinted at the presence of discrete processes: individual-neuron tuning changed between delay and execution epochs (e.g., Fig. 2C, top row) and some PCA factors were most active during one epoch (Fig. 2D, top row). Yet these hints are somewhat subtle.

In contrast, each SCA factor (Fig. 2E) was primarily active during only one epoch: preparation, execution, or postural-maintenance. Shortly after target onset, activity emerged in dimensions 1 and 2 and was sustained throughout the delay, a pattern typical of preparatory activity^21,26,50. Just before reach onset, activity collapsed in these dimensions but grew in dimensions 4, 6, 7, and 8. Dimensions 6–8 were occupied briefly: for approximately the same duration as the outward reach. Activity in dimensions 4 and 5 remained high as the target was held. These patterns are consistent with dimensions 1 and 2 capturing preparatory factors, dimensions 6–8 capturing execution-related factors, and dimensions 4 and 5 capturing posture-related factors. Thus, although SCA had no knowledge regarding the number of epochs or their boundaries, factor activity agreed with established and emerging beliefs.

Return reaches afford a test of the above interpretations. During return reaches, posture returns to roughly where it started. In agreement, putative posture-factor activity receded before the return reach, returning roughly to baseline. At the same time, activity reemerged in dimensions 1 and 3, likely reflecting preparation for the return reach. (That dimension 3 is occupied primarily prior to the return reach likely reflects kinematic differences relative to outward reaches, a point discussed later). Just before return-reach onset, activity of putatively preparatory factors decreased, while activity of putatively execution factors rose. Patterns of execution-related activity were ‘flipped’ between outward and return reaches, in agreement with reach-direction reversal. Results were similar for a second monkey (Fig. S2).

SCA thus confirms the known distinction between preparatory and execution factors, while also adding multiple nuances to our understanding of out-and-return sequences. First, it was recently shown that distinct posture dimensions are engaged when tasks are performed from a given experimenter-set posture⁵². SCA reveals that posture factors also track hand position across voluntary reaches (also see³¹). Factors with this property simply emerge when applying SCA, without need to specifically seek them. Had SCA been a standard method a decade ago, it would have been almost impossible not to discover this basic phenomenon. Second, present results reveal that preparation is brief before self-paced return reaches, suggesting that the standard plateau seen during the delay is likely atypical and reflects maintained readiness while awaiting an unpredictable go cue. Lastly, although SCA revealed a novel class of posture-related factors, it did not reveal ‘sequence-related’ factors, even though these certainly could have been present (e.g.⁵³) and are in other areas⁵⁴. Instead of employing novel factor-level events, return reaches simply employed events mirroring those during outward reaches. These results mesh with conclusions regarding externally-paced sequences^28,31.

SCA-factor activity also speaks to the linkage between preparation and execution. Consider activity in a state-space (Fig. 2F,G) defined by one preparatory dimension (SCA dimension 1) and two execution-related dimensions (SCA dimensions 6 and 8). Just before the outward reach, preparatory activity rotates into the plane defined by execution-related dimensions (gray plane). Activity within this plane then rotates in a consistent direction¹⁵. This view confirms prior conclusions (Fig. 7a of²⁵; Fig 2a of³³) and extends them: during return reaches (Fig. 2G), the same process unfolds in the same three dimensions. Preparatory activity is weaker (likely because return reaches are slower) and condition-order reverses (reflecting reversal of the reaches themselves). Yet the same neural events occur, with the same rotation from preparatory dimension into execution dimensions and the same rotation direction during execution.

Center-out reaching - Quantitative comparisons with PCA

The fact that each factor corresponds to a plausibly distinct process (preparation, execution, or posture) aids interpretation of SCA factors. In contrast, PCA factors (Fig. 2D) reflect mixtures of those processes. To quantitatively explore further, we compared with factors recovered by weighted PCA (wPCA, Methods). wPCA uses the same weighting matrix $(\mathbf{W})$ as SCA; the only substantial difference is that wPCA does not seek sparsity.

Seeking of sparsity did not decrease reconstruction accuracy: SCA and wPCA factors accounted for virtually identical total neural variance, including when generalizing to held-out neurons and conditions (Fig. 2I). The major difference was simply the degree to which each factor reflected a single putative computation. To quantify, we computed the cross-condition variance of activity in each dimension (its ‘occupancy’, Methods) during preparatory, execution-related, and posture-related epochs (Fig. 2H, left). For all 8 SCA dimensions, one epoch accounted for most (>60%) of that dimension’s total occupancy. Occupancy concentration was significantly higher for SCA-factors versus wPCA-factors (Fig. 2H, right, p<0.01 bootstrap test). This feature did not make SCA more ‘data intensive’ (Fig. S3).

SCA dimensions typically capture similar signals (and thus similar variance) as PCA dimensions, just in a different basis. The SCA basis is thus unlikely to ‘over sparsify’ responses. As a control, we applied SCA solely to delay-period activity, where preparatory-factor activity is generally viewed as sustained: after its initial rise, there is little known tendency towards ‘early’ or ‘late’ preparatory factors^25,26. Consequently, if SCA-identified factors approximate ground-truth factors, they should not be particularly sparse during the delay. Indeed they were not: occupancy concentration, across within-delay epochs, did not differ significantly between SCA and wPCA (p=0.68, bootstrap test).

Center-out reaching - Comparisons with supervised approaches

A desired feature of an unsupervised method is that it should perform almost as well as a supervised method that makes accurate assumptions, and better than a supervised method that makes imperfect assumptions. To find factors via supervision, we used the method of Elsayed et al.²⁵ to identify subspaces corresponding to preparation, execution, and posture. This approach divides data into three epochs types (Fig. 3A) and leverages considerable prior knowledge, including when preparation occurs with and without a delay period. If SCA naturally discovers those same divisions, each SCA factor should be a linear combination of supervised-method factors (Fig. 3C) that are restricted to be of only one type (e.g. only preparation-epoch factors or only execution-epoch factors). Indeed, across SCA factors, mean fit was 0.93±0.04 (R²±std). wPCA factors were more ‘mixed’: mean fit was 0.79±0.04, roughly as mixed as expected by chance (0.72±0.11 when randomly rotating the wPCA basis).

Figure 3. Comparisons of SCA with supervised approaches.

A. Schematic illustrating how supervision²⁵ was used to identify orthogonal subspaces, one per task epoch. ‘Proper’ supervision uses known epochs when preparation, execution, and posture-related activity occur. ‘Naive’ supervision uses only two epochs, and is unaware of any distinction between execution and posture-related activity. B. Factors discovered via proper supervision formed three categories (preparatory, execution, posture). Each factor found by the other methods was reconstructed via regression against properly supervised factors, limited to one category (whichever performed best). Mean and standard deviation of reconstruction performance (R²) were computed across 100 resampled populations. The resulting factor-separation metric nears unity if a method discovers the same factor categories as proper supervision. Factor separation was higher for SCA versus wPCA (p<0.01 via bootstrap), dPCA (p<0.01), and naive supervision (p=0.01). Lines with flanking dashed traces (at top) show reconstruction performance when using all supervised factors. The difference in R² when using all properly-supervised factors, versus one category, was smaller for SCA versus wPCA (p<0.01), dPCA (p=0.01), and naive supervision (p=0.01). Chance-level factor-separation was estimated by reconstructing randomly rotated wPCA factors. C. Eight ‘Properly supervised’ factors, ordered by variance explained within-category. D. Same for eight ‘Naively supervised’ factors. E. Same for eight dPCA factors.

Assuming that the supervised approach is ‘properly supervised’ (estimates factors that correspond closely to ground-truth distinctions), SCA comes quite close to estimating those ground-truth factors. Furthermore, SCA outperforms a ‘naively supervised’ approach that makes reasonable but imperfect assumptions. We applied the same supervised approach after dividing data into just two epoch types (Fig. 3A): a plausible approach if one appreciated the division between preparation and execution, but not the recently discovered posture factors. This approach recovers factors that mix movement and posture-related computations (Fig. 3D), and thus performs less-well than SCA (Fig. 3B).

Another common supervised approach, dPCA, learns factors that explain variance linked to defined sets of task conditions (reach directions in this experiment). dPCA factors were strongly ‘tuned’ for reach direction (as expected given dPCA’s goal) but mixed tuning related to preparation, execution, and posture (Fig. 3B,E). This is not a flaw of dPCA; the form of demixing it employs simply does not map onto the present goal.

Center-out reaching - practical considerations

SCA has three hyperparameters: ${\lambda }_{\text{sparse}}$ (which scales the penalty for non-sparse factors); ${\lambda }_{\text{orth}}$ (which scales the penalty for non-orthogonal latent-neuron mappings), and the number of requested factors. The outcome above – distinct preparatory, execution, and posture-related factors – was robust to relatively large changes in all hyperparameters (Fig. S4).

The primary impact of hyperparameters regarded a subtler issue: the degree to which activity during outwards and return reaches occurred within the same dimensions. The original analysis (Fig 2E) requested eight dimensions, and the same execution-related dimensions (6–8) captured activity during outward and return reaches. In contrast, when requesting twenty-four dimensions, outward- and return-reaches shared some execution dimensions but not others (Fig. S5A monkey B; S6 monkey A). This is largely expected: return reaches were considerably slower (Fig. 2B; 30% and 49% lower peak speeds, monkeys B and A) and thus likely to use overlapping-but-not-identical dimensions (including preparatory dimensions). Dimensions indeed differed when assessed independently of SCA. The alignment index²⁵ (which reaches unity when activity uses identical subspaces) between outward- and return-reach activity was 0.43. Alignment was thus incomplete, yet also much larger than for truly distinct factor-sets (e.g. alignment was 0.1 for delay-period activity versus outward-reach-related activity, similar to^25,26,28). When allowed more dimensions, SCA captures this partial overlap. At the same time, for practical reasons, one might sometimes prefer to request fewer factors, which forces SCA to focus on the more-complete divisions: between preparatory, execution-, and posture-related factors.

Importantly, when requesting too many factors, SCA is robust to yielding spurious sparsity in the factors: ‘extra’ dimensions largely capture noise rather than being splits of other factors (Fig. S5). This is true when using default values of ${\lambda }_{\text{orth}}$, which promotes approximately orthogonal dimensions. Over-sparsification did occur if ${\lambda }_{\text{orth}}=0$. Similar over-sparsification occurs (Fig. S7) when applying Independent Component Analysis⁵⁵ (ICA), a well-known unsupervised approach that can yield temporally-sparse factors⁵⁶ but lacks orthogonality constraints.

A related concern is that, by seeking sparsity, SCA could overemphasize ‘non-normal’ dynamics: activity rotating through dimensions without returning^57–59. For example, SCA execution factors in Fig. 2E were only modestly multiphasic, indicating rotations with a non-normal component. In practice, overemphasis appears minimal. SCA recovered the true oscillatory / multiphasic factors when applied to activity from an RNN that used such dynamics (Fig. S8). SCA also recovered strongly oscillatory factors for a second monkey (Fig. S8). This suggests that rotational dynamics can be closer to non-normal or oscillatory depending on the animal, and that SCA can help identify this heterogeneity.

Center-out reaching - Further comparisons with unsupervised approaches

We further explored how various unsupervised methods perform in the context of center-out-reaching (Fig. S9). Unlike in the main analyses, we did not subtract the cross-condition mean, partly to provide a complementary test-case but also because one method (non-negative matrix factorization, NMF) requires non-negative data. The neural data thus contain condition-invariant signals in addition to the direction-dependent signals analyzed above.

As above, SCA identified distinct factors reflecting preparation, execution and posture. SCA identified additional factors that were largely condition-invariant, including the putative ‘trigger signal’ that changes abruptly before both outward and return reaches^28,60. A variety of methods (PCA, factor analysis, and sparse PCA) did not demix; factors mixed preparation, posture, and/or execution.

Three methods (ICA, NMF, and PCA followed by a non-standard varimax rotation⁶¹) implicitly encourage sparsity and did demix factors, although not always as effectively as SCA. Concentration-by-epoch (Methods; higher indicates greater demixing) was 1.05, 1.30, and 1.12 (ICA, NMF, PCA+varimax), versus 1.30 for SCA. The similarity amongst these methods is reassuring: all three can encourage sparsity and thus should tend to perform similarly in many situations. Indeed, the non-standard variant of PCA+Varimax (recently employed in⁶¹), can be viewed as a step-wise approximation of SCA. One would thus hope that they produce similar results, and indeed they often do (our software toolbox provides code for PCA+varimax as a fast approximation of SCA).

However, these methods are less consistent, and can occasionally produce very different results from SCA (as in Fig. S1). In large part this is because none of these methods seek the two key features (sparsity and orthogonality of dimensions) as directly as SCA. Method-specific features were also sometimes relevant: because NMF uses non-negative factors and loadings, it cannot capture bidirectional modulation. NMF thus captures less variance than the other methods (Fig. S9H). Both NMF and (as noted above) ICA tend to over-sparsify factors (Fig. S10) when many dimensions are requested (a failure mode that SCA largely avoids) as they don’t explicitly seek orthogonality. Overall, no alternative method performed as well as SCA: either concentration was lower (factors were more mixed), condition-invariant signals were less-well isolated, variance captured was lower, factors could be misleadingly sparse. Still, examination of these methods highlights the importance of sparsity; implicit seeking of sparsity was the common feature of all methods that demixed factors fairly successfully.

Unimanual Cycling

Do more extended movements – spanning seconds – also involve distinct factors reflecting preparation, execution, and posture? As longer movements terminate, is there preparation to stop? Are distinct stages reused compositionally? Does a 7-cycle movement involve the same preparation to start (and stop?) as a 4-cycle movement? Such questions remain open because the nature and timing of these putative processes is unknown. This provides an opportunity to leverage the unsupervised nature of SCA.

We applied SCA to motor-cortex activity as monkeys cycled a hand-held pedal to traverse a virtual track (Fig. 4A)^54,62. Instructed by landscape color, progression required cycling forward versus backward. Monkeys began each trial by moving to a defined starting location. Shortly thereafter a target appeared, at a distance corresponding to one-half, one, two, four, or seven cycles. After a variable delay and following a go cue, monkeys cycled swiftly (~2 Hz, Fig. 4B) to the target and stopped upon it. The starting location was set such that the hand started either near the cycle’s top or bottom. The hand ended near that same location, excepting reversals during half-cycle movements. Postural muscle activity (Fig. 4C) reflected both top-versus-bottom position and cycling direction (which influenced exact start and stop positions).

Figure 4. SCA identifies distinct factor types – posture, preparatory, steady-state execution, and stopping – in motor-cortex unimanual cycling data.

A. Task schematic. Monkeys pedaled forward (top) or backward (middle) from an initial target to a final target. Legend applies to all panels. B. Vertical pedal position for twelve conditions: three distances (columns); forward (green) and backward (red); starting at cycle’s top (light traces) or bottom (dark traces). C. Trial-averaged EMG (trapezius) for the same conditions. D. Four SCA-identified factors (rows) for those same conditions. Factors were chosen to represent each category, and correspond to unordered dimensions 14, 20, 1 and 26.

SCA factors formed discrete sets reflecting preparation, steady-state execution, and posture. Figure 4D plots representative examples of each factor-type (all factors in Fig. S11; second monkey in Fig. S12). Preparatory factors were active between target and movement onset. Consistent with a role in preparation, preparatory-factor activity varied with starting position and cycling direction, which together determined the pending change in muscle activity. Preparatory-factor activity was similar for all distances greater than one cycle (Fig S11C and S12G) – supporting the recent conclusion that motor cortex lacks long-timescale information^28,54. Steady-state execution factors were rhythmically active throughout cycling and reflected cycling direction but not (ignoring a phase-shift) starting position, features shared with muscle activity. Posture factors were active between movements, regardless of target presence. Consistent with a postural role, their activity reflected the features that determined postural muscle activity: direction and stopping position.

Additionally, SCA identified ‘stopping’ factors, active for roughly 500 ms before movement terminated. These presumably reflected preparation to stop and/or a stopping-specific execution process. Stopping-factor activity reflected ending position and cycling direction, the two features that most influenced the final muscle-activity pattern. For example, the stopping-factor in Figure 4D is active when forward-cycling terminates at the bottom: during 7- and 4-cycle bottom-start movements and 0.5-cycle top-start movements.

Thus, SCA factors formed categories putatively related to preparation, steady-state execution, stopping, and postural maintenance. Within each category, both factor timing and factor ‘tuning’ agreed with the hypothesized role. Unlike during reaching, these factor-categories, and their likely interpretations, were not previously established. At the same time, three of the factor categories – preparatory, execution, and posture – map cleanly onto categories present during reaching, giving them a natural interpretation. Other aspects of interpretation are necessarily tentative. For example, do stopping factors principally reflect preparation to stop, stopping-specific execution computations, or both? Additional experimental manipulations are needed to resolve such questions. The key point is that SCA identifies factors that lend themselves to forming testable hypotheses, and can sometimes support firm conclusions immediately. For example, factors are indeed used compositionally: 7-cycle and 4-cycle movements involve identical factor-sets, just with a briefer bout of steady-state execution. Even half-cycle movements (whose duration is similar to a long reach) mostly reuse these same factors, with heavily truncated execution. Top- and bottom-start movements reuse some factors, but not others.

Although SCA factors extend what was known, their properties also fit reassuringly with independent findings. This was true not only regarding SCA-identified factor-categories, but also factor-categories that weren’t found. SCA did not identify top-start-specific or bottom-start-specific steady-state-execution factors. In agreement, subspace alignment was high during top-start and bottom-start conditions (0.85±0.05; mean and standard deviation across pairs of top-start/bottom-start conditions). Steady-state execution factors were not cycle-specific: they repeated across all middle cycles, in agreement with more conventional analyses^54,62. Thus, SCA does not invent sparse factors when they do not exist. Yet SCA does identify situation-specific factors when independent analyses suggest they should exist. Many SCA steady-state-execution factors were specific (or partially specific) to forward versus backward cycling (Fig. S11A, S12E). This agrees with prior PCA-based results^62,63 and with partial subspace alignment between forward- and backward-cycling activity (0.40±0.03 for this dataset). Stopping-factor activity also agrees with (and helps explain) prior⁵⁴ and current (Fig. S11F,G) alignment-based results.

Notably, the factor sets are not pre-determined but emerge naturally; SCA does not use labels indicating what factor types should be sought. This explains why dPCA is inappropriate in this context (Fig. S13). dPCA leverages labels regarding time and condition type. If those labels (e.g. cycling direction, distance, and starting position) don’t correspond to computational processes (preparing, steady-state-execution, stopping, posture), dPCA will not demix those processes.

Bimanual cycling

We applied SCA to motor-cortex recordings, from both hemispheres, during a bimanual cycling task⁴⁵. Data included previously analyzed conditions where only the left or right arm moved, and not-yet-analyzed conditions where both arms move simultaneously (Fig. 5A). Monkeys cycled to a target (seven cycles distant), either forward or backward, from a starting location at the cycle’s top or bottom. For bimanual movements, pedals were either in phase (bimanual-0) or out-of-phase (bimanual-180). Unlike a bicycle, moving one pedal did not cause the other to move. Thus, left-arm muscle activity (Fig. 5B, left) simply reflected what the left arm was doing, with little dependence on right-arm movement.

Figure 5. SCA identifies distinct left-arm and right-arm factors, used compositionally during bimanual cycling.

A. Top: Cycling required using one or both pedals. Bottom: Vertical hand positions for four trials (rows) illustrating the four handedness requirements: left-arm unimanual, right-arm unimanual, bimanual 0, and bimanual 180. B. Left. Trial-averaged EMG (with s.e.) during four conditions. Right. Trial-averaged firing rate (with s.e.) for an example motor-cortex neuron. C. Trajectory of two ‘left-arm’ SCA factors (those with highest variance) during forward cycling. Traces correspond to steady-state cycling across the four handedness requirements (legend). Top-start and bottom-start trajectories overlap. D. Same but for right-arm SCA factors. E. Comparison, for 50 SCA factors (symbols), of variance during bimanual versus unimanual cycling. Horizontal axis: variance predicted by superposition, computed by summing variances when each arm moves alone. F. Left: neural variance during bimanual cycling (black) is somewhat less than expected under superposition (gray). Bars give means across conditions. Dots atop each bar correspond to individual comparisons contributing to that mean (e.g. for bimanual cycling, the comparison could be made for both 0 and 180 degrees, and for both top- and bottom-start conditions). Right: Alignment indices for left-arm versus right-arm cycling (tan), bimanual versus unimanual (factors combined across arms) cycling (black), and top-start versus bottom-start unimanual cycling (gray). Bars show mean alignment for all relevant comparisons. Dots show individual comparisons contributing to that mean.

In contrast to the muscles, most motor-cortex neurons were active during movements of either arm (Fig. 5B)^45,46,64,65, even though descending drive is primarily contralateral. A potential resolution to this seeming paradox is distinct left-arm and right-arm factors (with potentially distinct downstream effects) as reported recently^45,46. Can SCA reproduce this result without supervision? Can it address whether bimanual cycling leverages compositionality? A recent study, where monkeys countered perturbations⁴⁷ found a compositional solution, as did a study of finger movements⁶⁶. Is bimanual cycling generated by superposition of left-arm and right-arm factors? Are there additional ‘bimanual specific’ factors?

We applied SCA to steady-state activity (cycles 2–6) from 586 neurons, during two cycling directions, two starting positions, and four handedness requirements. SCA reproduced, without supervision, the presence of distinct left-arm and right-arm factors. Left-arm factors were active when the left arm moved alone (Fig. 5C; purple trajectory is large) but not when the right arm moved alone (Fig. 5C; orange trajectory is very small). Right-arm factors (Fig. 5D) had the complementary property (orange trajectory is large and purple trajectory is small). During bimanual movements, left-arm and right-arm factors became simultaneously active (light-green and dark-green trajectories are sizable in both Fig. 5C and D). This was true not only of these example factors but in general (Fig. S14). If a factor was active when one arm moved alone, it was rarely active when the other arm moved alone, yet became active again during bimanual movements, consistent with bimanual movement reusing unimanual computations. In contrast, there was no evidence of reuse between unimanual conditions (almost no factors were shared between left-arm only and right-arm only cycling) even though there certainly could have been factors reflecting shared abstract parameters such as direction⁶⁴. Results were similar for a second monkey (Fig. S15).

Properties of SCA factors were confirmed by subspace-alignment analyses (Fig. 5F). There was near-zero alignment between left-arm-only and right-arm-only conditions, in agreement with distinct left-arm and right-arm factors. Alignment was high between top-start versus bottom-start conditions, consistent with SCA finding essentially no ‘top-start specific’ or ‘bottom-start specific’ factors. Alignment was high (but less so) when considering all bimanual versus all unimanual conditions, consistent with bimanual conditions reusing unimanual factors (but also, as described below, with the presence of some bimanual-specific factors).

Total neural variance during bimanual conditions was lower than the summed variances during component unimanual conditions (Fig. 5F, left). This was true of most individual dimensions (Fig. 5E) and is reflected in smaller bimanual-condition orbits (Fig. 5C,D; dark and light green trajectories). This replicates similar effects in other experiments involving multiple effectors^47,66. It is presently unclear whether this reflects dynamic-range limitations, or is feature or limitation of compositionality. Also in violation of idealized superposition, a few SCA factors were active only during bimanual movement. The unsupervised nature of SCA is thus helpful in highlighting features that a supervised method might miss. Along similar lines, SCA identified, primarily for monkey E, a handful of ‘ramping’ factors (Fig. S15). These agree with a behavioral idiosyncrasy that was pronounced in monkey E: speeding up when approaching the final target. A prediction, derived from prior findings⁶⁷, is that increasing speed will yield a helical motor-cortex population trajectory. This was confirmed by examining SCA-derived factors (Fig. S16).

SCA identifies factors related to specific mating behaviors in C. elegans

In mammalian cortex, neurons often exhibit heterogeneous responses with ‘mixed selectivity’^68–74, consistent with single-neuron activity reflecting multiple population factors^40,75–77. To probe SCA’s utility when neurons are more specialized, we considered single-trial data recorded during C. elegans mating⁷⁸, a complex behavior composed of discrete behavioral motifs (Fig. 6A). Prior work has mapped individual neurons in the posterior brain (the ganglia responsible for generating mating-related behaviors^79–81) of male C. elegans. Single-neuron activity is often tightly linked to one, or a few behavioral motifs^78,82–84.

Figure 6. SCA identifies factors related to mating motifs in C. elegans.

A. Behavioral motifs exhibited by worm-109; not all worms exhibited all behavioral motifs. B. Top: ethogram for a single mating bout (worm-109). Bottom: Activity of forty-nine neurons during that bout. C. Left: Four SCA factors during that bout. Dimension-numbers are intrinsically unordered. Right: Loadings describing how these four SCA factors impacted the activity of identified single-neurons. D. Correlations between all worm-109 factors and behaviors, for both SCA and PCA factors. E. For each SCA (or PCA) factor, we found the behavior with which it was most strongly correlated and took that correlation. Cumulative distributions plot those correlations across all factors and worms. Individual SCA factors had tighter relationships with individual behaviors, yielding a rightward shift (p<0.001, rank-sum test). F. Example SCA factors related to specific behavioral motifs. G. SCA loadings for the same motifs. There are different numbers of columns for each motif because not all worms exhibited the same motifs and SCA did not always identify a factor that clearly corresponded to that motif. The latter is partially due to different combinations of neurons being recorded across worms. Gray dashes indicate non-recorded neurons.

We analyzed calcium-imaging data from the posterior brain of eight male worms (49–54 simultaneously recorded and identified neurons, Fig. 6B). Behavior within a mating bout was classified into one of twelve motifs (see Methods, Fig. 6B, top). We applied SCA separately for each worm. As in all analyses, SCA was unsupervised: i.e. had no information regarding behavior.

In many individual SCA dimensions, activity was tightly related to a single behavioral motif. Consider the top row of Fig. 6C. This factor’s magnitude (white trace) is low except around vulva detection (orange). The three remaining factors have similarly tight relationships with copulation onset (red), turning (blue), and excretory pore detection (black). (All SCA factors and more worms: Fig. S17).

SCA loadings (Fig. 6C, right) agree with known functional roles of many neurons. Sensory neurons PCB, PCC, and HOB contribute to vulva detection^78,83. The factor active during vulva detection (dimension 5) contributes heavily to their activity. The turning-related factor (dimension 10) contributes heavily to neuron PVV, which drives turning behavior during mating⁷⁸.

We measured the correlation between each SCA factor and all behaviors (Methods). A given SCA factor was commonly closely related to just one behavior (Fig. 6D, top). In contrast, most wPCA factors were commonly weakly correlated with multiple behaviors (Fig. 6D, bottom). Across all SCA factors, in all worms, the mean correlation between the factors and the most closely related behavior was 0.47±0.02 (standard error) (Fig. 6E). Performing the same analysis with wPCA factors yielded significantly lower correlations (0.37±0.01, p<0.001, rank sum test), indicating the SCA factors bore a closer relationship to the timing of individual motifs.

We observed consistency across worms in many motif-related factors and their loadings (Fig. 6F). For example, we found SCA factors related to copulation in all worms that exhibited this behavioral motif (six of eight, Fig. 6F). When measuring similarity between loading vectors of individual worms, there was a modest but highly significant increase for SCA over wPCA (Fig. S18A,B): SCA loadings were more replicable across worms. In fact, SCA loadings could even generalize across worms - we could use loadings from one worm to predict behavior in other worms (Fig. S18C,D). Additionally, SCA recovered, across worms, similar relationships between individual neurons and particular factors. For copulation-related factors, SCA tended to learn large weights for neurons SPC, SPD, and HOA, neurons known to play important roles in copulation (Fig. 6G, red box)^78,82. For turning-related factors (Fig. 6G, blue box), SCA consistently recovered large weights for neuron PVV, recapitulating prior knowledge⁷⁸.

Identifying compositionality in a multitask RNN

Driscoll et al.¹³, extending work by Yang et al.³⁸, found that single networks, trained to perform fifteen cognitive tasks, learned a small number of component computations. These computational building blocks were used across multiple tasks and allowed networks to quickly learn new tasks. Although confirmation of compositional reuse required analyses of linearized dynamics, the presence of distinct building blocks was sometimes evident directly from activity. This was particularly true when network units used non-negative activation functions (softplus or rectified tanh) and individual units primarily participated in few component computations. For other activation functions (e.g. tanh), units tended to be active across many component computations (Fig. 7B), perhaps similar to the mixed selectivity prevalent in the frontal lobe. In such situations, might the presence of distinct computations still be discerned at the factor level?

Figure 7. SCA identifies compositionality in a multitask network.

A. Network architecture. Stimuli arrived in two ‘channels’ (analogous to two stimulus modalities), each using two dimensions. B. Normalized firing-rate variance (across time and conditions) of individual units, for different task epochs. Epochs were delineated by the timing of the network inputs. Epoch number differed by task. An individual unit was typically active across many epochs, preventing underlying computations from being apparent at the individual-unit level. C. Activity in three SCA dimensions during nine tasks. Traces are colored by stimulus angles. The first factor reflected the task-relevant stimulus. The second reflected the network’s ultimate decision. The third reflected the timing of network output. During the ‘Delay-pro’ and ‘Delay-anti’ tasks, the network received a directional stimulus and generated an output in the same, or opposite, direction after receiving a ‘go cue’. ‘Memory-pro’ and ‘-anti’ tasks were the same as the ‘Delay-’ tasks, except the network did not receive a continual directional stimulus. The remaining five tasks (Compare and Context Compare, detailed in Methods) required the network to compare the amplitude of two successive stimuli and respond in the direction of the larger.

We applied SCA to population activity from a network with mixed selectivity. For all tasks, the network received directional inputs (sine and cosine of different angles) and generated a two-dimensional directional output (Fig. 7A; see Methods and Fig. S19). SCA factors point towards the computational strategy used by the network. To illustrate, we highlight activity of three SCA factors during nine tasks (Fig. 7C; all factors during all tasks: Fig. S20). The network’s putative strategy can be summarized as follows: use one set of factors to consolidate incoming stimuli, another to reflect the ‘internal’ decision, and additional condition-invariant ‘trigger’ factors to determine when activity flows into output-potent dimensions. We describe each factor-type in turn.

Directional inputs employed two ‘channels’ (Fig. 7A, stimulus 1 and 2). Some tasks used only one channel per trial (e.g., Delay-pro, Compare-stim1 and -stim2). In others (e.g., ContextCompare-stim1), both channels were simultaneously active and one had to be ignored. SCA identified ‘stimulus factors’ that reflected the relevant stimulus, regardless of input channel. For example, activity in dimension 1 (Fig. 7C) tracks stimulus timecourse for either or both channels, as appropriate given the task. (SCA identified additional channel-specific dimensions: Fig. S19).

A second set of factors (e.g., Fig. 7C, dimension 6) reflected the internal decision – i.e. the signal that eventually determined network output. To appreciate how these factors behave, consider activity during Memory-pro and Memory-anti tasks. Both involve a brief directional stimulus. In the Memory-pro task, the network must report this direction after the go cue. In the Memory-anti task, the network must report the opposite direction. Activity grows in dimension 6 after the stimulus, but with opposite ordering for pro and anti tasks, reflecting what will become the ultimate ‘decision.’ During Compare- and ContextCompare tasks, activity becomes strong only after the second epoch because the decision requires comparing information across both.

A third set of factors reflects output timing: e.g. dimension 17 changes suddenly, following the go cue, in a similar way for all conditions and tasks. This is reminiscent of the condition-invariant ‘trigger signal’ observed during reaching in non-human primates⁶⁰, decision-making in rodents⁴¹, and speech in humans⁷⁷. The hypothesized role of the trigger signal is to translate activity to a region of state-space where local dynamics cause output-null activity (here decision-factor activity) to impact output-potent dimensions. This parcellation of network computations cannot be gleaned from single-unit activity (Fig. 7B) nor from PCA factors (Fig. S21).

Nonlinear SCA

One may sometimes wish to allow nonlinear relationships between latent factors and neural responses. We created a nonlinear extension of SCA that allows learned flexible mappings from factors to activity through a neural network within its architecture (Fig. 8A). This approach can be viewed as a sparse autoencoder with added orthonormality penalties. The cost function is almost identical to that of linear SCA, although we here encourage orthonormality in the weights in all layers of the neural network mapping factors to activity (Fig. 8A, Methods).

Figure 8. Nonlinear SCA discovers demixed factors and improves reconstruction accuracy in unimanual cycling data.

A. Architecture schematic. B. Reconstruction accuracy, on held-out data (Methods), versus the number of dimensions used. We compare nonlinear SCA, a vanilla autoencoder (no sparsity or orthonormality penalties), and linear SCA. C. Four example latent factors found with nonlinear SCA. Factors are labeled by their putative role and plotted as in Fig. 4D. D. Four example latent factors from a vanilla autoencoder.

When applied to unimanual cycling data (Fig. S22A), nonlinear SCA identified sets of factors – corresponding to preparation, steady-state movement, posture, and stopping – similar to linear-SCA factors (Fig. 8C). Thus, the presence of these distinct processes is supported by both linear and nonlinear methods. However, non-linear SCA needed fewer factors of each type (Fig. 8B): e.g. it achieved an R²>0.7 (on held-out data) using only 10 dimensions, as opposed to 40 for linear SCA. One straightforward reason for this savings relates to the fact that many neurons had steady-state execution firing rates that were essentially nonlinearly distorted versions of a sinusoid.

Both sparsity and orthonormality played an important role in discovering factors reflecting distinct computations. A standard nonlinear autoencoder, with an equivalent architecture but no sparsity or orthonormality encouragement, produced factors that mixed underlying task computations (Fig. 8D, Fig. S22B). Moreover, as it did above, encouraging orthonormality prevented ‘oversparsification’ (Fig. S23).

Quantitatively, nonlinear SCA’s predictive performance was superior to that of a standard autoencoder, likely due to overfitting by the latter. Thus, beyond improving interpretability of latent factors, SCA’s sparsity penalty may be serving as an effective regularizer. This suggests that encouragement of sparsity does more than produce a basis that aids interpretation; if the true factors tend to be non-synchronous, seeking this feature may improve factor-estimation accuracy.

Discussion

It is increasingly appreciated that a given neural population often performs multiple computations. SCA’s cost function is a straightforward implementation of two goals – seeking of sparsity and orthonormality – that should, based on first principles, be helpful for demixing neural computations. Empirically, SCA performed well across a range of datasets, confirming prior findings and revealing new ones. Intriguingly, every dataset we examined contained examples of distinct sets of factors, often used flexibly and/or compositionality. The prevalence of such structure, across species, brain areas and tasks, argues that SCA is likely to be broadly useful. Distinctions discovered by SCA, once known, were readily confirmed via other methods (e.g. computing subspace alignment). This highlights that SCA can be used either as a primary method or to discover factor-categories that are later isolated via supervision. We never found a situation where SCA discovered spurious factors that disagreed with subsequent analyses. For example, PCA-based subspace alignment was appropriately low versus high when examining situations where SCA did, versus didn’t, find distinct factors. This agrees with expectations given SCA’s cost function: when ground-truth factors are not sparse, it is unlikely that sparsity can be significantly increased without increasing one or both of the other two terms.

Nonlinear embeddings

Factors discovered by nonlinear SCA were qualitatively similar to the dominant factors discovered by linear SCA, during both unimanual cycling and reaching. In fact, using factors found by linear SCA, followed by nonlinear mappings, led to prediction performance nearly as good as that of nonlinear SCA (Fig. S22C). This aligns well with recent modeling work¹¹, where factors are defined as linear combinations of neurons’ spiking activity (as in PCA or linear SCA) and single-neuron firing rates are nonlinear functions of the factors. In networks where this conceptual framework applies, nonlinear SCA may thus be preferable. At the same time, the broad similarity of findings, between linear and nonlinear SCA, argues that the common use of linear techniques is reasonable in the context of the motor system. However, this is presumably not true in all systems; e.g. linear mappings may be very suboptimal when analyzing hippocampal populations.

Factor interpretation

Intentionally (given its unsupervised nature), SCA does not apply interpretation to the factors it discovers. Categories may naturally emerge (e.g. preparatory versus execution factors) but these distinctions are made by the user, not the method. Determining whether categories are interpreted correctly requires scientific, not statistical, inference. ‘Posture factors’ during reaching provide an example. The postural interpretation is suggested by the fact that two-dimensional factor activity roughly maps to two-dimensional hand position. Supporting that interpretation, posture-factor activity returns to baseline during return reaches. Yet even after tentatively accepting the postural interpretation, additional questions remain. For instance, does postural activity reflect proprioceptive feedback, posture-maintaining muscle commands, or an internal representation of location? Related points regard the ‘rules’ governing when and how factor-categories are used compositionally. In the past, specifically crafted experiments were needed to answer questions such as whether execution requires²⁶ versus merely benefits⁸⁵ from preparation, or whether rapid sequences involve dedicated computations⁵³ or reuse single-reach computations²⁸. Such strategies will remain essential when new factor categories are discovered.

Interpretation of SCA factors will also necessarily depend upon the appropriateness of SCA’s assumptions: that distinct computations use distinct factors, occupying largely orthogonal dimensions. Such properties are not guaranteed to be universal, but are likely common. In situations where expectations (or analysis goals) are different (e.g. seeking factors that map to known variables) one would use different methods^35,86–89. When SCA’s assumptions don’t apply, sparse factors may be absent. For example, SCA did not find individual-cycle-specific factors during steady-state cycling, in agreement with prior results in motor cortex (but unlike results in the supplementary motor area)⁵⁴. Thus, SCA can reject hypotheses, in addition to generating new ones.

Orthogonality

Promoting orthogonality reduces concerns that SCA may find sparse but unrepresentative factors: without orthogonality, and when requesting increasing numbers of dimensions, initially distinct factors will split (and continue splitting) into dimensions with increasingly collinear loadings. We observed this effect empirically both for SCA (when using no orthogonality term) and for ICA. In such cases, interpreting factors as distinct would be questionable without additional theoretical or empirical justification.

To date, many of the clearest examples of distinct factor categories involve close-to-orthogonal subspaces^{16,17,19,35,39–41}. If learning places computations in different subspaces, with no attempt to align them, high-dimensional geometry will ensure they are naturally near-orthogonal. Alternatively (or additionally) orthogonality may reduce interference between computations, allowing them to interact only when dynamics link their dimensions. For these reasons, encouraging orthogonality is generally thought to aid interpretation. Yet it does not eliminate the need to test hypotheses via further analyses and experiments. Beliefs regarding when orthogonal sets of dimensions do or don’t map onto distinct biological processes will essentially always depend upon further examination. When expectations of orthogonality don’t apply, one can relax this term within SCA’s cost function.

Sparsity

The notion of sparse single-neuron responses has a long history⁹⁰, and analyses have thus encouraged sparse loadings. An example is sparse PCA (sPCA)^91,92, which seeks factors that impact a subset of neurons, and is particularly useful for clustering⁹³. Both sPCA and SCA fall under the broad umbrella of dictionary learning^94,95: methods that decompose a matrix into a product of two matrices, one of which is encouraged to be sparse. Prior work has even referred to dictionary learning as sparse component analysis^96,97. Within neuroscience, dictionary learning is not typically used for estimating latent factors but rather as a preprocessing tool (e.g., for extracting neural activity from calcium imaging data^98–100 or for spike sorting¹⁰¹).

Given present goals, we were interested not in sparse loadings, but in factors that are sparsely active despite single-neuron responses with mixed and thus quite non-sparse activity. For example, it is typical for a neuron that is active during preparation to also be active during execution. Yet preparation and execution are distinct processes at the population level. By encouraging sparsity, the estimated factors naturally honor these two categories. This is true even though the factors are not truly sparse – simply more sparse than if they were mixed. Factor sparsity may become more prevalent (making SCA particularly useful) as experimental tasks become richer and recruit more component computations.

There have been only a few prior attempts to maximize factor sparsity within population activity. While ICA does not explicitly seek sparsity, minimizing mutual information between factors⁵⁵ can lead to sparse dimensions in practice⁵⁶. Dekleva et al⁶¹ used PCA followed by a varimax rotation of the factors, which leads to sparse factors and can be viewed as a stepwise approximation to SCA. Tolooshams et al.¹⁰² developed an approach based on a convolutional dictionary learning generative model to find sparse events (‘codes’), rather than continuously occurring factors, that reconstruct neural activity. Interestingly, within the Artificial Intelligence community, there has been success in using sparsity to find interpretable latents within Large Language Models^103–105.

The goal of maximizing factor sparsity is distinct from traditional theoretical ideas of sparse coding at the single-neuron level. Sparse coding typically arises if one encourages an autoencoder to use a sparse latent representation with more dimensions than input features⁹⁰. SCA, on the other hand, finds a lower-dimensional representation. We would thus not expect SCA to be particularly insightful when applied to neural activity for which sparse coding is the predominant framework.

Factors and neurons

The degree to which computationally distinct factors relate to physically distinct neurons appears to vary dramatically across brain areas and species. In C. elegans, where individual-neuron connectivity is preserved across animals, many SCA factors primarily reflected the activity of a small number of neurons. In this system, distinct processes are often reliant on the activity of single neurons (e.g., ‘excretory pore detection’ requires PHA activity⁷⁸) and can be understood, at the circuit level, in terms of interactions between individual neurons. In contrast, in monkey motor cortex, individual neurons have mixed selectivity, arguing that each neuron participates in many computations and reflects many factors. For example, there exist distinct left-arm and right-arm factors but relatively few ‘right-arm-only’ neurons, even in primary motor cortex⁴⁵. This appears to be a common property in much of cortex^70,75,76,106.

Traditionally, one often speaks of a given cortical area as performing ‘a computation’, or a small set of fixed computations. Yet our remarkable flexibility presumably involves the ability of areas to switch amongst a multitude of learned computations¹⁰⁷. Performing well in new situations may depend on compositionality: combining, from a repertoire of computations, those appropriate to the present moment. It is precisely in such situations where factor sparsity may be most pronounced. This could be largely invisible in single-neuron responses, which appear ‘mixed’ and confusing rather than sparse. Our results reveal that SCA can uncover distinct factor types even when this is hidden at the single-neuron level. If the above ideas regarding compositionality are correct, the need to do so will only grow.

Resource Availability

Lead Contact

• Requests for further information and resources should be directed to and will be fulfilled by the lead contact, Joshua I. Glaser (j-glaser@northwestern.edu).

Materials Availability

• This study did not generate new unique reagents.

Data and Code Availability

Data

• All data reported in this paper will be shared by the lead contact upon request.

Code

• A code package for SCA is available at https://github.com/glaserlab/sca.

• All original code has been deposited at Zenodo and is publicly available at https://doi.org/10.5281/zenodo.20214464 as of the date of publication.

Additional Information

• Any additional information required to reanalyze the data reported in this work paper is available from the Lead Contact upon request.

STAR Methods

EXPERIMENTAL MODEL AND STUDY PARTICIPANT DETAILS

Datasets

All analyzed datasets (center-out reaching²⁶, unimanual cycling⁶¹, bimanual cycling⁴⁴, center-out reaching network¹⁴, multi-task network¹³, and C. elegans mating⁷⁷) have previously been analyzed. Detailed descriptions of each can be found in the respective citations. Here, we briefly describe each dataset.

dataset	number of units
Multi-task RNN	128
Monkeys B and A, center-out reaching	124 and 130
Monkey N, maze task	118
Maze task RNN	231
Monkeys C and D, unimanual cycling	116 and 117
Monkeys F and E, bimanual cycling	738 and 586
Worms 109, 137, 153, 182, 184, 185, 714, and 2457, C. elegans mating	49, 53, 54, 50, 49, 51, 53, and 51

Center-out reaching

The animal protocols relating to the experiments involving rhesus macaques were approved by the Columbia University Institutional Animal Care and Use Committee. Single neuron and well-isolated multi-unit activity was collected from motor cortex (dorsal premotor cortex and primary motor cortex) while monkeys A and B performed center-out reaches. All recordings were performed in the hemisphere contralateral to the performing arm. Animals reached in eight directions during two contexts: cue-initiated and quasi-automatic. In the cue-initiated context, the animals performed standard center-out reaches with a random, unpredictable delay between 0–1000 ms. In the quasi-automatic context, the delay period was again 0–1000 ms long, and the peripheral target began to move outward from the center of the screen once the go-cue was delivered. In both conditions, the peripheral target needed to be held for 600 ms, after which reward was delivered. Following reward delivery, the monkeys reached back to the central touch-point to begin the next trial.

Due to extensive training, the monkeys’ behavior during cue-initiated and quasi-automatic conditions is extremely similar. Additionally, activity in motor cortex is also extremely similar²⁶. We therefore combined cue-initiated and quasi-automatic trials when creating trial-averaged PSTHs. When creating PSTHs, we excluded failed trials and trials where the monkeys’ behavior deviated from normal. This included trials where the outward reaction time was greater than 500 ms, the monkey’s kinematics were substantially different from the median kinematics, or the monkey’s hand moved during the ‘hold’ period between outward and return reaches. In practice, we primarily excluded trials because of behavior during the return reach, where behavior was less constrained by task requirements. All analyses only included data from conditions with a long (i.e., 300 ms) delay.

To construct trial-averaged PSTHs, we spliced neural activity from around target onset, outward reach onset, and return reach onset together to form a single trace that was continuous in time. For each unit, we first aligned spikes from a single trial, within a window of time, to the relevant task events. We used windows of −200 to 350 ms, −225 to 1000 ms, and −225 to 1000 ms for peri-target, peri-reach, and peri-return activity, respectively. We then concatenated each peri-event spike train, convolved each with a 25 ms Gaussian filter, and averaged across trials. Prior to performing SCA or PCA, we performed two additional preprocessing steps. First, we ‘soft-normalized’ the activity of each neuron; each neuron’s mean rate (across times and conditions) was normalized by the range of the rate (across times and conditions) + 5. We standardly use soft-normalization (e.g.,^15,28,61) to balance the desire for SCA or PCA to explain the responses of all neurons with the desire that weak responses not contribute on an equal footing with robust responses. As a second preprocessing step for Figures 2 and 3, the responses for each neuron were mean-centered at each time. We calculated the mean activity across all conditions of each neuron at each time point, and subtracted this mean activity from each condition’s response. This step ensures that dimensionality reduction focuses on dimensions where responses are selective across conditions, rather than dimensions where activity varies in a similar fashion across all conditions⁵⁹.

Unimanual cycling

Monkeys C and D performed unimanual cycling movements to traverse a virtual track⁶¹. All recordings were performed in motor cortex, contralateral to the performing arm. Here, only well-isolated single neurons were included in the analyses.

Each trial began with the monkey acquiring an initial target, whose location determined the starting location. A final target was then shown in the distance, cueing the monkey as to the number of required cycles. After a variable delay period (500–1000 ms), a go-cue was delivered. After acquiring the final target, the monkey needed to remain stationary for 1500 ms, after which they received a reward. Animals performed 20 unique conditions, each requiring 0.5, 1, 2, 4, or 7 cycles, forwards or backwards movements of the arm, starting at either the top or the bottom of a cycle.

To create trial-averaged PSTHs, we used a previously described time-warping procedure⁶¹. While behavior during the reaching task was brief, movements during the cycling task could unfold over multiple seconds, necessitating a more detailed alignment procedure. Briefly, the time-base for each individual trial was scaled such that the virtual position for that trial closely matched the mean position across all trials of a given condition. This procedure aligned neural activity (and behavior) across trials of a given condition. We then took a subsequent step (not used in⁶¹) to better align data across conditions; we applied a second scaling to the time-base calculated for each condition, such that the duration of each individual cycle was 500 ms (0.5 cycle conditions were not adjusted). Because both animals tended to cycle at approximately 2 Hz, this second adjustment was fairly minor. Prior to performing SCA, responses were soft-normalized, as described above.

Bimanual cycling

Monkeys E and F performed a cycling task similar to that described above; the animals performed cycling movements to traverse a virtual track⁴⁴. While monkeys C and D performed only unimanual cycling movements, monkeys E and F performed both unimanual and bimanual conditions. As above, each condition required the monkey to move between two virtual targets, with a given condition requiring either forwards or backwards arm movements, beginning at either the top or bottom of a cycle. Unlike the task performed by monkeys C and D, each condition was seven cycles long and required movements of either the left, right, or both arms.

For unimanual conditions, only one arm (the ‘performing arm’) was allowed to move; a trial was aborted if the ‘non-performing arm’ moved outside of a small window centered around the bottom of the cycle (±0.05 and ±0.07 cycles for monkey E and F, respectively). During bimanual conditions, the animal was required to move both arms, maintaining either a 0-degree (bimanual0 conditions) or 180-degree (bimanual180 conditions) phase offset between the two hands. Virtual position was determined by the average position of the two arms. Critically, the pedals were not ‘locked’ together; the monkeys needed to actively maintain a particular phase offset. If the angle between the two hands exceeded 180 degrees from the desired offset (i.e., if one arm became more than one half-cycle ahead of the other) the trial was aborted. While the position of the pedals were not yoked, we did provide a weak restorative force to help the animals maintain the instructed phase offset.

Recordings were made in the primary and premotor cortices of both hemispheres and included both single neurons and well-isolated multi-units. To generate trial-averaged PSTHs, we aligned single-trial spikes to movement onset and convolved each spike-train with a 25 ms Gaussian. We then used the same alignment procedure described previously⁴⁴. This procedure results in an adjusted time-base in which each individual cycle, for each condition, is 500 ms in duration. As was true for monkeys C and D, monkeys E and F cycled at approximately 2 Hz for all conditions, so the warping procedure provided only a small adjustment. All analyses of the bimanual cycling dataset only included data from the middle cycles (cycles 2–6). Prior to performing SCA, responses were soft-normalized, as described above.

C. elegans mating

Here, we analyzed calcium imaging data recorded from the posterior brain of eight male worms during mating⁷⁷. All worms co-expressed GCaMP6s and mNeptune in all neuronal nuclei. Individual neurons were identified via their location, morphology, and expression of specific fluorescent markers, and the occurrence of specific behavioral motifs was detected automatically using various kinematic measures (e.g., swimming velocity, tail velocity) as well as the orientation and position of the male worms relative to the hermaphrodites. All analyses involved single-trial fluorescence signals; for details of the preprocessing steps see⁷⁷. Each neuron’s response was fully normalized by its dynamic range prior to performing SCA or PCA.

Multitask RNN

The multitask network¹³ consisted of 128 recurrently connected units and received three inputs: fixation (1-dimensional), stimulus (4-dimensional), and context (15-dimensional). The fixation input served both as a ‘hold’ signal and a ‘go cue’, depending on the context. The set of stimuli contained two separate two-dimensional vectors composed of $A\text{sin}(\theta )$ and $A\text{cos}(\theta )$, where each vector encoded a different one-dimensional circular variable $\left({\theta }_1,{\theta }_2\right)$ scaled by an amplitude $A$. The context input indicates the current task on any given trial. Context input was encoded in a one-hot vector where the index associated with the current task was one and all other indices were zero. The network generated two outputs: a fixation output (1-dimensional) and a response angle (2-dimensional, the sine and cosine of the response angle). During training, the duration of the stimulus epochs (and memory epochs, when present) was determined by a random draw from a uniform distribution to prevent the network from predicting task period transitions.

The RNN was defined by standard functions:

$$\tau \frac{d\mathbf{h}}{\mathit{\text{dt}}}=-\mathbf{h}(t)+F\left({\mathbf{W}}_{\mathit{\text{rec}}}\mathbf{h}(t)+{\mathbf{W}}_{\mathit{\text{in}}}\mathbf{u}(t)+{\mathbf{b}}_{\mathit{\text{in}}}\right)$$

$$\mathbf{z}(t)={\mathbf{W}}_{\mathit{\text{out}}}\mathbf{h}(t)+{\mathbf{b}}_{\mathit{\text{out}}}$$

where $F$ is a tanh activation function, $\mathbf{h}$ is a vector of network activations, $\mathbf{u}$ are the inputs, $\mathbf{b}$ are offset terms, and $\mathbf{z}$ is the output vector. ${\mathbf{W}}_{\mathit{\text{rec}}}$, ${\mathbf{W}}_{\mathit{\text{in}}}$, and ${\mathbf{W}}_{\mathit{\text{out}}}$ are the recurrent, input, and output weight matrices, respectively. The network was trained with standard L2 penalties on the weights and rates. Additionally, private neuron noise and input noise were added during training. The weights were trained via backpropagation through time using the Adam optimizer¹⁰⁸. During training, the network performed all tasks in an interleaved order.

Below, we summarize the 15 tasks performed by the network.

Delayed Response:

Delay-pro: Move in the same direction as stimulus $({\varphi }_{\mathit{\text{response}}}={\theta }_{\mathit{\text{stimulus}}})$ after a delay. Delayed-anti: Move in opposite direction as stimulus ${(\varphi }_{\mathit{\text{response}}}={\theta }_{\mathit{\text{stimulus}}}+\pi )$ after a delay. Stimulus remains on throughout stimulus and response periods.

Memory Response:

Same as above, except the stimulus disappears during the memory and response period.

Reaction Timed:

Same as Delayed Response, but a response is required as soon as the stimulus is delivered.

Compare:

Move in the direction of the stimulus with the largest amplitude. Compare-stim1: Compare stimuli delivered via stimulus modality 1. Compare-stim2: Compare stimuli delivered via stimulus modality 2.

Context Compare:

Same as above, except stimuli are delivered using both modalities. ContextCompare-stim1: compare stimuli delivered via modality 1, ignore modality 2. ContextCompare-stim2: compare stimuli delivered via modality 2, ignore modality 1. ContextCompare-both: attend both modalities.

Category Match:

Network is delivered two sequential stimuli (using the same stimulus modality). Category-pro: immediately respond in the direction of the second stimulus if the two stimuli belong to the same category (${\theta }_{\mathit{\text{stim}}1},{\theta }_{\mathit{\text{stim}}2}<\pi$ or ${\theta }_{\mathit{\text{stim}}1},{\theta }_{\mathit{\text{stim}}2}>\pi$). Category-anti: immediately respond in the direction of the second stimuli if the two stimuli belong to opposite categories.

MatchToSample:

Network is delivered two sequential stimuli (using the same stimulus modality). MatchToSample-match: immediately respond in the direction of the second stimulus if the two stimuli are the same $\left({\theta }_{\mathit{\text{stim}}1}={\theta }_{\mathit{\text{stim}}2}\right)$. MatchToSample-nonmatch: immediately respond in the direction of the second stimulus if the two stimuli are 180° apart $({\theta }_{\mathit{\text{stim}}1}={\theta }_{\mathit{\text{stim}}2}+\pi )$

METHOD DETAILS

Sparse Component Analysis

Parameters and Cost Function:

Let ${\mathbf{X}}^{T\times N}$ be a matrix containing the activity of $N$ neurons over $T$ time bins. The goal of SCA is to find a low dimensional latent representation, with dimensionality $K$, that 1) accurately reconstructs the original data $\mathbf{X}$ via an affine mapping (linear mapping ${\mathbf{V}}^{K\times N}$ and offset ${{\mathit{b}}_{\mathit{v}}}^{1\times N}$), 2) has sparse latents, ${\mathbf{Z}}^{T\times K}$, and 3) encourages the mapping from the latent space to neural space to be orthonormal. We additionally include optional sample weighting, via the diagonal matrix ${\mathbf{W}}^{T\times T}$, which allows providing different importance to different time points within the cost function.

This cost function can be written as follows, with the summed terms corresponding to the three above goals, respectively:

$$\text{arg}\underset{\mathbf{Z},\mathbf{V},{\mathbf{b}}_{\mathbf{v}}}{\text{min}}{\left|\mathbf{W}\left(\mathbf{X}-\left(\mathbf{\text{ZV}}+{\mathit{b}}_{\mathit{v}}\right)\right)\right|}_F^2+{\lambda }_{\text{sparse}}|\mathbf{Z}{|}_1+{\lambda }_{\text{orth}}{\left|\mathbf{V}{\mathbf{V}}^{\top }-\mathbf{I}\right|}_F^2$$

In practice, rather than directly optimizing the latents, we learn the latents via a mapping from the neural activity space (which can be viewed as a linear autoencoder, Fig. 1E).

This is accomplished with the following cost function:

$$\text{arg}\underset{\mathbf{U},\mathbf{V},{\mathbf{b}}_{\mathbf{v}},{\mathbf{b}}_{\mathbf{u}}}{\text{min}}{\left|\mathbf{W}\left(\mathbf{X}-\left(\left(\mathbf{\text{XU}}+{\mathit{b}}_{\mathit{u}}\right)\mathbf{V}+{\mathit{b}}_{\mathit{v}}\right)\right)\right|}_F^2+{\lambda }_{\text{sparse}}{\left|\mathbf{X}\mathbf{U}+{\mathit{b}}_{\mathit{u}}\right|}_1+{\lambda }_{\text{orth}}{\left|\mathbf{V}{\mathbf{V}}^{\top }-\mathbf{I}\right|}_F^2,$$

where ${\mathbf{U}}^{N\times K}$ and ${{\mathit{b}}_{\mathit{u}}}^{1\times K}$ are the linear matrix and offset in the “encoder” part of the autoencoder, that map the neural activity to the low-dimensional space.

In addition to the orthonormality penalty, we also strictly constrain each row of $\mathbf{V}$ to have unit norm. Thus, even if ${\lambda }_{\mathit{\text{orth}}}$ is set to 0, optimization will not lead to a solution in which the magnitude of $\mathbf{Z}$ endlessly becomes smaller while $\mathbf{V}$ endlessly becomes larger. Additionally, this unit norm allows for easier interpretation of SCA factor magnitudes, as they can be interpreted as the magnitude of its effect on neural population activity.

We note that in our accompanying code package, we also include an option to enforce strict orthogonality in the recovered dimensions by performing manifold optimization¹⁰⁸. However, we have found that using the orthogonality penalty (as opposed to an orthogonality constraint) is advantageous for two primary reasons. First, while the two options generally produced very similar factors, we found that, on occasion, constraining SCA to recover orthogonal dimensions could produce factors that were more ‘mixed’. For example, the separation between posture-related factors and preparatory factors was less complete for monkey C (Fig. S11) if we required SCA to recover orthogonal dimensions. The second benefit was purely practical; fitting SCA with manifold optimization was slower.

Sample weighting:

When we use sample weighting, we default the sample weight at a given time point to be inversely proportional to the sum squared activity of the neural population at that time point. This has the purpose of encouraging the model to still use latent dimensions to reconstruct lower-activity time points (e.g. to care as much about motor preparation as motor execution, when the former has substantially lower activity).

Optimization:

All optimization is done in PyTorch with the Adam optimizer¹⁰⁹ using default hyperparameters. We use learning rate scheduling, where the initial learning rate is set at 1e-3, and the learning rate is halved every time the loss does not decrease for 100 iterations, until a minimum learning rate of 5e-4.

Initialization of parameters:

We initialize the model parameters using sample-weighted PCA on the data. That is, if ${\mathbf{Q}}^{N\times K}$ are the sample-weighted PCA loadings, we initialize with $\mathbf{U}=\mathbf{Q},\mathbf{V}={\mathbf{Q}}^{\top }$, ${\mathit{b}}_v=0,{\mathit{b}}_{\mathit{u}}=0$. In practice, we find nearly equivalent solutions using a random initialization of parameters, but convergence generally takes significantly longer. Both sample-weighted PCA and random initializations are available within our code package.

Hyperparameters:

In our code package, we have the following default settings for model hyperparameters. ${\lambda }_{\text{orth}}$ is defaulted so that, if the off-diagonal entries of ${\mathbf{V}\mathbf{V}}^{\top }$ were all equal to 0.1, the orthonormality penalty would be 10% of the initial reconstruction cost (with sample-weighted PCA initialization). ${\lambda }_{\text{sparse}}$ is defaulted so that the initial cost associated with the sparsity penalty is 10% of the initial reconstruction cost. This serves to make sparsity and orthogonality important, but not at great expense of reconstructing the neural data. These defaults allow for easy out-of-the-box model-fitting without hyperparameter tuning. Still, we have found that it is often slightly beneficial to increase ${\lambda }_{\text{sparse}}$ beyond this default value - i.e., further sparsity can be achieved without harming the reconstruction accuracy. Thus, for the demonstrations in this paper, we sometimes set ${\lambda }_{\text{sparse}}$ above the default value - using ${\lambda }_{\text{sparse}}$ values at the elbow of the plot of reconstruction accuracy versus ${\lambda }_{\text{sparse}}$. We also provide a code notebook so that users can also find ${\lambda }_{\text{sparse}}$ in this way. See below for a table of the hyperparameters used for each dataset.

Similarly, we have also seen a general robustness to the dimensionality chosen, with the primary change that occurs over differing dimensionalities detailed in Results: Center-out reaching - practical considerations. Importantly, we have seen that requesting too many dimensions generally yields highly similar ‘signal’ dimensions, with the extra dimensions being primarily capturing noise. For instance, in the reaching data, when overspecifying dimensionality, dimensions beyond 20 appear to be primarily noise (Fig. S5), which matches the dimensionality at which the performance on held-out data plateaus (Fig. 2I). We do note that this robustness does depend on keeping nearly orthogonal loadings (Fig. S7).

Advantages of neural-network based optimization:

We chose to optimize SCA as a linear autoencoder using standard neural network machinery for two primary reasons. 1) This framework scales well with increased dataset duration. The number of encoding and decoding weights depends only on the number of neurons and number of requested factors, whereas directly optimizing latents yields a parameter count that scales with the number of timepoints. Additionally, neural network machinery can easily allow for GPU use, providing improved speed for larger datasets. 2) Our approach makes the method readily extendable, as we have done to create a nonlinear version. As another possible example, if a dataset contains neural activity from two brain regions, the model can be easily altered to identify interpretable factors within one brain region that predict activity in the second region. Additionally, one can easily change the model’s cost function (e.g. including Poisson statistics). This flexibility will encourage further development of interpretable dimensionality reduction tools for neural population activity.

Nonlinear SCA

Parameters and Cost Function: As in our linear SCA description, we start by describing the mapping from latents, $\mathbf{Z}$ to neural activity, $\mathbf{X}$ (the ‘decoder’ part of the autoencoder). Let $g$ be the decoder function mapping the latent space to neural activity predictions, and ${\theta }_g$ represent the parameters (weights and biases) of the neural network $g$. Let ${\mathbf{V}}_i$ be the weights of layer $i$ in the decoder (so all ${\mathbf{V}}_i\in {\theta }_g$). We aim to find:

$$\text{arg}\underset{\mathbf{Z},{\theta }_{\mathit{g}}}{\text{min}}\left(|\mathbf{W}(\mathbf{X}-g(\mathbf{Z})){|}_F^2+{\lambda }_{\text{sparse}}|\mathbf{Z}{|}_1+{\lambda }_{\text{orth}}\sum _i{\left|{\mathbf{V}}_i{\mathbf{V}}_i^{\top }-\mathbf{I}\right|}_F^2\right)$$

As in linear SCA, we don’t directly optimize $\mathbf{Z}$, but rather learn the mapping from $\mathbf{X}$ to $\mathbf{Z}$. Here, this is a nonlinear neural network $f$. Let ${\theta }_f$ represent the parameters (weights and biases) of the neural network $f$. We thus optimize:

$$\text{arg}\underset{{\theta }_{\mathit{f}},{\theta }_{\mathit{g}}}{\text{min}}\left(|\mathbf{W}(\mathbf{X}-g(f(\mathbf{X}))){|}_F^2+{\lambda }_{\text{sparse}}|f(\mathbf{X}){|}_1+{\lambda }_{\text{orth}}\sum _i{\left|{\mathbf{V}}_i{\mathbf{V}}_i^{\top }-\mathbf{I}\right|}_F^2\right)$$

The three terms of the cost function, in order, relate to 1) the reconstruction cost, 2) sparsity of the latents, and 3) orthonormality. Here, we encourage the weights of each layer of the decoder neural network to be orthonormal. It’s important to note that the concept of orthonormality is defined for linear matrix transformations, rather than nonlinear functions. Here, our nonlinear mapping contains a nonlinearity between two matrix multiplications. In a linear network, the multiplication of two orthonormal matrices remains orthonormal. While the nonlinear activation functions between the matrices in the neural network will alter this mapping, our approach nonetheless promotes separate latents to have more distinct mappings to neural activity, and empirically to separate different computations involving different patterns of neural firing into different latent factors. We also note that while this orthonormality penalty was important for interpretation (Fig. S23), orthonormality had minimal impact on reconstruction accuracy (Fig. S22C).

Similar to linear SCA, in addition to the orthonormality penalty, we also strictly constrain each row of ${\mathbf{V}}_i$ to have unit norm, for all layers $i$. Thus, even if ${\lambda }_{\mathit{\text{orth}}}$ is set to 0, optimization will not lead to a solution in which the magnitude of $\mathbf{Z}$ endlessly becomes smaller while all ${\mathbf{V}}_{\mathbf{i}}$ endlessly become larger.

In all our demonstrations, $f$ and $g$ were neural networks with a single hidden layer with a tanh nonlinearity. For $K$ (the number of SCA dimensions) $<N/2$ (where $N$ is the number of neurons), the hidden layer contained $N/2$ units. For $K\ge N/2$, we let the hidden layer contain $K$ units.

Optimization:

Optimization was done as above in linear SCA.

Initialization of parameters and hyperparameters:

Neural network parameters were randomly initialized. As in linear SCA, we have default hyperparameter values that allow the technique to work out-of-the-box. ${\lambda }_{\text{orth}}$ was defaulted so that, if the off-diagonal entries of ${\mathbf{V}}_i{{\mathbf{V}}_i}^{\top }$, for both layers of the neural network, were all equal to 0.1, the total summed orthonormality penalty would be 100% of the sample-weighted PCA reconstruction cost. ${\lambda }_{\text{sparse}}$ was defaulted identically to SCA - the value that would lead to the sparsity term of the cost function leading to 10% of the sample-weighted PCA reconstruction loss.

Ordering of latent factors

There is not a meaningful intrinsic ordering to SCA dimensions. However, after model fitting, re-ordering dimensions can facilitate discovering a meaningful interpretation of the data. For instance, in the reaching data, we typically ordered dimensions by the time points at which the dimensions explained the most cross-condition variance, which allowed us to clearly observe dimensions that activated at distinct time epochs - i.e., preparation, then movement, then posture. Such a temporal ordering also helped us to discover ‘stopping’ dimensions in the unimanual cycling datasets, although we ultimately manually grouped these dimensions into their putative roles, after further examination, for the clearest presentation.

Dimensions can also be automatically sorted by the amount of neural activity explained, comparable to variance explained in PCA, which can also provide insight. For instance, the non-interpretable dimensions that appear like noise often have relatively low variance, so sorting in this manner can help to filter out such ‘noise’ dimensions. Still, it is important to note there are often interpretable signals that explain a low amount of variance overall, but a high amount of variance at a small number of time points (e.g. when stopping a movement or preparing a movement in the unimanual cycling task). For instance, the preparatory dimensions each explain about 1–2% of the total explained variance in the unimanual cycling data, despite being interpretable and computationally meaningful dimensions. Nonetheless, this activity-explained-based sorting can help to provide intuition for the extent to which each factor is present in the overall population activity. We provide code in our package for sorting factors in both these temporal-ordering and variance-ordering manners.

Responsible use of SCA

We want to give some advice regarding how to responsibly use SCA to decrease the risk of arriving at incorrect scientific conclusions. One set of possible concerns relates to sampling error (due to too few neurons or too few trials) leading to false positives or false negatives.

False positives:

By false positive, we mean the discovery of seemingly sparse and seemingly well-behaved factors (possibly suggesting some hypothesis) where none actually exist. For example, suppose that preparatory and execution activity actually involved the same set of factors (i.e. they involved different patterns of activity within the same subspace) but SCA returned distinct preparatory and execution factors. This would constitute a false positive. In practice, we have generally found SCA to be robust to false positives. As we highlight in Discussion, we never found a situation in which SCA found spurious factors; SCA factors always agreed with a known ground truth, or were validated by subsequent non-SCA-based analyses (e.g. subspace alignment). Furthermore, when applying SCA to poorly (minimally) filtered single-trial data, we did not find spurious factors, but noise. To be clear, this does not mean that the factor divisions found by SCA are necessarily the right ones for understanding computation – the degree to which this is true will depend on the degree to which the goals of SCA align with one’s scientific goals. Nor does it mean that initially-appealing interpretations will be correct – further analyses or experiments will almost always be needed, as with any method. What it does mean is that SCA is resistant to hallucinating false positives.

To see why, consider that each SCA latent is simply a weighted sum of the empirical neural activity. Those latents in turn form a basis set for the activity of all the single neurons. To create a false positive, SCA must take activity where there is no natural basis of sparsely active factors, yet choose weights so as to create such factors. To do so, optimization could follow the gradient in weight space that maximally reduces the sparsity term in SCA’s cost function. Yet this will come at the cost of increased reconstruction error (precisely because there is no natural sparse basis). Put differently, if there is no natural sparse basis, a change in weights that increases factor sparsity will typically decrease the sparsity term much less than it will increase reconstruction error. Optimization will thus avoid such changes. It is for this reason that the variance captured by SCA is typically almost as large as that captured by PCA – indeed the SCA basis set is often close to a rotated version of the PCA basis set (which is why PCA with a varimax rotation can be a good approximation to SCA).

These observations are reassuring, but also highlight that care must be taken when the orthogonality hyperparameter is substantially decreased (e.g see Figs. S5, S23). The same is true when the sparsity hyperparameter is turned up into the regime where reconstruction performance substantially degrades (which we do not recommend, as described above in Sparse Component Analysis: Hyperparameters).

If one remains concerned about false positives, particularly due to noisy or limited data, we recommend confirming the found factors on held-out data. Held-out data can take many forms, including held-out neurons, trials, or conditions. One can also replicate findings on another dataset from the same task, either from another session of the same animal or from a different animal (as we often did in this study). One can also replicate results in a variation of the original situation, e.g. a variation of the task.

False negatives:

False negatives are potentially a larger concern, in particular when data-limited. Suppose two factors are truly orthogonal in the full population, but one measures the activity of too few neurons. The loadings may then be accidentally correlated, leading to non-orthogonality simply due to lack of data. For instance, in the center-out reaching dataset, preparatory activity and movement-related activity occur in orthogonal dimensions when using all neurons. However, when using only 20 neurons, this scientific conclusion becomes less obvious, as some noisy latents are active both during preparatory and movement epochs (see 2nd-from-top and 2nd-from-bottom latents in Fig. S3). In general, the number of needed neurons will scale with the total number of factors that one seeks. In situations where one is concerned that neuron-counts are marginal, we would suggest applying SCA to simulated data to estimate the stability of results with neuron count.

Conceptual Interpretation:

In general, we think the larger concerns typically relate to the conceptual interpretation of the latent factors. SCA often produces factors that nicely lend themselves to interpretation, which is helpful, but only if one is careful. It can be a little too tempting, upon finding factors that are active before stopping to cycle, to then conclude ‘those must be related to preparing to stop’. Similarly, upon observing factors that mirror posture, it may be tempting to conclude ‘those must be related to posture’. Those are indeed reasonable interpretations, but that doesn’t reduce the need for follow-up analyses or experiments to test them. This is of course nothing new, or particular to SCA or even to dimensionality reduction. For example, motor-cortex preparatory activity (at both single-neuron and population levels) was called ‘preparatory’ for a long time before enough experiments were performed to truly nail down that that was indeed its role. As we describe in Discussion: Factor Interpretation, initial interpretations should not be fully accepted until follow-up experiments are used to test them.

SCA Runtime

To provide an approximate intuition of the runtime for fitting SCA, we provide examples below. These models were fit with CPU, on a 2020 MacBook Pro, 13” with M1 Chip and 8GB memory. Runtimes are provided until model convergence.

Reaching data (size 124 neurons × 2048 timepoints):

Linear SCA: 2 seconds (3000 epochs)

Nonlinear SCA: 29 seconds (15000 epochs)

Unimanual Cycling data (size 116 neurons × 7023 timepoints):

Linear SCA: 19 seconds (5000 epochs)

Nonlinear SCA: 1 minute 54 seconds (15000 epochs)

Sample-weighted Principal Components Analysis

Sample-weighted PCA (wPCA) is a variant of principal component analysis that includes providing a weight for each sample (time point). Formulated as a cost function, it aims to solve:

$$\text{arg}\underset{\mathbf{U}}{\text{min}}{\left|\mathbf{W}\left(\mathbf{X}-\mathbf{\text{XU}}{\mathbf{U}}^{\top }\right)\right|}_F^2\text{s.t.}{\mathbf{U}}^{\top }\mathbf{U}=\mathbf{I}$$

$\mathbf{U}$ is found by running singular value decomposition (SVD) on the matrix $\mathbf{W}\mathbf{X}$, in the same way that standard PCA runs SVD on the matrix $\mathbf{X}$.

Additional Comparison Methods

Beyond PCA, we compared SCA with several additional unsupervised dimensionality reduction approaches:

Factor analysis (FA):

FA is a commonly used linear probabilistic generative model that enables fitting separate noise levels on each neuron. We used the scikit-learn¹¹⁰ implementation with default hyperparameters.

Sparse PCA (sPCA):

This is a variant of PCA in which sparsity is encouraged in the loadings ($\mathbf{U}$ in the description of PCA in the above subsection). We used the scikit-learn implementation with default hyperparameters.

Non-negative matrix factorization (NMF):

NMF factorizes a non-negative matrix into the multiplication of two matrices with non-negative entries. We used the NIMFA package¹¹¹. For both simulations (Fig. S1) and reaching data, we used a NNDSVD initialization and increased the maximum number of iterations, to improve performance, particularly as convergence was not reached with the standard number of iterations. NMF can implicitly promote sparsity and orthogonality¹¹².

Independent component analysis (ICA):

ICA aims to find dimensions that minimize mutual information between those dimensions. We used the scikit-learn implementation of FastICA. Default hyperparameters were used for simulations (Fig. S1). For reaching data, we increased the maximum number of iterations to improve performance.

PCA followed by a varimax rotation of the latents:

This can be viewed as similar to a step-wise approximation of SCA. PCA first does dimensionality reduction, and ensures that the loadings are strictly orthonormal. Doing a varimax rotation on the factors, which finds a rotation of the latent factor space that maximizes the variance across factors at each time, leads to sparse latents. This is because having, for example, half the latents with zero activity and half the latents with high activity will have more variance than having all factors with moderate activity. PCA followed by a varimax rotation of the latents is part of our SCA code package.

SCA hyperparameters

As demonstrated in Figure S4, SCA produced similar results across a wide range of hyperparameter choices. Below we list the specific hyperparameters used for each dataset. ‘Default’, and rationale for non-default values is described above.

dataset	${\mathit{\lambda }}_{\mathit{\text{sparse}}}$	${\mathit{\lambda }}_{\mathit{\text{orth}}}$	number of factors
Multi-task RNN	default	default	25
center-out reaching (linear SCA)	default	default	8, 16, or 24
center-out reaching (nonlinear SCA)	0.01 (~2x default)	default	24
maze task	default	default	12
Maze task RNN	default	default	12
unimanual cycling (linear and nonlinear SCA)	0.03 (~10x default)	default	40
bimanual cycling	0.01 (~10x default)	default	50
C. elegans mating	0.05 (~10x default)	default	15

QUANTIFICATION AND STATISTICAL ANALYSIS

Ground-truth latent separability due to sparsity

We here provide further detail on the conditions under which encouraging sparsity via an L1 loss can automatically separate distinct neural processes into separate latents, using simplified ‘ground truth’ scenarios (expanding on Fig. 1F–I and surrounding text). Assume we have two true underlying latents (each corresponding to a neural process) that occur at times that are not completely overlapping. Let’s say each is a square pulse of magnitude 1, which occurs for duration $T$, and the two pulses overlap for duration $\tau$ (Fig. 1H). Our question is, in what scenarios will finding the sparsest latents, in terms of L1 loss, lead to finding the ground truth latents?

Background:

Assume we have $K$ underlying latents: ${\mathit{Z}}_{\mathit{\text{true}}}=\left[{\mathit{Z}}_{\mathbf{1}},\dots ,{\mathit{Z}}_{\mathit{k}}\right]\in {\mathbb{R}}^{TxK}$, and $\mathit{X}={\mathit{Z}}_{\mathit{\text{true}}}{\mathit{V}}_{\mathit{\text{true}}}$, where $\mathit{X}$ is the neural activity and ${\mathit{V}}_{\mathit{\text{true}}}$ is a loading matrix with rows of unit norm.

When inferring the latents, ${\mathbf{Z}}^{*}$, there are an infinite number of solutions with equally good reconstruction accuracy: $\mathit{X}={\mathbf{Z}}^{*}{\mathit{V}}^{*}={\mathit{Z}}_{\mathit{\text{true}}}\mathit{R}{\mathit{R}}^{-\mathbf{1}}{\mathit{V}}_{\mathit{\text{true}}}$ where ${\mathbf{Z}}^{*}={\mathit{Z}}_{\mathit{\text{true}}}\mathit{R}$ and ${\mathit{V}}^{*}={\mathit{R}}^{-\mathbf{1}}{\mathit{V}}_{\mathit{\text{true}}}$.

In SCA, we constrain the rows of ${\mathit{V}}^{*}$ to be unit norm, so that the L1 sparsity penalty on the latents, ${\left|{\mathbf{Z}}^{*}\right|}_1$, can’t be arbitrarily decreased by simply increasing the magnitude of ${\mathit{V}}^{*}$. We thus just consider $\mathit{R}$ matrices that are rotation matrices (which will keep ${\mathit{V}}^{*}$ to be unit norm). Still, there are infinite rotations of ${\mathit{Z}}_{\mathit{\text{true}}}$ (and counter-rotations of ${\mathit{V}}_{\mathit{\text{true}}}$), that will produce equal reconstructions of the data. In this framework, our question then becomes, for what underlying latents (i.e., for what ${\mathit{Z}}_{\mathit{\text{true}}}$), does ${\mathbf{Z}}^{*}={\mathit{Z}}_{\mathit{\text{true}}}$ (as opposed to any other ${\mathbf{Z}}^{*}={\mathit{Z}}_{\mathit{\text{true}}}\mathit{R}$), have the smallest ${\left|{\mathbf{Z}}^{*}\right|}_1$?

Answer:

We’ll start by considering the scenario when there is no temporal overlap between latent processes $(\tau =0)$. If we view the activity of these two latents in state space (as in Fig. 1G), when only the first latent, ${\mathit{Z}}_1$ is active, ${\mathbf{Z}}^{*}$ lies on the x-axis of a circle (Fig. 1G, solid black line), and when only ${\mathit{Z}}_2$ is active, ${\mathbf{Z}}^{*}$ lies on the y-axis of a circle (Fig. 1G, solid red line). For any such time point of ${\mathit{Z}}_{\mathit{\text{true}}}$ that lies on an axis, ${\left|{\mathit{Z}}_{\mathit{\text{true}}}\right|}_{\mathbf{1}}=1$. Any rotation (e.g. dashed lines in Fig. 1G) will increase ${\left|{\mathbf{Z}}^{*}\right|}_1$ above 1: $\left|{\mathbf{Z}}_{{\left.\mathbf{1}\right|}_1}^{*}+\right|{\mathbf{Z}}_{{\left.\mathbf{2}\right|}_1}^{*}\ge 1$. This is because of the geometrical fact that the hypotenuse is always shorter than the sum of sides of a triangle.

Next, we consider the more general scenario when there is temporal overlap between latent processes $(\tau >0)$. At timepoints in ${\mathit{Z}}_{\mathit{\text{true}}}$ where both ${\mathit{Z}}_{\mathit{1}}$ and ${\mathit{Z}}_{\mathbf{2}}$ are active (‘overlapping timepoints’ both with values of 1), any rotation of ${\mathbf{Z}}^{*}$ away from ${\mathit{Z}}_{\mathit{\text{true}}}$ will actually decrease ${\left|{\mathbf{Z}}^{*}\right|}_1$. For instance, consider rotating [1,1], with ${\left|{\mathbf{Z}}^{*}\right|}_1=2$, with a −45 degree rotation to $[\sqrt{2},0]$ with ${\left|{\mathbf{Z}}^{*}\right|}_1=\sqrt{2}$ (in fact, a 45 degree rotation will most decrease ${\left|{\mathbf{Z}}^{*}\right|}_1$ for overlapping points). Thus, depending how many overlapping vs non-overlapping timepoints there are, rotations may increase or decrease the total ${\left|{\mathbf{Z}}^{*}\right|}_1$. Analytically solving for the $\tau$ at which the decrease in ${\left|{\mathbf{Z}}^{*}\right|}_1$ from non-overlapping timepoints balances out the increase in ${\left|{\mathbf{Z}}^{*}\right|}_1$ from overlapping timepoints (deriving equations shown below), we get $\tau =(2-\sqrt{2})T\approx 0.586T$. Further overlap above this value leads to the ground truth latents not being the sparsest.

Deriving equations:

We calculate the maximum amount of overlap $\tau$ for which two latents of duration $T$ will have the smallest L1 value when not rotated, versus when the latents are rotated 45 degrees, as 45 (or −45) degrees is the rotation that will lead to maximum sparsity for overlapping timepoints in this scenario (Fig. 1J). To do so, we find when the L1 term with a 45 degree rotation becomes smaller than the L1 term with no rotation (when the ground truth latents are the sparsest).

The L1 term with no rotation is $2T$. This is because, for T time points, for 2 latents, each has an L1 of 1.

The L1 term with a 45 degree rotation is $\sqrt{2}\tau +2\sqrt{2}(T-\tau )$. Term 1 in this sum is because, for the $\tau$ overlapping timepoints, [1,1] gets rotated to $[0,\sqrt{2}]$, with $\text{L}1=\sqrt{2}$. Term 2 in the sum is because, for the $\text{T-}\tau$ overlapping timepoints, for the 2 latents, [1,0] and [0,1] get rotated to [$\sqrt{2}/2,\sqrt{2}/2$] and $[-\sqrt{2}/2,\sqrt{2}/2]$, respectively, each with an L1 of $\sqrt{2}$, resulting in a total L1 of $2\sqrt{2}(T-\tau )$.

Finding when the L1 term without rotation is smaller than with rotation:

$$\begin{array}{c}\sqrt{2}\tau +2\sqrt{2}(T-\tau )\ge 2T\\ \dots \\ \tau \le T(2-\sqrt{2})\approx 0.586T\end{array}$$

This analytical value is consistent with that empirically found in Fig. 1J.

Generality:

The general result that ${\mathit{Z}}_{\mathit{\text{true}}}$ is commonly the sparsest solution is robust to when we change the magnitude of latents or duration of latents, although the exact conditions on $\tau$ will slightly change. For instance, if one latent has magnitude 0.1 (and the other 1), they can overlap $~0.52T$ (~52%). When one latent has half the duration $(T/2)$, they can overlap $~0.43T$ (~86% overlap of the shorter duration latent and ~43% overlap of the longer duration latent).

Reaching analyses

Occupancy and occupancy concentration:

For the reaching datasets, we computed the occupancy (variance across conditions) of each dimension, $k$, at each timepoint, $t$, where $t$ began with target onset and ended 300 ms after the onset of the return reach:

$$\text{Occ}(t,k)=\frac{1}{C-1}\sum _{i=1}^C{\left({\mathbf{u}}_i^{\top }{\mathbf{r}}_{t,i}\right)}^2$$

where $C$ is the number of conditions, $\mathbf{u}$ is a vector of PCA or SCA neuron-latent mappings, and $\mathbf{r}$ is a vector of firing rates. For each dimension, we calculated the fraction of the total occupancy accounted for by the preparatory, execution, or postural epochs (e.g., Fig. 2H, left). For example, the fractional occupancy accounted for by the preparatory epoch is defined as:

$$\text{Occ}\_{\text{fraction}}_{\text{prep}}(k)=\frac{{\text{Occ}}_{\text{prep}}(k)}{{\text{Occ}}_{\text{prep}}(k)+{\text{Occ}}_{\text{exec}}(k)+{\text{Occ}}_{\text{post}}(k)}$$

Where $\text{Oc}{\text{c}}_{\text{prep}}$, $\text{Oc}{\text{c}}_{\text{exec}}$, and $\text{Oc}{\text{c}}_{\text{post}}$ are the sum of the occupancies during the preparatory, execution, and posture epochs. These three epochs are defined below.

epoch	time window(s)
preparatory	$T_{\mathit{\text{target}}}:T_{\mathit{\text{reach}}},T_{\mathit{\text{return}}}-300:T_{\mathit{\text{return}}}$
execution	$T_{\mathit{\text{reach}}}:T_{\mathit{\text{reach}}}+300,T_{\mathit{\text{return}}}:T_{\mathit{\text{return}}}+300$
posture	$T_{\mathit{\text{reach}}}+300:T_{\mathit{\text{return}}}-300$

$T_{\mathit{\text{target}}}$, $T_{\mathit{\text{reach}}}$, and $T_{\mathit{\text{return}}}$ are the times of target onset, outward reach onset, and return reach onset, and the time ranges are defined in milliseconds.

To summarize the distribution of occupancy within a single dimension, we calculated ‘occupancy concentration’: the sum of the absolute difference between all fractional occupancy pairs:

$$\text{Occ}\_\text{concentration}\left(k\right)=\mid \text{Occ}\_{\text{fraction}}_{\text{prep}}\left(k\right)-{\text{Occ}\_\text{fraction}}_{\text{exec}}\left(k\right)\mid +\mid \text{Occ}\_{\text{fraction}}_{\text{prep}}\left(k\right)-\text{Occ}\_{\text{fraction}}_{\text{post}}\left(k\right)|+\mid \text{Occ}\_{\text{fraction}}_{\text{exec}}\left(k\right)-\text{Occ}\_{\text{fraction}}_{\text{post}}\left(k\right)\mid$$

The maximum possible occupancy concentration for a single dimension would be 2 (e.g., fractional occupancies of 1, 0, and 0 for the preparatory, execution, and posture epochs, respectively). To summarize occupancy concentration across all dimensions, we simply calculated the median concentration across all dimensions (Fig. 2H, right).

Supervised dimensionality reduction:

To compare SCA and PCA latents to those recovered by a supervised method, we used the dimensionality reduction method documented in²⁵ (Fig. 3). This method identifies orthogonal sets of dimensions that capture a large fraction of variance during user-defined epochs:

$$\widehat{Q_{\text{prep}}},\widehat{Q_{\text{exec}}},\widehat{Q_{\text{post}}}=\text{arg}\underset{Q_{\text{prep}},Q_{\text{exec}},Q_{\text{post}}}{\text{max}}\frac{1}{2}(\frac{\text{Tr}\left(Q_{\text{prep}}^{\top }{C}_{\text{prep}}{Q}_{\text{prep}}\right)}{{\sum }_{i=1}^{d_{\text{prep}}}{\sigma }_{\text{prep}}\left(i\right)}+\frac{\text{Tr}\left(Q_{\text{exec}}^{\top }{C}_{\text{exec}}{Q}_{\text{exec}}\right)}{{\sum }_{i=1}^{d_{\text{exec}}}{\sigma }_{\text{exec}}\left(i\right)}+\frac{\text{Tr}\left(Q_{\text{post}}^{\top }{C}_{\text{post}}{Q}_{\text{post}}\right)}{{\sum }_{i=1}^{d_{\text{post}}}{\sigma }_{\text{post}}\left(i\right)})$$

$$\text{subject}\text{to}{Q}_{\text{prep}}^{\top }{Q}_{\text{exec}}=0,Q_{\text{prep}}^{\top }{Q}_{\text{post}}=0,Q_{\text{exec}}^{\top }{Q}_{\text{post}}=0$$

$$Q_{\text{prep}}^{\top }{Q}_{\text{prep}}=I,Q_{\text{exec}}^{\top }{Q}_{\text{exec}}=I,Q_{\text{post}}^{\top }{Q}_{\text{post}}=I$$

Where $Q$ are the loadings for the preparatory, execution, and posture-related spaces, $C$ are the covariance matrices of neural activity during preparatory, execution, and posture-related epochs, and $\sigma (i)$ is the $i^{\text{th}}$ singular value of $C$. For each epoch, we requested 12 dimensions, which were used as ground-truth computation-specific factors when evaluating other methods (Fig. 3).

In Figure 3, for properly supervised dimensionality reduction, the model was provided with preparatory, execution, and posture-related epochs. For naively supervised dimensionality reduction, the posture-related epoch was not provided; instead, execution and posture-related epochs along with the intervening time between them were grouped into a single “execution?” epoch.

Demixed PCA (dPCA):

We applied demixed PCA using reaching directions and time as task parameters. The model was fit by requesting eight components in each marginalization, and we restricted our analysis to the factors within direction-time interaction marginalization - that is, the eight condition-dependent factors.

Bi-cross-validation:

As a stringent test of whether SCA recovers structure that exists across conditions during reaching, we performed a bi-cross-validation analysis, which required generalization to both held-out conditions and held-out neurons. We first fit SCA using data from all conditions and a set of training neurons (90% of all neurons). We then used linear regression to fit the loadings between test-neurons and factors, using activity from 7 of 8 conditions. Finally, we use factor activity from the held-out condition and the loadings recovered via regression to predict the activity of the held-out neurons during the held-out condition. We repeated this process 80 times (10-fold validation for the neurons, 8-fold validation for the conditions).

Bimanual cycling analyses

Validating existence of a single speed dimension during bimanual cycling:

Training SCA on bimanual cycling activity from monkey E yielded a single dimension that was highly correlated with the monkey’s average cycling speed over a cycle (Figure S16E). To validate this result, we used linear regression to identify a single speed dimension from the neural activity from one condition. We then asked whether activity in this dimension was correlated with cycling speed during a second condition. More specifically, we used linear regression to solve for $\mathbf{b}$:

$$\mathbf{y}=\mathbf{\text{Xb}}$$

Where $\mathbf{\text{y}}$ is a vector corresponding to the angular cycling speed for a single condition after low-pass filtering (third-order Butterworth, cutoff frequency: 0.8 Hz) and $\mathbf{X}$ is a $T\times N$ matrix of trial-averaged firing rates for one bottom-start condition. To prevent overfitting, we used ridge-regression with $\lambda =0.01$. After calculating $b$, we projected neural activity from all top-start conditions into the same dimension and calculated the $R^2$ between the projection and low-pass filtered angular speed for each condition.

C. elegans analyses

Correlating factors with behavior:

The magnitude of individual SCA factors often correlated with the onset of specific behavioral motifs (e.g., Fig. 6C). To quantify this relationship (Fig. 6E), we generated boolean vectors that corresponded to when each individual behavioral motif occurred for each worm. We convolved each vector with a gamma distribution (shape parameter: 2, scale parameter: 5) to produce a continuous behavioral trace, then calculated the correlation between these behavioral traces and each SCA (or PCA) factor.

Generating pseudo-factors:

SCA often recovered qualitatively similar loading vectors for the same behavioral motif in different worms (e.g., Fig. 6G). We quantified this relationship in two ways. First, we measured the angle between loading vectors (Figure S18A,B). Second, we generated ‘pseudo-factors’ from the loadings of one worm and the neural activity of a second. If SCA is recovering truly similar loading vectors for different worms, then the magnitude of the pseudo-factor should correlate with the behavior of the worm that provided the neural activity.

For Figure S18C,D, we generated pseudo-factors using the loading vectors related to factors with high correlations to behavior. Specifically, we identified the loading vectors associated with the highest 25% of these correlations. We then generated pseudo-factors for each second worm (i.e., the worm other than the one that provided the loading vector) that exhibited the relevant behavioral motif. The critical result – that pseudo-factors generated from SCA loadings were more correlated than those generated from PCA loadings – held if we constructed pseudo-factors from all of the original loading vectors (mean correlation(standard error): 0.34(0.009) vs. 0.29(0.008), p < 0.001, SCA vs. PCA, rank sum test).

Nonlinear SCA - unimanual cycling quantitative performance

As a rigorous comparison between simple nonlinear autoencoders, SCA, and non-linear SCA, we performed a bi-cross-validation analysis across varying numbers of latent dimensions. For each model type, using $k$ latent dimensions (x-axis), we fit each model to 80% of the 116 neurons within motor cortex across all 20 trial conditions in the cycling dataset. We then trained a read-out regression model to map from the latent factors on 80% of the trials to the held-out 20% of neurons. We matched the expressivity of the read-out regressions to the corresponding decoders, using linear regression for linear models and a multi-layer perceptron (MLP) for nonlinear models. As shown in Fig. S22C, we also fit a MLP read-out to latents inferred using linear SCA. Finally, we evaluated the fit regression model by predicting activity of the same 20% of held-out neurons using latent factors on the 20% of trials withheld from the regression model. This entire procedure was repeated five times with independent splits of neurons and trial conditions, yielding a 5-fold cross-validation over neurons and a 5-fold cross-validation over trial conditions.

KEY RESOURCES TABLE

REAGENT OR RESOURCE	SOURCE	IDENTIFIER
Antibodies
Bacterial and virus strains
Biological samples
Chemicals, peptides, and recombinant proteins
Critical commercial assays
Deposited data
Experimental models: Cell lines
Experimental models: Organisms/strains
Oligonucleotides
Recombinant DNA
Software and algorithms
Sparse Component Analysis	This paper	https://github.com/glaserlab/sca.
scikit-learn	Pedregosa et al 2011110	https://scikit-learn.org/
Nimfa	Zitnik et al 2012111.	https://nimfa.biolab.si
Pytorch	Paszke et al 2019	https://pytorch.org
Other

Highlights.

• Sparse Component Analysis (SCA) is an unsupervised dimensionality reduction approach

• SCA parcellates activity into latent factors corresponding to separate computations

• SCA uncovers compositional reuse of latent factors across datasets