Addressing degeneracy in rodent studies of cognition

Abstract

In order to relate neural dynamics to cognitive computations, it is critical to determine whether subjects are employing behavioral strategies that actually use those computations, as opposed to simpler heuristics. Here, I describe the problem of adjudicating between degenerate behavioral algorithms and degenerate neural implementations in perceptual and value-based decision-making tasks. Multi-dimensional behavioral measures allow researchers to build arguments from numerous lines of evidence in favor of or against hypotheses about algorithmic strategies. Neural perturbations and individual differences can help resolve degenerate neural implementations. Rigorously contending with these issues is critical for leveraging rodent models to reveal mechanisms of cognition.

Introduction

Degeneracy is a phenomenon in which different solutions can produce identical outputs¹. DNA is degenerate, in that multiple codons can give rise to the same amino acid². Different combinations of ion channels can produce similar biophysical properties in neurons^3,4. At the circuit level, different combinations of synapse strengths, intrinsic conductances, and other neural properties can give rise to seemingly identical patterns of dynamic activity⁵.

The goal of this review is to highlight the problem of degeneracy in neuroscientific studies of cognition. Degeneracy poses challenges for studies across species, including rodents, non-human primates, and humans. However, in my view, the extent to which rodent studies contend with these issues is more variable and uneven than in primate studies. My hope is that greater awareness will increase the rigor and quality of mechanistic studies of cognition in rodents.

Marr famously argued that any information processing system, including the brain, can be studied at three levels of analysis⁶. The computational level seeks to understand the system’s goal or the problem it is trying to solve. The algorithmic level seeks to characterize the neural representations, transformations, and algorithms by which the computational goal is achieved. The implementational level describes the mechanisms by which those algorithms are instantiated in hardware, i.e., neural circuits. Most studies of cognition in animal models rely on fluid or food deprivation schedules and fluid or food reinforcers to incentivize task performance. Therefore, it is reasonable to assume that the computational goal of the animal is to maximize rewards and minimize various costs (effort, cognitive, time).

The thorny challenge is to identify the algorithms by which animals achieve that goal. Multiple algorithms can produce similar patterns of choice behavior, i.e. can be degenerate. Romo and colleagues highlighted this issue using a task in which they trained monkeys to compare the frequencies of two vibrotactile stimuli applied to the fingertip and report the higher frequency stimulus^7,8. The stimuli were separated by a delay to enable the study of working memory of the first stimulus during the delay period. However, if the first stimulus was fixed and only the second stimulus varied, the monkeys ignored the first stimulus and simply compared the second stimulus to a learned threshold, producing compelling psychometric functions and high accuracy⁷. Instead of fixing the first stimulus, when both stimuli varied randomly on each trial, the monkeys compared them, enabling the study of working memory^7–9. This is an example of algorithmic degeneracy: when the first stimulus was fixed, the threshold algorithm and parametric working memory both produced similar patterns of behavioral choices and psychometric functions.

At the implementational level, behavioral strategies can appear algorithmically identical, but differ in how they are implemented by neural circuits. Animal and human brains are remarkably robust to focal lesions or perturbations^10,11, suggesting that distributed networks can compensate for local perturbations or damage. This is an example of implementational degeneracy: multiple combinations of brain areas can support similar algorithmic strategies. This review will describe examples of algorithmic and implementational degeneracy during decision-making tasks, and provide basic prescriptions for resolving them.

Algorithmic degeneracy

There is a strong tradition of considering algorithmic degeneracy in accumulation of evidence tasks, in which subjects are presented with one or more sequences of momentary sensory evidence whose mean across time drives a binary decision^12,13. These tasks were designed to study the integration of evidence over time, and are often modeled as a drift diffusion process. Several algorithms can produce compelling psychometric performance in these tasks. In addition to integrating evidence throughout the entire trial, subjects might integrate evidence from brief, randomly selected epochs (a different 200ms window on each trial), or only integrate brief bursts of strong evidence. Because evidence in any time window reflects the generative trial statistics, it will likely provide evidence for the correct perceptual judgment, and that likelihood increases with stimulus strength. Therefore, these algorithms produce psychometric functions that are consistent with true integration (Figure 1).

Figure 1: Degenerate algorithms in accumulation of evidence.

Psychometric performance of model simulations solving the random-dot-motion task using different strategies: true integration, burst detection, or random snapshots. Reproduced from Stine et al., 2020 eLife.

One approach is to identify trials in which a non-integration algorithm would lead to errors: for burst detection, trials in which the first burst of stimuli favors the weaker evidence stream. Animals perform above chance on these trials, arguing against a burst detection algorithm^14,15. Experimental choices about task design, including whether stimuli are presented continuously or in discrete pulses, dictate which analyses are most informative for resolving algorithmic strategies^14–17. The psychometric function is necessary but not sufficient to infer the algorithm.

As another example, my lab developed a temporal wagering task in which rats reveal how much they value water rewards based on how long they wait for them^18,19 (Figure 2a). Rats experience blocks of trials with different average rewards, and how long they are willing to wait reflects the current reward and the block, such that rats wait less time for rewards in high reward blocks, when the opportunity cost is high^18–20, consistent with foraging theories^21,22 (Figure 2b–d). Rats could estimate the opportunity cost using degenerate algorithms. They might infer the hidden block and use a learned, block-specific estimate of opportunity cost, or estimate the opportunity cost via reinforcement learning, as the previous reward, or by divisive normalization²³ (Figure 2e).

Figure 2: Degenerate strategies produce block sensitivity in the temporal wagering task.

a. Schematic of behavioral paradigm. b. Block structure of task. c. Mean wait times on catch trials by reward in each block for one example, representative rat. d. Foraging theory-inspired behavioral model compares the value of the reward offer to the opportunity cost, or what the rat might miss out on by continuing to wait, which differs in each block. The starting point of the value function reflects the reward offer. e. Simulations of models that use different strategies to estimate the opportunity cost of time. The inference model infers the most likely block using Bayes’ Rule. The model-free reinforcement learning model estimates the opportunity cost as a running average of past rewards by a delta rule (simulation was with a learning rate of 0.3). The one trial back model treats the reward offer on the previous trial as the opportunity cost. The divisive normalization model divides the current offer by the average of past rewards. Units are arbitrary as scaling factors can determine the actual wait times in seconds.

We have used multi-dimensional behavioral measures, including the dynamics of behavioral changes at different block transitions, sensitivity to previous rewards, responses to highly informative single trials, and sensitivity to sequences of trials within a session, among others, to resolve the algorithm by which rats estimate opportunity cost^18,19. Across a range of metrics, rats’ behavior was most consistent with inferring the block and using a block-specific estimate of opportunity cost when deciding how long to wait. Bidirectional perturbation experiments and neural recordings have provided additional evidence for inferential strategies^19,24.

Within-trial degeneracy for different actions

Even on single trials, different algorithms can support sequential actions or aspects of behavior. In the temporal wagering task, rats also modulated how quickly they initiated trials based on the blocks, but multi-dimensional behavioral analyses revealed that they used a reinforcement learning algorithm (instead of state inference) to estimate the value of the environment when deciding how quickly to initiate trials^18,25–27. Initiation and wait times on the same trial, just seconds apart, reflected distinct algorithms that coarsely produced similar behavioral patterns¹⁸. Initiation and wait time decisions rely on distinct neural circuits: manipulating dopamine in the ventral striatum impacted initiation times but not wait times²⁶, while inactivating the orbitofrontal cortex (OFC) impacted wait times but not initiation times¹⁹. Studies in non-human primates have found evidence for similar phenomena²⁸. Multi-agent reinforcement learning models explicitly model behavior as reflecting combinations of different reinforcement learning algorithms^29–32. Monkeys performing the two-step task exhibited different combinations of reinforcement learning algorithms for choice versus response vigor on the same trials³³.

These results are also reminiscent of findings from the “Restaurant Row” task, in which humans and rodents make accept/reject decisions about whether to wait for rewards for a specified delay^34,35. Both the decision to accept the offer, and the decision to continue waiting, were sensitive to the value of the offer. However, only the decision to continue waiting was sensitive to sunk costs, in which the amount of time subjects waited increased their willingness to continue to wait. These sequential choices - accept an offer, wait for the offer - reflect distinct valuation processes that are differentially susceptible to sunk costs^34,35. Previous studies have shown that motivation, attention, and exploration can drift within sessions, causing different employed algorithms^36–39, indicating that algorithmic strategies can also be dynamic.

Prescriptions for algorithmic degeneracy

Certain task designs can reduce the possibility of degeneracy by discouraging the use of simple heuristics that would otherwise yield comparable performance. Memory-guided tasks present a transient stimulus whose location or identity must be reported after a delay period⁴⁰. While animals may keep the stimulus in working memory, they also might plan the behavioral report without necessarily keeping the first stimulus in memory. If one wants to study working memory separately from motor preparation, sequential delayed comparison paradigms, such as delayed match/non-match-to-sample tasks, separate the stimulus from the response but prevent the animal from anticipating the correct motor response during the delay period, although animals could still rely on embodied heuristics for working memory.

During task design, one should pre-emptively consider and design multiple, independent behavioral analyses that will be used to determine if the animal has actually learned the task, and is performing the desired algorithmic strategy. Multi-dimensional behavioral analyses that can adjudicate between degenerate strategies are only possible in tasks with a certain degree of richness. For instance, bandit tasks, which are widely used, provide low dimensional measures: binary choices and binary reward outcomes. Choice probabilities can be conditioned on reward history, but there are few other ways to evaluate behavior, making it difficult to develop model-independent analyses that can resolve algorithmic strategies. However, richness should be targeted, meaning tasks should include diagnostic manipulations that test competing hypotheses, not simply add complexity. For instance, in accumulation of evidence tasks, experimenters can parametrically vary the strength and precise timing of sensory evidence, as well as the duration of the stimulus. Variants of these tasks allow subjects to opt-out of the decision for a small but guaranteed reward, indicating their confidence in the perceptual judgment, which can be related to stimulus features^41–43.

Once one has settled on a task design, it is productive to develop behavioral models that implement different algorithmic strategies for the task. Simulating the behavior of these models can identify trial types or conditions in which they make different predictions, and thus inspire iterative improvements in task design and diagnostic behavioral analyses^{14,15,17,44–46} (for instance, trials in which the first burst of sensory evidence favors the incorrect choice in accumulation of evidence tasks). A model’s behavior depends on the parameter values that are used, so one might simulate the behavioral output of a model using the best parameters fit to data^44,45, or across a range of plausible parameter values⁴⁶. If parameters in a model trade-off, this limits interpretability.

How should one identify potential degenerate algorithms? If there are precedents in the literature, that is a good place to start. This is more likely if tasks are “deeply sampled” or well-studied across multiple experiments⁴⁷. If not, one will have to brainstorm, simulate possible heuristic strategies, and evaluate their performance. Addressing algorithmic degeneracy can be an extended process that spans multiple studies.

Neural recordings can help resolve algorithmic degeneracy. For centuries, economists were agnostic about the mental processes driving economic choices (based on subjective preferences) because there was no way to know if people were actually computing and comparing subjective values. One can only infer value from choices, so the behavioral definition of subjective value is circular⁴⁸. The finding that neurons in some brain regions encode value provides a measure of subjective value that is independent of choice, and breaks this circularity^49–54. As another example, the debate about whether neurons in monkey LIP exhibit ramping or stepping dynamics during evidence accumulation is a debate about LIP but also about algorithmic strategy: were the monkeys performing true integration, or a step-like algorithm like burst detection^55–57?

Degenerate neural implementations

Behavioral strategies can appear algorithmically identical, but differ in how they are implemented at the neural level. Work in mice and monkeys has shown that training history can change which cortical areas are required for behavior^58–60, indicating degenerate combinations of brain areas capable of supporting task performance. Studies of accumulation of evidence in monkeys have documented within-session behavioral compensation following inactivation of individual brain regions^61,62. The recruitment of unaffected circuits may contribute to such compensation. A study of accumulation of evidence in mice showed that behavior within a session fluctuated between states that appeared outwardly identical but differed in the causal requirement of the dorsomedial striatum⁶³. Animal and human brains are remarkably robust to focal lesions or perturbations^10,11, suggesting that distributed networks can compensate for local perturbations or damage. Adaptive recruitment of non-perturbed brain regions is thought to contribute to functional recovery following brain damage^64,65. Therefore, the same algorithmic strategy may reflect degenerate neural implementations: in these cases, reliance on different combinations of brain regions.

Perturbation experiments are more powerful for resolving how brain regions support behavior than regressing activity patterns against task variables, even if the recent proliferation of large-scale recordings methods facilitates the latter approach. The brain is a heavily interconnected dynamical system, and neural circuits can reflect task variables without causally contributing to behavior^66–68. Invertebrate central pattern generator circuits exhibit similar dynamics across subjects under baseline conditions, but temperature⁶⁹, pH⁷⁰, or modulatory⁷¹ perturbations reveal degenerate circuit implementations. Recurrent neural networks that learn about action values via synaptic plasticity or neural integration exhibit identical foraging behavior, but implementations are resolvable by perturbations⁷². Negative results of perturbation experiments can be difficult to interpret if compensation is rapid and difficult to resolve. Moreover, inactivating entire brain regions can produce null results that obscure a causal behavioral role for specific projection classes in those regions^73,74. However, perturbations that produce algorithmic deficits generate hypotheses about the behavioral or cognitive operation performed by the circuit^{9,19,24,43,75–77}, and distinguish whether a brain area’s activity is permissive versus instructive for behavior^78,79. Mice have become prominent models in neuroscience because of the availability of powerful experimental tools for monitoring but also for manipulating neural dynamics. Studies that exclusively rely on recordings fall short of realizing the potential of rodent models.

Implementational degeneracy was further highlighted by a recent study that leveraged high-throughput training to record from many rats performing a context-dependent accumulation of evidence task⁸⁰. They engineered recurrent neural network models (RNNs) that instantiated similar algorithmic strategies but different implementational mechanisms for context-dependent integration, and identified analyses of behavior and neural responses that distinguished these neural mechanisms. These analyses revealed that some rats achieved context-dependent accumulation by modulating the impact of sensory inputs on neural dynamics in a particular way, while others used different mechanisms, such as exhibiting distinct context-dependent recurrent dynamics. The authors resolved different neural mechanisms supporting similar algorithmic strategies⁸⁰.

Conclusions

Algorithmic strategies should be treated as hypotheses to be rigorously tested. Tasks designed with degeneracy in mind can support behavioral analyses that help adjudicate between degenerate algorithms. Behavioral models are powerful tools for identifying testable qualitative predictions about degenerate algorithms. Perturbation experiments should be used with behavioral modeling and thoughtful interpretation to identify deficits at the algorithmic level and resolve implementational degeneracy. Seriously contending with these issues is critical for characterizing neural mechanisms of cognition.