From Tripping and Falling to Ruminating and Worrying: A Meta-Control Account of Repetitive Negative Thinking

Abstract

Repetitive negative thinking (RNT) is a transdiagnostic construct that encompasses rumination and worry, yet what precisely is shared between rumination and worry is unclear. To clarify this, we develop a meta-control account of RNT. Meta-control refers to the reinforcement and control of mental behavior via similar computations as reinforce and control motor behavior. We propose rumination and worry are coarse terms for failure in meta-control, just as tripping and falling are coarse terms for failure in motor control. We delineate four meta-control stages and risk factors increasing the chance of failure at each, including open-ended thoughts (stage 1), individual differences influencing subgoal execution (stage 2) and switching (stage 3), and challenges inherent to learning adaptive mental behavior (stage 4). Distinguishing these stages therefore elucidates diverse processes that lead to the same behavior of excessive RNT. Our account also subsumes prominent clinical accounts of RNT into a computational cognitive neuroscience framework.

Why do we dwell on our flaws and past mistakes and fixate on the uncertainty of the future? And what do these tendencies have in common? An emerging consensus in clinical science is that perseverating on the past (rumination) and fixating on the uncertain future (worry) are intimately related—in fact, that both thinking patterns can be (partly) subsumed by a higher-order construct, such as repetitive negative thinking (RNT) [1–4]. This consensus is relatively new: for decades, rumination and worry had been thought of as both conceptually distinct and as differential risk factors for depressive [5] and generalized anxiety [6] disorders, respectively (reviewed in [4]). Yet, RNT is now understood as a transdiagnostic process: it is a risk factor for, and elevated within, most Axis I disorders [2,7], although most related to depression and anxiety [8]. And rumination and worry are now recognized as more similar than distinct [1,2,4,9]: factor analysis reveals that many rumination and worry items load onto a common factor that is more strongly associated with depression and anxiety than specific rumination and worry factors [9–13]; but see also [14]. The definitions of rumination and worry also clearly overlap: both thinking patterns are repetitive, difficult to control, and negatively valenced (although with somewhat different content, such as focus on the past vs. future; e.g., [1,2,4]; Figure 1).

Figure 1: Repetitive negative thinking (RNT) refers to what is shared between worry (W) and rumination (R).

This paper presents a computational account of what these thinking patterns share and also touches on how they diverge (see refs [4,8–11,13–15]).

In short, there is mounting psychometric, conceptual, and diagnostic evidence that rumination and worry are related. Yet, what is missing is an interdisciplinary basic scientific framework that makes testable predictions about the specific computations shared between rumination and worry. Here, we develop such a framework, grounded in the computational decision sciences [16]. From a decision-making perspective, rumination and worry both reflect attempts to facilitate adaptive future decisions. Indeed, a rational agent would have no reason to perseverate on the past or prospect about the future unless they estimated that doing so would improve their current or future decisions [17]. RNT may be what happens when this typically adaptive process goes awry.

Our account, which we describe in detail below, offers some key advantages over prior work. First, it outlines conditions under which meta-control is more likely to fail (resulting in worry or rumination) vs. when it is more likely to be productive (leading to adaptive problem solving or productive reflection). Hence, it overcomes the problem that some clinical accounts merely stipulate that rumination and worry are unproductive, without clarifying why or how (see [3,18–20] for other clarifications). Second, by grounding RNT in computational cognitive neuroscience, our framework should help to organize findings on the neural circuits and neuromodulators implicated in RNT [21–23]. Finally, our framework identifies distinct pathways—ranging from neurocomputational individual differences to life stressors to learning challenges—that increase the chance of meta-control failure and thus excessive RNT. Crucially, excess RNT, especially by vulnerable individuals, appears to be a key pathway to episodes of depression, anxiety, or other psychopathologies at specific time points [24,25]. Insight into the distinct pathways that lead to excess RNT may therefore facilitate powerful and tailored interventions that can prevent these episodes or mitigate their damage.

Our account is specifically based in meta-control research from computational cognitive neuroscience [21–23]. Theoretical and empirical research and neural-network modeling in the last two decades have suggested that adaptive control over cognitive actions, such as gating content into and out of working memory, scaffold on top of reinforcement learning mechanisms that evolved to learn the consequences, and guide the selection of, motor actions (e.g., [26–30]).

Here, we extend this insight to consider gating into working memory hypotheses with the potential to initiate rumination or worry, such as “I am socially incompetent” or “My spouse is ill.” Based on prior work [26,31], we identify gating in an overarching task as the first of four stages of meta-control (Figure 2). We take “rumination” and “worry” to be generic labels for what happens when one or more stages of meta-control fail, leading to an unproductive episode of thinking (just as “tripping” and “falling” are generic labels for what happens when one or more stages of motor control fail, leading to errant motor behavior¹). We equate rumination and worry with failure because these thinking patterns are often considered unproductive by definition [5,6] (although we acknowledge that other accounts use these terms differently (e.g., [3,20,32]).

Figure 2: The four stages of meta-control illustrated with mental multiplication (left) and extended to affective hypothesis evaluation that might instigate rumination (right).

This figure illustrates the stages of meta-control in two scenarios: In the left scenario, you are challenged to solve a difficult mental-multiplication problem in your head (34 × 19). In the right scenario, you are at a party conversing with someone who suddenly interrupts you, claiming that they need to use the restroom—leading you to consider the hypothesis “I am socially incompetent.”

Stage 1 of meta-control is to gate and maintain an overarching task goal in mind. In the left scenario, you must remember that you are multiplying 34 and 19 (rather than subtracting, adding, or mentally rearranging them). In the right (affective, self-relevant) scenario, you are trying to evaluate the hypothesis “I am socially incompetent.”

Stage 2 of meta-control is to complete a sequence of subproblems in the service of the overarching task. In the left scenario, completing just two subproblems solves the task : (i.) finding the easy product 34 × 20 and (ii.) subtracting 19 from it. In the right scenario, by contrast, the set of subproblems is much more open-ended: you might (i.) recall some memories and beliefs about other times you felt embarrassed at parties, (ii.) think of problems that you have had in your relationships, and (…) focus on any number of other emotionally or topically pertinent subproblems—each of which might elicit further subproblems. (In the figure, the circles represent mental activity patterns constituting different memories and beliefs represented by different colors.)

Stage 3 of meta-control is to switch between subproblems. Switching requires sufficient confidence that the previous subproblem has been solved or addressed. In the left scenario, you must be confident that 34 times 20 is 680 to move on to subtracting 34 from that product. In the right scenario, you must be sufficiently satisfied with the memories and beliefs that come to mind related to embarrassing yourself at parties to move on to relationship problems.

Stage 4 of meta-control is reinforcement of the operations that constituted the episode. Sometimes, reinforcement might be external. In the left scenario, the person who challenged you to mentally multiply might say, “You’ve got it! 646 is the answer.” In the right scenario, you might see evidence that seems to confirm your hypothesis, such as that your former conversation party has begun a new conversation without having stopped at the restroom. Yet, reinforcement can also presumably come internally—such as if you are confident that you have solved the multiplication problem, as well as from beliefs, internalized norms or goals (see [33]), and replays of an experience that seem to support your hypothesis [34]. Whatever its source, it may be more difficult to ascribe reinforcement to stages of meta-control far in time from its receipt, especially when many mental operations occur before receipt. (In the figure, reinforcement is denoted with a check mark and decreasing credit ascribed to earlier operations via fading gold stars.)

Four Stages of Meta-control Failures Leading to Rumination or Worry.

We identify four meta-control stages [26,31] at which failure may occur. Figure 2 introduces the four stages with the straightforward example of mental multiplication (left side) followed by hypothesis evaluation that might lead to RNT (right side). The remainder of this paper reviews the four stages and risk factors that make failure more likely at each stage. Our account highlights that numerous prominent clinical theories of RNT focus on a single stage of meta-control and forges links to decision science at each stage. It is thus unifying and consilient.

Stage 1 (Outer loop): Hypothesis selection.

This stage refers to the overarching task that guides an episode of meta-control. In meta-control research (e.g., [26,27]), the overarching task (outer loop) is often experimentally defined, and sometimes it is also straightforward in everyday life—such as when mentally multiplying two numbers (Figure 2, right; although see Box 1 for discussion of complications for even these problems). Yet, many overarching tasks, such as evaluating a hypothesis about oneself (Figure 2, right), are more nebulous and open-ended.² An influential clinical account argues that RNT is especially likely when “general, superordinate, and decontextualized mental representations” ([37], p. 260) guide (sub)goal or action selection [3,18,37]. From a meta-control perspective, a key issue with open-ended hypotheses is that they lead to numerous subproblems (see Figure 2, right, stage 2)—none of which have much chance of definitively resolving the hypothesis itself, or informing specific external actions. By contrast, concrete hypotheses (such as “I angered my friend at the party last night”) lead to more well-defined subproblems and associated actions (such as “I wonder if my friend did not like it when I teased her partner?” and plans to change specific future behavior).

Box 1: The challenge of defining and reinforcing subproblems.

The main text focuses on the distinct challenges of completing subproblems related to concrete vs. open-ended hypotheses, treating the set of subproblems as pre-defined. For instance, it assumes that the two subproblems shown in (Figure 2, left) will be the ones pursued to solve the mental-multiplication problem. In reality, defining subproblems is a challenging problem in its own right even for concrete hypotheses, to say nothing of more open-ended ones. For instance, one could attempt to solve the multiplication problem the long-form way, or via another multi-step strategy, such as using three steps all involving products and sums: 34 × 10 + 30 × 9 + 4 × 9.

That multiple subproblems can be pursued in the service of the same goal poses further challenges for credit assignment (Stage 4). For instance, variable strategies will lead to varying completion times of the overarching task; different subproblems will themselves vary in completion time due to fatigue and practice; and different strategies may lead to entirely different outcomes for the same overarching task. For instance, an attempt to solve 34 × 19 via a long-form strategy might lead to continued forgetting and needing to restart, and ultimately giving up—even for someone capable of solving the problem with a smarter strategy, such as the two-step solution in Figure 2.

In the case of affective open-ended hypotheses, the challenge of defining a set of subproblems compound, making credit assignment yet more challenging. A variety of other complications also come into play: for instance, negatively valenced hypotheses might trigger hard-wired Pavlovian responses that prevent gating and thus preclude subgoal execution [42] or undermine instrumental reinforcement (if one believes that an overarching negative hypothesis has been confirmed—and the resulting conclusion feels awful). Such interactions could be investigated experimentally in paradigms where the confirmation of a hypothesis can be dissociated from the valence of the outcome (e.g., [43]).

Subproblem selection and its consequences for reinforcement are also relevant to intervention. Repeatedly practicing mental behaviors learned in therapy (e.g., cognitive defusion, asking “how” rather than “why” questions, Socratic questioning) may make them easier to implement, and thereby decrease variability in delays and increase the chance of consistent success—leading to more consistent reinforcement. Indeed, computational modeling clarifies that meta-learning an effective control or gating policy can enhance the efficiency of computations and thereby maximize reward. Specifically, the policies that are easiest to learn via credit assignment may be those that are more efficient to use, because they involve fewer and clearer subgoals to complete [26,27,44,45]. One implication for psychotherapy theory is that certain skills might be recast as subproblems of others, rather than as wholly distinct strategies. For instance, cognitive defusion and asking how vs. why questions might be initial skills that pave the way for more complex strategies, such as Socratic questioning. This provides an alternate framing for debates about how distinct certain strategies are from each other [39] and raises the possibility of shaping complex skills by first establishing more basic ones [45].

Although selecting concrete (vs. open-ended) hypotheses may often lead to more tractable subproblems, concrete hypotheses can of course elicit more open-ended ones (“I angered my friend at the party” might lead to thinking “I am socially incompetent”). Notably, post-event processing [34], which has been formalized in terms of upward counterfactuals [38], is implicated in social anxiety and other psychopathology—implying that the tendency to revisit specific event details is a risk factor in its own right. Nonetheless, our account highlights that such thinking is most likely to be prolonged and unproductive when thoughts stray from specific situations and events and become open-ended.

Open-ended hypotheses pose a couple specific challenges from a computational perspective. For one, the inferences involved in such hypotheses (such as about one’s own and others’ traits, about dynamics like gossip propagating through complex social networks) may be intractably complex [39]. Second, when a hypothesis spawns many subproblems, this presumably makes it difficult to robustly maintain the hypothesis itself in mind [26] and to ascribe consequences to it [40] (see Box 1 and Stage 4).

The fraught nature of open-ended hypothesizing is well recognized. Indeed, key strategies in RNT treatments are to promote concrete thinking (thinking grounded in specific events and details and focused on “how” instead of “why” questions) and to track the deleterious consequences of open-ended thinking (e.g., [41]). Our account lays basic scientific groundwork for such strategies by identifying hypothesis evaluation as the first stage of meta-control. This should help to elucidate how open-ended hypothesizing interacts with risk factors at other stages.

Stage 2 (Inner Loop 1): Executing subgoals.

Once subgoals have been defined, they must be executed—and certain individual differences in execution may predispose toward RNT. Some differences may comparably increase the tendency to ruminate or worry. For instance, trait neuroticism may predispose toward RNT simply because more negative memories and polarized beliefs come to mind, irrespective of whether they concern the past or the future. Interestingly, self-reported “stickiness” of thinking predicts task disengagement [46] and may be a subjective marker of protracted subgoal execution. Other individual differences may predispose specifically to rumination or worry (see [14]). For instance, self-referential thinking is more core to the construct of rumination than worry and thus may have a stronger relationship to it (divergent validity). Recent years have seen advances in techniques for investigating self-judgment, ranging from computational modeling to event-related neuroimaging to real-world prediction (e.g., [47–50]). Yet, most work focused on depression rather than rumination (but see [51]) and no work of which we are aware examined divergent validity. Conversely, uncertainty aversion is more core to the construct of worry than rumination. Thus, individual differences underlying this tendency—in the tendency to differentiate within semantic space [52], for instance—may relate more to worry than rumination, although again divergent-validity studies are lacking.

Stage 3 (Inner Loop 2): Switching between subgoals.

Once a subgoal is completed, it must be recognized as such so that the next subgoal can begin. Recognition of subgoal completion is relatively straightforward in mental multiplication, but less so for open-ended hypotheses, which may lead subproblems to be processed in a disorganized way and with much revisiting (Figure 2).

Influential clinical theories posit that rumination and worry arise from a difficulty inhibiting, or switching between, mental representations. Stage 3 is one stage where this may pose difficulties—in completing subgoals and switching to new ones (see also [53]). Yet, there is disagreement about whether such difficulties occur with all types of content, or only negatively valenced content—in which case it is questionable whether the problem is switching per se [54–57]; see also [25,58]. [55] reviewed evidence suggesting that rumination may relate, in part, to a trait-like proclivity for stability over flexibility. The idea is that individual differences in this balance are neither uniformly good nor bad, but rather are helpful in different settings: stability is helpful when it is necessary to concentrate on a focal task and avoid distractions, and flexibility is helpful when task switching is needed. Yet [55] noted that higher trait-like stability will increase the propensity to ruminate when it is coupled with individual differences (e.g., neuroticism) or contexts (e.g., distressing life events) that increase negative affect (especially given that negative affect itself may stabilize cognition [55]).

It is still an open question whether RNT relates to valence-independent differences in stability vs. flexibility, in a way that can be consistently assessed behaviorally—especially given considerable practical challenges with such assessment (e.g., [59]) evident since the literature review in [55] and that most research has focused on rumination rather than worry or transdiagnostic RNT. One reason this question matters is that dopamine appears to regulate stability vs. flexibility balance, with higher striatal dopamine linked to flexibility and higher prefrontal dopamine linked to stability [60]. Such individual differences may inform tailored RNT treatments.

One way to stop perseverating on open-ended hypotheses besides completing subgoals is to engage with something external, such as an attention-demanding task [61] or a cue to potential reward [62]. Yet, when the propensity for RNT becomes entrenched, it may interfere with external tasks [46,63], including learning about reinforcement contingencies [64,65] (reviewed in [18,62,66]). Disrupted learning is an especially insidious effect, because it not only alters behavior in the moment, but also in related situations in the future, and because it may set off a feedback loop of withdrawal from the external world and preoccupation by RNT ([65] for discussion).

Stage 4: Learning from consequences.

Crucially, meta-control concerns not only the control of mental behavior but also learning its consequences through reinforcement, so that (ideally) mental behavior becomes more adaptive through learning [21,26,28]. Thus, the final stage³ of meta-control is learning the consequences of the mental operations performed. Many clinical accounts of RNT view it as governed by the principles of operant conditioning [67–70], hence meta-control is a natural formal framework [40].

One key learning challenge is that, unlike motor actions, mental behaviors entail carrying out one or more mental operations. Operations are thus necessarily interposed between the initiation of a mental action and the receipt of its outcome (for instance, subproblems are interposed between gating a hypothesis into mind and receiving reinforcement about the gating operation itself). In a task that directly compared the ability to learn optimal mental vs. motor behaviors through trial and error, we found that people had more difficulty learning optimal mental behaviors. Thus, it may be especially difficult to learn the consequences of the operations that constitute RNT—and to learn adaptive thinking patterns more generally [40]. The intrinsic challenge of learning about mental behavior might help explain why people are apt to neglect its negative consequences, as posited by many RNT theories [67–70]. Whereas [40] considered learning about simple mental operations that were well-defined in our task (in order to isolate the challenge of learning adaptive cognitive actions even in a simple case), as noted computational modeling has shown that credit assignment is more challenging when multiple subproblems are required (Stage 1) and multiple strategies can be used, some of which are more readily ascribed credit than others (Box 1) [26,27,44,45]. There also may be individual differences in learning from mental behaviors per se; we are currently investigating this possibility.

These learning challenges may be one reason that people are so prone to acquiring mental behavior with poor long-run consequences (such as gating in abstract hypotheses that might instigate RNT). An additional challenge, besides difficulty learning RNT’s negative consequences, is that RNT may actually have some positive consequences— which might outweigh its negative consequences (especially if they are more difficult to learn). Indeed, numerous sources of reinforcement for RNT (i.e., secondary gains) have been proposed, such as that people ruminate to conform to unjust gender norms [5,71], because it facilitates social bonding (e.g., through co-rumination [72]), or that it justifies abstention from their commitments [68]. One prominent account argues that RNT functions to preemptively lower affect so that, when negative experiences occur, they do not lead to a precipitous drop in affect [73]. This theory could be formalized via reinforcement learning if RNT decreases affect by lowering expectations, so that subsequent prediction errors are more positive. Increasing the plausibility that RNT functions to deflate expectations in this way, prediction errors—which are defined as outcomes minus expectations—have a strong influence on momentary mood [74]. However, the mechanics of ascribing reinforcement to an action that is aversive, yet which is believed to preclude a more-aversive outcome, are unclear (but see, e.g., [75,76], for general evidence that counterfactual outcomes and not choosing an action both influence reinforcement learning).

Interactions between stages.

A key advantage of our framework is that it allows theorizing about how individual differences will affect multiple stages and how stages interact. As one example, here we focus on how similar individual differences may influence both overarching hypotheses (stage 1) and subgoal execution (stage 2). Above, we discussed how negative self-judgment can influence subgoal execution (stage 2), yet a heightened tendency toward self-referential thinking is also tantamount to a tendency to consider open-ended hypotheses (stage 1)—given that self-schemas are by definition abstractions devoid of specific context [77]. From a computational perspective, decontextualized representations are thought to arise from the gradual incorporation of representations of specific, context-laden experiences stored in the hippocampal system into decontextualized representations of what is shared between them in neocortex [78]. One reason to build such representations is to enable recognizing that superficially distinct situations share contingencies (e.g., stimulus-response) [27]; see also [79] for discussion. Although this may often be adaptive, overgeneralization is also possible, such as when long-held negative views about oneself lead to an inaccurate assumption of poor efficacy in new situations [77].

Past computational research on self-judgment has focused on judgments and memories separately from learning (e.g., [47–50]). And research on the inference of shared learning contingencies has used arbitrary stimuli (e.g., fractal images) when studying situations that may share a (hidden) contingency structure (e.g., [27,80]). One novel direction would be to join these lines of research and examine how specific individual differences in self-referential processing (first examined separately) influence the inference of shared contingencies when self-referential (instead of arbitrary) stimuli are used as cues to shared structure.

Life events may also influence both stages. For instance, [20] construed rumination as fundamentally tied to goal progress: When a specific goal is blocked (such as failing to find a part while assembling a desk) the resulting repetitive thinking will be very specific, whereas thinking will be more abstract and open-ended in the face of setbacks (e.g., job loss) that influence many goals at once (Stage 1). Different overarching problems will also lead to different subgoals (Stage 2). For instance, life events involving death and separation are more predictive of depression than generalized anxiety symptoms, whereas those involving loss and danger are more predictive of generalized anxiety than depression [81]. This makes it plausible that distinct event types also differently predict rumination and worry (see [24,25]). Our framework clarifies that, although the content of the problems elicited by different life events will diverge, major life stressors will have a shared effect at the process level: namely, promoting open-ended thinking.

Conclusion.

We developed a basic scientific framework for understanding rumination and worry as failed meta-control. Of note, by introducing our framework in this brief article format, we were only able to cite rather than review some of the key literature that informed it (e.g., the neural bases of meta-control and RNT, executive function differences that might confer vulnerability for and/or result from RNT, methods of measuring RNT). The scope of our framework is also currently limited to RNT alone; we did not attempt to locate it within the wider constructs of perseverative or repetitive thought (e.g., [4]) or intrusive thinking and low mindfulness (e.g., [15,82]). Also regarding scope: Our framework focuses on RNT as active attempts to resolve a hypothesis gone awry, rather than passive RNT arising from habitual ineffective thinking patterns (although it is naturally compatible with habit-focused accounts, e.g., [67], and reinforcement-learning formalizations thereof).

Notwithstanding these limitations, we outlined a framework that establishes consilience between research on RNT and the decision sciences; elucidates diverse risk factors that can each lead to excess RNT; and lays a basic-science groundwork for specialized treatments [83] for rumination [41], worry [84], and transdiagnostic RNT [85,86]. Our framework makes testable predictions about how distinct processes (e.g., individual and/or situation-based differences in proneness to open-ended thinking, trait-like stability over flexibility, challenges of learning about mental behavior especially when compounded by open-ended thinking) may lead to the same behavioral output of excessive RNT.

Highlights:

• Rumination and worry overlap, yet what computations they share is unclear.

• We propose a four-stage meta-control theory based in computational cognitive neuroscience.

• When thinking fails at any of the four stages it can lead to rumination or worry.

• Rumination and worry are thus distinct forms of repetitive negative thinking with shared computational bases but also some important differences.

• Our theory makes testable predictions about distinct conditions and individual differences that produce the same output: repetitive negative thinking.

• Our theory integrates prominent clinical and computational cognitive neuroscience theories.

• It should promote consilient research across levels of analysis.

Glossary

Divergent validity

theoretically meaningful discriminability of one measure from another

Gating in

according to computational neuroscience models and experimental data (e.g., [26,28–31]), people can “gate” content into working memory, allowing it to be stored in a more robust form in the face of distracting or task-irrelevant content. Gating policies can be acquired via reinforcement learning (e.g., if entertaining and maintaining a joke in mind leads to actually telling the joke and people laughing in response, these processes will be more likely to be repeated in the future). Gating is assumed to be determined by prior experience and the current context and thus may happen partly out of voluntary control, although it also may be modulated by partly voluntary processes, such as thought suppression (e.g., [87]).

Meta-control

learning via reinforcement, and controlling the execution of, policies for mental behavior (e.g., working memory operations)

Pavlovian responses

“hard-wired” approach or avoid responses that can sometimes conflict with instrumental ones (such as withdrawing from a negatively valenced stimulus even if approaching it would lead to reward, or failing to entertain a negative hypothesis even if it would be in one’s best interest to do so)

Post-event processing

threatening appraisal of a social or other event after it has occurred

Secondary gains

positive consequences of behaviors that are maladaptive on balance

Trait neuroticism

the stable tendency to experience strong and frequent negative reactions