What is the nature of “internal content” prior to attentional selection?

A recurring distinction in attention research is between external attention, which selects among perceptual objects/representations, and internal attention, which selects among cognitive representations like memories, mental images, or goals (Chun et al., 2011). In contrast to our detailed knowledge of the mechanisms supporting external attention, the dynamics of internal attention remain poorly understood. Amir & Bernstein (this issue) develop an ambitious model that seeks to fill this gap and account for how internal attention integrates information from working memory, emotional states, and contextual demands to select among cognitive representations. To put it differently, in William James’s oft-quoted definition of attention as “the taking possession by the mind… of one out of what may seem several simultaneously possible objects or trains of thought”, researchers have often focused on the former target (perceptual objects), but rarely on the latter (trains of thought). Amir & Bernstein’s are to be applauded for their comprehensive attempt to characterize the dynamics of internal attention, especially as it relates to psychiatric symptoms like rumination.

We are generally quite enthusiastic about the dynamical systems approach developed by Amit & Bernstein to characterize internal attention in a computationally explicit manner. However, in this response, we highlight a fundamental question about internal attention dynamics that is left unaddressed in the model of Amit & Bernstein. The question revolves around the nature of the internal cognitive representations that become targeted by internal attention. Amir & Bernstein are clear that, once the representations are attended to, they enter working memory. But what is the nature of the representations before they are attended to? Here we suggest that there are three possible ways to interpret Amit & Bernstein’s characterization of their model, which are left ambiguous and implicit in the authors’ account. We refer to these as the generation, activation, and selection interpretations. Critically, these differing interpretations each have important implications for our understanding of the dynamics of internal attention. Below, we elucidate these three interpretations, spell out their implications, and propose avenues for testing them experimentally. In particular, we propose that experimenters could profitably employ cognitive neuroscience methods to distinguish these interpretations empirically.

Background

There is a basic picture of the internal landscape of the mind that Amir & Bernstein take for granted (illustrated in Figure 2 of their paper). According to this picture, there are a multitude of internal, cognitive representations – such as “spontaneous associative thoughts and memories”, “goal-directed thoughts” (p.19), mental images, etc. – which are competing for limited-capacity conscious processing. Internal attention functions as a bottleneck, selecting some of these representations into working memory (where they receive conscious processing), and leaving the others to remain unconscious. (In this response, we will follow Amir & Bernstein in the simplifying assumption of equating activation in working memory with being available to conscious awareness.)

This picture draws heavily on the analogy between internal and external attention. In the domain of external attention to perceptual targets/representations, something like this picture is well established (Dehaene et al., 2006). When viewing a complex visual scene, for instance, there is far more perceptual detail than can be consciously processed; attention helps gatekeep which percepts enter consciousness or working memory; and the unattended-to percepts typically remain outside awareness (Dehaene et al., 2006; Dehaene, 2014; Posner, 2011; Van Boxtel et al., 2010).¹ Dehaene et al. (2006) refer to these unattended-to percepts as “preconscious” – strong enough that they could have entered awareness, but left unconscious due to inattention.

However, there is significant ambiguity about how this picture applies to internal attention. In particular, in Amir & Bernstein’s model it is unclear what the nature is of the internal representations before they are attended to. Their model draws on at least three possible interpretations, between which they do not sufficiently differentiate.

The “generation” interpretation

First, Amir & Bernstein sometimes seem to be referring to situations where the internal representations did not exist before being “attended to”. Take, for instance, their discussion of repetitive negative thinking. They write (p.45):

“In A2T, repetitive negative thinking will be initiated, typically, when a negative representation (thought) is selected into WM in a state characterized by low contextual-demands for sustained focused attention. This representation then triggers or increases negative affect. Consequently, both the representation and affective state bias subsequent selection in favor of content- and affect- congruent thoughts (e.g., “Bad things always happen to me.”).”

It is unlikely that there were a multitude of fully-formed thoughts of different valences (e.g. “Bad things always happen to me”, “Good things always happen to me”, “Good and bad things both sometimes happen to me”, etc.) active in this hypothetical person’s mind, competing for entry to working memory. Rather, in repetitive negative thinking, the negative affect (or negative content in working memory) seem to feed back into a process that creates or generates more negative thoughts, not a process that selects between them.

We put “attended to” in scare quotes above because, on this interpretation of Amir & Bernstein’s model, their focal process likely does not qualify as attentional. It is not selecting between existing, competing representations; it is creating new ones. The process modeled by Amir & Bernstein in this case is more appropriately described as thought generation, not selection, and there’s little reason to suspect it would involve the same dynamics as internal attention. For instance, while it is plausible for internal attention to have similar dynamics across different domains of internal cognition (Chun et al., 2011), it seems much more likely that processes that generate both thoughts (e.g. “Bad things always happen to me”) and other types of cognitive representations (goals, plans, imaginations, etc.) will differ enormously across domains. The model that Amir & Bernstein propose to account for the generation of repetitive negative thoughts, then, is unlikely to generalize to other generative internal processes.

The “activation” interpretation

The second interpretation that Amir & Bernstein draw on is that the internal representations were latent before being attended to. An important distinction in memory research is between information represented in relatively static, durable patterns of neural wiring (e.g. in long-term memory) versus information represented in patterns of active, continuous neural firing (Goldman-Rakic, 1995; Myers et al., 2017; O’Reilly et al., 2000). A plausible interpretation of Amir & Bernstein’s model is that the internal content targeted by internal attention is lying latent in long-term memory, and internal attention activates it (and thereby brings it into working memory; Kiyonaga & Egner, 2013).

The activation interpretation is related to the generation interpretation. Generating novel representations (e.g. the thought “Bad things always happen to me”) likely involves activating latent long-term knowledge representations (e.g. semantic concept knowledge of the words “bad”, “things”, etc.) as an initial part of the generation process. But there are many examples in Amir & Bernstein’s paper that are more naturally described as activating existing latent representations rather than generating novel ones – for instance, automatically recalling negative autobiographical memories during rumination.

This interpretation – which we label the “activation” interpretation – is offered explicitly by Amir & Bernstein throughout the paper. For instance, they write that internal attention can operate over “stored” information (p. 6) and can involve “spreading activation” to “long-term memory representations” (p. 22). They also explicitly describe retrieval from long-term memory as an internal attentional process (e.g., “trying to recall the author and name of a paper to cite requires that internal attention be constrained to associated information in long-term memory, until the required information is retrieved”; p.35).

But the implications of this interpretation are insufficiently considered. For one, the overlap between memory retrieval and internal attention is more nuanced and controversial than Amir & Bernstein make it sound (Cabeza et al., 2008; De Brigard, 2012; Hutchinson et al., 2009). Canonically, attentional mechanisms select among active competing representations, not latent ones (Chun et al., 2011; Dehaene, 2014). For instance, when viewing a complex visual scene with many percepts competing for attention, unattended-to percepts are not being stored in latent, durable patterns of neural wiring in V1; they are being actively represented by patterns of neural firing (Dehaene et al., 2006). Indeed, for this reason, Dehaene’s influential model of how attention brings preconscious percepts into awareness explicitly restricts its scope to representations which are active, not latent (Dehaene, 2014). Of course, memory retrieval very well could have a “functional correspondence” with internal attention (Chun et al., 2011), and the two might share computational (Logan et al., 2021) and neural (Cabeza et al., 2008, 2011) mechanisms; but it is not clear whether they should be grouped under the same umbrella.

Moreover, even if the activation of latent representations from long-term memory is part of internal attention proper, this form of attention ought to be carefully distinguished from others. Lumping the various interpretations of internal attention together under a general definition (like “internal attention biases processing in favor of certain internally generated- or stored- mental representations”, p. 6) obscures important differences between cognitive processes that likely involve different computational mechanisms and neural substrates. Memory retrieval is unlikely to have the same dynamics as generating novel thoughts, and both likely differ from the process of selecting among already-active representations, which we consider next.

Finally, if the activation of latent representations from long-term memory is a core part of what Amir & Bernstein want to model, then they need to contrast their model, not just with the small number of alternate theories of internally-directed cognition (p. 32–33), but also with the myriad theories of memory retrieval. There is, for instance, a massive literature on the relationship between emotion and memory retrieval (Buchanan, 2007; Kensinger & Schacter, 2008; LeDoux, 1994). How does Amir & Bernstein’s model compare to those models? If they endorse the activation interpretation of their model, this is a question they need to answer.

The “selection” interpretation

The third possible interpretation of Amir & Bernstein’s model is subtly, but crucially, different from the activation interpretation. On this interpretation, there are multiple already-active internal representations competing for processing which internal attention selects among. For instance, there might be multiple streams of thought occurring simultaneously, multiple memories activated at once, or multiple active goals/task sets competing for behavioral control, and internal attention selects one to become the focus of working memory and conscious awareness. In the same way that, when viewing an external landscape, there are a multitude of blooming, buzzing perceptual representations waiting to be raised into awareness if attended to, so it is for the internal landscape.

This is the most natural interpretation of what Amir & Bernstein mean by attending to “internal events” (p. 5). Unlike in the generation or activation interpretations, where the attentional process is causing an internal event (e.g. the generation or activation of a thought, memory, mental image, etc.), on the selection interpretation the internal events are already occurring and attention is selecting which one(s) will make it into working memory. In other words, the internal events would be “preconscious” in Dehaene et al.’s sense of the term (Dehaene et al., 2006).

This interpretation has some appealing properties. The selection process in this case would be the closest internal analogue to external attention (Chun et al., 2011), and may therefore be the likeliest to share its computational mechanisms and neural substrates. Moreover, this picture of the internal landscape of the mind – with many ongoing thought streams hidden in darkness, waiting to be illuminated by internal attention – fits with foundational observations in clinical psychology. For instance, in his seminal book on cognitive-behavioral therapy (now considered the gold standard for evidence-based therapies; Hayes & Hofmann, 2017), Aaron Beck (1979) observed:

“Patients experienced specific types of thoughts of which they were only dimly aware and that they did not report... Unless they were directed to focus their attention on these thoughts, they were not likely to be very aware of them. Although these thoughts seemed to be on the periphery of the patients’ stream of consciousness, they appeared to play an important role in the psychic life of these patients... It seemed to me that I had tapped another level of consciousness in the recognition of automatic thoughts, perhaps analogous to the phenomenon described by Freud as “preconscious”...”

This picture of the mind is reflected in therapeutic practices like keeping thought journals, practicing attending to thoughts, and so on (Beck, 1991; Segal et al., 2018). The selection interpretation of internal attention, then, offers an appealing angle for what it means for internal attention to be helpful or adaptive in a clinical sense: People can use internal attention to bring into working memory the thought streams or mental processes that are actually underlying their present-moment behavior, rather than generating or activating ones which are more divorced from immediate reality (as in rumination).

Unfortunately, more work needs to be done to put this interpretation on solid ground. For instance, it is unclear how, mechanistically, this internal selection process would work. External attentional selection has been posited to operate in part by amplifying neural signals in perceptual brain regions (Dehaene et al., 2006). Would internal selection work in an analogous fashion? What is the proposed relationship between an “internal” selection mechanism that amplifies cognitive representations and an “external” selection mechanism that amplifies perceptual representations? Are they two mechanisms operating in parallel, one mechanism alternating between internal and external targets, or somewhere in between (Kiyonaga & Egner, 2013; Verschooren et al., 2021)? How is interference or competition between these two forms of selection handled (Chun et al., 2011; Kiyonaga & Egner, 2014)? These are the types of questions that need to be addressed to conceptually flesh out the selection interpretation of Amir & Bernstein’s model.

More worryingly, for the type of high-level cognition Amir & Bernstein focus on, there is surprisingly little evidence that there are active, preconscious thoughts (or other cognitive representations) that operate initially in the absence of consciousness, but are competing to be brought into awareness (Morris, 2021). In other words, the premise of the selection interpretation has rarely, if ever, been subject to experimental investigation. There are certainly many unconscious representations that appear to be activated and influence ongoing cognition and behavior, such as when people are rendering judgments, making decisions, interacting with social partners, and so on (Bargh & Morsella, 2008; Evans, 2008; Gigerenzer, 2007; Haidt, 2001; Hassin et al., 2004; Kahneman, 2011; Nisbett & Wilson, 1977). But it is far from clear that these activated, unconscious representations can be brought into awareness via internal attention. To the contrary, much research suggests that people are unable to accurately report the mental representations active in their minds and underlying their behavior, even when directed to attend to them – suggesting that people cannot actually attend to most of the “internal events” happening in their mind (Bargh & Morsella, 2008; Evans, 2008; Gigerenzer, 2007; Haidt, 2001; Hassin et al., 2004; Kahneman, 2011; Wilson, 2004); as Nisbett & Wilson (1977) put it, “There may be little or no direct introspective access to higher order cognitive processes.”

To illustrate what is needed to put the selection interpretation on solid empirical ground, it is helpful to sketch out an experiment that would support it. The experiment would utilize cognitive neuroscience methods (fMRI, EEG, MEG, fNIRS, etc). to demonstrate the existence of activated, preconscious, internal representations – representations that are unconscious (but still activated) when unattended to, and that become conscious when attended to. Consider a paradigm where participants are extensively trained to associate neutral cues with the act of imagining either faces or places, until seeing the cues automatically activates the corresponding mental images. Then, while neural activation data are being acquired, participants are shown a neutral cue under two conditions – one with low load on internal attention, and one with high load (e.g. mentally walking through all the rooms in their house) – and asked to report whether the corresponding mental image was activated during that trial. Finally, neural pattern classification approaches (e.g., multivariate pattern analysis; MVPA) could be used to decode the actual activation of the mental images during both conditions (Ragni et al., 2020; Reddy et al., 2010; Stokes et al., 2009). The pattern of results that would most strongly support the selection interpretation is as follows. People report that there was no mental image activated in the high load condition, suggesting that without attention the representation was unconscious. However, it is still possible to successfully decode the activation of the image from neural activity pattern, suggesting that the representation was active at an above threshold level in both conditions.² This pattern would provide rigorous causal evidence that internal attention can bring active, unconscious internal representations into awareness.

More broadly, this example illustrates the potential utility of cognitive neuroscience methods for testing (and distinguishing between) these three interpretations of the role of internal attention. The three interpretations differ in the nature of the internal representations prior to attentional selection – and hence, before they are available for explicit report. To characterize their nature, then, we need empirical methods which can identify and evaluate the presence of internal representations even when they are not accessible to conscious awareness. Cognitive neuroscience tools are likely to be important for such investigations.

Conclusion

To summarize: There are three different interpretations of the role of internal attention in Amir & Bernstein’s model, depending on the nature of the targets of that attention. Sometimes, it seems like Amir & Bernstein are trying to model the dynamics of a process that generates new internal content; other times, they appear to be trying to model a process that activates existing latent representations from long-term memory; and other times, they appear to be modeling a process that selects among already-activated representations. Each of these interpretations has problems. The first is likely not a unified attentional process; the second needs to engage more with the vast literature on memory retrieval; and the third is lacking in empirical evidence. More work needs to be done to distinguish between these interpretations both conceptually and empirically.

We applaud Amir & Bernstein for the ambition and scope of their model, which, by casting a wide light, helps illuminate these different possible interpretations of internal attention. Moreover, the ambiguity between these interpretations is not unique to Amir & Bernstein’s model; much work on internal attention, and internally-directed cognition more broadly, has not consistently distinguished between these possibilities. Rather than undermining Amir & Bernstein’s approach, these differing interpretations highlight how little is still known about internal attention, and help to map out the kind of work that needs to be done to put models of internal attention on solid ground. We believe this area is ripe for future investigation, particularly using the tools of modern cognitive neuroscience.