The Binary Structure of Event Files Generalizes to Abstract Features: A Nonhierarchical Explanation of Task Set Boundaries for the Congruency Sequence Effect

Abstract

Current views posit that forming and retrieving memories of ongoing events influences action control. However, the organizational structure of these memories, or event files, remains unclear. The hierarchical coding view posits a hierarchical structure, wherein task sets occupy a high level of the hierarchy. Here, the contents of an event file can be retrieved only if the task set repeats. In contrast, the binary coding view posits a nonhierarchical structure, which consists of a collection of independent, binary bindings between different feature pairs. In this view, repeating an abstract feature from a previous event (e.g., the previous trial’s S-R mapping) triggers the retrieval of the associated feature from the same binding (e.g., the previous trial’s congruency) even if the task set changes. To distinguish between these views, we investigated the nature of task set boundaries for the congruency sequence effect (CSE), an index of adaptive control that reflects event file formation and retrieval. Specifically, we investigated whether or not a CSE appears when the task set changes but the previous trial’s S-R mapping repeats. Three experiments involving a cross-modal prime-probe task yielded a CSE under these conditions and ruled out alternative explanations. These findings show that the typical binary structure of event files generalizes from concrete features (e.g., colors and locations) to abstract features (e.g., S-R mappings and task sets). Therefore, contrary to the hierarchical coding view, they provide a nonhierarchical explanation of task set boundaries for the CSE.

Engaging in purposeful behavior requires adapting to an ever-changing environment. Considered one of the three main cognitive-core functions (Diamond, 2013), such cognitive flexibility requires humans to combine internal information (e.g., task goals) with external contextual information to achieve a goal (e.g., Miller & Cohen, 2001). For instance, consider two pedestrians who are attempting to cross a busy intersection. If the walk sign does not function correctly, the pedestrians must adapt by shifting their attention from the typical source of information (i.e., the walk sign) to a new source (i.e., the traffic light) to decide when it is safe to cross the street. That is, to navigate the intersection safely, the pedestrians must combine their internal goal of crossing the street with external contextual information in a flexible manner.

To investigate how adaptive control processes operate in the laboratory, researchers measure performance (e.g., reaction time [RT] and/or error rate) in distractor-interference tasks (Eriksen & Eriksen, 1974; Eriksen & Schultz, 1979; Simon & Rudell, 1967; Stroop, 1935). For instance, consider the prime-probe task, wherein study participants ignore an initial prime (or distractor) and respond to a subsequent probe (or target) in every trial. In general, participants respond more slowly and less accurately when the prime signals a different (incongruent) response than the probe as compared with the same (congruent) response. However, this congruency effect—an index of overall distractibility—is smaller after incongruent trials than after congruent trials (Gratton et al., 1992). This congruency sequence effect, or CSE, may reflect adaptive control processes that minimize distraction from irrelevant stimuli (Botvinick et al., 2001; Gratton et al., 1992). In line with this view, a CSE appears in four-alternative-forced-choice (4-AFC) tasks even without feature integration (i.e., stimulus and response repetition) and contingency learning (i.e., stimulus frequency) confounds that can independently produce this sequential trial effect (Jiménez & Méndez, 2014; Kim & Cho, 2014; Schmidt & Weissman, 2014; Weissman et al., 2014).

The Attentional Shift and Response Modulation Accounts

There are two primary accounts of the CSE in confound-minimized 4-AFC tasks. The attentional shift account posits that control processes shift attention toward the target and away from the distractor to a greater extent after incongruent trials than after congruent trials (Botvinick, 2007; Botvinick et al., 2001; Dreisbach & Fischer, 2012; Gratton et al., 1992). The response modulation account posits that control processes modulate (i.e., inhibit and/or enhance) the response signaled by the distractor differently after incongruent trials than after congruent trials (Logan, 1985; Logan & Zbrodoff, 1979; Ridderinkhof, 2002; Stürmer et al., 2002; Weissman, Colter, Grant, & Bissett, 2017). Specifically, control processes (a) inhibit the response cued by the distractor (or activate the opposite response) after incongruent trials and/or (b) enhance the response cued by the distractor after congruent trials.

Three prior findings from the prime-probe task favor the response modulation account over the attentional shift account (Weissman et al., 2015). First, the CSE is larger when the prime appears before (vs. with) the probe. Because it takes time to modulate response activation (Ridderinkhof, 2002), this finding suggests that control processes underlying the CSE need time to modulate (e.g., inhibit) the response cued by the prime before participants respond to the probe. Second, when a long (i.e., 1,000 ms) interstimulus-interval (ISI) separates the prime’s offset from the probe’s onset, which eliminates the overall congruency effect, the CSE is associated with a positive congruency effect after congruent trials and a negative congruency effect after incongruent trials (i.e., faster response times in incongruent relative to congruent trials). The negative congruency effect after incongruent trials is inconsistent with the attentional shift account. Even completely ignoring the prime after incongruent trials could eliminate the congruency effect but not reverse it. In contrast, this finding supports the response modulation account. For example, if participants inhibit the response cued by the prime after incongruent trials, then response times to the subsequent probe should be longer in congruent trials, wherein the probe requires the same (inhibited) response, than in incongruent trials, wherein the probe requires a different (uninhibited) response, leading to a negative congruency effect. Third, the size of the CSE in the prime-probe task does not vary with whether there is a large overall congruency effect or no overall congruency effect. This finding weighs against variants of the attentional shift account wherein heightened response conflict in incongruent (vs congruent) trials—as indexed by the size of the congruency effect (Yeung et al., 2011)—triggers control processes underlying the CSE to shift attention toward the target (e.g., Botvinick et al., 2001). Consistent with this assertion, the size of the CSE is typically independent of the size of the congruency effect (Weissman et al., 2014, 2015).

The Episodic Retrieval View of the CSE

More broadly, the findings above are consistent with an emerging episodic retrieval view of the CSE (Dignath et al., 2019; Egner, 2014; Hazeltine et al., 2011; Schumacher & Hazeltine, 2016; Spapé & Hommel, 2008). Here, participants form an episodic memory of the previous trial that stores both (a) concrete features (e.g., stimuli and responses) and (b) abstract features (e.g., task sets, S-R mappings, control settings, and trial congruency). Repeating a previous-trial feature (e.g., the task set) in the current trial triggers the retrieval of a memory of the previous trial, which specifies its congruency. This biases control processes to treat the current trial as a congruency repetition. For instance, after an incongruent (vs congruent) trial in the prime-probe task, control processes may become biased to inhibit (vs enhance) the response associated with the prime. As we described earlier, this could lead to a smaller congruency effect after incongruent relative to congruent trials (i.e., a CSE).

Recent data from a modified prime-probe task support the episodic retrieval view (Grant & Weissman, 2019). As in the version of the standard prime-probe task that engenders a negative congruency effect after incongruent trials, a long (i.e., 1,000 ms) ISI separates the prime and probe. Unlike in the standard prime-probe task, however, participants respond to both the prime and the probe using the same S-R mapping (i.e., “identify the stimulus by making one of four possible keypresses”). The CSE in mean probe response time is much larger in the modified prime-probe task than in the standard prime-probe task, wherein participants do not respond to the prime. Further, this increase in CSE magnitude does not occur when participants respond to the prime in only the previous or current trial. These findings support the episodic retrieval view. Indeed, only in the modified prime-probe task does responding to the prime in the current trial constitute a repetition of the previous trial’s overall S-R mapping (i.e., a repetition of both the previous prime’s S-R mapping and the previous probe’s S-R mapping). The repetition of this abstract feature, which is not present in the standard prime-probe task, should serve as an additional retrieval cue for the memory of the previous trial, leading to an especially large CSE.

The Hierarchical and Binary Coding Views of the CSE

Although there is a great deal of support for the episodic retrieval account of the CSE, competing views differ with regard to the exact conditions under which repeating an abstract previous-trial feature engenders a CSE in confound-minimized tasks. The hierarchical coding view posits that information about the previous trial’s congruency, which gives rise to the CSE, is specific to the abstract task set that the participant used in the previous trial. Thus, the previous trial’s congruency is only retrieved when the task set repeats. This view is based on neurophysiological findings and computational models of prefrontal cortex (e.g., Badre, 2008; Miller & Cohen, 2001), which suggest that concrete and abstract features are encoded in memory within a hierarchical framework known as a task file (Schumacher & Hazeltine, 2016). Lower hierarchical levels of a task file consist of concrete features such as stimuli and responses. Higher hierarchical levels consist of abstract features such as task sets, stimulus-response (S-R) mappings, motivational factors, and task goals (Cookson et al., 2016, 2020; Hazeltine et al., 2011; Schumacher & Hazeltine, 2016).

An example is useful for illustrating how the hierarchical coding view explains task set boundaries for the CSE. Consider a cross-modal, standard prime-probe task wherein the prime and probe words “Left,” “Right,” “Up,” and “Down” appear in the visual modality in 50% of the trials and in the auditory modality in the other 50%. A hierarchical representation of this task would specify high-level visual and auditory task sets that are associated with low-level stimuli in the visual and auditory modalities, respectively (Schumacher & Hazeltine, 2016). Selecting the visual or auditory task set following the onset of a prime in the visual or auditory modality would activate the subset of the S-R mapping that relates (a) each of the four visual probe words to a specific response or (b) each of the four auditory probe words to a specific response. Critically, information about each trial’s congruency (i.e., whether the prime and probe activate the same or different responses) would be stored within the relevant subset of the S-R mapping, which, in turn, would be nested within the modality-specific task set (Hazeltine et al., 2011). Thus, the CSE would vanish if the task set changes in the next trial, because this would prevent the retrieval of the previous trial’s congruency. Consistent with this view, switching between modality-specific task sets typically eliminates the CSE (Grant et al., 2020; Hazeltine et al., 2011; Kreutzfeldt et al., 2016; Yang et al., 2017).

In contrast, the binary coding view posits that repeating the previous trial’s task set is just one of many ways to cue the retrieval of the previous trial’s congruency. According to this view, participants form an episodic memory of each trial known as an “event file” (Hommel, 1998). Unlike a task file, an event file consists of a nonhierarchical organization of independent, binary bindings between concrete and/or abstract features. Thus, the previous trial’s congruency—or associated control parameters—can be stored in multiple, independent binary bindings (e.g., a binding with the previous trial’s abstract task set, a binding with the previous trial’s abstract S-R mapping, etc.; Dignath et al., 2019). Consistent with the binary coding view, Dignath and colleagues (2019) observed a larger CSE in a prime-probe task when a salient contextual feature—the stimulus format (color word or color patch)—repeated (vs switched) across consecutive trials in the absence of stimulus and response repetitions. Given that repetitions of stimuli, responses, and other contextual features also increase the CSE (Hommel et al., 2004; Mayr et al., 2003; Spapé & Hommel, 2008), Dignath et al.’s (2019) finding suggests that the previous trial’s congruency can be stored in multiple independent bindings. Thus, the binary coding view predicts that repeating an abstract feature from the previous trial, such as the S-R mapping, should independently cue the retrieval of the previous trial’s congruency even when another abstract feature, such as the task set, changes.¹

In short, unlike the hierarchical coding view, the binary coding view posits that a CSE may appear when the task set changes. A CSE may appear, for example, when the modality-specific task set changes in a cross-modal version of the modified prime-probe task described earlier. The reason is that participants can use the same, nonhierarchically organized S-R mapping (i.e., identify the stimulus by making one of four possible keypresses) in each trial, regardless of whether the stimuli appear in the visual modality or the auditory modality. Here, responding to the current-trial prime is a repetition of the previous trial’s S-R mapping, which should independently trigger the retrieval of the previous trial’s congruency and engender a CSE.

The Present Study

We sought to distinguish between the hierarchical and binary coding views of task set boundaries for the CSE. In Experiment 1, we used a cross-modal version of the modified prime-probe task, wherein participants respond to both the prime and the probe. Consistent with the binary coding view, but not with the hierarchical coding view, we observed a significant CSE when the modality-specific task set changed. In Experiments 2 and 3, we further explored the binary coding view and ruled out two alternative explanations for our findings in Experiment 1.

Experiment 1

In Experiment 1, we sought to distinguish between the hierarchical and binary coding views by using a cross-modal version of the modified prime-probe task. Here, the prime and probe appear in the visual modality in half the trials and in the auditory modality in the other half. This allows participants to categorize each trial as “visual” or “auditory,” enabling them to form modality-specific task sets (Hazeltine et al., 2011). Participants respond to (a) the prime during the 1,150-ms ISI that separates prime offset from probe onset and (b) the probe after it appears (during a second ISI). As we described earlier, the hierarchical coding view predicts that changing the modality-specific task set across consecutive trials will eliminate the CSE just as it does in the cross-modal version of the standard prime-probe task (Grant et al., 2020; Hazeltine et al., 2011). In contrast, the binary coding view predicts that a significant CSE may appear even when the modality-specific task set changes.

Method

Participants

We preregistered our hypotheses, methods, and analyses on the Open Science Framework (OSF; https://osf.io/x75nq). As discussed in the preregistration, we conducted a power analysis (via G*Power 3.1.9.2; Faul et al., 2007) to determine the required sample size for observing an interaction among modality transition, previous trial congruency (i.e., trial N − 1 congruency), and current trial congruency (i.e., trial N congruency). To this end, we used the effect size for this interaction (${\eta }_p^2=.37$) from Experiment 1 of Hazeltine et al. (2011) and an alpha (α) of .05. The results indicated that a sample size of 32 participants would allow us to observe this interaction with greater than 99% power.

We collected data from 39 students at the University of Michigan, all of whom participated through the Intro Psych Subject Pool for course credit. We excluded all participants who self-reported a visual and/or auditory impairment (one participant) or completed the task with less than 75% overall accuracy (6 participants). We conducted statistical analyses on the data from the remaining 32 participants (seven male, 25 female; 31 right-handed, one left-handed; 18−20 years old, M = 18.66, SD = .70). No participants reported neurological (e.g., seizures, ADHD) or perceptual (e.g., uncorrected vision) impairments. The University of Michigan’s Behavioral Sciences Internal Review Board approved all experimental procedures. The raw data are available via the OSF (https://osf.io/a85bf/?view_only=c5bdeaf264e94370be943e111fb83960).

Task Stimuli and Experimental Design

The primes and probes were four direction words (i.e., left, right, up, and down) that could appear in the visual or auditory modality. The visual angles (visual words) and decibels (auditory words) were the same as those in one of our prior studies (Grant et al., 2020). To present the task stimuli and record participants’ responses (via a standard QWERTY keyboard), we used the Psychophysics Toolbox in MATLAB (Brainard, 1997). Participants viewed all visual stimuli from a distance of approximately 55 cm.

A 2-s fixation cross appeared at the beginning and end of each block. Each trial consisted of four events: (a) a prime; (b) an interstimulus-interval (ISI); (c) a probe; and (d) an intertrial-interval (ITI). The duration of each event is shown in Figure 1. We used a longer ISI between the prime and the probe than in prior studies (e.g., Hazeltine et al., 2011) to provide participants with enough time to respond to the prime before the probe appeared.

Figure 1.

The Prime-Probe Task Used in Experiment 1

Note. In each trial, a prime word preceded a probe word (the figure illustrates one trial). Trials contained either visual stimuli (shown above) or auditory stimuli. Visual stimuli appeared in white on a black background. Auditory stimuli were delivered via headphones. The number beneath each box indicates the length of the corresponding trial component in milliseconds (ms). The hands indicate the stimulus-response mapping.

There were sixteen prime-probe pairs (eight visual, eight auditory). In each modality (e.g., visual), half of the prime-probe pairs were congruent (e.g., UP-UP) and half were incongruent (e.g., UP-DOWN). Thus, in every block, there were four main trial types: congruent trials in the visual modality, congruent trials in the auditory modality, incongruent trials in the visual modality, and incongruent trials in the auditory modality. Each of these four trial types preceded and followed every trial type roughly equally often, separately for odd and even trials. Specifically, one sequential trial type appeared one less time than the others in each block, because there was no “previous trial type” for the first trial. However, because the trial sequence differed for each block, the underrepresented trial type varied randomly across blocks.

We avoided two prevalent confounds. To avoid feature integration confounds (Hommel et al., 2004; Mayr et al., 2003), we presented prime-probe pairs involving the words left and/or right in odd trials and the words up and/or down in even trials.² To avoid contingency learning confounds (Schmidt & De Houwer, 2011), we paired each prime (e.g., UP) equally often with the congruent (e.g., UP) and incongruent (e.g., DOWN) probe with which it could appear.

Procedure

We instructed participants to identify each word in a given trial by pressing the F key (to indicate left), the G key (to indicate right), the J key (to indicate up), or the N key (to indicate down; see Figure 1). If a participant (a) did not respond within .900 seconds of prime or probe onset or (b) pressed the wrong key (error trials), an error message appeared on the screen for 200 ms. Participants completed a practice block (64 trials) and twelve subsequent test blocks (64 trials each). Afterward, a research assistant explained the purpose of the study.

Data Analysis

Before analyzing the data, we excluded several trial types. Prior to analyzing mean probe response time (RT), we excluded practice trials, the first trial of each block, outliers (i.e., trials with RTs > 3 SD from the conditional mean), omitted responses, incorrect responses (i.e., errors), and trials immediately following omitted responses and errors. Prior to analyzing mean probe error rate (ER), we excluded the same trial types with the exception of errors, which were the dependent measure. In total, 13.2% of the trials were errors and .6% were outliers.

Following the exclusions above, we conducted separate, repeated-measures ANOVAs on mean probe RT and mean probe ER. There were four within-participants factors in each ANOVA: current trial modality (visual, auditory), modality transition (repeat, switch), trial N − 1 congruency (congruent, incongruent), and trial N congruency (congruent, incongruent). We included current trial modality to account for the variance this factor produced, but this factor was not of primary interest. Moreover, this factor did not influence the critical interaction among modality transition, trial N − 1 congruency, and trial N congruency in any of the present experiments. For this reason, we do not report our findings related to this factor below (but see Appendix A and B for supplemental ANOVAs of the data from Experiment 1 that include this factor).

Results

Mean Probe RT

There were significant main effects of (a) modality transition³, F(1, 31) = 28.03, p < .001, ${\eta }_p^2=.48$, (b) trial N − 1 congruency, F(1, 31) = 22.91, p < .001, ${\eta }_p^2=.43$, and (c) trial N congruency, F(1, 31) = 43.39, p < .001, ${\eta }_p^2=.58$. These main effects occurred because mean probe RT was slower in (a) in modality-repeat trials (480 ms) relative to modality-switch trials (471 ms), (b) after congruent trials (480 ms) than after incongruent trials (470 ms), and (c) in current incongruent trials (490 ms) relative to current congruent trials (460 ms).

There were also two significant two-way interactions. Specifically, there were interactions between modality transition and trial N congruency, F(1, 31) = 21.70, p < .001, ${\eta }_p^2=.41$, and between trial N − 1 congruency and trial N congruency, F(1, 31) = 95.31, p < .001, ${\eta }_p^2=.76$. These interactions occurred because the congruency effect was (a) larger in modality-switch trials (38 ms) than in modality-repeat trials (23 ms) and (b) smaller after incongruent trials (13 ms) than after congruent trials (49 ms; i.e., because there was a CSE).

Finally, there was a significant three-way interaction among modality transition, trial N − 1 congruency, and trial N congruency, F(1, 31) = 20.58, p < .001, ${\eta }_p^2=.40$ (Figure 2). Consistent with both the binary and hierarchical coding views, the CSE was larger in modality-repeat trials (48 ms; F(1, 31) = 85.88, p < .001, ${\eta }_p^2=.74$) than in modality-switch trials (24 ms; F[1, 31] = 39.71, p < .001, ${\eta }_p^2=.56$; Figure 2). Inconsistent with the hierarchical coding view, however, the CSE remained significant in modality-switch trials.

Figure 2.

The Congruency Sequence Effect (CSE) in Each of the Two Main Trial Types of Experiment 1: Modality-Repeat Trials and Modality-Switch Trials

Note. Previous trial congruency is indicated on the x axis (Prev Cong: previous congruent trial; Prev Incong: previous incongruent trial). Current trial congruency is indicated by line type (Incong: dashed line; Cong: black line). Reaction time (in ms) is indicated on the y axis. Error bars indicate ±1 SEM.

Mean Probe ER

There were three significant main effects of modality transition, F(1, 31) = 12.15, p = .001, ${\eta }_p^2=.28$, trial N − 1 congruency, F(1, 31) = 17.89, p < .001, ${\eta }_p^2=.37$, and trial N congruency, F(1, 31) = 18.07, p < .001, ${\eta }_p^2=.37$. These main effects occurred because mean ER was higher (a) in modality-repeat trials (11.8%) than in modality-switch trials (10.2%), (b) after congruent trials (12.1%) than after incongruent trials (9.9%), and (c) in current incongruent trials (12.2%) than in current congruent trials (9.8%).

Finally, there was a significant two-way interaction between trial N − 1 congruency and trial N congruency, F(1, 31) = 33.48, p < .001, ${\eta }_p^2=.52$. Indicating the presence of a CSE, the congruency effect was smaller after incongruent relative to congruent trials (.1% vs 4.6%).

Discussion

We observed a CSE in the modified version of the cross-modal prime-probe task even when the modality-specific task set changed. This result suggests that responding to the prime in the current trial, which constitutes a repetition of the previous trial’s S-R mapping, independently triggers the retrieval of the previous trial’s congruency even when participants switch to a different modality-specific task set. This outcome supports the binary coding view.

Mirroring prior findings from the standard version of the cross-modal prime-probe task (e.g., Grant et al., 2020; Hazeltine et al., 2011; Kreutzfeldt et al., 2016; Yang et al., 2017), we also observed a smaller CSE in modality-switch trials than in modality-repeat trials. This result suggests that repeating the previous trial’s S-R mapping by responding to the prime is not the sole determinant of CSE magnitude. Rather, it appears that repetitions of the previous trial’s (a) modality-specific task set and (b) S-R mapping serve as distinct retrieval cues for a memory of the previous trial, such that removing one of these cues makes episodic retrieval less efficient and reduces the CSE. This interpretation is also consistent with the binary coding view.

However, two alternatives to the binary coding view may account for the significant CSE that we observed when the task set changed. First, the use of a relatively long (i.e., 1,150 ms) prime-probe interval may provide participants with enough time to switch attention to the current-trial task set before the probe appears (“Orient” in Figure 3). In particular, it may allow participants to orient attention to the modality of the prime (e.g., visual), which always predicts the modality of the upcoming probe (e.g., visual), before the probe appears and thereby minimize task-switch costs that reduce the CSE (Egner, 2008; Kiesel et al., 2006). Consistent with this possibility, participants orient attention to the modality in which the prime appears when they know the probe is highly likely to appear in the same modality (Grant et al., 2020). Second, the use of a relatively long prime-probe interval may provide participants with enough time to subvocalize the prime word before the probe word appears (“Subvocalize” in Figure 3). This may allow participants to encode the stimuli verbally in both visual and auditory trials, leading to a reduced modality-specific boundary for the CSE. Consistent with this possibility, 1,150 ms is enough time to subvocalize one-syllable words (Landauer, 1962; Standing & Curtis, 1989).

Figure 3.

Two Alternative Explanations for Our Findings From Experiment 1, as Illustrated in a Single Trial

Note. Each horizontal panel represents a given explanation. The number beneath each vertical panel indicates the length of the corresponding trial component in milliseconds (ms).

Experiment 2

In Experiment 2, we sought to distinguish between the binary coding view and the two alternative explanations of our findings from Experiment 1 described earlier. To this end, we used a cross-modal version of the standard prime-probe task. This task was identical to the task in Experiment 1 with the sole exception that participants responded only to the probe. We reasoned that if, during the 1,150-ms ISI, participants (a) orient attention to the modality of the prime (“Orient” in Figure 3) and/or (b) subvocalize the prime word (“Subvocalize” in Figure 3), then we should observe a significant CSE in modality-switch trials just as in Experiment 1.

In contrast, the binary coding view predicts that changing the modality-specific task set will eliminate the CSE in a standard prime-probe task for two reasons. First, because the current trial’s task set (e.g., visual) does not match the previous trial’s task set (e. g., auditory), it does not cue the retrieval of the previous trial’s congruency (e.g., incongruent). Second, withholding a response to the prime in the current trial does not constitute a repetition of the previous trial’s overall S-R mapping (i.e., a repetition of both the previous prime’s S-R mapping and the previous probe’s S-R mapping) that can independently trigger the retrieval of the previous trial’s congruency. Indeed, neither the previous trial nor the current trial has an overall S-R mapping because participants use a different S-R mapping for the prime (i.e., do not respond to the stimulus) than for the probe (i.e., identify the stimulus by making one of four possible keypresses). Thus, when the modality-specific task set switches, neither the current task set nor the current S-R mapping can trigger the retrieval of the previous trial’s congruency. For this reason, changing the modality-specific task set should eliminate the CSE. Consistent with this prediction, the CSE is smaller in the standard version of the unimodal (i.e., visual-modality) prime-probe task, wherein the overall S-R mapping does not repeat, than in the modified version of the same task, wherein the overall S-R mapping does repeat (Grant & Weissman, 2019).

Method

Participants

We preregistered our hypotheses, methods, and analyses on the OSF (https://osf.io/x9n36). To determine the appropriate sample size, we conducted two power analyses (G*Power 3.1.9.2; Faul et al., 2007). First, we estimated the sample size needed to observe the critical three-way interaction from Experiment 1. Second, we estimated the sample size needed to observe a CSE in modality-switch trials. To conduct these power analyses, we used the effect sizes for the corresponding three-way (${\eta }_p^2=.40$) and two-way (${\eta }_p^2=.56$) interactions in Experiment 1 and an alpha (α) of .05. We found that a sample size of 32 participants would allow us to observe each of these interactions with greater than 99% power.

Thirty-eight students from the University of Michigan participated through the Intro Psych study Pool for course credit. We excluded participants who did not complete the task (one), did not follow the task instructions (one), experienced technical (i.e., computer-related) difficulties (one), and/or performed the task with less than 75% accuracy (three). We conducted our analyses on data from the remaining 32 participants (16 male, 16 female; 31 right-handed, one left-handed; 17−26 years old, M = 18.66, SD = 1.49). No participants reported impairments of a neurological (e. g., seizures, ADHD) and/or perceptual (e.g., uncorrected vision) nature. The University of Michigan’s Behavioral Sciences Internal Review Board approved all procedures. The raw data are available via the OSF (https://osf.io/gy7rj/?view_only=70b65aa28dfb4d2a9454d685a959e6fd).

Task Stimuli and Experimental Design

The stimuli, apparatus, and experimental design were identical to those in Experiment 1.

Procedure

The procedure was the same as that in Experiment 1 with one exception: rather than instructing participants to respond to both the prime and the probe, we instructed them to ignore the initial prime and respond only to the subsequent probe.

Data Analysis

The data analysis was identical to that in Experiment 1 (see Appendix C for supplemental ANOVAs that include this factor). On average, 5.9% of the trials were errors and .8% were outliers.

Results

Mean Probe RT

There was a significant main effect of modality transition, F(1, 31) = 57.76, p < .001, ${\eta }_p^2=.65$ and a significant two-way interaction between trial N − 1 congruency and trial N congruency, F(1, 31) = 31.20, p < .001, ${\eta }_p^2=.50$. These findings reflect (a) longer RT in modality-switch trials than in modality-repeat trials (548 ms vs 522 ms) and (b) a CSE: a smaller congruency effect after incongruent relative to congruent trials (−4 ms vs 15 ms).

Finally, there was a significant three-way interaction among modality transition, trial N − 1 congruency, and trial N congruency, F(1, 31) = 15.63, p < .001, ${\eta }_p^2=.34$. The CSE was larger in modality-repeat trials (33 ms; F[1, 31] = 42.48, p < .001, ${\eta }_p^2=.58$) than in modality-switch trials (6 ms; F[1, 31] = 1.69, p = .20, ${\eta }_p^2=.05$; Figure 4). Critically, the CSE was not significant in modality-switch trials. Thus, unlike in Experiment 1, a change in modality across consecutive trials eliminated the CSE.

Figure 4.

The Congruency Sequence Effect (CSE) in Each of the Two Main Trial Types of Experiment 2: Modality-Repeat Trials and Modality-Switch Trials

Note. Previous trial congruency is indicated on the x axis (Prev Cong: previous congruent trial; Prev Incong: previous incongruent trial). Current trial congruency is indicated by line type (Incong: dashed line; Cong: black line). Reaction time (in ms) is indicated on the y axis. Error bars indicate ±1 SEM.

Mean Probe ER

There were no significant effects (all p values > .25).

Exploratory Analysis

Our findings thus far suggest that task sets and S-R mappings both serve as boundaries for the CSE. However, two issues remain unclear. First, it remains unclear whether switching between S-R mappings reduces CSE magnitude in the cross-modal prime-probe task as it does in the visual prime-probe task (e.g., Grant & Weissman, 2019). Second, it remains unclear whether task sets and S-R mappings serve as independent or overlapping boundaries for the CSE.

To begin to address these issues, we conducted an exploratory mixed ANOVA wherein experiment (1, 2) served as the between-participants factor, current trial modality (visual, auditory) and modality transition (repeat, switch) served as the within-participants factors, and the CSE in mean probe RT served as the dependent measure. We made two predictions. First, if S-R mappings serve as boundaries for the CSE, then overall CSE magnitude should be lower in Experiment 2 than in Experiment 1. Second, if task sets and S-R mappings serve as independent (overlapping) boundaries for the CSE, then the influences of experiment and modality transition on CSE magnitude should sum additively (interact).

There were two significant main effects (Figure 5). First, as in the overall analyses, there was a main effect of modality transition, F(1, 62) = 34.87, p < .001, ${\eta }_p^2=.31$, because the CSE was smaller in modality-switch trials than in modality-repeat trials. Second, there was a main effect of experiment, F(1, 62) = 10.25, p = .002, ${\eta }_p^2=.14$, because the CSE was smaller in Experiment 2 than in Experiment 1. This result conceptually replicates our prior finding that making the prime task-irrelevant (vs task-relevant) reduces the CSE (Grant & Weissman, 2019).

Figure 5.

The Congruency Sequence Effect (CSE) Magnitude Across Experiments 1 and 2

Note. The dashed line indicates the CSE in Experiment 1. The solid line indicates the CSE in Experiment 2. Modality condition (repeat, switch) is indicated on the x axis. CSE magnitude (in ms) is indicated on the y axis.

Finally, experiment and modality transition did not interact, F < 1. Consistent with the binary coding view, this outcome suggests that task sets and S-R mappings serve as independent boundaries for the CSE. Because this finding comes only from an exploratory analysis, however, future studies will be needed to test this hypothesis in an a priori fashion.

Discussion

In line with the binary coding view, we found that switching between task sets eliminates the CSE in a cross-modal version of the standard prime-probe task. The exploratory results further suggest that task sets and S-R mappings serve as independent boundaries for the CSE. Because the standard prime-probe task in Experiment 2 matched the modified prime-probe task in Experiment 1 with the sole exception that participants did not respond to the prime, our findings suggest that the use of a long (i.e., 1,150 ms) prime-probe interval in Experiment 1 cannot explain the CSE that we observed in modality-switch trials. More broadly, our findings in Experiments 1 and 2 suggest that the absence of a CSE in modality-switch trials of the standard prime-probe task (Grant et al., 2020; Hazeltine et al., 2011; Kreutzfeldt et al., 2016; Yang et al., 2017) indexes the cumulative effects of switching between (a) task sets and (b) S-R mappings.

However, it is important to consider another possible interpretation of the significant CSE that we observed in the modality-switch trials of Experiment 1.⁴ To some extent, participants may have coded the visual and auditory trials as belonging to the same semantic category. Thus, in at least some of the trials, participants may have formed a single semantic task set (i.e., direction words; Left-Right-Up-Down) rather than a pair of distinct, modality-specific task sets (i.e., an auditory task set and a visual task set; Eder & Rothermund, 2008; Gast & Rothermund, 2010; Rothermund et al., 2009). This variant of the hierarchical coding view posits that repeating the previous trial’s semantic task set (direction words) triggers the retrieval of the previous trial’s congruency even when the sensory modality changes, thereby leading to a significant CSE in modality-switch trials. If this hypothesis is correct, our findings from Experiment 1 do not rule out the hierarchical coding view, because information about the previous trial’s congruency could be stored within the previous trial’s semantic task set.

This variant of the hierarchical coding view also appears to predict a CSE when the sensory modality changes in the standard cross-modal prime-probe task. Although we did not observe such a CSE in Experiment 2, it is possible that participants are more likely to group visual and auditory trials into a single, semantic task set when they respond to both the prime and the probe as they did in Experiment 1. Specifically, using the same overall S-R mapping in every trial may bias participants to group all of the trials into a single task set even if some of the trials are visual while others are auditory. To test this possibility, we conducted a third experiment.

Experiment 3

In Experiment 3, we sought to distinguish between the binary coding view and the semantic task set variant of the hierarchical coding view. To this end, we used a modified version of the cross-modal prime-probe task wherein the visual and auditory stimuli come from different semantic categories (i.e., letters vs direction words), rather than from the same semantic category (i.e., direction words) as in Experiments 1 and 2. Critically, in this task, participants cannot group the visual and auditory stimuli into the same semantic category.

Under these conditions, the binary coding view and the semantic task set variant of the hierarchical coding view make different predictions. As in Experiment 1, the binary coding view predicts that changing the sensory modality will reduce, but not eliminate, the CSE. Here, responding to both the prime and the probe in each trial leads to a repetition of the previous trial’s overall S-R mapping (i.e., identify the stimulus by making one of four possible keypresses), which serves as a retrieval cue for the previous trial’s congruency even when the sensory modality changes. In contrast, the semantic task set variant of the hierarchical coding view predicts that changing the sensory modality will eliminate the CSE, because there is no longer a repetition of the semantic task set when the modality changes in modality-switch trials. The absence of this repetition should prevent the retrieval of the previous trial’s congruency.

Method

Participants

We preregistered our hypotheses, methods, and analyses on the OSF (https://osf.io/3cq6r/?view_only=873cf95d9d8d497fbc6559852e1761ba). To determine the appropriate sample size, we conducted two power analyses (G*Power 3.1.9.2; Faul et al., 2007). First, we estimated the sample size needed to observe the critical three-way interaction from our prior experiment. Second, we estimated the sample size needed to observe a CSE in modality-switch trials. To conduct these power analyses, we used the effect sizes for the corresponding three-way (${\eta }_p^2=.40$) and two-way (${\eta }_p^2=.56$) interactions in Experiment 1 and an alpha (α) of .05. The results indicated that a sample size of 31 participants would allow us to observe each of these interactions with sufficient power (i.e., greater than 99% power). To be consistent with Experiment 1, however, we collected usable data from 32 participants.

Forty-eight students from the University of Michigan participated through the Intro Psych study Pool for course credit. We excluded five participants who did not follow the instructions, one participant who experienced technical (i.e., computer-related) difficulties, one participant with uncorrected hearing and/or vision, and nine participants who performed the task with less than 75% accuracy. None of the remaining 32 participants⁵ (12 male, 20 female; 18−28 years old, M = 18.84, SD = 1.78) reported any neurological impairments (e.g., seizures, ADHD). The University of Michigan’s Behavioral Sciences Internal Review Board determined that this experiment was exempt from oversight. The raw data are available via the OSF (https://osf.io/3cq6r/?view_only=873cf95d9d8d497fbc6559852e1761ba).

Task Stimuli and Experimental Design

The primes and probes were four direction words and four letters that could appear in the visual or auditory modality. In the visual modality, the primes and probes were “Left” (2.2° × 1.0°), “Right” (3.1° × 1.0°), “Up” (1.6° × 1.0°), “Down” (3.4° × 1.0°), “A” (.83° × 1.0°), “B” (.83° × 1.0°), “C” (.83° × 1.0°), and “D” (.83° × 1.0°). In the auditory modality, the primes and probes were “Left” (61.6 dB), “Right” (54.4 dB), “Up” (63.2 dB), “Down” (61.7 dB), “A” (61.1 dB), “B” (55.8 dB), “C” (60.4 dB), and “D” (62.0 dB). To present the task stimuli and record participants’ responses (via a standard QWERTY keyboard), we used the Builder software in PsychoPy (Peirce et al., 2019). Participants viewed the visual stimuli from a distance of about 55 cm. The apparatus and experimental design were identical to those in Experiment 1.

Procedure

The procedure was similar to that in Experiment 1. However, the auditory and visual stimuli for each participant came from different semantic categories (i.e., letters and direction words) rather than the same semantic category (i.e., direction words). We counterbalanced the category-modality pairings (e.g., letters—auditory; direction words—visual) across participants. Participants indicated that a prime or probe was “Left” or “A” by pressing the F key (left middle finger), “Right” or “B” by pressing the G key (left index finger), “Up” or “C” by pressing the J key (right middle finger), and “Up” or “C” by pressing the N key (right index finger).

Data Analysis

The data analysis was identical to that in Experiment 1 (see Appendix D and E for supplemental ANOVAs that include this factor). On average, 17.0% of the trials were errors and .5% were outliers.

Results

Mean Probe RT

There were two significant main effects.⁶ First, there was a main effect of trial N − 1 congruency, F(1, 31) = 19.93, p < .001, ${\eta }_p^2=.39$, because mean RT was longer after congruent trials than after incongruent trials (519 ms vs 511 ms). Second, there was a main effect of trial N congruency, F(1, 31) = 30.68, p < .001, ${\eta }_p^2=.50$. We observed this effect because mean RT was longer in incongruent trials than in congruent trials (529 ms vs. 502 ms).

There were also two significant two-way interactions. First, there was an interaction between modality transition and trial N congruency, F(1, 31) = 24.23, p < .001, ${\eta }_p^2=.44$, because the congruency effect was larger in modality-switch trials than in modality-repeat trials (36 ms vs 18 ms). Second, there was an interaction between trial N − 1 congruency and trial N congruency, F(1, 31) = 63.34, p < .001, ${\eta }_p^2=.67$: as expected, the congruency effect was smaller after incongruent trials (12 ms) than after congruent trials (43 ms).

Finally, there was a significant three-way interaction among modality transition, trial N − 1 congruency, and trial N congruency, F(1, 31) = 34.90, p < .001, ${\eta }_p^2=.53$, because the CSE was larger in modality-repeat trials (48 ms; F[1, 31] = 77.68, p < .001, ${\eta }_p^2=.72$) than in modality-switch trials (14 ms; F[1, 31] = 11.3, p = .002, ${\eta }_p^2=.27$; Figure 6). However, the CSE remained significant in modality-switch trials. Thus, as in Experiment 1, changing the sensory modality reduced, but did not eliminate, the CSE.

Figure 6.

The Congruency Sequence Effect (CSE) in Each of the Two Main Trial Types of Experiment 3: Modality-Repeat Trials and Modality-Switch Trials

Note. Previous trial congruency is indicated on the x axis (Prev Cong: previous congruent trial; Prev Incong: previous incongruent trial). Current trial congruency is indicated by line type (Incong: dashed line; Cong: black line). Reaction time (in ms) is indicated on the y axis. Error bars indicate ±1 SEM.

Mean Probe ER

There were three significant main effects. First, there was a main effect of modality transition, F(1, 31) = 26.42, p < .001, ${\eta }_p^2=.46$, because mean ER was higher in modality-repeat trials than in modality-switch trials (12.2% vs 8.6%). Second, there was a main effect of trial N − 1 congruency, F(1, 31) = 13.15, p = .001, ${\eta }_p^2=.30$. We observed this effect because mean ER was higher after congruent trials than after incongruent trials (11.3% vs 9.5%). Third, there was a main effect of trial N congruency, F(1, 31) = 12.67, p = .001, ${\eta }_p^2=.29$, because mean ER was higher in incongruent trials than in congruent trials (11.5% vs 9.4%).

Discussion

As in Experiment 1, changing the sensory modality reduced, but did not eliminate, the CSE. This result supports the binary coding view, wherein repeating the previous trial’s overall S-R mapping (i.e., identify the stimulus by making one of four possible keypresses) triggers the retrieval of the previous trial’s congruency even when the modality-specific task set changes. In contrast, this result does not support the semantic task set variant of the hierarchical coding view. Because the visual and auditory stimuli came from different semantic categories (i.e., direction words and letters), rather than the same category (i.e., direction words), participants could not form a single, semantic task set that would repeat even if the sensory modality changed.

General Discussion

We sought to distinguish between the hierarchical and binary coding views of task set boundaries for the CSE. Consistent with both views, we found that switching between modality-specific task sets reduces the CSE in both the modified and standard version of the cross-modal prime-probe task. Critically, however, we also found that switching between task sets does not eliminate the CSE in the modified version of the cross-modal prime-probe task. As we explained earlier, this outcome shows that repeating the previous trial’s overall S-R mapping (i.e., identify the stimulus by making one of four possible keypresses) triggers the retrieval of the previous trial’s congruency even when the modality-specific task set changes. Therefore, this outcome favors the binary coding view over the hierarchical coding view. It also has important implications for our understanding of task set boundaries for adaptive control.

Implications for the Binary Coding View

To our knowledge, the present findings are the first to favor the binary coding view over the hierarchical coding view. This outcome extends current views of action control such as the theory of event coding (Hommel et al., 2001) and the Binding and Retrieval in Action Control (BRAC) framework (Frings et al., 2020). The BRAC framework, for instance, focuses on how the formation and retrieval of binary bindings related to concrete features (e.g., stimuli and responses)—rather than abstract features (e.g., task sets and S-R mappings)—drives sequential-trial effects such as the CSE, task-switch costs, and negative priming.

For example, consider a standard 2-AFC flanker task involving the letters A and B wherein repetitions of stimuli and/or responses can occur in consecutive trials. Cognitive control accounts of the CSE posit that a previous incongruent trial (e.g., ABA) trial triggers an up-regulation of control processes that minimize distraction in the current incongruent trial (e.g., ABA), thereby reducing the congruency effect (e.g., Botvinick et al., 2001; Egner, 2008; Gratton et al., 1992). In contrast, the BRAC framework posits that repeating a previous-trial stimulus feature (e.g., the flanker and/or the target) triggers the retrieval of the previous trial’s response, which is stored in the same binary binding (Frings et al., 2020). Retrieving this response when the flankers and target repeat in consecutive incongruent trials facilitates performance in the current incongruent trial, which reduces the congruency effect and thereby leads to a CSE.

Frings et al. (2020) suggest that the BRAC framework can also account for the CSE when no stimuli or responses repeat across consecutive trials. In this framework, repeating the abstract task set or context cues the retrieval of the previous-trial control state. However, the manner in which binding and retrieval operate for such abstract features remains unclear. Along these lines, two findings from the present study suggest that the binary structure of event files extends to abstract features such as task sets and S-R mappings. First, we observed a robust CSE when the modality switched in the modified prime-probe task. This outcome suggests that repeating the previous trial’s S-R mapping triggers the retrieval of the previous trial’s congruency even when the modality-specific task set changes. Second, the exploratory analyses revealed additive effects of switching task sets and switching S-R mappings on CSE magnitude. This outcome further suggests that task sets and S-R mappings are stored in independent, binary bindings with the previous trial’s congruency. Critically, these findings extend the BRAC framework by showing that the formation and retrieval of binary bindings between abstract features can lead to a CSE.

It is important to note, however, that some findings suggest that bindings between abstract features are more complex than what the binary coding view assumes. For instance, consider findings from a recent study of negative priming, which is a phenomenon wherein the appearance of the previous-trial distractor as the current-trial target leads to performance costs (Mayr et al., 2018). In this prior study, retrieval of the previous response occurred when (a) the previous-trial distractor appeared as the current-trial target and (b) the broader auditory context (i.e., a 300- or 700-Hz sine tone played concurrently with the other stimuli) repeated. Such retrieval did not occur, however, when only the broader auditory context repeated. In other words, repeating the context on its own did not lead to the retrieval of the previous-trial distractor response. As Mayr et al. (2018) note, these findings suggest that the structure of an event file may sometimes be more complex than the binary structure proposed by current views of action control (Frings et al., 2020; Hommel et al., 2004). That is, bindings involving abstract features may not always be binary. We discuss this possibility in the next section.

Implications for the Hierarchical Coding View

Although one might interpret the present findings as evidence against the view that tasks are hierarchically organized (Schumacher & Hazeltine, 2016), they may simply indicate that tasks are hierarchically organized in some situations but not others. Along these lines, it is useful to consider the influence of task structure on whether a task set is organized hierarchically or non-hierarchically. In particular, recent findings suggest that a task set is organized hierarchically when the stimuli associated with each of two categories (e.g., faces and places) are mapped to finger responses on different hands but not when those stimuli are mapped to finger responses on the same hand as in the present study (Cookson et al., 2020).

The authors of this prior study assigned each participant to one of two conditions. In condition 1, the authors mapped four face stimuli to four fingers on the left hand and four place stimuli to four fingers on the right hand, or vice-versa. The authors argued that this task structure allows participants to form a hierarchical task representation wherein each stimulus category is associated with a different hand. Thus, cuing the higher-level stimulus category (face or place) should allow participants to prepare for the cued category and, subsequently, to prepare to respond with a particular hand. In condition 2, the authors mapped two face stimuli and two place stimuli to four interleaved fingers on each hand. The authors argued that this task structure does not allow participants to form a hierarchical task representation because each stimulus category is associated with two responses on each hand. Most important for present purposes, the authors found that cuing the stimulus category leads to hand-specific response preparation in condition 1 but not condition 2. Thus, they argued that task structure plays an important role in determining whether or not a task set is organized hierarchically (Cookson et al., 2016, 2020).

These findings may explain why we observed evidence for the binary coding view in the present study rather than for the hierarchical coding view. Specifically, analogous to condition 2 in the studies described above, we did not associate the stimuli in each sensory modality with responses on different hands. Rather, participants used the same fingers on each hand to indicate the identity of the prime and/or probe in every trial, regardless of whether these stimuli appeared in the visual or the auditory modality. Given these previous findings, the present task structure may have led participants to use a nonhierarchical (vs. hierarchical) task representation. To investigate this possibility, future studies could determine whether participants use a hierarchical task representation in studies of the CSE when a task maps different stimulus categories to different hands (e.g., Hazeltine et al., 2011, Experiment 3). If they do, then, unlike in the present study, a CSE should appear only when the task set repeats in consecutive trials.

Broader Implications

The present findings extend prior data suggesting distinct boundaries for the CSE, such as tasks sets (Grant et al., 2020; Hazeltine et al., 2011), conflict types (Egner, 2008; Schlaghecken & Maylor, 2020; but see Weissman, 2020), salient contextual features (Braem, Hickey, et al., 2014; Spapé & Hommel, 2008), and S-R mappings (Grant & Weissman, 2019; for a review, see Braem, Abrahamse, et al., 2014). Specifically, our findings show that the typical elimination of the CSE in modality-switch trials of the standard cross-modal prime-probe task (Grant et al., 2020; Hazeltine et al., 2011; Yang et al., 2017) reflects the cumulative effect of crossing two boundaries—one related to task sets and one related to S-R mappings—rather than a single boundary related to task sets. Further, our findings suggest that task sets and S-R mappings serve as independent boundaries for the CSE.

The present findings also support an emerging view wherein the control processes underlying the CSE contribute to cognition in ways that are broader than minimizing distraction from irrelevant stimuli (Grant & Weissman, 2019; Weissman, 2019). In particular, we observed a CSE in the modified prime-probe task of Experiment 1 even though the prime and probe were both task-relevant. This outcome is consistent with our prior suggestion that adaptive control processes engender a CSE in the prime-probe task via a two-step process (Grant & Weissman, 2019; Weissman et al., 2017). First, they form an implicit expectation that the previous trial’s congruency (e.g., congruent) will repeat in the current trial (Egner, 2014). Second, they combine this expectation with the response cued by the current prime (e.g., “Left”) to prepare a congruent response (e.g., left key) or an incongruent response (e.g., right key) to the upcoming probe. This two-step process occurs similarly in the standard and modified prime-probe tasks with a single exception. In the standard prime-probe task, control processes can prepare a congruent or incongruent response to the probe as soon as participants identify the prime. In the modified prime-probe task, however, such preparation can occur only after participants respond to the prime (Weissman, 2019). This view explains why a CSE appears regardless of whether the prime is task-irrelevant or task-relevant (Grant & Weissman, 2019; Weissman et al., 2017). It also explains why the size of the CSE is independent of the size of the congruency effect (Weissman et al., 2015, Experiment 3). In particular, this view posits that the previous trial’s congruency, rather than the degree of conflict it engenders (Botvinick et al., 2001), triggers control processes underlying the CSE.

The present view of the CSE differs from prevailing views wherein conflict processing and retrieving recently experienced states of selective attention (e.g., highly focused vs less focused) engender this sequential effect (e.g., Botvinick et al., 2001). However, as we reviewed in the Introduction (see The Attentional Shift and Response Modulation Accounts), these prevailing views appear unable to explain numerous aspects of the confound-minimized CSE in the prime-probe task. We note that other findings also suggest that (a) conflict does not trigger cognitive control processes underlying the CSE (Wendt et al., 2006) and (b) control processes modulate response-related processing, rather than selective attention, to engender this sequential effect (e.g., Ridderinkhof, 2002; Stürmer et al., 2002).

The present view of the CSE is also relevant to the prevailing interpretation of neural congruency effects and CSEs in brain imaging studies (e.g., Clayson & Larson, 2011; Forster et al., 2011; Pastötter et al., 2013). Researchers often interpret such effects as indexing the detection and subsequent resolution of response conflict (Botvinick, 2007; Botvinick et al., 2001; Egner, 2008). In this view, the anterior cingulate cortex (ACC), located along the medial walls of the frontal lobes, signals heightened conflict in incongruent relative to congruent trials to other brain regions (e.g., the dorsolateral prefrontal cortex, DLPFC). These regions then resolve conflict by increasing attention to the task-relevant stimulus dimension, leading to a smaller congruency effect in the next trial (i.e., a CSE).

The present view of the CSE does not contradict the assertion that neural congruency effects in the ACC reflect heightened response conflict in incongruent relative to congruent trials. It does suggest, however, that such “conflict-driven” effects may not trigger control processes underlying the CSE. Prior findings support this possibility. For example, neural congruency effects in the ACC—as measured with functional MRI (fMRI)—vanish when congruent and incongruent trials are matched for RT (Carp et al., 2010; Grinband et al., 2011) and, hence, conflict (Yeung et al., 2011). Nonetheless, the CSE in mean probe RT remains highly significant in the absence of a behavioral congruency effect (Weissman et al., 2015, Experiment 3; see also, the present Experiment 2). These findings suggest that objective trial congruency, rather than heightened response conflict, may serve as the crucial trigger of control processes underlying the CSE. Consequently, future studies might investigate which brain regions code for objective trial congruency independent of response conflict. To accomplish this objective, such studies could investigate which brain regions—and which measures of neural activity within those regions—distinguish between RT-matched congruent and incongruent trials.

Limitations

One may wonder why we observed faster (or equivalent) mean probe RT in modality-switch trials than in modality-repeat trials in the modified prime-probe tasks of Experiments 1 and 3. Indeed, performance is usually slower (rather than faster) when two trials involve different task sets than when they involve the same task set (Rogers & Monsell, 1995). Although this result is atypical, it is important to consider that responding to the prime in modality-switch trials requires participants to switch between modality-specific task sets long before the probe appears. Thus, it may eliminate probe-related switch costs. Consistent with this view, we observed slower responses to primes in modality-switch (vs modality-repeat) trials in both experiments (see Footnotes 3 and ⁶). Switching task sets when the prime appears - rather than repeating the same task set - likely also increases arousal, which may also facilitate mean probe RT in modality-switch trials (Rogers & Monsell, 1995). Consistent with both of these potential explanations, we observed the typical costs of switching between task sets in mean probe RT in the standard prime-probe task of Experiment 2, wherein participants responded only to the probe.

One may also wonder whether it is the heightened task-relevance of the prime, rather than the repetition of the previous trial’s S-R mapping, that leads to a larger CSE in the modified version of the cross-modal prime-probe task than in the standard version. The fact that the prime is task-relevant may increase the degree to which its relationship to the probe (i.e., congruent or incongruent) is (a) encoded in the previous trial (cf., Hommel, 2007) and (b) retrieved in the current trial (cf., Huffman et al., 2020), thereby increasing the size of the CSE. We cannot rule out this possibility. However, the CSE is not larger in the modified (vs standard) version of the visual prime-probe task when the prime is task-relevant but associated with a different S-R mapping than the probe (Weissman et al., 2017, Experiment 2). This finding suggests that repeating the previous trial’s S-R mapping in the modified version of the cross-modal prime-probe task increases the CSE independent of the prime’s task-relevance.

Finally, one may wonder whether boundaries for the CSE that are based on salient stimulus properties appear only in cross-modal tasks. Indeed, unlike a change in learned stimulus categories (Hazeltine et al., 2011) or learned S-R mappings (Lim & Cho, 2018), a change in sensory modality constitutes a qualitative change in the brain regions underlying performance (e.g., auditory vs visual cortex) that may exert a large impact on performance. Consistent with this possibility, even changes to conceptual representations that involve different sensory modalities (e.g., “the light is flickering” vs “the sound is echoing”) incur task-switch costs (Pecher et al., 2004; Scerrati et al., 2015). As we mentioned earlier, however, boundaries for the CSE that are based on salient stimulus properties also appear in visual tasks (Dignath et al., 2019). Therefore, such boundaries are not specific to cross-modal task protocols.

Conclusion

We sought to distinguish between the hierarchical and binary coding views of task set boundaries for the CSE. Consistent with the binary coding view, but not with the hierarchical coding view, repeating the previous trial’s S-R mapping engendered a CSE even when the task set changed. Further, exploratory analyses revealed additive effects of switching task sets and switching S-R mappings on CSE magnitude. These findings suggest that task sets and S-R mappings independently trigger the retrieval of the previous trial’s congruency. Thus, they extend current views of action control by showing that the binary structure of bindings for concrete features (i.e., stimuli and responses) in event files generalizes to abstract features (e.g., task sets and S-R mappings). Future studies investigating how the structure of episodic memory influences the CSE may provide additional insights into the nature and scope of adaptive control.

Appendix A. Experiment 1: Mean Probe RT and Mean Probe ER

For the interested reader, we provide the full ANOVA results for mean probe reaction time (RT) and mean probe error rate (ER) from Experiment 1. The four factors are current trial modality (visual, auditory), modality transition (repeat, switch), previous trial congruency (congruent, incongruent), and current trial congruency (congruent, incongruent).

Experiment 1 repeated-measures ANOVA

Key:

mod—current trial modality

mod trans—modality transition

pcong—previous trial congruency

ccong—current trial congruency

Dependent Measure	Effect	F	p	${\eta }_p^2$
Mean probe RT	mod	40.14	<.001	0.56
	mod trans	28.03	<.001	0.48
	pcong	22.91	<.001	0.43
	ccong	43.39	<.001	0.58
	Mod × Mod Trans	< 1	.99	0.00
	Mod × Pcong	<1	.93	0.00
	Mod Trans × Pcong	<1	.69	0.01
	Mod × Ccong	30.24	<.001	0.49
	Mod Trans × Ccong	21.70	<.001	0.41
	Pcong × Ccong	95.31	<.001	0.76
	Mod × Mod Trans × Pcong	<1	.90	0.00
	Mod × Mod Trans × Ccong	6.16	.02	0.17
	Mod × Pcong × Ccong	<1	.49	0.02
	Mod Trans × Pcong × Ccong	20.58	<.001	0.40
	Mod × Mod Trans × Pcong × Ccong	<1	.70	0.01
Mean probe ER	Mod	39.61	<.001	0.56
	mod trans	12.15	.001	0.28
	pcong	17.89	<.001	0.37
	ccong	18.07	<.001	0.37
	Mod × Mod Trans	<1	.65	0.01
	Mod × Pcong	2.22	.15	0.07
	Mod Trans × Pcong	1.99	.17	0.06
	Mod × Ccong	<1	.74	0.00
	Mod Trans × Ccong	<1	.66	0.01
	Pcong × Ccong	33.48	<.001	0.52
	Mod × Mod Trans × Pcong	<1	.98	0.00
	Mod × Mod Trans × Ccong	1.66	.21	0.05
	Mod × Pcong × Ccong	2.82	.10	0.08
	Mod Trans × Pcong × Ccong	<1	.50	0.02
	Mod × Mod Trans × Pcong × Ccong	3.78	.06	0.11

Appendix B. Experiment 1: Mean Prime RT and Mean Prime ER

For the interested reader, we provide the full ANOVA results for mean prime reaction time (RT) and mean prime error rate (ER) from Experiment 1. The three factors are current trial modality (visual, auditory), modality transition (repeat, switch), and previous trial congruency (congruent, incongruent).

Experiment 1 repeated-measures ANOVA

Key:

mod—current trial modality

mod trans—modality transition

pcong—previous trial congruency

ccong—current trial congruency

Dependent Measure	Effect	F	p	${\eta }_p^2$
Mean prime RT	mod	84.72	< .001	0.73
	mod trans	5.06	.03	0.14
	pcong	78.74	<.001	0.72
	Mod × Mod Trans	3.90	.06	0.11
	Mod × Pcong	5.08	.03	0.14
	Mod Trans × Pcong	17.85	<.001	0.37
	Mod × Mod Trans × Pcong	5.41	.03	0.15
Mean prime ER	mod	52.55	<.001	0.63
	mod trans	22.88	<.001	0.43
	pcong	13.56	<.001	0.30
	Mod × Mod Trans	3.18	.08	0.09
	mod × Pcong	2.68	.11	0.08
	Mod Trans × Pcong	1.00	.32	0.03
	Mod × Mod Trans × Pcong	<1	.53	0.01

Appendix C. Experiment 2: Mean Probe RT and Mean Probe ER

For the interested reader, we provide the full ANOVA results for mean probe reaction time (RT) and mean probe error rate (ER) from Experiment 2. The four factors are current trial modality (visual, auditory), modality transition (repeat, switch), previous trial congruency (congruent, incongruent), and current trial congruency (congruent, incongruent).

Experiment 2 repeated-measures ANOVA

Key:

mod—current trial modality

mod trans—modality transition

pcong—previous trial congruency

ccong—current trial congruency

Dependent Measure	Effect	F	p	${\eta }_p^2$
Mean probe RT	mod	35.34	<.001	0.53
	mod trans	57.76	<.001	0.65
	pcong	1.71	.20	0.05
	ccong	1.53	.23	0.05
	Mod × Mod Trans	10.07	.003	0.25
	Mod × Pcong	<1	.89	0.00
	Mod Trans × Pcong	<1	.48	0.02
	Mod × Ccong	28.69	<.001	0.48
	Mod Trans × Ccong	<1	.47	0.02
	Pcong × Ccong	31.20	<.001	0.50
	Mod × Mod Trans × Pcong	<1	.48	0.02
	Mod × Mod Trans × Ccong	3.32	.08	0.10
	Mod × Pcong × Ccong	<1	.80	0.00
	Mod Trans × Pcong × Ccong	15.63	<.001	0.00
	Mod × Mod Trans × Pcong × Ccong	<1	.98	0.00
Mean probe ER	mod	28.54	< .001	0.48
	mod trans	1.23	.28	0.04
	pcong	<1	.77	0.00
	ccong	<1	.45	0.02
	Mod × Mod Trans	1.36	.25	0.04
	Mod × Pcong	<1	.63	0.01
	Mod Trans × Pcong	<1	.43	0.02
	Mod × Ccong	1.16	.29	0.04
	Mod Trans × Ccong	1.09	.30	0.03
	Pcong × Ccong	2.15	.15	0.07
	Mod × Mod Trans × Pcong	4.59	.04	0.13
	Mod × Mod Trans × Ccong	<1	.43	0.02
	Mod × Pcong × Ccong	<1	.50	0.02
	Mod Trans × Pcong × Ccong	<1	.81	0.00
	Mod × Mod Trans × Pcong × Ccong	<1	.99	0.00

Appendix D. Experiment 3: Mean Probe RT and Mean Probe ER

For the interested reader, we provide the full ANOVA results for mean probe reaction time (RT) and mean probe error rate (ER) from Experiment 3. The four factors are current trial modality (visual, auditory), modality transition (repeat, switch), previous trial congruency (congruent, incongruent), and current trial congruency (congruent, incongruent).

Experiment 3 repeated-measures ANOVA

Key:

mod—current trial modality

mod trans—modality transition

pcong—previous trial congruency

ccong—current trial congruency

Dependent Measure	Effect	F	p	${\eta }_p^2$
Mean probe RT	mod	233.37	<.001	0.88
	mod trans	3.37	.08	0.10
	pcong	19.93	<.001	0.39
	ccong	30.68	<.001	0.50
	Mod × Mod Trans	<1	.65	0.01
	Mod × Pcong	4.07	.05	0.12
	Mod Trans × Pcong	<1	.39	0.02
	Mod × Ccong	10.25	.003	0.25
	Mod Trans × Ccong	24.23	<.001	0.44
	Pcong × Ccong	63.34	<.001	0.67
	Mod × Mod Trans × Pcong	<1	.70	0.01
	Mod × Mod Trans × Ccong	3.57	.07	0.10
	Mod × Pcong × Ccong	1.64	.21	0.05
	Mod Trans × Pcong × Ccong	34.90	<.001	0.53
	Mod × Mod Trans × Pcong × Ccong	<1	.82	0.00
Mean probe ER	mod	23.45	<.001	0.43
	mod trans	26.42	<.001	0.46
	pcong	13.15	.001	0.30
	ccong	12.67	.001	0.29
	Mod × Mod Trans	22.99	<.001	0.43
	Mod × Pcong	<1	.38	0.03
	Mod Trans × Pcong	1.71	.20	0.05
	Mod × Ccong	<1	.67	0.01
	Mod Trans × Ccong	<1	.48	0.02
	Pcong × Ccong	1.51	.23	0.05
	Mod × Mod Trans × Pcong	<1	.90	0.00
	Mod × Mod Trans × Ccong	1.51	.23	0.05
	Mod × Pcong × Ccong	9.68	.004	0.24
	Mod Trans × Pcong × Ccong	<1	.55	0.01
	Mod × Mod Trans × Pcong × Ccong	<1	.77	0.00

Appendix E. Experiment 3: Mean Prime RT and Mean Prime ER

For the interested reader, we provide the full ANOVA results for mean prime reaction time (RT) and mean prime error rate (ER) from Experiment 3. The three factors are current trial modality (visual, auditory), modality transition (repeat, switch), and previous trial congruency (congruent, incongruent).

Experiment 3 repeated-measures ANOVA Key:

mod—current trial modality

mod trans—modality transition

pcong—previous trial congruency

Dependent Measure	Effect	F	p	${\eta }_p^2$
Mean prime RT	mod	169.92	<.001	0.85
	mod trans	51.59	<.001	0.63
	pcong	39.84	<.001	0.56
	Mod × Mod Trans	1.57	.22	0.05
	Mod × Pcong	5.42	.027	0.15
	Mod Trans × Pcong	36.81	<.001	0.54
	Mod × Mod Trans × Pcong	2.23	.145	0.07
Mean prime ER	mod	23.71	<.001	0.43
	mod trans	27.87	<.001	0.47
	pcong	15.92	<.001	0.34
	Mod × Mod Trans	26.72	<.001	0.46
	Mod × Pcong	1.33	.257	0.04
	Mod Trans × Pcong	2.68	.112	0.08
	Mod × Mod Trans × Pcong	<1	.776	0.00