Pinpointing mechanisms of a mechanistic treatment: Dissociable roles for overt and covert attentional processes in acute and long-term outcomes following Attention Bias Modification

Abstract

Biased patterns of attention towards threat are implicated as key mechanisms in anxiety which can be modified through automated intervention (Attention Bias Modification; ABM). Intervention refinement and personalized dissemination efforts are substantially hindered by gaps in understanding the precise attentional components that underlie ABM’s effects on symptoms—particularly with respect to longer-term outcomes. Seventy adults with transdiagnostic anxiety were randomized to receive 8 sessions of active ABM (n=49) or sham training (n=21). Reaction time and eyetracking data, collected at baseline, post-training, and 1-month follow-up, dissociated multiple core attentional processes, spanning overt and covert processes of engagement and disengagement. Self-reported symptoms were collected out to 1-year follow-up. Covert disengagement bias was specifically reduced by ABM, unlike all other indices. Overt disengagement bias at baseline predicted acute post-ABM outcomes, while covert engagement bias was non-specifically predictive of symptom trajectories out to 1-year follow-up. Results suggest unique and dissociable roles for each discrete mechanism.

Preferential attention towards threat cues (henceforth, “attentional bias”) in anxious individuals is one of the most well-documented information processing patterns in clinical psychology. Using numerous diverse methodologies, and across virtually all forms of clinical and sub-clinical anxiety, a large literature supports a general pattern of attentional bias towards threat among anxious individuals (Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & van IJzendoorn, 2007; MacLeod & Grafton, 2016), which is theorized to promote pervasive overestimation of dangers in the environment and perpetuate anxiety over time. These well-documented patterns of experimental task performance eventually gave rise to explicit attempts to modify attentional patterns (“Attention Bias Modification”; ABM) in order to experimentally probe the mechanistic role of attention bias as a causal factor in anxiety and related forms of affective dysregulation (MacLeod, Rutherford, Campbell, Ebsworthy, & Holker, 2002). The success of this early work inspired further work testing ABM as an intervention strategy in clinical and high-anxiety samples. ABM has now been widely-studied as a clinical intervention, but results have been mixed. Numerous recent meta-analyses do support ABM’s efficacy in reducing symptoms of clinical anxiety, but only in a subset of individuals and studies (Heeren, Mogoase, Philippot, & McNally, 2015; Linetzky, Pergamin-Hight, Pine, & Bar-Haim, 2015; Price, Wallace, et al., 2016), and not all meta-analyses have supported a clinical effect (Cristea, Kok, & Cuijpers, 2015) [for a rebuttal, see, e.g., (Clarke, Notebaert, & Macleod, 2014)]. This heterogeneity in outcomes may be tied to the fact that attention is far from a unitary construct. Thus, to capitalize on the apparent potential of this mechanistic treatment strategy and point the way to further refinements and/or novel approaches, a more precise and sophisticated characterization of the attentional mechanisms involved is needed.

Attention acts as a filter, curating the set of perceptions an individual encounters on a moment-to-moment basis (Carrasco, 2011). The cacophony of stimuli, both internal and external, present in most environments far outpaces humans’ limited cognitive capacity; thus, attention will largely shape and define the subset of all possible information that is available for further processing. Basic cognitive science research has helped to delineate some of the multifaceted components involved in attention (Petersen & Posner, 2012). One prominent dichotomy is the distinction between overt visual attention—or the direction of eye gaze towards an overt visual target—and covert attention, which involves more subtle and internal processes, such as moving the covert ‘spotlight’ of attentional focus to more extensively process different regions of the visual field and periphery (Posner, Snyder, & Davidson, 1980; Weierich, Treat, & Hollingworth, 2008). A second dichotomy involves the distinction between a) being initially drawn towards threat-related cues, so that they are more likely to enter early stages of awareness (an “engagement” bias), and b) experiencing difficulty shifting attention away from threat cues once encountered, so that they are more likely to become the focus of perseverative cognition and/or elaborative processing (a “disengagement” bias). These disparate components of attention have distinct neurobiological bases (Petersen & Posner, 2012), and may produce unique clinical manifestations and/or call for distinct, process-based intervention prescriptions (Grafton & MacLeod, 2014; Grafton, Southworth, Watkins, & MacLeod, 2016; Rudaizky, Basanovic, & MacLeod, 2014). Across hundreds of largely cross-sectional studies comparing anxious and non-anxious individuals, using a variety of methods (e.g., reaction time measures, eyetracking of gaze patterns, temporal patterns of neural activity), evidence has emerged consistent with the presence of each of these types of bias among anxious individuals (Cisler & Koster, 2010; MacLeod & Grafton, 2016). However, no clear consensus has emerged to suggest which process, if any, is most characteristic of anxiety. In fact, this field of inquiry has been riddled by inconsistencies with regard to which attentional processes, in which populations, using which tasks, demonstrate baseline attentional biases and/or are impacted by ABM training [e.g., (Bantin, Stevens, Gerlach, & Hermann, 2016)], and the task of piecing together findings across studies that have tended to assess one or two subcomponents of attention, but are divergent from one another on a large number of other relevant features (e.g., measurement procedures, ABM procedures, clinical features of the sample, developmental factors, demographics, etc.), is particularly challenging.

When samples are defined based on a traditional categorical approach (e.g., specific diagnoses or specific elevated symptom scores), this difficulty pinpointing precise attentional mechanisms may reflect heterogeneity within anxious samples regarding the degree to which one type of bias or another is present. Developmental models of anxiety suggest that innate biases within the individual (e.g., fearful temperament, behavioral inhibition) can potentiate and exacerbate normative attention biases to threat, thereby increasing later risk for anxiety disorders (Morales, Fu, & Perez-Edgar, 2016). However, discrete components of attention may each be heterogeneous within and across anxious samples, and could exhibit distinct influences on long-term trajectories of anxiety. Heterogeneity within anxious samples is particularly relevant given that attentional patterns exist (and are quantified) along a continuum that ranges from extreme vigilance (or attentional bias towards threat) to extreme avoidance (or attentional bias away from threat). Though this latter pattern is less commonly discussed within the attentional bias literature, like threat vigilance, it is posited to be highly relevant to clinical manifestations of anxiety (e.g., promoting avoidance behaviors in the real world, interfering with elaborative processing of threat and concomitant habituation processes), and has been linked empirically to negative trajectories of mental health (Legerstee et al., 2010; Price, Tone, & Anderson, 2011; Price, Rosen, et al., 2015; Waters, Mogg, & Bradley, 2012). An alternative approach to traditional diagnosis-based group comparisons is to cast a wide net, cutting across traditional diagnostic boundaries in anxiety, and then apply process-based indices of attention in order to characterize mechanisms (e.g., overt and covert, engagement and disengagement biases, across a continuum from vigilance to avoidance) at the individual level. Within a mechanistic treatment framework, this approach can be paired with efforts to experimentally manipulate cross-cutting targets (e.g., with ABM) and to identify individual patients who will respond most favorably to such manipulations on the basis of baseline attentional profiles.

What do we know about the attentional mechanisms involved in ABM?

A large majority of previous ABM studies have utilized manual (button-press) responses on the “dot-probe” task (MacLeod & Mathews, 1988) as both the method of assessing attentional bias (before and after ABM) and as the method of modifying attentional bias (during ABM). More specifically, in the dominant ABM paradigm (MacLeod et al., 2002), participants practice repeated trials of the dot-probe task in which a probe stimulus always (or nearly always) appears in the previous location of a non-threat-related (e.g., neutral) cue as opposed to a threat-related cue, with the goal of guiding attention towards less threatening information over time. Meta-analyses—including one analysis utilizing participant-level data pooled across several hundred anxious participants (Price, Wallace, et al., 2016)—suggest this form of ABM exerts an effect on attentional bias assessed by dot-probe reaction times, consistent with a general decrease in attentional bias following active training (in relation to control conditions). Similarly, greater attentional bias exhibited by individuals at baseline—suggesting a good mechanistic fit for the intervention’s target—has been found to moderate symptomatic improvements following ABM in a subset of studies (Amir, 2011; Kuckertz et al., 2014; Price, Wallace, et al., 2016),

Unfortunately, there are several drawbacks to the use of manual responses on the dot-probe task as a mechanistic assessment in ABM trials which limit the conclusions that can currently be drawn from this literature. First, manual responses collected during the dot-probe task conflate overt and covert attention, as well as engagement and disengagement processes, precluding a better understanding of the essential mechanistic ingredients involved in reducing anxiety via attentional retraining. A greater degree of bias in reaction times on the task may indicate that attention was more readily engaged by threat cues—which may or may not have involved a bias in overt eye movements; or that it was relatively difficult to disengage attention from a threat cue—which again may include overt and/or covert processes—or any possible combination of these distinct components.

In addition, when manual responses on the dot-probe task are used both to modify attention bias as well as to assess it, the procedure provides only a proximal manipulation check, assessing whether the target performance pattern itself was well-learned by participants. A more clinically and theoretically pressing question is whether ABM is capable of creating generalized shifts in attentional patterns. If a core, clinically relevant biobehavioral phenomenon has indeed been altered, such changes would be expected to be evident even when measuring attention bias using distinct task procedures and/or stimulus and measurement modalities.

Pinpointing discrete attentional mechanisms

Unlike dot-probe reaction times, alternative assessment modalities offer the capacity to more precisely tease apart distinct attentional processes. For example, to quantify overt attentional patterns, one approach is to concurrently collect overt eye movement data during an attentional task (such as the dot-probe task) with an eyetracker. Because the eyetracker offers high temporal resolution (e.g., sampling eye gaze every 17ms or less), engagement processes, such as a bias in which stimulus receives the first “look” on any given trial, can be readily separated from disengagement processes, such as a delay in the time taken to reorient visual gaze whenever the task requires it (Heeren, Baeken, Vanderhasselt, Philippot, & de Raedt, 2015; Price, Kuckertz, et al., 2015; Ricketts et al., 2018). Covert attentional processes, though (by definition) not observable in overt eye movements, can also be separated into engagement vs. disengagement components with simple reaction time tasks. One task that has been used extensively for this purpose is the emotional spatial cueing task (Bar-Haim, Morag, & Glickman, 2011; Fox, Russo, Bowles, & Dutton, 2001; Posner et al., 1980). This task typically instructs participants to rest their eyes on a central fixation cross (thereby discouraging overt attentional shifts), and presents four types of trials: two trial types in which a briefly presented threat-related or non-threat-related cue (e.g., an emotional face), presented within peripheral vision, is followed by a target in the same location, providing an index of the tendency for attention to be preferentially engaged by threat-related cues early in processing; and two trial types in which the target appears in the position opposite to the briefly presented threat-related or non-threat-related cue, providing an index of the relative difficulty experienced when attention must be specifically disengaged from the threat location.

It is essential to note that the spatial cueing task has been critiqued for presenting potential confounds; in particular, the disengagement index it provides is not entirely separable from engagement processes, since attentional capture (engagement) by any given stimulus must occur before disengagement would be required (e.g., Grafton & MacLeod, 2014). Though newer tasks have more recently been developed to more precisely disentangle disengagement from engagement processes and remove other potential confounds (e.g., Grafton & MacLeod, 2014), the emotional spatial cueing task has been well-validated in terms of its relevance to anxious patients across a range of diagnoses and age groups (Cisler & Koster, 2010). If and when differential findings for engagement vs. disengagement indices are observed within a single data set (e.g., engagement patterns follow one trajectory while disengagement patterns follow another), this strengthens the possibility that the task achieved at least partial separation of the two constructs.

To date, a relatively small subset of ABM studies have made use of any other assessment strategy, including overt eyetracking indices or covert reaction-time based attention tasks, as a distinct method to assess attentional patterns before and after ABM. This is a critical gap given evidence that the attentional processes measured via dot-probe task reaction times are distinct from processes measured with alternative assessments (Dalgleish et al., 2003; Morales, Taber-Thomas, & Perez-Edgar, 2016; Ricketts et al., 2018). One study of standard dot-probe-based ABM among highly trait-anxious individuals found that when a single session of ABM was combined with prefrontal cortex brain stimulation, an eye-tracking index of disengagement (but not engagement) was specifically reduced (Heeren, Baeken, et al., 2015). An additional study in a healthy sample found that a single session of ABM impacted performance on an anti-saccade task—an eyetracking index of volitional inhibitory control (Chen, Clarke, Watson, MacLeod, & Guastella, 2015). However, this effect was seen irrespective of stimulus valence, suggesting ABM did not specifically impact threat-related overt attention in this healthy sample. Both studies delivered only a single session of ABM, and neither study included covert attentional bias assessment for comparison.

Among studies that used a spatial cueing task to assess covert attention following dot-probe-based ABM, three such studies in socially anxious samples (two using a single-session ABM protocol, and another using 8 repeated sessions delivered over 4 weeks) showed specific ameliorating effects of ABM on disengagement, but not engagement, processes, as assessed by the spatial cueing task (Amir, Beard, Taylor, et al., 2009; Amir, Weber, Beard, Bomyea, & Taylor, 2008; Heeren, Lievens, & Philippot, 2011). In each of these three cases, the samples also exhibited a group-level pattern of difficulty disengaging from threat at baseline, and the ABM procedures produced clinically relevant reductions in symptoms (in comparison to control conditions). Similarly, in a sample of chronically high-anxious 10-year-olds, a variant of ABM explicitly targeting disengagement bias produced both a reduction in disengagement bias and an improvement in symptoms (Bar-Haim et al., 2011). By contrast, other studies did not find group-level effects of ABM on spatial cueing task performance (Boettcher, Berger, & Renneberg, 2012; Enock, Hofmann, & McNally, 2014; Heeren, Mogoase, McNally, Schmitz, & Philippot, 2015; Julian, Beard, Schmidt, Powers, & Smits, 2012; Maoz, Abend, Fox, Pine, & Bar-Haim, 2013; Sigurjonsdottir, Bjornsson, Ludvigsdottir, & Kristjansson, 2014).

In addition to delineating treatment mechanisms more precisely, another goal of mechanistic treatment research is to identify process-related variables that could ideally be used to personalize treatment, tailoring the selection of strategies to the mechanisms that are in fact most relevant for an individual patient (Price, Paul, Schneider, & Siegle, 2013). To date, published findings on prediction of ABM clinical outcomes have been nearly entirely focused on acute outcomes immediately following ABM. While such findings speak to the proximal mechanisms of symptomatic relief, they offer little insight regarding the type of clinically impactful change that would substantially alter an individual’s mental health trajectory and its concomitant cumulative impact on physical health, disability, and functional attainment (Kessler, 2007). Both the impact of ABM on clinical outcomes beyond the acute post-training period, as well as any potential individual difference predictors of such impact, remain almost entirely unknown. While a small fraction of ABM studies have examined outcomes at 1-, 4- and/or 6-months post-treatment (Heeren, Mogoase, Philippot, et al., 2015; Linetzky et al., 2015; Rapee et al., 2013), investigation of clinical symptoms one year or more after the conclusion of training has been absent in published reports to date. This precludes both a thorough understanding of maintenance of gains as well as the detection of possible emergent effects that may appear for the first time during follow-up, as previously observed in a sample of remitted depressed patients (Browning, Holmes, Charles, Cowen, & Harmer, 2012). In addition to lack of clinical follow-up, ABM studies have not previously re-assessed attentional bias itself following an acute post-training assessment. Thus, the durability of any observed shifts in the target mechanism, in the absence of continued training sessions, is unknown.

Current Study

To inform a more complete understanding of ABM’s clinical impact and mechanisms, the present randomized controlled trial aimed to deliver ABM (or a control training condition) to a transdiagnostic clinically anxious sample. Attentional features were then used to parse the heterogeneity that was expected within this transdiagnostic sample, in an effort to understand the precise attentional processes that make ABM clinically beneficial for specific individuals. Consistent with the vast majority of ABM studies to date, ABM itself utilized a dot-probe task-based paradigm, promoting the relevance of current findings to the larger clinical ABM literature. We assessed attentional mechanisms using both overt and covert methods, each of which was designed to further help distinguish engagement from disengagement processes, and re-assessed these processes at a 1-month post-training follow-up, in an effort to chart the trajectory and durability of specific attentional shifts after training sessions were discontinued. We further aimed to enhance the clinical relevance of findings by focusing on symptom outcomes out to 1-year post-training, providing a glimpse at the potential for a brief, fully automated, mechanistic intervention to transform longer-term trajectories of mental health—with a focus on identifying key attentional features that may influence this impact.

Because attentional processes are multi-faceted, with unique neural and evolutionary underpinnings (Petersen & Posner, 2012; Weierich et al., 2008), we expected each attentional component to be dissociable from all others, and to play a unique mechanistic role in the context of ABM. Given relative consistency of prior findings regarding the impact of ABM on disengagement biases (Amir, Beard, Taylor, et al., 2009; Amir et al., 2008; Heeren et al., 2011), we predicted that ABM would ameliorate attentional disengagement, but not engagement, processes. In addition, previous research has suggested that attentional avoidance of threatening stimuli at baseline is a poor prognostic factor, acting both as a moderator of acute post-ABM outcomes [i.e, more avoidance at baseline suggests poorer fit for standard ABM procedures; (Kuckertz et al., 2014; Price, Wallace, et al., 2016)] and as a general predictor associated prospectively with increased anxiety and other internalizing problems, following standard cognitive-behavioral therapies and/or over longer-term follow-up (Fawcett et al., 1990; Gibb, Benas, Grassia, & McGeary, 2009; Legerstee et al., 2010; Price et al., 2011; Price, Rosen, et al., 2015; Waters et al., 2012). Thus, we predicted that avoidant attentional patterns (indicated by lower/more negative attentional bias scores) would predict poorer prognosis. Given the dearth of past research directly comparing discrete components of attention in clinical anxiety and/or ABM, we did not have a strong basis for further specific hypotheses.

As a secondary study aim, we examined the potential for poor or non-existent reliability in the attentional bias indices obtained in the current study. Manual reaction times indices taken during dot-probe tasks have been shown repeatedly to exhibit significant psychometric constraints (Price, Kuckertz, et al., 2015; Rodebaugh et al., 2016; Schmukle, 2005), increasing the risk of both Type I and Type II error when used as an outcome assessment and placing a highly constraining upper limit on the measure’s capacity to track with other indices of interest across individuals (e.g., symptom measures). Thus, the findings that address questions most germane to mechanistic treatment research (e.g., do symptom improvements depend upon achieving a successful change in the targeted attentional pattern? do individuals with strong impairments in the target mechanism benefit most?) are exceedingly difficult to reproduce and may contribute to null and mixed findings. Despite its importance (Hajcak, Meyer, & Kotov, 2017), few previous studies to our knowledge have reported on the reliability of alternative attentional bias indices (e.g., eye gaze; emotional spatial cueing task). In one prior study reporting on the reliability of engagement and disengagement indices derived from the spatial cueing task in a sample of GAD patients, poor to non-existent (<0 to .28) split-half reliability was obtained (Kinney, Boffa, & Amir, 2017). In another study, eyetracking indices collected during the dot-probe task among healthy youth showed significant, but still suboptimal (i.e., ICC’s<=.3), test-retest reliability (Price, Kuckertz, et al., 2015). Thus, the current study assessed the internal reliability of all examined attentional indices, in an effort to tackle head-on the psychometric constraints that have plagued this field of inquiry.

In summary, key components of the current work included: 1) comprehensive assessment of dissociable (overt/covert, engagement/disengagement) attentional features within a single sample; 2) administration of an adequate dose of ABM (e.g., 8 sessions) to promote lasting clinical change (e.g., Beard, Sawyer, & Hofmann, 2012); 3) repeated measurements of all levels of analysis at baseline, acute post-treatment, and follow-up; 4) prolonged (1-year) follow-up on clinical measures; 5) transparent reporting of reliability findings for all attentional bias measures; and 6) recruitment of a transdiagnostic clinical sample. Though a subset of these study features has been represented in prior ABM studies, which together motivate our hypotheses regarding discrete roles for discrete attentional components, we anticipated that the combined benefit of including these features within a single study would be to enable a more comprehensive picture to emerge regarding the key attentional mechanisms involved in transdiagnostic ABM outcomes, at multiple distinct and clinically meaningful timepoints.

Methods

Seventy unmedicated patients reporting clinically elevated levels of trait anxiety and associated clinician-rated disability (full inclusion/exclusion criteria described in Supplement) were randomized to receive active ABM (n=49) or a sham control variant (n=21) (clinicaltrials.gov: ). By design, uneven allocation was used so that available resources could be leveraged to maximize the sample size and statistical power in the active ABM group, as primary study aims and hypotheses concerned identifying mechanistic predictors of ABM response. The ABM sample size was selected based on an a priori power analysis suggesting that, for 0–20% attrition rates and two-tailed α=.05, the study had adequate (80%) power to detect medium effects (|r|=.38-.42) or larger, which were considered minimally sufficient for clinical relevance and are consistent with previous findings (e.g., Price, Wallace, et al., 2016). The sham sample was included so that analyses could also probe the specificity of findings to active ABM. 94% of randomized patients completed their assigned treatment condition and the acute post-treatment assessment, while 87% completed the 1-month and 79% completed the 1-year follow-up (CONSORT diagram: Figure 1). See Table 1 for additional sample characteristics. This study was approved by the local Institutional Review Board.

Figure 1.

CONSORT Flow Diagram

Table 1.

Demographic and Clinical Characteristics of the Intent-to-Treat Sample

	ABM (n=49)				Sham (n=21)
	Pre-treatment	Acute post-treatment			Pre-treatment	Acute post-treatment
Caucasian, n (%)	31 (63%)				16 (76%)
Female, n (%)	38 (78%)				16 (76%)
Age	30.78 (9.68)				29.86 (11.84)
Anxiety diagnosesmet, n (%):
GAD	42 (86%)				16 (76%)
SAD	16 (33%)				9 (43%)
Panic/agoraphobia	6 (12%)				3 (14%)
PTSD	7 (14%)				1 (5%)
Specific Phobia	5 (10%)				2 (10%)
OCD	5 (10%)				0 (0%)
Comorbid depressive disorder	15 (31%)				6 (29%)
Number of anxiety diagnoses	1.78 (1.09)	1.16 (1.05)			1.57 (0.93)	1.33 (1.15)
Remission of primary anxiety diagnosis, n (%)	-	15 (34%)¥			-	3 (14%)¥
Remission of all anxiety diagnoses, n (%)	-	10 (22%)¥			-	1 (5%)¥
Clinician-rated hypervigilance	4.54 (2.03)	3.90* (1.97)			5.24 (2.23)	4.19* (2.16)
Self-report symptom indices
	Pre-treatment	Acute post-treatment	1-month follow-up	1-year follow-up	Pre-treatment	Acute post-treatment	1-month follow-up	1-year follow-up
MASQ: Anxious Arousal	31.94 (10.51)	28.38* (9.41)	26.98* (7.85)	24.03* (8.45)	34.05 (10.72)	30.29 (13.92)	29.00* (12.42)	27.79* (11.86)
MASQ: General Distress	55.90 (15.06)	48.00* (13.46)	44.02* (11.28)	44.97* (15.06)	60.24 (17.79)	53.76* (22.86)	49.85* (19.54)	47.21* (22.95)
PSWQ	63.22¥ (10.79)	61.73 (10.42)	59.21* (12.43)	58.67* (11.47)	68.48¥ (8.18)	65.76 (12.59)	64.60* (11.20)	57.95* (13.93)

Note: Data presented as mean (SD) unless otherwise noted. PSWQ=Penn State Worry Questionnaire; MASQ=Mood and Anxiety Symptoms Questionnaire; GAD=Generalized Anxiety Disorder; SAD=Social Anxiety Disorder; PTSD=Posttraumatic Stress Disorder; OCD=Obsessive-Compulsive Disorder; NOS=Not Otherwise Specified.

^*Significant (p<.05) within-group decrease relative to baseline (according to paired t-test).

^¥There were non-significant trends at acute post-treatment suggesting a greater proportion of ABM participants were remitted from their primary anxiety diagnosis (Chi-square=2.79, p=.095) and all anxiety diagnoses (Chi-square=3.14, p=.076) relative to the control group. PSWQ was marginally greater at baseline in the control group than the ABM group [t(68)=1.995; p=.050002]. No other significant or trend-level differences were observed for ABM vs. control group on any demographic or clinical variable according to t-tests (continuous variables) or X² (categorical variables).

ABM and sham conditions

Patients and clinical assessors were successfully blinded to treatment allocation (see Supplement). The ABM and sham conditions were modeled after prior studies (e.g., Amir, Beard, Burns, & Bomyea, 2009) and described in detail in a previous report which focused on fMRI data collected in this sample (Price et al., In press). Briefly, participants in both groups completed 8 twice-weekly sessions, in the laboratory, using a dot-probe task to re-train attention. At baseline, ten idiographic threat words capturing the primary foci of anxiety were selected collaboratively by the participant and clinical interviewer, and were subsequently matched ideographically, on both subjective familiarity ratings and word length, to 10 neutral words drawn from a larger normative corpus used previously in ABM research (e.g., Amir, Beard, Burns, et al., 2009). To supplement this idiographic attention training with training pertinent to a broader and more generalized range of threat-related content, these idiographic words lists were supplemented by 20 threat words and 20 neutral words from the normative corpus. During training trials (300 administered at each training session), word pairs (80% threat-neutral; 20% neutral-neutral) were presented vertically for 500ms, followed by a probe (‘E’ or ‘F’) in either the upper or lower word location. Participants responded via button press to indicate the probe letter displayed. All text (words and probes) was presented in 14pt font, with the distance between the upper and lower screen position transcending a visual angle of 2°.

The only distinction between the ABM and control conditions was in the relationship between the probe location and the threat word in each word pair. In ABM, for 100% of threat-neutral trials (80% of all trials), the probe replaced the neutral word, thereby shaping attention away from threatening cues through practice. In the sham condition, the probe replaced either the threat or neutral word with equal likelihood.

Self-reported Symptoms

Although the primary outcome measure selected to index acute post-treatment outcomes in the trial was a clinician-rated index of hypervigilance, it was not feasible to collect this measure repeatedly across the full 1-year follow-up window. As one of the principle goals of the present analyses was to investigate longer-term outcomes, we therefore utilized self-report symptom measures, which were collected out to 1-year post-intervention, uniformly for all analyses. The primary self-report outcome for the trial was the Mood and Anxiety Symptoms Questionnaire (MASQ; 64-item short form), which was developed and validated to allow discrimination between anxiety- and depression-specific symptoms and general distress (Watson et al., 1995). The MASQ provides three subscales capable of capturing clinically relevant symptoms across a range of diagnoses: General Distress, Anxious Arousal, and Anhedonic Depression. The General Distress and Anxious Arousal subscales were of principle interest given their capacity to capture relevant symptom patterns within our transdiagnostic anxious patients. The Anhedonic Depression subscale, as well as the Penn State Worry Questionnaire (Meyer, Miller, Metzger, & Borkovec, 1990)), a well-validated measure of the severity and controllability of generalized worry, were included in exploratory analyses to assess the specificity of findings within the broader spectrum of internalizing symptoms (see Supplement).

Self-report symptom indices were collected at four timepoints: at a pre-training baseline visit (completed app. 1–2 weeks prior to the beginning of attention training); at acute post-training (within app. 1 week of completing the final computer training session); at a 1-month follow-up visit; and at 1-year follow-up. The first three assessments were completed at the clinical research facility, while the final assessment was completed remotely via a web-based platform. Internal consistency (Cronbach’s α) for the three MASQ subscales was acceptable to excellent (0.72–0.95) at every timepoint.

Attentional Bias Indices

Covert attention.

The emotional spatial cueing task was developed as a modification of covert attentional probes widely used in basic cognitive research to distinguish covert from overt attentional processes (Posner et al., 1980). It was identical to the task used in a prior study (Bar-Haim et al., 2011) and similar to emotional spatial cueing tasks used widely in attention research. As both the task paradigm (spatial cueing as opposed to dot-probe paradigm) and the stimulus modality (face pairs rather than word pairs) differed from those used within the ABM and control training sessions, the task provided a rigorous assessment of the generalization of training to a novel attentional task that assesses the target mechanism (attentional bias to threat) in a unique manner. In brief, participants are instructed to rest their eyes on a central fixation cross which remains on-screen throughout the task, to discourage overt eye movements. Consistent with a large literature utilizing this task to assess covert attention, compliance with this instruction was not explicitly assessed, but task features (e.g., placement of large, unambiguous spatial probes well within the field of peripheral vision) encourage compliance. On each trial, a single face was presented either left or right of the fixation cross located at the center of the screen. Face stimuli consisted of 12 different actors (50% male) displaying neutral or angry expressions from the NimStim stimulus set (Tottenham et al., 2009). After a 500ms pause, two black rectangle frames appeared to the left and right of the cross, one of which framed either a neutral or an angry face. After 500ms, the face was removed and a target (a star) appeared for 200ms at the center of one of the two rectangles. 75% of the trials were “valid” trials, meaning the target appeared at the location of the face cue, while the remaining 25% of the trial targets were “invalid”, meaning the target appeared on the opposite side of the screen. Participants pressed one of two buttons on a keyboard to indicate the location of the star. The next trial began 1,800ms after target offset. A beep tone was played to indicate an incorrect response or no response within the response window.

The task commenced with 8 practice trials comprised of 2 trials per condition. The participant could elect to repeat the practice as needed. 192 trials (96 of each emotion type) were then presented across two blocks. 25% of the trials of each emotion type were invalid cues and 75% were valid cues.

Analyses of behavioral spatial cueing task data across the full sample at baseline are presented in the Supplement (see “Analysis of Spatial Cueing Reaction Times at Baseline”).

Overt attention.

Eyetracking indices were collected during an assessment dot-probe task, which was identical to the sham training task described above, with some modifications [total of 300 trials were given, which included 150 trials with 500ms stimulus presentations (as above), randomly interspersed with 150 trials of longer (1500ms) duration]. The dot-probe task was administered individually in a quiet, moderately lit room on a monitor at ~68cm from the participant using Eprime software running on a PC. Eyetracking data were collected during the task using a table-mounted RK-768 eye-tracker, consisting of a video camera and infrared light source pointed at a participant’s eye and a device that tracked the location and size of the pupil and corneal reflection at 60 Hz (every 16.7 ms). As previously described (Price, Siegle, et al., 2013; Silk et al., 2012), data cleaning and preprocessing were applied in Matlab. Blinks were automatically identified and corrected using interpolation. Prior to the task, the eyetracker was calibrated using a 9-point sequential fixation task. Gaze position during the dot-probe task was scaled and offset based on each individual’s calibration parameters. Hand quality-checking was performed offline to ensure accurate registration of each participant’s gaze position to the two relevant screen positions (upper/lower) at the time that accurate responses to the probe in each of these positions were made. Participants (baseline: n=2; post-treatment: n=2; 1-month follow-up: n=2) whose data quality was deemed poor were excluded from all subsequent eyetracking analysis.

Eye fixations were defined as eye positions stable within 1° of visual angle for at least 100ms. Trials comprised of less than 25% fixation or with incorrect probe responses were removed from analysis. Fixations were then used to identify, for each individual: 1) trials in which the first (initial) fixation that was made following word pair onset fell within a region-of-interest defined by the threat word’s screen position (an index of engagement); and 2) trials in which the participant’s fixation, at the time of probe onset (i.e., just when the word pair was replaced by a probe), fell in the opposite word location from where the probe appeared (thus requiring disengagement from one location and an overt eye movement in order to respond to the probe accurately). These trials were separately identified as requiring disengagement from the threat word location (disengage-threat) or disengagement from the neutral word location (disengage-neutral). A minimum of 5 usable trials of each type (disengage-threat and disengage-neutral) was required in order for eyetracking data to be deemed usable at a given assessment point. Among included participants/sessions, an average of 38 trials per participant were available to generate overt disengagement bias scores. There were no significant differences for participants with vs. without usable eyetracking data on age, gender, or MASQ scores at any time point. The pattern of all findings described below was unchanged when using all available eyetracking data.

The covert (spatial cueing) and overt (dot-probe with eyetracking) tasks were completed at the same baseline, acute post-training, and 1-month follow-up visits as the self-report scales, but were not completed at the 1-year remote follow-up assessment.

Bias score calculation.

For the covert bias measures, error trials were removed and RT outliers (values +/− 2 interquartile ranges from the median) were rescaled within each dataset using a Windsorizing procedure that has been found to improve the psychometrics of attentional bias indices (Price, Kuckertz, et al., 2015). The average RT for each of the four trial types was then calculated per-subject at each assessment point and used to calculate attentional bias scores. Two primary bias scores were used to dissociate biases in early engagement with threat from biases in disengagement from threat, as follows: EngagementBias = NeutralValid_meanRT − AngryValid_meanRT; Disengagement Bias = AngryInvalid_meanRT − NeutralInvalid_meanRT. A third bias score was calculated but not included in primary analyses, as it collapses information from both of the primary bias indices, but was included in reliability analyses, because it provides an index akin to that generated by other attentional bias tasks (such as the dot-probe task) which may be useful for comparison. This “overall bias” score collapsed engagement and disengagement biases together to generate a single index, as previously described (Mogg, Holmes, Garner, & Bradley, 2008): Overall Bias = (AngryInvalid_meanRT − AngryValid_meanRT) − (NeutralInvalid_meanRT − NeutralValid_meanRT).

For the overt bias measures, we quantified eyetracking indices of engagement and disengagement during dot-probe task performance as in prior research (Price, Kuckertz, et al., 2015; Price, Rosen, et al., 2015; Ricketts et al., 2018). Disengagement bias was calculated from a subset of trials (see “overt attention” indices above) as a difference score between two values: (mean latency to initiate an eye movement when disengaging from the threat word location) - mean latency to initiate an eye movement when disengaging from the neutral word location). By calculating this index based only on trials in which a fixation was registered (at the time of probe onset) to each word type, the index provides a pure measure of disengagement, fully independent of any engagement bias present earlier within the trial.

To be comprehensive in capturing multiple aspects of attention that have shown relevance in anxiety research, overt engagement bias was also quantified. This bias was calculated as the percentage of valid threat-neutral trials (i.e., trials with >25% fixation timepoints; accurate probe response) in which the initial fixation made by the participant corresponded to the threat word location. However, this attentional bias was hypothesized to be less explicitly relevant to our ABM procedures, given that the modification procedures systematically intervened on attention at a fixed point in time—specifically, 500ms following threat stimulus onset. By contrast, a typical eye fixation lasts 100–300ms (Henderson & Hollingworth, 1998), implying that one’s initial fixation may be less relevant to optimizing performance on the ABM training procedures compared with subsequent fixation patterns and/or covert attentional shifts.

All attentional bias scores (both overt and covert) were calculated such that greater (positive) scores indicated greater attentional bias towards threat (a vigilant pattern), while smaller (negative) scores indicated preferential attention towards neutral cues (an avoidant pattern).

Reliability.

To examine split-half reliability of the final bias score indices at each assessment point, each of these bias scores were also calculated, exactly as above, separately for each of the two blocks of the task at each assessment point. A Spearman-Brown correction was applied to the correlation value across the two blocks to generate a split-half reliability value. The p-value for positive correlation coefficients was reported as an additional indicator of reliability, while correlation values that were not significantly greater than zero were considered indicative of a lack of reliability.

Analytic Strategy

Based on the study design and aims, primary analyses focused on prediction of outcomes following active ABM. The primary attentional mechanisms of interest (i.e., those hypothesized to be most relevant to our ABM procedures) were those that were quantified 500ms (or more) into affective stimulus presentation: overt disengagement; and covert engagement and disengagement. A fourth index, overt engagement (initial fixation bias) was included in exploratory analyses as a comparison index. We tested for individual attentional patterns that predicted symptomatic outcomes across the full sample as well as tests for moderation by training group (i.e., interaction effects: predictor * ABM vs. sham), with follow-up tests to determine whether prediction was significant in the ABM group alone. Predictors of acute post-treatment scores were tested in SPSS with multivariate linear regression models (covarying pre-treatment scores). Longitudinal analyses were conducted with Hierarchical Linear Modeling (HLM) software (Raudenbush & Bryk, 2002). Linear mixed models, with subject as a random factor, were used to test for an effect of relevant predictors (e.g., training condition, attentional indices at baseline) on the slope of outcomes (attentional patterns or symptoms) over time. Time was coded such that the intercept reflects values of the outcome variable at the baseline assessment. Missing data, which ranged from 0% of the dataset (symptom measures at baseline) to a max of 27% (symptom measures at 1-year follow-up) depending on the variable/timepoint, was handled via restricted maximum likelihood using an intent-to-treat approach. Missingness was not significantly related to training group (ABM vs. control) for any variable at any timepoint according to Chi-square tests (p’s>.11).

Results

Split-Half Reliability of Attentional Bias Indices

The split-half reliability values obtained at each timepoint are displayed in Table 2. Adequate reliability was evident (≥.85; p’s <.001) at the baseline assessment for all covert indices, including the secondary “overall bias” index, which is notable given previous observations of substantially lower or non-existent reliability for the dot-probe task in both anxious and non-anxious samples. Overt indices were moderately reliable (i.e., ≥.5, p’s<.01) at baseline. Split-half reliability at both of the post-training timepoints (acute post-training and 1-month follow-up) was much lower for the majority of indices and often non-significant. Because inadequate reliability is particularly problematic for analysis of individual differences [while group-level patterns may be more robust to such influences; (Price, Kuckertz, et al., 2015)], individual differences analyses (i.e., prediction of symptom trajectories over time) utilized baseline attentional bias indices only.

Table 2.

Spearman-Brown split-half reliability of attention bias indices as a function of visit

	Baseline		Acute Post-treatment		1-month Follow-up
	Reliability	p	Reliability	p	Reliability	p
Covert Attention Bias Index
Engagement Index	.91	<.001	.61	.01	<0	n/a
Disengagement Index	.86	<.001	<0	n/a	.56	.03
Overall Attentional Bias Index	.90	<.001	.36	.15	.06	.46
Overt Attention Bias Index
Engagement Index	.52	.007	.57	.002	.68	<.001
Disengagement Index	.54	.005	<0	n/a	<0	n/a

Note: Bold text indicates significantly different from 0 (p<.05)

Effects of Training Group

Effects of training group on attentional bias indices over time.

Intent-to-treat mixed models regression with covert disengagement bias as the dependent measure, training group (ABM or control) as a between-subjects factor, subject as a random factor, and timepoint (baseline, acute post-treatment, 1-month follow-up) as a fixed within-subject factor revealed a significant effect of training condition on the slope of disengagement bias over time, indicating that the trajectory of scores over time differed across training conditions: t(68) = −2.25, p = .027, r_{effect size} = .26. As shown in Figure 2A, covert disengagement bias decreased steadily over the course of the assessment points in the ABM group, such that individuals in this group became quicker to disengage from threat over the course of the baseline, acute post-treatment, and 1-month follow-up [within-group effect of time: t(48) = −2.48, p = .017, r_{effect size} = .34], while disengagement bias did not change significantly over time in the control training group [t(20) = 1.03, p = .31, r_{effect size} = .22].

Figure 2A.

Changes in covert disengagement bias over time as a function of training group

Identical mixed models regression analyses with the other two primary attentional indices (overt disengagement, covert engagement) as the dependent measure revealed no significant effects of group on either the slope or the intercept of attentional indices over the three assessment points (p’s>.11).

Effects of training condition on symptoms.

As shown in Table 1, an overall pattern of symptom improvements over time, across both self-report and clinician-rated indices, was present in both the ABM and the control groups, at both acute and long-term follow-up. Though significant between-group differences were not found for any clinical measure, two secondary clinician-rated outcomes (remission of primary anxiety diagnosis; remission of all anxiety diagnoses) showed non-significant trends towards more favorable acute outcomes in the ABM group compared to the control group.

Intent-to-treat mixed models regression with symptom scales (MASQ General Distress, MASQ Anxious Arousal) as the dependent measure, training group (ABM or control) as a between-subjects factor, subject as a random factor, and timepoint (baseline, acute post-treatment, 1-month follow-up, 1-year follow-up) as a fixed within-subject factor revealed a significant effect of time for each measure, consistent with a linear decrease in symptoms over time: [MASQ General Distress: t(68) = −3.67, p < .001, r_{effect size} = .41; MASQ Anxious Arousal: t(68) = −2.50, p = .015, r_{effect size} = .29]. There were no significant effects of training condition on either the slope or the intercept of symptoms for either subscale, suggesting no significant group differences in either the magnitude or the trajectory of symptoms over time.

Prediction of Acute and Long-Term Symptom Trajectories Over Time

Prediction of acute ABM outcomes using baseline attentional bias scores.

Each of the three attentional bias scores (overt disengagement; covert engagement and disengagement biases) was tested in a separate multivariate regression model as a moderator of acute post-training symptoms, controlling for baseline symptom scores and including group (ABM vs. sham) and predictor * group interaction terms. For overt disengagement bias, there was a significant moderation effect (group * overt disengagement bias interaction) observed in the multivariate model (group * overt disengagement bias: F_2,47 = 8.27, p = .001, partial η² = .26), which was present individually for both clinical outcome measures [MASQ General Distress: F_1,48 = 4.49, p = .039, partial η² = .09, adjusted R²=.45; MASQ Anxious Arousal: F_1,48 = 16.82, p < .001, partial η² = .26, adjusted R²=.66]. As illustrated in Figure 2B, the nature of the prediction effect was that lower overt disengagement bias at baseline predicted more favorable outcomes (i.e., lower MASQ scores) following active ABM, but not following sham treatment. In the ABM group alone, overt disengagement bias was a significant multivariate predictor of symptoms at acute post-treatment (F_2,29 = 3.62, p = .04, partial η² = .20), showing similar significant prediction effects on both MASQ General Distress and MASQ Anxious Arousal [MASQ General Distress: F_1,30 = 6.32, p = .018, partial η² = .17, adjusted R²=.55; MASQ Anxious Arousal: F_1,30 = 4.20, p = .049, partial η² = .12, adjusted R²=.71].

Figure 2B.

Relationship between baseline overt disengagement bias and acute post-treatment symptoms (controlling for baseline symptoms). The pattern of the finding was virtually identical for the second anxiety outcome measure (MASQ Anxious Arousal).

For covert disengagement bias, there was also a significant multivariate interaction (F_2,51 = 12.14, p < .001, partial η² = .32), but the nature of the interaction was distinct in two ways. First, the effect was significant for MASQ General Distress (F_1,52 = 7.47, p = .009, partial η² = .13, adjusted R²=.45) but not for MASQ Anxious Arousal (F_1,52 = 0.51, p = .48, partial η² = .01). Second, there were no significant prediction effects in the ABM group alone (p’s>.68; partial η² ≤ .01), suggesting the interaction effect was driven primarily by the control sample.

No significant prediction or moderation effects of covert engagement bias were found on acute post-training outcomes (p’s > .27).

Prediction of long-term ABM outcomes using baseline attentional bias scores.

Each of the three attentional bias scores (overt disengagement; covert engagement and disengagement bias) were added individually as between-subjects factors in separate mixed models predicting MASQ scores over time within the full sample. Covert engagement bias was a significant predictor of the slope of MASQ General Distress over time [MASQ General Distress: t(58) = −2.15, p = .036, r_{effect size} = .27; Figure 2C]. The nature of the prediction effect was that larger covert engagement bias scores at baseline predicted lower (and more sharply decreasing) MASQ General Distress scores over time (see Figure 2C). This effect of covert engagement bias was not present in identical mixed models predicting MASQ Anxious Arousal (p’s>.18), but did extend to a secondary measure of worry (see “Specificity of Prediction Findings to MASQ Anxiety Measures”, Supplement).

Figure 2C.

Regression equation plots depicting predicted symptom trajectories over time for participants with low (avoidant), mid (unbiased), and high (vigilant) levels of covert engagement bias at baseline.

This prediction effect was robust in the ABM sample alone. Covert engagement bias was a significant predictor of the slope of MASQ General Distress over time [t(29) = −.41, p = .022, r_{effect size} = .41]. However, unlike in the acute post-training analyses, no evidence of moderation of this significant prediction effect by training group was found when adding group*bias interaction effects to the model within the full sample (p’s>.22).

Neither of the two disengagement indices at baseline (overt and covert) predicted or moderated long-term symptom trajectories (p’s>.12).

Covariate Sensitivity Analyses

All findings reported above were maintained in models that included participant gender, age, and presence vs. absence of a diagnosis of GAD as covariates. There were no significant interaction effects involving GAD diagnosis in any analysis.

Exploratory Attentional Index: Overt Engagement

To test whether any of the above findings extended to the earliest stages of (overt) attentional processing, overt engagement bias was tested in identical models to those described above. There were no significant effects of training group on overt engagement over time; and overt engagement bias at baseline did not predict or moderate clinical symptoms, either at acute or longer-term follow-up (p’s > .36).

Discussion

Attentional Bias Modification (ABM) was developed initially as a mechanistic probe, and subsequently as a mechanistic intervention, with the intention of translating a basic cognitive science finding with well-established clinical relevance into a novel treatment for clinical anxiety. Over a decade later, considerable debate remains regarding the mechanisms that are, in fact, addressed by this approach, as well as its clinical utility (Cristea et al., 2015; MacLeod & Grafton, 2016; Price, Wallace, et al., 2016). In the present study, a heterogeneous (transdiagnostic) clinically anxious sample was recruited with the goal of revealing a better mechanistic understanding of how ABM works, and for whom. Results (summarized in Table S2 in Supplement) suggest that overt and covert engagement and disengagement processes—all of which are conflated by reaction times on the dot-probe task, the most commonly used method to both assess and modify attentional patterns in ABM studies—have unique and dissociable roles to play in ABM outcomes within a heterogeneous clinically anxious sample. While covert disengagement biases may be the most robustly modifiable feature via this widely-used form of ABM, heterogeneity in both overt disengagement and covert engagement may be the more clinically relevant prognostic factors following ABM. Individual differences in overt disengagement were linked to acute post-training outcomes specifically within the ABM group; while covert engagement bias was linked to longer-term symptom trajectories, out to a 1-year follow-up, across both training conditions. Thus, while low overt disengagement bias might index a particularly good therapeutic fit for attentional bias training procedures within an acute precision medicine framework, in the longer term, a distinct attentional profile (covert engagement bias) may exert a more indelible influence on prognosis.

The widely-used form of ABM used here conflates overt and covert engagement and disengagement processes; nevertheless, consistent with hypotheses and at least three previous studies (Amir, Beard, Taylor, et al., 2009; Amir et al., 2008; Heeren et al., 2011), there was a specific effect of the training only on covert disengagement, such that individuals in the ABM group (but not the control group) showed improved ability to disengage quickly from threat cues over the course of the study. Notably, this improved disengagement bias continued to decrease from the acute post-training assessment to the 1-month follow-up assessment, in the absence of further intervention (Figure 2). This raises the interesting possibility that, once improved disengagement capacity has been set in motion via laboratory-based training, further consolidation of the skill occurs (e.g., in reaction to cues encountered in real-world contexts) in a beneficial feed-forward loop. However, in spite of these persistent attentional shifts, there were no corresponding group differences in the trajectory of clinical symptoms over time—both the ABM and the control group exhibited significant decreases in clinical measures in relation to baseline. Although two measures of clinical remission (remission of primary diagnosis, and remission of all anxiety diagnoses), assessed via clinician interview, exhibited a non-significant trend in favor of better acute outcomes within the active ABM group (Table 1), no significant impact of ABM (relative to the control group) was observed on any clinical measure. The present study was not optimized to detect group differences or to settle questions relevant to ABM’s overall clinical efficacy, as power was disproportionately allocated towards the active ABM group in order to maximize the ability to capture links between biobehavioral processes and clinical outcomes within the active group. Vastly increased (e.g., 10-fold) sample sizes have been previously compiled to more adequately address the question of group-level effects of ABM in transdiagnostic clinical anxiety (Price, Wallace, et al., 2016), and while some of these analyses have generally supported a clinical benefit of ABM relative to control, other meta-analytic findings have not (Cristea et al., 2015). Broadly, meta-analytic findings to date have highlighted heterogeneity in outcome and the need to understand individual difference variables which moderate and/or predict clinical effects.

Though all of the symptom scales we examined decreased over the course of the study, the decreasing slope of symptoms over time was impacted at the individual level by attentional patterns measured at baseline, in a fairly complex pattern. Specifically, with regard to immediate symptom reductions, individuals who were relatively slow to shift their eye gaze away from threat words when required to do so (an overt disengagement bias) appeared to be a relatively poor fit for ABM, exhibiting higher levels of residual anxiety symptoms. The direction of this effect (i.e., a more vigilant attentional pattern predicting poor outcome post-ABM) was opposite to hypotheses. In contrast, consistent with hypotheses drawn from both longitudinal/observational and ABM studies, individuals with a more avoidant covert engagement bias at baseline (i.e., lower engagement bias values) reported symptoms of general distress that were more persistent, declining less steeply over the long-term follow-up out to the 1-year assessment point. The contributions of these two opposing attentional patterns (greater overt disengagement bias; lesser covert engagement bias), while both predicting generally poorer clinical outcomes, appeared to be distinct from one another, explaining trajectories of symptoms in different sets of individuals and at distinct assessment points.

Several possible explanations for the observed clinical prediction effects are apparent. First, larger difficulties with overt disengagement from threat words may have been a marker of poor fit for ABM, interfering with benefiting efficiently from the intervention in the short-term. This could be because the ABM intervention appeared to successfully modify covert disengagement, specifically, but not overt disengagement—thus, greater overt disengagement difficulties at baseline might represent an unmodified, persistent risk factor for anxiety. An alternative way to describe the current finding is to say that individuals low on overt disengagement bias (i.e., those who started treatment with a relatively strong ability to disengage overt eye gaze from threat words) brought a strength to the table that ABM was able to capitalize on to efficiently reduce symptoms. Consistent with “strengths-based” therapeutic approaches (Padesky & Mooney, 2012), individuals may be best able to benefit when a therapeutic strategy mobilizes their existing skill set. Interestingly, the direction of this effect for overt disengagement was in opposition to previous acute post-ABM findings that have suggested— when conflating across many attentional components—that ABM is best-suited and most beneficial for anxious individuals who exhibit greater vigilant bias towards threat at baseline (Amir, 2011; Kuckertz et al., 2014; Price, Wallace, et al., 2016). Thus, the present study’s disentanglement of discrete attentional processes may have enabled a novel pattern to emerge.

Conversely, larger (covert) engagement bias at baseline was an additional marker of good outcome, at least with respect to longer-term overall distress and worry, though this pattern could not be shown to be specific to the ABM condition, and may reflect a more general marker of good prognosis among anxious patients over a 1-year period. In interpreting this finding, it is important to note that attentional bias indices are quantified on a continuum which ranges from a strong threat-orientation at one extreme to a strong pattern of avoidance of threat at the other. Thus, individuals “low” on threat-oriented engagement bias were in fact “high” on an avoidant style of orienting. Just as rigid behavioral avoidance of threatening situations is considered a key behavioral mechanism maintaining anxiety over time, previous prospective studies have documented the detrimental impact of avoidant attentional patterns on the course of internalizing symptoms, including longitudinal trajectories out to 2-years post-assessment (Gibb et al., 2009; Price, Rosen, et al., 2015), suggesting the detrimental impact of avoidant attention may be most apparent over longer follow-up intervals. Taken together, individuals within our heterogeneous clinically anxious sample who exhibited a highly specific attentional profile at baseline—less overt disengagement difficulty, combined with greater covert engagement bias—experienced the greatest number of positive clinical outcomes following ABM, across distinct assessment points and measures.

As expected, overt engagement (initial fixation) was the only attentional pattern we quantified that showed neither an impact of ABM nor any predictive relationship to symptoms in the current sample. This is consistent with a prior study in high trait-anxious individuals who received a single session of a similar ABM protocol (combined with brain stimulation of the prefrontal cortex) and exhibited corresponding changes in overt disengagement, but not initial fixation patterns (Heeren, Baeken, et al., 2015). Given that typical dot-probe-based ABM protocols (including ours) systematically intervene on attention at a fixed point in time—specifically, 500ms following threat stimulus onset—initial fixation patterns may be less temporally synced with the treatment target, and therefore less relevant to this specific form of training.

Clinical Implications

Overall, the observed patterns across analyses suggest the divergent roles that discrete components of attention may play during a mechanistic intervention. While covert disengagement from angry faces was successfully altered by ABM in the present sample, these patterns did not prove clinically impactful—as evidenced by both a) null relationships between covert disengagement bias and symptom trajectories over time and b) the lack of clinical efficacy of ABM within the current sample as a whole (in spite of successfully modified covert disengagement patterns). Conversely, two other indices were clinically relevant in predicting symptoms over time, but were not reliably altered by the current, widely-used form of ABM. This double dissociation implicates a need for novel procedures that can more reliably impact specific components of attention. More precisely, procedures to specifically decrease overt disengagement biases, and/or increase covert engagement with threat cues (thereby reducing avoidant patterns of engagement) represent relatively under-studied directions within ABM research that could hold the key to achieving a more profound and enduring clinical impact. Novel operant conditioning (feedback-based) approaches that deliver continuous online feedback regarding moment-to-moment overt attention patterns (Lazarov, Pine, & Bar-Haim, 2017; Price, Greven, Siegle, Koster, & De Raedt, 2016) and temporally specific neural signatures (McTeague et al., 2018; Woody et al., 2017) are promising avenues for further study, with the potential to dissect, pinpoint, and target specific attentional components in a way that dot-probe-based procedures cannot.

Design Features and Reliability

Previous ABM studies have been limited by lack of longer-term follow-up, imprecise measurement methods for attentional mechanisms, and a primary reliance on group-level patterns with less focus on potential attentional heterogeneity within anxious samples. In the present analysis, we aimed to address these limitations. Additional strengths of the current study include the use of eyetracking indices to provide a complementary data modality during the specific task that was used for attention re-training; inclusion of a distinct, untrained task and stimulus set for covert attentional bias assessment, providing a rigorous test of the generalized impact of training on core attentional mechanisms; the use of intent-to-treat mixed models analyses to improve sensitivity and reduce bias; and explicit quantification of the reliability of attentional bias assessments at each timepoint.

Interestingly, while the reliability of reaction time bias scores on the dot-probe task (the dominant paradigm in attentional bias research) has been repeatedly questioned in both anxious and non-anxious samples (Price, Kuckertz, et al., 2015; Rodebaugh et al., 2016; Schmukle, 2005), our analyses strongly supported adequate reliability for the current spatial cueing paradigm at the baseline visit—an essential precondition for individual difference analysis (Hajcak, Meyer, & Kotov, 2017). In fact, the reliability indices obtained at baseline quite dramatically exceeded best-case-scenario reliability indices derived from the dot-probe task across multiple samples and analysis approaches in our previous work (Price, Kuckertz, et al., 2015). These findings may stem from the combined influence of analysis decisions [e.g., using Windsorized RT distributions (Price, Kuckertz, et al., 2015)] and specific task features. For example, stimuli were presented on the right and left of the screen, negating the influence of top vs. bottom probes implicated in prior reliability analyses of the dot-probe task (Price, Kuckertz, et al., 2015); and facial threat cues were used [rather than the word stimuli used in previous studies where reliability was reported to be lower (Kinney et al., 2017; Price, Kuckertz, et al., 2015))], which are evolutionarily salient stimuli that provoke a robust bottom-up neural response (Whalen et al., 2001). Eyetracking methods have been previously suggested to provide a potentially less “noisy” index of attentional patterns relative to reaction times. In the present dataset, split-half reliability of overt indices was more modest than for the covert (reaction-time-based) indices, but was still improved relative to prior reports using dot-probe reaction times.

At the acute post-treatment and follow-up assessments, there was a noticeable decrease in split-half reliability—although reliability values remained generally in a favorable range relative to previous reports on both spatial cueing and dot-probe tasks (Price, Kuckertz, et al., 2015; Rodebaugh et al., 2016). It is possible that the repeated assessment procedures, compounded by the attention training sessions themselves, may have induced fatigue, decreased motivation, perturbed the attentional system, and/or revealed to the participants unanticipated task performance strategies that undermined reliable assessment. Given that psychometric properties remain a critical and widely-noted issue within attentional bias research, with ongoing debate focused on the meaning and implications of reaction time fluctuations that occur within the course of a single assessment (Kruijt, Field, & Fox, 2016; Rodebaugh et al., 2016), we believe it is essential to routinely conduct and report reliability findings when attentional bias assessments are used, in order to amass a corpus of findings that will guide the field as a whole towards stronger and more robust methodology.

Limitations

As noted above, split-half reliability of attentional bias indices was strong only at the baseline assessment. This precluded individual differences analyses focused on the latter two timepoints (e.g., relating pre-to-post-training trajectories in attentional patterns to symptomatic change), and may have adversely impacted power for group-level analyses of attention bias over time. The transdiagnostic sample may limit generalization of findings to specific anxiety disorders, particularly for diagnoses less well-represented in the recruited sample, as the majority of the participants in the current study carried a diagnosis of GAD. The small sample size in the control group, compounded by attrition over the 1-year follow-up, constrained power for some analyses, particularly for identifying baseline factors (moderators) that interacted with training condition to predict longer-term outcomes. As highlighted above, the covert engagement and disengagement indices provided by the spatial cueing task are subject to possible contamination by unintended factors, such as overt attentional shifts (if the instruction to keep gaze on the center fixation cross was ignored), general response time slowing in the presence of threat cues (Mogg et al., 2008), and downstream effects of engagement bias on disengagement processes (Grafton & MacLeod, 2014). However, contrary to these concerns, in the present sample, covert engagement and disengagement bias scores were unrelated and showed distinguishable patterns and influences over time. By contrast, general slowing and/or “leakage” from the engagement to the disengagement index would be expected to impact individual participant’s scores on both bias indices to a uniform degree, resulting in spurious convergence of findings across the two measures. The specific form of spatial cueing task utilized here, in which targets are uniform stimuli (here, stars) on every trial, is further limited by the fact that the correct response (target position on the right or left of the screen) can be fully determined by its absence on the opposite side of the screen, even if no (covert) attentional shift to the target location is made. While this is an important limitation of the present task design, the prevalent use of location-based response tasks within the attention bias literature suggests that this form of the task frequently yields findings that are comparable to optimal task conditions (i.e., probe discrimination) (e.g., Bar-Haim et al., 2007; Bar-Haim et al., 2011; Heeren, Baeken, et al., 2015). This is potentially consistent with our own finding that a significant disengagement bias was observable across the full sample at baseline (see “Analysis of Spatial Cueing Reaction Times at Baseline” in Supplement). Finally, it was necessary (due to the specific covert focus of the spatial cueing task, in which participants are instructed to keep eyes fixated at screen center) to quantify overt attentional indices collected during a distinct task, which introduced additional method variance (e.g., stimulus types). Further integrative research is needed to combine methodological strengths across tasks and measurement modalities.

Conclusions

In summary, overt and covert attentional processes of engagement and disengagement, when assessed reliably, may play unique and dissociable roles in the context of a mechanistic treatment strategy designed to target threat-relevant attention. Specifically, findings suggest that the best baseline marker of efficient clinical gains immediately following ABM was having a pre-existing propensity to rapidly disengage overt eye gaze from threat words—a pattern which may have allowed these individuals to better capitalize on the ABM technique in the short-term. In contrast, an avoidant pattern of engagement with angry faces, captured in covert (i.e., internal) attentional processes rather than reflected in overt eye movements, was a more generalized, non-specific marker of poor prognosis over the longer term (1-year post-treatment). These differing directions of effect highlight the complexity and nuance of attentional mechanisms—even when measured within a single sample receiving uniform study procedures. A third feature, difficulties with covert disengagement, was the only pattern that was robustly attenuated following the automated attentional modification procedures. Notably, this attentional shift was observed on an independent task that was distinct in several ways from the training itself (different task, different stimulus modality), and continued to mount even out to 1-month after the intervention. However, this isolated change did not translate into robust, generalized effects of ABM (relative to sham) on symptoms, suggesting the pattern was modifiable, but not sufficiently clinically relevant within the present heterogeneous group of patients.

Consistent with an “experimental therapeutics” framework recently promoted by mental health research funding agencies (Insel, 2014), these findings are maximally informative in spite of null results. They suggest ABM, in its most widely-used “dot-probe-style” form, was: a) capable of moving one mechanistic target (covert threat disengagement), but the target appeared not to be sufficiently clinically relevant in the current sample; b) incapable of robustly modifying additional targets (overt disengagement, covert engagement) that were more clinically relevant over both short- and longer-term follow-up. This apparent disconnect between attentional and clinical outcomes suggests specific future directions for ABM refinement. In particular, in intervention development, anxious individuals exhibiting an attentional profile characterized by overt disengagement difficulties and/or covert avoidance of threat might benefit from increased clinical attention and preventative efforts, including potential attempts to more specifically remediate these patterns. More broadly, precise and reliable characterization of attentional patterns at the individual patient level, using comprehensive assessments delivered repeatedly and longitudinally, holds promise for both refining mechanistic interventions and subsequently matching these interventions to the most appropriate patients within a transdiagnostic, process-oriented framework.