Pooled Patient-Level Meta-analysis of Children and Adults Completing a Computer-Based Anxiety Intervention Targeting Attentional Bias

Abstract

Computer-based approaches, such as Attention Bias Modification (ABM), could help improve access to care for anxiety. Study-level meta-analyses of ABM have produced conflicting findings and leave critical questions unresolved regarding ABM’s mechanisms of action and clinical potential. We pooled patient-level datasets from randomized controlled trials of children and adults with high-anxiety. Attentional bias (AB) towards threat, the target mechanism of ABM, was tested as an outcome and a mechanistic mediator and moderator of anxiety reduction. Diagnostic remission and Liebowitz Social Anxiety Scale (LSAS) were clinical outcomes available in enough studies to enable pooling. Per-patient data were obtained on at least one outcome from 13/16 eligible studies [86% of eligible participants; n=778]. Significant main effects of ABM on diagnostic remission (ABM—22.6%, control—10.8%; OR=2.57; p=.006) and AB (β*(95%CI)=−.63(−.83, −.42); p<.00005) were observed. There was no main effect of ABM on LSAS. However, moderator analyses suggested ABM was effective for patients who were younger (≤37y), trained in the lab, and/or assessed by clinicians. Under the same conditions where ABM was effective, mechanistic links between AB and anxiety reduction were supported. Under these specific circumstances, ABM reduces anxiety and acts through its target mechanism, supporting ABM’s theoretical basis while simultaneously suggesting clinical indications and refinements to improve its currently limited clinical potential.

Introduction

Anxiety disorders are the most prevalent class of mental health disorders (Kessler, Chiu, Demler, Merikangas, & Walters, 2005), affecting approximately 812 million individuals annually worldwide (Baxter, Scott, Vos, & Whiteford, 2013). Clinical and subclinical forms of anxiety are associated with significant medical morbidity, disability, and public health burden (Kessler, 2007), with an estimated direct societal cost of over $42 billion per year in the U.S. (Greenberg et al., 1999). Efficacious treatments for anxiety, including cognitive-behavioral therapies and pharmacotherapy, have been available for decades, yet disorder prevalence rates remain notably consistent, with only 12.7% of patients receiving minimally adequate treatment (Wang et al., 2005). This observation has led to a call for interventions that take advantage of technology to increase patient access, reduce cost, and minimize aversive consequences, through the use of automated, computer-based procedures (Mohr, Burns, Schueller, Clarke, & Klinkman, 2013).

Current first-line treatments for clinical anxiety exhibit a 50–70% response plateau (Ballenger, 2004; Barlow, Gorman, Shear, & Woods, 2000; Hofmann & Smits, 2008; McEvoy, 2007), with high rates of relapse, low rates of remission, and little evidence to suggest which patients may benefit from which treatment options. These patterns underscore the need to continue refining existing treatments and developing novel interventions. Barriers to progress towards a more efficient and effective approach to anxiety treatment may include inadequate focus on theory-driven, mechanistic predictors of treatment outcome; the use of heterogeneous treatment protocols that require expert administration and have multiple likely mechanisms; and the current diagnostic nosology of psychiatry, which may obscure critical, transdiagnostic dimensions of biobehavioral functioning (Insel et al., 2010). Many recent clinical research efforts have therefore increasingly focused on mechanistic treatments, which are designed to target a well-defined, unitary mechanism, often with transdiagnostic relevance. Mechanistic intervention studies carry the potential to inform both the theory of conditions like anxiety—providing an experimental test of causality—and the clinical practice of how to efficiently deliver the “right” treatment to the “right” patient.

One such mechanistic intervention, Attention Bias Modification (ABM)(MacLeod & Clarke, 2015), is designed to directly target a well-replicated, posited mechanism of anxiety: selective attention to threat. Anxious individuals, across a wide range of clinical and subclinical definitions, exhibit attentional preferences towards threatening information (henceforth, ‘attentional bias’; AB)(Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & van IJzendoorn, 2007). ABM seeks to modify this AB through repeated attention retraining exercises. If AB plays a causal role in promoting anxiety (e.g., by fostering exaggerated perceptions of danger), reduction of AB should lead to reduction of symptoms. This approach represents a departure from gold-standard behavioral treatments for anxiety (e.g., cognitive-behavioral therapy), as it relies solely on implicit training of a cognitive pattern as opposed to effortful changes to thoughts and behaviors, and might therefore be beneficial and/or appealing to a distinct subset of patients. After an initial demonstration (MacLeod, Rutherford, Campbell, Ebsworthy, & Holker, 2002) in healthy individuals that AB could be experimentally manipulated using automated procedures, producing downstream effects on mood reactivity, intervention studies in clinical populations followed. Initial findings in small samples suggested the potential to ameliorate clinical symptoms, and even reverse clinical diagnoses (Amir, Beard, Burns, & Bomyea, 2009; Amir, Beard, Taylor, et al., 2009). However, larger subsequent studies, many using home/Internet-based administration, did not consistently confirm these findings (Clarke, Notebaert, & Macleod, 2014).

Considerable controversy remains regarding whether further research and clinical resources should be devoted to ABM. Central questions relevant to such a ‘go/no-go’ decision have not yet been resolved through standard study-level meta-analytic approaches. In spite of at least seven published meta-analyses examining the effects of ABM on measures of anxiety (Beard, Sawyer, & Hofmann, 2012; Cristea, Kok, & Cuijpers, 2015; Hakamata et al., 2010; Hallion & Ruscio, 2011; Heeren, Mogoase, Philippot, & McNally, 2015; Linetzky, Pergamin-Hight, Pine, & Bar-Haim, 2015; Mogoaşe, David, & Koster, 2014), meta-analytic conclusions have differed dramatically, ranging from no reliable effect on anxiety (Cristea et al., 2015), to effect sizes rivaling those of first-line anxiety treatments (Hakamata et al., 2010), to modest effects under constrained conditions (e.g., when training is delivered in the laboratory rather than at home; when anxiety is assessed by clinician ratings rather than self-report) (Heeren, Mogoase, et al., 2015; Linetzky et al., 2015). Ongoing debate is particularly focused around two key issues with clinical, pragmatic, and theoretical relevance: 1) whether the effects of ABM are clinically meaningful, for at least a subset of anxious patients—a question with relevance to clinical decision-making, particularly if subsets of anxious patients likely to benefit can be defined according to concrete, readily obtainable indices; and 2) whether symptom improvements are contingent upon successful change in the target mechanism (AB). This latter question is fundamental to the theoretical basis and future of ABM research; if supported, it would suggest that the mechanistic target of ABM (AB reduction) is valid, producing concomitant symptom relief when it is successfully ameliorated, while the ability to reliably manipulate the target is what requires further refinement (MacLeod & Clarke, 2015). While a subset of individual studies have reported evidence of such mediational patterns (Amir, Beard, Burns, et al., 2009; Amir, Beard, Taylor, et al., 2009; Kuckertz et al., 2014), many either do not assess this question or report null effects (and even among studies reporting mediation, findings have been inconsistent across anxiety scales). This is unsurprising given the substantial power constraints for testing mediation in small samples. Notably, both mechanistic and individualized prognostic questions are quite difficult to address using a standard meta-analytic approach, in which the only data available are study-level means included in the original reports, omitting crucial individual differences within samples (i.e., individuals not well-represented by the mean of their study) and substantially reducing power.

One key participant-level individual difference that could moderate ABM outcomes is patient age. Broadly speaking, the ameliorating effects of ABM on clinical anxiety have transgressed traditional age group boundaries, with both pediatric (Eldar et al., 2012; Waters, Pittaway, Mogg, Bradley, & Pine, 2013) and adult (Amir, Beard, Burns, et al., 2009; Amir, Beard, Taylor, et al., 2009; Schmidt, Richey, Buckner, & Timpano, 2009) anxiety patients showing beneficial effects in at least a subset of studies. However, there are several reasons to believe that age may be an important predictor of the potential to benefit (or not) from ABM. Age is strongly linked to ‘fluid’ intelligence, executive functions (including attentional control), and related capacities to learn novel skills, all of which follow an asymmetrical inverted-U curve across the lifespan, peaking in the early 20’s (Park et al., 2002). To the extent that uptake of novel attentional patterns during ABM relies on these cognitive functions, ABM might produce better outcomes among individuals proximal in age to this cognitive peak. In addition, cohort effects may exist, such that individuals in younger generations, who were raised in a technology-driven era, may be better matched to an automated intervention that relies on the ability to engage effectively and learn implicitly from a computerized task. Finally, there could be longitudinal changes in the mechanisms that contribute to anxiety; for instance, attentional bias towards threat could be a prominent factor contributing to the initial onset and early maintenance of anxiety (MacLeod & Hagan, 1992; See, MacLeod, & Bridle, 2009), but could be gradually replaced over the course of the illness by compensatory or secondary mechanisms that then begin to maintain anxiety on their own—for example, habitual avoidance of threat (Mogg, Bradley, Miles, & Dixon, 2004) or low self-efficacy (Bandura, 1988). Previous study-level meta-analyses examining age as a moderator have produced conflicting findings, and specifically suggested that age did not moderate social anxiety outcomes (Heeren, Mogoase, et al., 2015; Mogoaşe et al., 2014). However, as noted above, age can only be expressed as a study/condition mean in such standard meta-analyses, substantially limiting power to adequately address this question, and interfering with the ability to more precisely determine the age range for which ABM is effective.

The ABM literature, including several previous standard meta-analyses, also suggests the potential importance of two ‘study-level’ moderators that vary from one study to the next, but not from one individual to the next within a single study. The first is training location—specifically, meta-analyses suggest ABM reliably improves anxiety when delivered in a laboratory setting, but not at home (e.g., over the Internet) (Heeren, Mogoase, et al., 2015; Linetzky et al., 2015; Mogoaşe et al., 2014). One possible interpretation is that the relatively controlled environment of the laboratory facilitates uptake or retention of new attentional patterns because of the absence of distracting environmental factors. Highly focused attention may be particularly important for ABM, given that the attentional biases being modified tend to be subtle, on the order of 10–30ms in anxious samples on average (Bar-Haim et al., 2007). A second possibility is that the act of coming into the laboratory activates a relevant ‘fear structure’—particularly for socially anxious patients—and that such activation is necessary in order to provoke an attentional bias that can then be modified and/or to promote transfer of learning to real-life stressful situations. This hypothesis received preliminary support in an uncontrolled experiment (Kuckertz et al., 2014). By asking patients with social anxiety to complete a brief self-directed exposure (e.g., making a phone call) immediately prior to each session of ABM completed at home, effect sizes akin to previous laboratory-based studies were obtained.

The second study-level moderator suggested by previous meta-analyses is the individual making anxiety ratings: clinician or patient (self-report). Beneficial effects of ABM on anxiety have been more reliable when clinician-rated measures are used as outcomes (Linetzky et al., 2015; Mogoaşe et al., 2014). Consistent with larger effect sizes observed in psychotherapy trials when clinician-rated outcome measures are used (Cuijpers, Li, Hofmann, & Andersson, 2010), this finding may reflect greater accuracy or sensitivity to clinically meaningful change when clinical judgment is brought to bear. Assessment of highly personal experiences within the context of a social interaction (with the clinician) may also bring to light a more accurate view of vulnerabilities–particularly when social anxiety is part of the clinical picture.

As noted above, resolving the question of whether symptom improvements are contingent upon successful change in the target mechanism (AB) is essential to the future of ABM research. Specifically, the mechanistic theory that drives ABM suggests a mediational model, in which the degree of attentional shift away from threat, measured objectively based on task performance, ought to be linked to the degree of anxiety relief reported following ABM. If such a link can be demonstrated, it would suggest that: (a) the beneficial effects reported in a subset of ABM studies to date are, in fact, driven by a causal role for attentional bias in anxiety, rather than by the spurious influences of non-theorized factors [e.g., ‘experimenter effects’ and demand characteristics (Cristea et al., 2015)]; and (b) through further refinements to ABM procedures that successfully increase its effects on attention, beneficial effects on anxiety should likewise improve. A related mechanistic question concerns participant ‘fit’ for what ABM has to offer. Since attentional bias towards threat is what ABM remediates, the approach may only benefit those patients who have a relatively strong attentional bias in need of remediation. Like mediation, this question relies on individual differences in attentional patterns and anxiety outcomes, and is difficult to address in small samples or with standard meta-analytic techniques; however, previous analyses pooling participant-level data across more than one study (all of which were conducted by the same group of investigators) have provided preliminary support for this hypothesized form of moderation (Amir, Taylor, & Donohue, 2011; Kuckertz et al., 2014).

The current study sought to address these unresolved questions through a pooled patient-level ‘mega-analysis’ approach utilizing raw, per-patient data from randomized controlled trials (RCTs) of ABM, administered to individuals with elevated levels of anxiety. While preserving the advantages of conventional meta-analysis as a means of aggregating evidence across numerous studies, patient-level ‘mega-analysis’ offers unique advantages (Riley, Lambert, & Abo Zaid, 2010), including: 1) an order-of-magnitude increase in data points analyzed on each variable (many per study rather than one summary measure per study), which substantially increases power, particularly for testing moderators and mediators, 2) standardization of analytic procedures across studies, 3) quantification of clinically relevant outcomes not reported in the original studies (e.g., response rates), and 4) a substantially improved ability to ask questions that rely on individual differences within each sample, including critical mechanistic questions (e.g., does change in AB mediate symptomatic outcome?), and questions relevant to clinical decision-making (e.g., is ABM more helpful within a particular age range? do baseline levels of AB moderate outcome, suggesting a better ‘fit’ for the intervention target?).

A subset of prior meta-analyses, particularly those reporting minimal or no efficacy, have focused broadly on the effects of heterogeneous ‘cognitive bias modification’ programs (e.g., targeting attention, interpretations, etc.) across both healthy and anxious samples (e.g., Cristea et al., 2015), and have incorporated acute post-training measures of anxiety following a single session of ABM, an index of whether ABM manipulates state anxiety. Restricting analysis to more clinically relevant targets, such as measures of enduring clinical symptoms and/or mood reactivity following stress (an index of emotional vulnerability), may speak more directly to ABM’s clinical potential (MacLeod & Clarke, 2015). In the present mega-analysis, we sought to enhance the clinical relevance and theoretical clarity of findings by focusing on ABM specifically, in anxious samples (according to clinical diagnosis or symptom scale distributions), using outcome measures that capture enduring symptom experiences in daily life (rather than state mood effects), and incorporating direct measurement of change in ABM’s target, AB.

In summary, we aimed to clarify the potential role of ABM procedures in the treatment of anxiety across the lifespan by: 1) characterizing the impact of ABM (vs. control groups) on continuous and dichotomous measures of anxiety, including clinically meaningful benchmarks not routinely reported in original studies (diagnostic remission and response rates); 2) identifying patient and study-level characteristics that moderate ABM’s effect on symptoms (e.g., age, baseline AB, training setting - home or laboratory), suggesting ways to improve response rates through intervention refinement and/or individualized patient prescriptions; and 3) testing a mediational model in which ABM’s effects on its mechanistic target, AB, predict degree of symptom improvement, providing critical information relevant to the theoretical validity of the ABM approach. These questions are uniquely well suited to a pooled patient-level data approach.

Methods

Study Identification and Selection

All PRISMA and MARS guidelines were followed. The meta-analysis protocol, including all inclusion/exclusion criteria, was registered prior to beginning the literature review at: http://www.crd.york.ac.uk/PROSPERO/ (CRD42015019558). The purpose of the PROSPERO registry is to increase transparency and safeguard against possible bias resulting from post hoc study selection or selective reporting. PubMed, PsychINFO, EMBASE, and the Cochrane Central Register of Controlled Trials (CENTRAL) were searched over the period 01/01/2002–07/01/2015 using search terms and synonyms representing ABM ("attention* bias modification,” "attention* *training," "ABM") and anxiety (“anxiety”/exp, anxi*). Published meta-analyses and reviews were checked for additional relevant studies. The first author assessed eligibility of all records using the following initial inclusion criteria. First, the study population was required to exhibit elevated (moderate-to-clinical) levels of anxiety in any age group. This was designed to be as inclusive as possible, while maintaining a focus on clinically relevant levels of anxiety according to a dimensional, rather than a categorical, conceptualization of clinical anxiety. Thus, studies recruiting individuals with a clinical anxiety disorder or scoring above distributional cut-points on a validated measure of anxiety were included. Findings were quite similar, but slightly stronger, using clinically diagnosed samples only (see Supplement, sensitivity analysis #4). Second, a stand-alone ABM intervention was required, defined as any automated procedure (i.e., any computer task) designed to directly alter attention towards threat. No specific training procedure (e.g., dot-probe) was required to be used, as long as the theoretical target of the procedure, reduction in attention towards threat, was uniform. ABM given as an adjunct to other treatments was excluded because of our focus on mechanistic hypotheses which are relevant to ABM specifically; adjunctive therapy was expected to result in at least some individuals’ anxiety improving via non-attentional mechanisms, reducing power. Third, an RCT design was required to minimize bias. Allowable control conditions included sham or inverse (towards threat) variants of the same computer task, wait-list, or treatment-as-usual. Finally, we required a method to assess ABM’s posited mechanism, AB. In order to measure AB using reasonably uniform procedures, a dot-probe task (described below) was required at the start and end of the intervention, either as a distinct pre/post-training assessment, or as a first and last ABM training session (see details below). Although many alternative procedures can be used to assess AB [with possible advantages over the dot-probe; e.g., (Price et al., 2015), the dot-probe task was selected because it was expected to have been used in a large proportion of studies, enabling pooling data across numerous studies in order to provide a definitive test of ABM’s mechanism. In validation of this assumption, only 2 studies were excluded for not including a dot-probe assessment (see Figure 1).

Figure 1.

PRISMA Flowchart

Applying the criteria above resulted in a sample of 30 potentially eligible studies. However, pooled meta-analysis requires that the same index be used across studies to quantify anxiety post-treatment. No validated method exists to put individual patient scores, obtained on non-identical scales, into the same ‘space,’ and the assumptions that would be required in order to standardize disparate scales (e.g., that all scales measure the same construct; that all studies had a similar distribution of scores across their full sample) were deemed statistically inappropriate. Therefore, the specific anxiety assessment measures reported in the methods sections of the resulting 30 eligible studies were tallied. To maximize available data while meeting the constraint that uniform measures be used across studies, anxiety outcome measures were selected as those most frequently reported across the 30 otherwise-eligible studies. Two outcomes emerged as most prevalent: 1) the Liebowitz Social Anxiety Scale (LSAS), a widely used, validated measure of social anxiety [in either clinician-rated or self-report format, which are identical questionnaires producing highly convergent scores (Fresco et al., 2001)], used in 13 studies; and 2) re-assessment of the principle or inclusionary baseline diagnosis using a validated structured clinical interview, used in 9 studies.^a Inclusion of studies with either LSAS or diagnostic status evaluation pre- and post-training allowed for one set of analyses focused on social anxiety, consistent with the prevalence of social anxiety studies within the ABM literature, and a second set of analyses in a transdiagnostic sample. The primary effect of the additional outcome scale requirements imposed by our pooled patient-level approach was to eliminate protocols involving a single session of training, where change in clinical scales would not be expected, and was not measured (n=8 studies), and a few extensions to novel or relatively understudied patient populations (pediatric anxiety where clinical outcome measures have been less uniform, specific phobia, PTSD, subclinical samples high on worry or obsessive-compulsive traits; n=6 studies). The final list of eligible studies was highly comparable to the lists of studies analyzed in recent standard meta-analyses with a similar focus on clinical anxiety (Heeren, Mogoase, et al., 2015; Linetzky et al., 2015), suggesting the additional criteria necessary for pooling data did not create a misrepresentation of the studies available in the extant literature. After finalization of inclusion criteria, a second rater assessed eligibility of a random subset (20%) of records and obtained 100% agreement.

Authors of eligible studies were invited via email to contribute data. Repeated attempts were made if no response was received until all authors had replied. The following data were requested per-participant, with authors asked to contribute all available variables: group assignment, age, pre- and post-training LSAS, pre- and post-training principle/inclusionary diagnostic status, and pre- and post-training AB data. For AB data, raw trial-by-trial reaction time and accuracy data were requested for each participant to allow uniform methods to be applied in calculating AB indices. In cases where trial-level data were not retained, pre-calculated AB scores per-participant were requested. The pooled dataset was compiled and maintained locally by the first author. The first and second authors independently designed and conducted all analyses.

Quality Assessment and Data Extraction

Study quality was assessed based on information provided in the published manuscripts using 5 relevant criteria from the Cochrane Collaborations’ risk of bias tool (The Cochrane Collaboration, 2011). Two raters independently assessed risk of bias (mean Kappa=.68, indicating ‘substantial’ reliability). Disagreements were resolved by consensus. As recommended (The Cochrane Collaboration, 2011), sensitivity analyses utilized risk-of-bias study stratification rather than summary scores. For study-level characteristics used in descriptive and moderator analyses, design features were extracted by one rater and verified by the first author.

AB Assessment and Modification

The dot-probe task (discussed further in inclusion criteria above) is a widely-used reaction time (RT) measure of AB (MacLeod, Mathews, & Tata, 1986). In short, two stimuli are presented simultaneously (one threat-related and one comparison—typically neutral or positive) for a specific duration (e.g., 500ms). Both items are then removed and a neutral ‘probe’ appears in one of the two locations, which requires a response (e.g., indicate via button press whether the letter ‘E’ or ‘F’ is shown). AB towards threat is inferred when an individual’s mean RT to “incongruent” trials, in which the probe replaces the non-threat item in the pair, is longer than the mean RT to “congruent” trials, in which the probe replaces the threat item.

All studies that ultimately contributed data used a training variant of the dot-probe as the ABM intervention (MacLeod et al., 2002). Although this was not explicitly required by inclusion/exclusion criteria, the literature review revealed only 3 studies meeting all other eligibility criteria that used alternate automated training procedures, which reflects the prevalence of the dot-probe ABM approach in the literature to date. Of these 3 studies using alternate training procedures, 2 were excluded (see Figure 1) because they did not include a dot-probe assessment measure (prohibiting pooling with other studies to test our primary hypotheses regarding ABM’s target mechanism), and one was invited to participate but ultimately contributed no data. In the ABM variant of the task, the probe replaces the less threatening item in the stimulus pair with greater likelihood than the threat-related item, systematically drawing attention away from threat. Sessions are typically ≤20min with minimal-to-no contact with personnel. All studies included a sham control group completing an identical task without systematic contingency for the probe location (equal likelihood in threat or non-threat location). A subset of studies included other control groups (inverse training drawing attention towards threat: 3 studies; training using neutral-neutral stimulus pairs: 1 study). Procedural variables used in both assessment and training differed across included studies (Table 1).

Table 1.

Description of included studies

Study	Samplea	Age	ABM Interventionb	Control condition(s)b	Training Protocol	AB Assessmentb	Training Setting	Outcome variables received	Completer Rate
(Amir, Beard, Burns, et al., 2009)	Generalized Anxiety Disorder (n=29)	mean=25.5, SD=7.5, range=18–56	Threatening/neutral word pairs, probe replaces neutral; 34% neutral/neutral pairs	Sham training: Threatening/neutral word pairs, probe replaces threatening or neutral (50/50); 34% neutral/neutral pairs	2×/week for 4 weeks	Same as sham condition, with alternate word set	Laboratory	Diagnosis, raw AB data (assessment)	100%
(Amir, Beard, Taylor, et al., 2009)	Social Anxiety Disorder (n=48)	mean=29.4, SD=10.8, range=19–62	Disgust/neutral face pairs, probe replaces neutral; 20% neutral/neutral pairs	Sham training: Disgust/neutral face pairs, probe replaces disgust or neutral (50/50); 20% neutral/neutral pairs	2×/week for 4 weeks	First/last training session	Laboratory	LSAS-CR, Diagnosis, raw AB data (training)	92%
(Boettcher, Berger, & Renneberg, 2012)	Social Anxiety Disorder (n=68)	mean=37.9, SD=11.2, range=18–69	Disgust/neutral face pairs, probe replaces neutral; 20% neutral/neutral pairs	Sham training: Disgust/neutral face pairs, probe replaces disgust or neutral (50/50); 20% neutral/neutral pairs	2×/week for 4 weeks	First/last training session (unavailable for analysis)	Home	LSAS-SR, Diagnosis	94%
(Boettcher et al., 2013)	Social Anxiety Disorder (n=129)	mean=38.3, SD=12.3, range=18–65	Two training conditions (collapsed for present analyses): words alone or words+pictures; 33.3% neutral/positive, 33.3% neutral/negative and 33.3% positive/negative stimulus pairs, probe replaces less negative/more positive item in each pair. Stimuli presented for 1000ms (first half of trials each session), 500ms (second half of trials each session)	Two sham conditions (collapsed for present analyses): words alone or words + pictures; stimulus pairs and presentation times as described in ABM intervention column, probe replaces either more negative or more positive item in each pair (50/50) Two inverse conditions (collapsed for present analyses): words alone or words + pictures; stimulus pairs and presentation times as described in ABM intervention column, probe replaces more negative/less positive item in each pair	1×/day for 2 weeks	Same as the sham control condition with words and pictures; 500ms stimuli for all trials	Home	LSAS-SR, raw AB data (assessment)	96%
(Boettcher, Hasselrot, Sund, Andersson, & Carlbring, 2014)	Social Anxiety Disorder (n=133)	mean=33.4, SD=10.4, range=18–59	See footnotec	Sham training: 33.3% neutral/positive, 33.3% neutral/negative and 33.3% positive/negative stimulus pairs (words and pictures), probe replaces either more negative or more positive item in each pair (50/50) Inverse training: stimulus pairs as described above, probe replaces more negative/less positive item in each pair. Stimuli presented for 1000 ms (first half of trials each session), 500 ms second half of trials each session)	1×/day for 2 weeks	Same as sham condition; 500ms stimuli for all trials	Home	LSAS-SR, pre- calculated AB scores (assessment)	95.5%
(Britton et al., 2015)	High Social Anxiety (n=53)	mean=22, SD=3.1, range=18–30	Angry/neutral face pairs, probe replaces neutral; 20% neutral/neutral pairs	Sham training: Angry/neutral face pairs, probe replaces angry or neutral (50/50); 20% neutral/neutral pairs	1 training session in fMRI scanner + 2×/week for 4 weeks	Same as sham condition	Laboratory	LSAS-CR, Diagnosis, raw AB data (assessment)	83%
(Bunnell, Beidel, & Mesa, 2013)	Social Anxiety Disorder (n=32)	mean=24.3, SD=7.5, range=18–45	Disgust/neutral face pairs, probe replaces neutral; 20% neutral/neutral pairs	Sham training: Disgust/neutral face pairs, probe replaces disgust or neutral (50/50); 20% neutral/neutral pairs	2×/week for 4 weeks	First/last training session (unavailable for analysis)	Laboratory	LSAS-SR, Diagnosis	97%
(Carlbring et al., 2012)	Social Anxiety Disorder (n=79)	mean=36.5, SD=12.7, range=18–73	Disgust/neutral face pairs, probe replaces neutral; 20% neutral/neutral pairs	Sham training: Disgust/neutral face pairs, probe replaces disgust or neutral (50/50); 20% neutral/neutral pairs	2×/week for 4 weeks	First/last training session	Home	LSAS-SR, Diagnosis, raw AB data (training)	96%
(Eldar et al., 2012)	Mixed pediatric anxiety disorders (n=40)	mean=9.8, SD=1.9, range=7–15	Angry/neutral face pairs, probe replaces neutral	Sham training 1: Angry/neutral face pairs, probe replaces angry or neutral (50/50) Sham training 2: Neutral/neutral face pair probe replaces neutral	1×/week for 4 weeks	Similar to sham training 1, but with additional (untrained) stimuli also included	Laboratory	Diagnosis, raw AB data (assessment)	100%
(Fang, Sawyer, Aderka, & Hofmann, 2013)	Social Anxiety Disorder (n=32)	mean=25.1, SD=9.1, range=18–60	Disgust/neutral face pairs, probe replaces neutral; 20% neutral/neutral pairs	Sham training: Disgust/neutral face pairs, probe replaces disgust or neutral (50/50); 20% neutral/neutral pairs	2×/week for 4 weeks	First/last training session (unavailable for analysis)	Laboratory	LSAS-CR	97%
(Heeren, Reese, McNally, & Philippot, 2012)	Social Anxiety Disorder (n=60)	mean=21.9, SD=3.1, range=18–35	Angry/happy face pairs, probe replaces happy on 80% of trials	Sham training: Angry/happy face pairs with opposing training during alternating blocks—probe replaces happy (2/4 blocks) or angry (2/4 blocks) on 80% of trials within-block Inverse training: Angry/happy face pairs, probe replaces angry on 80% of trials	1×/day for 4 days	Angry/happy face pairs, probe replaces angry or happy (50/50)	Laboratory	LSAS-SR, pre- calculated AB scores (assessment)	95%
(Maoz, Abend, Fox, Pine, & Bar-Haim, 2013)	High Social Anxiety (n=51)	mean=22.7, SD=1.7, range=19–27	Subliminal (17ms) disgust/neutral face pairs, probe replaces neutral; 20% neutral/neutral pairs	Sham training: Subliminal (17ms) disgust/neutral face pairs, probe replaces threatening or neutral (50/50); 20% neutral/neutral pairs	2×/week for 2 weeks	Same as sham condition	Laboratory	LSAS-SR, raw AB data (assessment)	100%
(McNally, Enock, Tsai, & Tousian, 2013)	High Social Anxiety (n=90)	mean=36.5, SD=13.6, range=18–65	Disgust/joy face pairs, probe replaces joy	Sham training: Disgust/joy face pairs, probe replaces disgust or joy (50/50) Inverse training: Disgust/joy face pairs, probe replaces disgust	1–2× week for 4 weeks	Same as sham condition	Laboratory	LSAS-SR, raw AB data (assessment)	63%

Note: ABM=Attention Bias Modification; AB=Attention Bias; LSAS=Liebowitz Social Anxiety Scale; SR=self-report; CR=clinician-rated. All studies contributing data used a dot-probe task for ABM intervention.

^aN=number randomized.

^bStimulus presentation times for training and assessment were 500ms unless otherwise noted.

^cStudy did not administer standard ABM away from threat. Data from sham and inverse training groups was used to aid power and precision of estimates for control conditions. No findings were altered excluding this study (see Supplement, Sensitivity Analysis #3).

When raw trial-by-trial data were provided, outlier handling methods previously shown to improve test-retest reliability were applied (Price et al., 2015)—specifically, after removing trials with incorrect responses, reaction times were Winsorized across the study distribution to rescale outliers (values lying beyond the 25^th and 75^th percentiles by +/−1.5 interquartile ranges) to the nearest valid value. Per-participant bias scores were then calculated as mean RT to incongruent – mean RT to congruent trials (a more reliable (Price et al., 2015) and widely-used index than other possible contrasts, e.g. incongruent – neutral-neutral stimuli). In cases where only first and last training sessions were available (no separate pre-post dot-probe assessment; 2 studies), bias scores were calculated only for the sham group (as the training group completed no “congruent” trials) and used to improve power within this group.

Statistical Analysis

Analyses were conducted comparing ABM vs. all control conditions (collapsed into one group); results were similar when comparing ABM to sham specifically (see Supplement, sensitivy analysis #2). Outcomes were: remission of the principle/inclusionary diagnosis (“diagnostic remission”), post-training LSAS score (covarying pre-training LSAS in all analyses), and post-training AB (covarying pre-training AB). LSAS response (≥30% decrease from baseline) rates were calculated to provide further descriptive information on clinical effects. One-step individual patient data analyses (Riley et al., 2008) were completed using linear mixed effects regression models for post-training LSAS and AB and generalized linear mixed effects models with a logit link for diagnostic remission. All models included a random study effect to control for unobserved study heterogeneity; patient-level data was considered level 1 and study-level data was considered level 2. For interpretability, continuous variables were standardized across the full dataset and dichotomous variables were coded as .5 and −.5. We report standardized coefficients (β*) and odds ratios (OR) with 95% profile likelihood confidence intervals. Analyses were performed using R version 3.1.

Completer datasets were used for several reasons: a) completion rates were high—across the analyzed studies, 93% of randomized patients completed their assigned intervention and provided post-treatment data, suggesting low risk of bias due to the relatively small proportion of patients (7%) missing from completer datasets; b) completion rates did not differ by condition (p=.85), further suggesting low risk of bias (The Cochrane Collaboration, 2011) ; c) requesting previously published datasets was expected to increase participation; in the vast majority of publications, the published findings consisted of completer analyses.

Moderators

To assess the conditions under which ABM is most effective, we selected the following moderators a priori. Age, a participant-level variable expected to be available in all studies, was selected based on the observation that many of the strongest published ABM effects were observed in younger (e.g., pediatric and undergraduate) samples. Two study-level moderators were selected based on previous meta-analytic findings (Heeren, Mogoase, et al., 2015; Linetzky et al., 2015): training setting (home vs. lab) and clinician- vs. self-report assessment (relevant for LSAS only). Subsequently, we examined whether each of the tested moderators indicated different treatments depending on the value of that moderator, as indicated by a statistically significant moderator * treatment interaction effect in predicting outcome. We used the predicted ABM and control values to identify either the range of scores (for continuous moderators) or the subgroup(s) (for categorical moderators) where ABM was indicated to produce a preferable outcome relative to control. This approach allows for identification of moderator-defined subgroups where treatment effects are present.

Intervention Mechanisms

To assess theorized mechanistic links between AB and symptoms, we conducted additional moderator and mediator analyses. Baseline AB was tested as a moderator to determine whether the effect of ABM relative to control was greatest among those with high AB. To directly test ABM’s theorized mechanism (improvement of symptoms through improvement in AB), we performed statistical mediator analyses to determine whether ABM (and not control) reduced anxiety through reduction in AB. We first tested for mediation in the full sample. Subsequently, we followed published recommendations by (Kraemer, 2010), "…once a moderator of [treatment] on [outcome] is identified, the search for mediators should focus separately on moderator-defined subgroups." We therefore tested for mediation within the moderator-defined subgroups (using the 3 moderators described above) where a reliable treatment effect existed to be mediated. In all mediation analyses (full sample and subgroups), we tested whether change in AB mediated the relationship between treatment and change in LSAS using the MacArthur method (Kraemer, 2010), which requires a main effect of treatment on the mediator, and an effect of mediator or mediator*treatment (interaction) on the outcome. LSAS difference scores were used as the outcome for all mediator analyses because of low numbers of cases with both diagnostic and AB measures (n=27 ABM; n=90 control) and for consistency with previous mediation studies (Amir, Beard, Taylor, et al., 2009; Kuckertz et al., 2014).

Results

Study Selection

See Figure 1 for PRISMA flowchart. At least one usable outcome variable was obtained from 83% of eligible studies (13/16; total n=778; 86% of all possible eligible participants). This included AB data from 10 studies (n=523; 1 pediatric sample), LSAS data from 11 studies (n=693; n=456 with AB data; all adult samples), and diagnostic information from 6 studies (n=281). AB data were obtained in raw trial-by-trial form from 8 studies, pre-calculated form from 2 studies, and was unavailable from 3 of the participating studies because it was not retained.

The final pooled sample for the transdiagnostic outcome (diagnostic remission) was skewed towards adults with social anxiety (76%), reflecting the preponderance of adult social anxiety studies within the clinical ABM literature. However, because 2 studies meeting all a priori inclusion criteria contributed data, they were retained in primary analyses of diagnostic remission and AB (but did not contribute data to any analysis involving LSAS). The effect of this decision was assessed in sensitivity analyses (see Supplement, sensitivity analysis #3).

Among studies that did contribute data, 4 (31%) reported significant effects of ABM (over control) on anxiety outcomes and 9 reported null effects. Among studies that did not participate (3/16), two corresponding authors indicated that data were not retained and one was unable to access the data while on extended leave. Two (67%) of the non-participating studies reported significant effects of ABM on anxiety outcomes, and one reported null effects. Due to the high participation rate, risk of bias from selective study participation was judged to be low; however, clinical effects could slightly underestimate what would have been obtained from the full sample of 16 eligible studies. Table 1 presents descriptive characteristics of participating studies. The Supplement presents quality assessments for each study (e.g., blinding of assessor) and assessments of publication bias within the sample of included studies. Risk of bias according to formal study quality assessments tended to be low, and evidence of publication bias was not found for any outcome.

Anxiety Measures

Diagnosis

ABM was associated with a 2-fold increased likelihood of diagnostic remission post-training [ABM—22.6% (n=30), control—10.8% (n=16); OR(95% CI)=2.57(1.31,5.22); p=.006]. This effect was moderated by training setting [OR(95% CI)=0.13(.03, .54); p=.005] but not age (p=.366). For participants with a lab training setting (n=144), ABM was associated with an increased likelihood of diagnostic remission [ABM—34.85% (n=23); Control—8.97% (n=7); OR(95% CI)=6.16(2.44,17.40); p=.0002]. With home training (n=137), treatment was unrelated to diagnostic remission [ABM—10.45% (n=7); Control—12.86% (n=9); OR(95% CI)=0.79 (.26,2.26); p=.653]. In sensitivity analyses, the effect of ABM on diagnostic remission was not moderated when comparing social anxiety disorder (SAD) patients to all other diagnoses (p=.20). However, when using a smaller sample restricted to individuals with SAD, effects on diagnostic remission remained robust only for ABM performed in the lab (see Supplement, sensitivity analysis #5).

LSAS

There was no significant main effect of ABM vs. control on post-training LSAS [β*(95% CI)=−.05(−.16, .06); p=.392; ABM—29.2% responders; Control—23.8% responders]. However, all three moderators (excluding baseline AB, which is examined below) were significant [age*treatment β*(95%CI)=.12(.01, .23); p=.041; setting*treatmenty β*(95%CI)=.31(.08, .54); p=.009; rater*treatmenty β*(95%CI)= .38(.08, .67); p=.012]. A beneficial effect of ABM over control on post-training LSAS was observed in patients who were: younger (age≤37y), but not older; trained in the laboratory, but not at home; and/or had clinician-rated LSAS, but not self-report (statistical details and response rates in Figure 2A–C). All individuals with clinician-rated LSAS also had lab training; further decomposition suggested effects were apparent only among lab-trained, clinician-rated participants (see Supplement, “Decomposing Moderators”).

Figure 2.

Moderators of the effect of active ABM vs. control training on LSAS scores. Panels depict: A) moderation by age; B) moderation by training setting; C) moderation by rater; D) within participants trained in the lab setting only, moderation by baseline AB. Regression prediction lines based on models predicting LSAS post-training, controlling for baseline LSAS, with a random effect for study. Response rates based on reduction of at least 30% of baseline LSAS score. Response rates that do not change in tandem with regression lines are due to predicted post-training LSAS scores that differ, with no corresponding change in the observed number of patients who cross the dichotomized 30% threshold.

Attentional Bias

There was a significant main effect of ABM vs. control on post-training AB (controlling for pre-training AB), suggesting ABM was associated with decreased AB to threat post-training [β*(95%CI)=−.63(−.83,−.42); p <.00005]. Neither age [β*(95%CI)=.12(−.06, .31); p=.181] nor training setting [β*(95%CI)=−.26(−.77, .24); p=.304] moderated this effect.

Relationships Between AB and Anxiety

Pre-training, greater AB was associated with greater LSAS [β*(95%CI)=.09(.01, .17); p=.026]. In the full sample, post-training LSAS was not moderated or predicted by baseline AB, nor mediated by AB change (p’s>.27). Following (Kraemer, 2010), having identified moderator-defined subgroups in which ABM was effective on LSAS, change in AB was tested as a mediator of this effect. Mediation was supported in the younger subgroup [age<37, n=319; treatment→AB: β*(95%CI)=−.50(−.74,−.26), p<.001; treatment*AB→LSAS change: β*(95%CI)=.32(.04, .60), p=.023] and as a non-significant trend in the laboratory-trained subgroup: [n=216; treatment→AB: β*(95%CI)=−.40(−.65,−.16), p=.002; treatment*AB→LSAS change: β*(95%CI)=.28(−.05, .62), p=.096]. In both cases, greater reduction in AB following ABM (but not control) was associated with greater LSAS reduction. A similar interaction effect was observed among clinician-rated patients, but full mediation criteria were not met due to a non-significant effect of training on AB in this small subsample [n=51; treatment→AB: β*(95%CI)=−.01(−.42, .80), p=.95; treatment* AB→LSAS change: β*(95%CI)=.94(.19,1.70), p=.015).

Exploratory Sub-group Moderator Analyses

Because both the theoretical target of ABM and previous findings (Amir et al., 2011; Kuckertz et al., 2014) suggest high baseline AB should predict better response to ABM (but not control), exploratory analyses tested baseline AB as a moderator under optimal (efficacious) training conditions. Among laboratory-trained [AB-Pre*treatment β*(95%CI)=−.29(−.54,−.03), p=.028] and clinician-rated [AB-Pre*treatment β*(95%CI)=−.58(−1.16,−.007), p=.048] participants, baseline AB moderated LSAS, with higher baseline AB predicting better outcome in ABM but not control (see details in Figure 2D; identical pattern of effect observed in clinician-rated dataset). Baseline AB did not similarly moderate outcome among the younger subgroup (p=.307).

Sensitivity Analyses

Sensitivity analyses (e.g., effects of study quality, ABM vs. sham training, clinically diagnosed vs. non-diagnosed samples, and other study characteristics) suggested that primary findings were quite robust and became larger and more significant in the majority of cases (e.g., after removing studies with unclear risk of bias; after removing non-clinically diagnosed samples; see Supplement, sensitivity analyses #1 & #4). In other cases (e.g., comparing ABM to sham training specifically; sensitivity analysis #2 in Supplement), a subset of moderation effects dropped to a trend level in the smaller datasets, but the pattern of findings remained consistent.

Discussion

ABM is a low-cost, fully automated, non-invasive, brief (typically <30min/session delivered over ≤8 sessions), computer-based intervention for emotional disorders (most prominently, anxiety) that has generated considerable attention from researchers and clinicians over the past decade. We investigated symptom-level and attentional mechanistic effects of ABM, moderators of outcome, and evaluated hypothesized intervention mechanisms by compiling a pooled sample of 778 anxious participants, a roughly 11-fold increase over the mean sample size of individual ABM studies reviewed in recent meta-analyses (Heeren, Mogoase, et al., 2015; Linetzky et al., 2015). In a diagnostically heterogeneous sample (76% SAD, 13% GAD, 11% other anxiety disorder), ABM had a modest but significant effect on diagnostic status post-intervention, with 22.6% of patients achieving diagnostic remission compared to 10.8% receiving control training. Diagnostic remission increased to 34.85% when ABM was delivered in the laboratory, as compared to 8.97% for control, corresponding to an odds ratio greater than 6. No main effect of ABM over control groups was found on a continuous measure of social anxiety (29% vs. 24% response rate for ABM and control, respectively), suggesting there are substantial constraints on ABM’s efficacy in general, when collapsing across heterogeneous study conditions. However, a priori moderators were significant, with ABM producing greater reductions in symptoms (relative to control) among patients who were younger (<37y), who received training in the laboratory, and who were assessed by a clinician. Although mediation was not present in the full sample, it was present in moderator-indicated subgroups where ABM was shown to be particularly effective (most robustly, in younger patients), thus indicating mechanistic ties between symptom reduction and the explicit target of ABM, AB. Thus, our patient-level meta-analysis approach provides new insight on critical questions regarding the active mechanisms of ABM that have persisted even after multiple existing standard meta-analyses. Our approach also allowed for novel quantification of effects on intuitive, clinically relevant indices of outcome (diagnostic remission and response rates), and identification of patient-level moderators and empirical cut-points that have direct relevance to clinical decision making (e.g., age, elevated AB). Collectively, these findings could help to explain mixed results in the ABM literature, including mixed meta-analytic results, while simultaneously suggesting refinements at both the procedural and patient personalization levels.

Findings suggest several new insights and avenues for research and clinical translation. First, mediation findings, which were observed within subgroups where ABM was effective, provide a previously unattainable level of validation for ABM’s target mechanism, and suggest that innovations to improve the reliability and/or magnitude of ABM’s attentional effects should improve clinical effects. Notably, however, the pattern of subgroup-specific mediation (rather than mediation across the full sample) suggests a nuanced picture, which would not have emerged without the pooled patient-level approach to study individual differences in mechanism and outcome within the context of a meta-analytic dataset providing adequate variability in key moderators. Specifically, our findings suggest that both participant-level change in AB and favorable contextual factors (e.g., younger participants, trained in the lab) must converge in order for ABM to exert its maximum impact. In other words, favorable conditions must exist in order for ABM to promote translation of AB change into symptom change. In the absence of these conditions (e.g., older or home-trained participants), ABM’s effect on attentional patterns, which remained robust, failed to generalize to observable symptom improvement. A critical future direction for ABM research is therefore to develop novel and/or adjunctive techniques that foster such translation into clinical benefit for key groups identified here, as simply moving the target mechanism may be insufficient to relieve symptoms effectively across all age groups and settings.

Though not specifically required in our mega-analysis, all studies ultimately contributing data utilized a dot-probe training paradigm for ABM, which is increasingly viewed as suboptimal for robust attentional modulation (MacLeod & Clarke, 2015). As preliminary studies of alternative procedures to manipulate AB now begin to emerge (Notebaert, Clarke, Grafton, & MacLeod, 2015; Price, Greven, Siegle, Koster, & De Raedt, 2016; Urech, Krieger, Chesham, Mast, & Berger, 2015; Waters et al., 2015), our findings define key constraints on the dot-probe’s efficacy, suggesting useful benchmarks that novel techniques should aim to surpass (e.g., increasing efficacy and response rates, particularly in individuals over 37 and those trained outside the lab). Such efforts will be particularly informative if head-to-head comparisons with the dot-probe are made, and/or if studies retain a few transdiagnostic assessments common in the extant ABM literature, such as diagnostic remission and a dot-probe assessment task to quantify AB mechanism change, allowing direct comparison of effects in future pooled patient-level analyses.

Second, findings on both categorical (diagnostic remission) and continuous (LSAS) measures were consistent in suggesting that ABM was reliably efficacious on clinician-rated but not self-report indices, as reported previously for ABM (Linetzky et al., 2015) and consistent with larger clinician- than self-report effects in RCTs of psychotherapy (Cuijpers et al., 2010). One possible explanation is that the incorporation of clinical judgment and greater standardization of the assessment context make clinician ratings more accurate and sensitive than self-report, producing scores that are more reflexive when clinically meaningful change has occurred, and more stable when it has not. An alternative possibility is inadequate blinding of clinicians to treatment condition. However, this latter interpretation is contradicted by two factors in particular. First, quality ratings suggested risk of bias from non-blinded outcome assessment was low in 85% of studies, and findings became stronger after excluding studies with unclear risk of bias (Supplement: Table S1 & sensitivity analysis #1). Second, moderating and mediating effects were also observed for ABM’s posited mechanism, AB. As it is highly improbable that raters were aware of patients’ AB patterns obtained via laboratory tasks, these findings stand independent of this potential source of bias and support the accuracy of the anxiety assessments.

Consistent with previous study-level meta-analyses (Heeren, Mogoase, et al., 2015; Linetzky et al., 2015), ABM had greater beneficial effects on both clinician-rated diagnostic remission and LSAS when training was delivered in the laboratory. Although the apparent benefit of administering ABM in a controlled (i.e., non-home-based) setting places important constraints on dissemination, preliminary data suggest a variety of controlled settings [e.g., primary care (Beard, Weisberg, & Amir, 2011)] and/or adjunctive techniques [e.g., self-exposure to fear triggers (Kuckertz et al., 2014); using virtual reality devices to isolate the patient from the immediate environment (Urech et al., 2015)] could ultimately prove sufficient.

Finally, our pooled data approach also allowed us to identify a patient-level demographic moderator—age. Previous study-level meta-analyses examining age as a moderator (expressed as a study/condition mean) have produced conflicting findings and suggested that age did not moderate social anxiety outcomes (Heeren, Mogoase, et al., 2015; Mogoaşe et al., 2014). Here, utilizing individual differences on a readily obtainable baseline characteristic, we found that socially anxious young adults age 37 or younger benefited from standard ABM training attention away from threat, while participants above this age did not—a finding with direct implications for clinical decision-making. No such moderation effect was observed for diagnostic remission, which could be due to decreased power when testing moderation of a dichotomous outcome and/or the inclusion of a wider, younger age range in this analysis (extending to pediatric anxiety), where chronological age may have alternate (e.g., non-linear) effects. Indeed, mirroring the inverted-U shape that many relevant cognitive functions follow across the lifespan, a recent report in a pediatric anxiety sample found a reversed relationship in youth, where older children benefited more from ABM than younger children (Pergamin-Hight, Pine, Fox, & Bar-Haim, In press). It could also be significant that the moderating effect of age was present only on the LSAS, where both clinician- and self-report methods were included, while the effect was absent for diagnostic remission, where clinician ratings were consistently used. As noted above, clinician ratings may have improved sensitivity, and thus may yield more consistent results across the lifespan. However, there was no evidence of significant age differences for studies using clinician vs. self-report LSAS (see Supplement, “Decomposing Moderators”). Thus, the benefit for younger participants in terms of LSAS reduction doesn’t appear to be explained by coincidental differences in anxiety measurement procedures across studies. Similarly, although clinician-rated LSAS was more likely to be used in lab-based studies, the moderating effect of training location was present even for diagnostic remission—which was exclusively clinician-rated—suggesting moderating effects of training location and rater were also at least partially independent.

It is noteworthy that the control conditions utilized in ABM studies provide methodologically stringent comparisons that share many features of ABM (e.g., completion of a cognitive task in the context of threatening stimuli). Under certain circumstances, control conditions outperformed active ABM—for example, among individuals trained in the lab who began with low/negative AB scores, suggesting attentional preference away from threat, an avoidant (rather than a vigilant) pattern (Figure 2D). A fundamental premise of mechanistic treatment research is that not all forms of anxiety are likely to be rooted in a unitary mechanism. However, the use of interventions with a ‘distilled’ unitary target should facilitate process-based parsing of individuals who represent the best fit for a particular procedure. Our data help to identify individuals for whom standard ABM procedures may be a poor fit, because their anxiety may be maintained by alternate mechanisms—specifically, older participants, for whom age-related or longitudinal changes may have reduced the primacy of attentional bias in promoting anxiety; and individuals already showing attentional avoidance of threat at baseline, for whom standard ABM procedures would offer no solution. Consistent with recent RCT findings in PTSD (Badura-Brack et al., 2015), ‘control’ conditions could in these cases provide a useful alternate form of training for specific patient subgroups (e.g., avoidant and/or older patients) via similar, but not identical, mechanisms—e.g., flexible attentional allocation in the context of threat, decreased avoidance of threat—particularly when overall processing speed and cognitive flexibility are degraded (e.g., by age, or being outside a controlled laboratory environment). Expectancy effects cannot be ruled out as an alternative explanation, although it is not obvious why the specific conditions that increased control response rates (e.g., home-based training, older age) would increase expectancy. Consistent with a precision medicine perspective, future studies should explore the possible benefits of alternative forms of training for specific patients and/or settings, while utilizing inert control conditions to match expectancies.

Findings have transdiagnostic implications for theory, research, and treatment of child and adult clinical anxiety. From a theoretical standpoint, robust effects of ABM were observed on clinician-rated diagnostic remission across diagnoses and age groups, which is consistent with the relevance of attentional bias as a theorized mechanistic treatment target that cuts across traditional clinical subgroups. From a research standpoint, ABM exemplifies the mechanistic treatment research approach—a growing area within clinical psychology and psychiatry. Our findings highlight the utility of not only testing whether mechanistic treatments improve symptoms, but explicitly assessing the degree of change within the mechanism itself. Failure to do so could lead to the erroneous conclusion that theorized mechanisms are not valid treatment targets, when in fact it is the methodology used to engage the mechanistic target that requires refinement (MacLeod & Grafton, In press). From a clinical standpoint, findings highlight both the promise and the limitations of the first generation of ABM techniques, which have primarily used a modified dot-probe paradigm. If this particular form of ABM is applied clinically, our findings suggest it will be most likely to reduce anxiety if delivered in a controlled, laboratory-like environment, and if given to younger patients (when treating social anxiety symptoms, at least). Personalized pre-selection of patients high on attentional bias may also be useful, although such efforts are currently stymied by poor psychometrics of widely-used performance-based indices (Price et al., 2015), which are inadequate to draw confident conclusions at the individual patient level, and lack of availability of these measures in many clinical settings. Development of a patient-report measure of vigilance to threat with strong psychometrics may be a useful future direction. Ideally, the breadth of ABM’s clinical impact can ultimately be extended through development of refined modification methods. For example, efforts are underway to make ABM more engaging [e.g., “game-ified” forms of ABM; (Dennis & O’Toole, 2014; Urech et al., 2015)] and to synergistically combine ABM with methods designed to enhance uptake of learning [e.g., brain stimulation (Clarke, Browning, Hammond, Notebaert, & MacLeod, 2014; Heeren, Baeken, Vanderhasselt, Philippot, & de Raedt, 2015)].

Limitations

Our focus on intervention mechanisms within a pooled patient-level approach, while offering several unique quantitative advantages, required consistent assessment measures to be used across studies, placing some constraints on the datasets available for meta-analysis. Our list of eligible studies was nevertheless highly comparable to recent study-level meta-analyses with a similar focus on clinical anxiety (Heeren, Mogoase, et al., 2015; Linetzky et al., 2015), and study-level moderator findings were also convergent with those reports, suggesting these additional criteria may not have substantially influenced findings. We were constrained by certain aspects of the available published datasets, including unclear risk of bias in some studies (Table S1), lack of follow-up data, and the fact that the mediator and clinical outcome were assessed contemporaneously. Without establishing the temporal precedence of change in mechanism, support for mediation is incomplete; however, causality in the opposing direction (i.e., anxiety reduction leading to AB reduction) would predict a generic link between AB reduction and anxiety reduction (in both active and control training), whereas in our data, the link was present only in active ABM. Reliance on completer datasets could alter effects, although high completion rates mitigate this impact. While use of completer datasets could lead to inflated effect sizes, effects calculated here could also underestimate the true magnitude and impact of clinical effects. For instance, two of the three eligible studies that did not contribute data reported strong positive findings, and the control conditions used in ABM studies (e.g., sham training—which replicates numerous potentially beneficial features of ABM) may offer stronger blinding and a more stringent comparator than is routinely achieved in behavioral treatment trials. Both AB assessment and training methods varied across studies. Furthermore, reliability of reaction time measurements of AB is suboptimal, even when methods to improve stability are applied (Price et al., 2015). Although these factors likely brought noise to the AB signal, expected ABM effects on AB were strongly evident, and hypothesized mechanistic links from AB to symptom-level outcomes were detectable in our comparatively large pooled sample. Subgroup definitions and unavailable data resulted in small samples for some analyses, reducing confidence in results, though samples remained large relative to previous patient-level analyses. Few studies that recruited pediatric or non-socially anxious patients could be identified utilizing uniform outcome measures. Given the transdiagnostic, cross-developmental relevance of AB (Bar-Haim et al., 2007), and our finding that ABM effects were more robust when multiple diagnoses and younger participants (including pediatric patients) were included, recruitment of heterogeneous samples is an important goal for future work.

Conclusions

The efficacy and posited mechanisms of a computer-based anxiety intervention, ABM, were partially validated in this pooled patient-level mega-analysis. Clinical significance of ABM shows much room for improvement. For example, response rates on a continuous measure of social anxiety—a clinically relevant outcome not routinely reported in ABM studies, but quantified through our pooled patient-level data—were uniformly below 40%, and efficacy on this measure was apparent only when specific conditions were met (clinician-rated assessment, laboratory training setting, patient age≤37). However, mixed findings reported in the literature, in both individual studies and prior meta-analyses, may not reflect a lack of potential for the intervention or an invalid theoretical framework, but rather reflect that identifiable contextual factors (training setting, type of rater) and individual differences (in age, baseline AB, and ‘trainability’ of AB) interact to constrain the circumstances where ABM produces benefit. Given its low cost and apparent acceptability to patients (e.g., low drop-out rates), and with further refinement based on the current findings, ABM might have utility within a stepped-care model, as at least a quarter of patients gained notable relief within a ~1-month timeframe using this automated, minimally invasive approach. Our pooled patient-level approach helped to clarify the precise combination of conditions that promote symptomatic benefit across individuals and studies, informing mechanistic theories of anxiety, evidence-based clinical decision-making, and further ABM refinements.

Highlights.

• Transdiagnostic remission was 2.5 times higher for active modification vs. control

• Anxiety decreased only when training sessions were completed in the laboratory

• Social anxiety decreased significantly only in participants <37 years old

• Reductions in anxiety were mediated by reductions in the mechanism, attention bias