Efficacy of Attention Bias Modification Using Threat and Appetitive Stimuli: A Meta-Analytic Review

Abstract

Attention Bias Modification (ABM) protocols aim to modify attentional biases underlying many forms of pathology. Our objective was to conduct an effect size analysis of ABM across a wide range of samples and psychological problems. We conducted a literature search using PubMed, PsycInfo, and author searches to identify randomized studies that examined the effects of ABM on attention and subjective experiences. We identified 37 studies (41 experiments) totaling 2,135 participants who were randomized to training toward neutral, positive, threat, or appetitive stimuli or to a control condition. The effect size estimate for changes in attentional bias was large for the neutral vs. threat comparisons (g =1.06), neutral vs. appetitive (g =1.41), and neutral vs. control comparisons (g = 0.80), and small for positive vs. control (g =0.24). The effects of ABM on attention bias were moderated by stimulus type (words vs pictures) and sample characteristics (healthy vs. high symptomatology). Effect sizes of ABM on subjective experiences ranged from 0.03 to 0.60 for post-challenge outcomes, −0.31 to 0.51 for post-treatment, and were moderated by number of training sessions, stimulus type and stimulus orientation (top/bottom vs. left/right). Fail-safe N calculations suggested that the effect size estimates were robust for the training effects on attentional biases, but not for the effect on subjective experiences. ABM studies using threat stimuli produced significant effects on attention bias across comparison conditions, whereas appetitive stimuli produced changes in attention only when comparing appetitive vs. neutral conditions. ABM has a moderate and robust effect on attention bias when using threat stimuli. Further studies are needed to determine whether these effects are also robust when using appetitive stimuli and for affecting subjective experiences.

Attention bias, the tendency to selectively attend to disorder-relevant stimuli, is implicated in the etiology and maintenance of psychopathology (Rapee & Heimberg, 1997; Williams, Watts, MacLeod, & Mathews, 1997). A meta-analysis of 172 studies found that attention bias toward threat stimuli is reliably associated with anxiety (Bar-Haim, Dominique, Pergamin, Bakermans-Kranenburg, & van IJzendoorn, 2007). Attention biases have also been associated with depression (Eizenman, 2003; Koster, de Raedt, Leyman, & De Lissnyder, 2010). Moreover, a number of other forms of psychopathology have been characterized by attention bias toward appetitive stimuli rather than threat, i.e., eating disorders (Glauert, Rhodes, Fink, & Grammer, 2010; Shafran, Lee, Cooper, Palmer, & Fairburn, 2007), smoking (Ehrman, 2002), alcohol use (Townshend & Duka, 2001), and sexual dysfunction (Beard & Amir, 2010). Evidence from genetic studies ( e.g., Beevers, Gibb, McGeary, & Miller, 2007; Caspi, Hariri, Holmes, Uher, & Moffitt, 2010; Gibb, Benas, Grassia, & McGeary, 2009) and prospective studies (Fox, Cahill, & Zougkou, 2010; Macleod & Hagan, 1992; van den Hout, Tenney, Huygens, Merckelbach, & Kindt, 1995) suggests that attention bias is not merely associated with psychopathology, but may constitute a vulnerability factor for developing psychopathology in response to stress.

Visual probe tasks are commonly used to assess attention. For example, in a typical dot probe task, two stimuli are presented quickly (e.g., 500 ms) on a computer monitor simultaneously. One of the stimuli is neutral (e.g., the word ‘chair’), whereas the other one is disorder-relevant (e.g., the word ‘disease’). After the brief presentation of these stimuli, the words disappear, and a probe (e.g., the letter ‘E’ or ‘F’) appears in the prior location of one of the words. The participant’s task is to identify the probe quickly and accurately by pressing a corresponding button. Biased attention is inferred from faster reaction times to identify probes replacing disorder-relevant compared to neutral stimuli (MacLeod, et al., 1986).

The original version of the dot probe task was designed to assess bias, and thus probes replaced neutral and disorder-relevant stimuli with equal frequency. By altering the contingency between the location of probes and disorder-relevant stimuli, the dot probe task can also be used to modify attention. If probes always replace neutral or positive stimuli, attention may be directed away from disorder-relevant stimuli. In these neutral or positive training conditions, acquiring an attention bias away from disorder-relevant stimuli will facilitate faster reaction times (i.e., better performance) on the task. Similarly, the task can also be used to induce a bias toward disorder-relevant stimuli when probes always replace the disorder-relevant stimuli. Finally, visual probe tasks have the unique methodological advantage of having an extremely well-matched control condition. In the control condition, probes replace neutral or positive and disorder-relevant stimuli with equal frequency. Thus, the control is simply the assessment version of the dot probe task.

To date, most Attention Bias Modification (ABM) studies have utilized aversive stimuli (e.g., threat words, sad faces) as disorder-relevant stimuli and positive or neutral stimuli (e.g., happy or neutral faces) as the other stimuli in the task¹. However, recently a growing number of studies have utilized appetitive stimuli (e.g., alcohol, smoking, food) and examined effects on motivational outcomes (e.g., desire to drink). Additionally, while most ABM studies have utilized probe detection tasks, a number of studies trained attention via a visual search paradigm. In the visual search task, participants repeatedly identify the location of a smiling face among a matrix of angry faces. In both types of tasks, acquiring an attention bias away from disorder-relevant stimuli will facilitate faster reaction times (i.e., better performance) on the tasks.

The aim of all of these ABM trainings is to modify attention via repeated practice on cognitive tasks (for qualitative reviews of ABM, see Bar-Haim, 2010; Browning, Holmes, & Harmer, 2010; Beard, 2011). Thus, ABM training may alter cognitive biases through a more implicit, experiential process compared to the explicit, verbal process of psychotherapy. This training is assumed to alter attentional processes that are not considered to be under volitional control, providing a different method of influencing this stage of information processing compared to existing treatments.

Researchers have mostly used ABM tasks to test the causal relationship between attention bias and emotional vulnerability or motivational states. To this end, researchers attempt to induce biases in healthy individuals in a single experimental session and examine the effect on responses to a laboratory challenge. Challenge tasks are typically related to the specific psychopathology under study. For example, socially anxious participants may undergo an impromptu speech, whereas heavy drinkers may complete an alcohol taste test. Recently, researchers have translated findings from single-session experiments into multi-session treatments, primarily for anxiety disorders.

ABM has grown rapidly over the last decade, and two quantitative reviews of the literature have been published. Hakamata and colleagues (2010) conducted a specific review limiting their examination to the effect of dot probe ABM tasks on attention bias and anxiety. Results revealed that ABM produced a large effect on attention bias (d = 1.16, CI = .82–1.50) and a medium effect on anxiety (d = 0.61, CI = .42–.81). Effect sizes for anxiety were larger on trait versus state measures, words versus face stimuli, and top-bottom versus left-right orientation of stimuli. Additionally, the number of sessions moderated effects on attention bias. These results were promising, yet tentatively based on only 12 experiments. Hallion and Ruscio (2011) recently extended the Hakamata findings by including 21 ABM studies in their meta-analysis and by examining ABM’s effect on depression in addition to anxiety. Results revealed small, but reliable effects on attention (g = .29), anxiety (g = .23), and non-significant effects on depression (g = .12). Effects on anxiety were larger for studies that included more than one training session (g = .40), although this was a non-significant trend.

Thus, the magnitude of ABM’s effects are unclear given the discrepant findings of these prior reviews, with Hakamata suggesting large effects on attention and medium effects on anxiety while Hallion and Ruscio suggesting small effects on both outcomes. Additionally, the Hakamata review concluded that effects on anxiety were reliable, but did not apply standard fail-safe N guidelines when interpreting effect sizes. Moreover, the Hallion and Ruscio review did not examine a number of task characteristics as potential moderators; thus, the moderators revealed by Hakamata await replication in a larger sample. Such findings have direct implications for how future ABM treatment should be delivered. Finally, none of the prior quantitative or qualitative reviews have included the growing number of ABM studies that utilize appetitive stimuli (e.g., alcohol, smoking, food) rather than aversive stimuli (e.g., threat words, sad faces) as disorder-relevant stimuli. Given the inherent differences in approach-avoidance tendencies related to such stimuli, it is possible that ABM may have different effects for appetitive versus threat stimuli. For example, it may be more difficult to train individuals to attend away from alcohol cues, given that the approach system is involved in problematic drinking (e.g., Field, Kiernan, Eastwood, & Child, 2008), whereas anxiety is characterized by behavioral avoidance (e.g., Barlow, 2002).

Thus, the current study extends prior reviews by including 33 additional experiments compared to Hakamata, examining several potentially important moderators (e.g., orientation of stimulus) identified in Hakamata’s review that were not examined in the Hallion and Ruscio review and thus await replication, and including studies which utilized appetitive stimuli. Our objective was to provide a comprehensive quantitative, meta-analytic review of the efficacy of ABM. We reviewed studies examining the effects of ABM on attention bias and subjective experiences (e.g., anxiety, depression, urge to drink alcohol or smoke). To determine the robustness of this new approach to testing cognitive models, it is important to quantitatively examine ABM’s ability to affect attention in healthy individuals, in addition to its ability to alleviate symptoms in analogue and clinical samples. Thus, similar to the Hallion review, we included studies that utilized healthy participants and studies that induced biases toward disorder-relevant stimuli, in addition to studies with samples of high levels of symptomatology (i.e., analogue, clinical).

ABM is not proposed to be an effective mood manipulation, but rather to affect an individual’s vulnerability to respond emotionally or behaviorally to emotional or motivational cues. Thus, we hypothesized that ABM would not have a direct effect on subjective experience (i.e., post-training). We expected ABM to have an effect on responses to challenge tasks (i.e., post-challenge) and following multi-session protocols (i.e., post-treatment). Based on the Hakamata review, we expected word stimuli and top-bottom orientation in training tasks to produce larger effects. However, this was a tentative hypothesis, given that Hakamata et al. (2010) only examined 12 experiments and Hallion & Ruscio (2011) did not examine these potential moderators. Finally, based on prior findings (Hakamta et al., 2010; Hallion & Ruscio, 2011), we expected stronger magnitude of training (i.e., number of sessions) to produce larger effects.

Methods

Eligibility Criteria

We determined that an intervention qualified as an ABM if it (a) directly targeted attention bias, and (b) modified attention via “extensive practice on a cognitive task designed to encourage and facilitate the desired cognitive change (Koster et al., p. 3).” Additionally, studies meeting the following criteria were eligible for inclusion: (1) included a measure of attention bias or subjective experience; (2) randomized participants; and (3) provided sufficient data to perform effect size analyses (i.e., means and standard deviations, t or F values, change scores, frequencies, or probability levels). Authors were contacted for additional data when needed.

As our primary aim was to provide a comprehensive examination of ABM’s effects on attention and emotional or motivational outcomes, our inclusion/exclusion criteria were broader than those of the Hakamata review. Specifically, Hakamata et al. only examined studies examining the effect of ABM on symptoms of anxiety. Additionally, they excluded studies not utilizing a dot-probe task. Our inclusion/exclusion criteria differed from the Hallion & Ruscio analysis in that they excluded all studies employing stimuli not directly relevant to anxiety or depression (e.g., food, cigarettes, alcohol). These studies were included in our analysis and were of particular interest in comparing the effects of appetitive vs. aversive stimuli. Additionally, the Hallion & Ruscio study included only studies in which the sample population was psychologically healthy or diagnosed with anxiety or depression. We did not employ this criterion, and therefore included studies examining a wider range of pathology (e.g., alcohol dependence, smoking). However, we limited our examination to studies testing ABM, rather than including two different types of interventions as did Hallion and Ruscio (i.e., ABM and interpretation bias modification).

Search

We performed the meta-analysis in accordance with the PRISMA guidelines (Liberati et al., 2009). Searches were conducted in PubMed and PsycInfo to identify studies published between the first available year and February 15, 2011. The following search terms were used: cognitive * bias * modification, attention * bias * modification, and attention * bias* training. Additionally, a manual review of authors identified through database searches was conducted. Articles related to the topic of ABM were selected for further examination.

Data Abstraction

Two of the authors (CB, ATS) independently extracted numerical data and coded potential moderators for each study. For examining the effect on subjective experience, the primary measure identified in each study report was extracted to examine group differences immediately after a single session of training (i.e., post-training), in response to a challenge task, such as an impromptu speech (i.e., post-challenge), and after a multi-session treatment (i.e., post-treatment). If a study did not identify a primary outcome, the authors (CB, ATS) identified the measure that best assessed the construct targeted by the study.

Study Characteristics

Meta regression analyses and the Q and I² statistics were used to determine whether effect sizes varied as a function of clinical characteristics (healthy vs. high symptomatology), type of pathology targeted (anxiety vs. depression vs. alcohol vs. smoking), study year, and training characteristics (number of sessions, number of training trials, stimulus orientation [i.e., top/bottom vs. left/right], stimulus modality [i.e., words vs. pictures]). Please note that stimulus orientation is only relevant for probe detection tasks, whereas the other training characteristics apply to all training tasks. Continuous measures were assessed by meta-regression. The Q and I² statistics were used to assess heterogeneity and categorical moderators (Huendo-Medina, Sánchez-Meca, Marin-Martinez, & Botella, 2006). The Q statistic indicates whether heterogeneity is present or absent, while the I² assesses the degree of heterogeneity on a 0 to 1 scale, with 0 representing complete homogeneity and 1 representing complete heterogeneity. Both the heterogeneity within each group (Q_within), and the heterogeneity between the groups (Q_between), was assessed. The grouping variable (i.e., the moderator) was considered significant when the between-groups heterogeneity was significant.

Quantitative Data Synthesis

Effect sizes were calculated using Hedges’s g and its 95% confidence interval. Hedges’s g is a variation of Cohen's d that corrects for biases due to small sample sizes (Hedges & Olkin, 1985). All effect sizes were calculated using random effects models because the studies included were assumed to be only a sample of the entire population of studies (Hedges & Vevea, 1998).

The effect size estimates for individual studies were combined to obtain a summary statistic. We calculated an average Hedges’s g effect size for attention bias and a separate Hedges’s g effect size for studies that included measures of subjective experience. We calculated effect sizes separately for each type of comparison condition: neutral (e.g., neutral faces) vs. control, positive (e.g., smiling faces) vs. control, and neutral vs. disorder-relevant (e.g., angry faces/alcohol pictures). For those studies that examined changes in subjective experience, we calculated effect sizes at post-training, post-challenge, and post-treatment when applicable. The magnitude of Hedges’s g may be interpreted using Cohen’s (Cohen, 1988) recommendations of small (0.2), medium (0.5), and large (0.8). In cases where the correlation between pre-and post measures was unavailable but necessary to calculate pre-post effect sizes, we followed the recommendation by Rosenthal (Rosenthal, 1993) and assumed a conservative estimation of r = 0.7. All analyses were completed using the software program Comprehensive Meta-Analysis, Version 2 (Borenstein, Hedges, Higgins, & Rothstein, 2005).

Validity Assessment

To address the file-drawer problem (i.e., the fact that studies with non-significant results are less likely to be published than those reporting significant results and can thereby bias meta-analytic results), we computed the fail-safe N (Rosenthal, 1991; Rosenthal & Rubin, 1988) which is an estimate of the number of unpublished studies reporting effect sizes of zero needed to nullify the significant effect. We used the following formula: $X=\frac{K(K{\overline{Z}}^2-2.706)}{2.706}$, where K is the number of studies included in the meta-analysis and Z̄ is the mean Z obtained from the K studies. According to Rosenthal (Rosenthal, 1991), if the required number of studies (X) to reduce the overall effect size to a non-significant level exceeds 5K + 10, the effect size can be considered robust. Fail-safe N values were calculated only for effect sizes that were significant.

Moderator Analyses

We examined the following potential moderators: clinical characteristics (healthy vs. high symptomatology); type of pathology targeted (anxiety vs. depression vs. alcohol vs. smoking); stimulus orientation (top/bottom vs. left/right); stimulus modality (words vs. pictures) of training task; stimulus modality (words vs. pictures) of assessment task; publication year; number of sessions; and number of training trials.

Results

Study Selection and Characteristics

Our initial searches identified 921 potentially relevant articles of which 37 studies (41 experiments) and a total of 2,135 participants met our inclusion criteria and were included in the meta-analysis (see Figure 1). In all studies, participants were randomized and blind to training condition. Table 1 details the characteristics of the included studies. For those studies reporting attention bias data, 17 experiments compared neutral vs. control conditions, nine compared neutral vs. disorder-relevant conditions, and six compared positive vs. control conditions (the numbers do not add up to 30 because some studies fell in more than one category). For the studies examining the effect on subjective experience, most experiments examined a neutral vs. control condition (eight studies after training; 11 after a challenge; and eight after treatment), followed by neutral vs. disorder-relevant condition (14 studies at post-training; 12 after a challenge; and none after treatment), and positive vs. control (two studies after training, two after a stressor, and three after treatment); (most experiments included more than one time point). All studies utilized either visual probe tasks or visual search tasks as the ABM method.

Figure 1.

Flow diagram of study selection process.

Table 1.

Description of studies

Study	Year	Sample Type	Condition Type	Type of bias test	Type of training	Outcome Post-training	Outcome Post-challenge	Outcome Post-treatment	# of sessions	Total # of trials
Amir et al.	2009a	GAD dx	control vs. neutral	Dot probe words	Dot probe words	none	None	HRSA	8	1280
Amir et al.	2009b	Social Phobia dx	control vs. neutral	Posner words	Dot probe pictures	none	None	LSAS	8	1024
Amir et al.	2008	High social anxiety	control vs. neutral	Posner words	Dot probe pictures	STAI-S	STAI-S	none	1	128
Attwood et al.	2008	Smokers	appetitive vs. neutral	Dot probe pictures	Dot probe pictures	QSU-brief score	QSU-brief score	none	1	512
Baert et al.	2010	High depression	control vs. positive	Posner words	Posner words	None	None	BDI-II	10	2000
Study 1
Study 2		MDD dx	control vs. positive	Posner words	Posner words	none	none	BDI-II	10	2000
Bar-Haim et al.	2011	High anxious children	control vs. neutral	Posner faces	Posner faces	None	Analog mood scale - anxiety	STAI-C	2	1536
Browning et al.	2009	Healthy adults	threat vs. neutral	Dot probe pictures	Dot probe words	STAI-S	None	none	1	576
Study 1
emsp;Study 2		Healthy adults	threat vs. neutral	n/a	Dot probe words	STAI-S	None	none	1	576
Dandeneau & Baldwin	2009	Healthy adults	control vs. positive	Dot probe pictures	Visual search	None	State self-esteem scale	None	1	112
Dandeneau et al.	2007	Low & High self-esteem	Control vs. positive	Dot probe pictures	Visual search	POMS- confident scale	None	None	1	112
Study 2b
Study 3a		Healthy adults	Control vs. positive	n/a	Visual search	None	STAI-S	None	5	400
Dandeneau & Baldwin	2004	Low & High self-esteem	Control vs. positive	Stroop words	Visual search	None	None	None	1	112
Eldar & Bar-Haim	2010	High anxious and non-anxious	control vs. neutral	Dot probe pictures	Dot probe pictures	STAI-S	None	none	1	480
Eldar et al.	2008	Healthy children	threat vs. neutral	Dot probe pictures	Dot probe pictures	Analog mood scale-anxiety	Analog mood scale-anxiety	none	2	672
Field et al.*	2007	Heavy drinkers	control vs. neutral appetitive vs. neutral	Dot probe pictures	Dot probe pictures	Urge to drink (VAS)	Urge to drink (VAS)	none	1	960
Field et al.*	2009	Smokers	Control vs. neutral appetitive vs. neutral	Dot probe pictures	Dot probe pictures	Urge to smoke (VAS)	Option to smoke	none	1	896
Field & Eastwood	2005	Heavy drinkers	appetitive vs. neutral	Dot probe pictures	Dot probe pictures	DAQ-urge to drink	DAQ-urge to drink	none	1	896
Harris & Menzies	1998	Healthy adults	threat vs. neutral	Dot probe words	Dot probe words	SPQ	None	none	1	40
Hayes et al.	2010	GAD/High worriers	control vs. neutral	Dot probe words	Dot probe words	VAS-anxiety	VAS-anxiety	none	1	480
Hazen et al.	2009	Clinical worriers	control vs. neutral	Dot probe words	Dot probe words	none	None	PSWQ	5	1020
Klumpp & Amir	2010	High social anxiety	control vs. neutral	n/a	Dot probe pictures	STAI-S	STAI-S	none	1	128
Koster et al.	2010	Healthy adults	control vs. neutral	Dot probe pictures	Dot probe pictures	none	None	STAI-T	5	900
Krebs et al.	2010	Healthy adults	threat vs. neutral	Dot probe words	Dot probe words	VAS-anxiety	VAS-anxiety	None	1	576
Li et al.	2008	High social anxiety	Control vs. positive	Dot probe pictures	Dot probe pictures	None	None	SIAS	7	3360
MacLeod et al.	2002	Healthy adults	threat vs. neutral	Dot probe words	Dot probe words	Analogue mood scale-anxiety	Analogue mood scale-anxiety	none	1	576
Study 1
Study 2		Healthy adults	threat vs. neutral	Dot probe words	Dot probe words	Analogue mood scale-anxiety	Analogue mood scale-anxiety	none	1	576
MacLeod et al.	2007	Healthy adults	threat vs. neutral	Dot probe words	Dot probe words	None	None	none	1	288
Study 2
McGowan et al.	2009	Healthy adults	threat vs. neutral	Dot probe words	Dot probe words	None	Threshold pain	none	1	320
McHugh et al.	2010	Smokers	control vs. neutral	Dot probe pictures	Dot probe pictures	QSU-brief craving	QSU-brief craving	none	1	476
McMillan	2009	High social anxiety	Control vs. positive	Dot probe pictures	Visual search	POMS-A	None	None	1	112
Najmi & Amir	2010	High OCD symptoms	control vs. neutral	Dot probe words	Dot probe words	None	BAT - % steps completed overall	none	1	288
Reese et al.	2010	High spider fears	control vs. neutral	Dot probe pictures	Dot probe pictures	VAS-anxiety	BAT - # steps completed	none	1	768
Schmidt et al.	2009	Social Phobia dx	control vs. neutral	n/a	Dot probe pictures	None	None	LSAS	8	1024
Schoenmakers et al.	2007	Heavy drinkers	control vs. neutral	Dot probe pictures	Dot probe pictures	None	Alcohol craving	none	1	576
Schoenmakers et al.	2010	Alcohol dependence dx	control vs. neutral	Dot probe pictures	Dot probe pictures	None	None	DAQ-craving (mild desires)	5	2640
See et al.	2009	Recent high school graduates	control vs. neutral	Dot probe words	Dot probe words	None	STAI-S	none	15	2880
Smith & Rieger	2006	Healthy females	appetitive vs. neutral	Dot probe words	Dot probe words	POMS-A	PASTAS-state subscale	none	1	320
Smith & Rieger*	2009	Healthy females	appetitive vs. neutral	Dot probe words	Dot probe words	POMS-A	PASTAS-state subscale	none	1	240
Van Bockstaele et al.	2011	Healthy adults	threat vs. neutral	Dot probe pictures	Dot probe	SPQ pictures	BAT - distance to spider	none	1	288
Wells & Beevers	2010	High depression	neutral vs. control	Dot probe pictures	Dot probe pictures	None	None	BDI-II	4	166

Note: Alcohol craving = participants indicated urge to drink on an analogue scale; Analogue mood scale – anxiety = participants indicated anxiety on an analogue scale; Anxiety (VAS) = Anxiety Visual Analogue Scale; BAT - % of steps completed = behavioral approach test, percentage of overall steps completed; BDI-II = Beck Depression Inventory II (Beck, Steer, & Brown, 1996); DAQ – urge to drink = Desires for Alcohol Questionnaire – urge to drink (Love, James, & Willner, 1998); DAQ-craving (mild desires) = Desires for Alcohol Questionnaire (Love, James, & Willner, 1998); HRSA = Hamilton Rating Scale for Anxiety (Hamilton, 1959); LSAS = Liebowitz Social Anxiety Scale (Liebowitz, 1987); PASTAS-state subscale = Physical Appearance State and Trait Anxiety Scale – state subscale (Reed, Thompson, Brannick, & Sacco, 1991); POMS-A = Profile of Mood States, anxiety subscale (McNair et al., 1971); POMS-confident scale = Profile of Mood States, confident subscale (McNair et al., 1971); PSWQ = Penn State Worry Questionnaire (Meyer, Miller, Metzger, & Borkovec, 1990); QSU-brief score = Questionnaire of Smoking Urges – brief version (Tiffany & Drobes, 1991); SPQ = Spider Phobia Questionnaire (Klorman, Weerts, Hastings, Melamed, & Lang, 1974); STAI-C = State-Trait Anxiety Inventory for Children (Spielberger, Edwards, Lushene, Montuori, & Platzek, 1973); STAI-S = State-Trait Anxiety Inventory, state version (Spielberger, 1983); STAI-T = State-Trait Anxiety Inventory, trait version (Spielberger, Gorsuch, Lushene, Vagg, & Jacobs, 1983); Threshold pain = time taken (sec) for participant to first register pain; Urge to drink (VAS) = Urge to drink, Visual Analogue Scale; Urge to smoke (VAS) = Urge to smoke, Visual Analogue Scale.

^*Denotes studies that included additional comparison conditions not included in the analyses. In additional to comparisons listed, Field et al. (2007) and Field et al. (2009) examined control vs. threat; Smith & Reiger (2009) examined neutral vs. positive and threat vs. positive.

Across the experiments, the most frequent type of sample was healthy individuals (19 experiments, n = 898 participants), followed by analogue samples (18 experiments, 993 participants), and clinical samples (7 experiments, 244 participants) (experiment numbers do not add up to 41 because several experiments included more than one type of sample). Within the high symptomatology samples, the most common pathology studied was social anxiety (n = 6), followed by generalized anxiety (n = 5), alcohol use (n = 4), smoking (n = 3), depression (n = 2), low self-esteem (n = 2), fear of spiders (n = 1), and obsessive-compulsive symptoms (n = 1).

Quantitative Data Synthesis

Effect on Attention Bias

Pre-post effect sizes

As depicted in Table 2, effect sizes were calculated separately for each of the comparison conditions. Within each comparison condition, we also present the effect sizes separately for studies utilizing healthy and high symptomatology samples. The random effects meta-analysis yielded the following average pre-post effect size estimates (Hedges’s g): neutral vs. control condition (g = 0.80; 95% CI: 0.49–1.12, p < .001), positive vs. control condition (g = 0.235; 95% CI: 0.020–0.449, p < .05), and neutral vs. disorder-relevant (g = 1.19; 95% CI: 0.96–1.41 , p< .001). For the neutral vs. control comparison condition, the See et al. (2009) study had a very large effect size (Hedges’s g = 4.76). Eliminating this study from the analysis did not change the general results (Hedges’s g = 0.72; 95% CI: 0.49–0.95, p < .01).

Table 2.

Effect size analysis of studies examining attentional bias

Comparison condition	Study	Hedges’s g	95% CI	p-value
Attend Neutral vs. Control
(neutral & threat stimuli)	Amir et al. 2009a	0.787	0.051–1.524	0.036
	Amir et al. 2009b	0.554	−0.038–1.145	0.067
	Amir et al, 2008	0.407	0.002–0.812	0.049
	Bar-Haim et al., 2011	0.660	−0.016–1.336	0.056
	Eldar & Bar-Haim, 2010 (low anxiety)	0.000	−0.696–0.696	1.000
	Eldar & Bar-Haim, 2010 (high anxiety)	0.341	−0.361–1.043	0.341
	Hayes et al., 2010	0.658	0.086–1.230	0.024
	Hazen et al., 2009	1.194	0.205–2.183	0.018
	Koster et al., 2010	1.341	0.723–1.960	0.000
	Najmi & Amir, 2010	0.822	0.264–1.380	0.004
	Reese et al., 2010	0.789	0.173–1.423	0.012
	See et al., 2009	4.760	3.552–5.969	0.000
	Wells & Beevers, 2010	1.536	0.784–2.289	0.000
(neutral & appetitive)	Field et al., 2007	0.623	0.000–1.245	0.050
	Field et al., 2009	0.171	−0.398–0.740	0.557
	McHugh et al., 2010	0.015	−0.525–0.556	0.956
	Schoenmakers et al., 2007	0.669	0.280–1.058	0.001
	Schoenmakers et al., 2010	0.329	−0.306–0.964	0.310
	Overall Threat Studies	0.958	0.552–1.365	0.000
	Overall Appetitive Studies	0.394	0.129–0.659	0.004
	Overall Studies	0.799	0.491–1.107	0.000
	Healthy Samples Only	1.619	0.864–2.375	0.000
	High Symptomatology Samples Only	0.614	0.305–0.923	0.000
Attend Positive vs. Control
(positive & threat stimuli)	Baert et al., 2010 (study 1)	0.258	−0.301–0.818	0.365
	Baert et al., 2010 (study 2)	0.568	−0.099–1.236	0.095
	Dandeneau & Baldwin, 2009	0.133	−0.210–0.477	0.447
	Dandeneau et al., 2007 (2b) (low self-esteem)	0.414	−0.055–0.883	0.084
	Dandeneau et al., 2007 (2b) (high self-esteem)	−0.164	−0.605–0.276	0.465
	Dandeneau & Baldwin, 2004 (2) (low self-esteem)	0.898	0.084–1.711	0.031
	Dandeneau & Baldwin, 2004 (2) (high self-esteem)	−0.317	−1.086–0.452	0.419
	Li et al., 2008	0.556	0.268–1.380	0.186
	Overall Positive Studies	0.235	0.020–0.449	0.032
	Healthy Samples Only	−0.017	−0.272–0.239	0.899
	High Symptomatology	0.475	0.197–0.753	0.001
Attend Neutral vs. Attend Disorder-relevant
(neutral & threat stimuli)	Browning et al., 2009 (study 1)	1.155	0.316–1.994	0.007
	Krebs et al., 2009	1.297	0.551–2.044	0.001
	MacLeod et al., 2007	0.571	0.0631–1.080	0.028
	McGowan et al., 2009	0.946	0.544–1.349	0.000
	Van Bockstaele et al., 2011	1.530	0.982–2.078	0.000
(neutral & appetitive)	Attwood et al., 2008	1.357	0.772–1.942	0.000
	Field & Eastwood, 2005	1.364	0.687–2.041	0.000
	Field et al., 2007	1.526	0.832–2.219	0.000
	Field et al., 2009	1.401	0.772–2.031	0.000
	Overall Threat Studies	1.061	0.719–1.403	0.000
	Overall Appetitive Studies	1.406	1.085–1.727	0.000
	Overall All Studies	1.192	0.963–1.420	0.000
	Healthy Samples Overall	1.034	0.783–1.286	0.000
	High Symptomatology Samples Only	1.406	1.081–1.732	0.000

Publication bias

Fail-safe N values were calculated for the pre-post attention bias effect sizes. The fail-safe N value for the attention bias effect size for the neutral vs. control comparison condition was robust at 427 (z-value = 10.00), indicating that 427 unpublished studies with effect sizes of zero would be necessary to nullify this result. When excluding the See et al. (2009) study, the fail-safe N value remains robust at 73 (z-value = 8.59). However, for the positive vs. control condition, the fail-safe N value was 4 (z-value = 2.41), suggesting that this result is unreliable and should be considered preliminary. The fail-safe N value was robust for the neutral vs. disorder-relevant condition (321, z-value = 11.85).

Threat vs. Appetitive Stimuli

Effect sizes were calculated separately for studies utilizing threat stimuli and those using appetitive stimuli within each comparison condition. For the neutral vs. control comparison condition, the pre-post effect size estimate was large for threat studies (g = 0.96; 95% CI: 0.55–1.37, p < .001), and small for appetitive studies (g = 0.39; 95% CI: 0.13–0.66, p = .004). For the neutral vs. disorder-relevant comparison condition, the pre-post effect size estimate was large for the neutral vs. threat studies (g = 1.06; 95% CI: 0.72–1.40, p < .001), and for the neutral vs. appetitive studies (g = 1.41; 95% CI: 1.09–1.73, p < .001). The fail-safe N value was robust for the threat studies (neutral vs. control = 179, neutral vs. threat = 84). For appetitive studies, the fail-safe N value (N = 73) was robust for the neutral vs. appetitive condition, but it was not robust for the neutral vs. control condition (N= 8).

Moderator Analyses

All meta-regression analyses performed on the neutral vs. control condition excluded the See et al. (2009) study due to its unusually large effect size. For the neutral vs. control condition, when comparing the type of pathology targeted, the largest mean effect size was for the study targeting depression (g =1.54; CI = 0.78–2.29, p < 0.001), followed by studies targeting anxiety (g =0.68; CI = 0.46–0.83, p < 0.001), alcohol (g =0.59; CI = 0.29–0.88, p < 0.001), and smoking (g =0.09; CI = −0.31–0.49, p = 0.66 ). The degree of heterogeneity within the groups was non-significant (Q_within = 10.84, df = 12, p = 0.54; I² = 0.0%), whereas the degree of heterogeneity between the groups was significant (Q_between = 13.18, df = 3, p = 0.004; I² = 77.2%). However, this effect was driven by the small effects observed in the two smoking studies. When these studies were removed, the type of pathology targeted no longer moderated effects on attention bias.

For the positive vs. control condition, sample type was a significant moderator. The mean effect size for studies with healthy samples was −0.02 (95% CI = −0.27–0.24, p = 0.90, n.s.), and the mean effect size for studies with high symptomatology samples was 0.48 (95% CI = 0.20–0.75, p = 0.001). The degree of heterogeneity within the groups was non-significant (Q_within = 3.54, df = 6, p = 0.74, n.s.; I² = 0.0%), whereas the degree of heterogeneity between the groups was significant (Q_between = 6.51, df = 1, p = 0.011; I² = 84.6%).

For the neutral vs. disorder-relevant condition, when comparing the stimulus modality (i.e., words vs. pictures) of the training paradigm, results indicated that the mean effect size for studies utilizing pictures was 1.44 (CI = 1.16–1.72, p < 0.001), and the mean effect size for those utilizing words was 0.91 (95% CI = 0.63–1.18, p < 0.001). The degree of heterogeneity within the groups was non-significant (Q_within = 3.4, df = 7, p = 0.85, n.s.; I² = 0.0%), whereas the degree of heterogeneity between the groups was significant (Q_between = 7.12, df = 1, p = 0.008; I² = 85.9%). This indicates that the effects of ABM on attention bias in studies utilizing pictures are significantly greater than the effects of ABM in studies utilizing words for this comparison condition. This same result emerged when comparing the modality of the test paradigm for this condition (pictures = 1.41 (95% CI = 1.15–1.67, p < 0.001), words = 0.88 (95% CI = 0.59–1.17, p < 0.001). The degree of heterogeneity within the groups was non-significant (Q_within = 3.42, df = 7, p = 0.84, n.s.; I² = 0.0%), whereas the degree of heterogeneity between the groups was significant (Q_between = 7.11, df = 1, p = 0.008; I² = 85.9%).

Effect on Subjective Experience

Post-Training Effects

As hypothesized, analyses for post-training effects revealed small, non-significant effects. The random effects meta-analysis yielded the following average pre-post effect size estimate (Hedges’s g): neutral vs. control (g = 0.01, 95% CI: −0.17–0.20; p = .89), positive vs. control (g = 0.09, 95% CI: −0.30–0.47; p = .66), and neutral vs. disorder-relevant (g = 0.03, 95% CI: −0.12–0.19, p = .70).

Post-Challenge Effects

The effect of ABM on subjective experiences at post-challenge and post-treatment is presented in Table 3. For post-challenge effects, the random effects meta-analysis yielded the following average pre-post effect size estimate (Hedges’s g): neutral vs. control (g = 0.22, 95% CI: −0.02–0.45; p = .07), positive vs. control (g = 0.60, 95% CI: −0.08–1.28; p = .08), neutral vs. disorder-relevant (g = 0.40, 95% CI: 0.22–0.57; p < 0.001).

Table 3.

Effect size analysis of studies examining subjective experience

Condition and Time-point	Study	Primary Outcome	Hedges’s G	95% CI	p-value
Attend Neutral vs. Control
Post-challenge
(neutral & threat stimuli)	Amir et al., 2008	STAI-S	0.450	0.044–0.856	0.030
	Bar-Haim et al., 2011	Anxiety analog scale	0.465	−0.202–1.132	0.172
	Hayes et al., 2010	Anxiety (VAS)	0.445	−0.119–1.008	0.122
	Klumpp & Amir, 2010	STAI-S	−0.543	−1.091–0.006	0.052
	Najmi & Amir, 2010	BAT - % of steps	0.789	0.241–1.355	0.005
	Reese et al., 2010	BAT - # of steps	−0.088	−0.689–0.513	0.774
	See et al., 2009	STAI-S	0.569	−0.054–1.192	0.074
(neutral & appetitive)	Field et al., 2007	Urge to drink (VAS)	0.144	−0.464–0.752	0.643
	Field et al., 2009	Option to smoke	0.400	−0.940–1.740	0.558
	McHugh et al., 2010	QSU- brief craving	0.056	−0.485–0.597	0.840
	Schoenmakers et al., 2007	Alcohol craving	−0.059	−0.437–0.319	0.759
	Overall Threat Studies		0.299	−0.039–0.637	0.083
	Overall Appetitive Studies		0.028	−0.242–0.299	0.837
	Overall All Studies		0.217	−0.016–0.451	0.068
	Healthy Samples Only		0.569	−0.054–1.192	0.074
	High Symptomatology Only		0.186	−0.060–0.431	0.138
Post-treatment
(neutral & threat stimuli)	Amir et al., 2009a	HRSA	0.893	0.149–1.638	0.019
	Amir et al., 2009b	LSAS	1.144	0.517–1.772	0.000
	Bar-Haim et al., 2011	STAI-C	−0.142	−0.801–0.516	0.672
	Hazen et al., 2009	PSWQ	0.935	0.101–1.769	0.028
	Koster et al., 2010	STAI-T	−0.152	−0.721–0.418	0.601
	Schmidt et al., 2009	LSAS	0.561	−0.091–1.213	0.092
	Wells & Beevers, 2010	BDI-II	0.304	−0.357–0.966	0.368
(neutral & appetitive)	Schoenmakers et al., 2010	DAQ – mild craving	−0.031	−0.661–0.600	0.924
	Overall Threat Studies		0.482	0.080–0.884	0.019
	Overall Appetitive Studies		−0.310	−0.661–0.600	0.924
	Overall All Studies		0.414	0.048–0.780	0.027
	Healthy Samples Only		−0.152	−0.721–0.418	0.601
	High Symptomatology Only		0.506	0.128–0.883	0.009
Attend Positive vs. Control
Post-challenge
(positive & threat stimuli)	Dandeneau & Baldwin, 2009	State Self-Esteem Scale	0.334	−0.011–0.679	0.058
	Dandeneau et al., 2007 (study 3a)	STAI-S	1.055	0.242–1.868	0.011
	Overall Positive Studies(all are healthy samples)		0.597	−0.083–1.276	0.085
Post-treatment
(positive & threat stimuli)	Beart et al., 2010 (study 1)	BDI-II	−0.494	−1.059–0.072	0.087
	Beart et al., 2010 (study 2)	BDI-II	0.213	−0.443–0.869	0.525
	Li et al., 2008	SIAS	0.711	−0.124–1.545	0.095
	Overall Positive Studies		0.093	−0.592–0.777	0.791
	(all are high symptomatology)
Attend Neutral vs. Attend Disorder-relevant
Post-challenge
(neutral & threat stimuli)	Eldar et al., 2008	Anxiety analog scale	0.685	−0.083–1.452	0.080
	Krebs et al., 2009	VAS - anxiety	0.293	−0.386–0.972	0.398
	MacLeod et al., 2002 (study 1)	Analogue scale anxiety	0.328	−0.159–0.816	0.187
	MacLeod et al., 2002 (study 2)	Analogue scale anxiety	0.748	0.247–1.249	0.003
	McGowan et al., 2009 (threat instruct.)	Threshold pain	0.239	−0.304–0.783	0.388
	McGowan et al., 2009 (no-threat instr.)	Threshold pain	0.645	0.100–1.191	0.020
	Van Bockstaele et al., 2011	BAT- distance to spider	0.111	−0.370–0.592	0.651
(neutral & appetitive)	Attwood et al., 2008	QSU- Brief score	0.077	−0.448–0.603	0.773
	Field & Eastwood, 2005	DAQ - urge to drink	0.468	−0.148–1.084	0.137
	Field et al., 2007	Urge to drink (VAS)	0.306	−0.313–0.924	0.333
	Field et al., 2009	Option to smoke	−0.828	−2.060–0.404	0.188
	Smith & Reiger, 2006	PASTAS - (state)	0.580	0.016–1.144	0.044
	Smith & Reiger, 2009	PASTAS - (state)	0.823	0.202–1.445	0.009
	Overall Threat Studies		0.412	0.187–0.637	0.000
	Overall Appetitive Studies		0.362	0.041–0.684	0.027
	Overall All Studies		0.396	0.222–0.569	0.000
	Healthy Samples Only		0.474	0.276–0.672	0.000
	High Symptomatology Only		0.185	−0.139–0.510	0.262

Note: Refer to Table 1 note for information regarding primary outcomes.

Effect sizes on subjective experience may differ from other reviews because we examined primary outcome measures, whereas Hallion and Ruscio (2011) examined composite scores of all administered measures.

Post-Treatment Effects

We calculated pre-post effect sizes for studies assessing changes in symptoms following a multi-session ABM. For neutral vs. control conditions, the random effects meta-analysis yielded an average pre-post effect size estimate (Hedges’s g) of 0.41 (95% CI: 0.05–0.78, p = .03). This effect appeared to be driven entirely by studies utilizing sample with high symptomatology (g = 0.51), rather than studies of healthy samples (g = −0.15). For positive vs. control conditions, the analysis yielded an effect size of (0.09 (95% CI: −0.59–0.79, p = .79). None of the studies in the neutral vs. disorder-relevant condition involved multi-session treatments.

Publication Bias

Although most post-challenge and post-treatment effects were significant and in the moderate range, none of these effect sizes were robust according to a priori fail-safe N guidelines (Rosenthal, 1991). Thus, all effect sizes for subjective experience outcomes should be interpreted with caution until further evidence is available.

Threat vs. Appetitive Studies

Effect sizes were calculated separately for studies utilizing threat stimuli and those using appetitive stimuli within each comparison condition. For the neutral vs. control condition, the post-challenge effect size estimate for threat studies was not significant (g = 0.30, 95% CI: −0.04–0.64, p = .083). However, the post-treatment effect for threat studies was significant and medium in size (g = 0.48, 95% CI: 0.08–0.88, p = .019). For appetitive studies, neither post-challenge, (g = 0.03, 95% CI: −0.24–0.30, p = .84), nor post-treatment effects (g = −0.031, 95% CI: −0.66–0.60, p = .92) were significant.

For the neutral vs. disorder-relevant comparison condition, the post-challenge effect was significant for both threat (g = 0.41, 95% CI: 0.19–0.64, p < .001) and appetitive studies (g = 0.36, 95% CI: 0.04–0.68, p = .027). However, similar to the overall subjective experience effect sizes, the fail-safe N values were not robust for any of the threat or appetitive subjective experience outcome effect sizes.

Moderator Analyses

For the neutral vs. control condition at post-challenge, the mean effect size for studies utilizing pictures was 0.07 (CI = −0.16–0.31, p = 0.54, n.s.), and the mean effect size for those utilizing words was 0.62 (95% CI = 0.16–1.09, p = 0.009). The degree of heterogeneity within the groups was non-significant (Q_within = 11.28, df = 8, p = 0.19, n.s.; I² = 29.0%), whereas the degree of heterogeneity between the groups was significant (Q_between = 4.27, df = 1, p = 0.039; I² = 76.6%). This indicates that the effects of ABM utilizing words on subjective experience are significantly greater than those utilizing pictures.

For the neutral vs. control condition at post-treatment, orientation of the stimuli was a moderator. The mean effect size for studies utilizing a top/bottom orientation was 0.88 (CI =0.53–1.23, p < 0.001), and the mean effect size for those utilizing a left/right orientation was −0.02 (95% CI = −0.33–0.30, p = 0.91, n.s.). The degree of heterogeneity within the groups was non-significant (Q_within = 2.88, df = 6, p = 0.82, n.s.; I² = 0.0%), whereas the degree of heterogeneity between the groups was significant (Q_between = 14.09, df = 1, p < 0.001; I² = 92.9.0%). This indicates that a top/bottom orientation was more effective than a left/right orientation. Finally, Hedges’ g was moderated by the number of training sessions for this condition at post-treatment (β = −0.176, SE = 0.066, p = 0.007), with a greater number of training sessions resulting in larger effect sizes.

For the neutral vs. disorder-relevant condition at post-challenge there were no moderators (there were no studies examining post-treatment effects). There were too few studies in the positive vs. control condition at post-challenge (N = 2) and post-treatment (N = 3) to run moderator analyses.

Discussion

We examined the effects of ABM on attention bias and subjective experiences across various forms of psychopathology. We identified 37 studies, of which we analyzed 41 experiments with 2,135 participants to derive effect size estimates. The current results confirm that ABM has a reliable effect on attention bias. Similar to Hakamata et al. (2010), we found statistically significant and large effects of ABM on attention with large fail-safe calculations. These large effect sizes are in contrast to small effects reported in Hallion and Ruscio (2011). As expected and similar to prior findings (Hallion & Ruscio, 2011), studies that compared two active trainings yielded larger effects than studies that compared a neutral or positive induction versus a control condition.

Results revealed that effect size estimates for subjective experiences obtained directly following ABM were small and non-significant across all comparison conditions. Thus, ABM is not an effective state emotional or motivational manipulation. In contrast, the effect size estimates for emotional and motivational responses to laboratory and natural challenges were small, but significant. Our effect sizes for these outcomes were in line with Hallion and Ruscio (2011), which also obtained small effect sizes. When applied as a multi-session protocol, ABM effects on symptomatology were also significant. Hakamata et al. (2010) obtained similar effect sizes specifically for changes in anxiety, but those authors interpreted the effect sizes as reliable and supportive of ABM as a treatment for anxiety. However, similar to the current results, the fail-safe N obtained by Hakamata and colleagues was not robust, and thus we conclude that both reviews suggest that there is currently insufficient data to determine ABM’s effect on subjective experiences.

The current review is the first to include studies which utilized appetitive stimuli. When examining effect sizes separately for these studies, results suggest that only studies comparing neutral vs. appetitive conditions (i.e., two active trainings) produced significant effects on attention and subjective experiences. These initial findings suggest that ABM may not be as promising a treatment for some disorders, such as alcohol or nicotine dependence, compared to anxiety. One explanation for these findings is that it may be more difficult to direct attention away from appetitive stimuli than aversive stimuli. However, these findings are based on a small number of appetitive studies per condition. Additionally, no study directly compared the efficacy of ABM with threat stimuli vs. appetitive stimuli. Future reviews are warranted after more studies are completed.

Observed effect sizes were unrelated to publication year, but were related to sample and task characteristics. Consistent with Hakamata et al. (2010), training with top-bottom orientation was more effective than left-right orientation. This effect may be in part due to the fact that the largest effect sizes were obtained from three multi-session treatment studies for clinically anxious individuals, which all utilized top-down orientation. Multi-session studies that used left-right orientation included a variety of samples (healthy, depressed, alcoholics, children) and obtained smaller effects. In Hakamata et al. (2010), all of the studies targeted anxiety, and most of the multi-session studies used top-bottom.

Pictures were superior to words, but only for neutral vs. disorder-relevant condition effects on attention, which were almost exclusively tested in healthy controls. Words were more effective than pictures, but only when comparing neutral vs. control conditions and only for subjective experiences at post-challenge. Thus, there does not appear to be a consistent superior stimulus type across ABM in the current study. Hakamata et al. (2010) found that words were superior to pictures, but only when comparing neutral vs. control condition effects on attention.

Hakamata et al. (2010) found that number of sessions moderated effects on attention. The Hallion and Ruscio (2011) review also obtained this trend, but for emotional outcomes. In the current study, number of sessions moderated subjective experience effects even when the See et al. outlier study was not included in the analyses. This study included substantially more sessions of training (n = 15) than other studies and a much larger effect size, providing further support for a dose-response relationship. Thus, it is clear that future ABM studies should include multiple sessions in order to obtain larger and perhaps more reliable effects on attention and subjective experience.

The results of this study are limited to the meta-analytic technique and, therefore, are dependent on the study selection criteria, the quality of the included studies, expectancy effects, and statistical assumptions about the true values of the included studies. In order to limit any possible biases, we adopted a relatively conservative approach. Following the recommendations by Moses and colleagues (2002) and Hedges and Vevea (1998), we analyzed the effect sizes using a random effect model. Perhaps the most important bias of meta-analyses is the expectancy effect. Cotton and Cook (1982) recommended early on that the investigators of meta-analyses explicitly state their personal view with regards to the outcome in order to acknowledge and possibly avoid the expectancy effect. At the outset of our review, the second and third authors were not involved with ABM. Given the first author’s work in the ABM field and the positive results from prior meta-analyses, we did expect to find significant effects. However, all authors remained skeptical about the size of the effects across different forms of psychopathology and maintained equipoise throughout the review process. Another limitation relates to the small number of studies targeting depression and smoking, which prevented us from drawing conclusions about whether the type of pathology targeted moderates outcomes. Finally, given the few studies that included a follow-up assessment, we were unable to examine duration of effects.

Despite these limitations, the current quantitative review of randomized experiments suggests that ABM is a robust method for modifying attention bias across a wide range of samples and stimuli. However, additional randomized controlled trials are needed before conclusions can be made about ABM’s effect on emotional and motivational outcomes. It is intuitive that effects on attention, the construct being manipulated, would be larger than those on subjective experience. Most studies delivered only a single session of ABM, and this not be an adequate dose of ABM to produce reliable effects on subjective experience. Additional studies are particularly needed to determine the utility of ABM using appetitive stimuli. Given the potential clinical utility (e.g., standardized and computerized delivery, no therapist contact, inexpensive) of this novel approach, larger definitive trials are warranted.

Highlights.

• We reviewed 33 experiments to examine ABM’s effect on attention and subjective experiences.

• ABM had robust effects on attention.

• ABM had small to medium effects on subjective experiences, but these were not robust

• Effects were stronger for studies using threat compared to appetitive stimuli

• ABM is an efficacious method for affecting attention bias

Appendix

Procedural Notes

1. We included studies that compared neutral vs. control, positive vs. control, or neutral vs. disorder-relevant training conditions. Other comparison conditions (neutral vs. positive) were not examined due to an insufficient number of studies examining these comparisons.

2. Amir et al. (2008, 2009): Similar to the original papers, we used data from the invalid social threat trials as an attention bias index.

3. Baert (2010, Study 1): We analyzed data from the entire depressed sample rather than splitting the sample into mildly and moderately/severely depressed as the authors did in exploratory analyses.

4. Dandeneau et al. (2007, study 2a & 2b): Original paper presented simple slope analyses; we calculated effect sizes from M and SDs.

5. Hayes et al. (2010): (a) This sample comprised a majority of patients diagnosed with GAD with the remaining participants scoring in the clinical range on the PSWQ. (b) The attention bias test trials were embedded within the training block and thus the effect size is based on first half of training compared to second half of training.

6. Krebs et al. (2009): We only included data from groups receiving the standard instructions to remain consistent with other studies.

7. Li et al. (2008): For attention bias effect size calculations, Day 1 data was used as pre data, and Day 7 data was used as post data.

8. MacLeod et al. (2002, Study 1): We only included data from the 500 ms trials to remain consistent with other studies.

9. Smith & Reiger (2006, 2009): We only examined negative shape/weight word conditions, not negative emotional word control conditions.

10. The majority of studies used the no-contingency, assessment version of the dot probe as the control condition. However, please note that See et al. (2009) and Wells and Beevers (2010) used a no-training control.

11. Please note that Baert et al. (2010) and Bar Haim et al. (2011) utilized a visual probe ABM task that presents only one stimulus, rather than two stimuli. In the neutral and positive conditions, probes replaced neutral/positive stimuli and showed up in the opposite location of threat stimuli.

PRISMA checklist of items to include when reporting a systematic review or meta-analysis

Heading	Subheading	Descriptor	Reported? (Yes/N)	Page number
Title		Identify the report as a systematic review, meta-analysis, or both	Yes	1
Abstract	Structured summary	Provide a structured summary including, as applicable, background, objectives, data sources, study eligibility criteria, participants, interventions, study appraisal and synthesis methods, results, limitations, conclusions and implications of key findings, systematic review registration number	Yes	2
Introduction	Rationale	Describe the rationale for the review in the context of what is already known	Yes	4–7
	Objectives	Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS)	Yes	8
Methods	Protocol and registration	Indicate if a review protocol exists, if and where it can be accessed (such as web address), and, if available, provide registration information including registration number	N/A
	Eligibility criteria	Specify study characteristics (such as PICOS, length of follow-up) and report characteristics (such as years considered, language, publication status) used as criteria for eligibility, giving rationale	Yes	9
	Information sources	Describe all information sources (such as databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched	Yes	10
	Search	Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated	Yes	10
	Study selection	State the process for selecting studies (that is, screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis)	Yes	10
	Data collection process	Describe method of data extraction from reports (such as piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators	Yes	10
	Data items	List and define all variables for which data were sought (such as PICOS, funding sources) and any assumptions and simplifications made	Yes	10
	Risk of bias in individual studies	Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis	Yes	11–12
	Summary measures	State the principal summary measures (such as risk ratio, difference in means).	Yes	11–12
	Synthesis of results	Describe the methods of handling data and combining results of studies, if done, including measures of consistency (such as I2 statistic) for each meta-analysis	Yes	11–12
	Risk of bias across studies	Specify any assessment of risk of bias that may affect the cumulative evidence (such as publication bias, selective reporting within studies)	Yes	11–12
	Additional analyses	Describe methods of additional analyses (such as sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified	Yes	11–12
Results	Study selection	Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram	Yes	Figure 1
	Study characteristics	For each study, present characteristics for which data were extracted (such as study size, PICOS, follow-up period) and provide the citations	Yes	12–13
	Risk of bias within studies	Present data on risk of bias of each study and, if available, any outcome-level assessment (see item 12).	Yes	12–13
	Results of individual studies	For all outcomes considered (benefits or harms), present for each study (a) simple summary data for each intervention group and (b) effect estimates and confidence intervals, ideally with a forest plot	Yes	Tables 2 & 3
	Synthesis of results	Present results of each meta-analysis done, including confidence intervals and measures of consistency	Yes	13–18
	Risk of bias across studies	Present results of any assessment of risk of bias across studies (see item 15)	Yes	13–18
	Additional analysis	Give results of additional analyses, if done (such as sensitivity or subgroup analyses, meta-regression) (see item 16)	Yes	13–18
Discussion	Summary of evidence	Summarise the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (such as health care providers, users, and policy makers)	Yes	19–20
	Limitations	Discuss limitations at study and outcome level (such as risk of bias), and at review level (such as incomplete retrieval of identified research, reporting bias)	Yes	21
	Conclusions	Provide a general interpretation of the results in the context of other evidence, and implications for future research	Yes	20–222
Funding	Funding	Describe sources of funding for the systematic review and other support (such as supply of data) and role of funders for the systematic review	Yes	23