Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Aug 1.
Published in final edited form as: Behav Res Ther. 2023 Jun 2;167:104344. doi: 10.1016/j.brat.2023.104344

A systematic review with meta-analysis of cognitive bias modification interventions for anger and aggression

Nicole K Ciesinski a,*, McKenzie Himelein-Wachowiak a, Lynette C Krick a, Kristen M Sorgi-Wilson b, Joey CY Cheung a, Michael S McCloskey a
PMCID: PMC10526745  NIHMSID: NIHMS1909176  PMID: 37307657

Abstract

Aggression and anger are associated with interpretation and attention biases. Such biases have become treatment targets for anger and aggressive behavior in cognitive bias modification (CBM) interventions. Several studies have evaluated the efficacy of CBM for the treatment of anger and aggressive behavior, with inconsistent results. The present study meta-analytically analyzed 29 randomized controlled trial studies (N=2334) published in EBSCOhost and PubMed between March 2013 and March 2023 assessing the efficacy of CBM for anger and/or aggression. Included studies delivered CBMs that addressed either attention biases, interpretation biases, or both. Risk of publication bias and potential moderating effects of several participant-, treatment- and study-related factors were assessed. CBM significantly outperformed control conditions in the treatment of aggression (Hedge’s G=0.23,95%CI[0.35,0.11], p<.001) and anger (Hedge’s G=0.18,95%CI[0.28,0.07], p=.001) independent of treatment dose, participant demographic characteristics, and study quality, though overall effects were small. Follow-up analyses demonstrated that only CBMs targeting interpretation bias were efficacious for aggression outcomes, but not when baseline aggression was accounted for. Findings suggest that CBM demonstrates efficacy for the treatment aggressive behavior and to a lesser extent, anger.

Keywords: cognitive bias modification, aggression, anger, meta-analysis, randomized controlled trials


Aggression and its most common emotional antecedent, anger, pose substantial costs to public health and society. Aggression is a leading cause of death and non-fatal injury in the United States, accounting for approximately 1.5 million hospitalizations and 21,000 homicides per year (CDC, 2021; FBI, 2022). Globally, approximately one billion children have been subjected to some form of violence in the past year, one in three women have been victims of aggression in their lifetime, and in 2019 alone, interpersonal violence resulted in 475,000 deaths worldwide (WHO, 2022). Furthermore, aggression results in billions of dollars worldwide in annual medical expenses and lost productivity (Corso et al., 2007), which is estimated to be as high as 11% of the world’s gross domestic product (WHO, 2022). These data do not account for the short- and long-term physical and psychological impacts on victims (Krug et al., 2002) or aggressors (Griffin et al., 2019; Rynar & Coccaro, 2018), as well as the intergenerational transmission of aggression (Conger et al., 2003). Chronic and/or high levels of anger are also associated with a range of physical health problems, such as inflammation (Barlow et al., 2019), hypertension (Harburg et al., 2003), heart disease (Smith et al., 2004), and early mortality (Trudel-Fitzgerald et al., 2021). Thus, efficacious interventions for both aggressive behavior and chronic or dysregulated anger hold substantial public health importance worldwide.

Social information processing theory delineates six cognitive processes that contribute to the development and maintenance of aggression: (1) selective attention to and encoding of social cues, (2) interpretation of these cues (e.g., intent attribution), (3) selection of desired goals/outcomes of social interaction, (4) generation of possible responses, (5) response evaluation and selection, and (6) behavioral enactment (Crick & Dodge, 1994). Though this model was developed to explain aggression in children, decades of research reveal that biases in cognitive processes are commonly implicated in anger and aggression across the lifespan. For example, hostile attribution (i.e., interpretation) bias, the tendency to misattribute others’ behavior to hostile motives, has long been found to be associated with anger and aggressive behavior in children (Martinelli et al., 2018), adolescents (Dodge et al., 1990), and adults (Klein Tuente et al., 2019). Similarly, attentional biases toward threatening information such as words or faces are augmented among children (Miller & Johnston, 2019) and adults (Chan et al., 2010) with higher trait anger and/or aggression. The association between cognitive biases and dysregulated anger and aggression suggests that attention and interpretation biases may be important targets for intervention among individuals who experience such behavioral and emotional dysregulation.

Cognitive bias modification (CBM) interventions were initially developed to treat the dysfunctional attentional (Amir et al., 2008; Schultz & Heimberg, 2008) and interpretive (Beard & Amir, 2008; Coles et al., 2008) biases to threat in anxiety disorders. These computerized CBM interventions can be broadly divided into two types: interpretation bias modification (IBM) and attention bias modification (ABM). IBM interventions train individuals to interpret ambiguous social cues (e.g., interpersonal scenarios, facial expressions) in more benign, non-hostile ways, such that more positive or neutral interpretations of the cues (as opposed to hostile or threatening interpretations) are reinforced. ABM interventions employ basic cognitive psychology paradigms (e.g., dot probe task) to create a contingency wherein responses that involve directing attention away from a threatening cue (e.g., hostility-related word, angry face) are reinforced.

IBM (Beard & Amir, 2008; Peters et al., 2009) and ABM (Amir, Beard, Burns, et al., 2009; Amir, Beard, Taylor, et al., 2009) interventions have demonstrated efficacy in the treatment of a range of anxiety disorders and have also been applied to other populations such as individuals diagnosed with substance use disorders (Cristea et al., 2016) and depression (Fodor et al., 2020). Notably, aggression and anger share several mechanisms of aberrant threat processing with anxiety, namely, attentional (Coccaro et al., 2007; Maoz et al., 2017; Schultz & Heimberg, 2008) and interpretive (Coccaro et al., 2009; Coles et al., 2008; Wenzel & Lystad, 2005) biases toward threat. Furthermore, anxiety, anger, and aggression commonly co-occur (Bubier & Drabick, 2009; Kessler et al., 2006; Versella et al., 2016), suggesting shared risk processes and the potential for anger and aggression treatments to be informed by the anxiety disorder treatment literature.

Given the overlap in cognitive biases associated with anxiety, anger, and aggression, combined with the empirical support for CBM in reducing anxiety (e.g., Beard & Amir, 2008; Amir, Beard, Burns, et al., 2009), researchers began to evaluate CBM interventions for the treatment of dysregulated anger and aggression. Several studies demonstrated significant, positive effects of CBMs on anger (Dillon et al., 2020; Hawkins & Cougle, 2013; Smith et al., 2018) and aggressive behavior (Schmidt & Vereenooghe, 2021; van Teffelen et al., 2021). However, other studies demonstrate no effect of CBM on aggression (e.g., Kuin et al., 2020; Sprunger, 2019) or anger (e.g., AlMoghrabi et al., 2019; Hiemstra et al., 2019), and some studies even demonstrate increases in anger post-treatment relative to controls (e.g., Krans et al., 2014; Osgood et al., 2021). As such, it is unclear whether CBM is an efficacious treatment for these target outcomes.

The inconsistent findings across studies assessing the delivery of CBM for anger and aggression may be a result of the heterogeneity in populations studied (e.g., varying age, gender, symptomatology), treatment adaptations employed (e.g., ABM vs. IBM, duration and frequency of treatment sessions), and/or study methodological rigor. One review of twelve meta-analyses on CBMs for a variety of clinical conditions (that did not include anger or aggression) noted mixed results with respect to potential moderating effects of gender, age, baseline level of symptomatology, and CBM type on treatment outcomes such as change in biases and clinical symptoms (Jones & Sharpe, 2017). Most meta-analyses demonstrated that CBMs were effective independent of age, gender, and level of clinical symptoms, while other meta-analyses found younger participants (Mogoaşe et al., 2014), females (Menne-Lothmann et al., 2014), and samples with either lower (Cristea et al., 2015) or higher (Beard et al., 2012; Menne-Lothmann et al., 2014) baseline symptomotology benefitted more. Moreover, one meta-analysis demonstrated that IBMs were more successful in treating anxiety and depression than ABMs (Hallion & Ruscio, 2011). Finally, separate meta-analytic reviews of the efficacy of CBM for clinical anxiety and depression (Cristea et al., 2015) and substance addictions (Cristea et al., 2016) both found that methodological rigor (assessed via Cochrane Collaboration’s Risk of bias assessment tool) significantly moderated results in both studies, such that study quality had a negative relationship with effect size. These findings corroborate previous evidence (Kjaergard et al., 2001; Schulz et al., 1995) that risk of bias in a study can be associated with artificial inflation of effect sizes. Thus, it is possible that the efficacy of CBMs for anger and aggression may vary depending on participant (i.e., age, gender, level of symptomatology), treatment (i.e., CBM type, duration, session frequency), and/or study (i.e., risk of bias) factors.

Meta-analyses offer the opportunity to statistically combine results and synthesize conclusions to critically evaluate therapeutic efficacy, as well as identify potential moderators obscuring individual study findings. However, to our knowledge, no meta-analytic review has been performed to assess the efficacy of CBMs in the treatment of aggression and anger. The aim of the present meta-analysis is twofold: (1) to evaluate existing randomized controlled trials to assess whether CBMs outperform control conditions in the treatment of both clinical and non-clinical levels of anger and aggression; and (2) to identify which conditions (i.e., sample characteristics, treatment adaptations, methodological rigor) may moderate CBM treatment efficacy. It was hypothesized that CBM would significantly outperform control conditions in the treatment of aggression, and to a lesser extent anger, given that many trials showed significant positive clinical effects of CBM on these outcomes (e.g., Dillon et al., 2020; Hawkins & Cougle, 2013; Schmidt & Vereenooghe, 2021; van Teffelen et al., 2021), with only few studies conversely reporting increases in anger throughout CBM intervention relative to control conditions (e.g., Krans et al., 2014; Osgood et al., 2021). Based on the equivocal nature of the research on potential moderators of CBM efficacy, no specific hypotheses were made with respect to participant-related (i.e., age, gender, level of symptomatology) and treatment-related (i.e., CBM type, treatment duration, number of treatment sessions) factors. However, it was hypothesized that risk of bias would significantly moderate study findings, with larger effects being associated with lower methodological rigor, due to evidence of effect size inflation in studies with higher risk of bias (e.g., Kjaergard et al., 2001; Cristea et al., 2015).

Method

The present study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines (Moher et al., 2015).

Information Sources and Search Strategy

A systematic search was performed in the electronic databases of EBSCOhost (Academic Search Complete & Main Edition, PsycArticles, PsycInfo, Health and Psychosocial Instruments, Psychology and Behavioral Sciences Collection) and PubMed up to April 21, 2023. Population (human) and language (English) search filters were imposed, and year of publication was unbounded. The following search terms were entered, joined by the operator “AND”:

  1. “attribution bias” OR “interpretation bias” OR “cognitive bias” OR “attention bias”

  2. “modification” OR “intervention” OR “training”

  3. “anger” OR “angry” OR “aggression” OR “aggressive” OR “violent” OR “violence” OR “hostile” OR “hostility”

Additionally, reference lists of articles that cited eligible studies were reviewed and authors of included articles and related research were contacted for any unpublished data related to the topic.

Eligibility Criteria

We used the PICOS components (i.e., participants, interventions, comparators, outcomes, and study design (Moher et al., 2015) to refine our research questions and formulate the eligibility criteria. Study inclusion criteria were: (1) original empirical research (i.e., no reviews, theoretical literature, meta-analyses, or case studies); (2) studies assessing the delivery of a CBM (i.e., ABM and/or IBM) intervention using a randomized controlled trial design; (3) studies reporting at least one quantitative outcome measure of anger or aggressive behavior; (4) post-treatment means and standard deviations are available in-text or per request for each outcome variable and treatment condition; and (5) studies are written or translated in English. No limits were imposed on gender, age, or diagnostic status (e.g., non-psychiatric vs. diagnosed with one or more mental disorders) of study participants. Both peer-reviewed journal articles and unpublished dissertations were included in the search and analysis to reduce the effects of publication bias.

Study Selection

Our search identified a total of 181 records as displayed in the PRISMA (Moher et al., 2015) diagram (Figure 1). Database searches were exported to and organized in Excel. To increase objectivity and reduce the likelihood of human error in our study selection process, four trained reviewers (NKC, KHW, LCK, KSW) independently assessed each of the identified records for eligibility, including both title/abstract review and full-text review. After removing duplicate articles, initial screening was performed by each of the four reviewers by reading through titles and abstracts of each identified study. Studies that clearly failed to meet eligibility criteria at this stage were removed from further processing, and the reason for exclusion was noted. Inter-rater reliability (IRR) for title/abstract screening was substantial (κ=.700) among the four raters. Next, all four reviewers performed a full-text review on each of the remaining studies to assess for eligibility, and reasons for exclusion were recorded. IRR for full-text screening was excellent (κ=.879) among the four raters. Discrepancies in eligibility assessments were discussed by the reviewers, and final eligibility determinations were made in all cases by unanimous agreement.

Figure 1.

Figure 1.

PRISMA study screening and selection diagram

Data Extraction and Management

For each study deemed eligible for inclusion, the following information was extracted from the article or requested from the corresponding author: author names, publication year, title, abstract, sample size (for each group, both pre- and post-trial), sample characteristics (e.g., mean age, percentage female in the sample, diagnostic status), anger measures, aggression measures, type of CBM intervention (e.g., IBM vs ABM), detailed information about treatment and control conditions, total treatment duration, frequency and total number of treatment sessions, as well as pre-trial (if available) and post-trial means and standard deviations for anger and aggression measures. For one study otherwise eligible for inclusion (Stoddard et al., 2016), means and standard deviations for outcome measures of interest were inaccessible, thus the study was excluded from the analysis.

As not all studies included pre-trial data on anger and/or aggression, effect sizes for the meta-analyses were conducted in two ways. Standardized mean differences (SMD or Hedge’s G) were calculated using post-trial means for each condition (e.g., CBM vs. control) divided by their pooled standard deviation (Deeks et al., 2008) in order to include a greater pool of studies in the analysis (k=29). Additionally, SMDs (Hedge’s G) were calculated for the subset of studies (k=23) that included pre-trial anger and/or aggression data by calculating the difference between pre- and post-trial means for each condition (e.g., CBM vs. control) divided by their pooled standard deviation in order to enhance sensitivity of treatment-related symptom change. Additionally, standard errors and assigned weights were calculated for each included study for both effect size calculations. All calculations were performed by the first author and confirmed by a co-author. For studies containing multiple anger or aggression measures, effect sizes were calculated for each measure, then averaged to create a single anger or aggression effect size per study, after which standard errors and weights were calculated.

Meta-Analysis

Data were analyzed in R version 4.2.0 using the metafor package and Hedge’s G estimator. Quantitative analyses of the efficacy of CBM for anger and aggression outcomes were conducted using a random effects meta-analytic model, which was utilized due to the heterogeneity of effect sizes across included studies. All outcome measures included in the analysis were scored such that higher scores indicated greater levels of anger or aggression. Separate meta-analyses for the effect of CBM on anger and the effect of CBM on aggression outcomes were conducted for each effect size calculation (i.e., post-trial-only and post-pre-trial effects). Meta-regression analyses were then conducted to determine whether participant-related (i.e., age, gender, level of symptomatology), treatment-related (i.e., CBM type [IBM vs. ABM], treatment duration in weeks, number of treatment sessions), or study-related (risk of bias) factors influenced study findings. Level of symptomatology was operationalized as low (i.e., samples with no requirement of elevated baseline levels of anger and/or aggression symptoms) and high (i.e., samples with an inclusion criterion related to elevated baseline anger and/or aggression levels).

The I2 index was calculated to assess to statistical heterogeneity of pooled effect sizes among included studies (Higgins et al., 2003). The I2 index is derived from Cochran’s Q statistic, which is calculated by summing the squared deviations of the individual study effect sizes from the pooled effect size (Sidik & Jonkman, 2005). The test’s p-value is obtained by comparing the I2 statistic to a χ2 distribution, wherein the null hypothesis holds that all included studies represent the same effect (homogeneity), with variance due to chance.

Risk of Bias

Doi plots (Furuya-Kanamori et al., 2018) were created for each outcome (anger and aggression) to assess the potential influence of publication bias on meta-analytic results. To statistically assess the asymmetry of effect sizes surrounding the overall effect size represented by the Doi plots, the LFK (Luis Fuyura-Kanamori) index was calculated since it has shown to be a more sensitive measure of effect size asymmetry due to publication bias in meta-analyses with small sample sizes (i.e., 5–20 studies) than the commonly used Egger’s Regression test (Furuya-Kanamori et al., 2018).

The revised Cochrane Risk of Bias tool for randomized trials was used to assess study quality and bias (Sterne et al., 2019). Five domains of bias were evaluated for each study: bias arising from the randomization process, bias due to deviations from the intended interventions (with focus on assignment to intervention), bias due to missing outcome data, bias in measurement of the outcome, and bias in selection of the reported result. Four authors (NKC, KHW, LCK, KSW) separately coded each included study across the five domains of bias, and discrepant codes were discussed and resolved by unanimous agreement. IRR was substantial (κ=.655) among the raters.

Results

Study and Sample Characteristics

Following full-text review and resolutions of discrepancies regarding study eligibility among reviewers, 25 randomized controlled trials published online between March 2013 and March 2023 were deemed eligible for inclusion in the meta-analyses. Of the 25 studies, three (Hiemstra et al., 2019; Osgood et al., 2021; Penton-Voak et al., 2013) included multiple (between two and three) studies with unique samples, each eligible for inclusion in the analysis as separate studies. Thus, 29 studies were eligible for inclusion in the final sample. Additionally, two of the 29 studies were unpublished dissertations (Dillon, 2016; Sprunger, 2019). Table 1 includes details about sample characteristics, design, and methodology of the included studies. The overall sample consisted of 2,334 participants with a mean age of 22.93 years (SD=5.14), ranging from 10 to 62 years of age. Participants across studies were drawn from a variety of different populations, including but not limited to undergraduate students (k=10), youth exhibiting high aggression (k=5), youth offenders (k=2), adult violent offenders (k=1), adults exhibiting high trait anger (k=3), heavy drinkers (k=2), individuals experiencing symptoms of depression (k=2), children with neurodevelopmental disorders (k=1), children diagnosed with disruptive mood dysregulation disorder (k=1), schoolchildren (k=2), and healthy adult volunteers (k=4). The type of CBM intervention varied across studies, with 24 studies administering IBM paradigms and 5 studies administering ABM paradigms. One study delivered both ABM and IBM interventions (AlMoghrabi et al., 2022).

Table 1.

Description of included studies (N = 29)

First Author, Year Sample size Mean age % Female Population Attention or interpretation bias Intervention condition Control condition # Treatment sessions Duration (weeks) Outcome measures
AlMoghrabi 2018 80 21.7 50 Undergraduates Interpretation CBM-I (positive training) CBM-I (negative training) 1 0.14 Anger: BPAQ Anger, VAS State Anger
Aggression: BPAQ Physical & Verbal Aggression; TAP
AlMoghrabi 2019 80 20.6 50 Undergraduates Attention Attention Bias Modification Training (positive training) Attention Bias Modification Training (negative training) 1 0.14 Anger: BPAQ Anger, VAS State Anger
Aggression: BPAQ State Physical & Verbal Aggression
AlMoghrabi 2021 80 21.4 50 Undergraduates Interpretation CBM-I (positive training) CBM-I (neutral training) 1 0.14 Anger: BPAQ Anger, VAS State Anger
Aggression: BPAQ State Physical & Verbal Aggression
AlMoghrabi 2022 120 21.1 50 Undergraduates Both CBM-AI (positive training) CBM-AI (neutral training) 1 0.14 Anger: BPAQ Anger, VAS State Anger
Aggression: BPAQ State Physical & Verbal Aggression
Cougle 2017 58 40.7 69.0 Adults with Alcohol Use Disorder and elevated trait anger Interpretation IBM-Hostility Credible Healthy Videos Control 8 4 Anger: STAXI Trait Anger & STAXI Anger Expression Out
Aggression: n/a
Dillon 2016 57 18.9 63.2 Undergraduates high in trait anger Interpretation Interpretation Bias Modification-Hostility Interpretation Bias Modification-Control (non-hostile/social) 4 2 Anger: STAXI Trait Anger, Past-Week Anger
Aggression: Past-week # of disagreements
Haller 2022 44 12.0 43 Disruptive Mood Dysregulation Disorder Interpretation Interpretation Bias Training Interpretation Bias Sham 4 0.57 Anger: Clinical Global Impression-Severity for DMDD
Aggression: n/a
Hawkins 2013 135 18.9 71.1 Undergraduates high in trait anger Interpretation Interpretation Training Control Training (non-threat) 1 0.14 Anger: State Anger during training, during insult, following insult
Aggression: n/a
Hiemstra 2019: Study 1 59 11.8 0 Clinically referred young boys with aggressive behavior problems Interpretation Emotion Recognition Modification Training Emotion Recognition Control Training (no feedback) 5 0.71 Anger: n/a
Aggression: IRPA Total Score, Teacher Tally of Aggression in Class
Hiemstra 2019: Study 2 75 11.5 0 Clinically referred young boys with aggressive behavior problems Interpretation Emotion Recognition Modification Training Emotion Recognition Control Training (no feedback) 3 0.43 Anger: STAXI-CA State Anger
Aggression: n/a
Krans 2014 99 19.2 68.7 Undergraduates Interpretation Interpretation Bias Training (negative views about rumination) Interpretation Bias Training (positive views about rumination) 1 0.14 Anger: APQ Anger
Aggression: APQ Aggressive Action
Kuin 2020 86 39.5 0 Adult male offenders (vast majority violent) Interpretation Emotion Recognition Modification Training Emotion Recognition Control Training (no feedback) 5 0.71 Anger: NAS, OSAB Anger/Irritability
Aggression: Weekly Self-Report of Aggression, OSAB Aggression
Osgood 2021: Study 1 120 35.6 45.8 Healthy adult volunteers Interpretation Hostile Bias Modification Training Placebo Control 1 0.14 Anger: Imagined Anger
Aggression: Imagined Aggression
Osgood 2021: Study 2 217 38.4 97 Healthy adult volunteers Interpretation Hostile Bias Modification Training Placebo Control 1 0.14 Questionnaire - Anger
Aggression: # Aggressive Driving Behaviors
Penton-Voak 2013: Experiment 1 40 22.7 65 Healthy adult volunteers Interpretation Emotion Recognition Modification Training Emotion Recognition Control Training (no feedback) 1 0.14 Anger: STAXI State Anger
Aggression: n/a
Penton-Voak 2013: Experiment 2 46 13.3 28.3 Aggressive adolescents at high risk of criminality Interpretation Emotion Recognition Modification Training Emotion Recognition Control Training (no feedback) 4 0.71 Anger: n/a
Aggression: Self-Reported Aggression, Staff-Reported Aggression
Penton-Voak 2013: Experiment 3 53 24.7 60.4 Healthy adult volunteers Interpretation Visual adaptation No visual adaptation 1 0.14 Anger: STAXI State Anger
Aggression: n/a
Ren 2021 56 17.5 0 Male juvenile delinquents Interpretation CBM-I Waitlist Control 4 4 Anger: BPAQ Anger
Aggression: BPAQ Physical & Verbal Aggression
Schmidt 2021 67 12.2 49.3 Children with neurodevelopmental disorders/special educational needs Interpretation CBM-I Same stimuli, focus on memory/facts 3 3 Anger: n/a
Aggression: RPQ Reactive Aggression
Smith 2018 40 19.3 82.5 Undergraduates currently experiencing a major depressive episode Interpretation Interpretation Bias Modification-Hostility Healthy Videos Control 8 4 Anger: STAXI Trait Anger
Aggression: n/a
Smith 2019 72 19.1 80.6 Undergraduates currently experiencing major depressive episode symptoms Interpretation Interpretation Bias Modification-Depressogenic Healthy Videos Control 4 2 Anger: CAS
Aggression: n/a
Sprunger 2019 28 24.1 42.9 Community sample, couples with recent IPV and heavy drinking Attention Attention Bias Modification Training Placebo Control 1 0.14 Anger: n/a
Aggression: TAP Physical & Verbal Aggression
Van Bockstaele 2020 39 14.0 35.9 Adolescents with teacher-reported high aggression Interpretation Hostile Attribution Bias Modification Retest Control 5 4 Anger: n/a
Aggression: RPQ Total Score
Van Teffelen 2021: Study 2 135 39.2 39 Combination of clinical and community sample with emotion dysregulation Interpretation CBM-I Active Control Training 8 4 Anger: STAXI State Anger
Aggression: FOAS, FOAT, Voodoo Doll Task
Vassilopoulos 2015 34 10.7 11.8 Aggressive children (based on peer and teacher ratings) Interpretation Attribution Training Retest Control 3 1 Anger: n/a
Aggression: Aggression Scale
Vassilopoulos 2022 129 11.2 48.8 5th and 6th grade school children Interpretation Attribution Training Retest Control 3 1 Anger: Imagined anger following vignettes
Aggression: RPQ
Wilkowski 2015 108 19.9 64.8 Undergraduates Attention Attention Cognitive Control-Hostile Attention Cognitive Control-Neutral 1 0.14 Anger: BPAQ Anger
Aggression: BPAQ Physical & Verbal Aggression, TAP
Zeng 2023 121 13.9 53.7 Middle school students Interpretation CBM-I Placebo Control 4 4 Anger: n/a
Aggression: CATQ, RPSQ
Zhao 2022 56 17.6 0 Young male offenders Attention Attention Bias Modification Placebo Control 4 4 Anger: BPAQ Anger
Aggression: BPAQ Physical & Verbal Aggression

Note: APQ = Aggressive Provocation Questionnaire; BPAQ = Buss Perry Aggression Questionnaire; CAS = Clinical Anger Scale; CATQ = Cyber Aggression Typology Questionnaire; CBM-AI = cognitive bias modification-attention/interpretation; CBM-I = cognitive bias modification- interpretation; DMDD = Disruptive Mood Dysregulation Disorder; Forms of Aggression State Questionnaire; FOAT = Forms of Aggression Trait Questionnaire; IPV = Intimate Partner Violence; IRPA = Instrument for Reactive and Proactive Aggression; NAS = Novaco Anger Scale; OSAB = Observation Scale for Aggressive Behavior; RPQ = Reactive/Proactive Aggression Questionnaire; RPSQ = Reactive and Proactive Cyber Aggression Questionnaire; STAXI = State-Trait Anger Expression Inventory; STAXI C/A = STAXI Child/Adolescent; TAP = Taylor Aggression Paradigm; VAS = Visual Analogue Scale

The tasks used for the ABM and IBM interventions varied across studies as well. For example, the IBM interventions included, but were not limited to, facial emotion interpretation modification through shifting of angry-to-happy facial morph balance points (e.g., Hiemstra et al., 2019; Kuin et al., 2020; Penton-Voak et al., 2013), word fragment completion tasks to encourage benign interpretations of scenarios (e.g., Dillon, 2016; Hawkins & Cougle, 2013; Osgood et al., 2021; Van Bockstaele et al., 2020), the Word-Sentence Association Paradigm (WSAP; van Teffelen et al., 2021; Zeng et al., 2023), and ambiguous scenario interpretation training (e.g., AlMoghrabi et al., 2021; Vassilopoulos et al., 2015). ABM interventions included social mishap viewing with instructions to fixate gaze on the part of the image that demonstrates intent of the harm-doer (AlMoghrabi et al., 2022; AlMoghrabi et al., 2019), visual search paradigms of angry and happy/neutral faces (Sprunger, 2019; Zhao et al., 2022), and the Flanker Task with hostile primes more strongly associated with incongruent trials (Wilkowski et al., 2015). Control conditions included but were not limited to placebo control interventions (i.e., tasks identical to the active intervention but providing non-informative and/or non-modification feedback; AlMoghrabi et al., 2022; Haller et al., 2022; Kuin et al., 2020; Osgood et al., 2021; Penton-Voak et al., 2013; Sprunger, 2019; van Teffelen et al., 2021; Zeng et al., 2023), interventions to train the opposite bias as that trained in the CBM condition (AlMoghrabi et al., 2018; Krans et al., 2014), healthy video viewing (Cougle et al., 2017; Smith et al., 2018; Smith et al., 2019), a waitlist control group (Ren et al., 2021), and retest control groups (Van Bockstaele et al., 2020; Vassilopoulos et al., 2015; Vassilopoulos & Brouzos, 2022). Across the full sample, treatment duration ranged from 1 day to 4 weeks, and the total number of treatment sessions ranged from 1 to 8.

Primary Outcome Measures

Anger

Of the 29 studies included in the meta-analysis, 22 (n=2119) included anger1 outcome measures (AlMoghrabi et al., 2022; AlMoghrabi et al., 2021; AlMoghrabi et al., 2018; AlMoghrabi et al., 2019; Cougle et al., 2017; Dillon, 2016; Haller et al., 2022; Hawkins & Cougle, 2013; Hiemstra et al., 2019; Krans et al., 2014; Kuin et al., 2020; Osgood et al., 2021; Penton-Voak et al., 2013; Ren et al., 2021; Smith et al., 2018; Smith et al., 2019; van Teffelen et al., 2021; Vassilopoulos & Brouzos, 2022; Wilkowski et al., 2015; Zhao et al., 2022), and 18 of the 22 studies included pre-trial anger means. Anger measures included the State-Trait Anger Expression Inventory (k=7), Buss Perry Aggression Questionnaire (BPAQ)-Trait Anger subscale (k=7), visual analogue scale of state anger (k=4), Clinical Anger Scale (k=1), state anger ratings (k=1), Aggressive Provocation Questionnaire (APQ)-Anger subscale (k=1), Novaco Anger Scale (k=1), Observation Scale for Aggressive Behavior (OASB)-Anger/Irritability subscale (k=1), imagined anger following vignettes (k=2), State Emotion Questionnaire-Anger subscale (k=1), and Clinical Global Impression-Severity for Disruptive Mood Dysregulation Disorder (k=1, for which scores were determined by clinicians blind to experimental condition). Table 2 and Figure 2 present effect sizes and meta-analytic results for the 22 studies reporting at least one anger outcome. Random effects meta-analysis of post-trial anger means (N=22) demonstrated a nonsignificant trend for CBM outperforming control conditions (mean Hedge’s G=0.13,95%CI[0.26,0.00], p=.050). Random effects meta-analysis accounting for the difference between pre- and post-trial means (N=18) demonstrated that CBM significantly outperformed control conditions in reducing anger (mean Hedge’s G=0.18,95%CI[0.28,0.07], p<.001).

Table 2.

Summary effect sizes, measures of heterogeneity, moderators, and bias (N = 29)

Post-Treatment-Only Effect Size Anger Aggression

Number of studies 22 21
Number of participants 2119 1817
Random effect: Hedge’s G [95% CI]* −0.13 [−0.26, 0.00] −0.24 [−0.38, −0.11]
Heterogeneity: I2 46.96% (p = 0.002) 46.37% (p = 0.03)
Moderation effects: Hedge’s G [95% CI]*
 Age 0.01 [−0.00, 0.02] 0.01 [−0.01, 0.02]
 Percent female in sample 0.00 [−0.00, 0.01] −0.00 [−0.01, 0.01]
 Level of Symptomatology −0.03 [−0.31, 0.25] −0.07 [−0.21, 0.35]
 Interpretation vs. Attention Bias 0.06 [−0.28, 0.40] 0.35 [0.07, 0.63]
 Treatment duration −0.03 [−0.12, 0.06] −0.02 [−0.11, 0.06]
 Number of treatment sessions −0.02 [−0.07, 0.04] −0.00 [−0.07, 0.07]
 Risk of Bias 0.03 [−0.28, 0.33] −0.06 [−0.37, 0.26]
LFK Index −1.51 (minor asymmetry) −0.68 (no asymmetry)

Post-Pre-Treatment Effect Size Anger Aggression

Number of studies 18 15
Number of participants 1538 1186
Random effect: Hedge’s G [95% CI]* −0.18 [−0.27, −0.07] −0.23 [−0.35, −0.11]
Heterogeneity: I2 30.96% (p = 0.17) 5.30% (p = 0.39)
Moderation effects: Hedge’s G [95% CI]*
 Age 0.01 [−0.00, 0.02] −0.01 [−0.01, 0.02]
 Percent female in sample 0.00 [−0.00, 0.01] 0.00 [−0.01, 0.01]
 Level of Symptomatology −0.03 [−0.24, 0.18] 0.00 [−0.24, 0.24]
 Interpretation vs. Attention Bias 0.14 [−0.15, 0.43] 0.16 [−0.14, 0.46]
 Treatment duration 0.01 [−0.06, 0.07] −0.05 [−0.12, 0.02]
 Number of treatment sessions −0.01 [−0.05, 0.04] −0.03 [−0.08, 0.03]
 Risk of Bias −0.01 [−0.23, 0.22] −0.08 [−0.117, 0.33]
LFK Index −0.86 (no asymmetry) 0.39 (no asymmetry)

Note: Findings that emerged as significant (p < .05) are bolded; findings that were trending nonsignificant (p < .10) are italicized; CI = confidence interval; LFK = Luis Fuyura-Kanamori

*

Hedge’s G effect size interpretation: 0.2 (small), 0.5 (medium), 0.8 (large)

Figure 2.

Figure 2.

Random effect of cognitive bias modification on anger

Aggression

Twenty one (n=1817) of the 29 studies included in the meta-analysis reported quantitative measures of aggressive behavior, or these data were available upon request (AlMoghrabi et al., 2022; AlMoghrabi et al., 2021; AlMoghrabi et al., 2018; AlMoghrabi et al., 2019; Dillon, 2016; Hiemstra et al., 2019; Krans et al., 2014; Kuin et al., 2020; Osgood et al., 2021; Penton-Voak et al., 2013; Ren et al., 2021; Schmidt & Vereenooghe, 2021; Sprunger, 2019; Van Bockstaele et al., 2020; van Teffelen et al., 2021; Vassilopoulos et al., 2015; Vassilopoulos et al., 2022; Wilkowski et al., 2015; Zeng et al., 2023; Zhao et al., 2022), and 15 of the studies included pre-trial aggression means. Aggression measures included BPAQ Verbal and Physical Aggression subscales (k=7), staff-rated aggression (k=2), Reactive/Proactive Aggression Questionnaire (k=3), Taylor Aggression Paradigm (k=3), past-week number of disagreements (k=1), Instrument for Reactive and Proactive Aggression (k=1), teacher tally of aggression (k=1), APQ-Aggressive Action subscale (k=1), Cyber Aggression Typology Questionnaire (k=1), Reactive and Proactive Cyber Aggression Questionnaire (k=1), weekly self-report of aggression (k=2), OASB-Aggression subscale (k=1), imagined aggression following vignettes (k=1), self-reported aggressive driving (k=1), staff-reported aggression (k=1), the Aggression Scale (k=1), Forms of Aggression State and Trait Aggression Questionnaires (k=1), and the Voodoo Doll Task (k=1). Table 2 and Figure 3 present effect sizes and meta-analytic results for the 21 studies reporting at least one aggression outcome. Random effects meta-analysis of post-trial aggression means (N=21) demonstrated that CBM significantly outperformed control conditions (mean Hedge’s G=0.24,95%CI[0.38,0.11], p<.001). Random effects meta-analysis accounting for the difference between pre- and post-trial means (N=15) also demonstrated that CBM significantly outperformed control conditions in reducing aggression (mean Hedge’s G=0.23,95%CI[0.35,0.11], p<.001).

Figure 3.

Figure 3.

Random effect of cognitive bias modification on aggression

Risk of Bias

Visual examination of the Doi plots (Figure 4) and overall pattern of effect sizes suggest a somewhat asymmetrical distribution for anger with the post-treatment-only effect size. The Doi plot for the post-pre-treatment effect sizes for anger, as well as both Doi plots for the aggression outcome, did not suggest asymmetry. These findings were supported by the LFK index, which demonstrated no asymmetry for studies with aggression outcomes (LFK indices = −0.68 and 0.39) as well as studies reporting pre-trial means for anger (LFK index = −0.86), whereas the LFK index demonstrated minor asymmetry for studies reporting only post-trial means for anger outcomes (LFK index = −1.51). These results suggest that the overall effect sizes for the aggression meta-analyses are likely adequate estimators of the true effect sizes in the population, whereas there is evidence of potential minor publication bias for the studies included in the post-treatment-only anger meta-analysis.

Figure 4.

Figure 4.

Doi plots for publication bias in random effects estimates of anger and aggression

As all studies included in the present meta-analyses were randomized controlled trials, the Cochrane RoB 2 tool was used to assess study quality and risk of bias across each of the included studies. For the meta-analysis of anger outcomes, six studies were judged as ‘low risk’ (i.e., all five domains were rated as ‘low risk’), 16 studies were judged by raters as having ‘some concerns’ (i.e., between 1–3 of the five domains were rated as ‘some concerns’, the remaining were ‘low risk’), and no studies were rated as being ‘high risk’. For the meta-analysis of aggression outcomes, five studies were deemed as ‘low risk’, 15 studies were rated as having ‘some concerns’, and one study was deemed to be ‘high risk’ (i.e., one of five domains was rated as ‘high risk’, one domain was rated as ‘some concerns’, and the remaining three domains were rated as ‘low risk’). The study rated as ‘high risk’ of bias (Sprunger, 2019) was given this designation because there was substantial dropout post-randomization and only the completer sample was analyzed, rather than using an intent-to-treat analysis to correct for bias due to attrition post-randomization. All other domains for this study were rated as ‘low risk’ or ‘some concerns’. See Supplement for raters’ risk of bias judgments across all five domains for each study included in the analyses.

Effect Size Heterogeneity

The I2 Index and Cochran’s Q-Statistic were calculated to assess the degree of effect size heterogeneity across studies. Study effects calculated using post-trial-only means demonstrated significant heterogeneity for both anger (I2=46.96%, Q[df=21:44.47,p=.002,τ2=0.04), and aggression (I2=46.37%, Q[df=20]:33.90,p=.027,τ2=0.04). However, study effects calculated using post-pre-trial means did not demonstrate significant heterogeneity for either anger (I2=30.96%, Q[df=17]:22.26,p=.175,τ2=0), or aggression (I2=5.30%, Q[df=14]:14.78,p=.39,τ2=0). Of note, for the I2 and Q statistics, a small average sample size (N¯<80) and/or number of studies (k<20) results in low statistical power (β<0.8; Huedo-Medina et al., 2006). As such, for the present study, I2 and Q statistics calculated for both post-pre-trial effect sizes (anger: k=18,N¯=85; aggression: k=15;N¯=79) are likely underpowered due to the availability and nature of studies meeting inclusion criteria. I2 and Q statistics calculated for both post-trial-only effect sizes (anger: k=22,N¯=88; aggression: k=21,N¯=87) are considered sufficiently powered.

Meta-Regression Analyses

As effect size heterogeneity was significant for both anger and aggression outcomes for the higher-powered but lower sensitivity effect size (post-trial-only), meta-regression analyses were run for these effects. Treatment type (ABM vs. IBM) significantly moderated the effects of CBM on aggression outcomes (SMD=0.35,95%CI[0.07,0.63]; p=.013). Follow up meta-analyses of studies reporting aggression outcomes with only ABM interventions (k=4) and those reporting only IBM interventions (k=16) demonstrated that IBM-only interventions significantly outperformed control conditions in the treatment of aggression (mean Hedge’s G=0.27,95%CI[0.43,0.12], p<.001), whereas ABM-only interventions did not outperform controls (mean Hedge’s G=0.03,95%CI[0.19,0.26], p=.763). Treatment type did not significantly moderate the effect of CBM on anger outcomes. The moderating effects of participant variables (i.e., mean participant age, percent female in the sample, level of symptomatology), other treatment variables (i.e., number of treatment sessions and treatment duration in weeks), and study quality (i.e., risk of bias) were non-significant (all ps>.05) for both anger and aggression outcomes.

Effect size heterogeneity was nonsignificant for both anger and aggression outcomes for the lower-powered but higher sensitivity effect size (post-pre-trial), thus meta-regressions were run as exploratory analyses. No variables (participant-related, treatment-related, or study quality) significantly moderated overall anger or aggression meta-analytic findings (all ps>.05) for the post-pre-trial effects. Table 2 presents all meta-regression results.

Discussion

The present meta-analyses synthesized results across 29 randomized controlled trials published between 2013 and 2023 to assess the overall impact of CBM interventions on anger and aggressive behavior. Meta-analytic effect sizes were calculated in two ways to maximize statistical power as well as sensitivity to treatment-related symptom changes.

Consistent with our hypotheses, results of both the higher-powered (post-trial-only) and the higher sensitivity (post-pre-trial) meta-analytic effect sizes demonstrate that CBMs outperform control conditions in the treatment of aggressive behavior, supporting the overall efficacy of CBM as an aggression treatment. However, these results should be interpreted keeping in mind that the meta-analytic effect sizes were small (−0.24 and −0.23) with Figure 3 demonstrating that only 5/21 and 3/15 studies respectively showed significant intervention effects, limiting clinical utility of CBM as a standalone treatment for aggression.

Meta-regression analyses demonstrated that participant age, gender, and level of clinical symptoms at baseline did not influence meta-analytic results for aggression outcomes, suggesting that CBMs appear to be efficacious in treating aggressive behavior independent of these participant factors. In contrast, CBM type (i.e., IBM vs. ABM) did significantly moderate the effects of CBM on aggression outcomes, with IBM (but not ABM) interventions outperforming control conditions in the treatment of aggressive behavior. However, this was only true for the higher-powered but lower sensitivity effect size calculation (post-trial-only), whereas the lower-powered but higher sensitivity effects (post-pre-trial) demonstrated no significant differences between the efficacy of ABMs and IBMs for aggression. Notably, this analysis only included two (Sprunger, 2019; Wilkowski et al., 2015) of the four ABM trials that reported aggression outcomes and thus was very underpowered. As such, while ABMs appear to be effective in reducing other clinical symptoms similarly associated with an amplified attentional bias toward threatening information, such as clinical anxiety (Linetzky et al., 2015), it remains unclear whether ABMs alone produce the same effect on aggression. Study quality and other treatment variables (i.e., duration in weeks, number of treatment sessions) did not moderate meta-analytic findings, demonstrating that CBM type (IBM vs. ABM) was the sole factor assessed impacting the overall efficacy of CBMs for aggression. This is consistent with a previous meta-analysis on the efficacy of CBM for the treatment of anxiety and depression, which demonstrated that IBMs had a significantly stronger effect on cognitive biases than did ABMs (Hallion & Ruscio, 2011). Multiple factors may explain why ABMs fail to produce the same effect on aggression as IBMs. First, the sample of studies administering ABM-only interventions and reporting aggression outcomes was very small (k=4) and thus likely underpowered to detect true effects compared to studies involving IBM-only interventions (k=16). Additionally, two (Wilkowski et al., 2015; Zhao et al., 2022) of the four studies in the ABM analysis demonstrated that though there were no significant overall effects on aggression, significant interactions between baseline cognitive biases and treatment-related aggression changes were found, such that only those with high baseline cognitive biases exhibited significant reductions in aggression post-ABM treatment, suggesting that the efficacy of ABM may depend on the severity of attentional bias at baseline. Relatedly, mediation analyses conducted in studies of ABM for anxiety (e.g., Kuckertz et al., 2014) have repeatedly demonstrated that only participants who experience reductions in attention bias from pre- to post-treatment exhibit treatment-related symptom changes (i.e., social anxiety), suggesting that ABM may work indirectly on behavior only if it successfully reduces attentional bias. Considering the small sample of ABMs and the inability of the present study to assess baseline cognitive bias and/or treatment-related changes in cognitive biases, more research is needed to evaluate the efficacy of ABM for aggression and for which groups, and under which conditions, it is most efficacious. Overall, these results support the notion of CBM, particularly IBM, as an efficacious treatment demonstrating small but significant effects on aggression, and highlight the importance of hostile interpretation bias as a treatment target for aggressive behavior (Klein Tuente et al., 2019; Martinelli et al., 2018).

The present meta-analytic findings provide limited support for the efficacy of CBM in the treatment of anger, also consistent with hypotheses. Only the higher sensitivity/lower powered effect (post-pre-trial) was statistically significant (p<.001) for the anger outcome, whereas the lower sensitivity/higher powered effect (post-trial-only) was trending nonsignificant (p=.050). Additionally, both effect size calculations demonstrated small effects (−0.18 and −0.13 respectively) for CBM’s efficacy in the treatment of anger. These findings suggest that the treatment has small but significant effects only when accounting for baseline anger. The overall effect of CBM on anger outcomes was not moderated by any variable probed in the meta-regression analyses, namely number of treatment sessions administered, duration of treatment, CBM type (ABM vs IBM), age of participants, percent female in the sample, level of clinical symptoms as baseline, or study quality. The small but significant efficacy of CBM in reducing anger from pre- to post-treatment is consistent with research demonstrating the prevalence of cognitive biases among anger-prone populations (e.g., Mellentin et al., 2015) and the efficacy of CBMs in treating aggression that is specifically reactive, or anger-based, in nature (e.g., Schmidt & Vereenooghe, 2021; Van Bockstaele et al., 2020; Zeng et al., 2023). An important caveat to these findings is that effect sizes of −0.13 and −0.18 are small and may not be clinically meaningful, which is supported by the fact that only 4/22 and 3/18 studies, respectively, showed significant intervention effects as evidenced by the 95% confidence intervals in Figure 2. Thus, current CBMs also may not be best suited as a standalone treatment for anger, rather potentially a low-cost adjunct to enhance ongoing treatment (e.g., psychotherapy).

Aside from CBM type (ABM vs IBM) significantly moderating treatment efficacy for aggression outcomes, all other meta-regression analyses returned nonsignificant. Regarding participant variables, our findings demonstrate that CBMs appear to be similarly efficacious regardless of age, gender, and level of anger/aggression symptomatology, which corroborates a 2015 review of meta-analyses on CBMs for a range of disorders reporting that most meta-analyses on the topic demonstrated that CBM efficacy was not moderated by participant age, gender, and level of clinical symptoms (Jones & Sharpe, 2017). The present findings are promising as they suggest that CBMs can be applied to a wide range of populations in need of intervention for aggressive behavior and anger, with minimal variation in efficacy across different demographic and clinical groups. Additionally, despite substantial range in treatment duration (1 day to 4 weeks) and number of treatment sessions (1 to 8 sessions), neither of these variables significantly moderated study effects, suggesting that lower doses of CBM (i.e., shorter duration and/or fewer sessions) have similar efficacy for the treatment of anger and aggression as do higher doses of CBM. Thus, according to the present results, even single-session CBMs may produce a significant albeit small remediating effect on aggression and anger, which may serve as a useful low-cost and low-burden adjunctive intervention for individuals across a range of settings who may benefit from such additional support where it is typically limited or unavailable (e.g., juvenile or adult correctional facilities, schools, hospitals) or as addition to ongoing care (e.g., outpatient psychotherapy). Finally, contrary to hypotheses, study methodological rigor did not significantly moderate meta-analytic results. This is surprising given that previous meta-analytic reviews of CBM for other disorders (e.g., Cristea et al., 2016) have shown that higher risk of bias is associated with higher effect sizes. The present review only contained one study with a high risk of bias, whereas the majority (k=21) were rated as having ‘some concerns’ and the remaining (k=6) were rated as ‘low risk’, thus there may not have been enough variability across the three categories of bias level to detect significant moderation effects. Regardless, researchers should continue to conduct CBM RCTs in accordance with high methodological standards (e.g., trial pre-registration, double-blindness if possible, intent-to-treat analysis) to enhance confidence in study findings.

The present findings should be interpreted considering the study’s limitations. First, only five studies contained ABM interventions, limiting our ability to investigate with sufficient power the efficacy of ABM specifically for the treatment of anger and aggression. Additionally, though several studies found significant moderating effects of baseline variables such as hostile interpretation bias (Ren et al., 2021; Wilkowski et al., 2015) and attention bias (Zhao et al., 2022), it was not possible to include these variables in the present meta-regression analyses due to the heterogeneity in measures used across studies and inability to standardize across such measures. Moreover, there was substantial variability across interventions that qualified as IBM and ABM (e.g., IBMs using words vs. faces vs. other pictures), and the numbers of these different variants of IBM and ABM were too small to statistically analyze as potential moderators of treatment outcome. Relatedly, sample type (e.g., prisoners, undergraduates, children) varied across included studies; however, these subgroups were also too small to statistically analyze for potential moderating effects which limited investigation of which populations CBM may be most helpful.

Despite these limitations, the present study also had some noteworthy strengths. Meta-analytic effect sizes were calculated in two ways to maximize both statistical power and sensitivity to treatment-related symptom changes, enhancing overall rigor of the study. Relatedly, meta-analytic results between the two effect size calculations largely replicated one another for each outcome of interest (i.e., anger and aggression), demonstrating robustness of findings. Quality of each included study was evaluated by four independent raters using the Cochrane Risk of Bias Tool for Randomized Controlled Trials (Sterne et al., 2019), and the impact of study quality on meta-analytic findings was probed via meta-regression analyses. All included studies, apart from one, were deemed to be ‘low risk’ or ‘some concerns’ of bias, demonstrating moderate to high quality of studies included in the analyses and enhancing confidence in study findings. Moreover, we assessed for heterogeneity in effect sizes for each meta-analysis conducted and investigated several important variables for potential moderating effects on meta-analytic findings, including participant characteristics, features of the CBM treatments, and study quality. Finally, both published (peer-reviewed journal articles; k=27) and unpublished (dissertations; k=2) research was included in the present study, reducing the effects of publication bias on the results. Moreover, authors of included studies and researchers conducting related work were contacted for any unpublished data related to the topic, thus the present results generally reflect the current status of research (regardless of publication status) on CBMs for the treatment of anger and aggression.

Results of the present study highlight directions for future research. Very few included studies (Cougle et al., 2017; Dillon, 2016; Haller et al., 2022; Kuin et al., 2020; Penton-Voak et al., 2013; Zeng et al., 2023) assessed long-term impacts of CBM on the outcomes of interest, thus more longitudinal work is needed to evaluate the long-term effects of CBM on anger and aggression outcomes. Additionally, future investigations of the efficacy of CBM would continue to benefit from quantifying baseline cognitive-affective variables (such as hostile interpretation bias and attentional bias), potentially using consistent measures across studies, such that moderating effects of these baseline variables on treatment outcome can be probed across studies. This would allow future research to assess which populations might benefit most from CBM interventions. Moreover, additional studies are needed to evaluate the effect of ABM interventions for aggression and anger given that the present sample only contained five studies. Finally, increased consistency across studies in interventions (e.g., WSAP, facial emotion interpretation, word fragment completion as IBMs) used for the CBM treatment may lead to more consistent findings regarding the effect of CBM on anger and aggressive behavior.

The present findings support the notion of CBM as a promising treatment—particularly as a supplement to ongoing care—for aggressive behavior, and to a lesser extent, anger, across a variety of samples and interventions. The only significant moderator observed in our analyses was the type of CBM intervention in the treatment of aggression, such that interventions that employed IBM appeared to significantly outperform control conditions at post-treatment. However, more research on ABMs specifically is needed to assess whether this finding is due to a true effect or the underpowered nature of ABM meta-analysis. The overall significant effects of CBM on aggressive behavior and anger demonstrate the importance of continued exploration of the efficacy of this treatment for these outcomes specifically and for emotional/behavioral dysregulation more broadly.

Highlights.

  • Aggression and dysregulated anger pose public health and economic costs to society.

  • Cognitive bias modification may be an effective treatment for these difficulties.

  • Meta-analyses were performed to assess CBM’s efficacy for anger and aggression.

  • CBM demonstrates efficacy for the treatment of both aggression and anger.

Role of Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Footnotes

Declarations of Interest

The authors declare no conflict of interest.

Review Registration and Protocol

This review was not registered, and a protocol was not prepared.

Submission Declaration and Verification

All authors reviewed and approved the submitted manuscript. The article is the authors’ original work, has not received prior publication and is not under consideration for publication elsewhere.

1

This also included measures of irritability (k=2), as despite some conceptual differences, the terms are often used interchangeably. Analyses excluding studies reporting irritability outcomes did not change the overall pattern of results; thus, irritability measures were included in the overall anger meta-analyses.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Availability of Data, Code, and Other Materials

No materials included in this review are currently publicly available.

References

  1. AlMoghrabi N, Franken IHA, Mayer B, & Huijding J (2022). A Single-Session Combined Cognitive Bias Modification Training Targeting Attention and Interpretation Biases in Aggression. Behaviour Change, 39(1), 1–20. [Google Scholar]
  2. AlMoghrabi N, Franken IHA, Mayer B, van der Schoot M, & Huijding J (2021). CBM-I Training and Its Effect on Interpretations of Intent, Facial Expressions, Attention and Aggressive Behavior. Europe’s Journal of Psychology, 17(2), 13–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. AlMoghrabi N, Huijding J, & Franken I (2018). The effects of a novel hostile interpretation bias modification paradigm on hostile interpretations, mood, and aggressive behavior. Journal of Behavior Therapy and Experimental Psychiatry, 58, 36–42. [DOI] [PubMed] [Google Scholar]
  4. AlMoghrabi N, Huijding J, Mayer B, & Franken I (2019). Gaze-contingent Attention Bias Modification Training and its Effect on Attention, Interpretations, Mood, and Aggressive Behavior. Cognitive Therapy and Research, 43(5), 861–873. [Google Scholar]
  5. Amir N, Beard C, Burns M, & Bomyea J (2009). Attention modification program in individuals with generalized anxiety disorder. J Abnorm Psychol, 118(1), 28–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Amir N, Beard C, Taylor CT, Klumpp H, Elias J, Burns M, & Chen X (2009). Attention training in individuals with generalized social phobia: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 77(5), 961–973. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Amir N, Weber G, Beard C, Bomyea J, & Taylor CT (2008). The effect of a single-session attention modification program on response to a public-speaking challenge in socially anxious individuals. Journal of Abnormal Psychology, 117(4), 860–868. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Barlow MA, Wrosch C, Gouin J-P, & Kunzmann U (2019). Is anger, but not sadness, associated with chronic inflammation and illness in older adulthood? Psychology and Aging, 34(3), 330–340. [DOI] [PubMed] [Google Scholar]
  9. Beard C, & Amir N (2008). A multi-session interpretation modification program: changes in interpretation and social anxiety symptoms. Behaviour Research and Therapy, 46(10), 1135–1141. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Beard C, Sawyer AT, & Hofmann SG (2012). Efficacy of attention bias modification using threat and appetitive stimuli: A meta-analytic review. Behavior Therapy, 43(4), 724–740. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bubier JL, & Drabick DAG (2009). Co-occurring anxiety and disruptive behavior disorders: The roles of anxious symptoms, reactive aggression, and shared risk processes. Clinical Psychology Review, 29(7), 658–669. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. CDC. (2021). Community Violence Prevention. Center for Disease Control. https://www.cdc.gov/violenceprevention/communityviolence/ [Google Scholar]
  13. Chan SC, Raine A, & Lee TM (2010). Attentional bias towards negative affect stimuli and reactive aggression in male batterers. Psychiatry Research, 176(2–3), 246–249. [DOI] [PubMed] [Google Scholar]
  14. Ciesinski NK, Sorgi-Wilson KM, Cheung JC, Chen EY, & McCloskey MS (2022). The effect of dialectical behavior therapy on anger and aggressive behavior: A systematic review with meta-analysis. Behaviour Research and Therapy, 154, 104122. [DOI] [PubMed] [Google Scholar]
  15. Coccaro EF, McCloskey MS, Fitzgerald DA, & Phan KL (2007). Amygdala and orbitofrontal reactivity to social threat in individuals with impulsive aggression. Biological Psychiatry, 62(2), 168–178. [DOI] [PubMed] [Google Scholar]
  16. Coccaro EF, Noblett KL, & McCloskey MS (2009). Attributional and emotional responses to socially ambiguous cues: validation of a new assessment of social/emotional information processing in healthy adults and impulsive aggressive patients. Journal of Psychiatry Research, 43(10), 915–925. [DOI] [PubMed] [Google Scholar]
  17. Coles ME, Heimberg RG, & Schofield CA (2008). Interpretation of facial expressions and social anxiety: Specificity and source of biases. Cognition and Emotion, 22(6). [Google Scholar]
  18. Conger RD, Neppl T, Kim KJ, & Scaramella L (2003). Angry and aggressive behavior across three generations: a prospective, longitudinal study of parents and children. J Abnormal Child Psychology, 31(2), 143–160. [DOI] [PubMed] [Google Scholar]
  19. Corso PS, Mercy JA, Simon TR, Finkelstein EA, & Miller TR (2007). Medical costs and productivity losses due to interpersonal and self-directed violence in the United States. American Journal of Preventative Medicine, 32(6), 474–482. [DOI] [PubMed] [Google Scholar]
  20. Cougle JR, Summers BJ, Allan NP, Dillon KH, Smith HL, Okey SA, & Harvey AM (2017). Hostile interpretation training for individuals with alcohol use disorder and elevated trait anger: A controlled trial of a web-based intervention. Behaviour Research and Therapy, 99, 57–66. [DOI] [PubMed] [Google Scholar]
  21. Crick NR, & Dodge KA (1994). A review and reformulation of social information-processing mechanisms in children’s social adjustment. Psychological Bulletin,115,74–91. [Google Scholar]
  22. Cristea I, Kok R, & Cuijpers P (2015). Efficacy of cognitive bias modification interventions in anxiety and depression: Meta-analysis. The British Journal of Psychiatry, 206(1), 7–16. [DOI] [PubMed] [Google Scholar]
  23. Cristea IA, Kok RN, & Cuijpers P (2016). The Effectiveness of Cognitive Bias Modification Interventions for Substance Addictions: A Meta-Analysis. PLOS ONE, 11(9), e0162226. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Deeks JJ, Higgins JPT, & Altman DG (2008). Analysing Data and Undertaking Meta-Analyses. Cochrane Handbook for Systematic Reviews of Interventions, 243–296. [Google Scholar]
  25. Dillon KH (2016). Evaluation of a computerized multi-session interpretation modification program for the treatment of anger. APA PsycInfo. [Google Scholar]
  26. Dillon KH, Medenblik AM, Mosher TM, Elbogen EB, Morland LA, & Beckham JC (2020). Using Interpretation Bias Modification to Reduce Anger in Veterans with Posttraumatic Stress Disorder: A Pilot Study. Journal of Traumatic Stress. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Dodge KA, Price JM, Bachorowski JA, & Newman JP (1990). Hostile attributional biases in severely aggressive adolescents. Journal of Abnormal Psychology,99(4),385–92. [DOI] [PubMed] [Google Scholar]
  28. FBI. (2022). Violent Crime. Federal Bureau of Investigation. https://www.fbi.gov/investigate/violent-crime [Google Scholar]
  29. Fodor LA, Georgescu R, Cuijpers P, Szamoskozi Ş, David D, Furukawa TA, & Cristea IA (2020). Efficacy of cognitive bias modification interventions in anxiety and depressive disorders: a systematic review and network meta-analysis. The Lancet Psychiatry, 7(6), 506–514. [DOI] [PubMed] [Google Scholar]
  30. Furuya-Kanamori L, Barendregt JJ, & Doi SAR (2018). A new improved graphical and quantitative method for detecting bias in meta-analysis. International Journal of Evidence Based Healthcare, 16(4), 195–203. [DOI] [PubMed] [Google Scholar]
  31. Griffin BJ, Purcell N, Burkman K, Litz BT, Bryan CJ, . . . Maguen S. (2019). Moral Injury: An Integrative Review. Journal of Traumatic Stress, 32(3), 350–362. [DOI] [PubMed] [Google Scholar]
  32. Haller SP, Stoddard J, Botz-Zapp C, Clayton M, MacGillivray C, Perhamus G, . . . Brotman MA (2022). A Randomized Controlled Trial of Computerized Interpretation Bias Training for Disruptive Mood Dysregulation Disorder: A Fast-Fail Study. Journal of the American Academy of Child & Adolescent Psychiatry, 61(1), 37–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Hallion LS, & Ruscio AM (2011). A meta-analysis of the effect of cognitive bias modification on anxiety and depression. Psychological Bulletin, 137(6), 940–958. [DOI] [PubMed] [Google Scholar]
  34. Harburg E, Julius M, Kaciroti N, Gleiberman L, & Schork MA (2003). Expressive/suppressive anger-coping responses, gender, and types of mortality: a 17-year follow-up (Tecumseh, Michigan, 1971–1988). Psychosomatic Medicine, 65(4), 588–597. [DOI] [PubMed] [Google Scholar]
  35. Hawkins KA, & Cougle JR (2013). Effects of Interpretation Training on Hostile Attribution Bias and Reactivity to Interpersonal Insult. Behavior Therapy, 44(3), 479–488. [DOI] [PubMed] [Google Scholar]
  36. Hiemstra W, De Castro BO, & Thomaes S (2019). Reducing Aggressive Children’s Hostile Attributions: A Cognitive Bias Modification Procedure. Cognitive Therapy and Research, 43(2), 387–398. [Google Scholar]
  37. Higgins JPT, Thompson SG, Deeks JJ, & Altman DG (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Huedo-Medina TB, Sánchez-Meca J, Marín-Martínez F, & Botella J (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods, 11(2), 193–206. [DOI] [PubMed] [Google Scholar]
  39. Jones EB, & Sharpe L (2017). Cognitive bias modification: A review of meta-analyses. Journal of Affective Disorders, 223, 175–183. [DOI] [PubMed] [Google Scholar]
  40. Kessler RC, Coccaro EF, Fava M, Jaeger S, Jin R, & Walters E (2006). The prevalence and correlates of DSM-IV intermittent explosive disorder in the National Comorbidity Survey Replication. Archives of General Psychiatry, 63(6), 669–678. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Kjaergard LL, Villumsen J, & Gluud C (2001). Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine, 135(11), 982–989. [DOI] [PubMed] [Google Scholar]
  42. Klein Tuente S, Bogaerts S, & Veling W (2019). Hostile attribution bias and aggression in adults - a systematic review. Aggression and Violent Behavior, 46, 66–81. [Google Scholar]
  43. Krans J, Moulds ML, Grisham JR, Lang TJ, & Denson TF (2014). Evaluating the Effect of Meta-Cognitive Beliefs about Angry Rumination on Anger with Cognitive Bias Modification. Journal of Experimental Psychopathology, 5(3), 259–271. [Google Scholar]
  44. Krug EG, Mercy JA, Dahlberg LL, & Zwi AB (2002). The world report on violence and health [Review]. Lancet, 360(9339), 1083–1088. [DOI] [PubMed] [Google Scholar]
  45. Kuckertz JM, Gildebrant E, Liliequist B, Karlström P, Väppling C, Bodlund O, . . . Carlbring P. (2014). Moderation and mediation of the effect of attention training in social anxiety disorder. Behaviour Research and Therapy, 53, 30–40. [DOI] [PubMed] [Google Scholar]
  46. Kuin NC, Masthoff EDM, Nunnink VN, Munafò MR, & Penton-Voak IS (2020). Changing perception: A randomized controlled trial of emotion recognition training to reduce anger and aggression in violent offenders. Psychology of Violence, 10, 400–410. [Google Scholar]
  47. Linehan MM (1993). Cognitive-behavioral treatment of borderline personality disorder. Guilford Press. [Google Scholar]
  48. Linetzky M, Pergamin-Hight L, Pine DS, & Bar-Haim Y (2015). Quantitative evaluation of the clinical efficacy of attention bias modification treatment for anxiety disorders. Depression and Anxiety, 32(6), 383–391. [DOI] [PubMed] [Google Scholar]
  49. Maoz K, Adler AB, Bliese PD, Sipos ML, Quartana PJ, & Bar-Haim Y (2017). Attention and interpretation processes and trait anger experience, expression, and control. Cognition and Emotion, 31(7), 1453–1464. [DOI] [PubMed] [Google Scholar]
  50. Martinelli A, Ackermann K, Bernhard A, Freitag CM, & Schwenck C (2018). Hostile attribution bias and aggression in children and adolescents: A systematic literature review on the influence of aggression subtype and gender. Aggression and Violent Behavior, 39, 25–32. [Google Scholar]
  51. Mellentin AI, Dervisevic A, Stenager E, Pilegaard M, & Kirk U (2015). Seeing enemies? A systematic review of anger bias in the perception of facial expressions among anger-prone and aggressive populations. Aggression and Violent Behavior, 25, 373–383. [Google Scholar]
  52. Menne-Lothmann C, Viechtbauer W, Höhn P, Kasanova Z, Haller SP,. . . Lau, J. (2014). How to Boost Positive Interpretations? A Meta-Analysis of the Effectiveness of Cognitive Bias Modification for Interpretation. PLOS ONE, 9(6), e100925. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Miller NV, & Johnston C (2019). Social threat attentional bias in childhood: Relations to aggression and hostile intent attributions. Aggressive Behavior, 45(3), 245–254. [DOI] [PubMed] [Google Scholar]
  54. Mogoaşe C, David D, & Koster EHW (2014). Clinical Efficacy of Attentional Bias Modification Procedures: An Updated Meta-Analysis. Journal of Clinical Psychology, 70(12), 1133–1157. [DOI] [PubMed] [Google Scholar]
  55. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, . . . Group, P.-P. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4, 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Osgood JM, Kase SE, Zaroukian EG, & Quartana PJ (2021). Online Intervention Reduces Hostile Attribution Bias, Anger, Aggressive Driving, and Cyber-Aggression, Results of Two Randomized Trials. Cognitive Therapy and Research, 45(2), 310–321. [Google Scholar]
  57. Penton-Voak IS, Thomas J, Gage SH, McMurran M, McDonald S, & Munafò MR (2013). Increasing recognition of happiness in ambiguous facial expressions reduces anger and aggressive behavior. Psychological Science, 24(5), 688–697. [DOI] [PubMed] [Google Scholar]
  58. Peters KD, Constans JI, & Mathews A (2009). Experimental modification of attribution processes. Journal of Abnormal Psychology, .120(1), pp. [DOI] [PubMed] [Google Scholar]
  59. Ren Z, Zhao Z, Yu X, Zhang L, & Li X (2021). Modification of hostile interpretation bias and self-reported aggression in juvenile delinquents: A randomized controlled trial. International Journal of Clinical and Health Psychology, 21(2), 100226. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Rynar L, & Coccaro EF (2018). Psychosocial impairment in DSM-5 intermittent explosive disorder. Psychiatry Research, 264, 91–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Schmidt NB, & Vereenooghe L (2021). Targeting hostile attributions in inclusive schools through online cognitive bias modification: A randomised experiment. Behaviour Research and Therapy, 146, 103949. [DOI] [PubMed] [Google Scholar]
  62. Schultz LT, & Heimberg RG (2008). Attentional focus in social anxiety disorder: potential for interactive processes. Clinical Psychology Review, 28(7), 1206–1221. [DOI] [PubMed] [Google Scholar]
  63. Schulz KF, Chalmers I, Hayes RJ, & Altman DG (1995). Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Jama, 273(5), 408–412. [DOI] [PubMed] [Google Scholar]
  64. Sidik K, & Jonkman JN (2005). Simple heterogeneity variance estimation for meta-analysis. Journal of the Royal Statistical Society: Series C (Applied Statistics), 54(2), 367–384. [Google Scholar]
  65. Smith HL, Dillon KH, & Cougle JR (2018). Modification of Hostile Interpretation Bias in Depression: A Randomized Controlled Trial. Behavior Therapy, 49(2), 198–211. [DOI] [PubMed] [Google Scholar]
  66. Smith HL, McDermott KA, Carlton CN, & Cougle JR (2019). Predictors of Symptom Outcome in Interpretation Bias Modification for Dysphoria. Behavior Therapy, 50(3), 646–658. [DOI] [PubMed] [Google Scholar]
  67. Smith TW, Glazer K, Ruiz JM, & Gallo LC (2004). Hostility, anger, aggressiveness, and coronary heart disease: an interpersonal perspective on personality, emotion, and health. Journal of Personality, 72(6), 1217–1270. [DOI] [PubMed] [Google Scholar]
  68. Sprunger JG (2019). Randomized controlled trial of an attention-based intervention for alcohol-facilitated intimate partner aggression. APA PsycInfo. [Google Scholar]
  69. Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, . . . Higgins JPT. (2019). RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ, 366, l4898. [DOI] [PubMed] [Google Scholar]
  70. Stoddard J, Sharif-Askary B, Harkins EA, Frank HR, Brotman MA, Penton-Voak IS, ... & Leibenluft E. (2016). An open pilot study of training hostile interpretation bias to treat disruptive mood dysregulation disorder. Journal of Child and Adolescent Psychopharmacology, 26(1), 49–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Trudel-Fitzgerald C, Reduron LR, Kawachi I, & Kubzansky LD (2021). Specificity in Associations of Anger Frequency and Expression With Different Causes of Mortality Over 20 Years. Psychosomatic Medicine, 83(5). [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Van Bockstaele B, van der Molen MJ, van Nieuwenhuijzen M, & Salemink E (2020). Modification of hostile attribution bias reduces self-reported reactive aggressive behavior in adolescents. Journal of Expermiental Child Psychology, 194, 104811. [DOI] [PubMed] [Google Scholar]
  73. van Teffelen MW, Lobbestael J, Voncken MJ, Cougle JR, & Peeters F (2021). Interpretation bias modification for hostility: A randomized clinical trial. Journal of Consulting and Clinical Psychology, 89, 421–434. [DOI] [PubMed] [Google Scholar]
  74. Vassilopoulos SP, Brouzos A, & Andreou E (2015). A Multi-Session Attribution Modification Program for Children with Aggressive Behaviour: Changes in Attributions, Emotional Reaction Estimates, and Self-Reported Aggression. Behavioural and Cognitive Psychotherapy, 43(5), 538–548. [DOI] [PubMed] [Google Scholar]
  75. Vassilopoulos SP, & Brouzos A (2022). A multi-session attribution modification program for children: Effects on hostile attributions and reactive/proactive aggression. Hellenic Journal of Psychology, 19(1), 69–82. [Google Scholar]
  76. Versella MV, Piccirillo ML, Potter CM, Olino TM, & Heimberg RG (2016). Anger profiles in social anxiety disorder. Journal of Anxiety Disorders, 37, 21–29. [DOI] [PubMed] [Google Scholar]
  77. Wenzel A, & Lystad C (2005). Interpretation biases in angry and anxious individuals. Behaviour Research and Therapy, 43(8), 1045–1054. [DOI] [PubMed] [Google Scholar]
  78. WHO. (2022). WHO Violence Prevention Unit: approach, objectives and activities, 2022–2026 In: World Health Organization. [Google Scholar]
  79. Wilkowski BM, Crowe SE, & Ferguson EL (2015). Learning to keep your cool: Reducing aggression through the experimental modification of cognitive control. Cognition and Emotion, 29(2), 251–265. [DOI] [PubMed] [Google Scholar]
  80. Zeng K, Cao F, Wu Y, Zhang M, & Ding X (2023). Effects of interpretation bias modification on hostile attribution bias and reactive cyber-aggression in chinese adolescents: A randomized controlled trial. Current Psychology: A Journal for Diverse Perspectives on Diverse Psychological Issues. Advance online publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Zhao Z, Yu X, Ren Z, Zhang L, & Li X (2022). The remediating effect of Attention Bias Modification on aggression in young offenders with antisocial tendency: A randomized controlled trial. Journal of Behavior Therapy and Experimental Psychiatry, 75, 101711. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

No materials included in this review are currently publicly available.

RESOURCES