Abstract
Aggression replacement training (ART) is widely used to reduce aggression. Results regarding its effectiveness, however, are inconclusive regarding adults and specific populations displaying severe aggression. The current open uncontrolled treatment study aimed at assessing the social skills and anger control modules of the ART to reduce aggression in forensic psychiatric outpatients (FPOs). Furthermore, characteristics associated with treatment outcome and dropout were examined. The results suggested that aggression changed during the ART. In addition, higher baseline levels of trait aggression were associated with greater reductions of aggression, whereas more cognitive distortions were associated with less reduction. Treatment dropouts were characterized by higher levels of psychopathic traits, proactive aggression, and more weekly substance use. As there was a considerable amount of dropout; it is important to assess risk of dropping out of treatment and, subsequently, improve treatment motivation. This might enhance treatment adherence which may lead to a more successful reduction of aggression.
Keywords: aggressive behavior, treatment, responsivity, forensic psychiatric patients
Disproportionate aggressive behavior, defined as aggression disproportionate to the provocation (Siever, 2008), can have detrimental effects on victims, perpetrator, and society (Gentile & Gellig, 2012; Patel & Taylor, 2012; World Health Organization [WHO], 2007). Victims of violence, for instance, are at high risk of serious and lasting physical damage and psychological problems, ranging from posttraumatic stress disorder to depression, anxiety disorders, substance abuse, and suicidal behavior (Krug, Mercy, Dahlberg, & Zwi, 2002). Individuals who show aggressive and violent behavior, however, are also characterized by multiple (social) problems, such as isolation, greater legal costs, and absenteeism from work (Gentile & Gellig, 2012; Patel & Taylor, 2012; WHO, 2007). In addition, aggression is one of the most important reasons for referral to forensic psychiatric institutions (Smith & Humphreys, 1997). Considering the significant health, social, and economic consequences of aggression, there is a critical need for effective treatments of maladaptive aggressive behavior.
Generally, interventions based on principles of cognitive behavioral therapy (CBT) are the interventions of first choice. A frequently used CBT-based intervention for aggressive behavior is the aggression replacement training (ART; Brännström, Kaunitz, Andershed, South, & Smedslund, 2016). The ART was originally developed for use in children and adolescents and consists of three modules: (a) social skills training, which focuses on responding in a prosocial way to difficult situations instead of using aggression; (b) anger control training, which teaches techniques to exert more control over aggressive thoughts and aggressive impulses; and (c) moral reasoning training, where one learns to recognize certain cognitive distortions relating to aggression by themselves and think in a less egocentric way, by means of group discussions (Glick & Goldstein, 1987; Goldstein, Glick, & Gibbs, 1998). Originally, the modules are given in three weekly sessions during 10 weeks. Over the years, the original ART had been modified and applied for other settings, outcomes, and populations (Brännström et al., 2016).
Commonly, among either juvenile or adult criminal offenders, as well as young forensic psychiatric patients, positive results of the ART have been reported on recidivism, cognitive distortions, and antisocial and aggressive behavior (Brännström et al., 2016; Hornsveld, 2005; Hornsveld, Kraaimaat, Muris, Zwets, & Kanters, 2015; Hornsveld, Nijman, Hollin, & Kraaimaat, 2008; Hornsveld, Nijman, & Kraaimaat, 2008; Hornsveld, van Dam-Baggen, Leenaars, & Jonkers, 2004; Landenberger & Lipsey, 2005; Lipsey, Chapman, & Landenberger, 2001; Lipsey, Landenberger, & Wilson, 2007). Still, results regarding the effectiveness of the ART might be considered as inconclusive. A recent systematic review documented that, at an individual study level, ART indeed had positive effects on recidivism and anger control, social skills, and moral reasoning in adolescents as well as in adults (Brännström et al., 2016). However, overall the quality of the reviewed studies was found to be insufficient to conclusively claim that the ART is effective with risk rations ranging from .25 to .75. One of the major concerns is that the target groups have not been described sufficiently. For instance, the behavior targeted by the ART is often described as antisocial, which is rather broad, which makes it unclear for which type of (specific) behavior the ART is most suitable as treatment. Furthermore, studies conducted among adult populations consisted of heterogeneous samples (Brännström et al., 2016), including violent offenders, individuals convicted for crimes such as robbery, and even offenders with a history of psychosis. Such heterogeneous samples make it unclear to which population the ART is most applicable. Further concerns are inadequate description of missing or incomplete data and author bias might have been occurred, as studies have been conducted by researchers associated with the development of the ART (Brännström et al., 2016). Considering the widespread use of interventions such as the ART, more research is needed to explore its effectiveness in specific and clearly described populations. The current study, therefore, tries to overcome one of the shortcomings of previous research by focusing on components of the ART among a specific population: forensic psychiatric outpatients (FPOs) with severe aggression regulation problems.
Furthermore, it is of great importance to elucidate which characteristics are associated with treatment response and to investigate to which subgroup of individuals this treatment is suitable. This is in line with the risk–need–responsivity (RNR) model for offender rehabilitation, which postulates that offenders at higher risk of reoffending will benefit most from more intensive treatment, that only those factors associated with reductions in recidivism should be targeted during treatment, and that interventions should be matched to offender characteristics (Andrews et al., 1990; Polaschek, 2012). Focusing on these factors would enable clinical practice to develop personalized treatment, instead of or alongside group interventions, which may enhance treatment adherence and improve the effectiveness, which is of great importance as dropout rates among outpatients receiving aggression treatment are high (Hornsveld, Nijman, Hollin, & Kraaimaat, 2008). In the long run, this might result in a more successful reduction of aggressive behavior.
Previous studies consider interventions such as the ART to be particularly effective in patients with impulsive/reactive aggression, low in psychopathic traits, and with a high disposition to anger at the beginning of treatment (Hornsveld, 2005; Hornsveld et al., 2014; Hornsveld, Nijman, Hollin, & Kraaimaat, 2008; Hornsveld, Nijman, & Kraaimaat, 2008; Hornsveld et al., 2004). Moreover, patients who did not complete ART exhibited more chronically antisocial behavior and higher levels of psychopathy, and were at higher risk of violent recidivism (Hornsveld et al., 2014; Hornsveld, Nijman, & Kraaimaat, 2008). This is in agreement with studies showing that limited effects of ART on adult recidivism were solely found among individuals who did not complete the intervention (Brännström et al., 2016). Another feature that seems to be important to consider is the impact of distorted cognitions. Cognitive distortions are defined as inaccurate attitudes, thoughts, or beliefs regarding own or other’s behavior and are associated with externalizing tendencies, such as aggression (Barriga & Gibbs, 1996; Barriga, Landau, Stinson, Liau, & Gibbs, 2000). A recent meta-analysis suggested that successful interventions on cognitive distortions could result in a decrease in externalizing problem behavior (Helmond, Overbeek, Brugman, & Gibbs, 2015). High levels of cognitive distortions, when untreated, might then also be related to a less successful reduction of externalizing problems, such as aggressive behavior. In addition, substance use is closely related to antisocial and offending behavior and occurs frequently in forensic psychiatric patients (Doran, Luczak, Bekman, Koutsenok, & Brown, 2012; Ruiz, Douglas, Edens, Nikolova, & Lilienfeld, 2012; van der Kraan et al., 2014). Furthermore, the comorbidity of substance misuse is relatively high in forensic psychiatric settings (van der Kraan et al., 2014). It, moreover, is suggested that when substance problems are not the primary diagnosis, it still is important to increase insight in substance use as a risk factor for the recurrence of criminal or antisocial behavior (van der Kraan et al., 2014). Furthermore, it was thought that this increase in insight is associated with treatment motivation and, therefore, also with treatment effectiveness. High levels of substance use in combination with limited insight in the consequences and associations with current aggressive behavior might then also be negatively related to treatment response.
Taken together, studies regarding the ART do not provide clear-cut evidence for a specific population of adults displaying clinical levels of aggressive behavior. Furthermore, it is unknown whether reactive and trait aggression, psychopathic traits, cognitive distortions, and substance use are associated with the response or dropout of the ART in adults. The current open uncontrolled trial study, therefore, was designed (a) to assess whether aggressive behavior reduced during the social skills and anger control modules of the ART in a specific population of adult FPOs with aggression regulation problems; (b) to examine whether trait and type of aggression, cognitive distortions, psychopathic traits, and substance use at baseline moderated the change in aggression during the ART; and (3) to investigate whether treatment dropouts differed from treatment completers in aforementioned characteristics.
Method
Participants
In the period from January 1, 2012, to June 15, 2015, 963 FPOs were referred to “Kairos,” the outpatient unit of Forensic Psychiatric Clinic the Pomestichting in Nijmegen, the Netherlands, because of aggression regulation problems. Inclusion to the study required to meet each of the following criteria: (a) a diagnosis of an antisocial, borderline, and/or narcissistic personality disorder, and/or the intermittent explosive disorder (IED); and (b) a total score of 5 points or higher on the Social Dysfunction and Aggression Scale (SDAS; Wistedt et al., 1990). In addition, FPOs were excluded if there was a current major depression, current severe addiction, or lifetime bipolar disorder or psychosis. They were excluded because, in such a condition, a proper treatment of these disorders will be a priority and, furthermore, these conditions will seriously limit the responsivity of patients to ART. In the current study, 213 (of the initial 963 FPOs) male and female FPOs were eligible and willing to participate. An overview of reasons for exclusion of the remaining 750 FPOs is provided in Table 1.
Table 1.
Reason for Exclusion.
| N | |
|---|---|
| Total | 750 |
| Reason | |
| Negative decision by therapist due to severity of psychopathology | 12 |
| Dropout after intake/not suitable for treatment | 102 |
| Refused to participate | 324 |
| Exclusion criteria | |
| Current major depression | 52 |
| Lifetime psychosis | 25 |
| Current severe alcohol/drug dependency | 56 |
| Insufficient understanding of Dutch language | 12 |
| No current aggressive behavior (only past) | 167 |
Of the 213 FPOs, 44 were excluded because of no show at the screening appointment (N = 22), current major depression (N = 2), no current aggressive behavior (N = 10), not willing to participate anymore (N = 4), or no show at the baseline assessment start of treatment (N = 6). Eventually, 169 FPOs participated in the present study, of which 125 performed the halfway measurement and 115 performed the end-of-treatment measurement as well. Of the 169 participating FPOs, 58 dropped out of the current study because of no show during intervention (N = 47), reference to other type of treatment (for addiction or for autism, N = 7), not willing to participate anymore (N = 2), and death (N = 2). Unfortunately, no extra follow-up information regarding this dropout group is available, only the data collected during the baseline measurement. Furthermore, admission to Kairos occurs on either obligatory (e.g., when sentenced by a judge) or voluntary basis (based on reference by general practitioner). In the current study, 124 patients were referred voluntarily and 45 obligatory.
Demographic information is provided in Table 2. All participants obtained a monetary compensation. The current study was approved by the regional Ethics Committee, CMO region Arnhem-Nijmegen, the Netherlands. The initial design included a waiting-list control group. However, this design did not get ethical approval of the Ethics Committee. Due to the vulnerability of this population and their critical need of psychological care, the committee reasoned that these patients should receive immediate care and considered a waiting-list control group as not ethical. The current study, therefore, was an uncontrolled open trial study. This is one of the difficulties of conducting research in clinical settings as well as that it was not possible to deliver the ART components as they were originally intended. Due to the nature of the study, no causal conclusions can be drawn. Nevertheless, cross-sectional studies are thought to be more generalizable to the general population of the population of interest, more suitable to identify risk factors and, therefore, of significant clinical relevance (Besen & Gan, 2014).
Table 2.
Demographic Information.
| M/N | |
|---|---|
| Age | M = 35.79 (SD = 10.94) |
| Male | N = 159 |
| Female | N = 10 |
| IQa | M = 87.83 (SD = 11.91) |
| Alcohol use, unit/week | N = 112; M = 7.99 (SD = 15.34) |
| Cannabis use, joint/week | N = 53; M = 4.31 (SD = 13.54) |
| Use of medication | N = 67 |
| Diagnosis | |
| Antisocial personality disorder | N = 68 |
| Borderline personality disorder | N = 35 |
| Narcissistic personality disorder | N = 6 |
| Intermittent explosive disorder | N = 145 |
| ADHD | N = 46 |
| History of depressive disorder | N = 110 |
Note. ADHD = attention-deficit/hyperactivity disorder.
As measured by using the Dutch Adult Reading Test (Schmand, Bakker, Saan, & Louman, 1991).
Measures
The SDAS (Wistedt et al., 1990) is an observer scale that measures the severity of actual aggressive behavior. The score on the SDAS was used as the primary outcome measure. It consists of nine items measuring outward aggression and two items measuring inward aggression. Items have to be scored on a 4-point Likert-type scale with 0 = not present and 4 = severely to extremely present as extremes. The SDAS has adequate observer reliability (Wistedt et al., 1990). The Dutch translation has adequate internal consistency and interrater reliability (Kobes, Nijman, & Bulten, 2012). In the current study, due to lack of observers, the SDAS was used as self-report. Participants had to rate their aggressive behavior over a period of 3 months. The SDAS was administered at baseline (over the last 3 months), halfway (over the last 2 weeks), and end-of-treatment measurement (over the last 2 weeks). In the current study, the SDAS as a self-report demonstrated acceptable test–retest stability: intraclass correlation coefficient (ICC) baseline—halfway measurement = .686, p < .01; halfway—end-of-treatment measurement = .763, p < .01; baseline—end-of-treatment measurement = .651, p < .01; and acceptable internal consistency, Cronbach’s α baseline = .76, halfway measurement = .82, end-of-treatment measurement = .82.
The Clinical Global Impression (CGI) Scale (Guy, 1976) is a widely used global rating of the severity of symptoms and the overall change from baseline to endpoint. In the current study, clinicians were asked to rate the severity of, and change in, aggressive behavior. Both questions are rated by the clinician on a 7-point Likert-type scale with for Question 1, 1 = normal, not ill and 7 = among the most extremely ill patients as extremes, regarding severity and for Question 2, 1 = very much improved and 7 = very much worse as extremes with respect to improvement. The CGI was filled out at baseline, halfway, and end-of-treatment measurement.
The Reactive Proactive Questionnaire (RPQ; Cima, Raine, Meesters, & Popma, 2013; Raine et al., 2006) is a 23-item self-report questionnaire to measure reactive and proactive aggression. The reactive subscale consists of 11 items, whereas the proactive subscale consists of 12 items. The items are rated 0 (never), 1 (sometimes), or 2 (often). The Dutch translation has good internal consistency and adequate convergent, criterion, and construct validity (Cima et al., 2013). In the current study, the internal consistency has also proven to be good (Cronbach’s α = .91 for baseline measurement and .93 for end-of-treatment measurement). The RPQ was administered at baseline and end of treatment.
The Aggression Questionnaire (AQ; Buss & Perry, 1992) is a self-report questionnaire to assess an overall trait level of aggression. It consists of 29 items, which are divided into four subscales: physical aggression, verbal aggression, anger, and hostility. The items are scored on a 5-point Likert-type scale (1 = extremely unlike me to 5 = extremely like me). The Dutch translation has adequate psychometric properties (Morren & Meesters, 2002). In the present study, the internal consistency has also proven to be good (Cronbach’s α = .91 for baseline measurement and .93 for end-of-treatment measurement). The AQ was administered at baseline and end-of-treatment measurement.
The How I Think (HIT) Questionnaire (Barriga & Gibbs, 1996) is a 54-item self-report questionnaire to assess self-serving cognitive distortions. The items are divided into four cognitive distortion subscales (self-centered, blaming others, minimizing/labeling, assuming the worst) and four behavioral referent categories (physical aggression, opposition defiance, lying, stealing). Items have to be answered on a 6-point Likert-type scale. The Dutch translation has proven reliability and validity (Nas, Brugman, & Koops, 2008). In the current study, the internal consistency has also proven to be good (Cronbach’s α = .93 for baseline measurement and .94 for end-of-treatment measurement). The HIT was administered at baseline and end-of-treatment measurement.
The Self-Report Psychopathy–Short-Form (SRP-SF; Paulhus, Neumann, & Hare, 2015) is a self-report measure of adult psychopathic features. The SRP-SF consists of 29 statements, which are divided into four subscales: interpersonal manipulation, callous affect, erratic life styles, and criminal tendencies. Participants have to rate the extent to which they agree with these statements on a 5-point Likert-type scale (1 = disagree strongly, 5 = agree strongly). The Dutch version of the SRP-SF has good internal consistency and test–retest reliability (Gordts, Uzieblo, Neumann, Van den Bussche, & Rossi, 2017). In the present study, the internal consistency has also proven to be good (Cronbach’s α = .88). The SRP-SF was only administered at baseline measurement.
Intervention
All FPOs were referred to the ART. Besides ART for general aggression and violence, ART was also offered for perpetrators of intimate partner violence. This version of the ART is identical to the regular ART, except that the partners of the FPOs were involved during this intervention (N = 58). Both the regular ART as well as the ART for domestic violence perpetrators consisted, as offered by “Kairos,” of two of the three original modules: (a) social skills training and (b) anger control training. Both interventions occurred either in groups (N = 116) or individually (N = 46) and consisted of two 90-min weekly sessions during 12 weeks. The first 10 weeks consisted of the social skills and anger control training. Week 11 consisted of a session to integrate all that was learned in the previous weeks. Finally, Week 12 consisted of an evaluation session. Indication for ART was determined by a multidisciplinary team. The ART therapists (all clinicians at “Kairos,” not involved in the current study as a researcher) were all formerly trained in applying the ART and, in addition, made use of a detailed intervention manual and participated in intervision.
Procedure
Clinicians at “Kairos” asked FPOs who were referred to ART (group or individual), whether they agreed to be contacted about the study. When they agreed, FPOs were contacted by the researcher. All FPOs, received treatment as indicated whether they participated in the study or not.
After receiving information about the nature of the study, a consent form was signed. The FPOs were screened by the researchers (who were also trained clinicians in the use of these interviews) with the Structured Clinical Interview for DSM-IV Axis II personality disorders (SCID-II; Weertman, Arntz, & Kerkhofs, 2000), the research criteria set for Intermittent Explosive Disorder (IED-IR; Coccaro, Kavoussi, Berman, & Lish, 1998), the MINI International Neuropsychiatric Interview for axis I disorders (MINI; Sheehan et al., 1998; Van Vliet & De Beurs, 2007), and the Measurement of Addictions for Triage and Evaluation (MATE)-Crimi for substance misuse, dependency, and its association with criminal behavior (Schippers & Broekman, 2010) regarding the aforementioned inclusion and exclusion criteria.
Once FPOs were suitable for participation, they proceeded with the baseline measurement, which was completed before the start of their treatment. Halfway, after 6 weeks, the level of aggressive behavior was determined by use of the SDAS. An end-of-treatment measurement took place after 12 weeks. The FPOs were instructed not to use alcohol or drugs 24 hr prior to any of the assessments. The CGI was administered by clinicians at baseline, halfway, and end of treatment.
Statistical Analysis
First, a MANOVA was conducted to investigate whether there were differences on aggression severity, clinician rating of aggression, reactive and proactive aggression, trait aggression, cognitive distortions, psychopathy, and measures of substance use, between FPOs who received ART as usual or ART for domestic violence perpetrators and FPOs who received group or individual ART. Subsequently, it was examined whether aggressive behavior changed over time, Research Question 1, by using a linear mixed model (SPSS, version 24). One advantage of this analysis is that it is possible to include individuals with incomplete data, without imputing data (Heck, Thomas, & Tabata, 2013). As the aggression assessment halfway and end of treatment were not completed for all participants, this method was favored. The basic model was a repeated-measures design with aggressive behavior as measured with the SDAS as dependent variable and time of measurement (baseline, halfway, end of treatment) as within-subjects factor. Repeated covariance type was set at diagonal, which assumes heterogeneous variances and no correlation between elements (Heck et al., 2013). With respect to time, the slope was set as a fixed effect and the intercept as a random effect. This random effect was defined to assess variation in the dependent variable because variation among individuals, regarding change in aggression over time, was assumed (Bolker et al., 2009; West, Welch, & Galecki, 2014). The covariance type for the random effects was set at unstructured, which is a completely general covariance matrix (Heck et al., 2013). Subsequently, a similar linear mixed model was conducted now with the CGI improvement as dependent variable and time of measurement (baseline, halfway, end of treatment) as within-subjects factor, to examine whether aggressive behavior decreased during treatment according to clinicians.
Next, it was investigated whether trait aggression, cognitive distortions, reactive and proactive aggression, psychopathy, and substance use measured at baseline moderated the effect of time, Research Question 2, by adding main effects of trait aggression, cognitive distortions, reactive and proactive aggression, psychopathy and weekly alcohol and cannabis use and two-way interactions of trait aggression, cognitive distortions, reactive and proactive aggression, psychopathy, and substance use with time to the basic model. Only the basic model with the SDAS as dependent variable was used, as the SDAS was the main outcome measure of the current study. To be able to interpret the results, the variables were centered; the sample mean was subtracted from the individuals mean.
Finally, to investigate the underlying differences between treatment dropouts and treatment completers, Research Question 3, a MANOVA regarding baseline measurements was conducted.
Results
Table 3 displays the means on all questionnaires and the weekly alcohol and cannabis use. Means are presented for the total sample as well as separately for the treatment dropouts and completers.
Table 3.
Descriptives (M, SD) of the Completers Versus the Dropouts and Total Sample.
| Completers (N = 111) | Dropouts (N = 58) | Total sample (N = 169) | |
|---|---|---|---|
| CGI baseline | M = 3.67 (SD = 0.56) | M = 3.84 (SD = 0.44) | M = 3.72 (SD = 0.53) |
| CGI halfway | M = 2.99 (SD = 0.66) | — | M = 2.99 (SD = 0.66) |
| CGI end of treatment | M = 2.83 (SD = 0.87) | — | M = 2.83 (SD = 0.87) |
| SDAS baseline | M = 15.14 (SD = 6.19) | M = 15.14 (SD = 7.09) | M = 15.14 (SD = 6.49) |
| SDAS halfway | M = 10.79 (SD = 6.67) | — | M = 10.79 (SD = 6.67) |
| SDAS end of treatment | M = 8.48 (SD = 6.35) | — | M = 8.48 (SD = 6.35) |
| AQ | M = 91.99 (SD = 18.41) | M = 95.00 (SD = 19.21) | M = 93.02 (SD = 18.69) |
| RPQ reactive | M = 11.95 (SD = 4.54) | M = 13.21 (SD = 5.34) | M = 12.38 (SD = 4.85) |
| RPQ proactive | M = 3.81 (SD = 3.74) | M = 5.33 (SD = 4.66) | M = 4.33 (SD = 4.13) |
| HIT | M = 2.41 (SD = 0.73) | M = 2.56 (SD = 0.81) | M = 2.46 (SD = 0.76) |
| Alcohola | M = 6.05 (SD = 11.53) | M = 11.67 (SD = 20.35) | M = 7.99 (SD = 15.34) |
| Cannabisa | M = 2.53 (SD = 7.61) | M = 7.71 (SD = 20.28) | M = 4.31 (SD = 13.54) |
| SRP-SFa | M = 66.75 (SD = 16.56) | M = 73.82 (SD = 19.05) | M = 69.15 (SD = 17.71) |
Note. Alcohol and cannabis use is defined as mean units/joint per week. The mean total score on the improvement subscale of the Clinical Global Impression Scale (CGI), the Social Dysfunction and Aggression Scale (SDAS), the Aggression Questionnaire (AQ), the reactive and proactive aggression subscale of the Reactive Proactive Questionnaire (RPQ), the How I Think (HIT) Questionnaire and the Self-Report Psychopathy–Short Form (SRP-SF) are reported.
Groups significantly differ from each other at least p < .05.
Differences Among FPOs
A MANOVA was conducted to investigate whether FPOs who received regular ART versus ART for domestic violence perpetrators and FPOs who received group versus individual treatment differed regarding type, trait and severity of aggression, substance use, cognitive distortions, and psychopathy. Bonferroni correction was used to control for multiple comparisons. No significant multivariate effects of ART versus ART for domestic violence and group versus individual treatment emerged, Wilks’ λ = .791, F(13, 68) = 1.383, p = .191, η² = .209; Wilks’ λ = .890, F(13, 68) = 0.645, p = .808, η² = .110, respectively. Therefore, in subsequent analyses, the FPOs were considered as one sample.
Effect on Aggressive Behavior
The analyses of both basic models “time, SDAS” (treatment responsivity based on self-report) as well as “time, CGI” (treatment responsivity based on clinician report) revealed a significant main effect of time, indicating that aggressive behavior significantly decreased during treatment. A graphic representation of the change in aggressive behavior (SDAS) is displayed in Figure 1.1 As an effect size is not provided by linear mixed models, Cohen’s d was calculated by dividing the mean difference between pre- and post-SDAS scores by the pooled standard deviation: 15.19 – 8.48 / (√((6.142 + 6.252) / 2)) = 1.08.
Figure 1.
Graphical representation of the change in aggressive behavior according to the self-report of FPOs (SDAS).
Note. Baseline: N = 169, SD = 6.49; Week 6: N = 125, SD = 6.69; end of treatment: N = 115, SD = 6.25. FPO = forensic psychiatric outpatient; SDAS = Social Dysfunction and Aggression Scale.
The basic model (time, SDAS) was extended by adding trait aggression, cognitive distortions, reactive and proactive aggression, psychopathy, and weekly alcohol and weekly cannabis use as possible predictor variables. Significant main effects of trait aggression, cognitive distortions, reactive and proactive aggression, and psychopathy emerged, suggesting that these characteristics were associated with differences in aggressive behavior. In the subsequent model, two-way interactions were included to examine which characteristics might explain variability in aggression reduction during treatment. The analysis of this model revealed a significant main effect of time, trait aggression, reactive and proactive aggression; significant interactions of Time × Trait Aggression, and a marginal significant interaction of Time × Cognitive Distortions (see Table 4). In the final model, nonsignificant interactions were removed. The results suggest that the disposition to act aggressively and cognitive distortions, measured at baseline, were associated with the course of treatment; that is, high baseline score on trait aggression leads to a more rapid decrease of aggression, whereas high baseline scores on cognitive distortions lead to a less rapid reduction of aggression. A graphic representation of both significant interaction effects are displayed in Figures 2 and 3. For this graph, predicted values were calculated by using the regression equation of the final model. For Figure 2, percentile scores of the AQ, measure for trait aggression, and mean scores of all other predictors were used. For Figure 3, percentile scores of the HIT, measure of cognitive distortions, and mean scores of all other predictors were used.
Table 4.
Results of Linear Mixed Model.
| Model | Parameter | Estimate | 95% CI | t | df | p |
|---|---|---|---|---|---|---|
| Basic Model 1 | Intercept | 14.91 | [13.81, 16.02] | 26.82 | 114.388 | <.001 |
| Time (SDAS) | −3.28 | [−3.86, −2.69] | −11.21 | 114.199 | <.001 | |
| Basic Model 2 | Intercept | 3.65 | [3.54, 3.77] | 63.62 | 92.511 | <.001 |
| Time (CGI) | −0.46 | [−0.57, −36] | −8.77 | 96.490 | <.001 | |
| Model including main effects baseline characteristics | Intercept | 14.91 | [13.81, 16.02] | 26.82 | 114.388 | <.001 |
| Time (SDAS) | −3.28 | [−3.86, −2.69] | −11.21 | 114.199 | <.001 | |
| AQ | 0.16 | [0.12, 0.21] | 7.32 | 113.76 | <.001 | |
| HIT | 3.38 | [2.24, 4.53] | 5.84 | 113.24 | <.001 | |
| RPQ proactive | 0.30 | [0.06, 0.54] | 2.51 | 113.03 | .013 | |
| RPQ reactive | 0.56 | [0.38, 0.74] | 6.09 | 113.25 | <.001 | |
| SRP-SF | 0.14 | [0.09, 0.19] | 5.23 | 109.513 | <.001 | |
| Alcohol use | 0.06 | [−0.03, 0.14] | 1.29 | 112.639 | .197 | |
| Cannabis use | 0.12 | [−0.04, 0.27] | 1.47 | 112.856 | .145 | |
| Model including significant main effects + interaction effects | Intercept | 15.00 | [14.12, 15.89] | 33.81 | 107.108 | <.001 |
| Time (SDAS) | −3.32 | [−3.90, −2.74] | −11.34 | 106.168 | <.001 | |
| AQ | 0.11 | [0.04, 0.19] | 3.03 | 108.27 | .003 | |
| HIT | 0.69 | [−1.28, 2.67] | 0.69 | 107.28 | .488 | |
| RPQ proactive | −0.55 | [−0.91, −0.19] | −3.07 | 107.449 | .003 | |
| RPQ reactive | 0.57 | [0.25, 0.89] | 3.58 | 107.95 | <.001 | |
| SRP-SF | 0.04 | [−0.05, 0.13] | 0.830 | 107.14 | .408 | |
| Time × AQ | −0.06 | [−0.11, 0.01] | −2.37 | 106.33 | .019 | |
| Time × HIT | 1.24 | [−0.07, 2.55] | 1.88 | 106.192 | .063 | |
| Time × RPQ proactive | −0.03 | [−0.27, 0.20] | −0.26 | 106.216 | .797 | |
| Time × RPQ reactive | −0.02 | [−0.22, −0.19] | −0.16 | 106.28 | .873 | |
| Time × SRP-SF | −0.00 | [−0.06, 0.05] | −0.15 | 106.172 | .881 | |
| Final model | Intercept | 15.00 | [14.13, 15.88] | 33.91 | 108.311 | <.001 |
| Time (SDAS) | −3.32 | [−3.90, −2.75] | −11.48 | 109.156 | <.001 | |
| AQ | 0.12 | [0.05, 0.19] | 3.26 | 119.755 | <.001 | |
| HIT | 0.81 | [−1.07, 2.68] | 0.85 | 120.602 | .396 | |
| RPQ proactive | −0.57 | [−0.88, −0.27] | −3.71 | 106.933 | <.001 | |
| RPQ reactive | 0.56 | [0.28, 0.83] | 4.05 | 107.408 | <.001 | |
| SRP-SF | 0.03 | [−0.04, 0.11] | 0.873 | 106.640 | .385 | |
| Time × AQ | −0.06 | [−0.11, 0.02] | −3.09 | 109.139 | .003 | |
| Time × HIT | 1.09 | [0.07, 2.11] | 2.12 | 109.072 | .036 |
Note. All results were comparable when excluding women (N = 10) and FPOs with IQ < 75 (N = 21) from analyses. CI = confidence interval; SDAS = Social Dysfunction and Aggression Scale; CGI Scale = Clinical Global Impression Scale; AQ = Aggression Questionnaire; HIT = How I Think Questionnaire; RPQ = Reactive Proactive Questionnaire; SRP-SF = Self-Report Psychopathy–Short Form; FPO = forensic psychiatric outpatient.
Figure 2.
Graphical representation of interaction effect of Aggression Questionnaire and course of treatment.
Figure 3.
Graphical representation of interaction effect of How I Think and course of treatment.
Dropout
Means and SDs for baseline measures of treatment dropouts and completers are presented in Table 3. Bonferroni correction was used to control for multiple comparisons. A significant multivariate effect of group was found, Wilks’ λ = .893, F(7, 155) = 2.66, p = .013, η² = .107. Separate univariate ANOVAs revealed that dropouts, compared with completers, used more weekly alcohol and cannabis, and displayed more proactive aggression and more psychopathic traits: F(1, 161) = 5.80, p = .017, η² = .035; F(1, 161) = 5.35, p = .022, η² = .032; F(1, 161) = 3.91, p = .050, η² = .024; and F(1, 161) = 5.29, p = .023, η² = .032, respectively. No significant group differences were found on AQ, HIT, and RPQ reactive: F(1, 161) = 0.67, p = .414, η² = .004; F(1, 161) = 1.37, p = .244, η² = .008; and F(1, 161) = 2.15, p = .144, η² = .013 respectively.
Discussion
The present study investigated the change in aggressive behavior during an intervention based on the anger control and social skills modules of the ART among FPOs with severe aggressive behavior. It was found that aggressive behavior decreased during the intervention according to clinicians and the self-report of FPOs. The current study complements previous findings by revealing that the social skills and anger control modules of the ART reduce aggression in a specific population of adult FPOs displaying clinical levels of aggression. Moreover, in agreement with previous studies (Hornsveld, 2005), the present results showed that a larger tendency to aggressive behavior at baseline measurement was associated with a more rapid decrease in aggression over the course of treatment. This may indicate regression toward the mean. We, however, believe that it is unlikely that this is the major mechanism. Cognitive distortions, however, were found to be negatively related to treatment outcome; more cognitive distortions were associated with a less rapid reduction of aggressive behavior. It is important to note that it might be possible that other, or maybe no, associations between cognitive distortions and treatment outcome could have occurred when the moral reasoning module was included in the intervention. With regard to psychopathic traits, substance use, and aggression subtype, no interactions were found with treatment outcome. This is in contrast with previous studies, showing that the ART was, in particular, effective in individuals displaying reactive aggression (Hornsveld, Nijman, Hollin, & Kraaimaat, 2008; Hornsveld et al., 2004). A possible explanation for this discrepancy is that the patients in the present study were all diagnosed with aggression regulation disorders and mainly displayed reactive aggressive behavior. It is possible that there was too little variation in aggression subtype and that, therefore, the current study mainly focused on FPOs with reactive aggressive behavior.
Regarding treatment dropout, higher levels of proactive aggression and psychopathic traits emerged to be associated with dropout, which is in agreement with previous studies (Hornsveld et al., 2014). In addition, more weekly alcohol and cannabis use were reported by FPOs who dropped out of treatment. Problems regarding substance use occur frequently in FPOs (Ruiz et al., 2012; van der Kraan et al., 2014). In the present study, FPOs with substance abuse or dependency were excluded from participation. The weekly substance use, however, still is relatively high. Previously, it was suggested that substance use may function as risk factor for recidivism (van der Kraan et al., 2014). The current results suggest that it is important to take into account the amount of substance use as it also may be a reason to drop out. Furthermore, as suggested by van der Kraan et al. (2014), it might be necessary to increase insight in substance use as this might be related to treatment adherence and/or motivation.
The current findings may have implications with respect to forensic mental health settings. As suggested by the RNR model for offender rehabilitation, interventions should be matched to offender characteristics (Andrews et al., 1990; Polaschek, 2012). The disposition to act aggressively, cognitive distortions, substance use, and psychopathy were found to be associated with treatment response and dropout. This knowledge can be used to make an informed decision of whether ART is suitable for a specific individual patient and to develop personalized treatment. For instance, it may be important to determine the magnitude of cognitive distortions before the start of treatment. This may be of particular importance regarding group interventions: More focus on altering these distortions might be necessary to reduce aggression equally rapidly as patients with less cognitive distortions. It also might be of interest to examine whether type of cognitive distortions is associated with treatment outcome. In the end, this would result in interventions that are more concentrated on altering these specific distortions and, therefore, maybe more successful in reducing aggression. Moreover, the current results might advocate the addition of the moral reasoning training/module, as this module focuses on cognitive distortions. Future research needs to elucidate whether adding this module is sufficient in reducing cognitive distortions and results in a more successful decrease of aggression in a population of individuals with severe aggression.
In addition, the current study provided information on the differences between treatment dropouts and completers but not on how to decrease dropout rates. When there is a high risk of dropout, special attention may be needed on treatment motivation. Treatment motivation is considered to be a crucial factor related to treatment outcome and is associated with treatment-related behavior such as compliance, engagement, and adherence (Drieschner, Lammers, & van der Staak, 2004). An intervention that may be helpful to increase motivation is motivational interviewing (MI). MI is a directive, client-centered, method for improving intrinsic motivation to change through exploring and resolving ambivalence (Miller & Rollnick, 2002). Previous studies showed that MI improved treatment adherence and the motivation to change in criminal offenders and individuals with substance use disorders (Crane & Eckhardt, 2013; Doran et al., 2012; Dunn, Deroo, & Rivara, 2001; Feldstein & Ginsburg, 2006; McMurran, 2009). It would be highly interesting to explore the effectiveness of MI as pretreatment in reducing dropout rates.
The present study and results have a number of limitations, which merit further comment. First, the ART examined in the current study only consisted of two modules instead of three. Therefore, the current results are not generalizable to the use of the entire ART and only accounts for the social skills and anger control module. As there exist a lot of variations in the application of the ART (Brännström et al., 2016), this is not the first study examining a different setup as compared with the original ART. These variations result in difficulties in comparing treatment effectiveness studies. In future research, the focus should perhaps be on the modules separately to be able to compare different study and treatment setups but also to understand the working mechanisms of and the added value of each module. Subsequently, as this was an open uncontrolled trial study and lacked a comparison or control group, the results have to be interpreted with care. Third, except for the CGI, all measurements consisted of self-report. It is questionable whether a population of FPOs is fully able to reflect on their own behavior and whether they are willing to answer genuinely. Furthermore, the SDAS, originally developed as an observer scale, has only once been used as a self-report measure (Smeijers, Rinck, Bulten, van den Heuvel, & Verkes, 2017). In addition, no follow-up measurement was included. This would enable one to determine the long-term effects of the ART and to distinguish whether specific individuals have a higher tendency to recidivate. Fourth, as only a few women participated, the current findings may not be generalizable to a female population with aggression regulation problems. Future research is needed to elucidate the effectiveness over a proper control condition and also more research is needed on the working mechanisms of the ART in forensic psychiatric clinical practice.
Notwithstanding the limitations and nature (open uncontrolled trial) of the current study, the present results suggest that aggressive behavior changed during the social skills and anger control modules of the ART in adult populations displaying severe levels of aggression. Subsequently, it seems to be of importance to examine patients before the start of their treatment regarding cognitive distortions, psychopathic traits, and substance use. Using this knowledge enables clinical practice to make an indication of the chance of dropout and to target an intervention more specifically towards the individual patient, which, in the end, probably in combination with MI, will enhance treatment adherence and reduce aggressive behavior more successfully.
Acknowledgments
The authors would like to thank the staff in the outpatient clinics of Kairos, Pompestichting at Nijmegen, the Netherlands, for their support in recruiting participants.
The change in aggressive behavior was similar in all subgroups: regular aggression replacement training (ART), ART for domestic violence perpetrators, group and individual treatment, male and female, voluntary, and obligatory.
Footnotes
Declaration of Conflicting Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: In the past 3 years, Jan Buitelaar has been a consultant to/member of advisory board of/and/or speaker for Janssen Cilag BV, Eli Lilly, Bristol-Myer Squibb, Shering Plough, UCB, Shire, Novartis, and Servier. He is not an employee of any of these companies, and not a stock shareholder of any of these companies. He has no other financial or material support, including expert testimony, patents, or royalties. On behalf of all other authors, the corresponding author states that there is no conflict of interest.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by NWO/National Initiative Brain & Cognition (NIHC, Grant NWO-056-24-011). Furthermore, Jan Buitelaar is supported by the European Union 7th Framework programs AGGRESSOTYPE (602805) and MATRICS (603016).
References
- Andrews D. A., Zinger I., Hoge R. D., Bonta J., Gendreau P., Cullen F. T. (1990). Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis. Criminology, 28, 369-404. [Google Scholar]
- Barriga A. Q., Gibbs J. C. (1996). Measuring cognitive distortion in antisocial youth: Development and preliminary validation of the “How I Think” Questionnaire. Aggressive Behavior, 22, 333-343. [Google Scholar]
- Barriga A. Q., Landau J. R., Stinson B. L., Liau A. K., Gibbs J. C. (2000). Cognitive distortion and problem behaviors in adolescents. Criminal Justice and Behavior, 27, 36-56. [Google Scholar]
- Besen J., Gan S. D. (2014). A critical evaluation of clinical research study designs. The Journal of Investigative Dermatology, 134(3), e18. [DOI] [PubMed] [Google Scholar]
- Bolker B. M., Brooks M. E., Clark C. J., Geange S. W., Poulsen J. R., Stevens M. H. H., White J.-S. S. (2009). Generalized linear mixed models: A practical guide for ecology and evolution. Trends in Ecology & Evolution, 24, 127-135. [DOI] [PubMed] [Google Scholar]
- Brännström L., Kaunitz C., Andershed A.-K., South S., Smedslund G. (2016). Aggression Replacement Training (ART) for reducing antisocial behavior in adolescents and adults: A systematic review. Aggression and Violent Behavior, 27, 30-41. [Google Scholar]
- Buss A. H., Perry M. (1992). The Aggression Questionnaire. Journal of Personality and Social Psychology, 63, 452-459. doi: 10.1037/0022-3514.63.3.452 [DOI] [PubMed] [Google Scholar]
- Cima M., Raine A., Meesters C., Popma A. (2013). Validation of the Dutch Reactive Proactive Questionnaire (RPQ): Differential correlates of reactive and proactive aggression from childhood to adulthood. Aggressive Behavior, 39, 99-113. doi: 10.1002/ab.21458 [DOI] [PubMed] [Google Scholar]
- Coccaro E. F., Kavoussi R. J., Berman M. E., Lish J. D. (1998). Intermittent explosive disorder-revised: Development, reliability, and validity of research criteria. Comprehensive Psychiatry, 39, 368-376. doi: 10.1016/S0010-440X(98)90050-5 [DOI] [PubMed] [Google Scholar]
- Crane C. A., Eckhardt C. I. (2013). Evaluation of a single-session brief motivational enhancement intervention for partner abusive men. Journal of Counseling Psychology, 60, 180-187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Doran N., Luczak S. E., Bekman N., Koutsenok I., Brown S. A. (2012). Adolescent substance use and aggression a review. Criminal Justice and Behavior, 39, 748-769. [Google Scholar]
- Drieschner K. H., Lammers S. M., van der Staak C. P. (2004). Treatment motivation: An attempt for clarification of an ambiguous concept. Clinical Psychology Review, 23, 1115-1137. [DOI] [PubMed] [Google Scholar]
- Dunn C., Deroo L., Rivara F. P. (2001). The use of brief interventions adapted from motivational interviewing across behavioral domains: A systematic review. Addiction, 96, 1725-1742. [DOI] [PubMed] [Google Scholar]
- Feldstein S. W., Ginsburg J. I. (2006). Motivational interviewing with dually diagnosed adolescents in juvenile justice settings. Brief Treatment and Crisis Intervention, 6, 218-233. [Google Scholar]
- Gentile J. P., Gellig P. M. (2012). Aggression. In Gentile G. (Ed.), Psychiatry of intellectual disability (pp. 210-249). Oxford, UK: John Wiley. [Google Scholar]
- Glick B., Goldstein A. P. (1987). Aggression replacement training. Journal of Counseling & Development, 65, 356-362. [Google Scholar]
- Goldstein A. P., Glick B., Gibbs J. C. (1998). Aggression replacement training: A comprehensive intervention for aggressive youth (Rev. ed.). Champaign, IL: Research Press. [Google Scholar]
- Gordts S., Uzieblo K., Neumann C., Van den Bussche E., Rossi G. (2017). Validity of the Self-Report Psychopathy Scales (SRP-III full and short versions) in a community sample. Assessment, 24, 308-325. [DOI] [PubMed] [Google Scholar]
- Guy W. (1976). ECDEU assessment manual for psychopharmacology: 1976. Rockville, MD: U.S. Department of Health, Education, and Welfare. [Google Scholar]
- Heck R. H., Thomas S. L., Tabata L. N. (2013). Multilevel and longitudinal modeling with IBM SPSS. New York, NY: Routledge. [Google Scholar]
- Helmond P., Overbeek G., Brugman D., Gibbs J. C. (2015). A meta-analysis on cognitive distortions and externalizing problem behavior: Associations, moderators, and treatment effectiveness. Criminal Justice and Behavior, 42, 245-262. [Google Scholar]
- Hornsveld R. H. J. (2005). Evaluation of aggression control therapy for violent forensic psychiatric patients. Psychology, Crime & Law, 11, 403-410. doi: 10.1080/10683160500255596 [DOI] [Google Scholar]
- Hornsveld R. H. J., Kraaimaat F. W., Muris P., Zwets A. J., Kanters T. (2015). Aggression replacement training for violent young men in a forensic psychiatric outpatient clinic. Journal of Interpersonal Violence, 30, 3174-3191. [DOI] [PubMed] [Google Scholar]
- Hornsveld R. H. J., Nijman H. L., Hollin C. R., Kraaimaat F. W. (2008). Aggression control therapy for violent forensic psychiatric patients: Method and clinical practice. International Journal of Offender Therapy and Comparative Criminology, 52, 222-233. doi: 10.1177/0306624X07303876 [DOI] [PubMed] [Google Scholar]
- Hornsveld R. H. J., Nijman H. L. I., Kraaimaat F. W. (2008). Aggression control therapy for violent forensic psychiatric patients: First results. Psychology, Crime & Law, 14, 1-18. doi: 10.1080/10683160701340569 [DOI] [PubMed] [Google Scholar]
- Hornsveld R. H. J., van Dam-Baggen R., Leenaars E., Jonkers P. (2004). Aggression control therapy for violent forensic psychiatric patients: method and clinical practice. Tijdschrift voor Psychotherapie, 30(1), 13-22. [DOI] [PubMed] [Google Scholar]
- Kobes M. H., Nijman H. H., Bulten E. B. (2012). Assessing aggressive behavior in forensic psychiatric patients: Validity and clinical utility of combining two instruments. Archives of Psychiatric Nursing, 26, 487-494. [DOI] [PubMed] [Google Scholar]
- Krug E. G., Mercy J. A., Dahlberg L. L., Zwi A. B. (2002). The world report on violence and health. The lancet, 360(9339), 1083-1088. [DOI] [PubMed] [Google Scholar]
- Landenberger N. A., Lipsey M. W. (2005). The positive effects of cognitive–behavioral programs for offenders: A meta-analysis of factors associated with effective treatment. Journal of Experimental Criminology, 1, 451-476. [Google Scholar]
- Lipsey M. W., Chapman G. L., Landenberger N. A. (2001). Cognitive-behavioral programs for offenders. The ANNALS of the American Academy of Political and Social Science, 578, 144-157. [Google Scholar]
- Lipsey M. W., Landenberger N. A., Wilson S. J. (2007). Effects of cognitive-behavioral programs for criminal offenders. Campbell systematic reviews, 6, 1-27. [Google Scholar]
- McMurran M. (2009). Motivational interviewing with offenders: A systematic review. Legal and Criminological Psychology, 14, 83-100. [Google Scholar]
- Miller W., Rollnick S. (2002). Motivational interviewing: Preparing people for change (2nd ed.). New York, NY: Guilford Press. [Google Scholar]
- Morren M., Meesters C. (2002). Validation of the Dutch version of the Aggression Questionnaire in adolescent male offenders. Aggressive Behavior, 28, 87-96. doi: 10.1002/ab.90010 [DOI] [Google Scholar]
- Nas C. N., Brugman D., Koops W. (2008). Measuring self-serving cognitive distortions with the “How I Think” Questionnaire. European Journal of Psychological Assessment, 24, 181-189. [Google Scholar]
- Patel D., Taylor R. M. (2012). Social and economic costs of violence: Workshop summary (Institute of Medicine, Forum on Global Violence Prevention, & National Research Council). Washington, DC: National Academies Press. [PubMed] [Google Scholar]
- Paulhus D., Neumann C., Hare R. (2015). Manual for the self-report psychopathy scale (4th ed.). Toronto, Ontario, Canada: Multi-Health Systems. [Google Scholar]
- Polaschek D. L. (2012). An appraisal of the Risk–Need–Responsivity (RNR) model of offender rehabilitation and its application in correctional treatment. Legal and Criminological Psychology, 17, 1-17. [Google Scholar]
- Raine A., Dodge K., Loeber R., Gatzke-Kopp L., Lynam D., Reynolds C., . . .Liu J. H. (2006). The reactive-proactive Aggression Questionnaire: Differential correlates of reactive and proactive aggression in adolescent boys. Aggressive Behavior, 32, 159-171. doi: 10.1002/ab.20115 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ruiz M. A., Douglas K. S., Edens J. F., Nikolova N. L., Lilienfeld S. O. (2012). Co-occurring mental health and substance use problems in offenders: Implications for risk assessment. Psychological Assessment, 24, 77-87. [DOI] [PubMed] [Google Scholar]
- Schippers G. M., Broekman T. G. (2010). MATE-Crimi 2.01. Manual and protocol. Nijmegen, The Netherlands: Bêta Boeken. [Google Scholar]
- Schmand B. A., Bakker D., Saan R. J., Louman J. (1991). The Dutch Adult Reading Test: a measure of the premorbid level of intelligence. Tijdschrift voor Gerontologie en Geriatrie, 22, 15-19. [PubMed] [Google Scholar]
- Sheehan D., Lecrubier Y., Sheehan K., Amorim P., Janavs J., Weiler E. (1998). The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59, 22-33. doi: 10.1016/s0924-9338(97)83296-8 [DOI] [PubMed] [Google Scholar]
- Siever L. J. (2008). Neurobiology of aggression and violence. American Journal of Psychiatry, 165, 429-442. doi: 10.1176/appi.ajp.2008.07111774 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smeijers D., Rinck M., Bulten E., van den Heuvel T., Verkes R. J. (2017). Generalized hostile interpretation bias regarding facial expressions: Characteristic of pathological aggressive behavior. Aggressive Behavior, 43, 386-397. [DOI] [PubMed] [Google Scholar]
- Smith A. D., Humphreys M. (1997). Characteristics of in-patients transferred to a locked ward in a Scottish psychiatric hospital. Health Bulletin, 55, 77-82. [PubMed] [Google Scholar]
- van der Kraan J., Verkes R. J., Goethals K., Vissers A., Brazil I., Bulten E. (2014). Substance use disorders in forensic psychiatric patients. International Journal of Law and Psychiatry, 37, 155-159. [DOI] [PubMed] [Google Scholar]
- Van Vliet I., De Beurs E. (2007). Het Mini Internationaal Neuropsychiatrisch Interview (MINI). Een kort gestructureerd diagnostisch psychiatrisch interview voor DSM-IV en ICD-10-stoornissen [The Mini International Neuropsychiatric Interview (MINI). A short structured diagnostic psychiatric interview for DSM-IV and ICD-10 disorders]. Tijdschrift voor Psychiatrie, 49, 393-397. [PubMed] [Google Scholar]
- Weertman A., Arntz A., Kerkhofs M. (2000). SCID-II: Structured clinical interview for DSM-IV axis II personality disorders. Lisse, The Netherlands: Swets & Zeitlinger. [Google Scholar]
- West B. T., Welch K. B., Galecki A. T. (2014). Linear mixed models: A practical guide using statistical software. Boca Raton, FL: CRC Press. [Google Scholar]
- Wistedt B., Rasmussen A., Pedersen L., Malm U., Träskman-Bendz L., Wakelin J., Bech P. (1990). The development of an observer-scale for measuring social dysfunction and aggression. Pharmacopsychiatry, 23, 249-252. doi: 10.1055/s-2007-1014514 [DOI] [PubMed] [Google Scholar]
- World Health Organization. (2007). Third milestones of a global campaign for violence prevention report, 2007: Scaling up. Geneva, Switzerland: World Health Organization. [Google Scholar]



