Structured Assessment of Violence Risk in Schizophrenia and Other Psychiatric Disorders: A Systematic Review of the Validity, Reliability, and Item Content of 10 Available Instruments

Abstract

Objectives: To undertake a systematic review on structured violence risk assessment tools in individuals with schizophrenia. Methods: A systematic search was conducted from 1990 to 2011 to identify violence risk assessment tools and studies examining their predictive validity. Item content of the identified instruments was analyzed, and areas under the curve (AUC) from the studies were extracted. In addition, an 11-item checklist was developed to assess the utility and psychometric properties of these tools. Results: Ten risk assessment tools designed to predict community violence in psychiatric patients were identified, but only 2 studies reporting predictive validity estimates in patients with schizophrenia were found (median AUC = 0.69; interquartile range = 0.60–0.77). When inclusion criteria was broadened to include studies measuring accuracy for any diagnostic group, mixed evidence of predictive validity was found, with median AUCs ranging from 0.62 to 0.85 depending on the population. Item content included mostly clinical, sociodemographic, and criminal history factors. As only 1 tool included a neurobiological item, a structured review of brain-based and cognitive risk factors for violence was included, and 3 clusters (neurocognitive ability, neurocognitive awareness, and attitudinal cognition) were identified. Conclusions: While a number of violence risk assessment tools exist that can be used to predict the likelihood of community violence in psychiatric patients, there is currently little direct evidence for their utility in individuals with schizophrenia. In addition, there is large variation in item content between instruments, and further research is necessary to determine whether the inclusion of alternative factors could improve risk assessment.

Introduction

Current treatment guidelines published by the American Psychiatric Association¹ and the UK’s National Institute for Health and Clinical Excellence² recommend that violence risk be assessed for individuals diagnosed with schizophrenia. While unstructured assessments of risk, in which mental health professionals use their clinical experience to make subjective judgments as to whether an individual is likely to offend, remain common,³ a recent metareview (a systematic overview of systematic reviews and meta-analyses) identified over 120 structured instruments that have been designed for use in predicting violence in psychiatric and correctional populations.⁴ Furthermore, questionnaire surveys suggest that some form of structured risk assessment is currently used by over 80% of forensic psychologists in the United States⁵ and in over 80% of forensic and 70% of general psychiatric hospitals in the United Kingdom.^3,6

Generally, structured risk assessment tools adopt 1 of 2 approaches: actuarial prediction or structured clinical judgment (SCJ). Actuarial risk assessment tools are composed of weighted factors found to have been empirically associated with the likelihood of future offending. Numerical values are assigned to each factor, item scores are summed, and the resulting risk score is translated into a probabilistic estimate of future violence. As they rely on statistical algorithms rather than professional judgment, actuarial tools are generally considered more reliable than clinical predictions.⁷ However, their reliance on predominantly historical information has led to arguments that actuarial instruments are not useful in helping clinicians make patient-centered treatment decisions.⁸

In the structured clinical approach to risk assessment, clinicians use a set of factors that are empirically or theoretically associated with offending to guide their predictions of future violence.⁹ Proponents argue that clinically based instruments take individual differences into consideration and can, therefore, be used to inform decisions relating to risk management and treatment planning. While the use of SCJ measures appears to be increasing in psychiatric settings,⁶ there is currently no clear evidence that such instruments outperform the predictive accuracy of actuarial tools.^4,10

Assessing Violence Risk in Schizophrenia

Using structured risk assessment instruments to predict violent outcomes in individuals diagnosed with schizophrenia poses a number of challenges. Although evidence for a positive association between schizophrenia and violence has been found in large-scale epidemiological investigations in different countries with varying designs and outcomes,¹¹ violent behavior is not common in this population (estimated prevalence between 10 and 15%).^12,13 This relatively low base rate has led some to argue that violent behavior cannot be predicted without a high number of false positives.¹⁴ In keeping, a meta-analysis by Large and colleagues¹⁵ found that in order to prevent one stranger homicide, 35 000 patients with schizophrenia judged to be high risk would need to be detained. Despite this, the public expectation for accurate risk assessment has remained high, with reviews of media literature¹⁶ and surveys conducted in Western countries¹⁷ continuing to identify schizophrenia as the hospitalized population most commonly associated with violence.

While previous reviews have explored the utility of risk assessment tools in psychiatric patients more generally,⁴ no review has investigated the evidence base for the utility of structured risk measures in solely schizophrenia. Given the number of tools available to clinicians for such purposes, composed of different factors and designed using different approaches to prediction, we have conducted a systematic review to identify and describe the psychometric properties and item content of those measures aiming to predict outpatient violence risk in psychiatric patients, including schizophrenia.

Methods

Search Strategy

Identifying Risk Assessment Tools.

Risk assessment tools designed to predict the likelihood of community violence in psychiatric populations were identified using PsycINFO, EMBASE, MEDLINE, the US National Criminal Justice Reference Service Abstracts, and combinations of the following Boolean keywords: violen*, risk, assess*, predict*, tool*, instrument*, measure*, mental*, and psychiatr*. The search was restricted to tools whose calibration studies (for actuarial instruments) or manuals (for SCJ instruments) had been published between January 1, 1990 and January 19, 2011. Additional instruments were located using previous reviews and through discussion with researchers in the field. Instruments in all languages were considered.

Actuarial risk assessment tools were included if their calibration sample was composed of adult psychiatric patients discharged from hospital, and interpersonal violence was the outcome. SCJ instruments were included if their manual stated that they were designed for predicting community violence in mentally disordered adults.

Instruments were excluded if they were designed solely to predict inpatient violence or if they were intended for use by a single institution or ward. Also excluded were tools developed to predict the likelihood of antisocial behavior solely while on temporary leave, as our primary interest was violent offending in individuals who had been formally discharged from hospital. Finally, instruments whose predictive validity has yet to have been measured using outpatient samples and multivariate models that currently lack guidelines for actuarial use were excluded.

The initial search for violence risk assessment tools identified a total of 158 instruments (see online supplementary material for Appendix 1). When personality assessments, behavioral checklists, and symptom scales were excluded, 104 eligible tools remained. When inclusion and exclusion criteria were applied, this number was reduced to 11 instruments. Though meeting initial inclusion criteria, the Violence Prediction Scheme¹⁸ was excluded because the tool’s authors recently stated that the measure should not be used to predict future violence.¹⁹ Therefore, the final number of risk assessment tools included was 10.

Identifying Predictive Validity Studies.

Studies investigating the predictive validity of the 10 tools were identified using the same databases and time specifications as above, using the acronyms and full names of the instruments as search criteria. Additional articles were located through reference lists, annotated bibliographies, and discussion with researchers in the field. Studies in all languages and those not published in peer-reviewed journals (ie, government reports, conference presentations, doctoral dissertations, and Master’s theses) were considered.

Studies were included if their titles, abstracts, or methods sections demonstrated testing of a tool’s validity in predicting community violence in psychiatric patients diagnosed with schizophrenia. When multiple predictive validity estimates were available because different tools were administered to the same participants, effect sizes were extracted for each of the tools and counted as different samples. Therefore, the same study could contribute multiple samples. When effect sizes were reported for multiple forms of violence, those estimates corresponding to the most sensitive form were included. We considered violent (including sexual) offending to be the most sensitive outcome followed by violent (nonsexual) offending and then sexual offending only.

When samples overlapped, that with the most participants was included to avoid double counting (see online supplementary material for Appendix 2). When overlapping samples contained the same number of participants, that which reported effect sizes for the fewest number of subgroups was included. In cases where unique indices of construct (convergent or divergent) validity or reliability were reported, studies were included regardless of whether their samples overlapped; however, only the predictive validity estimates from the largest of the overlapping samples were included in descriptive analyses.

Samples of remanded prisoners transferred for psychiatric evaluations were excluded as were samples composed of both discharged psychiatric patients and released prisoners where separate effect sizes were not reported for the patient group. Studies were also excluded if they only included select scales of a tool because we wished to compare complete instruments. Finally, studies were excluded if they measured the psychometric properties of a pilot version of one of the tools.

A preliminary search identified only 2 studies that met inclusion criteria and measured the predictive validity of 1 of the 10 instruments in discharged patients with schizophrenia.^20,21 Therefore, the search was broadened to any replication study that investigated predictive accuracy for psychiatric patients more generally because such studies would have included participants with schizophrenia. The initial search using these broader search limits identified a total of 2420 records, which was reduced to 113 on critically reading abstracts (see online supplementary material for Appendix 3). After inclusion and exclusion criteria were applied, a final total of 35 studies were identified (see online supplementary material for Appendix 4).

Data Extraction

One of the authors (J.S.) extracted 28 descriptive and demographic characteristics of the predictive validity studies. When information was unclear, S.F. was consulted. When predictive validity estimates were reported for multiple lengths of follow-up, effect sizes for the longest follow-up period were extracted. Effect sizes were converted to the area under the curve (AUC) using formulae published by Cohen²² and Lipsey and Wilson.²³ As variance parameters were commonly missing, median values were used for descriptive analyses of tool performance by instrument and across diagnostic groups.

Item Content Analyses

The item content of the 10 identified tools was classified into 4 categories: (1) criminal history, (2) sociodemographic, (3) clinical, and (4) neurobiological. A criminal history factor was defined as an item concerning previous antisocial acts (eg, frequency and severity of offending). A sociodemographic factor was defined as an item relating to an individual’s social functioning (eg, frequency and quality of social contact), living circumstances (eg, security level of setting in the community), or nonclinical background characteristics (eg, gender, age, ethnicity). A clinical factor was defined as an item concerning an individual’s psychological functioning (eg, impulsivity, coping, empathy) or a background characteristic potentially linked to psychopathology (eg, parental alcohol or drug use). A neurobiological factor was defined as an item relating to an individual’s brain function. In order to test the reliability of the extraction, a research assistant working independently of the authors (Christie Leung) classified items into these 4 categories after receiving copies of the calibration studies and the tools’ manuals (where applicable). A high level of interrater agreement was found (κ = 0.88).²⁴ When raters differed in their classification of an item, S.F. was consulted and a consensus was reached.

Content and Reporting Characteristics Checklist

Following the approach of Fitzpatrick and colleagues²⁵ in their systematic review of measures in forensic mental health research, 2 of the authors (J.S. and S.F.) developed a checklist to examine item content and reporting characteristics. The checklist was composed of 11 items pertaining to risk assessment tools organized into 3 scales (item content, validity, and reliability):

1. Are static factors included as item content?

2. Are dynamic factors included as item content?

3. Are risk factors included as item content?

4. Are protective factors included as item content?

5. Has the predictive validity of the instrument been measured prospectively?

6. Has the predictive validity of the instrument been measured in civil psychiatric patients?

7. Has the predictive validity of the instrument been measured in forensic psychiatric patients?

8. Has the convergent validity of the instrument been tested against other risk assessment tools?

9. Has the divergent validity of the instrument been tested against other risk assessment tools?

10. Has the interrater reliability of the instrument been measured?

11. Has the internal consistency of the instrument been measured?

A static factor was defined as a historical or unchangeable trait, while a dynamic factor was defined as a present potentially changeable mental state or condition. Items were considered risk factors if their presence was associated with an increase in the likelihood of future violence, whereas items were considered protective factors if their presence was associated with a decrease in the likelihood of violence. Predictive validity was defined as the ability to accurately identify patients who would or would not commit an act of violence when discharged into the community. A prospective study was defined as an investigation in which a risk assessment tool was administered and participants were followed longitudinally for outcomes. Forensic patients were defined as individuals who were receiving psychiatric treatment in a secure hospital, whereas civil patients were those individuals who were engaged with mental health services but not in secure settings. Convergent validity was defined as the degree to which a risk assessment tool positively correlated with another tool that measured the same construct (ie, risk for or protection against future violence), whereas divergent validity was defined as the degree to which a risk assessment tool did not correlate with another instrument that measured a different construct. Interrater reliability was defined as the consistency of instrument scores or risk judgments when conducted by more than 1 rater. Finally, internal consistency was defined as the degree to which items on a risk assessment tool were correlated with each another.

Results

Tool Selection

The following 10 instruments were identified as risk assessment tools designed to predict the likelihood of community violence in psychiatric samples: the Classification of Violence Risk (COVR),²⁶ the Historical, Clinical, Risk Management-20 (HCR-20),⁹ the Historische, Klinische, Toekomstige-30 (HKT-30),²⁷ the Structured Assessment of Protective Factors (SAPROF),⁸ the Structured Outcome Assessment and Community Risk Monitoring (SORM),²⁸ the Short-Term Assessment of Risk and Treatability (START),²⁹ the Violence Risk Appraisal Guide (VRAG),¹⁹ the Violence Risk Screening-10 (V-RISK-10),³⁰ the Violence Risk Scale (VRS),³¹ and an actuarial instrument developed as part of the UK700 study to predict violence in individuals diagnosed with psychotic disorders.³² The descriptive characteristics of these measures are outlined in table 1.

Table 1.

Descriptive Characteristics of the 10 Violence Risk Assessment Tools Included in the Systematic Review

Tool	Current Manual	Act vs SCJ	Developed for Forensic vs Civil Patients	Items	Scoring System	Scales
COVR	Monahan et al26	Act	Civil	Variable	N/A (iterative classification tree)	N/A
HCR-20	Webster et al9	SCJ	Forensic	20	+0 = item not present	Historical factors
					+1 = item possibly present	Clinical factors
					+2 = item definitely present	Risk management factors
HKT-30	Werkgroep Pilotstudy Risicotaxatie Forensische Psychiatrie27	SCJ	Forensic	30	+0 = item suggests a low risk of violence	Historical factors
					+1 = item suggests a low-moderate risk of violence	Clinical factors
					+2 = item suggests a moderate risk of violence	Future factors
					+3 = item suggests a moderate-high risk of violence
					+4 = item suggests a high risk of violence
SAPROF	de Vogel et al8	SCJ	Forensic	17	+0 = item not present	Internal factors
					+1 = item present to some extent	Motivational factors
					+2 = item clearly present	External factors
SORM	Grann et al28	SCJ	Forensic	30	N/A (different scoring systems and a statement of risk/protective effect for each item)	Current services and interventions
						Social situation
						Social network
						Clinical factors
						Subjective ratings
						Criterion variables
START	Webster et al29	SCJ	Forensic	20	±0 = item is not a strength/vulnerability	Strengths
					±1 = item indicates a moderate strength/vulnerability	Vulnerabilities
					±2 = item indicates a substantial strength/vulnerability
UK700 riskinstrument	Wootton et al32	Act	Civil	4	Score +11 if assault in previous 2 years	N/A
					Add +2 if male
					Add +6 if drug use in previous year
					Add +20
					Subtract 0.3*age at assessment
					Divide total by 3.5
VRAG	Quinsey et al19	Act	Forensic	12	N/A (different weighted values for different items)	N/A
V-RISK-10	Hartvig et al30	SCJ	Civil	10	+0 = item does not apply	N/A
					+1 = item may apply to a moderately severe degree
					+2 = item definitely applies to a severe degree
VRS	Wong and Gordon31	Act	Forensic	26	+0 = item suggests no increased risk of violence	Static factors
					+1 = item suggests a low risk of violence	Dynamic factorsa
					+2 = item suggests a moderate risk of violence
					+3 = item suggests a high risk of violence

Note: Act, actuarial instrument; SCJ, structured clinical judgment instrument; COVR, Classification of Violence Risk; HCR-20, Historical, Clinical, Risk Management-20; HKT-30, Historische, Klinische, Toekomstige-30; SAPROF, Structured Assessment of Protective Factors; SORM, Structured Outcome Assessment and Community Risk Monitoring; START, Short-Term Assessment of Risk and Treatability; VRAG, Violence Risk Appraisal Guide; V-RISK-10, Violence Risk Screening-10; VRS, Violence Risk Scale.

^aScores on the dynamic scale of the VRS can change with different stages of treatment.

Study Characteristics

Information was collected on 11 720 participants in 46 samples from 35 studies that investigated the ability of these 10 instruments to accurately predict community violence. The tools with the most samples were the HCR-20 (k samples = 20; 43.5%), VRAG (k = 9; 19.6%), and HKT-30 (k = 5; 10.9%). The majority of the samples (k = 30; 65.2%) used an SCJ instrument. There was a trend of more predictive validity studies having been published in recent years (χ² trend = 7.07, P < .01). Studies were conducted in 7 countries: Canada (n = 100; 0.9% of all participants), The Netherlands (n = 1558; 13.3%), Norway (n = 381; 3.3%), Sweden (n = 638; 5.4%), the United Kingdom (n = 7102; 60.6%), the United States (n = 1837; 15.7%), and Yugoslavia (n = 104; 0.9%). Instruments were administered by doctoral-level mental health professionals in 8 (17.4%) samples, computers in 4 (8.7%) samples, nurses in 1 (2.2%) sample, graduate students in 1 (2.2%) sample, and by a mix of qualified individuals in 3 (6.5%) samples. Risk assessment tools were scored using file review information only in 25 (54.4%) samples, interview or self-report only in 6 (13.0%) samples, and a mixture of sources in 13 (28.3%) samples. All but one of the samples (97.8%) was identified using convenience-sampling methodology. Additional demographic and study design characteristics are provided in table 2.

Table 2.

Descriptive and Demographic Characteristics of 46 Samples Investigating the Utility of Risk Assessment Tools in Predicting Community Violence in Psychiatric Populations

Category	Subcategory	Group	Number of k = 46 (%)
Study information	Source of study	Journal article	32 (69.6)
		Conference	5 (10.9)
		Dissertation/Thesis	4 (8.7)
		Government report	3 (6.5)
		Manual	2 (4.3)
	Study location	United States	3 (6.5)
		United Kingdom	19 (41.3)
		Other European Union	23 (50.0)
		Canada	1 (2.2)
Tool information	Type of tool	Actuarial	16 (34.8)
		SCJ	30 (65.2)
	Tool used	COVR	4 (8.7)
		HCR-20	20 (43.5)
		HKT-30	5 (10.9)
		SAPROF	2 (4.3)
		SORM	1 (2.2)
		START	1 (2.2)
		UK700 risk instrument	1 (2.2)
		VRAG	9 (19.6)
		V-RISK-10	1 (2.2)
		VRS	2 (4.3)
Study methodology	Study design	Prospective	15 (32.6)
		Retrospective	31 (67.4)
	Study setting	Forensic psychiatric	35 (76.1)
		Civil psychiatric	8 (17.4)
		Mixed	3 (6.4)
Outcome characteristics	Length of follow-up (mo)	Mean (SD)	43.2 (39.8)
	Type of outcome	Charge/arrest/conviction	26 (56.5)
		Self-report	0 (0.0)
		Collateral report	1 (2.2)
		Mixed	15 (32.6)
		Unstated/Unclear	4 (8.7)
Sample size	Mean (SD)		261 (303)
Diagnosis of schizophrenia	Mean number of participants per sample (SD)		87 (76)
Diagnosis of psychotic disorder	Mean number of participants per sample (SD)		139 (205)
Age	Mean age in years (SD)		35.4 (4.7)
White ethnic background	Mean number of participants per sample (SD)		237 (251)
Male sex	Mean number of participants per sample (SD)		217 (267)

Note: Abbreviations are explained in the first footnote to table 1. k, number of samples.

Item Content Analysis

The most common items were clinical factors followed by sociodemographic characteristics, criminal history variables, and neurobiological factors (table 3). The clinical item that appeared most frequently was previous and/or current substance abuse (N tools = 9). The most common sociodemographic item was employment (N = 7). The most common criminal history variable was any previous and/or current violence (N = 6). Finally, only 1 neurobiological factor was identified: history of head injury with or without loss of consciousness (N = 1).

Table 3.

Descriptive Analysis of Item Content From 10 Risk Assessment Tools Designed to Predict the Likelihood of Community Violence in Psychiatric Populations

Category	Item Content	Number of N = 10 Tools
Criminal history	Previous and/or current violence (general)	••••••
	Prior conditional release failure/escapes	••••
	Previous nonviolent offense	•••
	Inpatient aggression or self-harm	••
	Frequency and seriousness of arrests as an adult	•
	Frequency and seriousness of arrests as a juvenile	•
	Gender of victim (for index offense)	•
	Parental history of arrest	•
	Previous and/or current threats (verbal or physical)	•
	Previous and/or current use of weapons	•
	Previous property offense	•
	Previous sentences resulting in incarceration/hospitalization	•
	Previous sexual offense	•
	Severity of victim injury (for index offense)	•
Sociodemographic	Employment	•••••••
	Exposure to destabilizers	••••••
	Social support	•••••
	Economic/financial situation	••••
	Intimate relationship	••••
	Age at assessment	•••
	Leisure/recreational activities	•••
	Age at first violent conviction/incident	••
	Frequency and quality of socializing	••
	Gender	••
	Homeless	••
	Lived with both biological parents during childhood	••
	Living conditions supervised by mental health professionals	••
	Marital history	••
	Occupational skillset/training	••
	Social skills	••
	Acculturation problems	•
	Age at first hospitalization	•
	Age at index offense	•
	Contact with biological, adopted, or stepchildren	•
	Contact with family	•
	Criminal peers	•
	Ethnicity	•
	Level of education	•
	Living alone	•
	Living in private residence	•
	Security level of release setting	•
Clinical	Previous and/or current substance abusea	•••••••••
	Insight into illness and/or behavior	••••••
	Previous and/or current symptoms of psychosis	••••••
	Coping	•••••
	Impulsivity/impulse control	•••••
	Medication/medication compliance	•••••
	Previous and/or current diagnosis of a personality disorder	•••••
	Treatability/motivation for treatment	•••••
	Criminal/negative attitudes	••••
	Planning/future plans	••••
	Psychopathy	••••
	Self-care/ability to perform daily chores	••••
Clinical (cont’d)	Affective state (elevated or depressed mood)	•••
	Agreement on conditions/rules	•••
	Early adjustment	•••
	Empathy	•••
	Previous and/or current diagnosis of major mental illness	•••
	Contact with community support worker	••
	History of inpatient psychiatric treatment	••
	Hostile reaction style	••
	Mandatory treatment or probation order	••
	Previous and/or current diagnosis of a psychotic disorder	••
	Previous and/or current diagnosis of schizophrenia	••
	Previous and/or current symptoms of anxiety	••
	Responsibility toward crime	••
	Suicidal ideation, threats, or previous suicide attempt	••
	Attachment style in childhood	•
	Barratt Impulsiveness Scale	•
	Contact with physical health care services	•
	Global Assessment of Functioning	•
	History of treatment in correctional institution	•
	Homicidal thoughts	•
	Intelligence	•
	Novaco Anger Scale	•
	Number of prior hospitalizations	•
	Personal problem present at intake	•
	Previous and/or current diagnosis of a mood disorder	•
	Recent change in medication	•
	Schedule of Imagined Violence	•
	Self-report of future violence risk	•
	Self-report of overall physical and mental health	•
	Self-report of overall quality of life	•
	Sexual preoccupation	•
	Stress level	•
	Suspiciousness	•
	Parental alcohol abuse	•
	Parental drug abuse	•
	Parental hospitalization	•
	Parents argued	•
	Victim of child abuse	•
	Violent lifestyle	•
Neurobiological	History of head injury (with or without LOC)	•

Note: •, item present on one risk assessment tool; LOC, loss of consciousness. When items on the Classification of Violence Risk overlapped (eg, diagnosis of drug or alcohol abuse, drug abuse, alcohol abuse), the content was pooled and considered a single item. Therefore, the 106 potential risk factors described by Monahan and colleagues³⁵ were reduced to 41 factors.

^aItem appears on both the Clinical and Future scales of the Historische, Klinische, Toekomstige-30 but was only counted once for this instrument.

Content and Reporting Characteristics Checklist

Each of the 10 tools was graded using our 11-item checklist (table 4). Those instruments meeting the most checklist criteria included the HCR-20 (N criteria satisfied = 10), HKT-30 (N = 8), VRAG (N = 8), and VRS (N = 8). The most common criterion met was the inclusion of risk factors (N = 9). In relation to validity, it was the prospective measurement of predictive validity (N = 7), and on the reliability subscale, it was the testing of interrater reliability (N = 8). Less common were the inclusion of protective factors as item content (N = 5), the testing of divergent validity (N = 4), and the measurement of internal consistency (N = 1).

Table 4.

Overview of the Item Content of 10 Violence Risk Assessment Tools Developed for Psychiatric Populations and the Reporting Characteristics of Their Outpatient Prediction Literatures

Scale	Grading Criteria	Risk Assessment Tool
		COVR	HCR-20	HKT-30	SAPROF	SORM	START	UK700	VRAG	V-RISK-10	VRS
Item content	Static factors included	•	•	•	•	—	—	•	•	•	•
	Dynamic factors included	•	•	•	•	•	•	—	—	•	•
	Risk factors included	•	•	•	—	•	•	•	•	•	•
	Protective factors included	—	—	—	•	•	•	•	•	—	—
Validitya	Predictive validity tested prospectively	•	•	•	—	•	—	—	•	•	•
	Predictive validity tested in civil psychiatric patients	•	•	•	—	—	—	•	•	•	—
	Predictive validity tested in forensic psychiatric patients	—	•	•	•	•	•	—	•	—	•
	Convergent validity tested	—	•	•	•	—	•	—	•	—	•
	Divergent validity tested	—	•	—	•	—	•	—	—	—	•
Reliabilitya	Interrater reliability tested	—b	•	•	•	•	•	—	•	•	•
	Internal consistency tested	—c	•	—	—	—	—	—	—	—	—

Note: Abbreviations are explained in the first footnote to table 1.

^aIn the context of predicting the likelihood of community violence.

^bAs the COVR is administered using computer software and self-report, the instrument’s interrater reliability cannot be evaluated.

^cAs the COVR uses an iterative classification tree to assess violence risk, the item content on the instrument will differ depending on the individual. Therefore, internal consistency cannot be evaluated.

Psychometric Properties

Predictive Validity.

Independent predictive validity estimates were reported in 44 (95.7%) of the included samples. Performance indicators included the AUC (k cases = 43), sensitivity (k = 8), odds ratio (OR; k = 7), specificity (k = 6), positive predictive value (PPV; k = 4), negative predictive value (NPV; k = 4), product-moment correlation coefficient (r_pm; k = 2), false positive rate (k = 1), and the number needed to detain (NND; k = 1). At least one AUC, OR, or r_pm significantly above chance was reported for all of the included instruments. A comparison of these effect sizes converted into AUCs is provided in table 5. For 2 instruments (HCR-20 and VRAG) effect sizes were reported for different diagnostic groups (table 5). One interesting finding was that there appeared to be high levels of heterogeneity in these effect sizes across diagnostic groups.

Table 5.

Comparison of the Median AUC Produced by Risk Assessment Tools Designed for Predicting Violence in Psychiatric Populations When Used With Outpatient Samples in Different Diagnostic Categories

Tool	Diagnostic Category	n	k	LOFU (mo)	Median	IQR
HCR-20	Psychosisa,S6,S9	696	2	18.0	0.71	0.67–0.74
	Mental retardationS8,S9	260	2	42.0	0.80	0.79–0.80
	Mood disorderS9	100	1	24.0	0.67	—
	Personality disorderS9	160	1	24.0	0.62	—
	PTSDS13	104	1	12.0	0.85	—
	Substance abuseS9	116	1	24.0	0.65	—
	Heterogeneous samplesS1–S5,S7–S12,S14–S17,S34	3365	16	55.9	0.69	0.65–0.80
VRAG	SchizophreniaS21,S22	165	2	64.4	0.69	0.60–0.77
	Mental retardationS8	145	1	60.0	0.73	—
	Heterogeneous samplesS5,S8,S18–S20,S34	2963	6	37.1	0.72	0.66–0.76
COVR	Heterogeneous samplesS23–S26	1520	4	4.6	0.70	0.61–0.79
HKT-30	Heterogeneous samplesS12,S17,S27–S29	492	5	88.3	0.72	0.65–0.73
SAPROF	Heterogeneous samplesS1	188	1	36.0	0.67	—
SORM	Heterogeneous samplesS7	74	1	10.0	0.71	—
START	Heterogeneous samplesS30	50	1	12.0	0.77	—
UK700	Heterogeneous samplesS31	708	1	24.0	0.71	—
V-RISK-10	Heterogeneous samplesS32	381	1	12.0	0.75	—
VRS	Heterogeneous samplesS33,S34	164	2	58.8	0.73	0.68–0.78

Note: Abbreviations are explained in the first footnote to table 1. n, sample size; k, number of samples; LOFU, length of follow-up (fixed effects mean where applicable); IQR, interquartile range; PTSD, post traumatic stress disorder; AUC, area under the curve. Heterogeneous samples were those composed of <100% participants diagnosed with any specific category of disorder. Final risk judgment AUCs were reported where available. The number of diagnostic samples does not equal the total number of HCR-20 and VRAG samples because effect sizes for multiple diagnostic groups were reported in one study.^S9 References for all samples in online supplementary material for Appendix 4.

^aIncluding diagnoses of schizophrenia, schizophreniform disorder, schizoaffective disorder, delusional disorder, and schizotypal personality disorder (International Classification of Diseases [ICD-10] codes F20–F29).

Construct Validity and Reliability.

Convergent validity estimates were reported for 6 tools: the HCR-20, HKT-30, SAPROF, START, VRAG, and VRS (see online supplementary material for Appendix 5). Evidence of significant divergent validity was found for the HCR-20, SAPROF, START, and VRS.

Independent reliability estimates were reported for 27 (58.7%) samples. At least 1 intraclass correlation coefficient above 0.75 or kappa coefficient (κ) above 0.80, indicative of excellent interrater reliability,^24,33 was identified for the HCR-20, HKT-30, SAPROF, SORM, START, VRAG, V-RISK-10, and VRS. In addition, the HCR-20 was found to have a high level of internal consistency (α = .82).³⁴

Discussion

This systematic review has investigated the utility of structured risk assessment in predicting community violence in psychiatric populations, including individuals with schizophrenia. Ten assessment schemes were identified, 7 that were developed for mentally disordered offenders, and 3 others that were designed for use with civil psychiatric patients. We collected data on the psychometric and predictive qualities of these tools from 46 samples (based on 35 studies) involving 11 720 patients in 7 countries. Our review explored 3 main areas: (1) whether available risk assessment tools are useful in predicting community violence in patients with schizophrenia, (2) what is the evidence base for the psychometric properties of these instruments, and (3) what items frequently appear on these measures.

The principal finding of this review was that there was little direct evidence to support the use of these risk assessment tools in schizophrenia, specifically. Rather, the psychometric properties and predictive accuracy of available instruments have been studied in generic psychiatric samples, and the assumption is that these properties are applicable to patients with schizophrenia. As an implication of this finding is that future research should be conducted on available instruments, we have presented an 11-item checklist to enable clinicians to determine the goodness of fit in terms of content as well as the strength of evidence in support of any particular risk measure. We also found that only 1 neurobiological factor appears on the instruments, that of head injury.

Predictive Utility in Schizophrenia

Although our initial aim was to review risk assessment instruments in schizophrenia, we identified only 2 studies that reported effect sizes specifically for this diagnostic group.^20,21 Therefore, we also included more general psychiatric samples because they would have included patients with schizophrenia. For 1 of the instruments, there was a trend for lower predictive accuracy in schizophrenia than other diagnoses. However, as the number of studies comparing schizophrenia with other diagnoses was limited, these findings should be viewed as hypothesis generating at most.

Replication studies of 3 instruments, the VRAG, HCR-20, and the UK700 risk instrument, included over 50% participants with psychosis but did not provide separate effect sizes for schizophrenia. Thus, one implication of the review is that reporting psychometric properties for specific diagnostic groups, if statistical power permits, would be helpful. As most studies provided information on heterogeneous samples of psychiatric patients, it is uncertain how the utility of risk assessment differs by diagnosis. For example, the same instrument may perform well in identifying high-risk individuals diagnosed with substance abuse but poorly in persons with mood disorder. Future reviewers could quantitatively synthesize individual participant data from primary studies in order to improve statistical power and explore utility by psychiatric diagnosis.

A related issue concerns the effect sizes used to report on predictive validity. Nearly all of the studies included in the present review reported the AUC as their primary outcome statistic. The AUC is currently considered the effect size of choice in the risk assessment literature because it provides a global estimate of tool utility and is independent of the base rate of violence.³⁶ The AUC does not, however, discriminate between instruments that perform well at identifying high vs low risk individuals. Reliance on the AUC as the effect size of choice in this literature provides one possible explanation as to why no diagnosis-specific instruments for predicting community violence are currently available: “High” AUC values for heterogeneous groups of psychiatric patients may have led researchers, clinicians, and policymakers to believe that instruments perform well for all diagnostic groups. However, it is problematic to suggest that structured instruments would be able to identify high-risk individuals with the same accuracy in groups with higher and lower base rates of violence.

It could be argued that judgments of dangerousness cannot be made without taking the base rate of violence into account. Therefore, base rate dependent outcome statistics such as the PPV—the proportion of individuals who are predicted to offend who actually offend—and the NPV—the proportion of individuals who are predicted by a tool not to offend who do not—may be useful in the psychiatric risk assessment literature. We recommend that future studies that measure predictive validity for specific diagnostic groups report at least 3 effect estimates: a global effect size (eg, AUC or the diagnostic odds ratio), a “rule in” effect size which provides information about how well a tool performs in identifying high-risk individuals (eg, PPV or the NND), and a “rule out” effect size that provides information about how accurately a tool identifies low-risk individuals (eg, NPV).

Content and Reporting Characteristics Checklist

We developed an 11-item checklist to assess the content and psychometric evidence bases of risk assessment tools. None of the reviewed instruments met all 11 criteria, with the average tool fulfilling 7 items. Researchers may find this checklist helpful when deciding which psychometric properties to report (predictive validity, construct validity, interrater reliability, internal consistency), especially in relation to divergent validity and internal consistency estimates, which have not been reported for more than half of the instruments included in the present review. In addition, clinicians and policymakers could use the checklist as an adjunct for assessing a particular instrument's evidence base before deciding whether to implement it into clinical practice.

Item Content

The identified tools measured primarily criminal history, sociodemographic, and clinical factors, of which, in order of frequency, substance abuse, employment status, and markers of previous crime were the most common. More recent cohort studies^12,13 and narrative reviews³⁷ on violence risk in schizophrenia underline the importance of these factors. At the same time, there were many items that appeared on only 1 of the 10 included instruments, suggesting that an updated review of risk factor research is necessary for their further development. As few neurobiological factors were included as item content (history of head injury included on one measure), a synthesis of the evidence on brain-based and cognitive (collectively “neurocognitive”) risk factors associated with violence in schizophrenia could begin this work.

Neurocognitive Factors.

A number of biological and cognitive factors, many of which have not been included on currently available instruments, have been found to be associated with violence in individuals diagnosed with schizophrenia. While it is uncertain whether the inclusion of such items would increase predictive validity, neurocognitive factors offer a different perspective, and for a clinician to perform a comprehensive assessment of violence risk, that perspective may be useful.

A search of PsycINFO and MEDLINE was performed for articles published prior to April 17, 2011 using combinations of the following Boolean keywords: aggression, schizophrenia, risk, cognit*, neurocognit*, and neurobiological. Our definition of cognition, as commonly used in schizophrenia research, encompasses a broad engagement of learning and understanding of the world. Our definition is limited to performance-based abilities that include “the ability to attend to things in a selective and focused way, to concentrate over a period of time, to learn new information and skills, to plan, to determine strategies for actions and to execute them, to comprehend language and to use verbal skills for communication and self-expression, and to retain information and manipulate it to solve complex problems”.^38(p25) All these abilities are impaired to some extent in individuals with schizophrenia³⁸ and may be implicated for increased risk for aggressive behavior. On the basis of the electronic search, 2 of the authors (M.S. and J.R.) identified 3 biological-cognitive factor clusters that have been associated with violent offending in this population: neurocognitive ability, neurocognitive awareness, and attitudinal cognition.

Neurocognitive Ability

Prefrontal cortical dysfunction has been implicated as a possible anatomical correlate of violent behavior. Damage to the prefrontal area is associated with heightened aggression, emotional outbursts, disorganization, and risk-taking behavior.³⁹ Executive functioning is believed to be subserved by prefrontal cortical functioning as indexed by the ability to inhibit behavioral responses to stimuli, solve problems, adapt to changes in the environment and learn from the consequences of behavior. Executive dysfunction is very prominent in schizophrenia. Numerous investigators have reported that patients with schizophrenia with histories of aggression manifest poorer cognitive functioning than their nonviolent peers on cognitive tasks, particularly executive functioning tasks.^40,41 Serper et al⁴² examined the relationship between executive functioning, psychiatric symptomatology, and aggressive behavior in 85 patients presenting with schizophrenia over an acute hospital admission using structure equation modelling techniques. In this study, it was found that the level of patients’ executive impairment significantly predicted the formation of positive and negative symptoms, which in turn significantly contributed to the manifestation of inpatient aggressive behavior. Executive dysfunction also directly predicted inpatient aggressive behavior. Combining the indirect and direct effects, 59% of the inpatient aggression measure factor variance was accounted for by the measures of patients with executive dysfunction and clinical symptom severity. It was concluded that executive dysfunction may not possess the behavioral inhibition skills needed to cope with the presence of symptoms and other stressful events that accompany acute psychosis and hospitalization. This may result, consequently, in increased manifestations of aggressive behavior. While this study used inpatient aggression as its outcome, the link between executive dysfunction and aggression may be generalizable to outpatient antisocial behavior. Enhancing cognitive functioning may have treatment implications for schizophrenia patients who have prior histories of aggressive behavior. In a recent study, improvement of cognitive functioning in patients with schizophrenia has been found to decrease aggression. Krakowski et al⁴¹ found improvement on a composite index of cognitive performance (which included executive functioning) was significantly associated with a decrease in aggression in schizophrenia patients treated with olanzapine.

Not all studies, however, support the association between cognitive deficits and aggression in schizophrenia.⁴³ Mixed findings have been reported to be associated with varying definitions of aggression, participant selection characteristics, and methodological heterogeneity. While further investigation is necessary, the evidence to date implies that aggression takes 2 forms: manipulative, cold, premeditated, executively complex, antisocial aggression vs impulsive, hot, executively simple aggression. The former would contain thoughtout aggression designed to gain power or status or to inflict injury. The latter would contain impulsive reactionary aggression designed to defend against perceived threats. This two-path system may explain the discrepant findings and concords with Naudts and Hodgins’ meta-analysis, where they posit that patients with a longer aggressive history show better executive functioning than those with a short or no aggressive history.⁴³ To date, it is unknown whether varying executive patterns predict different types of aggression motivation in individuals with schizophrenia.

Neurocognitive Awareness

The cognitive awareness literature is diverse and controversial, involving intangible elements that are difficult to measure validly. Elements of awareness potentially related to violence include insight into one’s own illness and behavior and empathy for the target of aggression, both of which have been included on currently available risk assessment instruments (eg, HCR-20, SAPROF, START), as well as awareness of one’s own emotion, awareness of aggressive thoughts, and awareness of one’s life situation and behavioral tendencies. Many potential deficits have not been studied sufficiently to assess their relationship to aggression in schizophrenia, particularly theory of mind. Theory of mind, or the meta-cognitive representation of others’ mental states, is theorized to inhibit antisocial behavior and therefore to be deficient in people who aggress. Development of theory of mind in children is associated with a decrease in proactive aggression.⁴⁴ Perhaps also relevant is the work of Goldberg and colleagues,⁴⁵ who found that inpatients who committed acts of aggression were differentiated from their nonaggressive peers by exhibiting significantly higher self-serving theory of mind biases, attributing more positive attributes to themselves relative to their perceptions of how others viewed them.

A related awareness concept is insight into illness. A lack of insight, or anosognosia, is common in individuals with severe mental illness. Many studies have found significant negative correlations between insight into illness and aggression in patients with schizophrenia.^13,44,46,47 However, medication nonadherence, positive symptoms, and psychopathy have been proposed as common-cause explanations of this covariance. Treatment noncompliance alone predicts an increase in the likelihood of violent behavior.⁴⁸ Evidence from studies appears to conflict whether poor insight and treatment noncompliance independently predict increased aggression⁴⁴ or if poor insight only increases the risk of violence when substance abuse and medication noncompliance are also present.⁴⁸ As the insight-compliance relationship may be bidirectional, there is need for further specification. Likewise, Lincoln and Hodgins⁴⁶ found that the correlation between insight and aggression did not add significant variance beyond that from patients’ psychopathy and psychoticism. Interestingly, there is evidence that low insight is rooted in frontal lobe dysfunction, suggesting that poor neurocognitive capability may affect aggressive behavior in schizophrenia through its anosognosic effects.⁴⁹ Other awareness factors such as quality of life, suicidal ideation, and empathy have limited or no evidence for or against a link. Interestingly, the literature search also did not identify any studies exploring a link between homicidal ideation and violent behavior in patients diagnosed with schizophrenia.

One promising direction in need of further investigation is the role of emotional awareness in aggression in individuals with schizophrenia. Individuals vary in the degree to which they value and pay attention to their own emotional state. These differences refer to the degree that individuals notice, think about, and monitor their mood states.⁵⁰ Low emotional awareness has been found to predict higher psychoticism,⁵⁰ and it may thereby explain part of the hot impulsive type of aggression. No studies (to our knowledge) have explored this potential relationship.

Attitudinal Cognitions

Attitudinal cognitions, specifically hostile attribution biases, have received attention in violence risk research. The hostile attribution bias has been linked to paranoia in prodromal, first episode, and chronic schizophrenia.⁵¹ Berman et al⁵² found that the attributions of omnipotence and malevolence to hallucinated voices were robust predictors of aggressive behavior. Attributions of personal powerlessness and inadequacy that patients make about their psychotic experiences are also reflected in their interpersonal attributions toward others.⁵² A sense of being powerless and controlled by others may lead hallucinators to harbor shame, hostility, and resentment in interpersonal situations. It may be the case that individuals with schizophrenia who harbor a social sense of powerlessness and inferiority may increase their level of hostility and paranoia which could result in acts of aggression directed toward those who they see as controlling or manipulating them.⁵²

Conclusions

A number of structured risk assessment tools exist that can be used to predict the likelihood of community violence in individuals diagnosed with schizophrenia. There is some evidence for the validity and reliability of these instruments when used with psychiatric patients. However, little evidence exists for the psychometric properties of these tools in individuals with schizophrenia, specifically. Updated reviews of risk factors may assist in developing more accurate instruments, and we have presented one such review on neurocognitive factors.

Supplementary Material

Supplementary material is available at http://schizophreniabulletin.oxfordjournals.org.