Multisite study of Aggressive Behavior Risk Assessment Tool in emergency departments

Abstract

Objective

Violence is a major preventable problem in emergency departments (EDs), and validated screening tools are needed to identify potentially violent patients. We aimed to test the utility of the Aggressive Behavior Risk Assessment Tool (ABRAT) for screening patients in the ED.

Methods

A prospective cohort study was conducted among adult and pediatric patients aged ≥10 years visiting 3 emergency departments in Michigan between May 1, 2021, and June 30, 2021. Triage nurses completed the 16‐item checklist using electronic health records (EHRs), and the occurrence of violent incidents were collected before ED disposition. A multivariate logistic regression model was applied to select a parsimonious set of items.

Results

Among 10,554 patients, 127 had ≥1 violent incidents (1.2%). The regression model resulted in a 7‐item ABRAT for EDs, including history of aggression and mental illness and reason for visit, as well as 4 violent behavior indicators. Receiver operating characteristics analysis showed that the area under the curve was 0.91 (95% confidence interval [CI], 0.87–0.95), with a sensitivity of 84.3% (95% CI, 76.5%–89.9%) and specificity of 95.3% (95% CI, 94.8%–95.7%) at the optimal cutoff score of 1. An alternative cutoff score of 4 for identifying patients at high risk for violence had a sensitivity and specificity of 70.1% and 98.9%, respectively.

Conclusion

The ABRAT for EDs appears to be a simple yet comprehensive checklist with a high sensitivity and specificity for identifying potentially violent patients in EDs. The availability of such a screening checklist in the EHR may allow rapid identification of high‐risk patients and implementation of focused mitigation measures to protect emergency staff and patients.

1. INTRODUCTION

1.1. Background

Violence in emergency departments (EDs) is a major preventable problem for healthcare workers and patients. Recently, the American College of Emergency Physicians (ACEP) and Emergency Nurses Association (ENA) launched a joint campaign, “No Silence on ED Violence,” in an effort to reduce ED violence, protect emergency staff, and increase awareness. ¹ Compared with other nurses, emergency nurses were more likely to experience physical assaults. ² A survey of 7169 emergency nurses by ENA showed that 54.5% and 12.1% experienced overall violence and physical assaults, respectively, during the previous week. ³ Furthermore, 13.4% of the nurses subjected to physical violence sustained injuries, mostly blunt trauma, contusions, or bruises. Similarly, a survey of 3539 emergency physicians showed that 47% had been physically assaulted, primarily by patients. ⁴ Notably, 77% of the emergency physicians felt that ED violence has a negative impact on patient care.

In view of the high incidence of violence in EDs and obvious negative consequences, there is an urgent need to prevent or mitigate such events. Although universal precaution for violence has been recommended by the Occupational Safety and Health Administration (OSHA) for healthcare workers, ⁵ such an indiscriminate expectation of violence from all patients in the ED could dilute the effectiveness of violence mitigation measures. Furthermore, emergency staff taking such a precaution with every patient would further raise barriers to cultivating therapeutic relationship with patients. An alternative violence mitigation strategy is the identification of those patients at high risk of violence so that focused preventive measures can be implemented proactively. The Joint Commission has suggested using a checklist or questionnaire to assess patient irritability, confusion, or threatening behavior. ⁶ Compared with unstructured clinical judgment for detecting potentially violent patients, a structured risk assessment tool was found to be more accurate. ⁷

1.2. Importance

There is a dearth of validated violence risk assessment tools useful for EDs. Before a tool can be considered useful, it must be validated in a prospective study showing satisfactory sensitivity and specificity. The sensitivity indicates the tool's ability to correctly identify violent patients prospectively of all those who become violent. ⁸ If a tool has low sensitivity, many violent patients would be missed. In contrast, the specificity measures the tool's ability to correctly identify non‐violent patients prospectively of all non‐violent patients. If a tool's specificity is low, many of the non‐violent patients could be wrongly labeled as violent. Furthermore, the violence risk assessment tools should also include the common behavioral indicators of violence. ⁹ , ¹⁰ A past history of violence is also an important indicator of violent behaviors in the ED. ¹¹ Currently available violence risk assessment tools for potential use in EDs include the STAMP framework, ¹² Broset Violence Checklist (BVC), ¹³ and Aggressive Behavior Risk Assessment Tool (ABRAT). ¹⁴

The STAMP violence risk assessment framework includes 5 potential violent behavior indicators observed in an ED setting: staring, tone and volume of voice, anxiety, mumbling, and pacing. ¹² These items were selected through thematic analyses of direct observation, semistructured interviews, and field interviews among 20 emergency nurses. However, this framework has not been tested for usefulness among patients in the ED, and its predictive properties are unknown. Nevertheless, STAMP has been suggested by the Centers for Disease Control and Prevention as a violence risk assessment tool. ¹⁵

The Bottom Line

This multisite cohort study tested the utility of the Aggressive Behavior Risk Assessment Tool among 10,554 patients visiting emergency departments. It showed a sensitivity of 84.3% and specificity of 95.3% for identifying violent patients during an emergency department stay.

Another tool, the BVC, was originally developed for use in psychiatric units. ¹³ This 6‐item checklist consists of confusion, irritability, boisterousness, verbally or physically threatening, and attacking objects. Among 2064 patients in the ED, the sensitivity and specificity were 45.7% and 99.4%, respectively, at a cutoff score of 3. ¹⁶ The low sensitivity of the BVC indicates that more than half of the violent patients were missed. These limited options indicate a need for a validated ED‐specific violence risk assessment tool with better sensitivity and specificity in identifying violent patients. ¹⁷ , ¹⁸

ABRAT showed satisfactory sensitivity and specificity for identifying potentially violent patients in a study of 2063 medical–surgical patients. ¹⁴ It was further tested among 724 residents admitted to 25 long‐term care homes and revised as ABRAT for Long–Term Care with satisfactory sensitivity and specificity. ¹⁹ However, ABRAT has not been previously tested among patients in the ED.

1.3. Goals of this investigation

The purpose of this study was to determine the utility of the ABRAT for EDs in identifying patients at risk for violence in EDs. The specific aims were to (1) estimate the prevalence of violent incidents among patients in the ED, (2) select a parsimonious set of ABRAT items, and (3) determine the optimal cutoff scores in identifying patients at risk for violence and estimate the sensitivity and specificity of the ABRAT for EDs.

2. METHODS

2.1. Study design

A prospective cohort study was conducted at 3 EDs associated with an integrated health system in a large urban city in Michigan between May 1, 2021, and June 30, 2021. A total of 10 items of the original ABRAT and 6 common violent behavior indicators in EDs were combined, resulting in a 16‐item expanded ABRAT. The 6 ED‐specific items were history of drug/alcohol abuse, history of mental illness, hallucination, police custody, pacing, and signs of intoxication/drug use. ⁹ , ¹⁰ In the current study, violent incidents were classified as physical assault, physical threat, harassment, sexual assault, verbal abuse, verbal threat, and aggression not otherwise specified. Physical assault involves physical harm or unwanted physical contact on another individual, such as biting, hitting, kicking, and scratching. The physical or verbal threat is expressed intent to cause bodily harm. Verbal abuse is an expression of anger or hostility intended to demean, insult, or berate another individual. The 16 expanded ABRAT items and violent incidents were incorporated into the electronic health record (EHR).

2.2. Selection of participants and setting

All patients aged ≥10 years visiting 3 EDs during the study period were included.

The 3 EDs were the following: a level I adult trauma center with 84 care spaces servicing a 535‐bed medical center and a 250‐bed heart center, a level III adult trauma center with 33 care spaces servicing a 255‐bed hospital, and a level I pediatric trauma center with 36 care spaces servicing a 234‐bed children's hospital and 108‐bed neonatal intensive care unit. For patients with >1 ED visit during the study, only the first visit was included.

2.3. Measurements

This study used the 16‐item expanded‐ABRAT that combined the original 10‐item ABRAT with 6 additional items relevant to violent behaviors among patients in the ED. The original ABRAT was previously developed and tested as a pen‐and‐paper checklist among 2063 patients within 24 hours of admission to medical–surgical units. ¹⁴ It consists of 10 indicators of violent behaviors among hospitalized patients, including a history of aggression as well as aggressive/threatening behavior, agitation, anxiety, confusion, shouting, mumbling, staring, and signs/symptoms of mania. Each item was assessed on a 2‐point scale (presence = 1; absence = 0) with the summation scores ranging from 0 to 10 and higher scores indicating higher risks of violence. The receiver operating characteristics (ROC) analysis showed the area under the curve (AUC) of 0.82 (95% confidence interval [CI], 0.75–0.90) and sensitivity and specificity at a cutoff score of 1 with 71.4% and 89.3%, respectively. The interrater reliability was reported as a Cohen's κ of 0.66 and an interrater agreement percentage of 92.9%. ¹⁴

2.4. Data collection procedures

Before initiation of the study, the emergency staff educational rollout included posting and emailing an educational flier and frequently asked questions. These were also posted prominently in each ED. Leaders of each unit discussed the study at their ED huddles during the first 2 weeks of the study to ensure awareness and address any questions. Spot checks and feedback were performed weekly to ensure adherence to the required documentation in the EHR.

Triage nurses completed the expanded ABRAT as a part of the routine triage assessment for patients entering the ED directly. For patients arriving by ambulance, the expanded ABRAT was completed during the initial nursing assessment in the EDs. For the collection of violent incidents, a violence checklist, which included violence types and an optional free‐text box, was completed for each patient by the assigned nurse before disposition from the ED. Any free‐text entry in the checklist was coded into the violence types by the honest broker and then validated by 1 of the authors. At 2 months after the study initiation, data were retrieved from the EHR, including the ED visit reasons (EDVRs), expanded ABRAT items, violent incidents, arrival‐in‐room time, age, sex, and acuity levels.

2.5. Statistical analysis

Descriptive statistics of means, frequencies, and percentages were calculated to summarize sample characteristics and the prevalence of violent incidents. Each expanded ABRAT item and the violent incidents were assessed as dichotomous variables. Bivariate correlation procedures using the Kendall τ test were performed to explore correlations between dichotomous violent incidents, expanded ABRAT items, EDVRs, and demographic data. All variables that correlated with violent incidents (r ≥ 0.10) were entered into multivariate logistic regression model with a backward elimination method to select a parsimonious set of items. The odds ratio (OR) from the logistic regression model was used to derive and calculate the weighted summation scores of the ABRAT for EDs.

The ROC analysis was then performed to determine the optimal cutoff score of the ABRAT for EDs in predicting violent incidents and estimate the corresponding sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Data were analyzed using the SPSS version 28.0 (IBM Corporation, Armonk, NY), and the significance level was set at P value < 0.01.

2.6. Ethical considerations

This study was reviewed by the organization's institutional review board (IRB). The IRB determined that the study did not meet the definition of human subjects research according to the US federal regulations. The ABRAT items were considered to be a part of the standard nursing assessments. An honest broker, employed by the study site research department, was used to extract de‐identified data from the EHR and create a de‐identified data set for analysis according to Health Insurance Portability and Accountability Act privacy rule. ²⁰ To further anonymize the data, patients aged >89 years were aggregated as ≥90 years.

3. RESULTS

3.1. Characteristics of study patients

Of 28,387 patients visiting 3 EDs during the study period, 4817 were aged <10 years. Of the remaining 23,570 eligible patients, 13,016 had missing expanded ABRAT assessments, resulting in a sample size of 10,554 patients (44.8% completion rate). Table 1 shows the sample characteristics of the overall patient population and those with violent incidents. A total of 127 patients of 10,554 had ≥1 violent incidents (1.2%). For the various EDVRs, only 5.9% of all patients had probable mental health reasons for the visit (627 of 10,554), whilst 78.7% of the violent patients has probable mental health reasons for the visit (100 of 127). The violent group was younger (33 years vs 46 years), included more men (55.1% vs 43.2%), and had higher emergent acuities (2.17 vs 2.76) and shorter arrival‐in‐room time (3.8 minutes vs 17.2 minutes) compared with the overall patients. However, no racial differences were found between the violent group and overall patients.

TABLE 1.

Demographic characteristics

	Overall patients, N = 10,554	Patients with violent incidents, n = 127
Age, years	46 (10–90+)	33 (10–88)
Male sex	4555 (43.2)	70 (55.1)
Race/ethnicity
Non‐Hispanic White	6991 (66.2)	89 (66.9)
African American	1969 (18.7)	23 (18.1)
Hispanic	1089 (10.3)	13 (10.2)
Asian/Pacific Islander	92 (0.9)	0
Multiracial	155 (1.5)	3 (2.4)
American Indian	24 (0.2)	0
Other/unknown	234 (2.2)	3 (2.4)
Acuity level a	2.76 (1–5)	2.17 (1–4)
ED arrival‐to‐room time, minutes	17.2 (0–197)	3.8 (0–67)
EDVR, ≥2% for either column
Non–mental health conditions
Abdominal pain	1168 (11.5)	1 (0.8)
Chest pain	743 (7.0)	1 (0.8)
Difficulty breathing/SOB/cough	764 (7.2)	4 (3.1)
Fall	429 (4.1)	0
Back pain/injury	343 (3.2)	1 (0.8)
Leg/hip pain	304 (2.9)	4 (3.1)
Headache	244 (2.3)	0
Flank pain	237 (2.2)	0
Dizziness	227 (2.2)	1 (0.8)
Motor vehicle/cycle/pedestrian crash	222 (2.1)	2 (1.6)
Wound/laceration/rib pain/assault victim	138 (1.3)	5 (3.9)
Probable mental health conditions
Suicidal ideation/suicide attempt	166 (1.6)	17 (13.4)
Altered mental status/confusion/delusion	126 (1.2)	11 (8.7)
Psychiatric evaluation	89 (0.8)	25 (19.7)
Alcohol intoxication/problem/DT	83 (0.8)	4 (3.1)
Ingestion	81 (0.8)	5 (3.9)
Behavioral health concern	29 (0.3)	11 (8.7)
Aggressive behavior	19 (0.2)	16 (12.6)
Hallucinations	18 (0.2)	4 (3.1)
Agitation	11 (0.1)	3 (2.4)
Homicidal	5 (0.0)	4 (3.1)

Note: Values are expressed as mean (range) or n (percentage). Percentages do not add to 100% because of 2% cutoff and rounding.

Abbreviations: DT, delirium tremens; ED, emergency department; EDVR, emergency department visit reason; SOB, shortness of breath.

^aAcuity level of the ED visit: 1, immediate; 2, emergent; 3, urgent; 4, less urgent; and 5, non‐urgent.

The EDVRs for overall patients and violent patients are also shown in Table 1. The most common mental health EDVRs among 127 violent patients were psychiatric evaluation (19.7%), suicidal ideation/suicide attempt (13.4%), and aggressive behavior (12.6%). In each EDVR, aggressive behavior (84.2%; 16 of 19), behavioral health concern (37.9%; 11 of 29), psychiatric evaluation (28.1%; 25 of 89), and suicidal ideation/suicide attempt (10.2%; 17 of 166) had the highest rates of violence. In contrast, very few violent incidents occurred among patients with pain as the EDVR, such as abdominal pain, chest pain, back pain, headache, or flank pain. Among 127 patients who had ≥1 type of violent incident, the most common events were a verbal threat (29.1%), aggression not otherwise specified (26.8%), verbal abuse (25.2%), physical assault (25.2%), and physical threat (18.1%).

3.2. Item selection: ABRAT for EDs

Table 2 presents the results of bivariate Kendall τ tests showing correlations between expanded ABRAT items, EDVRs and violent incidents. Of 16 expanded ABRAT items and 4 EDVRs, 15 met the correlation coefficient criterion (r ≥ 0.10). From the logistic regression model, 6 items from the expanded ABRAT and 4 EDVRs emerged as predictors of violent incidents (Table 3). Because each patient had only a single EDVR or chief complaint, the EDVR was considered as a single item. Thus, the parsimonious set of 7 items that best predicts violent incidents was named ABRAT for EDs, comprising histories of aggression and mental illness, EDVR, and 4 behavior indicators of violence, that is, aggressive/threatening behavior, agitation, staring, and confusion.

TABLE 2.

Bivariate correlations with violent incidents (N = 10,554)

Predictors	r
History of aggression	0.45*
History of mental illness	0.24*
History of drug/alcohol abuse	0.16*
Aggressive/threatening	0.44*
Threatening to leave	0.28*
Agitated	0.41*
Staring	0.24*
Police custody	0.15*
Mumbling	0.19*
Confused	0.15*
Anxious	0.23*
Shouting	0.32*
Signs/symptoms of mania	0.22*
Pacing	0.18*
Hallucinating	0.13*
EDVR: suicidal ideation/suicide attempt	0.13*
EDVR: behavioral health concern	0.18*
EDVR: psychiatric evaluation	0.23*
EDVR: aggressive behavior	0.32*

Abbreviation: EDVR, emergency department visit reason.

^* P < 0.001 by bivariate correlation with Kendall τ test. Only items with r ≥ 0.10 are shown.

TABLE 3.

Multivariate logistic regression model for predicting violent incidents (N = 10,554)

Predictors	OR	95% CI	P value	Weight value
History of aggression	16	8–31	<0.001	4
History of mental illness	3	2–7	<0.001	1
Aggressive/threatening	24	7–80	<0.001	5
Agitation	18	7–45	<0.001	4
Staring	10	3–42	<0.001	2
Confusion	5	2–14	0.004	1
EDVR: aggressive behavior	71	15–340	<0.001	6
EDVR: Behavioral health concern	18	6–56	<0.001	4
EDVR: Psychiatric evaluation	12	5–26	<0.001	3
EDVR: Suicidal ideation/suicide attempt	5	2–12	<0.001	1

Abbreviations: CI, confidence interval; EDVR, emergency department visit reason; OR, odds ratio.

Based on the OR from the logistic regression model, a weight value was assigned to each item ranging from 1 (OR ≤5) to 6 (OR >30) as shown in Table 3. The weighted ABRAT summation scores were calculated according the following formula:

$$\text{ABRAT}\text{score}=\sum _{i=1}^7{\mathrm{W}}_i{\mathrm{X}}_i$$

where W is the assigned weight value for each item (ranging from 1 to 6), and X is the response value (0 or 1) for the item. As an example, for a confused patient with a history of aggression and visiting the ED for psychiatric evaluation, the calculated ABRAT score would be 8 (1 + 4 + 3).

3.3. Sensitivity and specificity of the ABRAT for EDs

The ROC analysis was used to assess the potential utility of the ABRAT for EDs in discriminating between violent and non‐violent patients. The AUC of 1.0 would indicate a perfect discriminant ability, whereas an AUC of 0.5 indicates no discriminant ability. ¹¹ As shown in Figure 1, the AUC for the ABRAT for EDs was 0.91 (95% CI, 0.87–0.95; P < 0.001), indicating a high discriminant ability. The ROC analysis was also used to determine the optimal cutoff score with acceptable sensitivity and specificity. Table 4 shows the sensitivity, specificity, PPV, and NPV at various cutoff scores. The cutoff score of 1 has a sensitivity and specificity of 84.3% (95% CI, 76.5%–89.9%) and 95.3% (95% CI, 94.8%–95.7%), respectively, whereas the PPV and NPV are 17.8% (95% CI, 14.9%–21.2%) and 99.8% (95% CI, 99.7%–99.9%), respectively. This indicates that the ABRAT for EDs would correctly identify 84.3% of violent patients (true positive) while correctly identifying 95.3% of non‐violent patients (true negative). At an alternate cutoff score of 4, the sensitivity and specificity were 70.1% (95% CI, 61.2%–77.7%) and 98.9% (95% CI, 98.7%–99.1%), respectively, whereas the PPV and NPV were 44.3% (95% CI, 37.3%–51.4%) and 99.6% (95% CI, 99.5%–99.7%), respectively. The PPV of 44.3% indicates that almost half of the patients with scores of ≥4 will become violent during their ED visits. In contrast, the NPV of 99.6% indicates that almost all of the patients with the ABRAT scores <4 will be non‐violent.

FIGURE 1.

Receiver operating characteristics curve: Aggressive Behavior Risk Assessment Tool (ABRAT) for emergency departments (EDs). Diagonal line represents the line of no discrimination with an area under the curve of 0.5. Solid curve represents the ABRAT for EDs with an area under the curve of 0.91 (95% confidence interval, 0.87–0.95; P < 0.001)

TABLE 4.

Characteristics of the summation score cutoffs: Aggressive Behavior Risk Assessment Tool for emergency departments

Score cutoff	Sensitivity (95% CI), %	Specificity (95% CI), %	PPV (95% CI), %	NPV (95% CI), %
1	84.3 (76.5–89.9)	95.3 (94.8–95.7)	17.8 (14.9–21.2)	99.8 (99.7–99.9)
2	74.8 (66.2–81.9)	98.1 (97.9–98.4)	33.0 (27.6–38.8)	99.7 (99.6–99.8)
3	72.4 (63.7–79.8)	98.5 (98.3–98.8)	37.6 (31.5–44.0)	99.7 (99.5–99.8)
4	70.1 (61.2–77.7)	98.9 (98.7–99.1)	44.3 (37.3–51.4)	99.6 (99.5–99.7)
5	61.4 (52.3–69.8)	99.5 (99.4–99.6)	61.4 (52.3–69.8)	99.5 (99.4–99.6)

Abbreviations: CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

Figure 2 shows the percentage of violent patients at various ABRAT scores. Among 9954 patients with ABRAT scores of 0, only 20 were violent (0.2%). Among 399 patients with ABRAT scores between 1 and 3, 18 were violent (4.5%). In contrast, among 201 patients with ABRAT scores ≥4, 89 became violent (44.3%).

FIGURE 2.

Percentage of violent patients versus Aggressive Behavior Risk Assessment Tool (ABRAT) scores. The ratios shown are violent patients/total patients at various ABRAT scores

4. LIMITATIONS

This study has certain limitations. First, despite the large sample size of >10,000 patients, the completion rate of ABRAT was only 44.8%, which may limit internal validity of the study. In addition, the study was conducted in a single health system located in 1 geographic region, which may limit the generalizability of findings to other settings. Second, there were only 127 patients with violent incidents, which limited the precision of the reported sensitivity and specificity. Although the nurses were specifically trained to capture all violent incidents in the EHR, it is possible that some violent incidents were not captured. Third, the severity of the violent incidents and violence‐related injuries were not collected in this study. Finally, the data were collected only for 2 months during the COVID‐19 pandemic, which may limit applicability beyond the pandemic period. Future studies are needed to confirm the findings as well as to determine whether early identification of potentially violent patients and preemptive interventions focused on a small subset of high‐risk patients in the ED would prevent or mitigate violent incidents.

5. DISCUSSION

The ABRAT for EDs is a simple yet comprehensive risk assessment checklist with high sensitivity and specificity for identifying patients at risk for violence in EDs. To our knowledge, this is the largest multisite study of >10,000 patients in the ED that demonstrates the utility of a violence risk assessment tool. This 7‐item checklist includes histories of aggression and mental illness, EDVRs, and 4 behavior indicators of violence. The availability of such a validated violence screening tool could make the implementation of targeted interventions possible for a safer ED environment.

Notably, 4 reasons in the EDVR item were identified as predictors of violent incidents in the multivariate logistic regression analysis and included in the ABRAT for EDs. All 4 reasons in the EDVR item were related to mental health conditions, such as aggressive behavior, behavioral health concern, psychiatric evaluation, or suicidal ideation/suicide attempt. This is likely due to the current pattern of mental health emergencies being handled through the EDs.^21,22 In this study, only 3% (303 of 10,554) of all patients visiting the EDs had 1 of the 4 reasons in the EDVR item, whereas more than half (69 of 127) of the violent patients had the same 4 reasons. Therefore, including them in the checklist appears to be reasonable. Although the violent patients were younger than overall patients (33 years vs 46 years), age was not a predictor of violent incidents.

The ABRAT for EDs takes about 2 minutes to complete and was not perceived to be burdensome by nurses who participated in the study. Leveraging information available from the EHR could further reduce the documentation burden on emergency nurses for completing the checklist. The EDVR or chief complaint is usually 1 of the first pieces of information collected during the ED visit, sometimes even before the patient's arrival in ED. If a patient presents to ED with 1 of the 4 reasons in EDVR item, the information could be retrieved from the EHR and automatically incorporated into the calculation of the ABRAT score. Furthermore, for those patients with preexisting data in the EHR, the past histories of aggression or mental illness could also be automatically retrieved from the EHR. If so, the busy triage nurse may need to assess only 4 violent behavior indicators to complete the checklist: aggressive/threatening, agitation, staring, and confusion. Thus, with the computerized retrieval of history and EDVR from the EHR, the checklist could be completed quickly for most patients.

For calculating the ABRAT score, each item was assigned a weight value ranging from 1 to 6 to account for the strength of each item's association with patient violence represented by its OR from the logistic regression model. Based on the ABRAT scores, the ROC analysis showed an AUC of 0.91, indicating that the ABRAT for EDs has an excellent likelihood of discriminating between violent and non‐violent patients. One common method of choosing an optimal cutoff score is to select the point on the ROC curve farthest away from the diagonal line of non‐discrimination, that is, the cutoff score of 1 in the current study. Although this cutoff score of 1 is associated with high sensitivity (84.3%), which allows most of the violent patients to be identified, the PPV is only 17.8%, indicating that only a modest fraction of the patients with ABRAT scores ≥1 will be violent. On the other hand, a higher cutoff score of 4 is associated with a lower sensitivity of 70.1%, but the PPV is much higher (44.3%), indicating that nearly half of the patients with ABRAT scores ≥4 will be violent.

To strike a balance between sensitivity and PPV, we propose the following 3 categories of violence risk according to the ABRAT scores: high risk (scores ≥4), medium risk (scores 1–3), and low risk (score of 0). In this proposal, the high‐risk category cutoff score of 4 with the sensitivity of 70.1% would allow emergency staff to focus their attention on a small number of patients, nearly half of whom are expected to become violent during the ED visit (44.3%; 89 of 201 patients). The medium‐risk patients in the ED (scores of 1, 2, or 3) have a 10‐fold lower risk of violence (4.5%; 18 of 399 patients), and reassessments of these patients may be helpful to further differentiate violent versus non‐violent patients. The low‐risk patients are expected to have an extremely low risk of violence (0.2%; 20 of 9954 patients). This categorization scheme would allow the identification of patients at moderate and high risk of violence as well as focusing violence mitigation measures on high‐risk patients.

Surprisingly, the prevalence of violent patients in this study was only 1.2% (127 of 10,554 patients). A similar low rate was also observed in Australia, where the prevalence of violent patients in the ED was 1.7%. ¹⁶ In a previous ABRAT study of medical–surgical patients, the prevalence was higher at 2.7% (56 of 2063 patients). ¹⁴ Furthermore, the prevalence was even higher among long‐term care residents at 7.3% (53 of 724 residents). ¹⁹ These differences may be attributed to the different lengths of time the violence data were collected from each patient. The violence data for the long‐term care residents were collected >30 days after admission, and the data for medical–surgical patients were collected during their entire hospitalization, usually lasting a few days. In contrast, the violence data for patients in the ED were collected over much shorter periods as suggested by the median ED visit lengths of 4.6 and 2.4 hours for the admitted and discharged patients, respectively, in the United States.²³ Therefore, the low (1.2%) prevalence of violence in the ED (1.2 violent incidents per 100 patients) may be attributed to the short stays in the ED with rapid patient turnover. In contrast, the high percentage of patient violence experienced by emergency staff (violent incidents per 100 emergency healthcare workers) is likely related to each healthcare worker caring for a large number of patients as well as seeing patients with psychiatric disorders.^1,24,25

The Joint Commission suggests using a checklist to identify patients at risk for violence, ⁶ and OSHA recommends universal precautions against violence. ⁵ The application of indiscriminate precautions would exhaust resources and limit the effectiveness of preventive measures. We suggest an alternative strategy of using ABRAT to identify those few patients at high risk of violence so that focused preventive measures can be implemented. Some proposed preventive measures include an interprofessional huddle when a patient is identified as high risk, a behavioral management plan, and the use of specially trained behavioral health technicians. However, there is a dearth of evidence that supports the effectiveness of such preventive measures. Interventional studies are planned to test the efficacy of focused strategies based on the identification of potentially violent patients using ABRAT.

In conclusion, this study of >10,000 patients showed the utility of ABRAT for EDs with a high sensitivity and specificity for identifying potentially violent patients. It is a simple yet comprehensive violence risk‐assessment checklist that incorporates patient histories, EDVRs, and behavioral indicators of violence. The availability of such a screening checklist in the EHR may allow rapid identification of high‐risk patients and implementation of focused mitigation measures to protect emergency staff and patients.

CONFLICT OF INTEREST

Son Chae Kim is a copyright holder of the Aggressive Behavior Risk Assessment Tool. The other authors declare no conflict of interest.

AUTHOR CONTRIBUTIONS

Son Chae Kim and Jennifer Kaiser conceived and designed the study. Julie Bulson, Tracy Hosford, Ashleigh Nurski, Carol Sadat, and Nicole Kalinowski contributed to the study design. Jennifer Kaiser, Julie Bulson, Tracy Hosford, Ashleigh Nurski, Carol Sadat, and Nicole Kalinowski contributed to data collection. Son Chae Kim performed statistical analysis and interpretation and drafted the manuscript. All authors contributed to reviewing and editing the final draft.

Biography

Son Chae Kim, PhD, RN, is a Professor of Nursing at Point Loma Nazarene University in San Diego, CA.

Kim SC, Kaiser J, Bulson J, et al. Multisite study of Aggressive Behavior Risk Assessment Tool in emergency departments. JACEP Open. 2022;3:e12693. 10.1002/emp2.12693