Improving the evaluation of non-accidental trauma across multiple specialties at a single institution

Abstract

Background

Based on the presenting injury, patients undergoing abuse evaluation may be managed by different specialties. Our local child abuse specialist expressed concern over the variability in evaluation of patients presenting with injuries concerning for non-accidental trauma (NAT). The aim of this quality improvement project was to increase the percentage of patients for whom there is a concern for NAT who receive a guideline-adherent evaluation from 7.7% to 50% in 6 months’ time.

Methods

A committee of physician stakeholders developed criteria for a complete NAT evaluation which were integrated into an order panel with built-in clinical guidance for test selection within our electronic medical record. Data on the completeness of NAT evaluation in paediatric patients 0–18 years of age were collected before and after the order panel release and analysed by admitting service, injury category and equity factors.

Results

This initiative increased the percentage of patients with a guideline-adherent evaluation from a mean of 7.7% to 25% within 6 months’ time. The number of days between patients with complete evaluations decreased from 63 days to 35 days. Order panel utilisation increased to 55%, and the percentage of evaluation opportunities was more complete when the order panel was used (79% vs 92%).

Conclusions

Standardisation of NAT evaluations through creation of an order panel with a clinical decision tool resulted in more guideline-adherent evaluations. The potential reduction of bias in such evaluations remains an area of interest.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• There is variation in non-accidental trauma (NAT) evaluation across providers. There have been significant efforts to improve the identification of NAT, yet most studies to improve evaluation of NAT are restricted to a subset of patients or individual tests.

WHAT THIS STUDY ADDS

• This study adds to the scarce literature available on standardising the NAT evaluation to uncover both occult injury and screen for underlying medical conditions across all hospital locations and specialties.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• This study supports the use of EMR tools, specifically in the form of an order panel, to help guide clinicians in the evaluation of NAT. Future studies can build on this work to increase the percentage of complete evaluations when there is suspicion for NAT.

Introduction

Variability in non-accidental trauma (NAT) evaluation can impact timely and complete workups. This variability is linked to a variety of factors including practice patterns of individual providers. One study demonstrated significant site variability in the evaluation of abusive head trauma, thought to be related to implicit bias of local physicians.¹ Another study found inconsistency in the frequency of haematological testing in children presenting with bruising or bleeding in the context of suspected NAT, ranging from 12% to 62% for individual tests.² This variability, whether rooted in bias, lack of recognition or other factors; places children at risk for incomplete evaluations and therefore missed evidence of NAT. A variety of studies have demonstrated that children who are victims of abuse are at risk of recurrent abuse and further injury,³,⁵ and appropriate recognition and evaluation of abuse may help prevent further injury.⁶

Much of the focus on improving the management of NAT has been directed towards screening tools to assist with identification of abuse patients⁷,¹¹ and implementation of management protocols.¹²,¹⁴ Despite 75% of providers using management protocols in a recent nationwide survey of level 1 and 2 trauma centres,¹⁵ studies on the impact of these protocols are mostly limited to subsets of patients,^{13 14 16} tests¹⁷,²⁰ or practice locations.¹² Rather than focusing on a specific subset of patients or evaluation components, we attempted to standardise the NAT evaluation to uncover both occult injury and screen for underlying medical conditions across all hospital locations and specialties. We hypothesised that NAT evaluation completeness varies by admitting service, and injury type, and can be improved by an institutional guideline to standardise the evaluation. The aim of this quality improvement intervention is to increase the percentage of patients for which there is a concern for NAT who receive a guideline-adherent evaluation from 7.7% to 50% in 6 months’ time.

Methods

Context

Our local child abuse specialist expressed concern over the variability in evaluation of patients presenting with injuries suspicious for NAT, as many patients were not receiving complete evaluations despite the availability of consultation. Project team members included a child abuse specialist, a quality improvement expert and a paediatric hospital medicine fellow. This single-centre project took place in a Mid-Atlantic suburban children’s hospital-within-a-hospital. There are 226 paediatric beds and an average of 7000 general paediatric ward admissions and 45 000 paediatric emergency department visits per year. The hospital is a level 1 trauma centre.

The hospital’s Forensic Assessment and Consultation Team (FACT) consists of two forensic physicians, along with forensic nurse examiners. They are available for inpatient and outpatient consultations 24 hours a day, as well as medical record review for cases involving physical abuse, sexual abuse and neglect. Once a consult order is placed the provider typically receives preliminary recommendations over the phone. The patient is then seen by the FACT consultant in the hospital within 24–48 hours and finalised recommendations are outlined for the primary team to consider. Based on the presenting injury, patients undergoing NAT evaluation at our institution are managed by different specialties including paediatric emergency medicine (PEM), paediatric intensive care (PICU), paediatric hospital medicine (PHM), trauma surgery and orthopaedic surgery.

Patient involvement

Patients and the public were not involved in the design, conduct, reporting or dissemination of our study.

Population

This project included paediatric patients receiving an evaluation for NAT that presented to the emergency department or were admitted to our children’s hospital. Patients were identified using a FACT consult with associated consultant documentation. In order to isolate patients at risk for NAT, the FACT physician assigned a likelihood of abuse rating from 1 to 7 using a modified Likert scale from Lindberg et al.²¹ Patients with a likelihood scale equating to ‘definitely not’ (1) or ‘no’ (2) concern for inflicted injury were excluded. In addition, patients for which there was only a concern for sexual abuse or neglect were excluded.

Interventions

A committee was formed to address the variability encountered in the initial evaluation of patients for whom there was a concern for NAT. This committee included the FACT specialist along with physician stakeholders from PEM, PICU, PHM, orthopaedics and trauma surgery. We reviewed the existing literature and developed a unified understanding of the goal and key drivers to success (online supplemental figure 1).

Once agreement was obtained on what constitutes a complete evaluation using existing guidelines and expert opinion where necessary, we created a clinical decision tool consisting of listed indications for each testing component. We then developed an order panel within our EMR (EMR; Epic Systems, Verona, Wisconsin). The accordion style order panel is grouped by measurements, laboratory testing, imaging studies and relevant consults. It lists the indication for each evaluation component to provide clinical guidance. Committee team members held educational sessions within their respective departments and informed colleagues of the order panel via email, at departmental meetings and at trainee didactic sessions. Trainee didactic sessions included information on searching for and use of the order panel, in addition to general education on NAT evaluation. Additionally, the information was added to existing trainee handbooks. Patient charts with FACT consultation were manually reviewed to determine all diagnostics that were obtained, the timing of these tests and whether the evaluation contained all indicated tests for NAT evaluation as per the operational definitions (table 1).

Table 1. Criteria used for defining a complete evaluation for NAT.

Item(s)	Age	Indication
Measurements
Head circumference	0–24 months	All patients*
Weight	0–18 years	All patients*
Length/height	0–18 years	All patients*
Laboratory
CBC with differential, comprehensive metabolic panel, lipase	0–24 months	All patients*
PT/aPTT, fibrinogen, Von Willebrand cofactor activity and antigen, factor levels (VII, VIII, IX, XIII)	0–18 years	Patient with unexplained bleeding or bruising:not clearly explained as determined on manual review of FACT consultant documentation
Vitamin D, PTH, phosphorus	0–18 years	Patient presenting with a fracture(s)Exception: isolated skull fracture, or consistent with accidental injury as per FACT consultant documentation
Toxicology testing	0–18 years	Patient with concern indicated by history or examination
Imaging
Head CT	0–6 months	All patients*
Head CT with 3D reconstruction	0–6 months	All patients 0–6 months*and for any patient requiring head CT due to concern for NAT
MRI brain	0–18 years	If recommended by FACT specialist for evaluation of intracranial injury
MRI spine	0–6 months	All patients receiving MRI brain
Skeletal survey	0–2 years	All patients*
CT abdomen and pelvis	0–18 years	For patients with AST/ALT >80 or presenting with evidence of abdominal trauma

^*Routine evaluation if there is primary medical provider concern for NAT.

FACT, Forensic Assessment and Consultation TeamNAT, non-accidental trauma

We shared results with project champions along the way to motivate and encourage. The Model for Improvement framework was used, testing interventions for improvement through Plan-Do-Study-Act cycles.

Evolution of interventions over time

There was consideration for development of a separate clinical practice guideline. This was abandoned prior to implementation in favour of a simplified approach to include the clinical decision prompts directly into the order panel. This avoided the extra step of referencing a clinical practice guideline while using the order panel. Small adjustments were also made to the order panel following provider feedback including clarification of test indications.

Measures

Our main outcome measure was the percentage of patients for whom there is a suspicion of NAT who receive a complete evaluation. Using age and injury type, we determined if each medical evaluation was complete based on guidance from key studies on the evaluation of NAT including the 2015 American Academy of Pediatrics clinical report on Evaluation of Suspected Physical Child Abuse.²²,²⁸ Where recommendations required interpretation, we used child abuse specialist expert opinion to determine the necessity of the test. We analysed the percentage of evaluation opportunities completed per patient and across patients (number of evaluation components completed divided by the total number of evaluation components indicated), the number of days between patients with a complete NAT evaluation, utilisation of the order panel (process measure) and median time to completion of the evaluation (balancing measure). Time to completion of NAT evaluation was chosen rather than length of stay because patients may reside in the hospital after completion of the NAT evaluation while awaiting safe disposition planning by child protective services or for management of other medical conditions.

Analysis

Data on the completeness of evaluation for NAT were collected prior to the development of the EMR order panel from 1 January to 14 October 2020 (baseline) and after order panel release from 15 October 2020 to January 2023 (postintervention). We created control charts in Excel’s QI Macros using Montgomery rules to identify special cause variation.²⁹ Centrelines were calculated separately for preintervention and postintervention populations specifically when comparing the proportion of complete evaluations. Although race and ethnicity are social constructs without scientific or biological meaning, we chose to analyse these variables, in addition to language and payer, to explore potential inequities. We used χ², Fisher’s exact and Mann-Whitney U testing to analyse the difference in evaluations among specialty services, injury categories, equity factors, preintervention and postintervention.

Results

90 cases (27 baseline, 63 postintervention) of suspected NAT were evaluated via FACT consultation between January 2020 and January 2023. Online supplemental table 1 compares the preintervention and postintervention patient populations. Fewer patients presented to the paediatric emergency department (73% vs 96%, p=0.01) in the postintervention group. While there were no significant differences in the primary admitting service, it is worth noting that zero patients were admitted to the orthopaedics service in the preintervention population.

Following order panel implementation, the percentage of patients with fully complete evaluations increased from a mean of 7.7% to 25% (figure 1A). Figure 1B shows that the number of days between patients with complete evaluations decreased from 63 days to 35 days. Total evaluation opportunities completed over time increased from 80% to 87% (figure 1C). Order panel utilisation increased to 55% (figure 1D).

Figure 1. P chart showing the proportion of children with fully complete evaluations (A). T chart showing the number of days between patients with complete NAT evaluations (B). P chart showing the proportion of total evaluation opportunities performed in children for whom NAT was suspected over time (C). P chart showing the proportion of children in which the order panel was used (D). Interventions are annotated. ED, emergency department; LCL, lower control limit; NAT, non-accidental trauma; PHM, paediatric hospital medicine; UCL, upper control limit.

While there was no difference between the number of patients with fully complete evaluations when comparing patients using the order panel to those without using the order panel, there was a significantly higher percentage of total evaluation opportunities completed across patients for whom the order panel was used (79% vs 92%, p≤0.00001 via χ² testing). There was also a significantly higher median percentage of evaluation opportunities completed per patient when the order panel was used (83% vs 93%, p=0.004 via Mann-Whitney testing).

Results are summarised in table 2. The percentage of recommended evaluations completed per patient did not differ by admitting service in the baseline population. When categorising testing by injury type, evaluation for abdominal trauma when indicated was least likely to be complete compared with bruising/bleeding or fracture evaluation (72% vs 90%, 92%, respectively, both p values <0.001 via χ² testing). Evaluation for abdominal trauma increased to 85% following order panel implementation (p=0.009 via χ² testing), but this evaluation remained the least likely to be complete when compared with other injury types.

Table 2. Summary of results before and following order panel implementation.

	Baseline(n=27)	Postintervention(n=63)	P value
Median time to complete evaluation (hours)	30.2	61.6	0.0005**
Median % testing completed per patient	82%	92%	0.005**
Patients with complete evaluations by injury category***
Bruising	1 (14%)	6 (33%)	0.63
Extremity fractures	0 (0%)	5 (25%)	0.14
Head trauma (skull fracture and/or intracranial bleed)	1 (8%)	10 (32%)	0.14
Tests completed by test category
Bleeding/bruising	85 (90%)	245 (95%)	0.16*
Fractures	101 (92%)	263 (94%)	0.38*
Head trauma	87 (87%)	274 (92%)	0.14*
Abdominal trauma	58 (72%)	166 (85%)	0.009*
Select test completion when indicated
Abdomen/pelvis CT	3 (75%)	8 (80%)	1
Head circumference	17 (71%)	46 (85%)	0.138*
Lipase	6 (23%)	37 (61%)	0.001*
Head CT—3D reconstruction	17 (81%)	35 (65%)	0.26*
Complete evaluations by equity factors
Race
Black	0 (0%)	1 (20%)	1
Asian	0 (0%)	0 (0%)	1
White	1 (14%)	8 (33%)	0.64
Ethnicity
Hispanic	0 (0%)	2 (15%)	0.54
Non-Hispanic	2 (9%)	12 (30%)	0.11
Language
Non-English	0 (0%)	2 (25%)	0.48
English	2 (10%)	14 (25%)	0.21
Insurance
Public	1 (6%)	5 (13%)	0.66
Private	1 (9%)	11 (48%)	0.05
Total evaluation opportunities completed by equity factors
Race
Black	32 (89%)	49 (88%)	1
Asian	36 (80%)	22 (79%)	0.88*
White	60 (77%)	247 (88%)	0.01*
Ethnicity
Hispanic	46 (71%)	144 (86%)	0.008*
Non-Hispanic	186 (82%)	415 (87%)	0.06*
Language
Non-English	45 (70%)	98 (92%)	0.0003*
English	187 (82%)	583 (86%)	0.74*
Insurance
Public	131 (78%)	443 (86%)	0.010*
Private	131 (78%)	238 (89%)	0.002*

Note: * = Chi Square Testing. ** = Mann-Whitney U Testing. All other p-values are the result of Fisher exact testing. *** Patients could be included in multiple categories based on presenting injury.

P-values considered significant (<0.05) are bolded.

Online supplemental table 2 shows results by individual test completion. The most frequently missed tests were lipase, abdomen and pelvis CT, and spinal MRI. Head circumference was the most frequently missed measurement. There were significant increases in lipase and fibrinogen testing following order panel implementation. Abdomen and pelvis CT and head circumference testing also increased postintervention, but these increases were not statistically significant. There was an increase in our balancing measure of median time to complete evaluation (30.2–61.6 hours, p=0.005 via Mann-Whitney U testing).

When results were compared across race, ethnicity, language and insurance type in the baseline population, there were no differences in the number of fully complete evaluations, but patients with a preferred language of English had a higher percentage of total evaluation opportunities completed when compared with patients with a preferred language other than English (p=0.048 via χ² testing). Patients self-identifying as Black had the highest median percentage complete evaluation per patient (90% compared with 78% for those identifying as white and 78% for those identifying as Asian, p>0.05 via Fisher’s exact). Following order panel implementation, there were more patients with private insurance with complete evaluations as compared with public insurance (p=0.003 via Fisher’s exact testing), but no differences when comparing the total evaluation opportunities by equity factors. When comparing before and following the intervention, white, Hispanic, non-English speaking, and both private and public insurance groups had an increase in total evaluation opportunities completed following intervention implementation (all p values<0.02 via χ² testing). Patients identifying as Black were the only group to have a decrease in percent evaluation opportunities completed, but this difference was not significant (p=1 via Fisher’s exact).

Discussion

This quality improvement initiative increased the percentage of patients with a complete, guideline adherent evaluation from a mean of 7.7% to 25%, but did not reach the goal of 50%. Additionally, the percentage of evaluation opportunities was more complete (79%–92%) when the order panel was used. Thus, standardising the evaluation of NAT across subspecialty services and hospital location using an order panel with an embedded clinical decision tool outlining testing indications increased the completeness of evaluations.

Despite this success, there was large month-to-month variability in patients with complete NAT evaluations and in order panel uptake as demonstrated in figure 1A,D. This variability, although perhaps linked, is largely impacted by the small number of patients within each data point. Barriers to order panel utilisation included an initial lack of order panel tracking in real-time during the first year after implementation due to limited staff availability. This prevented us from reacting with timely interventions. Continued barriers to utilisation include the need for providers to search for the NAT order panel, as there is no central repository of paediatric order panels in our EMR.

There were some notable discrepancies between observed and anticipated outcomes. We initially hypothesised that utilisation of the order panel would help expedite the NAT evaluation and decrease the time to complete evaluation (balancing measure). This time, however, actually increased in the postintervention group. It is possible that implementation of the order panel prompted additional testing which prolonged the evaluation duration.

In addition, there were no differences by primary specialty service in our baseline population. Regarding injury categories, evaluation for abdominal trauma was the least likely to be completed. Both lipase and abdominal and pelvic CT were identified via Pareto analysis as tests that were frequently missed in the preintervention population. There was success in significantly increasing guideline adherent lipase measurement and overall evaluation for intra-abdominal trauma in the baseline versus postintervention population. However, it remained the least likely category to be complete across injury types, and this is an area to target with additional education and direct prompting within the order panel’s clinical guidance. Finally, though there were no differences in complete evaluations across racial categories in the postintervention population, there was an increase in the percentage of evaluation opportunities completed specifically in white patients when comparing the baseline and postintervention populations. The percentage of complete evaluation for white patients subsequently equalled that of black patients in the postintervention population at 88%.

Regarding more general discrepancies, it is also worth noting that the mean for complete NAT evaluations was 7.7% prior to order panel implementation. Given that FACT consultation guides NAT evaluations, it is important to elaborate on the barriers to evaluation completion as it relates to one of our study limitations. NAT evaluation is often nuanced and challenging to define. In our efforts to standardise this measurement we compiled the operational definitions outlined in table 1 to apply to all paediatric patients during chart review using clinical guidance from existing literature and the expert opinion of our institutional specialist. If any one of the indicated metrics was not obtained the evaluation was labelled as incomplete.

Practically, however there are a variety of reasons that certain studies were not obtained. For example, 3D reconstruction of the head CT was almost universally not obtained in patients who were transferred from outside facilities after the head CT had already been obtained. Though missing the 3D reconstruction component would label an evaluation as incomplete, it would be medically appropriate to defer this given the risk of additional radiation associated with repeat CT scans. We additionally noted evaluations missing vitamin D levels which were secondary to sample clotting and subsequent inability to repeat phlebotomy. This would label an evaluation as incomplete despite awareness of the need to obtain vitamin D. Though the above examples do not constitute medically inappropriate actions, the operational definitions were applied to maintain consistency of best practice.

While we recognise the limitations in such a stringent approach, this also relates to what our project adds to the existing literature on guideline-adherent NAT evaluations. When compared with other similar studies, our order panel uptake was higher than that of Riney et al (55% vs 20%)¹² but they achieved a higher overall percentage of guideline adherent evaluations at 69% of patients. Similarly, Greene et al achieved 100% completion of their management bundles specifically involving five diagnostic components for infants less than 7 months old.¹⁴ Overall, while other studies demonstrated higher rates of guideline-adherent evaluation, these studies often focused on specific clinical scenarios with unambiguous management recommendations. Rather than focusing on a specific age or diagnostic metric our guideline evaluation required additional recommended tests as well as measurements, but more broadly our study approach attempted assessment of NAT evaluation across age group and injury type despite the associated clinical ambiguity. Though there are challenges in quantifying certain elements, for instance whether a bleeding evaluation is indicated, we used in depth chart review to pursue a more global approach to each NAT evaluation. We therefore did not expect 100% complete evaluations even with order panel utilisation, but sought to increase completeness across settings through a standardised order panel with clinical prompts.

Conclusions

Implementation of a NAT order panel increased the percentage of paediatric patients with a guideline-compliant evaluation across all locations and specialty services in our institution. With targeted education of appropriate front-line providers, this order panel has increased in usage, partially because it provides convenience to ordering providers. It also prompts providers to consider a standardised approach to NAT evaluation based on age and injury type, regardless of race, ethnicity, language or insurance type. Given the number of studies demonstrating bias in the evaluation of NAT¹³⁰,³⁴ it is worth noting the significant increase in completed evaluation opportunities for white patients when comparing our baseline and postintervention population. The potential for reduction of bias in the evaluation of NAT remains an area of future interest as related to this quality improvement intervention.

Data availability statement

Data are available upon reasonable request.