Development of an electronic medical record–based child physical abuse alert system

Abstract

Objective

Physical abuse is a leading cause of pediatric morbidity and mortality. Physicians do not consistently screen for abuse, even in high-risk situations. Alerts in the electronic medical record may help improve screening rates, resulting in early identification and improved outcomes.

Methods

Triggers to identify children < 2 years old at risk for physical abuse were coded into the electronic medical record at a freestanding pediatric hospital with a level 1 trauma center. The system was run in “silent mode”; physicians were unaware of the system, but study personnel received data on children who triggered the alert system. Sensitivity, specificity, and negative and positive predictive values of the child abuse alert system for identifying physical abuse were calculated.

Results

Thirty age-specific triggers were embedded into the electronic medical record. From October 21, 2014, through April 6, 2015, the system was in silent mode. All 226 children who triggered the alert system were considered subjects. Mean (SD) age was 9.1 (6.5) months. All triggers were activated at least once. Sensitivity was 96.8% (95% CI, 92.4–100.0%), specificity was 98.5% (95% CI, 98.3.5–98.7), and positive and negative predictive values were 26.5% (95% CI, 21.2–32.8%) and 99.9% (95% CI, 99.9–100.0%), respectively, for identifying children < 2 years old with possible, probable, or definite physical abuse.

Discussion/Conclusion

Triggers embedded into the electronic medical record can identify young children with who need to be evaluated for physical abuse with high sensitivity and specificity.

INTRODUCTION

Child maltreatment is a leading cause of death and disability in children. Annually in the United States, more than 3 million reports are made to Child Protective Services and almost 1600 children die due to maltreatment, almost 4 times the annual number of pediatric cancer deaths.¹ Failure to recognize abuse in its milder forms may result in repeated abuse and increased morbidity and mortality.^2–6 Studies demonstrate that a significant proportion of children diagnosed with physical abuse had been previously evaluated by a physician who did not recognize the abuse.^2–8 There are also economic implications of early diagnosis; the cost of each nonfatal child abuse case is estimated at $210 000, compared with $1.2 million for each fatality.⁹

The American Academy of Pediatrics (AAP) has evidence-based recommendations for which children should be screened for physical abuse and which tests should be done as part of that screening process.^10–12 Despite these recommendations, physicians do not consistently screen for abuse, even in high-risk situations.^13,14 There are also disparities in physician screening practices related to patient characteristics, with publicly insured nonwhite children more likely to be screened than privately insured white children with the same injuries.^2,13–19

Rangel et al. implemented a clinical guideline recommending that a skeletal survey be performed in all infants with a skull fracture or intracranial injury unless the injury was witnessed by a non-family member. Prior to implementation of the guideline, black infants underwent skeletal surveys more frequently than white infants; after implementation, there was no racial difference in skeletal survey completion, and the rate of abuse diagnosis increased.¹⁶

In general, clinical guidelines alone are insufficient to improve care on a long-term basis.^20–22 Emerging literature demonstrates that the electronic medical record (EMR) can be used to improve screening rates for many diseases, allow for early intervention, decrease disparities, and improve outcomes.^23–26 No studies, to our knowledge, have used the EMR to improve standardization of care in suspected physical abuse.

The current study was designed to determine whether triggers embedded in the EMR could alert physicians to the possibility of physical abuse with high sensitivity and acceptable specificity. This type of system could then be linked to order sets that include the AAP-recommended evaluation of suspected physical abuse. The current practice is for physicians to remember the AAP recommendations and order each of the recommended tests individually. The ultimate goal of the child abuse alert system is to both identify at-risk children and link the treating physician to an appropriate order set/power plan to ensure proper evaluation. The current study focuses on the first of these goals, development of the child abuse alert system to identify at-risk children.

METHODS

The protocol was approved by the Institutional Review Board of the University of Pittsburgh. Waiver of informed consent was approved for all subjects.

Inclusion and exclusion criteria

Subjects were defined as children < 2 years old who met one of the criteria that activated a trigger embedded into the EMR. There were no exclusion criteria.

Setting

The study took place in the emergency department (ED) at Children’s Hospital of Pittsburgh of the University of Pittsburgh Medical Center (CHP), a level 1 trauma center. Cerner Millennium^® (Cerner Corp, Kansas, MO, USA) is the EMR used in the CHP ED.

Embedding of triggers into the EMR

Thirty triggers were embedded into the EMR. The combination of all these triggers will hereafter be referred to as the child abuse alert system. Individual triggers were selected after review of child abuse literature and electronic data entry fields available in the CHP EMR. The triggers incorporate discrete fields used throughout a patient’s encounter. A constraint of the current version of our EMR is the inability of free text to generate alerts. Among children at risk for physical abuse, we were particularly interested in identifying those whose injuries met one of the specific scenarios for which the AAP has guidelines for evaluation^10–12: fracture in a noncruising infant, bruise in a <6-month-old, bruise in a noncruising 6- to 12-month-old, intracranial injury in a noncruising infant not in a motor-vehicle crash, and injury reported to Child Protective Services for concern of physical abuse in a child <2 years old.

Running the system in silent mode

Once coding was complete, the system ran in silent mode. During this time, research staff saw which clinicians would receive alerts if the system were live. There was no impact on clinical care, and physicians did not know that the system was being tested. During this time, study personnel, including the hospital’s chief medical informatics officer (SS), child abuse pediatrician (RB), and research coordinator (EH), identified coding errors (eg, use of an “or” term instead of an “and” term) and triggers that were producing a large number of over-triggers (eg, chief complaint of mouth injury in children <1 year old). Errors were corrected and over-triggers removed by hospital information technology personnel. All the reported data, including the specificity and sensitivity of the alert system, are based on analysis performed after all coding changes were complete.

Methods and measurements

Patient-specific data

The following data were directly downloaded from the EMR for children who triggered the alert system: date of ED visit, chief complaint, age, race, gender, zip code, insurance (private, public, uninsured), the trigger that activated the system (ie, first trigger), and a list of all fields that appeared earlier in the coding than the first trigger and potentially could have activated the system. Race information was collected during the registration process as part of clinical care.

Each time the alert system was activated, a co-investigator (JF) reviewed the encounter to determine whether it was appropriate for a physician to be concerned about physical abuse given the clinical scenario (Table 2). Due to ethical concerns that a potentially abused child could be identified by the alert system and not by the treating physician, a child abuse pediatrician (RB) reviewed the ED documentation for all 226 patients who triggered the child abuse alert system. When there was a concern that child abuse should have been considered, the case was referred to the chief of the division of emergency medicine (RS).

Table 2.

Classification system for determining whether activation of the child abuse alert system was appropriate

Possible reasons why it was appropriate for a physician to be concerned about physical abuse (n = 180)

Possible reasons why it was not appropriate for a physician to be concerned about physical abuse (n = 46)

Patient has an injury that should result in screening for physical abuse based on the AAP guidelines (eg, bruise, petechiae, burn, subconjunctival hemorrhage, fracture, or intracranial hemorrhage in a child who is not yet cruising, or bruising, fracture, intracranial hemorrhage, or burn of concern for abuse in a mobile child (n = 122) Patient has a nonspecific symptom that is known to be associated with child abuse, or there is concern that physical abuse is the etiology of an infant’s symptoms (eg, fussy infant, infant with apparent life-threatening event or apnea, vomiting without diarrhea, failure to thrive) (n = 18) Patient is the sibling of a child with known or suspected physical abuse (n = 5) Patient has been involved in an incident of domestic violence (n = 2) Patient has been a victim of child abuse in the past (n = 1) An adult (eg, parent, bystander, or professional) raised concerns of abuse (n = 19) Unexplained death (eg, possible co-sleeping/unsafe sleep) (n = 3) Misassessment by a medical professional (eg, a nurse believes that an infant has a bruise and documents this, causing the system to trigger, but the physician ultimately decides that it is not a bruise, but rather a birthmark) (n = 10)

Concern is for another type of abuse (eg, sexual abuse, neglect) but not physical abuse (n = 27) A miscode resulted in the trigger (eg, the chief complaint of “assault” inadvertently entered when it should have been “asthma”; the system had already triggered prior to correction) (n = 18) There is an injury, but it is not appropriate that a physician would be concerned given the situation (eg, injury witnessed in a public place by a disinterested adult, an isolated toddler’s fracture in a walking 11-month-old) (n = 1)

Defining the first trigger

The first trigger was defined as the one that was entered in the EMR first and activated the child abuse alert system. When 2 triggers are entered at identical times, the trigger the system recognizes first is the one in an earlier line of code. Because the order of the coding affects how the EMR triggers, if there is a second trigger activated and that trigger happens to be after the first trigger in the coding, the second trigger is not recognized by the system as a potential trigger. Therefore, it is not possible to know all the triggers a patient could potentially have activated.

Defining the reference standard

The reference standard against which the child abuse alert system was evaluated was the CHP Child Protection Team’s (CPT) assessment of abuse. The CPT assessment as the reference standard for defining abuse has been used previously by our group and others in abuse-related clinical research.^2,27–30 The CHP CPT is composed of physicians with expertise in child maltreatment. The team reviews all reports of suspected maltreatment made by any hospital-based mandated reporter to Child Protective Services, the government agency tasked with responding to concerns of child maltreatment. The CPT also reviews all trauma-related admissions for children <2 years old to assess whether an abuse evaluation is needed and provides consultation for any child for whom hospital staff is concerned about maltreatment. The CPT provides written documentation for all consultations and assesses the likelihood of abuse as “not abuse,” “possible/concerning for abuse,” “probable/highly concerning for abuse,” or “definitive abuse.”

Validation of the reference standard

The concern that a child in the true negative group could potentially be a false negative (eg, patient missed by the child abuse alert system, by the ED physician caring for the child, and by the CPT) was considered. Given the size of the true negative cohort, it would have been impractical to review all the ED records to determine if any patients in this cohort could have been false negatives. While review of a random selection of children who did not trigger the system to assess for the possibility of a false negative was considered, an alternative approach was used. A nonrandom sample of children who were at high risk of being false negatives was instead selected. Children who were evaluated by the CPT in the year after silent mode were selected, and all the ED visits for these patients that occurred during silent mode were reviewed to determine whether they represented a missed abusive event. Ideally, we would have also had access to another nonrandom sample of children who did not trigger the system, children who were evaluated at other EDs and/or referred to Child Protective Services for concerns of physical abuse in the year after the silent mode period.

Hospital-specific data

In order to assess whether children whose ED encounters resulted in activation of the alert system were representative of children evaluated in the CHP ED, race and insurance status were compared between these groups. Trigger rates as a proportion of all children seen in the ED were calculated each month.

Statistical analysis

Demographics and descriptive data were evaluated using descriptive statistics. P < .05 was considered statistically significant. All statistics were performed using IBM SPSS version 23.0.

The sensitivity, specificity, and negative and positive predictive values (NPV and PPV) for identifying physical abuse were calculated. True positives were defined as children who triggered the alert system and were subsequently classified by the hospital-based CPT as having possible, probable, or definite physical abuse. True negatives were children who were seen in the ED, did not trigger the alert system, and were either not evaluated by the CPT or were evaluated but not classified as having possible, probable, or definite physical abuse. False negatives were children who did not trigger the system, but were classified by the CPT as having possible, probable, or definite physical abuse. False positives were children who triggered the alert system, but for whom it was not reasonable that a physician would consider the possibility of abuse (eg, over-triggers).

RESULTS

The abuse alert trigger system ran on silent mode from October 21, 2014, through April 6, 2015. During this time, 226 children triggered an alert. Mean (SD) age was 9.1 (6.5) months. The gender and race of children < 2 years old who triggered were similar to those of all children <2 years old seen in the CHP ED. More children <2 years old with public insurance triggered alerts compared with all children <2 years old seen in the CHP ED (Table 3).

Table 3.

Comparison of demographics of all children seen at CHP emergency department in fiscal year 2015 (July 1, 2014, to June 30, 2015) and those who triggered the child physical abuse alert

	Children who triggered (n = 226) (%)	All children < 2 years old seen in CHP ED (n = 21 349) (%)	P value
Gender (% male)	61	56	NS
Race (% Caucasian)	68	60	NS
Insurance (% with public insurance)	77	62	<0.03

NS, not significant.

The mean (SD) trigger rate as a proportion of all children evaluated in the ED was 0.6 (0.1)% (Table 4).

The most common first triggers were an order for a skeletal survey (31.9%), concern for abuse or report of child abuse documented on the prearrival form (16.8%), and chief complaint of assault or suspected child abuse and neglect (15.9%). Each embedded trigger alerted at least once (Table 1

Table 5.

2 × 2 Contingency table

		Diagnosis of possible, probable, or definite physical abuse
		Positive	Negative
Trigger fired	Positive	60	166	PPV = 26.5%
	Negative	2	10 710	NPV = 100.0%
		Sensitivity = 96.8%	Specificity = 98.5%

Table 4.

Trigger rates as a proportion of all patients seen in the ED during silent mode

Month	No. of ED patients	No. of patients who triggered the alert system	Trigger rate (percentage of all patients)
October 21–31, 2014	2386	18	0.8
November 2014	6353	34	0.5
December 2014	7654	48	0.6
January 2015	6820	35	0.5
February 2015	6569	39	0.6
March 2015	7141	42	0.6
April 1–6, 2015	1311	10	0.8
Total	38 234	226	Mean (SD) 0.6 (0.1)

).

Table 1.

Descriptions of the 30 triggers coded into the EMR

Discrete field that results in a trigger/position of professional who enters data in this field	Name and value of discrete field that results in a trigger when combined with patient age	Patient age (in years) that results in a trigger when present with discrete field	Criteria that prevent activation of the child abuse alert system	Number of instances in which this was the first trigger (percentage of all first triggers)
Prearrival documentation /nurse	Is there concern for abuse or neglect? – “yes” response	<2	NA	38 (16.8)
	OR
	Has a ChildLinea been filed? − “yes” response
Chief complaint/nurse	Assault OR SCAN	<2	NA	36 (15.9)
Chief complaint/nurse	Bruising OR Burn OR Petechiae OR Fracture	<1	NA	19 (8.4)
Mechanism of injury (completed when patient assessed to be a trauma patient)/nurse	Assault OR SCAN OR Burn	<2 for Assault or SCAN; <1 for Burn	If “Mechanism of Injury” response to “MVC” = Yes	2 (0.9)
Documentation of “skin characteristic” OR “head and neck assessment”/nurse	Burn OR bruise OR petechiae on “skin characteristic”; subconjunctival hemorrhage on “head and neck assessment”	<1	If “bruise” and patient age 6–11.99 months and response to “Is child able to cruise or walk?” = Yesb	15 (6.7)
Documentation of “head and neck assessment”/nurse	Left ear condition – bruising	<2	NA	1 (0.4)
	OR
	Right ear condition – bruising
Social work screen/nurse	Reason for consult = SCAN	<2	If “Mechanism of Injury” response to “MVC” = Yes	7 (3.1)
Orders placed/physician or APP	Orthopedic consult for fracture AND/OR	<1	If “Mechanism of Injury” response to “MVC” = Yes	6 (2.7)
	Neurosurgery consult for skull fracture AND/OR Neurosurgery consult for intracranial hemorrhage
Orders placed	X-rayc AND medication order for fentanyl, midazolam, ketorolac, naloxone, or flumazenil	<1	NA	0d
Order placed/physician or APP	Skeletal survey	<2	NA	72 (31.9)
Order placed/physician or APP	Consult social work reason “SCAN”	<2	NA	9 (3.9)
Social work documentation/social worker	Type of referral “SCAN” OR action “ChildLine-physical abuse”	<2	If “Mechanism of Injury” response to “MVC” = Yes	6 (2.7)
Discharge instruction/physician or APP	Burn – first degree, burn – second degree OR Fracture – clavicle, Fracture – humerus	<1	If “Mechanism of Injury” response to “MVC” = Yes	9 (3.9)
	Fracture (cast care), hand fracture
Discharge instruction/physician or APP	Contusion OR shoulder dislocation	<1	If response to “Is child able to cruise or walk?” = Yesb	6 (2.7)

^aChildLine is the report to Child Protective Services in Pennsylvania.

^bIf this discrete field is not completed, the trigger system assumes the answer is no for children <1 year and yes for children >1 year.

^cX-rays included in the trigger: Clavicle left; Clavicle right; Foot 3 views minimum left; Foot 3 views minimum right; Fingers 2 views minimum left; Fingers 2 views minimum right; Knee 3 views left; Knee 3 views right; Shoulder 2 views minimum left; Shoulder 2 views minimum right; Femur 2 views left; Femur 2 views right; Humerus 2 views minimum left; Humerus 2 views minimum right; Skull 4 views minimum; Ankle 2 views left; Ankle 2 views right; Ankle 3 views minimum left; Ankle 3 views minimum right; Elbow 2 views left; Elbow 2 views right; Elbow 3 views minimum left; Elbow 3 views minimum right; Tibia and fibula 2 views left; Tibia and fibula 2 views right; Wrist 2 views left; Wrist 2 views right; Wrist 3 views minimum left; Wrist 3 views minimum right; Foot 2 views left; Foot 2 views right; Hand 2 views minimum left; Hand 2 views minimum right; Knee 1 or 2 views left; Knee 1 or 2 views right; Knee Lt Comp 4 or more views; Knee Rt Comp 4 or more views; Knees bilateral AP standing.

^dTrigger for sedation medication + X-ray in a patient <1 year was never the first trigger, but was a later trigger.

Abbreviations: NA, not applicable; APP, advanced practice provider; SCAN, suspected child abuse and neglect; MVC, motor vehicle crash.

Sensitivity of the trigger system

During silent mode, 62 children <2 years old were diagnosed with possible, probable, or definite physical abuse by the CPT; 60 triggered the alert system and 2 did not. The sensitivity of the trigger system for children <2 years old was, therefore, 96.8% (95% CI, 92.4–100%).

The 2 physical abuse cases missed by the alert system (false negatives) were reviewed to determine whether a change in the trigger system could potentially increase the sensitivity without significantly impacting the over-trigger rate. One patient was in the ED < 1 h before being transferred to the pediatric intensive care unit. Because of the severity of the injuries, there was no real-time documentation in the EMR. This infant was immediately identified clinically as needing consultation by the CPT, and would also have been identified by the CPT because the infant was admitted for trauma. The other infant not identified by the child abuse alert system was evaluated by the CPT after a skeletal survey performed as part of the inpatient admission for failure to thrive demonstrated a fracture. During the ED visit, there had been no concern for physical abuse and the concern was only failure to thrive.

Specificity of the trigger system

During silent mode, 10 710 children < 2 years old did not trigger the alert system and were either not evaluated by the CPT or were evaluated but not assessed as having possible, probable, or definite abuse (ie, true negatives). There were 46 children < 2 years old who triggered the alert system but for whom there was no reason to be concerned about physical abuse (Table 2); the reasons for over-triggers were “concern for neglect or sexual abuse, not physical abuse” (n = 27), a miscode (n = 18), and a birth injury recognized prior to the ED visit (n = 1). There were also 121 subjects for whom it was reasonable to be concerned about abuse, but who were ultimately not diagnosed as having possible, probable, or definite abuse. These patients are, therefore, considered in the cohort of false positives. The specificity of the trigger system for children <2 years old was, therefore, 98.5% (95% CI, 98.3–98.7). The PPV and NPV were 26.5% (95% CI, 21.2–32.8 %) and 100% (95% CI, 99.9–100%), respectively.

There were 210 children evaluated by the CPT in the year after silent mode. Of these, 40 had a total of 61 ED visits that had occurred during the silent mode period. Ninety-seven percent (59/61) of the visits were unrelated to trauma; the majority were for respiratory symptoms and/or fever. Two visits represented possible opportunities to identify maltreatment. The first was a 6-month-old with a closed head injury and a bruise who was observed for 4 h and discharged. The subsequent CPT referral was for a tooth injury in a 10-month-old and was assessed as not being the result of abuse. The second patient was a 9-month-old infant who was seen in the ED for a routine physical examination prior to foster care placement due to poor housing conditions. In the physician note (which is free text and cannot be used to trigger) a “dime-sized brown bruise on left upper calf” was noted. There was no concern for abuse. The subsequent CPT referral was for medical neglect. Based on these data, it is unlikely that there is a significant number of false negatives in the true negative cohort.

Alert variables present prior to actual system trigger

In 4% (9/226) of cases, there was an earlier point during the visit when the alert system would have triggered but did not because of nonspecific documentation: lack of documentation of the concern of abuse by an outside hospital (n = 5), documentation of “injury” (not a trigger) as a chief complaint instead of “fracture” (a trigger) in an infant presenting with a known fracture (n = 2), and free-text (vs discrete field) documentation of bruising by a nurse (n = 2).

DISCUSSION

This study demonstrates that it is possible to embed a series of triggers into the EMR to identify young children who may need a physical abuse evaluation. The high sensitivity for identification of possible, probable, or definite physical abuse in children <2 years old is encouraging, since this is the age group with the highest morbidity and mortality from abuse. While there is no acceptable rate of over-triggering/false positives, the risk of missed diagnosis must be balanced with the possibility of “alert fatigue.”³¹ The PPV of 26.5% suggests that 3 of every 4 alerts do not lead to a diagnosis of possible, probable, or definite physical abuse. While this may be perceived as a low PPV, an argument could be made that this is far too high given the morbidity and mortality from missed child physical abuse. In addition, since cases of physical abuse in children <2 years old are relatively infrequent in the context of all children seen in pediatric EDs and even lower for any given physician, alert fatigue is unlikely to be a significant problem even with a low PPV. Importantly, although the number of false positive cases was 166, in 121 (73%) of these cases, it was reasonable for the physician to be concerned about abuse even though, ultimately, the diagnosis of possible, probable, or definite abuse was not made.

The earlier in a patient encounter the first trigger occurs, the more likely the trigger will influence care; ideally, the provider would receive the alert before entering a patient’s room for the initial evaluation. As a result, a trigger such as prearrival documentation is more likely to influence clinical care than an order for an orthopedic consult for a fracture. The child abuse alert system is designed to trigger when a chart is opened, an order is placed, or a patient is discharged; assessment of when the trigger actually would occur in practice could not be evaluated while the system was in silent mode, but will be assessed when the system is live.

While more than half of the patients who triggered the child abuse alert system had a trigger, which suggests that there was already clinical concern for abuse, clinical experience suggests that just because a physician or nurse at a referring hospital has a concern for abuse, this does not necessarily translate into physician knowledge of that concern at the referral hospital. When the alert system is live (and not in silent mode), triggering based on prearrival information, for example, ensures that the physician caring for the patient is aware of the concern for abuse raised by other medical professionals. Similarly, just because a skeletal survey is ordered does not necessarily mean that the physician has followed the AAP guidelines for evaluation of physical abuse, which includes neuroimaging in young infants as well as bloodwork to evaluate for occult abdominal injury and other medical conditions that can mimic abuse. While the current study was focused on whether the child abuse alert system can identify children with possible physical abuse, identification of risk is not sufficient and must be followed by proper evaluation for physical abuse, which is the reason why skeletal survey was included as a trigger. In 20%–30% of cases, the child abuse alert appeared to raise the first clinical concern for abuse. Interestingly, this number is similar to the proportion of physical abuse cases that are thought to be “missed” based on clinical judgment alone.^3,4,32

Our data demonstrate that more children with public insurance triggered the alert system compared with the overall population of similarly aged children evaluated in the CHP ED. This is consistent with data demonstrating that poverty is a risk for child abuse.^33,34

Determining the NPV of a screening test for physical abuse can be difficult when the reference standard (eg, the CPT assessment) is not 100% sensitive for abuse. While there is no way to be sure that there are no false negatives within the true negative group, our approach of reviewing the ED records for all patients for whom there was concern for abuse in the year after the end of silent mode is likely to identify the vast majority of missed cases and is an approach that has been used by other child abuse–related studies.^2,32,35 Even chart review of all 10 771 children in the true negative cohort could not guarantee that all false negatives would be identified; identification of missed cases within this cohort would depend on accurate EMR documentation of a comprehensive history and physical exam. The fact that no missed cases of abuse were identified by our approach strongly supports our reference standard.

While older children can also be victims of physical abuse, our alert system focused on young children, since morbidity and mortality are greatest in this age group. It is also the age group for which the EMR is likely to be most helpful, since there are certain physical exam and radiologic findings that are relatively specific for abuse but lose their specificity in older children. For example, a fracture in an infant is almost always concerning for abuse; the same injury in a mobile child is far more likely to be accidental. Even within our cohort of children <2 years old, there were many more for triggers in children <1 year old compared with those 1–2 years old.

Although this study was conducted in a pediatric ED in an academic center, most children who receive care in EDs in the United States are evaluated in general EDs, where the clinicians providing care have less training than pediatricians related to identification of child abuse. Several studies have demonstrated disparities in the rate of abuse diagnosis between adult and pediatric physicians and suggest that missed cases of abuse occur with higher frequency in general EDs.^3,13,15,32 Future research will focus on the use of a similar trigger system in non-pediatric and community EDs.

Finally, the goal of the current study was to investigate the feasibility of developing an embedded child abuse alert system specifically for young children with injuries that might be the result of abuse. It was not meant to identify other forms of abuse, such as sexual abuse, and does not help to identify when infants with nonspecific symptoms such as fussiness should be evaluated for abuse. While these are both important clinical issues, they are beyond the scope of this alert system. Assessing the accuracy of the alert system in silent mode was a prerequisite to moving it to live mode. Had the sensitivity and specificity not been adequate, incorporating the alert system into clinical practice could potentially do more harm than good. As a result, the current study should be seen only as a necessary first step; the next step is to evaluate whether this system affects clinical practice.

Limitations

The most important limitation relates to the quality of the reference standard; it is possible that a patient would not be known to the CPT, would not trigger the alert system, and would not raise clinical concern for abuse. Because of this limitation, the sensitivity of the trigger system may be an overestimate. We do not think that this is a significant limitation, however, since there were no cases of missed abuse identified during our review of prior ED visits for all children who were evaluated by the CPT in the year after the silent mode period. The inability of free text to generate alerts and the inability to know all the triggers that could have occurred for a given patient are limitations of the EMR system and the coding approach, respectively. Finally, scalability is a limitation, given the different EMRs in different hospital systems as well as the differences among Cerner builds within the same hospital system; however, this is a limitation of the approach of using the EMR rather than of the alert system per se.

CONCLUSION

We demonstrated that it is possible to embed a set of triggers into the EMR that alert physicians to young children who are at risk for physical abuse. A randomized controlled trial comparing the medical evaluation and compliance with AAP guidelines in cases in which physicians do and do not receive alerts is under way.

Funding

The study was supported by PCORI-AD-12‐11‐4956. The statements in this article are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute, its Board of Governors or Methodology Committee.

Competing Interests

The authors have no conflicts of interest relevant to this article to disclose.

Contributors

RPB: Substantial contributions to the conception or design of the work, drafting the work, and final approval of the version to be published, and agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

RAS: Acquisition and interpretation of data, revision for important intellectual content, final approval of the version, and agreement with the accuracy and integrity of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

JF: Acquisition, analysis, and interpretation of data, revision for important intellectual content, final approval of the version, and agreement with the accuracy and integrity of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

EH: Acquisition, analysis, and interpretation of data, revision for important intellectual content, final approval of the version, and agreement with the accuracy and integrity of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

SS: Analysis and interpretation of data, revision for important intellectual content, final approval of the version to be published, and agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

TMcG: Analysis and interpretation of data, revision for important intellectual content, final approval of the version to be published, and agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.