Validation of the Pittsburgh Infant Brain Injury Score for Abusive Head Trauma

Abstract

BACKGROUND:

Abusive head trauma is the leading cause of death from physical abuse. Misdiagnosis of abusive head trauma as well as other types of brain abnormalities in infants is common and contributes to increased morbidity and mortality. We previously derived the Pittsburgh Infant Brain Injury Score (PIBIS), a clinical prediction rule to assist physicians deciding which high-risk infants should undergo computed tomography of the head.

METHODS:

Well-appearing infants 30 to 364 days of age with temperature <38.3°C, no history of trauma, and a symptom associated with an increased risk of having a brain abnormality were eligible for enrollment in this prospective, multicenter clinical prediction rule validation. By using a predefined neuroimaging paradigm, subjects were classified as cases or controls. The sensitivity, specificity, and negative and positive predictive values of the rule for prediction of brain injury were calculated.

RESULTS:

A total of 1040 infants were enrolled: 214 cases and 826 controls. The 5-point PIBIS included abnormality on dermatologic examination (2 points), age ≥3.0 months (1 point), head circumference >85th percentile (1 point), and serum hemoglobin <11.2g/dL (1 point). At a score of 2, the sensitivity and specificity for abnormal neuroimaging was 93.3% (95% confidence interval 89.0%–96.3%) and 53% (95% confidence interval 49.3%–57.1%), respectively.

CONCLUSIONS:

Our data suggest that the PIBIS accurately identifies infants who would benefit from neuroimaging to evaluate for brain injury. An implementation analysis is needed before the PIBIS can be integrated into clinical practice.

What’s Known on This Subject:

Abusive head trauma (AHT) is the leading cause of death from physical abuse. Identification of AHT, particularly in its mild forms, is difficult; missing the diagnosis can result in increased morbidity and mortality.

What This Study Adds:

The Pittsburgh Infant Brain Injury Score may be able to assist physicians to decide whether an infant at increased risk for AHT would benefit from computed tomography of the head.

Abusive head trauma (AHT) is the leading cause of death from traumatic brain injury in infants^1–3 and the leading cause of death from physical abuse in the United States.⁴

A retrospective study using the Centers for Disease Control and Prevention definition of AHT demonstrated a rate of ∼1 in 3000 infants.⁵ Studies in other countries suggest similar incidences.^6,7

Proper diagnosis of mild AHT is difficult because caretakers rarely provide an accurate history,⁸ infants present with nonspecific symptoms, such as vomiting or fussiness, and physical examination is often normal.^9–11 As a result, misdiagnosis is common and can have catastrophic medical consequences.^11–13 In a landmark study,¹² 31% (54/173) of children diagnosed with AHT were evaluated previously by a physician for symptoms compatible with brain injury. A multicenter study 15 years later demonstrated a similar rate of missed diagnoses (M. Letson, MD, MEd, personal communication, 2016), suggesting that early, accurate diagnosis of AHT continues to be challenging.

Although AHT is the leading cause of morbidity and mortality from brain injury in infants, infants with atraumatic neurologic abnormalities, such as hydrocephalus or a brain tumor, and infants with traumatic injuries that are not due to abuse, can present with the same symptoms as infants with AHT. Timely diagnosis of these non-AHT–related brain abnormalities also can be difficult for the same reasons as early recognition of AHT can be difficult: physicians may not consider a brain abnormality as a cause of the infant’s symptoms.

Clinical prediction rules (CPR) are tools that quantify the contributions that components of the history, physical examination, and laboratory tests make toward a patient’s diagnosis.¹⁴ CPRs are particularly useful in diseases in which clinical stakes are high and clinical experience and intuition are insensitive.¹⁵

The Pittsburgh Infant Brain Injury Score (PIBIS) CPR was retrospectively derived based on data from 187 infants (150 without brain injury and 37 with mild AHT) who presented to a tertiary care children’s hospital for evaluation of nonspecific symptoms (R.P.B., unpublished data). A 5-step CPR derivation process was performed.¹⁶ Five predictor variables were identified: age ≥3 months, head circumference percentile >90%, serum hemoglobin <11.2 g/dL, abnormality on neurologic or dermatologic examination, and a previous emergency department (ED) visit for a high-risk symptom. The receiver operator characteristic (ROC) curve using these predictor variables showed an area under the curve (AUC) of 0.87 (95% confidence interval [CI] 0.80–0.95).

The current study was designed as a multicenter, prospective validation and refinement of the PIBIS CPR.

Methods

Subjects

The protocol was approved by the institutional review boards at Children’s Hospital of Pittsburgh of UPMC (CHP), Primary Children’s Hospital in Salt Lake City, Utah (SLC), and Ann & Robert H. Lurie Children’s Hospital of Chicago (CHG). Enrollment at CHP started on October 1, 2006, SLC on June 1, 2010, and CHG on January 1, 2011. Consent was obtained except in cases of suspected abuse in which a waiver of informed consent was approved by all 3 institutional review boards.

Enrollment Criteria

Children were eligible if they were 30 to 364 days of age, well-appearing, and presented to a participating ED with a temperature <38.3°C, without a history of trauma and for evaluation of a symptom that is associated with an increased risk of AHT^11,12 (Table 1). The only exclusion criterion was having a previously abnormal computed tomography (CT) scan of the head.

TABLE 1.

Study Inclusion Criteria

Inclusion Criteria: All 4 Criteria Must Be Met	Definition
30–364 d of agea AND	Self-explanatory
Well-appearing AND	Defined as GCS score of 13–15 OR by description of the attending physician when no GCS score assigned
Temperature <38.3°C AND	Defined as no measured temperature ≥38.3°C in the previous 24 h
No history of trauma AND	History of trauma not given by caretaker as the reason for seeking medical care. If history of trauma was later provided by caretakers, this was not considered to be a history of trauma for purposes of eligibility
Seeking medical evaluation for 1 of the following symptoms	ALTE as defined by the National Institutes of Health17
(1) ALTE/apnea	Vomiting without diarrhea defined as >4 episodes of vomiting in the previous 24 h OR ≥3 episodes of vomiting per 24 h for the previous 48 h
(2) vomiting without diarrhea
(3) seizures or seizurelike activity
(4) soft tissue swelling of the scalp
(5) bruising
(6) other nonspecific neurologic symptom not described above, such as lethargy, fussiness, or poor feeding

GCS; Glasgow Coma Scale Score.

^aChildren <30 d of age were excluded because the validation of PIBIS was part of a larger study evaluating the use of serum biomarkers to identify brain injury. Because serum biomarkers of brain injury are often abnormal in healthy infants <30 d of age, neonates were excluded from the entire study.

Measures

At Enrollment

History of present illness, past medical history including previous ED visits, results of laboratory and radiologic testing, neurologic and dermatologic examination findings, serum hemoglobin, head circumference, and discharge diagnoses were collected. Neurologic and dermatologic examinations were recorded prospectively by the attending ED physician by using a checklist (Fig 1). If serum hemoglobin or neuroimaging were not done for clinical care, consent was obtained to participate in research. Due to increasing concern by treating physicians regarding radiation exposure, no research CTs were performed after May 1, 2012. No data about socioeconomic status or social history were collected.

FIGURE 1.

Physical Examination Assessment for Clinical Prediction Rule Study.

Follow-up

Subjects were tracked by medical record review for 6 months after enrollment or up to 1 year of age, whichever came later. The goal of follow-up was to identify subjects with abnormal neuroimaging during the follow-up period and/or those who had neuroimaging performed to follow up on symptoms at enrollment.

Classification of Subjects

Subjects were classified as cases or controls based on the results of neuroimaging. Neuroimaging was classified dichotomously by using a paradigm developed a priori (Table 2). Controls included subjects with normal neuroimaging or no neuroimaging at enrollment and during follow-up. Cases included subjects with abnormal neuroimaging at enrollment or during follow-up. Cases were further classified into those with possible traumatic, probable/definite traumatic, and atraumatic abnormalities.

TABLE 2.

Neuroimaging Classification Paradigm

Classification	Definition and Examples
Normal/clinically insignificant abnormality	Clinically insignificant abnormality defined as an incidental finding that does not result in any follow-up or is unrelated to the clinical presentation. These include the following:
	(1) mild prominence/enlargement or asymmetry of ventricles
	(2) prominent suture(s) or vascular grooves
	(3) enlarged posterior fossa
	(4) plagiocephaly
	(5) mild volume loss
	(6) resolving cephalohematomaa
	(7) small cysts
	(8) isolated soft tissue swellingb
	(9) sequelae of birth trauma (eg, periventricular leukomalacia, intraventricular hemosiderin, incidental finding of posterior fossa subdural)
	(10) benign extra-axial fluid of infancy
Equivocal	Equivocal defined as an interpretation prefaced by “possible,” “probable,” “suspicious for,” “cannot rule out,” or “versus.” All findings initially assessed as equivocal were subsequently categorized as “normal/clinically insignificant abnormality” or “abnormal” based on clinical testing (eg, additional CTs or MRIs) occurring during the follow-up period. If no follow-up testing was performed during the follow-up period, then the equivocal finding was considered to be “normal/clinically insignificant.”
Abnormal	Three categories:
	Probable/definite traumac
	(1) most cases of acute extra-axial hemorrhage
	(2) skull fracture/skull fracture with underlying intracranial hemorrhage
	(3) intraparenchymal contusion/hemorrhage
	Possible trauma
	(1) cases of acute extra-axial hemorrhage with atypical clinical circumstance (eg, underlying bleeding disorder, with moderate/severe volume loss, with significant extra-axial spaces)
	(2) chronic SDH (without acute SDH)
	(3) moderate or severe volume loss
	(4) laminar necrosis
	(5) encephalomalacia
	(6) cerebral edema (vasogenic or cytotoxic/stroke): localized or diffuse
	Not trauma
	(1) mass lesions/tumors/cavernoma
	(2) hydrocephalus
	(3) craniosynostosis/skeletal dysplasias/other bony abnormalities
	(4) any type of cortical dysplasia
	(5) Misc including tuberous sclerosis, Dandy-Walker malformation, arteriovenous malformation, stroke

SDH, subdural hemorrhage.

^aCan be traumatic, but is so commonly related to birth trauma in infants that it would not influence clinical care.

^bCan be traumatic, but in the absence of a skull fracture or other evidence of trauma, it is far more likely to be due to positioning or a normal variant and would not influence clinical care.

^cAll cases of AHT were classified as probable/definite trauma.

CTs and MRIs were interpreted as part of clinical care and by a study neuroradiologist (L.F.). When there was a difference between interpretations, a pediatric neurosurgeon (E.C.T.K.) reviewed the images and consensus was reached. SLC and CHG images were de-identified.

AHT diagnosis was defined as a brain injury that was assessed by each site’s hospital-based Child Protection Team (CPT) as being due to definite or probable, but not possible, abuse. Using the assessment of the CPT is a common way to define AHT in clinical research.^1,12,18

Statistical Analysis

Demographics, descriptive data, and site-to-site comparisons were calculated by using descriptive statistics. A P < .05 was considered statistically significant. Listwise deletion was used to handle missing data. All statistics were performed using IBM SPSS version 22.0 (IBM SPSS Statistics, IBM Corporation, Chicago, IL).

PIBIS Validation and Refinement

The 5 predictor variables identified in the retrospective derivation were reevaluated; variables with a nonsignificant log odds ratio and that failed to increase logistic regression model fit statistics when predicting group (case versus control) were dropped from further analysis. For the remaining predictor variables, the cut-points identified in the retrospective derivation, as well as several additional cut-points, were tested. Additional cut-points were examined to see whether minor changes would improve the odds ratio, model fit statistics, and/or accuracy of PIBIS. The additional cut-points were based on mathematical, rather than clinical, criteria. For example, the previous cutoff for hemoglobin was 11.3 g/dL; 11.1, 11.2, 11.4, and 11.5g/dL were, therefore, tested. Each predictor variable was then recoded as a binary value and logistic regression was used to evaluate the relationship between each binary predictor variable and group.

Weights were assigned to each variable based on the relative magnitude of the regression coefficients in the logistic regression and these were used to generate the PIBIS. The formulas developed with these weights were evaluated by using ROC curves.

Results

From October 2006 to April 2014, 1040 subjects were enrolled: 801 at CHP, 138 at SLC and 101 at CHG. Mean (SD) age was 4.7 (3.1) months; 52% were boys and 78% were white. Fifty-nine percent came directly to the ED and 40% were transferred from a physician office (30%) or referring hospital (10%). Subjects presented due to seizurelike activity (25%), apparent life-threatening event (ALTE) (19%), vomiting (16%), fussiness (16%), scalp swelling (8%), bruise (6%), or a combination of symptoms (10%).

Classification of Subjects as Cases Versus Controls

Of the 1040 subjects, 826 (79%) were classified as controls and 214 (21%) as cases. There was no difference between sites in the proportion of cases. Overall, 507 (61%) of 826 of controls underwent head CT and/or MRI at enrollment or during follow-up. Eighty-three (10%) of the CTs were performed for research.

Among the 214 cases, 213 had neuroimaging at enrollment. The case without imaging at enrollment presented with seizures 2 weeks after enrollment; MRI showed chronic subdural hemorrhage. Of the 214 neuroimaging abnormalities, 166 (78%) were probable/definite trauma, 29 (13%) were possible trauma, and 19 (9%) were atraumatic abnormalities that included hydrocephalus (n = 6), cortical dysplasia (n = 7), tumors (n = 2), stroke (n = 1), arteriovenous malformation (n = 1), craniosynostosis (n = 1), and tuberous sclerosis (n = 1).

AHT was diagnosed in 109 subjects; all 109 were classified as probable/definite trauma by the paradigm in Table 2. There were also 51 subjects assessed by the CPT as having possible AHT; 28 had skull fractures without intracranial hemorrhage, 12 had skull fractures with an underlying hemorrhage, 5 had chronic subdural hemorrhages, and 5 had acute extra-axial hemorrhage with atypical clinical circumstances (eg, moderate/severe volume loss).

With the exception of children who presented with scalp swelling, in whom 75 (87%) of 86 were cases, a similar proportion of subjects with each presenting symptom was classified as cases: 35 (14%) of 260 infants with a seizure, 23 (12%) of 197 with an ALTE/apnea, 24 (15%) of 165 with fussiness, 19 (12%) of 161 with vomiting, and 13 (20%) of 65 with a bruise.

Subjects With Noncranial Abuse

Sixty-nine subjects without brain injury were diagnosed with noncranial abuse; 34 had abusive fracture(s) and 35 had abusive dermatologic injury.

Evaluation of Predictor Variables By Using Logistic Regression

The variables “a previous ED visit for a high-risk symptom” and “neurologic examination” had nonsignificant log odds ratio and failed to increase the logistic regression model fit statistics and were, therefore, removed from further analysis. The previously established cut-points for the other predictor variables were robust; the only variable for which there was a change in the cutoff relative to the derivation was head circumference, which decreased from 90% to 85%. When all 3 sites were included, the overall χ² test was statistically significant (χ² = 243.18, P < .001). Using the cut-points in the logistic regression yielded sensitivity and specificity that were nearly identical to the data in their continuous form (Table 3).

TABLE 3.

Logistic Regression Coefficients for PIBIS

Variables	B	SE	Wald	df	P	Exp(B)	95% CI for EXP(B)
							Lower	Upper
Abnormality on dermatologic examination	2.242	0.201	124.833	1	.000	9.414	6.353	13.951
Age ≥3.0 mo	0.730	0.216	11.391	1	.001	2.075	1.358	3.171
Head circumference >85th percentile	1.221	0.227	28.952	1	.000	3.392	2.174	5.292
Hemoglobin <11.2 g/dL	0.950	0.201	22.433	1	.000	2.586	1.745	3.832

ROC Curve

The 5-point PIBIS was computed by using the following weights: abnormal dermatologic examination (2 points), age ≥3.0 months (1 point), head circumference >85th percentile (1 point), and serum hemoglobin <11.2 g/dL (1 point) (Table 4). The ROC curve using these cutoffs had an AUC of 0.83 (95% CI 0.80–0.86) (Fig 2). The AUCs for each site were similar: CHP 0.83 (95% CI 0.80–0.86), CHG 0.82 (95% CI 0.72–0.91), and SLC 0.83 (95% CI 0.74–0.91).

TABLE 4.

PIBIS

Variable	Points
Abnormality on dermatologic examination	2
Age ≥3.0 mo	1
Head circumference >85th percentile	1
Hemoglobin <11.2 g/dL	1

FIGURE 2.

ROC for 3 sites combined, ROC curve demonstrating ability to discriminate cases from controls. AUC = 0.83 (95% CI 0.80–0.86), P < .00.

Missing Data

Eighty-one percent of subjects had data available for all 4 CPR variables, 18% had 1 missing variable, and 1% had 2 missing variables. There was no difference between sites in terms of the proportion of subjects with full data (χ² = 9.35, P = .53). There was a difference in the proportion of subjects with missing data by group: 3% of subjects with abnormal neuroimaging, 11% of subjects with normal neuroimaging, and 41% of subjects without neuroimaging had missing data.

Sensitivity, Specificity, and Negative and Positive Predictive Values

Sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV) were calculated for the 862 subjects with complete data. The sensitivity for identification of abnormal neuroimaging at a score of 2 was 93.0% (95% CI 89.0%–96.0%). The specificity for detection of abnormal neuroimaging at a score of 2 was 53.0% (95% CI 49.0%–57.0%). The NPV of a score of <2 for detection of abnormal neuroimaging was 96.0% (95% CI 93.6%–97.9%), meaning that if a patient had a score of 0 or 1, the clinician could be 96% confident that the infant did not have a brain injury. The PPV of a score ≥2 for detection of abnormal neuroimaging was 39.0% (95% CI 34.8%–43.6%). There was no difference in the sensitivity, specificity, NPV or PPV among sites (Table 5).

TABLE 5.

Sensitivity and Specificity of the PIBIS for Abnormal Neuroimaging at Each Score

Score	Sensitivity	Specificity
0	1.00	0
1	0.99	0.12
2	0.93	0.53
3	0.81	0.75
4	0.45	0.90
5	0.12	1.00

Of the 69 subjects with noncranial abuse, 64 had data available for all 4 predictor variables and 89% (57/64) had a PIBIS score ≥2.

Discussion

This multicenter prospective CPR validation demonstrates that 4 easily measureable clinical variables (dermatologic examination, age, head circumference, and serum hemoglobin) can help physicians assess which well-appearing high-risk infants might benefit from a head CT. The consistency in the data among the 3 sites and between the retrospective derivation and the prospective validation support the robustness of this CPR. As with all CPRs, PIBIS is meant to supplement, not replace, clinical judgment. PIBIS was designed to be used in well-appearing infants in whom brain injury may not be part of the initial differential diagnosis; it should not be used on infants needing emergent neuroimaging.

The goal of PIBIS is not to decrease the number of infants who undergo the screening test, head CTs in this case. Instead, the goal is to target head CTs to those infants who are most likely to have a positive result, an important goal given the growing concern about head CT use in young children.¹⁹ Consistent with this goal is that the proportion of infants who would undergo head CT by using PIBIS (55%) is almost identical to the proportion who underwent head CT as part of routine clinical care (59%). In the case of mild AHT in which published data suggest that clinical judgment has a sensitivity of ∼70%,¹² our validation suggests that this sensitivity can be increased to >90% without an increase in head CT use. At the same time, the very high NPV when the PIBIS is 0 or 1 (96% [95% CI 93.6%–97.9%]) demonstrates that there is a cohort of low-risk infants who can be safely discharged without neuroimaging.

PIBIS is not designed to diagnose AHT, but to prompt the treating physician to consider the brain as a possible etiology of an infant’s symptoms. This is an important distinction because brain abnormalities, such as a brain tumor, can present with the same symptoms as AHT. That just >50% (109/214) of the children with abnormal neuroimaging had definite/probable AHT is evidence that AHT is just one of many conditions that may present with soft neurologic signs. At the time PIBIS is applied, the cause of the presenting symptom is not known; as a result, sensitivity, specificity, NPV, and PPV were calculated for brain abnormalities in general rather than for AHT in particular. Importantly, PIBIS appears to be able to identify infants with both atraumatic and traumatic abnormalities. If a head CT demonstrates abnormalities that may be consistent with AHT, then a full evaluation for AHT would need to be strongly considered. The relatively small number of patients with atraumatic conditions relative to the number of patients with AHT is consistent with the fact that the incidence of AHT is higher than that of brain tumors²⁰ or hydrocephalus.²¹ The incidence of AHT in this study was higher than what we predicted when performing our initial sample size calculation. This was likely because the study took place over a period that included the Great Recession when the rate of AHT increased in multiple regions of the country, including western Pennsylvania, where most of the study subjects were enrolled.^22,23

PIBIS was developed to maximize sensitivity with the goal of being able to identify >90% of well-appearing infants with AHT. Our data suggest that using PIBIS with a cutoff score of 2 would identify >90% of children with abnormal neuroimaging with a specificity of just >50%. Increasing the cutoff to 3 would decrease sensitivity to 81%, but would increase the specificity to 75%. Deciding whether to recommend a cutoff of 2 or 3 depends, in part, on the risk of neuroimaging. Although head CT is currently the gold-standard test to evaluate for brain injury in an ED setting and the one that was used in the current study, new data suggest that a rapid brain MRI may be a viable alternative.^24–26 By eliminating radiation exposure, a rapid brain MRI would change the risk-benefit ratio and the PIBIS score at which one might obtain neuroimaging.

The fact that close to 90% of the subjects with noncranial abuse had a PIBIS score of ≥2 is related to the dermatologic variable. Only dermatologic abnormalities consistent with trauma, and not atraumatic findings, such as rashes, were included in this assessment. Studies have demonstrated high rates of abnormal head CTs in infants with bruising,^13,27 which has led the American Academy of Pediatrics to recommend neuroimaging in nonmobile infants with noncranial abuse.²⁸ PIBIS is, therefore, consistent with current clinical recommendations.

The lack of predictive value of “a previous ED visit for a high-risk symptom” and “neurologic examination” was surprising given their predictive strength in the derivation. We hypothesize that the lack of predictive value of the previous ED visit relates to recall bias in the retrospective derivation. Physicians routinely ask about previous ED visits when evaluating an infant for suspected abuse, but may not ask about this history for all infants presenting with nonspecific complaints. The lack of predictive value of the neurologic examination likely relates to the transient and subtle nature of neurologic abnormalities, such as fussiness.

We recognize that there is controversy about whether to dichotomize continuous predictor variables.²⁹ Although accuracy is clearly important, ease of use is also critical for success of any CPR. For this reason, we dichotomized the predictor variables, similar to other studies.^16,30,31

Because not all subjects had neuroimaging, some cases may have been misclassified as controls. By tracking all subjects during the follow-up period, we sought to decrease the possibility of misclassification. Furthermore, both CHP and SLC, in which 90% of the subjects were enrolled, are the only regional level I trauma centers; all children with suspected AHT in these regions would be evaluated at the participating hospital.

We cannot assess whether the study itself affected physicians’ practice. In most cases, the treating physician obtained consent to allow investigators to approach families and was, therefore, aware that the study was designed to develop a CPR for identification of brain injury. This may have prompted the treating physician to consider brain injury in the differential diagnosis and, therefore, order a head CT.

A final limitation is this was a convenience sample. During the study period, other children with similar symptoms were evaluated in the study EDs but were not approached for enrollment. There is no reason to believe, or are there data to suggest, that infants who come to an ED with nonspecific symptoms at 2 am, for example, are different from infants who come at 4 pm as it relates to the variables in PIBIS or to the likelihood of having a brain abnormality.

Conclusions

In conclusion, our study suggests that PIBIS can identify infants at increased risk for brain injury who should undergo neuroimaging. As with all CPR, implementation analysis is essential before incorporating PIBIS into clinical practice to determine whether PIBIS improves identification of AHT and/or changes the use of neuroimaging to screen for brain injury in the ED setting.

Glossary

AHT

abusive head trauma

ALTE

apparent life-threatening event

AUC

area under the curve

CHG

Ann & Robert H. Lurie Children’s Hospital in Chicago, IL

CHP

Children’s Hospital of Pittsburgh of UPMC

CI

confidence interval

CPR

clinical prediction rule

CPT

Child Protection Team

CT

computed tomography

ED

emergency department

NPV

negative predictive value

PIBIS

Pittsburgh Infant Brain Injury Score

PPV

positive predictive value

ROC

receiver operator characteristic curve

SLC

Primary Children’s Hospital in Salt Lake City, Utah