Evidence for significant misdiagnosis of abusive head trauma in pediBIRN data

Abstract

Independently witnessed events are used as a reference standard to robustly categorize accidental or non-abusive head trauma (non-AHT) cases in the pediBIRN data set of acutely symptomatic infants with closed head injuries. Findings in such independently witnessed non-AHT cases are compared to findings in cases that were diagnosed as AHT but were not independently witnessed. The data shows that 14% of independently witnessed non-AHT cases are misdiagnosed as AHT, and that risk factors for misdiagnosis include acute encephalopathy, bilateral or interhemispheric SDH, and/or severe retinal hemorrhages, findings that are commonly associated with AHT. The data also shows that “dense retinal hemorrhages extending to the periphery” are not highly suggestive of AHT, as they also occur in independently witnessed non-AHT cases.

Highlights

• •When using independent witness events as a reference standard, pediBIRN data shows that:
• •Non Abusive Head Trauma (non-AHT) is commonly misdiagnosed as AHT.
• •Rates of misdiagnosis increase when acute encephalopathy, non-focal Subdural Hemorrhage (SDH), and/or severe retinal hemorrhage (RH) are present.
• •Severe, widespread RHs occur in accidental injury cases, and are not highly suggestive of AHT.
• •A robust relation is shown between severity of intracranial pathological conditions and rates of severe RHs.

1. Introduction

Much research attempting to validate the diagnoses of abusive head trauma (AHT) has been criticized for relying on circular reasoning [[1], [2], [3], [4]]. Such studies uses cases diagnosed as AHT [5] as a reference standard, with those diagnoses based on findings already associated with AHT, in order to determine findings associated with AHT (see Fig. 1A [6]). The results of such studies simply reflect the findings that were used in making the diagnosis, i.e., the findings widely believed to be associated with AHT.

Fig. 1.

A Circular Reasoning: findings such as retinal hemorrhages (RH) are used as diagnostic criteria to diagnose AHT, so RH will have high prevalence in cases diagnosed with AHT, and will be “validated” as a diagnostic criterion for AHT. B Judicial Admissions: findings such as RH are used as diagnostic criteria to diagnose AHT, so RH will have high prevalence in cases diagnosed with AHT who enter the judicial process, some of whom make admissions. The findings in judicial admission cases are determined by the criteria used to diagnose abuse. The circle is the same as in A, with an added step. C Spontaneous Admissions: admissions made independently of diagnosis can have any findings, which have not been pre-determined by assumed diagnostic criteria. Alongside independently witnessed AHT and non-AHT, clinical data from spontaneous admissions has clinical value.

To break from this fundamental flaw in methodology, cases where events were witnessed provide the promise of a “gold standard” for research in the field [7]. Although caveats remain around witnessed events, particularly the potential for motivated or biased witnesses and the issue of causation, unbiased independently witnessed events are the best reference standard for trying to understand which findings are associated with trauma, both abusive (AHT) and accidental (non-AHT).

This study analyses the Pediatric Brain Injury Research Network (pediBIRN) data set of acutely symptomatic patients under 3 years of age to examine and compare abusive head trauma (AHT) and non-AHT cases in infants. Independently witnessed events are used as a reference standard to understand how often accidents or non-AHT cases are misdiagnosed as AHT, and what are the risk factors for such misdiagnosis.

The paper is organized as follows: in Section 2, the pediBIRN data is presented, and reasons provided as to why admitted cases do not provide a robust reference standard, and why independently witnessed events provide a more reliable standard. In Section 3, cases of independently witnessed non-AHT are compared to cases that were diagnosed as AHT and the implications of applying the adopted reference standard to this prospective, uniform set of data are discussed. Conclusions are summarized in Section 5.

2. Materials and methods

2.1. PediBIRN data

The PediBIRN data [8] contains information on 973 acutely symptomatic patients with ages less than 3 years, all hospitalized for intensive care in one of 18 participating academic medical centers, between February 2011 and March 2021. All patients had closed head injuries that were confirmed by either CT or MRI. Patients with pre-existing brain abnormalities were excluded, as were those involved in motor vehicle collisions The pediBIRN collaboration did not release the raw data publicly, and turned down a request for the data. However, some data is publicly available from tables in the published articles, Hymel et al., 2022 [9] (Hymel22 hereafter) and Boos et al., 2022 [10] (Boos22 hereafter). Further, full data for 420 [11] of the 973 cases were released with an earlier publication [12]. These 420 cases come from 8 academic medical centers. Although further interesting analysis may be possible with the entire data set, the publicly available data within the tables of the published articles combined with the full data for 420 cases, allows for substantial analysis and to draw inferences and make conclusions.

For full details of the PediBIRN data, refer to Hymel22 and references therein. Crucial aspects of the data are summarized here:

Investigators were led through the same series of questions to attain uniform data.

• 1.Was the child's head injury event witnessed and described thoroughly by an unbiased, independent observer? Answer options were:

• (a)Yes, and was described by this observer as an ‘accidental’ or ‘non-abusive’ head injury event,
• (b)Yes, and was described by this observer as an ‘inflicted’ or ‘abusive’ head injury event, or
• (c)“No or unknown.”

If the investigators answered (c), they were also required to answer.

• 2.Was the person responsible for this child when he or she was head-injured—or first became clearly and persistently ill—asked to explain what happened?

If investigators responded Yes, they were then asked.

• 3.Which of the following statements best summarizes this caregiver's explanation for the child's head injuries and acute clinical presentation? Answer options included:
• (a)The caregiver described an ‘accidental’ or ‘non-abusive’ head injury event
• (b)The caregiver clearly admitted ‘inflicted’ or ‘abusive’ head trauma
• (c)The caregiver specifically denied that the child experienced any head trauma before he or she became symptomatic, or
• (d)The caregiver refused to explain what happened.

Based on these responses, the data was categorized as i) admitted or independently witnessed AHT, ii) independently witnessed non-AHT or iii) diagnosed AHT that was neither admitted nor independently witnessed. Investigators were also asked to provide the final consensus diagnosis of treating and consulting physicians. Answer options were definitive non-AHT, probable non-AHT, undetermined, probable AHT, and definitive AHT.

2.2. Reference standard: Unbiased, independently witnessed events

Unbiased, independently witnessed events are used as a reference standard for classification as non-AHT. Different studies have used different criteria for independent witnesses. One study [13] requires that independent witnesses to AHT be a non-family member such as a neighbor or a stranger in a public setting, while another study [14] includes family members as “independent” witnesses, although they do not claim those witnesses to be “unbiased”. In terms of independently witnessed non-AHT, one study [15] requires that the event was witnessed by at least two people, whilst another study [16] requires the event was witnessed by at least two people or one person who is not involved in the care of the child.

The pediBIRN data has 100 such cases of “unbiased independently witnessed” non-AHT. No detailed information is provided in the pediBIRN data regarding the meaning of “unbiased” and “independent”, or the identity of the witness, which is listed as a possible weakness of this study.

The full pediBIRN data set also has 9 cases of witnessed AHT. In Hymel22, the 9 cases of witnessed AHT are grouped with cases of admitted AHT so cannot be separately analyzed. In the 420 publicly available cases there are only 3 cases of independently witnessed AHT, so not enough from which to draw inferences. However, the k-means clustering analysis from Boos22 shows that the findings the 9 cases of witnessed AHT in the full data set cluster closely with the findings of the cases of witnessed non-AHT. In other words, for the pediBIRN data set, findings are similar in all independently witnessed trauma cases, whether AHT or non-AHT.

Nevertheless, due to the low numbers of witnessed AHT cases and their mixing with admitted cases in Hymel22, it is made clear throughout this article that the analysis is made on cases of witnessed non-AHT, and it is from those cases that conclusions are drawn.

2.3. Admitted AHT cases in pediBIRN data

The pediBIRN data also provides findings for admitted AHT cases. However, the data does not include the circumstances in which the admissions were made. To their credit, Hymel22 cite this as a limitation of their study. This methodological limitation risks biasing the results and injecting circular reasoning into the analysis [17].

Psychologists have studied [18] a range of circumstances in which admissions and confessions are made, providing information on their reliability. In the context of AHT studies, an important distinction can be made between admissions made spontaneously, prior to medical conclusions and prior to accusations of abuse/shaking being made, and those made during the judicial process, so after medical examination, and after accusations have been made [19]. These will be referred to as spontaneous and judicial admissions respectively1:

Spontaneous admissions to abuse were made spontaneously, independent of the assessment of a physician. Assessment by a physician comes after the admission. In spontaneous admissions, the traumatic abuse (and the mechanism, possibly including shaking) is suggested by the admitter.

Judicial admissions to abuse were made after assessment by a physician, after accusations are made, during the investigative and/or judicial process. In judicial admissions, the traumatic abuse (and often the mechanism) is suggested by the doctor and/or interrogator, after medical findings have been determined.

The types of injuries that can be associated with cases of spontaneous admissions is unlimited, meaning a data set of spontaneous admissions could be used to determine which findings are typically caused by AHT, and in particular by shaking. By contrast, the types of injuries associated with cases of judicial admissions is determined by the physician, based on findings that are believed to be diagnostic of AHT. The findings in a data set containing judicial admissions are thus already determined based on the findings believed to be diagnostic of AHT, and have high risk of circular reasoning (see Fig. 1B vs 1C).

2.3.1. The data rules out a significant fraction of spontaneous admissions

It is acknowledged that “few witnessed shaken infants have signs and symptoms of AHT” [20]. The majority of witnessed shaking events lead to no neurological findings [21]. If all people who shook an infant and took them to the hospital to be assessed, most would not show signs and symptoms of AHT. Therefore, if there were a statistically significant number of spontaneous admissions in admitted/witnessed AHT group of pediBIRN data, there would be a significant number of cases without neurological findings. These cases without neurological findings would mean that the set of cases with spontaneous admissions must be statistically different to the set of cases diagnosed AHT by physicians.

On the other hand, the cases with judicial admissions must have the signs and symptoms associated with AHT by physicians, because the first step is the identification of such findings. Therefore, there must be no statistical difference between findings in this group, and the AHT cases that were diagnosed by physicians. The fact that there is not a statistical difference in these two groups in the pediBIRN data (see Table 3 of Hymel22) means that one can infer that the admitted/witnessed AHT cases in the pediBIRN must be dominated by judicial admissions.

If the data is dominated by judicial admissions (as it seems) there is a risk of circular reasoning because the cases in both cohorts, those admitted and those diagnosed, were selected based on having the medical findings that the physicians use to diagnose AHT. The diagnostic test is the first step of the reference test [22]. Therefore, the similarity between admitted AHT and diagnosed AHT in the pediBIRN data, as found by Hymel22, is an inevitable outcome of the methodology employed. This problem of circularity when using judicial admissions is shown in panel B of Fig. 1.

2.3.2. Are all admissions in pediBIRN data true confessions to violent shaking?

The pediBIRN data may include admissions that were incentivized by plea bargain, the promise or hope of a lighter sentence, and/or the promise or hope of returning children to the home and care of the partner of the person who confessed. A lack of information makes it very difficult to assess how many admissions in the pediBIRN were incentivized, and how many risk factors and red flags for false confession are present in each case. Further still, it is not clear that all admissions were to violent shaking, or whether some admissions were to lesser acts.

It is for these reasons that researchers who attempt to use admissions in AHT studies have been repeatedly urged to provide information on the circumstances in which admissions are made [23,24]. Without information such as what accusations were being made, what was admitted, who first suggested the shaking mechanism, whether child custody proceedings were running concurrently, whether offers of charges for lesser crimes or lower sentences were made, there remains a considerable risk of bias and circularity in the data when admitted cases are used as a reference standard.

Given all these factors presented in this section (2.3), including risk of circular reasoning and risk of incentivized, pragmatic, or coerced false admissions, judicial admissions do not make a reliable scientific reference standard for understanding the findings associated with AHT. Thus, admitted cases are not used as a reference standard in this study.

2.4. Statistical testing

Comparisons are made between all cases and independently witnessed non-AHT cases, and then between independently witnessed non-AHT that were diagnosed AHT cases and cases diagnosed as AHT with no independent witness. The chi2_contingency from the python scipy.stats module is used to calculate p values using the chi-squared test. p values were recalculated using the Fisher exact test for Table 2 using fisher_exact from the same module. p-values <0.05 are considered “significant” but are provided in each case for clarity of the degree of significance.

Table 2.

Findings in cases of witnessed non-AHT that were diagnosed as AHT (n=14, of which 12 had ophthalmologic exams), compared to cases diagnosed as AHT that were not witnessed. p values and odds ratios (with 95% confidence interval) are shown. p values using the Fisher exact test are shown in brackets.

	Witnessed non-AHT diagnosed as AHT	Diagnosed AHT	p value	Odds ratio
Acute respiratory compromise	10/14 (71%)	270/438 (62%)	0.46 (0.58)	1.2 (0.4–3.8)
Acute circulatory compromise	8/14 (57%)	170/438 (39%)	0.17 (0.18)	1.5 (0.5–4.3)
Seizure	7/14 (50%)	228/428 (52%)	0.88 (1.0)	1.0 (0.3–2.8)
Acute encephalopathy >24 h	8/14 (57%)	211/438 (48%)	0.51 (0.59)	1.2 (0.4–3.5)
Bilateral and/or interhemispheric SDH	10/14 (71%)	318/438 (73%)	0.92 (1.0)	1.0 (0.3–3.2)
Bilateral and/or subcortical hypoxia, ischemia, or swelling	7/14 (50%)	220/438 (50%)	0.99 (1.0)	1.0 (0.3–2.9)
Extracranial fracture(s) moderately or highly specific for abuse	2/14 (14%)	143/410 (35%)	0.11 (0.15)	0.4 (0.1–1.9)
Retinoschisis	3/12 (25%)	48/425 (11%)	0.15 (0.15)	2.2 (0.6–8.5)
Severe Retinal Hemorrhage	6/12 (50%)	257/425 (60%)	0.47 (0.55)	0.8 (0.3–2.6)

3. Results and discussion

3.1. Findings in cases of witnessed non-AHT

In Table 1 all cases are compared with cases of witnessed non-AHT. For all cases, the 420 case data set is used, of which 313 had ophthalmologic exams. Cases of witnessed non-AHT (n=100 of which 38 had ophthalmologic exams), come from the full data set of 937 cases. The p values and odds ratios (with 95% confidence interval) show the comparison between findings in these two categories.

Table 1.

The first column shows findings from all cases from the 420 case data set (of which 313 had ophthalmologic exams). The second column shows the finings for cases of witnessed non-AHT (n=100 of which 38 had ophthalmologic exams), coming from the full data set of 937 cases. The p values and odds ratios (with 95% confidence interval) for these two categories are shown.

	All cases	Witnessed non-AHT	p value	Odds ratio
Acute respiratory compromise	180/420 (43%)	31/100 (31%)	0.03	1.4 (0.9–2.2)
Acute circulatory compromise	147/420 (35%)	19/100 (19%)	0.002	1.8 (1.1–3.2)
Seizure	139/420 (33%)	16/100 (16%)	<0.001	2.2 (1.2–3.9)
Acute encephalopathya	223/420 (53%)	47/100 (47%)	0.273	1.1 (0.7–1.7)
Acute encephalopathy >24 hb	110/420 (26%)	16/100 (16%)	0.033	1.6 (0.9–2.9)
Subdural Hemorrhage (SDH)	303/420 (72%)	52/100 (52%)	<0.001	1.4 (0.9–2.2)
Unilateral SDH	103/420 (25%)	27/100 (27%)	0.607	0.9 (0.6–1.5)
Bilateral SDH	167/420 (40%)	20/100 (20%)	<0.001	2.0 (1.2–3.4)
Hypoxia, ischemia, and/or brain swelling	142/420 (34%)	22/100 (22%)	0.022	1.5 (0.9–2.6)
Any skull Fracture	201/420 (48%)	74/100 (74%)	<0.001	0.6 (0.4–1.1)
Retinoschisis	27/313 (9%)	3/38 (8%)	0.879	1.1 (0.3–3.8)
Severe Retinal Hemorrhagec	133/313 (43%)	7/38 (18%)	0.004	2.3 (1.0–5.4)

^aAcute encephalopathy: alteration or loss of consciousness at the scene of injury, during transport, in the Emergency Room, or prior to hospital admission.

^bAcute encephalopathy >24: as above where loss of consciousness lasts greater than 24 h.

^cRetinal Hemorrhage(s): described by an ophthalmologist as “dense, extensive, covering a large surface area, and/or extending to the ora serrata.“

The two categories of cases shown in Table 1 show significant differences in a range of findings. Notably, witnessed non-AHT cases have higher rates of skull fractures (p=<0.001). This may be expected, as witnessed non-AHT cases are generally accidents such as falls that result in blunt force trauma, whilst all cases may include non-traumatic causes such as bleeding disorders that would be significantly less likely to result in skull fractures. The higher rates of skull fractures therefore provide a degree of corroborating evidence for the veracity of the eyewitness accounts.

Table 1 shows that witnessed non-AHT cases have significantly lower rates of hypoxia, ischemia and/or brain swelling (p=0.022), and also have significantly lower rates of findings that are often associated with AHT, including seizure (p < 0.001), bilateral SDH (p < 0.001), acute encephalopathy that lasts 24 h or more (p=0.033) and severe RH (p=0.004). Thus, these findings commonly associated with AHT are not strongly associated with blunt force trauma.

Nevertheless, witnessed non-AHT cases do have such findings, just at lower rates. A natural interpretation is that blunt force trauma is a cause of intra cranial findings commonly associated with AHT, but it is not the most common cause of such findings.

3.2. Severe RHs are common

Of the 313 cases that had eye exams, 133 (43%) had severe RHs, described by an ophthalmologist as “dense, extensive, covering a large surface area, and/or extending to the ora serrata”. The rate of severe RHs increases to 71% when there is hypoxia, ischaema and/or brain swelling, and to 77% when such hypoxia is bilateral and/or involving the subcortical brain. Similarly, the rate of severe RHs increases to 58% for cases with acute encephalopathy, and 73% for cases where the encephalopathy lasts >24 h. This correlation between degree of intracranial pathological conditions and the degree of RH is now well established [25] in the literature and is supported by the pediBIRN data.

3.3. Accidental trauma misdiagnosed as AHT

When using witnessed events as a reference standard, the pediBIRN data shows that 14/100 witnessed non-AHT cases have been diagnosed as AHT. So, simply following from the adopted reference standard, the data shows that 14% of accidents are misdiagnosed as AHT.

Table 2 uses data from the full 937 cases and shows the rates of various findings in cases of witnessed non-AHT that were diagnosed as AHT (n=14, of which 12 had ophthalmologic exams), compared to cases diagnosed AHT, i.e., those diagnosed by physicians that were neither witnessed nor admitted (n=438 of which 425 had ophthalmologic exams). Also shown are the p values and odds ratios between these categories of cases.

The p values in Table 2 show that there are no significant statistical differences in the findings between cases of witnessed non-AHT diagnosed as AHT and cases that were diagnosed AHT but where there was no independent witness.

As 14 is a relatively small number of cases to do statistical analysis, results are also checked using the Fisher exact test, shown in brackets in Table 2. No qualitative difference was found between the Fisher exact and chi-squared tests. Whilst each individual finding has a relatively large 95% confidence interval, the association between findings in the two categories is systematic. That is, the findings commonly associated with AHT all show no significant difference between the two categories, and all show odds ratios that are close to 1. This systematic trend greatly decreases the likelihood that the association is based on chance, and greatly increases confidence in the result.

Having similar findings in cases diagnosed AHT and cases of witnessed non-AHT that were diagnosed as AHT is not surprising, because both groups rely on diagnosis by the same physicians, who make the diagnosis by reference to findings that are commonly associated with AHT. The interesting part is that the data shows that a subset of witnessed non-AHT cases can cause findings that physicians associate with AHT, and that when this occurs, the physicians tend to misdiagnose the cases as AHT, even though there is an independent unbiased witness attesting that it was non-AHT.

3.4. Risk factors for misdiagnosis

Applying the adopted reference standard, the data shows that.

• •15 cases of witnessed non-AHT had bilateral and/or subcortical hypoxia, ischemia, and/or swelling. Of those, 7 (44%) were misdiagnosed as AHT.
• •17 cases of witnessed non-AHT had acute encephalopathy lasting >24 h. Of these, 8 (47%) were misdiagnosed as AHT.
• •25 cases of witnessed non-AHT had bilateral and/or interhemispheric SDH. Of these, 10 (40%) were misdiagnosed as AHT.
• •7 cases of witnessed non-AHT had severe RH. Of these 6 (86%) were misdiagnosed as AHT.
• •All 3 cases (100%) of witnessed non-AHT with retinoschisis were misdiagnosed as AHT.

In summary, high risk factors for misdiagnosis of AHT are hypoxia, ischemia, and/or swelling, acute encephalopathy, bilateral and/or interhemispheric SDH, and severe RH.

3.5. Severe retinal hemorrhages occur in non-AHT

Severe RHs are significantly less common in witnessed non-AHT than in the total population of cases, but they do occur. Within the witnessed non-AHT cases, 38 underwent retinal exams. Of those, 7 (18%) were found to have RH that were severe. This rate is even more striking when one considers the [26] correlation between degree of intracranial pathological conditions and the degree of RH. Using the 420 cases, 11 cases of witnessed non-AHT that had acute encephalopathy, of which 4 (36%) had severe RH. Of the 5 witnessed non-AHT cases with acute encephalopathy lasting more than 24 h, 3 (60%) had severe RH. Of the 6 cases that had hypoxia, ischaema and/or swelling, 3 (50%) had severe RHs.

Although these non-AHT case numbers are small, it seems that the relation between the extent of intracranial pathological conditions holds for cases of independently witnessed non-AHT and is not specific to AHT. Indeed, the relation holds in pediBIRN data if one selects only cases where the accused person denied any trauma had occurred, it holds when selecting only cases where the accused makes admissions during the judicial process, it holds for cases where the doctor's final diagnosis was AHT, and it holds for cases where the doctors' final diagnosis was non-AHT. It appears to be a very robust relation.

3.6. Strengths and weaknesses

A strength of our study is that the pediBIRN data is large, prospective, and uniform and that the methodology does not rely on circular reasoning. Limitations include the fact that the 937 case data set is not fully available publicly, which would allow better statistics in the instances where 420 of the 937 cases are used; a lack of information surrounding the witnesses and whether they are truly unbiased and independent; the issue of causation which will always be a caveat when using witnessed or admitted cases as a reference standard.

4. Conclusions

Given the inability to conduct randomized control trials, unbiased, independently witnessed events are the best reference standard available for classifying cases as being abusive verses non abusive head trauma and avoiding circular reasoning.

When using unbiased independently witnessed events as a reference standard, pediBIRN data shows that misdiagnosing accidents as AHT is common, particularly when acute encephalopathy, bilateral or interhemispheric SDH, and/or severe RH, are present. As intracranial pathological conditions become more severe, the rate of misdiagnosis of accidents as AHT rises rapidly.

Misdiagnoses of non-AHT cases occur despite the statistical differences in findings of witnessed non-AHT cases compared to diagnosed AHT cases. Table 1, Table 2 demonstrate that there is nevertheless an overlap between the findings in non-AHT cases and the findings used by physicians to diagnose abuse. When such an overlap exists, there is a significant risk of misdiagnosis.

Misdiagnosed of undetected or unknown medical conditions, or evidence for other causes of findings associated with AHT such as the hypoxic cascade hypothesis put forward by Geddes [27], would imply additional misdiagnoses on top of misdiagnosed non-AHT.

The data also shows that retinal hemorrhages that are “dense, extensive, covering a large surface area, and/or extending to the ora serrata.” are common and occur relatively commonly in non-AHT. This supports a growing body of research [[28], [29], [30], [31], [32], [33], [34]] that has shown that severe retinal hemorrhages in infants can have a range of causes. The known relation between severity of intracranial pathological conditions and rates of severe retinal hemorrhages is also found in the pediBIRN data and seems to be a very robust relation.

Declarations of competing interest

None.