Retinal Hemorrhage in Abusive Head Trauma: Finding a Common Language

Abstract

Purpose:

To assess the performance of a refined Web-based tool for documenting retinal hemorrhage characteristics in suspected abusive head trauma.

Methods:

Using a comprehensive tabular secure platform, with access to digital images in color, black and white, and 4-zone system schematic overlay, four pediatric ophthalmologists performed pilot testing with 80 images for tool refinement. In a second phase, retinal hemorrhages were documented by number, zone, and type. Interobserver agreement was calculated using the Fleiss kappa coefficient. Intraobserver agreement was calculated using Cohen’s kappa statistic. We used surface area mapping software for further analysis.

Results:

Interobserver agreement was good (kappa 0.4–0.6) and very good (kappa 0.6–0.8) for all questions in Zone A (peripapillary). For zones C (midperiphery) and D (peripheral retina), agreement was very good for all questions except number of hemorrhages, for which agreement was good. Zone B (macula) showed good and fair agreement except for superficial hemorrhage, for which agreement was poor. There was very good intraobserver agreement for number (kappa 0.68, 0.65, 0.67) and type of hemorrhages in zones A, B, and C. Surface area mapping results revealed no significant differences between zones A and B. Zones C and D had significantly less hemorrhage than A and B.

Conclusions:

Our tool performed with good or very good interobserver and intraobserver agreement in almost all domains. We attribute zone B underperformance to the significant increased area covered by hemorrhages compared to zones C and D and the lack of contrast with normal anatomical structures in zone A.

INTRODUCTION

Retinal hemorrhages are a cardinal manifestation of abusive head trauma characterized by repeated acceleration-deceleration with or without blunt head impact (shaken baby syndrome, SBS¹) occurring in approximately 85% of victims.^2,3 Research involving surviving and deceased victims, animal models, mechanical dummies, and finite element analysis have demonstrated the important role of vitreoretinal traction in the pathophysiology of these hemorrhages.⁴ In addition to the distinctive traumatic macular retinoschisis as a result of such traction, first described by Greenwald and associates in 1986,⁵ there is a predilection for hemorrhages to occur at other areas where the vitreous is most adherent to the retina, including the vitreous base.³ Many investigators have demonstrated the high specificity and sensitivity of specific patterns of retinal hemorrhage for SBS as opposed to accidental head trauma, in particular too-numerous-to-count multilayered hemorrhages extending to the ora seratta.^6–10

The ophthalmologist thus plays a critical role in the documentation of retinal findings in cases where SBS is suspected.¹¹ The recent study by Chhabra and coworkers¹² shows that there was a good agreement between pediatric ophthalmologists identifying severity of retinal hemorrhages based on images of SBS retinopathy. Likewise, postmortem documentation by both gross and histologic methods plays an important role in the diagnosis and medicolegal adjudication of these cases.¹³ Yet, there remains no universally accepted standard for recording the retinal findings. The generic term retinal hemorrhage, although commonly used in clinical notes and the medical literature, is clearly insufficient, and the value of more exacting terminology is supported by the studies that demonstrate the diagnostic significance of specific types, numbers, and distribution patterns of hemorrhages.^6–10 Many investigators have used their own, unvalidated, unique systems to grade or describe hemorrhages for the purpose of single studies in a wide variety of conditions associated with retinal hemorrhage.^14–29

Four studies have specifically set out to evaluate a system of retinal hemorrhage documentation, all of which were primarily based on retinal photography. Using a new three-zone classification system, Fleck and coworkers³⁰ defined peripapillary, posterior pole, and peripheral retinal zones and then had four examiners (2 pediatric ophthalmologists, 1 pediatric ophthalmology fellow, and 1 pediatric neurologist) review 31 hard-copy digital photographs from abused children and those with retinal hemorrhage from other causes. Photographs had a variety of hemorrhage type and number. No provisions were made for the identification of macular retinoschisis or perimacular retinal folds. The examiners used a clear acetate overlay to define the zones and were asked only to indicate the zone of 142 specific nominated hemorrhages. Only photographs centered on the posterior pole using the 130° lens of the RetCam system (Clarity Medical Systems, Pleasanton, California) were used. Using Cohen’s unweighted kappa score, the investigators found excellent interobserver and intraobserver agreement, but a disagreement between observers on zone location of specific hemorrhages, ranging from 5.63% to 8.45%.

The same group published a follow-up study, which differed from the initial study only in the allowance for the examiners to review the photos on a computer monitor as well as hard copies, the request that they identify the type of the same nominated hemorrhages (vitreous, preretinal/subhyaloid, subinternal limiting membrane, superficial intraretinal [flame], deeper intraretinal [dot/blot], subretinal, retinoschisis cavity, or indeterminate), and the ability to also view a black-and-white version of the image.³¹ The examiners then conferred on the image classifications and narrowed the choice to five types (vitreous, preretinal, nerve fiber layer, intraretinal/subretinal, or indeterminate) and the assessment was conducted again. Using the initial eight categories, only fair interobserver agreement was found. Only slight improvement was noted with the revised hemorrhage-type classifications.

Another group studied a two-zone system, which was created by collapsing zones 2 and 3 of the International Classification of Retinopathy of Prematurity³² into one zone along with the Zone 1 of the same system.³³ Seven pediatric ophthalmologists were asked to indicate many features of the hemorrhages, including number, location, and type, on 105 images from 21 eyes of 21 victims of abusive head injury captured using the 130° lens of the RetCam system and viewed on a laptop and by a digital projector. For each case, 5 images over 4 minutes were viewed, including primary position and each of the four fundus quadrants. One month later, 6 of the initial 7 examiners, one additional pediatric ophthalmologist, and 7 residents repeated the exercise. In general, they found excellent or very good interobserver agreement for presence, number, and location of hemorrhages; very good agreement for hemorrhage size and morphology; and fair to good agreement for hemorrhage layer and other findings. There was a wide range of intraobserver agreement, ranging from poor to almost perfect.

In 2011, two of the current authors published our Retinal Hemorrhage Assessment Tool and evaluated its performance using four examiners and 80 photographs.³⁴ The four zones were constructed based on the known features of retinal hemorrhage in SBS, such that areas with different anatomy relative to vitreoretinal adhesions were isolated. The color photographs were viewed on a Web-based platform and examiners were asked to answer 69 multiple-choice questions for each image regarding hemorrhage number, location, and type. Multiple views of the retina were used, and images were selected randomly to represent a wide variety and both RetCam and Kowa (Kowa Company Ltd, Tokyo, Japan) systems. We found good or very good interobserver agreement in the peripapillary zone but only fair agreement for all other zones with regard to number of hemorrhages. Intraobserver agreement was excellent only for number of hemorrhages in the two most posterior zones.

Disappointed with the results of our initial study, and noting the limitations (ie, including too few zones, including zones that did not respect the pathophysiologic anatomy of SBS retinopathy, assessing only nominated specific hemorrhages) and potential strengths of the other tools, we set out to combine the methods previously reported with our own in hopes of developing an improved system for retinal hemorrhage documentation. We enhanced our tool by adding options to use an overlay zone template, black-and-white images, and a streamlined questionnaire. We then repeated our study with the same photographs and four examiners. Herein we report the results of this trial.

We hypothesize that the current refined tool, using lessons learned from other investigators and our prior study, and respecting the anatomy and pathophysiology of the hemorrhagic retinopathy of SBS, will perform at a higher level than we previously reported. In doing so, we believe this tool may offer a common international language for clinical, educational, and research communication with regard to this important ophthalmic manifestation of child abuse.

METHODS

This study was granted exempt status from the Wills Eye Institute Institutional Review Board. The Web-based questionnaire consisted of three elements: images, retinal hemorrhage description tool, and questionnaire. We used the same 80 de-identified retina images as used in our previous study.³⁴ The photographs had been taken with either the RetCam or Kowa retinal camera and included a broad range of retinal hemorrhage pathology, mainly from cases of abusive head injury but also some generated by accidental injury or nontraumatic retinal causes. The retinal images were selected to ensure broad representation of hemorrhage type, pattern of distribution, and number. The images were uploaded without identifying information and made available for reviewers in three formats: (1) the original color unaltered image; (2) the black-and-white version of the same image; and (3) the original image plus an overlay of the retinal hemorrhage description tool and zones preplaced according to anatomical landmarks. The respective zone letter was indicated within the zone. On photographs where the anatomical landmarks were not clearly visible, the zones were attributed to be the best possible estimate. Examiners were able to freely “toggle” between the three formats for each image.

The overlay placed on the retinal images divides the retina into four circular areas labeled as aones A through D (Figure 1). The division is based on easily identifiable anatomical landmarks and the known hemorrhage patterns in SBS retinopathy,^2,3 which roughly correspond to the anatomy of the vitreoretinal interface in children.⁴ Zone A (yellow circle) is a peripapillary circle centered on the middle of the optic disc with a diameter three times that of the optic disc. Zone B (green circle) is the posterior pole circle, the center of which lies on a line bisecting the retinal arcades and located approximately on the fovea. The nasal edge of Zone B is tangential to the nasal edge of zone A. Zone C (black circle) is the midperipheral circle ,the edges of which run approximately through the vortex veins (yellow dots). Zone D consists of the peripheral retina beyond zone C. Photographs were selected to show a variety of zone combinations (ie, just zone A, zones A through D, just zones C and D). These written definitions were available to the examiner at all times while assessing images.

FIGURE 1.

Schematic of retinal hemorrhage assessment tool. Retina is divided into four zones: A (yellow circle), B (green circle), C (black circle), and D (peripheral retina beyond zone C). The nasal edge of zone B is tangential to the nasal edge of zone A. The edges of the midperipheral zone C run approximately through the vortex veins (yellow dots). The x represents the fovea.

The images and the questionnaire were posted on a secure Web page managed by Syync LLC (Philadelphia, Pennsylvania). Every examiner was given a user name, a password, and a designated week to review the 80 images. Examiners could pause at any time, return at another time or on another day, and if desired, revise prior answers. The study consisted of two phases: the first phase was used to evaluate the functionality of the Web page and the second phase to evaluate the performance of the Tool. All examiners were experienced pediatric ophthalmologists with a special interest in abusive head trauma who have peer-reviewed publications about the ocular manifestations of SBS (see “Acknowledgments”). Three of the same examiners from the previous study were used for the first phase of this study, along with a new examiner. As one of the prior examiners was also a coauthor of two of the previous studies, we mutually agreed that a conflict of interest may exist, and therefore he was replaced with another examiner. For the second phase of the study, we used three of the same reviewers from phase 1 plus a new reviewer on account of the unavailability of one examiner from phase 1.

We created a tabular format from our prior³⁴ questionnaire (Figure 2). Section 1 asked the number of hemorrhages present in each zone, section 2 the type of hemorrhages present in each zone, and section 3 whether or not papilledema was present. For each question, examiners had the option of replying “cannot determine.” At the end of phase 1 we evaluated the examiner answers to detect problems with user interface and data entry. We found that some answers in section 2 were inconsistent with those in section 1. For example, examiners may have selected “no hemorrhages” for a zone in section 1 and yet offered answers in section 2 indicating hemorrhage type in that same zone. We therefore introduced a different format—forced-choice and mutually exclusive selections and automatic inactivation of choices—to disallow inconsistent entries (Figure 3). For example, if an examiner selected “no hemorrhages” in a zone in Section 1, the Web page disabled the respective zone in section 2 (type of hemorrhage). For phase 2, the examiners were given 2 days to again review the 80 images using the new format.

FIGURE 2.

Phase 1 questionnaire. Four pediatric ophthalmologists used this tabular format for the 80 images, and based on their feedback, the questionnaire was then refined (see Figure 3).

FIGURE 3.

Phase 2 questionnaire. Screenshot showing the standard final Web page, including zone schematic and an image under review using the overlay option. The tabular format questionnaire reflects alterations based on the feedback of the examiners in phase 1 (see Figure 2). This new format, with forced-choice, mutually exclusive selections and automatic inactivation of choices, prevents inconsistent entries..

As shown in Figure 3, for both phases 1 and 2, while assessing any image, examiners always had a full view, which included the zone schematic, the photograph being assessed (and the option to toggle to alternative formats: color, with schematic, or black and white), and complete questionnaire.

Examiners were instructed to provide an answer for all visible zones even if the image showed only a partial area of one or more zones. The number of hemorrhages referred to only the number of hemorrhages visible in a partial zone. Reviewers were instructed to choose “cannot determine” only if they could not see any part of a zone at all in the image or if the quality of the photograph limited the ability to respond accurately. Hemorrhages in a more posterior zone are not included in that of the more peripheral zone. Hemorrhages that covered the border of two were counted only once in the more posterior zone. After each examiner completed phase 2, we reviewed the data to identify questions that were not answered, in which case an email with the respective question and image was send to each reviewer asking him or her to complete the data for that image and question.

For the statistical analysis, interobserver agreement was assessed using the Fleiss multi-rater kappa statistic. For all questions, “cannot determine” was analyzed as an acceptable option so that the kappa measures the level of agreement among the raters with respect to the options in section 1 for the number of hemorrhages (none, <10, 10–19, >20, cannot determine) in each zone, the options in section 2 (yes, no, cannot determine) for the type of hemorrhages in each zone, and the options for the single question in section 3.

Intraobserver agreement on each question was assessed for each rater using Cohen’s kappa statistic.³⁵ An overall estimate of intraobserver agreement was also calculated for each question. For some questions (eg, vitreous hemorrhage in zones A and B, and subretinal hemorrhage in zone A), some of the raters selected the same response for most images. In these cases, kappa may not be able to be calculated or may underestimate the level of agreement. We therefore report the percentage agreement (the number of photos where the rater gave the same rating both times divided by the total number of photos rated) for each question and rater and for each question across all raters. We defined a kappa of 0.81 to 0.99 as excellent, 0.61 to 0.80 as very good, 0.41 to 0.60 as good agreement, 0.21 to 0.40 as fair, and 0.01 to 0.20 as poor agreement. Calculations were performed using the %inter_rater macro in SAS 9.3 (http://www.ccitonline.org/jking/homepage/interrater.html, accessed December 7, 2012).

We explored the possible relationship between amount of hemorrhages in a zone and performance using the hemorrhage assessment system (HAS) developed by Aslam and coworkers.³⁶ Each of the 80 images was reviewed by two of the authors (J.A.C., A.V.L.) to identify by consensus retinal hemorrhages of any size in each zone, which were then circled by freehand drawing within the computer software. The HAS software then calculates the percentage of surface area visible within a zone on the image that is subtended by hemorrhage within that zone. We used the images with overlay to facilitate the analysis of each zone. For statistical analysis, group differences in means of fractions of retinal hemorrhage areas for each zone (A, B, C, and D) were assessed using repeated-measures analysis of variance. Covariance structure was autoregressive. Multiple comparisons adjustments were done using the methods of Tukey. Data was square-root transformed for normality.

RESULTS

Kappa values are presented in Table 1 and Table 2 for each question, along with 95% confidence intervals. Agreement was good for all questions in zone A except for preretinal hemorrhages, for which the agreement was very good. For zones C and D, agreement was very good for all questions except number of hemorrhages, for which the agreement was good. Zone B showed good agreement for retinoschisis and preretinal hemorrhage, but only fair agreement in other fields except for superficial hemorrhage, for which the agreement was poor. The kappa for papilledema assessment was good (0.593, CI 0.509, 0.676).

TABLE 1.

KAPPA STATISTICS FOR INTEROBSERVER AGREEMENT FOR THE RETINAL HEMORRHAGE TOOL

CATEGORY	KAPPA (95% CI)
	ZONE A	ZONE B	ZONE C	ZONE D
Number of hemorrhages	0.540 (0.459, 0.622)	0.375 (0.309, 0.441)	0.595 (0.534, 0.655)	0.593 (0.317, 0.868)
Retinoschisis	0.581 (0.440, 0.721)	0.539 (0.396, 0.683)	0.761 (0.658, 0.863)	0.690 (0.420, 0.959)
Superficial intraretinal hemorrhages	0.495 (0.344, 0.646)	0.167 (0.026, 0.307)	0.761 (0.658, 0.863)	0.690 (0.420, 0.959)
Deeper intraretinal hemorrhages	0.495 (0.344, 0.646)	0.377 (0.179, 0.575)	0.761 (0.658, 0.863)	0.690 (0.420, 0.959)
Pre-retinal/subhyaloid hemorrhage	0.610 (0.521, 0.699)	0.595 (0.504, 0.686)	0.663 (0.588, 0.739)	0.628 (0.352, 0.904)
Vitreous hemorrhage	0.481 (0.273, 0.688)	0.276 (−0.01, 0.563)	0.746 (0.640, 0.852)	0.690 (0.420, 0.959)
Subretinal hemorrhage	0.570 (0.337, 0.803)	0.309 (0.041, 0.578)	0.740 (0.635, 0.845)	0.681 (0.411, 0.951)

CI, confidence interval.

TABLE 2.

KAPPA STATISTICS FOR INTRAOBSERVER AGREEMENT FOR THE RETINAL HEMORRHAGE TOOL^*

CATEGORY	KAPPA (95% CI)[% AGREEMENT]
	ZONE A	ZONE B	ZONE C
Number of hemorrhages	0.68 (0.47, 0.89) [82.5]	0.65 (0.47, 0.83) [75.0]	0.67 (0.49, 0.86) [82.5]
Retinoschisis	0.50 (0.25, 0.75) [77.5]	0.65 (0.42, 0.87) [82.5]	0.70 (0.44, 0.96) [90.0]
Superficial intraretinal hemorrhages	0.53 (0.30, 0.77) [80.0]	0.67 (0.44, 0.90) [85.0]	0.71 (0.50, 0.91) [85.0]
Deeper intraretinal hemorrhages	0.39 (0.20, 0.58) [65.0]	0.75 (0.49, 1.01) [92.5]	0.77 (0.58, 0.96) [90.0]
Pre-retinal/subhyaloid hemorrhage	0.53 (0.17, 0.88) [82.5]	0.65 (0.40, 0.90) [87.5]	0.73 (0.57, 0.89) [80.0]
Vitreous hemorrhage	−0.05 (−0.20, 0.10) [87.5]	0.03 (−0.12, 0.17) [92.5]	0.72 (0.44, 0.99) [92.5]
Subretinal hemorrhage	0.03 (−0.15, 0.21) [72.5]	0.77 (0.55, 0.98) [90.0]	0.59 (0.40, 0.78) [75.0]

CI, confidence interval.

^*Zone D was not tested for intrarater agreement as none of the selected images for this analysis demonstrated zone D.

There was very good intraobserver agreement for number of hemorrhages in zones A, B, and C (kappa 0.68, 0.65, 0.67). For types of hemorrhages in zone A, kappas generally indicated good agreement (range 0.39–0.53) with the exception of vitreous and subretinal hemorrhage, where most photos were rated “no.” For these questions, kappa was near 0, though agreement was fairly high (87.5% and 72.5% on average). Zone A deeper intraretinal hemorrhages had the lowest level of intraobserver agreement (65%). For types of hemorrhages in zone B, kappas indicated very good agreement (range 0.65–0.77). Agreement was high for vitreous hemorrhage (92.5%). Types of hemorrhages in zone C also demonstrated very good agreement (kappa 0.59–0.77). Agreement on assessment of papilledema was also very good (0.81). Zone D was not tested for intrarater agreement, as none of the selected repeated images demonstrated zone D.

Each image may contain all or part of zones A, B, C, and/or D. Data were analyzed for 223 zones from the 80 images (Table 3). The main effect among zones was statistically significant (P < 0.001). There was no statistically significant difference in the amount of hemorrhage between zones A and B (P = 1.0). Zones C and D had statistically significantly less hemorrhage than both zones A and B (P ≤ 0.007). Zone C had statistically significantly more hemorrhage than zone D (P = 0.018).

TABLE 3.

DIFFERENCE IN MEAN HEMORRHAGE AREA^*

ZONE (NUMBER OF AREA ZONES)†	MEAN HEMORRHAGE AREA (95% CI)	Zone B	DIFFERENCE (95% CI) [P VALUE] Zone C	Zone D
A (75)	0.510 (0.449, 0.571)	−0.001 (−0.053,0.051) [1.0]	0.120 (0.048, 0.192) [0.007]	0.294 (0.166, 0.421) [<0.001]
B (79)	0.511 (0.451, 0.571)	—	0.121 (0.063, 0.180) [<0.001]	0.294 (0.171, 0.418) [<0.001]
C (56)	0.390 (0.323, 0.456)	—	—	0.173 (0.058, 0.288) [0.018]
D (13)	0.217 (0.096, 0.337)	—	—	—

CI, confidence interval.

^*Square root scale.

^†Total zone images analyzed: 223 zones.

DISCUSSION

Retinopathy of prematurity perhaps represents the finest example of the impact that an internationally accepted classification and documentation system³² can have on clinical care and scientific progress for a pediatric retinal disorder. This classification system has facilitated numerous prospective studies spanning two decades that have changed the face of clinical care and prevented blindness in countless infants.^37,38 Similar advances have also been made in retinoblastoma and other forms of eye cancer.³⁹ Despite the intense medicolegal ramifications of the ophthalmic findings in abusive head trauma, no such system has been universally accepted.

A classification system must be congruent with the anatomy of the disorder. The retinopathy of prematurity system is clearly based on the pathophysiology of that disease, in which retinal vessels grow centripetally from the optic nerve and reach the nasal ora seratta before temporal vascularization is complete. The clinical characteristics of SBS retinopathy have been well described in the two largest studies,^2,3 and further work has indicated how these features help to distinguish SBS from accidental injury.^6–10 In addition to the multiplicity and multilayered nature of these hemorrhages, there appears to be a particular predilection for hemorrhages in the peripheral retina and the presence of macular retinoschisis with or without paramacular folds. Using a variety of methods, multiple studies have shown the important role of vitreoretinal traction and shear in the pathophysiology of SBS retinopathy, a finding which offers an anatomical explanation of these features of the hemorrhages, which in large part distinguish SBS from accidental injury.⁴ Midperipheral sparing of hemorrhagic retinopathy is not uncommon in SBS,^40,41 whereas one postmortem study reported a high frequency in this zone.²⁰ Unlike the studies of Fleck,³⁰ Mulvihill,³¹ and Ng,³³ we believe that our zone system is the first to account for all of these features by more accurately representing the relevant anatomy and clinical findings. None of these previously reported systems allows for the representation of specific areas of vitreoretinal attachment or the midperipheral zone. The use of the retinopathy of prematurity classification³³ is particularly unsuited to the biology of SBS retinopathy.

Although the zone system was not revised, the modification that we made in our tool following our first publication³⁴ resulted in improved performance in almost every domain. Several factors may have contributed. The forced-choice tabular format is certainly much easier to use than the initial 69-question tool. Our examiners, who participated in the first study as well as this one, gave a resounding positive reinforcement in this regard through personal communications. Capitalizing on the methods of Fleck³⁰ and Mulvihill,³¹ we also feel that the addition of schematic zonal overlays and the use of black-and-white versions of the images, as well as the ability to rapidly toggle between these versions and the original color image, may have improved performance. Yet, zone B underperformed in interobserver agreement relative to the other zones. We believe this may in part be attributed to the usual overabundance of hemorrhages present in this area compared to zones C and D and as confirmed by our mapping anlysis.³ We previously hypothesized that confluence of hemorrhages may make the recognition of hemorrhages by a nonophthalmologist difficult as there is a loss of contrast with surrounding normal structures.⁴² We believe that the contrast offered by the optic nerve in the peripapillary zone A allows a better discernment for hemorrhages even for the ophthalmologist. This was supported by the underperformance of zone B compared to zone A, despite there being no difference in the area covered by hemorrhages in these two zones.

The surface area analysis revealed a preponderance of hemorrhages in midperipheral zone C compared to peripheral zone D. This finding contrasts with previous reports suggesting a relative sparing of the midperipheral retina in SBS, presumably due to the lesser vitreoretinal adherence in this area.^43–45 We suspect that the lower incidence of images with zone D represented may in part be responsible for creating this artifact, as evidenced by the wider confidence interval for mean hemorrhage surface area in zone D (CI 0.096, 0.337) as compared to zone C (CI 0.323, 0.456) (Table 3). We also believe that this preponderance of hemorrhages in zone C could reflect the increased difficulty of photographing zone D and the near impossibility of imaging all of zone D in one image. In addition, not all of our images were from victims of SBS, and some include disorders for which the pattern of retinal hemorrhage may differ from SBS. Therefore, our results should not in any way be taken to challenge the concept of vitreoretinal traction in the pathophysiology of SBS retinopathy. Rather, the finding of increased percentage of surface area occupied by hemorrhage in zone C as compared to zone D is more likely simply artifact induced by the methods and limits of retinal photography (ie, an artifact of the photographic process).

It may seem that in some areas, the studies of Fleck,³⁰ Mulvihill,³¹ and Ng³³ outperformed our tool. The use of fewer zones—for example, the reduction to two zones by Ng and coworkers—will almost guarantee higher performance, as the examiners have less choice for possible variation. In the studies by Fleck and Mulvihill, only designated isolated retinal hemorrhages were assessed, once again simplifying choice and decreasing the likelihood of variation. Our examiners were asked to perform a more “real life” assessment in that all hemorrhages were assessed, using different views, variable clinical severity, different cameras, and different lens magnifications. One might expect that this complexity would hamper interobserver agreement, but our tool withstood this challenge. We are intrigued, though, that our tool performed particularly poorly for superficial retinal hemorrhages in zone B. Perhaps the fact that these hemorrhages fade more rapidly, and are more thin and tenuous, allows them to be harder to recognize, especially in this zone of more severe retinopathy. The detection of retinoschisis was good or very good in all zones. This is particularly important given the epidemiology of this distinctive lesion, which has only otherwise been reported in the SBS age range (generally ≤5 years old) in rare cases of fatal motor vehicle accidents,⁴⁶ two cases of fatal head crush injury,^47,48 and an 11-meter fall onto concrete.⁴⁹ We are not surprised that the performance of our tool with regard to papilledema was only good. Digital photography removed the 3-dimensional clues that play a role in recognition of subtle disc elevation, and hemorrhages may obscure the disc margins. Fortunately, papilledema is uncommon in SBS, occurring in less than 8% of victims.^2,3

Our tool has the advantage of use via an electronic secure Web-based medium that could potentially allow for remote application via any Internet connection, if not bedside applicability. The retinopathy of prematurity system is another example of how a robust classification system allows for telemedicine application.⁵⁰ Although RetCam images could be rapidly downloaded from the camera, they would still need to be uploaded onto the Web-based platform, and the generation of the zone schematic overlays takes some time, therefore creating a challenge for bedside use. There is some urgency to the performance of the clinical ophthalmic consultation.¹¹ But the true value of a documentation tool lies rather in the ability to potentially generate objective measures to create severity scores, prognosis indices, and enhanced specificity and sensitivity analyses.

Perhaps the greatest limitation to the development of any retinal hemorrhage documentation tool is the absence of a “gold standard.” Even experienced examiners may differ at the bedside as to the type of hemorrhage or whether a given area has a certain number of hemorrhages. As the severity of the retinopathy increases, these distinctions become even more difficult. Even postmortem evaluation cannot clearly discern what was seen clinically, as the passage of time between clinical examination and autopsy can result in either resolution or even progression of the hemorrhages.⁵¹ Our study was not designed to compare different camera systems or assess the reliability of using photographs to determine retinal findings. That agreement was not higher may suggest a need for caution in using photography for this purpose. We made the same conclusion in our first published study of the tool,³⁴ but the improved performance herein is encouraging. Our tool is also limited by its dependence on digital imaging, which may not be available throughout the world. The zone system, however, is based on easily identified retinal landmarks. Techniques have been suggested for using biometry to judge the size of the optic nerve in vivo with indirect and slit-lamp biomicroscopy.^52,53 Perhaps adaptation of such techniques may be helpful in conjunction with the current proposed tool to allow some degree of objective bedside documentation. Other factors that require further study include speed of examiners in assessing images, inter-image reliability using images from the same patient (perhaps similar to the second phase of the study by Ng and coworkers³³), and a formal assessment of examiner ease of use.¹

As we continue to refine and experiment with this tool, we are pleased to see the performance increasing. Using kappa statistics in a complicated assessment, excellent scores are extremely hard to obtain and perhaps unreasonable to expect. Yet, our tool showed good or very good levels of interobserver agreement on every parameter in three of the zones and intraobserver agreement for every zone. Establishing a robust classification system for documenting retinal hemorrhages in cases of abusive head trauma may serve to further this highly medicolegal field to help ensure optimum diagnostic accuracy, clinical care, and understanding of pathophysiology. A reliable tool could also allow for future studies of the diagnostic sensitivity and specificity of retinal hemorrhage for abuse and correlations of various retinal hemorrhage parameters with other aspects of the abuse.