Validating use of ICD-10 diagnosis codes in identifying physical abuse among young children

Abstract

Objective:

Evaluate the positive predictive value of International Classification of Disease, 10^th Revision, Clinical Modification (ICD-10-CM) codes in identifying young children diagnosed with physical abuse.

Methods:

We extracted 230 charts of children <24 months of age who had any emergency department, inpatient, or ambulatory care encounters between Oct 1, 2015 and Sept 30, 2020 coded using ICD-10-CM codes suggestive of physical abuse. Electronic health records were reviewed to determine if physical abuse was considered during the medical encounter and assess the level of diagnostic certainty for physical abuse. Positive predictive value of each ICD-10-CM code was assessed.

Results:

Of 230 charts with ICD-10 codes concerning for physical abuse, 209 (91%) had documentation that a diagnosis of physical abuse was considered during an encounter. The majority of cases, 138 (60%), were rated as definitely or likely abuse, 36 cases (16%) were indeterminate, and 35 (15%) were likely or definitely accidental injury. Other forms of suspected maltreatment were discussed in 16 (7%) charts and 5 (2%) had no documented concerns for child maltreatment. The positive predictive values of the specific ICD-10 codes for encounters rated as definitely or likely abuse varied considerably, ranging from 0.89 (0.80 – 0.99) for T74.12 “Adult and child abuse, neglect, and other maltreatment, confirmed” to 0.24 (95% CI: 0.06 – 0.42) for Z04.72 “Encounter for examination and observation following alleged child physical abuse.”

Conclusion:

ICD-10-CM codes identify young children who experience physical abuse, but certain codes have a higher positive predictive value than others.

Introduction

Administrative and discharge datasets are an important source of health services information pertaining to child physical abuse.¹ International Classification of Disease, Clinical Modification (ICD-CM) codes are used to document diagnoses as part of routine medical care and are therefore available within clinical and administrative datasets to identify children who have experienced physical abuse. ICD-CM codes have been used widely to evaluate epidemiologic trends in abuse, clinical care practices for abused patients, and disparities in diagnosis or care of abused children.^2–7

Most validation of ICD-CM codes for identification of abuse was done with ICD-9-CM codes rather than the ICD-10-CM codes currently used in practice. ICD-9-CM codes were limited in their ability to distinguish between suspected and confirmed abuse, contributing to inconsistency in their application.⁸ Theoretically the transition from ICD-9-CM to the currently used ICD-10-CM codes provided opportunity for increased specificity in maltreatment diagnoses, by creating distinct codes for suspected and confirmed abuse.^3,9,10 Yet comparison of trends in administrative codes for the identification of abuse hospitalizations suggests instability in code utilization after the transition to ICD-10-CM.¹⁰ One retrospective analysis of 115 hospitalized patients included in a trauma registry found that among patients with clinically confirmed abuse, only 63% had ICD-10-CM codes identifying abuse while 6% of trauma registry patients without evidence of abuse were incorrectly over-coded as “suspected abuse.”¹¹ Given the potential research and public health surveillance implications of changes in administrative coding, it is important to evaluate the informativeness of ICD-10-CM codes.

More broadly, little is known about the accuracy of ICD-CM codes to identify cases of physical abuse outside of acute care settings. Though ICD-CM codes are assigned in multiple clinical care settings, prior validation for detection of physical abuse were largely done in inpatients evaluated by child protection teams (CPTs) or trauma teams, with the exception of a single, older study which included patients evaluated in an emergency department and inpatient setting.^12–16 Validation using patients undergoing child abuse evaluation by a CPT allowed for estimation of sensitivity and specificity of ICD-CM codes, but limits generalizability beyond the subset of patients evaluated by CPTs in hospital settings. Another study estimated the incidence of inflicted abusive head trauma (AHT) among hospitalized infants and found that the incidence estimates were similar to those in previously published using active surveillance.¹⁷ These results suggest that ICD-CM codes may provide reasonable estimates of infants hospitalized with AHT but do not provide information on the accuracy of ICD-CM codes in an individual infant. The informativeness of abuse-related ICD-CM codes beyond the population of children evaluated by trauma and child protection teams in the acute care settings, has not been well characterized.

With the transition to ICD-10-CM we sought to assess the positive predictive value of ICD-10-CM codes in identifying young children who experienced physical abuse and to do so within a regional care network, using codes assigned in inpatient, emergency department, urgent care, primary care, and specialty care settings.

Methods

Study Population and Data Extraction

Children < 24 months old receiving primary care through a large regional pediatric primary care system, as evidenced by at least one primary care visit from October 2015 to September 2020, were eligible for inclusion, if they had any encounter with at least one of the 21 ICD-10-CM codes for physical abuse (T74.12XX, T74.4XXX T76.12XX, Z04.72), assault (X92-X99, Y00-Y04 & Y08-Y09), or perpetrator of assault, neglect or maltreatment (Y07.XX) assigned (list and title of codes detailed in Table 2). In instances in which multiple codes for physical abuse were assigned in the initial encounter, all codes were included in analysis. We restricted evaluation to patients who had established care within our primary care network with at least one primary care visit prior to diagnosis to allow for contextual review of clinician intent in assigning codes. Our institution is a tertiary care referral center, and children present to radiology, the emergency department, or specialty care for evaluation and management of injuries; absent clinical context, it can be difficult to make determinations about diagnostic certainty. We focused on children under the age of 24 months as they are frequently evaluated in pediatric primary care ¹⁸ and are at increased risk for fatal abuse compared to older children.^19,20 Inpatient, emergency department, urgent care, and outpatient encounters were queried in the institutional clinical data warehouse (CDW) and patients were included if they had ICD-10-CM codes for abuse assigned in any of those settings. If patients had more than one episode of care in which ICD-10-CM codes were assigned, only the first episode of care was reviewed. ICD-10-CM codes, patient demographic information, date of initial diagnosis, and clinical setting were imported into a REDCap^21,22 database. This study was determined to be exempt from review by the Institutional Review Board.

Table 2.

Positive predictive value of ICD-10-CM code in identifying physical abuse.

		Records Reviewed	Positive Predictive Value (95% CI)
			Abuse Considered (n=209)	Definitely or Likely Abuse (n=138)
T74.12	Child abuse, neglect, and other maltreatment, confirmed	38	0.97 (0.86 – 1.00)	0.89 (0.75 – 0.97)
Y09	Assault by unspecified means	14	0.93 (0.66 – 1.00)	0.86 (0.57 – 0.98)
Y07	Perpetrator of assault, maltreatment and neglect	43	0.86 (0.72 – 0.95)	0.72 (0.56 – 0.85)
T76.12	Child abuse, neglect, and other maltreatment (suspected)	140	0.94 (0.88 – 0.97)	0.59 (0.50 – 0.67)
Y04	Assault by bodily force	23	0.91 (0.72 – 0.99)	0.52 (0.31 – 0.73)
Z04.72	Examination following alleged child physical abuse	21	0.81 (0.58 – 0.95)	0.24 (0.08 – 0.47)
Y08.89XA	Assault by other specified means, initial encounter	5	--	--
X95	Assault by other and unspecified firearm and gun discharge	2	--	--
T74.4	Shaken Infant Syndrome	2	--	--
X93	Assault by handgun discharge	1	--	--

Positive predictive value for physical abuse associated with each code was calculated when more than 5 cases were reviewed. Patients may have had multiple ICD-10-CM codes applied; therefore these do not sum to 230.

Chart Review

Identified episodes of care were reviewed independently by study team members including general pediatricians (n=3) and child abuse pediatricians (n=2) to determine whether the patient had sustained physical abuse (Figure 1). First, the documentation was reviewed to determine whether physical abuse was considered during the episode of care. Among cases in which documentation indicated a physical abuse diagnosis had been considered, reviewers assessed the level of certainty for physical abuse using the following categories: definitely accidental, likely accidental, likely abuse, definitely abuse, medically indeterminate, and cannot be determined (Appendix 1).¹³ This categorization was adapted based on abuse certainty scales used in the literature^13,16,23 and clinical practice; we added an additional category (“indeterminate”) to the 5-point scale used clinically when child abuse pediatricians evaluate children for abuse at our institution. This additional category allowed for capture of additional uncertainty present in the broader population of children not undergoing comprehensive evaluation by a child abuse team. Reviewers also indicated whether a report of suspected physical abuse was filed with the local child protective services (CPS) agency and whether the CPS agency instituted a safety plan during the episode of care. Safety plans could include in-home protective services as well as placement in foster care or kinship care.

Figure 1: Chart Review Strategy and Findings.

Cases reviewed to identify child abuse. Cases which referenced multiple forms of abuse including physical abuse (n=7) were included in review. Cases were also classified by level of certainty for physical abuse using the following categories: definitely accidental, likely accidental, likely abuse, definitely abuse, medically indeterminate, and cannot determine. These categories have been grouped for purposes of analysis. CDW: clinical data warehouse.

Reviewers were asked to consider all notes, imaging, and lab data during the initial episode of care in which the ICD-10-CM codes were assigned. For instance, if a history of abuse was obtained in primary care and the patient was referred to the emergency department where the diagnosis was made and a report was filed, reviewers were asked to consider both the primary care and the emergency department visit documentation in making their determination. To assess interrater reliability, 17% (38 of 230) of charts were reviewed by more than one reviewer. Reviewers were blinded to each other’s assessments.

Analysis

Demographic characteristics and frequencies of specific ICD-10-CM codes were tabulated. Certainty of diagnosis was grouped (definitely/likely abuse, definitely/likely accidental injury, or medically indeterminate/reviewer could not determine) for purposes of analysis. The positive predictive values (PPV) for both consideration of abuse and for diagnosing definitely/likely physical abuse were calculated for each individual ICD-10-CM code that appeared more than 5 times. Interrater reliability was assessed using percent agreement and a Fleiss kappa statistic.²⁴ Results were stratified by clinical encounter site (emergency department/urgent care, inpatient, ambulatory specialty care, or primary care). Analyses were completed in R-Studio²⁵ and Stata.²⁶

Results:

We evaluated 230 encounters identified by ICD-10-CM codes for abuse or assault, including 51 (23%) outpatient encounters, 72 (31%) emergency department or urgent care encounters, and 107 (47%) hospitalizations. Most encounters (n=177, 77%) had a single ICD-CM-10 code; 6 encounters had 3 or more codes assigned. Across encounters, 209 (91%) had consideration of concerns for possible physical abuse documented in the medical record including 7 (3%) which described multiple forms of abuse; these 209 cases were reviewed in further detail (Figure 1). The 21 charts which either discussed another form of abuse (n=16) or did not reference any type of abuse (n=5) were not evaluated further. There were no significant demographic differences between children who experienced physical abuse as compared with children identified by ICD-10-CM code but did not experience physical abuse (Table 1).

Table 1:

Demographic Characteristics of encounters (n=230) identified by ICD-10-CM codes for abuse or assault, by certainty of abuse

	Records Reviewed	Maltreatment classification
		No abuse or non-physical abuse (N=21)	Definitely/Likely Accidental Injury (N=35)	Indeterminate (N=36)	Definitely/Likely Physical Abuse (N=138)	P *
Age
0-4 months	102 (44%)	7 (33%)	16 (46%)	14 (39%)	65 (47%)	0.53
5-12 months	60 (26%)	4 (19%)	8 (23%)	10 (28%)	38 (28%)
13-24 months	68 (30%)	10 (48%)	11 (31%)	12 (33%)	35 (25%)
Gender
Male	135 (59%)	10 (48%)	21 (60%)	18 (50%)	86 (62%)	0.40
Female	95 (41%)	11 (52%)	14 (40%)	18 (50%)	52 (38%)
Race
Black	139 (60%)	15 71(%)	21 (60%)	14 (39%)	89 (64%)	0.07
White	50 (22%)	3 (14%)	7 (20%)	14 (39%)	26 (19%)
Other or Multiracial	37 (16%)	2 (10%)	7 (20%)	6 (17%)	22 (16%)
Asian	4 (2%)	1 (5%)	0	2 (6%)	1 (<1%)
Ethnicity
Hispanic or Latino	19 (8%)	0	4 (11%)	3 (8%)	12 (9%)	0.50
Not Hispanic	211 (92%)	21 (100%)	31 (89%)	33 (92%)	126 (91%)
Insurance **
Private	47 (20%)	2 (10%)	8 (23%)	9 (25%)	28 (20%)	0.59
Public	177 (80%)	18 (86%)	27 (77%)	27 (75%)	104 (75%)
Care Setting
Inpatient	107 (47%)	9 (43%)	13 (37%)	9 (25%)	76 (71%)	0.01
Emergency or Urgent Care	72 (31%)	7 (33%)	16 (46%)	13 (36%)	36 (50%)
Outpatient	51 (23%)	5 (24%)	6 (17%)	14 (39%)	26 (51%)
Report made to child protective services
Report made	197 (86%)	14 (67%)	24 (69%)	30 (83%)	137 (99%)	<0.01
Not reported	33 (14%)	7 (33%)	11 (31%)	6 (17%)	1 (<1%)
Safety Plan
Documented	177 (77%)	11 (52%)	19 (54%)	19 (53%)	128 (93%)	<0.01
No Safety Plan Documented	53 (23%)	10 (48%)	16 (46%)	17 (47%)	10 (7%)

^*p value reflects the significance associated with Pearson chi squared test

^**6 records had incomplete or missing data with respect to payor

In charts referring to physical abuse, reviewers identified that in 138 of the 230 cases (60%) the documentation suggested that the child definitely/likely experienced abuse (Figure 2). Reflective of this clinical certainty, 137 (99%) of these 138 cases referenced a report to CPS and 128 of 138 cases (93%) included description of safety plan. In 35 of the 230 cases (15%) documentation suggested that the child definitely/likely experienced accidental injury or findings were attributed to an underlying condition rather than abuse. An example of a case identified as “likely accidental” was a toddler who presented with a supracondylar fracture sustained while outside of parents’ care but ultimately deemed by the emergency department team to be accidental. In 36 of the 230 cases (16%) the etiology of the child’s injury could not be determined, or the chart reviewer was unable to ascertain the level of certainty of diagnosis based on the documented information. An example of an “indeterminate” case was that of a young child evaluated with skeletal survey (which was normal) after her sibling presented with subdural hemorrhages. There was 96% reviewer agreement in identifying cases describing physical abuse. There was good interrater agreement in evaluating the degree of diagnostic certainty (Fleiss kappa statistic, 0.63).

Figure 2: ICD-10-CM code use in identifying physical abuse.

ICD-10 code use among the 209 patients whose medical records discussed physical abuse. Patients may have had multiple ICD-10-CM codes applied; therefore these do not sum to 209.

Both use and predictiveness of codes in identifying cases that were definitely or likely abuse varied widely (Table 2); several codes were used fewer than 5 times. The T76.12 code for “Child, abuse neglect, and other maltreatment (suspected) was used 140 times, but other codes were rarely issued such as the T74.4 code for “Shaken Infant Syndrome” which appeared in only 2 records (Table 1). The T76.12 code appeared across all settings while the Y08.89 and Y09 were only used in emergency department, urgent care or inpatient settings. The code for “Adult and child abuse, neglect, and other maltreatment, confirmed” (T74) was most predictive in identifying children who had experienced physical abuse (PPV 0.89; 95% CI: 0.80 – 0.99).

Discussion:

We evaluated the positive predictive value of individual ICD-10-CM codes in identifying children who experienced physical abuse, using data from emergency department, inpatient, and ambulatory care settings. The positive predictive values of specific ICD-10-CM codes varied considerably, ranging from 0.89 (0.80 – 0.99) for T74 “Adult and child abuse, neglect, and other maltreatment, confirmed” to 0.24 (95% CI: 0.06 – 0.42) for Z04.72 “Encounter for examination and observation following alleged child physical abuse.” This finding provides important validation information for health services researchers studying child physical abuse using clinical or administrative datasets. ICD-CM codes are used widely in health services and clinical research to evaluate child abuse.

In the prior literature, ICD-9-CM diagnostic codes demonstrated utility in identifying abused patients among hospitalized or emergency department patients undergoing abuse evaluations by child protection or trauma teams.^11–15 Recent work suggests ICD-10-CM codes instability in identification of abuse hospitalizations.¹⁰ Our sample was not derived from a population undergoing evaluation by a child protection team and included both inpatient and outpatient visits, and therefore speaks to the broader utility of these diagnostic codes in elucidating trends in diagnosis and clinical practice pertaining to abused children. We found that certain codes have a higher positive predictive value than others.

Our findings must be contextualized within limitations of our approach. Because we identified these patients using ICD-10-CM codes (rather than a clinical registry of children diagnosed with abuse), we were unable to evaluate sensitivity or specificity of ICD-10 codes. One single center study of Pediatric Level 1 Trauma patients demonstrated ICD-10-CM code sensitivity of 55.6% and 22.2% for inpatients and outpatients, respectively, and specificity of 78.6% for inpatients and 86.3% for outpatients.²⁷ Though these estimates are in keeping with an earlier ICD-9-CM validation study,¹⁶ the authors of this work attribute some of the limitations in sensitivity and specificity to use of ICD-10-CM code Z04.72 (Examination and observation following alleged physical abuse) and the application of codes for suspected physical abuse. Indeed, in our study Z04.72 had the lowest PPV (0.24, 95% CI: 0.06 – 0.42). Similarly, the most frequently applied code in our study was T76.12 (Child abuse, neglect, and other maltreatment, suspected) which had a PPV of 0.59 (95% CI: 0.50 – 0.67). Because of our limited sample, we were unable to evaluate predictiveness of uncommonly used codes and combinations of different codes. Finally, we explored ICD-10-CM codes assigned within a single health system. There are institutional differences in the way ICD codes are assigned.^10,13 Future work exploring the informativeness of ICD-10 codes across institutions is needed.

Despite these limitations, our results demonstrate that the assignment of certain ICD-10-CM codes are highly suggestive of a diagnosis of physical abuse in a young patient. Thus, these ICD-10-CM codes can be used in health services research to study the evaluation, treatment and outcomes of young children who experience physical abuse, with the caveat that some abused children may not be captured.

Conclusions

Administrative and clinical data are important sources of information about abuse incidence and diagnosis. The presence of ICD-10-CM code T74 “Adult and child abuse, neglect, and other maltreatment, confirmed” reliably indicates that a young child was diagnosed with abuse. Future work should further evaluate the sensitivity of specific ICD-10 codes to identify children diagnosed with physical abuse.

What’s New:

Certain ICD-10-CM codes reliably identify young children who have experienced physical abuse in administrative data within a general pediatric population.