Predictive value of indicators for identifying child maltreatment and intimate partner violence in coded electronic health records: a systematic review and meta-analysis

Abstract

Objective

Electronic health records (EHRs) are routinely used to identify family violence, yet reliable evidence of their validity remains limited. We conducted a systematic review and meta-analysis to evaluate the positive predictive values (PPVs) of coded indicators in EHRs for identifying intimate partner violence (IPV) and child maltreatment (CM), including prenatal neglect.

Methods

We searched 18 electronic databases between January 1980 and May 2020 for studies comparing any coded indicator of IPV or CM including prenatal neglect defined as neonatal abstinence syndrome (NAS) or fetal alcohol syndrome (FAS), against an independent reference standard. We pooled PPVs for each indicator using random effects meta-analyses.

Results

We included 88 studies (3 875 183 individuals) involving 15 indicators for identifying CM in the prenatal period and childhood (0–18 years) and five indicators for IPV among women of reproductive age (12–50 years). Based on the International Classification of Disease system, the pooled PPV was over 80% for NAS (16 studies) but lower for FAS (<40%; seven studies). For young children, primary diagnoses of CM, specific injury presentations (eg, rib fractures and retinal haemorrhages) and assaults showed a high PPV for CM (pooled PPVs: 55.9%–87.8%). Indicators of IPV in women had a high PPV, with primary diagnoses correctly identifying IPV in >85% of cases.

Conclusions

Coded indicators in EHRs have a high likelihood of correctly classifying types of CM and IPV across the life course, providing a useful tool for assessment, support and monitoring of high-risk groups in health services and research.

What is already known on this topic?

• Electronic health records (EHRs) are readily available and are increasingly used to identify different forms of family violence in practice and public health.

• Few studies of EHRs provide comprehensive estimates on the positive predictive values (PPVs) for coded indicators of family violence including child maltreatment, prenatal neglect (neonatal abstinence syndrome or fetal alcohol syndrome) and intimate partner violence.

What this study adds?

• This comprehensive meta-analysis provides PPVs of coded indicators in EHRs for different forms of family violence based on external independent reference standards.

• We show that routinely coded indicators of family violence have high predictive value for identifying at-risk groups who may benefit from targeted interventions.

• Findings emphasise that improving the quality and use of available coded indicators for identifying groups affected by family violence across data systems should be a public health priority.

Introduction

Intimate partner violence (IPV) and child maltreatment (CM) are forms of family violence that often go unnoticed by services,^1–3 despite repeated recommendations by the WHO to improve monitoring efforts.^{4 5} CM and IPV refer to any act of commission or omission that causes biopsychosocial harm to a child, a future child or partner.^{6 7} Statutory definitions of CM in the UK include fetal alcohol syndrome (FAS) and neonatal abstinence syndrome (NAS) due to neglect or harm during pregnancy.⁸

Assessing health records for detailed information on family violence is time consuming and expensive. Instead, studies and services are increasingly using routinely coded electronic health records (EHRs) for assessing family violence.^{9 10} Coded EHRs allow for longitudinal population-based assessments, automated early warning systems and identification of high-risk populations for targeted interventions at relatively low costs.^11–13 However, the potential utility of EHRs to support surveillance and clinical decisions is often undermined by reported quality issues and coded conditions are rarely validated externally.^{14 15} Unable to check the data themselves for accuracy, large-scale studies (eg, Global Burden of Disease Study) and services rely on routinely coded indicators based on the International Classification of Diseases (ICD) with unknown predictive values.^{10 16 17} The validity of coded indicators are compounded by varying case definitions, ad hoc classifications by coders5 15 18–20 and under-recording due to clinician fears of potential harm or lack of awareness.^{18 21}

To our knowledge, no previous review has estimated the positive predictive values (PPVs) of coded indicators for different forms of family violence including CM, prenatal neglect (NAS or FAS) and IPV based on external independent reference standards (ie, not using other codes in the EHR to validate indicators). This meta-analysis provides a comprehensive summary of PPVs for multiple coded indicators in EHRs aimed at identifying family violence in general healthcare settings, compared with an independent reference standard.

Methods

We followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses for Diagnostic Test Accuracy Studies and the Meta-analysis of Observational Studies in Epidemiology guidelines.^{22 23} The review protocol was published in the PROSPERO registry (CRD 42019139300),²⁴ with any protocol deviations provided in online supplementary table S1.

Search strategy

We searched 18 electronic databases and 20 selected journals for studies published between 1 January 1970 and 24 May 2020 (complete search strategy in online supplementary table S2). Reference lists of eligible full-text articles were hand-searched and 18 frequently cited authors of eligible studies were contacted for article recommendations.

Study eligibility

Three independent reviewers (SS, RA and MS) screened abstracts, full texts and corresponding reference lists of articles using Covidence’s systematic review software.²⁵ Any disagreement over study inclusions between reviewers was resolved by a family violence expert (RG). We included: (1) studies with data to calculate the PPV of a coded EHR indicator for a specific family violence outcome as reported by at least three other eligible studies,²⁶ (2) studies from primary care, paediatric units (including trauma centres) or general hospital settings, (3) studies published in English/Swedish/German and (4) studies distinguishing family violence cases from non-cases (tables 1–2; online supplementary table S3-S4).

Table 1.

Key definitions of electronic health records (EHRs) and family violence

EHRs Routinely collected healthcare data stored and accessible electronically as part of routine care or as part of administrative databases for wider patient, practice and billing management purposes.
Coded indicators of EHRs Single or combination of codes aimed at identifying different forms of family violence (table 2). Codes must be based on a recognised classification system (eg, International Classification of Disease system) assigned to patients’ EHRs by a clinician or by a professional coder based on medical records (eg, discharge documentation).
Family violence outcomes Any of the below outcomes obtained independently from the coded indicator (ie, not using other codes in the EHR to validate the indicator):
Child maltreatment (CM) Any act of commission or omission by a parent or caregiver resulting in harm, the potential for harm or threat of harm to a child (0–12 years) or an adolescent (13–19 years) including neglect, psychological, physical, sexual and emotional abuse. Harm does not need to be intended.6–8 6–8
Neonatal abstinence syndrome (NAS) or fetal alcohol syndrome (FAS) Statutory guidelines definition of CM in the UK include any form of maternal omission or commission during pregnancy with potential for significant biopsychosocial harm to the future child such as NAS and FAS.8 This may include compliance with prescribed medication (clinician approved) and/or maternal substance misuse during pregnancy, resulting in drug withdrawal symptoms in newborns and/or harm to the future child's biopsychosocial functioning/appearance.
Intimate partner violence (IPV) Any incident of threatening behaviour, violence or abuse (psychological, physical, sexual, financial or emotional) between adults who are, or have been, intimate partners or family members.7 In this review, studies are predominately restricted to estimates of women aged 15–50 years.
Eligible methods for obtaining independent reference standards
Independent and blinded/non-blinded manual chart review of full medical, social and/or criminal records (either as extracted or by recoding charts for direct comparison with originally assigned codes).
The individual meets criteria defined by a validated instrument (eg, self-report survey) completed within 1 month of the coded EHR.
Clinician confirms case via a self-report survey administered to the service.
Linkage of the same individual to another independent non-health-related database that provides concordance of the diagnosis by a qualified professional (eg, social worker and police).
Linkage of the same individual to another healthcare database that provides concordance of the diagnosis (eg, similar diagnosis recorded in both hospital discharge and general practitioner records).
Level of certainty for distinguishing cases of family violence from non-cases in the information obtained by the reference standard Quality ratings correspond to the level of certainty that the reference standard is a true measure of CM, NAS, FAS or IPV. Ratings 1–5 apply to CM, 1–4 apply to NAS and FAS and 1–3 apply to IPV.	Quality rating
CM confirmed at case conference or family, civil or criminal court proceedings; admitted by the perpetrator; or witnessed abuse AND non-CM actively excluded by stated criteria (eg, witnessed accidental cause, caused by metabolic bone diseases etc).	5 (high)
CM confirmed by stated criteria including multidisciplinary assessment AND non-CM actively excluded by stated criteria.	4
CM/IPV defined by stated criteria AND source verifying non-CM/IPV merely stated.	3
CM/IPV stated but no supporting detail is given AND source verifying non-CM/IPV merely stated.	2
Suspected CM/IPV AND no criteria stated for verifying non-CM/IPV.	1 (low)

The criteria for assessing studies of EHRs was adapted from McCormick et al,¹³⁴ and the quality ratings for the level of certainty of CM in the reference standard were adapted from Kemp et al.²⁹ The complete quality assessment tool is provided in online supplementary table S3.

Table 2.

Coded indicators of specific family violence outcomes as measured by the included studies

Outcome	Indicator and period*	Descriptions*	No. data sources†	Frequently used codes‡
	Prenatal neglect/substance misuse during pregnancy		20
Neonatal abstinence syndrome (NAS)	NAS: primary diagnosis	Diagnosis of substance withdrawal symptoms in newborns, typically within the first few days after birth, resulting from drug use (commonly opioids) during pregnancy.135	1336 39–43 45	ICD-9: 779.5. ICD-10: P96.1.
	Newborn affected by drugs	Less specific indicator of NAS, defined by the fetus being affected by maternal substances, noxious influences or medications NOS.	739 43	ICD-9: 760.72. ICD-10: P04.4x.
Fetal alcohol syndrome (FAS)	FAS: primary diagnosis	FAS is one type of a spectrum of disorders linked to alcohol exposure during pregnancy. Symptoms include abnormal facial features, deficiencies in growth (including brain/ neurophysiology) and/or neurobehavioural impairment in children (0–19 years, typically diagnosed 2–5 years).136	746–52	ICD-9: 760.71. ICD-10: Q86.0, P04.3.
	Child maltreatment (CM) among children aged 0–19 years		50
CM	CM primary diagnoses	Any diagnosis of CM including neglect, abandonment, physical, sexual and psychological/emotional abuse (0–18 years).	1914 55 59 60 62 66 68 71 77 80 82 85 87 96 97	ICD-9: 995.5x, E967.x, E960.x-968.x. ICD-10: T74.x, Y06.x, Y07.x.
	Abusive head trauma (AHT)	Skull and/or intracranial injuries due to inflicted blunt impact/penetration and/or violent shaking (0–5 years). Coding clusters include different degrees of certainty: ‘definite or presumptive’ and ‘probable’ AHT. Other common terms are ‘shaking infant syndrome’ or ’whiplash shaking baby syndrome’.137	462 80 85 138	ICD-9: As in CM + 781.0-781.4, 800.x-801.x, 854.0 x, 950.0-950.3. ICD-10: As in CM + S06.0-S06.9, S09.7-S09.8, T90.5.
	Traumatic brain injury (TBI)	Skull and/or intracranial injuries and concussion (<5 years of age) characterised by the level of loss of consciousness or altered mental status, ranging from mild TBI (in most cases a concussion) to severe TBI (patient is comatose, unable to open their eyes and so on).139	757 66 69 76 80 84 92	ICD-9: 800.0-801.9, 803.0-804.9, 850.1-850.5, 850.9, 851.0-854.1, 959.01. ICD-10: S02.0, S02.1x, S02.8, S02.91, S04.02, S04.03.x, S04.04x, S06.x, S07.1, T74.4.
	Skull fractures	A fracture of one or more bones of the cranial vault or skull base.	1053 57 70 72 73 75 83 88 95 96	ICD-9: 800.x-804.x ICD-10: S02.x
	Subdural haematoma	A collection of blood between the dura and arachnoid coverings of the brain. In this review, subdural haematoma includes ICD-coded subdural haemorrhages.	557 76 88 94 96	ICD-9: 852.2, 432.1. ICD-10: I62.0, S06.57.
	Retinal haemorrhage	Extravasation of blood into the retina, between the retina and the subhyaloid or preretinal or between the retina and the subretina.	465 81 88 96	ICD-9: 362.81. ICD-10: H35.6.
	Upper limb fractures	In this review, referred to as fractures of the humerus, radius or the ulna.	474 86 89 91 96	ICD-9: 812.x, 813.x ICD-10: S42.2-S42.9, T10.
	Lower limb fractures	In this review, referred to as fractures of the femur, diaphysis, tibia or fibula.	654 56 63 67 89 96	ICD-9: 820.x, 821.x. ICD-10: S72.x, S82.x, T12.
	Rib fractures	A break in the rib bone of the thoracic skeleton.	464 74 88 89	ICD-9: 807.0-807.1x ICD-10: S22.3, S22.4.
	Multiple burns	Burns and corrosions of multiple and unspecified body regions.	358 88 96	ICD-9: 946.0. ICD-10: T29.x-T32.x.
	Assaults	Intentional injuries (sexual, physical, emotional or psychological) inflicted by another person.	659 77 78 82 88	ICD-9: E960.0-E969.0, V71.6. ICD-10: X85-Y09.
	Assaults/CM/adversity	Combination of multiple broad codes deemed to meet the threshold for potential child harm including CM, assaults, drug or alcohol intoxication, poisoning of undetermined cause, concerns for psychosocial circumstances, specific fractures and/or sexual diseases in children (0–18 years).	661 64 82 90 93	ICD-9: As in CM + V15.4, V61.2, V71.8, E960.1, E967. ICD-10: X85-Y09, Z04.0-Z04.8, Z60.0-Z91.8.
	Intimate partner violence (IPV) among women aged 12–55 years		18
IPV	IPV primary diagnosis	Any diagnosis of partner maltreatment syndromes including neglect, abandonment, physical, sexual and psychological abuse.140	3111 114 115	ICD-9: E967.3, 995.80, 995.81. ICD-10: T74.1, Y06.0, Y07.0.
	Assaults	Intentional injuries (sexual, physical, emotional or psychological) inflicted by another person.	10100 102 105 107–109 112–114 116	ICD-9: E960.0-E969.0, V71.6, V71.5. ICD-10: X85-Y09.
	Head, face and neck injuries	Maxillofacial fractures (mandible fractures, zygomatic complex fractures and intracranial injuries) and crushing injuries of head and neck.	499 103 107 110	ICD-9: 802.0-802.9. ICD-10: S02.2, S02.6, S06.x-S07.x.
	Ocular injuries (eg, orbital fractures)	Ocular injuries such as fractures to the orbital floor and sustained scleral laceration or ruptures.	499 103 106 117	ICD-9: 802.6, 871.0, 871.1. ICD-10: S02.3, S05.1, S05.2-S05.3.
	Contusions of upper body	Bruises or contusions of the throat, breast, upper arms/shoulders and eyelids, periocular areas and scalp.	3101 107 110	ICD-9: 920.x-924.x. ICD-10: S10.0-S10.9, S20.0, S40.x, S05.1, S00.1, S05.7.

*Indicator definitions represent condensed descriptions for the overall category and are not representative of all ICD code descriptions.

†References represent the number of unique publications per indicator and do not necessarily match the corresponding number of data sources, as one publication can provide estimates for one indicator from several unique data sources. The same reference can also apply to multiple indicators, as one reference can report on estimates for multiple indicators.

‡Complete lists of codes by individual studies and indicators are provided in online supplementary table S8.

“x” indicates any character from fourth through seven.ICD, International Classification of Diseases; NOS, not otherwise specified.

Indicators and outcomes of family violence

An overview of key definitions of indicators and outcomes is provided in tables 1–2. Briefly, indicators were defined as any coded marker or risk factor for family violence, ranging from specific primary diagnostic codes (eg, ‘T74.1 physical abuse’) to injuries (eg, rib fractures and retinal haemorrhages), assaults and combinations of adversity-related codes (table 2). For IPV, we predominately included indicators for women (eg, >80% of study sample were women), as the health consequences for women are higher, and the prevalence among men is significantly lower.²⁷ In terms of family violence, men’s parental status is also more difficult to ascertain than women’s. Outcomes were defined as mutually exclusive categories of family violence according to different life periods: NAS or FAS (representing the prenatal period),⁸ any form of CM (representing childhood) or IPV (representing women of reproductive age), respectively. Outcomes had to be ascertained and verified by an independent reference standard such as recorded multidisciplinary decisions of family violence extracted from chart reviews (table 1).

Data extraction and quality assessments

Using a piloted standardised extraction form, three reviewers independently extracted relevant study characteristics. If studies reported separate estimates for multiple codes, reference standards or age criteria, we extracted all estimates and prioritised those based on the criterion most frequently used by other studies to increase overall homogeneity. We requested additional information from 25 authors, 6 of whom responded within the 2-month deadline and were included. The risk of bias was assessed by the same three reviewers using a revised version of the Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2; online supplementary table S4).²⁸ We rated each study's reference standard according to assessment quality and exclusion criteria based on previous reviews (table 1).²⁹

Data synthesis

The PPVs for each family violence outcome were calculated as the proportion of identified cases by an indicator validated as true cases by an independent reference standard. For studies that reported only sensitivity and specificity, we obtained PPVs using Bayes’ theorem.³⁰ PPVs were pooled using random effects intercept logistic regression models with the logit transformation when at least three studies were available for the same outcome.³¹ The model accounts for potentially misleading back transformations of PPVs when pooling studies with highly variable sample sizes.³¹ Where applicable, we also examined documentation quality and coding errors by pooling the proportions of coded medical charts with missing key information (full procedures in online supplementary table S5).

We measured the extent to which the PPVs varied between studies (ie, between-study heterogeneity).³² As the PPV is a measure of proportions, we measured the between-study heterogeneity using the I² statistic (>75%=indicates substantial heterogeneity),³² standard χ² tests, prediction intervals (where the PPV is expected for 95% of similar future studies)³² and subgroup analyses when at least four studies were available for each subgroup.²⁴ We used random effects meta-regression to assess for the impact of publication year on the between-study heterogeneity. The influence of individual studies was explored by serially omitting different studies from the overall estimates.

Publication bias occurs when studies with favourable results (ie, high PPV) are more likely to be published than unfavourable results.³³ We explored publication bias by plotting the PPV against the SEs of individual estimates using funnel plots for indicators with at least 10 studies for the same outcome. To test for statistically significant differences in funnel plot asymmetry (ie, publication bias), we used Egger’s test and Begg’s and Mazumdar rank correlation test.³³ We used R (V.3.6.1) and the ‘meta’ package with the ‘metaprop’ command to perform the analyses.^{34 35}

Results

Study characteristics

Details of the included study characteristics are provided in online supplementary tables S6–S8 and tables S12–15. In total, 65 cross-sectional and 23 longitudinal studies (81 unique publications), involving 20 indicators and 3 875 183 individuals from 11 different countries met the inclusion criteria (figure 1). Overall, 13 studies provided indicators for NAS,^36–45 7 for FAS,^46–52 50 for CM (0–18 years)^{14 53–98} and 18 for IPV among women (12–50 years).^99–116 Most studies were from the USA (72 studies, 81%), with a minority from Australia (8 studies, 9%) and Europe (4 studies, 4%). The majority of indicators comprised different ICD-9 coding clusters (64 studies) or modified versions of ICD-10 (14 studies). Chart reviews with predefined family violence criteria were the most frequent type of reference standard for all outcomes (75 studies, median sample size: 301 participants). The smallest study included 38 children,⁹⁴ involving all ICD-9 coded subdural haematomas (SDH) at one hospital over 10 years. The study was retained as it involved all potential cases presenting to a generic children’s hospital and as SDHs are deemed to be extremely rare in young children and an important indicator for CM.¹¹⁷ We included one unpublished dissertation from 2019,⁵⁹ specifically assessing ICD codes for CM.

Figure 1.

Flow diagram of the 88 selected studies for this systematic review. CM, child maltreatment; FAS, fetal alcohol syndrome; IPV, intimate partner violence; NAS, neonatal abstinence syndrome.

Study quality assessment

Individual QUADAS-2 scores for studies are provided in online supplementary table S7. Overall, 31 studies (35%) were rated as high risk of bias and lower quality in most domains, and 57 studies (65%) were rated as low risk of bias and higher quality in most domains. The majority of studies were rated lower as they did not mask the outcome when reviewing coded EHRs (95%) or used a lower rated reference standard (42%; a rating <4 for CM; rating <3 for NAS, FAS or IPV).

Pooled estimates of PPVs

Table 3 depicts details for each pooled PPV and between-study heterogeneity by indicator and outcome. Individual forest plots of PPVs and study-specific ICD codes are provided in online supplementary figures S1-S19 and tables S6-S8.

Table 3.

Pooled positive predictive values for NAS, FAS, CM and IPV

Family violence outcome	Indicator and age cut-off in years	Studies	Ref std. cases/indicator positive	Pooled PPV (95% CI)	Between-study heterogeneity
					(95% prediction interval)*	I2 %
	Prenatal
NAS	NAS primary diagnosis	16	3030/3796	80.9 (71.0 to 87.9)	(32.3–97.4)	97.4
	Newborn affected by drugs	7	384/2202	27.2 (16.8 to 40.8)	(6.5–66.7)	95.0
FAS	FAS primary diagnosis	7	777/2687	39.3 (25.3 to 55.4)	(9.7–79.6)	97.6
	Childhood
CM	CM primary diagnoses ≤18	19	3090/3717	87.8 (83.4 to 91.2)	(63.5–96.8)	92.1
	Abusive head trauma ≤4†	4	437/477	91.6 (88.8 to 93.8)	(83.8–96.7)	36.7
	Traumatic brain injury ≤2	7	410/1770	22.9 (15.3 to 32.9)	(7.0–53.9)	94.2
	Skull fractures ≤4	10	603/2872	22.1 (18.5 to 26.2)	(13.2–34.7)	79.7
	Subdural haematoma/haemorrhage ≤3†	5	118/211	55.9 (49.2 to 62.5)	(49.2–62.5)	0.0
	Retinal haemorrhages ≤4†	4	60/74	81.1 (70.5 to 88.5)	(70.5–88.5)	0.0
	Upper limb fractures ≤3	4	249/298	38.5 (11.3 to 75.6)	(1.6–96.1)	94.9
	Lower limb fractures ≤3	6	90/272	24.0 (12.5 to 41.1)	(4.7–66.8)	82.0
	Rib fractures ≤4	4	90/126	88.3 (55.2 to 97.9)	(17.7–99.6)	84.7
	Multiple burns ≤2	3	70/261	22.5 (8.4 to 47.7)	(3.3-.70.9)	84.3
	Assaults ≤19	6	218/331	77.0 (46.6 to 92.7)	(19.1–97.9)	97.1
	Assaults/CM/adversity ≤19	6	1893/4033	55.9 (37.0 to 73.3)	(14.4–90.5)	98.6
	Women predominantly of reproductive age
IPV	IPV primary diagnosis ≤55	3	1946/2438	86.1 (72.2 to 93.6)	(53.0–97.1)	96.3
	Ocular injuries ≤50†	4	22/668	3.3 (2.2 to 5.0)	(2.2–5.0)	0.0
	Facial fractures ≤50	4	58/1550	11.1 (3.0 to 33.9)	(0.6–70.5)	94.2
	Upper body contusions ≤50	3	85/325	26.5 (18.1 to 37.0)	(13.7–44.9)	64.9
	Assaults ≤65	10	6073/20919	31.6 (22.3 to 42.7)	(9.2–67.9)	99.4

Each pooled positive predictive value represents the overall logit transformed estimate by indicator and outcome. Pooled estimates are based on random effects meta-analyses, unless otherwise specified. Individual forest plots are shown in the supplement.

*A 95% prediction interval is a measure of between-study variation and approximates where the PPV is to be expected for 95% of similar future studies.

†Estimates based on a fixed-effects meta-analysis due to lower between-study heterogeneity (ie, I ² <50%).

CM, child maltreatment; EHR, electronic health record; FAS, fetal alcohol syndrome; IPV, intimate partner violence; NAS, neonatal abstinence syndrome; PPV, positive predictive value; Ref std, reference standard.

Neonatal abstinence syndrome

The pooled PPV of primary ICD diagnoses for NAS was 80.9% (95% CI 71.0% to 87.9%), and the PPV ranged from 31.8% to 98.2%, with substantial between-study heterogeneity. Most studies (54%) used the Finnegan scale (a validated 21-item scale for documented symptom severity) to determine the accuracy of the coded diagnosis,¹¹⁸ with a cut-off score of 8 as a reference standard (except for one study, cut-off score of 4).⁴¹ The remaining studies used recorded clinician diagnosis or required pharmacological NAS treatment as reference standards.

Fetal alcohol syndrome

For FAS, we found a low pooled PPV of 39.3% (95% CI 25.3% to 55.4%), and the PPV ranged from 14.0% to 66.9%, with large between-study heterogeneity. All studies used ICD-9 codes to identify cases (eg, alcohol affecting fetus) and required 3–5 prespecified criteria to be met in chart reviews, including facial anomalies, prenatal growth deficiencies and maternal alcohol exposure (online supplementary table S6).

Child maltreatment

The pooled PPV of primary diagnoses for CM was 87.8% (95% CI 83.4% to 91.2%) among children aged 0–18 years across 19 studies, with significant between-study heterogeneity. Individual PPVs from studies ranged from 65.0% to 100.0%.

Overall, 37 studies assessed 11 different indicators of CM (online supplementary figures S3-S14). The pooled PPVs ranged from 88.3% (95% CI 55.2% to 97.9%) for rib fractures to 19.6% (95% CI 8.9% to 37.9%) for multiple burn injuries in children under 5 years. The between-study heterogeneity ranged from small to large (I² range 0.0%–98.5%). Four studies also assessed the PPV of poisonings (range 7.0%–95.0%),78 88 96 98 but the extreme heterogeneity precluded reliable pooling (prediction interval: 0%–100%). The majority of studies assessing injury indicators of CM actively excluded transport injuries (28 studies, 65%), metabolic bone diseases (10 studies) and birth injuries (9 studies) from the CM group.

Subgroup analyses for NAS, FAS and CM

Details of each subgroup analysis are provided in online supplementary tables S9–S11. For primary diagnoses of CM, we found significantly higher PPVs of CM in studies from inpatient settings (PPV=90.6%) compared with studies from emergency departments (EDs; PPV=80.8%) and in studies that applied a lower rated reference standard (89.4%; a rating <4) relative to a higher rated reference standard (70.8%; a rating ≥4). We found no significant differences when comparing subgroups by coding systems (ICD-9 vs ICD-10), age (younger vs older children) or publication year (p>0.179).

Intimate partner violence

Three studies assessed primary ICD diagnoses of IPV among women presenting to EDs (main age range: 12–55 years), with a pooled PPV of 86.1% (95% CI 72.2% to 93.6%). Individual PPVs from studies ranged from 73.6% to 94.4%. Two studies used the Flitcraft criteria as a reference standard (eg, prespecified IPV criteria) and included both violence by ex and current partners as cases. One study from Hong Kong included only IPV within cohabiting couples as cases and used medical chart reviews to verify that diagnoses matched any documented disclosure of IPV.¹¹⁵

We found 16 studies (10 in EDs/6 inpatient) that assessed four different injury-related presentations of IPV among predominantly women (primary age range: 12–55 years). The pooled PPVs were low, ranging from 31.6% (22.3%–42.7%) for assault-related codes to 3.3% (2.2%–5.0%) for orbital floor fractures. All studies used ICD-9 codes to identify presentations, except for one Finnish study that used ICD-10 codes.¹⁰⁸ The between-study heterogeneity ranged from low to substantial across indicators (I ² range=0.0%–99.4%). For assaults and facial fractures, meta-regressions showed that more recent studies were associated with a lower PPV of IPV (p<0.001). Three studies involved a small proportion of men (4%–17%),^{108 113 114} as estimates could not be separated from women. No study focused on pregnant or elderly women.

Coding errors for NAS, CM and IPV

The proportion of misclassifications (false positives) due to coding errors were on average 2.1% (95% CI 0.8% to 5.6%) across nine studies on NAS, CM, IPV and assaults (range I ²=97.3%–99.7%; online supplementary figure S20).

Documentation quality of assault indicators for IPV

We found that on average 28.0% (95% CI 14.7% to 46.8%; 18074/31224 charts; six studies) of assault coded cases among women had no recorded perpetrator information in the underlying medical charts (ie, missing data), preventing any classification/coding of IPV (online supplementary figure S21; table S5).

Sensitivity analyses

Serially excluding individual studies revealed that no study significantly impacted the between-study heterogeneity, and it remained substantial across all pooled PPVs (online supplementary tables S9-S10).

Publication bias

The funnel plot of studies reporting on the PPV for primary CM diagnoses was asymmetric (Egger’s test p=0.001, slope=0.999, rank correlation test: p=0.248; online supplementary figure S23), meaning that studies with higher PPVs were potentially more likely to be published. We found no evidence of funnel plot asymmetry for all other indicators with at least 10 or more studies (Egger’s test: p>0.366, rank correlation test: p>0.190; online supplementary figures S22-25).

Discussion

This is the largest meta-analysis to investigate the predictive value of indicators for family violence in EHRs, involving over 3.8 million individuals across 11 countries. Despite the large between-study heterogeneity, the results highlight that EHRs provide consistently high PPV for CM or IPV. We found that more than 8 in 10 coded primary diagnoses were confirmed as cases of NAS, CM and IPV. The findings also indicate that 8 in 10 recorded rib fractures and retinal haemorrhages met criteria for CM, and 1 in 3 assault-related presentations among women met criteria for IPV. Given the consistent recommendations for improved surveillance of violence,¹¹⁹ our findings underscore the utility of using commonly available coded medical data to evaluate services for at-risk groups across the life course. However, estimates varied depending on indicator and outcome, with substantial heterogeneity.

Compared with all other indicators of CM, FAS showed the lowest PPV. This most likely reflects the poor availability of specific ICD-9 codes for FAS, along with the complexity of the diagnosis. All included studies used the ICD-9 code ‘760.71’ focusing on fetuses being affected by alcohol without further description. Yet, the applied reference standard across studies required additional FAS criteria to be met, including facial anomalies and growth deficiencies.^46–52 There is also an absence of FAS criteria that are widely recognised by clinicians,¹²⁰ which may explain a higher underlying rate of misclassifications. Further studies on the accuracy of specific FAS codes using the ICD-10 are needed and may yield higher validity.

The high PPV of coded high-risk injuries of CM aligns with findings from previous reviews based on clinician diagnoses across paediatric healthcare settings. Compared with Kemp et al’s meta-analyses of 32 studies on fractures,²⁹ our results showed higher PPVs for rib fractures (88.3% vs 70.9%) and similar estimates for lower limb fractures but lower estimates for skull (22.1% vs 30.1%) and upper limb fractures (38.5% vs 47.6%). Our PPV for retinal haemorrhages and burn injuries were also consistent with previous reviews of children across settings.^{121 122} While there are significant methodological differences between this study and previous work (eg, overall age criteria), the consistency between findings suggests that some coded injury patterns could be considered as a broader measure of CM in EHRs to aid identification of high-risk groups.

Coded injury patterns of IPV, such as orbital fractures, provided relatively low PPVs. This is not surprising as the PPV is related to the underlying prevalence of IPV. For example, most studies on IPV were conducted in large populations (eg, EDs) and investigated injury patterns applicable to a wide range of causes. Women are also known to under-report IPV because of safety reasons and stigma.^{123 124} The under-reporting is consistent with the findings that more than 1 in 4 pooled female assault coded records were missing perpetrator information in the underlying medical charts. Still, broader assault-related presentations and upper body contusions showed higher PPVs of IPV (>25%),¹²⁵ and their utility in combination with other risk factors might yield comparable predictive accuracy in further studies.

Limitations

This review has important limitations. First, NAS and FAS are not always recognised as forms of prenatal neglect or CM, and the categorisation should be considered with caution to prevent stigma and barriers to help-seeking. Women who misuse substances may be unaware of their pregnancy, and opioids can be prescribed during pregnancy by clinicians for pain or opioid addiction treatment, increasing the risk of NAS.

Second, PPVs were analysed without accounting for study prevalences and within-study correlations of family violence.¹²⁶ As a result, pooled estimates from studies with higher underlying prevalence estimates will generally lead to higher PPVs. However, we aimed to minimise variation of underlying prevalences by only including studies from general hospitals or paediatric settings.

Third, we were unable to obtain adequate data to reliably pool estimates on specificity and sensitivity. However, these absolute accuracy measures were not the focus of this review. The high volume of eligible patients presenting to healthcare combined with the rare occurrence and under-reporting of all outcomes limits the feasibility to apply a reference standard to non-coded cases to ascertain false negatives. Reliable measures on sensitivity are therefore unlikely to be obtained.

Fourth, the between-study heterogeneity of the pooled PPVs was substantial. As in most large meta-analyses, several factors not examined in the subgroup analyses may have influenced our results. Many estimates were ascertained from lower quality reference standards, prone to circularity bias (ie, using suspicion of CM as a reference standard in the absence of other explanations).¹²⁷ Still, the consistency of studied ICD codes, combined with the larger samples and the high PPV across indicators, suggests that these estimates are valid and merit further study.

Finally, identifying CM and IPV in practice is complex. Obtaining information is a difficult task, as patients often under-report their experiences or symptoms, and high-risk groups such as infants cannot communicate. Similarly, some symptoms addressed by the reference standard (eg, linkage to social service assessments) may not have been conveyed to the clinician. EHRs may thus lead to potentially missed diagnoses or misclassifications. Estimates of indicators, therefore, reflect the ‘best data’ available and should be viewed in terms of routinely recorded indicators to help inform decisions about the likelihood of abuse, rather than definitive diagnoses.

Implications

More than a billion children and women aged 0–45 years globally reported being victims of abuse in 2014.^{128 129} Yet, in the UK, studies show that only about 1 in 3 violence-related ED visits for children and adults appear in police records, and self-report studies reveal that 1 in 5 affected women feel reluctant to report abuse to healthcare.^{3 130 131} In response to WHO’s priorities on addressing gaps in violence prevention, our findings highlight the potential to improve targeted care using routine EHRs to identify, prevent and support high-risk groups of family violence. On a service level, coded indicators of co-occurring family violence have the potential to be incorporated into computerised clinical decision support systems or risk prediction models to flag potential at-risk individuals.¹³² In the UK, linkage of family members EHRs could also allow for a ‘Think-Family’ approach, where indicators have the potential to identify vulnerable children through mothers or vice versa.¹³³ Despite these potential implications, it remains unknown whether the benefits of using automated EHR systems to identify at-risk individuals outweigh potential harms including stigma, legal consequences, trust and reduced help-seeking.