Assessment of the Accuracy of Identification of Selected Disabilities and Conditions in Hospital Discharge Data for Pregnant Women

Abstract

Background

Linked birth certificate–hospital discharge records are a valuable resource for examining pregnancy outcomes among women with disability conditions. Few studies relying on these data have been able to assess the accuracy of identification of pre-existing disability conditions. We assessed the accuracy of International Classification of Diseases version 9 (ICD9) codes for identifying selected physical, sensory, and intellectual conditions that may result in disability. As ICD9 codes were utilized until recently in most states, this information is useful to inform analyses with these records.

Methods

We reviewed 280 of 311 (90%) medical records of pregnant women with disabilities based on ICD9 codes and 390 of 8,337 (5%) records of pregnant women without disabilities who had deliveries at a large university medical center. We estimated sensitivity, specificity, and positive predictive values (PPV) using the medical record as gold standard. We adjusted for verification bias using inverse probability weighting and imputation.

Results

The estimated sensitivity of ICD9 codes to identify women with disabilities with deliveries 2009–2012 was 44%; PPV was 98%, improving over time. Although sensitivity was <50% for some conditions, PPVs were 87%−100% for all conditions except intellectual disability (67%). Many physical conditions had complete verification and no underreporting.

Conclusions

These results are helpful for new studies using historical data comparing outcomes among women with and without these conditions, and to inform interpretation of results from earlier studies. Assessment of the accuracy of disabilities as identified by ICD version 10 codes is warranted.

Introduction

Linked birth certificate–hospital discharge records provide a valuable resource for examining conditions among women with deliveries and their infants.^1–3 They do not rely on self-report and are population-based. International Classification of Diseases (ICD) codes in hospital discharge data allow screening for specific conditions during pregnancy⁴ or maternal and child outcomes^5,6 with greater accuracy than birth certificates alone,^7,8 and measurement of conditions not included in birth certificates.^9–12 They are useful in studies requiring linkages to other files, or historical pregnancy information.^13–15

Inference from ICD codes relies on their accuracy for identifying persons with the conditions under study. For some conditions, such as cardiovascular conditions,^16,17 ICD codes may with high sensitivity and positive predictive value (PPV) distinguish affected and non-affected persons. Few studies relying on hospital discharge data have assessed accuracy of the condition under examination. For analyses of pregnancy outcomes or conditions using linked birth-hospital discharge data, inaccuracies are compounded by reliance on the presence of relevant codes during delivery hospitalization only, making identification of non-pregnancy/delivery-related conditions problematic.

Such linked records are important for assessing pregnancy outcomes among women with physical or other disabilities, given their increasing numbers,¹⁸ and the lack of relevant population-based registries. To inform these analyses using linked birth-hospital discharge records for women with selected conditions (physical, sensory, intellectual) that may result in disability, we reviewed medical records to assess accuracy of identifying these women based on ICD version 9 (ICD9) codes.

Methods

Live birth/fetal death records in Washington State are routinely linked to hospital discharge records for non-federal facilities. This enriched file includes birth and/or fetal death and hospital discharge records for the mother’s and infant’s delivery hospitalization. ICD9 codes identified all women with selected physical, sensory, or intellectual conditions with deliveries from 1987 to 2012 (N=4,227) (Table 1). We tried to encompass conditions identifying persons with limitations in movement, sensory, or cognitive or intellectual functioning generally included in studies of women with disability.^18–21 These were categorized as physical disability (rheumatoid arthritis, multiple sclerosis, cerebral palsy, paralysis/hemiplegia, spina bifida, spinal cord injury, myasthenia gravis, muscular dystrophy, ankylosing spondylitis, stroke late effects, and polio late effects); sensory conditions (visual impairment/blindness, hearing impairment/deafness); and intellectual disability.

Table 1.

ICD9 codes used to identify selected disabilities in hospital discharge records.

Condition	ICD codes
Physical disabilities
Multiple sclerosis	340
Cerebral palsy	343.x
Rheumatoid arthritis	714.x, 725.x
Ankylosing spondylitis	720.0
Spina bifida	741.x
Paralysis/hemiplegia	344.x, 342.x
Spinal injuries	806.x, 905.1, 907.x, 952.x, 953.x
Stroke – late effects	438.x
Polio (w/ paralysis, spinal, +late effects)	045.1x, 045.0x, 045.9x, 138.x
Myasthenia gravis	358.x
Muscular & myotonic dystrophies	359.x
Chondrodystrophy	756.4
Intellectual disability	317.x – 319.x
Sensory disabilities
Hearing impairment/deafness	389.x
Visual impairment/blindness	369.x, 950.x

ICD=International Classification of Disease

With human subjects’ protection board approvals, we attempted to review electronic medical records for all deliveries at a high-level maternity care hospital to assess the accuracy of hospital discharge data to identify pregnant women with these conditions during 1987–2008 (N=210) and 2009–2012 (N=101). Ultimately 179 (85%) and 101 (100%) were reviewed from each time period, respectively. From among 8,337 women without these conditions with deliveries at that same hospital during 2009–2012, we identified and reviewed 390 randomly selected records. We were unable to review comparison records for the earlier time period.

We estimated the validity of identifying women with any disability; with physical, sensory, or intellectual disability; and with specific conditions. For deliveries 1987–2008 we estimated the PPV (proportion of women with a positive medical record among those with positive ICD codes for the condition). For deliveries 2009–2012, we additionally estimated, using the medical record as gold standard, the sensitivity (proportion of women identified by ICD9 codes among those with a disability documented in the medical record) and specificity (proportion of those without disability correctly identified among those without the disability per the medical record). A woman was considered to have a condition if it was indicated in her medical or obstetric record prior to/around the time of delivery hospitalization (by mention of earlier diagnosis or presence of chronic or congenital conditions). Occurrence of acute conditions (e.g., stroke) during delivery hospitalization was not considered verification of a pre-existing condition.

We used inverse probability weighting and imputation to adjust for verification bias assuming that results of medical records reviewed represented results that would have been observed if all records had been reviewed.²² To estimate the total expected number of women with a correctly identified disability per the medical record, we multiplied each cell in the 2 × 2 table by the inverse probability of the chart being reviewed and either being correctly identified or incorrectly identified in the medical chart (depending on the cell).²² For example, if 162/179 records during 1987–2008 of women with disability were validated by medical record, we multiplied 162/179 by the total number of women with disability with deliveries during that time period (210) to impute the expected number of women with correctly identified disability, had all 210 charts been reviewed. Thus, the expected number of true positives would be 162/179 × 210=190; the expected number of false positives would be 17/179 × 210 = 20; and the PPV would be 190/210 × 100 = 91%.

Results

The estimated PPV for using ICD9 codes to identify women with any disability increased from 91% during 1987–2008 to 98% during 2009–2012 (Table 2). PPVs for all specific conditions also improved with lowest PPV during 1987–2008 of 62% for sensory disability, improving to 98% in 2009–2012. In 1987–2008, the PPV was 63% for intellectual disability, but >78% for the remaining conditions with several >90%.

Table 2.

Agreement between hospital discharge and medical record (MR) reporting of selected disability conditions among women with deliveries at a teaching hospital, by time period.

	1987–2008				2009–2012
	Deliveries with ICD code for condition				Deliveries with ICD code for condition				Deliveries without ICD codes for conditions (N=390)a		Validity estimates
Condition	No.	Reviewed	Noted in MR	PPV	No.	Reviewed	Noted in MR	Noted %	Noted in MR	Noted %	Sensitivity	Specificity	PPV
Any disability condition	210	179	162	91	101	101	99	98	6	1.5	44	100b	98
Any physical disability	159	143	129	90	58	58	57	98	3	0.8	47	100b	98
Any sensory disability	39	29	18	62	41	41	40	98	4	1.0	32	100b	98
Intellectual disability	13	8	5	63	3	3	2	67	0	0.0	100	100b	67
Specific conditions
Rheumatoid arthritis	39	32	25	78	14	14	14	100	1	0.3	40	100	100
Multiple sclerosis	25	22	21	96	10	10	10	100	0	0.0	100	100	100
Chondrodystrophy	3	3	3	100	1	1	1	100	0	0.0	100	100	100
Cerebral palsy	10	9	9	100	2	2	2	100	0	0.0	100	100	100
Paralysis/hemiplegia	22	21	17	81	8	8	7	88	0	0.0	100	100b	88
Spina bifida	13	12	11	92	5	5	5	100	1	0.3	19	100	100
Spinal injury sequelae	7	7	6	86	2	2	2	100	0	0.0	100	100	100
Myasthenia gravis	13	13	12	92	4	4	4	100	0	0.0	100	100	100
Muscular dystrophy	8	7	7	100	2	2	2	100	0	0.0	100	100	100
Ankylosing spondylitis	9	9	8	89	2	2	2	100	0	0.0	100	100	100
Stroke – late effects	11	10	9	90	5	5	5	100	1	0.3	19	100	100
Polio - late effects	7	6	6	100	3	3	3	100	0	0.0	100	100	100
Visual impairment/blindness	9	7	7	100	10	10	10	100	1	0.3	32	100	100
Hearing impairment/deafness	30	22	21	96	31	31	30	97	3	0.8	32	100b	97

^a390 records reviewed for an estimated 8,337 total deliveries at this facility during 2009–2012 (2.38% of 2,093,710 total live birth/fetal death deliveries to women without the selected conditions in the state).

^bValue prior to rounding is 99.9%.

During 2009–2012, sensitivity estimates were 44% for any disability, 47% for physical, and 32% for sensory disabilities; specificities were >99%. Sensitivity was also low for rheumatoid arthritis, spina bifida, stroke, blindness, and deafness. Specificities for identifying all conditions were ≥99%. PPV estimates were >87% for all conditions except intellectual disability (67%).

Discussion

There is a dearth of information on the accuracy of ICD9 codes for most disability conditions, and we identified no studies focused on accuracy among pregnant women. Overall specificity (>99%) was high, but sensitivity was low (44%). Specificity was >99% for all specific conditions but sensitivity was <50% for sensory disabilities, rheumatoid arthritis, spina bifida, and stroke. These findings have important implications for using ICD9 codes to identify pregnant women with these conditions.

Low sensitivity can be improved by combining multiple codes and increasing the number of code fields from discharge records or clinic visits. The improvement in our PPV estimates over time may reflect this, given the increase in ICD diagnosis codes within the State hospital discharge record in 1994 from 5 fields to 9 fields. Norwegian hospital discharge records verified only 28.5% of patients with a single ICD9 code for traumatic spinal cord injury, but when screening included three relevant ICD9 codes for spinal cord injury (806.x, 907.2, and 952.x) in several fields of a hospital discharge record, 34.7% were verified.²³ That this value is lower than what we observed could be attributable to the timing of the code assignment, as only patients hospitalized immediately or soon after a trauma event were included; thus, they assessed accuracy of ICD codes for identifying incident spinal cord injury. We assessed accuracy of ICD9 codes for identifying prevalent disability due to spinal cord injury. Our accuracy may also be greater due to obstetrical providers’ awareness of challenges in caring for women with decreased mobility during multiple prenatal care visits vs. potential uncertainty in spinal cord injury diagnoses soon after trauma.

Screening codes across multiple visits improves accuracy. A recent study found a single ICD9 discharge code for muscular dystrophy accurately predicted muscular dystrophy 48.4% of the time, whereas the accuracy of ten ICD9 codes accumulated over 10 outpatient or inpatient visits was 96%,²⁴ similar to the high level of accuracy observed in our data. However, not all conditions have multiple associated codes, and combining codes from multiple visits is not always feasible. When the target population consists of women with deliveries, screening codes across multiple visits would require longitudinal linkage of hospital discharge data to records of prior hospitalizations (which many women do not have) or to records of previous outpatient visits which is beyond the scope of many studies.

In our study, specificity was high for all conditions, supporting use of ICD9 codes for identification of pregnant women with these disabilities. Non-differential misclassification of dichotomous exposure generally results in attenuated measures of association.²⁵ The extent to which this bias occurs depends on disease prevalence and specificity, such that bias decreases as both prevalence and specificity increase.²⁶ Imperfect sensitivity is a concern with regard to precision, because it results in a reduced number of study subjects, but can be compensated for by increasing the control:case ratio.²⁶ In addition, the odds ratio is biased to a greater extent by the false positive rate than by the false negative rate.²⁷ Thus, specificity, rather than sensitivity, is most crucial for validity.

Results of our project may be used to calculate bias-adjusted risk estimates from prior studies with these data. Methods exist for using sensitivity and specificity of exposure measurement to calculate bias-adjusted risk estimates,²⁸ which may result in substantively different conclusions,²⁹ emphasizing the importance of understanding the accuracy of exposure ascertainment. We applied formulas for bias-adjusting relative risks (RRs) to selected results of earlier analyses of pregnancy outcomes among women with intellectual disability using our data.⁹ This method entails calculating the bias-adjusted proportions of exposed and non-exposed individuals in the diseased and non-diseased groups using the observable proportions in each category in combination with the sensitivity and specificity of the classification. The unadjusted RRs of cesarean section and preeclampsia among women with intellectual disability were 1.22 and 2.18, respectively. Applying the sensitivity and specificity results from our study (assuming these are similar for cases and comparison women), the corresponding estimates would have been 1.33 and 2.45.

Our study has several limitations. We examined deliveries at a large teaching hospital; women cared for at high-level obstetric facilities include greater proportions of high-risk pregnancies, and it is likely that accuracy of reporting varies by type of facility, or that disability diagnoses documented at other facilities were not entered into medical records at the delivery hospital if pre-delivery care occurred elsewhere. Although there is little research regarding the prevalence among women of reproductive age, self-reported disability (typically defined as difficulty with functional or daily living activities, use of assistive aid such as a wheelchair or crutches, or limitations in ability to work at job or around the house) in reproductive age women is approximately 12%.³⁰ Because PPV depends on prevalence, and due to differences in how disability is defined, our PPV results may not be generalizable to the population at large. In addition, our sample size was small for several conditions. Finally, the weighting method we used assumes reviewed records represented what we would observe had all records been reviewed; we assumed the probability of a medical record being reviewed was not associated with the probability of observing a code in that chart. If non-reviewed charts were less likely to confirm case status, our calculated sensitivity and specificity would be spuriously high, resulting in a conservative estimate of the bias-adjusted odds ratio. In contrast, if non-reviewed charts were more likely to confirm case status, our calculated sensitivity and specificity would be spuriously low, and corrections would yield inflated risk estimates. Despite these limitations, linked birth-hospital discharge records are valuable for population-based research including populations less likely to enroll in clinical studies, and able to explore conditions without medical record reviews or clinical examination.

Although ICD9 has been replaced by ICD10 coding, knowing the accuracy of ICD9 codes is helpful for studies using historical data, especially if only one hospital discharge record is available for screening. The high specificity of ICD9 codes used in our study supports the validity of past research using these codes. However, studies using codes we identified to have low sensitivity may suffer from lack of precision. As “Big Data” cohorts become more commonplace in epidemiologic research (e.g, the National Institutes of Health “All of Us” project),³¹ reliance on ICD codes for outcome assessment may increase.³² Given the utility of ICD codes for conducting research among pregnant women with disabilities, future research should assess the accuracy of disabilities as identified by ICD10 codes; given their expanded fields, improved accuracy is expected.