The Validity of Discharge Billing Codes Reflecting Severe Maternal Morbidity

Abstract

BACKGROUND

Discharge diagnoses are used to track national trends and patterns of maternal morbidity. There are few data regarding the validity of the International Classification of Diseases (ICD) codes used for this purpose. The goal of our study was to try to better understand the validity of administrative data being used to monitor and assess trends in morbidity.

METHODS

Hospital stay billing records were queried to identify all delivery admissions at the Massachusetts General Hospital for the time period 2001 to 2011 and the University of Michigan Health System for the time period 2005 to 2011. From this, we identified patients with ICD-9-Clinical Modification (CM) diagnosis and procedure codes indicative of severe maternal morbidity. Each patient was classified with 1 of 18 different medical/obstetric categories (conditions or procedures) based on the ICD-9-CM code that was recorded. Within each category, 20 patients from each institution were selected at random, and the corresponding medical charts were reviewed to determine whether the ICD-9-CM code was assigned correctly. The percentage of correct codes for each of 18 preselected clinical categories was calculated yielding a positive predictive value (PPV) and 99% confidence interval (CI).

RESULTS

The overall number of correctly assigned ICD-9-CM codes, or PPV, was 218 of 255 (86%; CI, 79%–90%) and 154 of 188 (82%; CI, 74%–88%) at Massachusetts General Hospital and University of Michigan Health System, respectively (combined PPV, 372/443 [84%; CI, 79–88%]). Codes within 4 categories (Hysterectomy, Pulmonary edema, Disorders of fluid, electrolyte and acid–base balance, and Sepsis) had a 99% lower confidence limit ≥75%. Codes within 8 additional categories demonstrated a 99% lower confidence limit between 74% and 50% (Acute respiratory distress, Ventilation, Other complications of obstetric surgery, Disorders of coagulation, Cardiomonitoring, Acute renal failure, Thromboembolism, and Shock). Codes within 6 clinical categories demonstrated a 99% lower confidence limit <50% (Puerperal cerebrovascular disorders, Conversion of cardiac rhythm, Acute heart failure [includes arrest and fibrillation], Eclampsia, Neurotrauma, and Severe anesthesia complications).

CONCLUSIONS

ICD-9-CM codes capturing severe maternal morbidity during delivery hospitalization demonstrate a range of PPVs. The PPV was high when objective supportive evidence, such as laboratory values or procedure documentation supported the ICD-9-CM code. The PPV was low when greater judgment, interpretation, and synthesis of the clinical data (signs and symptoms) was required to support a code, such as with the category Severe anesthesia complications. As a result, these codes should be used for administrative research with more caution compared with codes primarily defined by objective data.

Recent analysis of available data within the United States health care system indicates a steady increase in severe maternal morbidity and mortality since the 1990s.^1–5 Continued effort is aimed at understanding the causes of these trends by monitoring adverse health events not resulting in death^6–8 and developing tools such as prediction models to better identify and care for those with increased health risk.^9,10 Clinical researchers, public health epidemiologists, health care economists, and hospital administrators frequently use the International Classification of Diseases (ICD) codes documented in hospital discharge billing data to identify cases with severe maternal complications.

Administrative data have also been used increasingly to help answer questions in clinical epidemiology and comparative effectiveness research; however, results may be affected by the quality of the underlying administrative data used for such purposes.¹¹ At present, little is known about the validity of ICD codes representing severe maternal morbidity. Therefore, studies that assess ICD code validity are essential to substantiate the quality or identify limitations of utilized administrative data. We aimed to define the positive predictive value (PPV) of ICD codes used to identify maternal morbidity in epidemiologic studies in a sample of delivery admissions drawn from 2 large academic medical centers.

METHODS

For this study, we queried billing records to identify all obstetric deliveries at the Massachusetts General Hospital (MGH) from 2001 to 2011 and at the University of Michigan Health System (UMHS) from 2005 to 2011. Time periods reflect the availability of electronic patient billing data based on when each institution’s electronic billing infrastructure was established. This study was approved by the Partners Institutional Review Board, Boston, Massachusetts (2011P001432) and the University of Michigan Institutional Review Board, Ann Arbor, Michigan (IRB00001999). The requirement for written informed consent was waived by the institutional review boards.

The MGH and UMHS are teaching hospitals located in Boston, Massachusetts, and Ann Arbor, Michigan, respectively. The MGH is the largest teaching hospital of Harvard Medical School, with 950 inpatient beds and approximately 48,000 inpatients annually, including approximately 3500 deliveries annually.^12,13 The UMHS is one of the largest teaching hospitals in the Midwest, with 993 inpatient beds and approximately 45,000 inpatients annually, including approximately 3800 deliveries annually.^14,15

From the population of obstetric deliveries at each institution, we identified deliveries with ICD, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes indicative of severe maternal morbidity. We then grouped these deliveries into 1 of 18 preselected clinical categories of severe maternal morbidity (conditions or procedures) based on the ICD-9-CM code that was recorded.

Informed by studies that have previously used health care utilization data to describe the patterns of severe maternal morbidity,^5–7 the authors focused on 18 clinical categories felt to reflect high acuity and the most severe maternal morbidity and end-organ injury. We reviewed the codes used to define these clinical categories in previously published algorithms^5–7 and supplemented the codes with those identified based on our own review of the 2012 ICD-9-CM coding manual. The 18 clinical categories included in our analysis, and the ICD-9-CM codes used to comprise these categories, are shown in Table 1.

Table 1.

ICD-9-CM Diagnosis and Procedure Codes Used to Describe Severe Maternal Morbidity

Maternal Morbidity Categories	ICD-9-CM Codes Reviewed
Acute heart failure (includes arrest and fibrillation)	99.60, 99.63, 415, 427.41, 427.42, 427.5, 428.1, 428.21, 428.23, 428.31, 428.33, 428.41, 428.43, 997.1
Acute renal failure	584, 669.3
Acute respiratory distress	518.0, 518.5, 518.81, 518.82, 518.84, 799.1
Cardiomonitoring	89.6
Conversion of cardiac rhythm	99.6
Disorders of fluid, electrolyte, and acid–base balance	276
Disorders of coagulation	286.6, 286.7, 286.9, 287.4, 649.3, 666.3
Eclampsia	642.6
Hysterectomy	68.3–68.9
Neurotrauma	801, 803–805, 851–854
Other complications of obstetric surgery	669.4
Puerperal cerebrovascular disorders	325, 346.6, 348.1, 348.3, 348.5, 430–432, 433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.01, 434.11, 434.91, 436, 437 674.0, 671.5, 997.02
Pulmonary edema	428.1, 514, 518.4
Sepsis	038, 995.91, 995.92
Severe anesthesia complications	668.0–668.9, 995.86
Shock	669.11, 785.5, 995.0, 995.4, 995.94, 998.0, 999.4
Thromboembolism	415.1, 673.0, 673.1, 673.2, 673.3, 673.8
Ventilation	93.90, 96.03–96.05, 96.7

Unless specifically listed, all subclassifications were included in the data query for higher order codes.

Abbreviation: ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification.

Data Source

At the MGH, we used the Partner Enterprise Master Patient Index with records from January 1, 2001, through December 31, 2011. We queried this patient billing database for all obstetric records using ICD-9-CM delivery code V27.x and, concomitantly, ICD-9-CM codes representing maternal morbidity (as shown in Table 1). All records represented events during the delivery hospitalization. We queried the University of Michigan Data Warehouse database using the same methodology for the period of January 1, 2005, through December 31, 2011. These selected records served as our sampling frame.

Sample Selection Methodology

At each institution, within each of the 18 clinical categories, 20 patients were chosen randomly (Rand function, Microsoft Excel, version 14.4.2, Microsoft Corp, Redmond, WA) and the associated medical chart reviewed in detail for supportive clinical information. If the number of patients within the preselected category was ≤20, all patients and their associated records were reviewed. If a patient’s medical chart was missing or unavailable, an alternate patient within the same morbidity category was selected randomly (Rand function) for review, if available. Finally, the results from each institution were combined for aggregate analysis.

Medical Record/Chart Review Methodology

Electronic and paper medical charts were used as a source of clinical information. At each institution, 2 physicians (MGH: M.S. and H.M.; UMHS: J.M. and B.R.) reviewed the billing record and medical chart associated with each code to determine correctness. Evidence within the patient’s medical chart, such as operative procedure note, laboratory data, and physician assessment, was reviewed in detail to substantiate the billing code assigned. Although both physician reviewers were aware of the maternal morbidity ICD-9-CM code before reviewing the record, neither physician was aware of the other’s assessment until the conclusion of data acquisition. If the 2 reviewers disagreed on whether the ICD-9-CM code correctly reflected the patient condition, a third physician (MGH: B.B.; UMHS: N.S.) rendered the definitive opinion as to whether the ICD-9-CM code was correct. For the MGH data set only, length of stay and intensive care unit (ICU) admission data also were collected.

In some cases, a patient’s billing record included >1 ICD-9-CM code of interest for our study sample. For example, a patient might require ventilation (96.7) for respiratory distress (518.5) and undergo invasive monitoring (89.6). In this circumstance, the case was eligible to be counted toward the validation analysis of each clinical category.

We analyzed each delivery record where the associated ICD-9-CM code was deemed incorrect by the above process and identified themes post hoc that summarized possible explanations for incorrect coding (categories A, B, and C).

To assess interrater consistency, we analyzed the rate of disagreement between the primary physician reviewers who examined ICD-9-CM code supportive evidence. We identified categories post hoc that summarized possible explanations for these disagreements (categories I and II).

All data for this study were recorded and analyzed in Microsoft Excel for Mac 2011, version 14.4.2 (Microsoft Corp, Redmond, WA) and were maintained in an encrypted computer with password access restricted to the study team. Paper medical charts were housed in a secure location within the institution’s medical records department. All patient identifiers, including date and year of delivery, were removed from the data set before analysis.

Statistical Analysis

Using the medical chart as the gold standard and comparing it with the identified billing records, we calculated the percentage of correct ICD-9-CM codes in each maternal morbidity category, resulting in a PPV and 99% confidence interval (CI). PPV is defined as the number of true positives (where the ICD-9-CM code from billing record and clinical diagnosis from medical record agree on a positive diagnosis) divided by the sum of all positives identified from parturient billing records, which include true positives and false positives. The Wilson Score interval was used to calculate each CI, and we emphasize the lower confidence limit given the wide intervals in our study.¹⁶ Analyses were performed on MGH and UMHS data sets individually and then combined for aggregate results.

For the MGH data set only, we performed a subgroup analysis, restricting codes within our sample frame to those associated with a hospitalization ≥3 days, ≥7 days, and ICU stay, and then calculated the PPV for the codes remaining in each maternal morbidity category. Although imposing a length of stay or ICU restriction decreases the number of cases available for assessment, it may increase the specificity of codes for surveillance efforts; codes for severe events, such as amniotic fluid embolism, cardiac arrest, or stroke, are typically associated with longer hospital stays.^7,17,18

Identified coding errors were categorized according to error type for analysis. Cohen’s κ coefficient was calculated to measure interrater agreement between the study teams’ physician chart reviewers.¹⁹ Disagreements between physician reviewers were investigated and categorized for analysis.

RESULTS

Sample Population

The query of hospital billing records for ICD-9-CM diagnosis and procedure codes of interest yielded 687 patient deliveries (N_MGH = 474; N_UMHS = 213; Figure 1) with ICD-9-CM codes reflecting maternal morbidity. As described previously, patients were sorted into 1 of the 18 preselected clinical categories by ICD-9-CM code. After we selected patients from each category, a total of 443 patient billing records and associated medical charts underwent a detailed review (N_MGH = 255; N_UMHS = 188). Eight patients selected had incomplete medical charts precluding analysis, and alternate patients were selected randomly (N_MGH = 7; N_UMHS = 1).

Figure 1.

Study population. The number in parentheses (N) indicates incomplete records for which alternates were selected. MGH indicates Massachusetts General Hospital; UMHS, University of Michigan Health System.

Positive Predictive Value

The overall number of correctly assigned ICD-9-CM codes, or PPV, was 218 of 255 (85%; CI, 79%–90%) and 154 of 188 (82%; CI, 74%–88%) at MGH and UMHS, respectively (combined PPV, 372/443 [84%; CI, 79%–88%]). The PPV and 99% CI for the codes reflecting each maternal morbidity category are listed in Table 2. Codes within 6 clinical categories demonstrated a 99% lower confidence limit <50% (Puerperal cerebrovascular disorders, Conversion of cardiac rhythm, Acute heart failure [includes arrest and fibrillation], Eclampsia, Neurotrauma, and Severe anesthesia complications). Codes within 8 additional categories demonstrated a 99% lower confidence limit between 74% and 50% (Acute respiratory distress, Ventilation, Other complications of obstetric surgery, Disorders of coagulation, Cardiomonitoring, Acute renal failure, Thromboembolism, Shock). Codes within 4 categories (Hysterectomy; Pulmonary edema; Disorders of fluid, electrolyte, and acid–base balance; and Sepsis) had a 99% lower confidence limit ≥75%.

Table 2.

Conditions Reflective of Maternal Morbidity Identified by ICD-9-CM Codes, Number of Cases Identified, Positive Predictive Value, and 99% CIs

	MGH				UMHS				Combined
Conditions	Query Total	Reviewed	Correct	PPV (%)	Query Total	Reviewed	Correct	PPV (%)	Query Total	Reviewed	Correct	PPV (%)	99% CI
Pulmonary edema	25	20	20	100	4	4	4	100	29	24	24	100	78%–100%
Sepsis	15	15	15	100	5	5	5	100	20	20	20	100	75%–100%
Hysterectomy	70	20	20	100	19	19	19	100	89	39	39	100	86%–100%
Cardiomonitoring	7	7	7	100	2	2	2	100	9	9	9	100	58%–100%
Conversion of cardiac rhythm	2	2	2	100	2	2	2	100	4	4	4	100	38%–100%
Disorders of fluid, electrolyte, and acid–base balance	107	20	20	100	40	20	18	90	147	40	38	95	78%–99%
Acute respiratory distress	28	20	18	90	9	9	9	100	37	29	27	93	72%–99%
Ventilation	19	18	17	95	20	20	18	90	39	38	35	92	74%–98%
Other complications of obstetric surgery	20	20	18	90	4	4	4	100	24	24	22	92	67%–98%
Shock	6	6	5	83	8	8	7	88	14	14	12	86	51%–97%
Disorders of coagulation	56	20	15	75	25	20	17	85	81	40	32	80	60%–91%
Acute renal failure	17	17	14	82	16	16	12	75	33	33	26	79	57%–91%
Neurotrauma	2	1	1	100	3	3	2	67	5	4	3	75	22%–97%
Thromboembolism	14	14	11	79	18	18	13	72	32	32	24	75	52%–89%
Acute heart failure (includes arrest and fibrillation)	6	6	4	67	8	8	6	75	14	14	10	71	38%–91%
Puerperal cerebrovascular disorders	33	20	17	85	18	18	9	50	51	38	26	68	48%–84%
Eclampsia	38	20	10	50	9	9	6	67	47	29	16	55	33%–76%
Severe anesthesia complications	9	9	4	44	3	3	1	33	12	12	5	42	15%–74%
Total	474	255	218	86	213	188	154	82	687	443	372	84	79%–88%

Abbreviations: CI, 99% confidence interval; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; MGH, Massachusetts General Hospital; PPV, positive predictive value; UMHS, University of Michigan Health System.

Results of a length of stay restriction on the MGH data set are provided in Table 3. Total PPV (the sum of all categories) is shown with each level of restriction: Total PPV_overall = 86% (CI, 79%–90%), total PPV_LOS>3days = 89% (82%–93%), total PPV_{LOS>7 days} = 92% (83%–97%), and total PPV_ICU 98% (89%–99%).

Table 3.

Effect of Length of Stay and ICU Restriction on Positive Predictive Value—MGH Data Only

	Overall PPV			LOS ≥ 3 d			LOS ≥ 7 d			ICU Admission
Conditions	Reviewed	Correct	PPV (%)	Reviewed	Correct	PPV (%)	Reviewed	Correct	PPV (%)	Reviewed	Correct	PPV (%)
Pulmonary edema	20	20	100	19	19	100	10	10	100	3	3	100
Sepsis	15	15	100	15	15	100	12	12	100	5	5	100
Hysterectomy	20	20	100	20	20	100	9	9	100	10	10	100
Cardiomonitoring	7	7	100	7	7	100	4	4	100	7	7	100
Conversion of cardiac rhythm	2	2	100	1	1	100	1	1	100	1	1	100
Disorders of fluid, electrolyte, and acid–base balance	20	20	100	16	16	100	4	4	100	4	4	100
Acute respiratory distress	20	18	90	20	18	90	13	11	85	6	6	100
Ventilation	18	17	95	17	17	100	11	11	100	15	15	100
Other complications of obstetric surgery	20	18	90	19	17	89	3	2	67	2	2	100
Shock	6	5	83	4	4	100	1	1	100	4	4	100
Disorders of coagulation	20	15	75	16	13	81	12	10	83	6	5	83
Acute renal failure	17	14	82	17	14	82	5	5	100	6	6	100
Neurotrauma	1	1	100	1	1	100	1	1	100	1	1	100
Thromboembolism	14	11	79	13	11	85	9	8	89	3	3	100
Acute heart failure (includes arrest and fibrillation)	6	4	67	5	3	60	3	2	67	3	3	100
Puerperal cerebrovascular disorders	20	17	85	13	12	92	2	2	100	4	4	100
Eclampsia	20	10	50	15	9	60	3	2	67	0	0	N/A
Severe anesthesia complications	9	4	44	7	3	43	1	1	100	2	1	50
Total	255	218	86	225	200	89	104	96	92	82	80	98
99% CI		79%–90%			82%–93%			83–97%			89%–99%

Abbreviations: CI, 99% confidence interval; ICU, intensive care unit; LOS, length of stay; MGH, Massachusetts General Hospital; PPV, positive predictive value.

Review of all 71 identified coding inaccuracies yielded 3 categories of errors (Table 4). A detailed description for each error is presented in Supplemental Digital Content 1, Table 1, http://links.lww.com/AA/B451. The most common error identified was lack of clinical or laboratory evidence supporting the condition (category A). Other error categories included history of related condition, but no clinical or laboratory evidence of defect impacting current pregnancy (category B), and confusion between possible or expected side effects and actual complications (category C).

Table 4.

Maternal Morbidity Categories and Description of Coding Errors

Maternal Morbidity Category	Total Errors	A	B	C
Disorders of fluid, electrolyte, and acid–base balance	2	2	0	0
Acute respiratory distress	2	2	0	0
Ventilation	3	3	0	0
Other complications of obstetric surgery	2	1	0	1
Shock	2	2	0	0
Disorders of coagulation	8	4	3	1
Acute renal failure	7	7	0	0
Neurotrauma	1	1	0	0
Thromboembolism	8	1	7	0
Acute heart failure (includes arrest and fibrillation)	4	4	0	0
Puerperal cerebrovascular disorders	12	5	7	0
Eclampsia	13	0	0	13
Severe anesthesia complications	7	2	0	5
	71	34	17	20

A, Lack of clinical or laboratory evidence supporting the condition; B, History of related condition, but no clinical or laboratory evidence of diagnosis impacting current pregnancy; C, Confusion between possible or expected side effects and actual complications. Total errors by category: A = 34, B = 17, and C = 20.

Disagreements Between Physician Reviewers

There were a total of 42 disagreements of 443 (9.5%) charts evaluated by the 2 primary physician reviewers (N_MGH = 33/255 [13.3%]; N_UMHS = 9/188 [4.8%]). Overall, Cohen’s κ statistic for the 2 primary reviewers was 0.80 (κ_MGH = 0.72; κ_UMHS = 0.90; Supplemental Digital Content 2, Table 2, http://links.lww.com/AA/B452). Disagreements occurred because data were either overlooked (category I) or misinterpreted (category II) (Supplemental Digital Content 3, Table 3, http://links.lww.com/AA/B453).

DISCUSSION

Hospital administrative data are an attractive data source because of wide availability and low cost; however, the codes used to record relevant conditions may be subject to error. Our analysis of supporting ICD-9-CM codes illustrates a range of PPVs for severe maternal morbidity categories. Codes demonstrated a high PPV when supporting medical information containing objective data, such as laboratory values, pathology results, or procedure documentation. The PPV was low when greater judgment, interpretation, and synthesis of clinical information (signs and symptoms) was required to support the ICD-9-CM code, such as with the category Severe anesthesia complications. These ICD-9-CM codes should be used for administrative research with more caution compared with codes primarily defined by objective data.

As illustrated in Supplemental Digital Content 1, Table 1, http://links.lww.com/AA/B451, errors in coding identified in this study corresponded to 1 of 3 categories. Lack of clinical or laboratory evidence supporting the condition (category A) was the most common reason for coding errors—either a code was placed erroneously or a charted medical condition misunderstood. Inaccuracies in coding for some disorders including Puerperal cerebrovascular disorders and Thromboembolism occurred when chronic conditions were miscoded as acute conditions and did not reflect a new peripartum morbidity (category B). For other conditions, most notably Severe anesthesia complications, Other complications of obstetric surgery, and Disorders of coagulation, inaccuracies occurred because of confusion between possible or expected side effects and actual complications (category C); examples related to Severe anesthesia complications are discussed below.

Coding for procedures demonstrated a high PPV such as with categories Hysterectomy (PPV, 100%; CI, 86%–100%) and Ventilation (PPV, 92%; CI, 74%–98%). Previous studies in which authors validated coding for obstetric and non-obstetric procedures also demonstrated high PPVs.²⁰ Conditions such as Sepsis or Disorders of fluid, electrolyte and acid-base balance also demonstrate a high PPV, perhaps because of the availability of objective data (medications, vital signs, and laboratory values), which too is consistent with previous validation studies.²¹

For conditions such as Eclampsia and Disorders of coagulation, chart review occasionally illustrated the presence of related diagnoses, but no evidence for the condition of interest. For example, patients with preeclampsia and no evidence of seizure were miscoded with eclampsia, reflecting misunderstanding of the definition or unintentional error. Our observation of low coding PPV for eclampsia is consistent with Geller et al²² in which the PPV of eclampsia was 41.7%.

The PPV for coding for anesthesia-related complication was also low in our study (PPV, 42%; CI, 15%–74%); however, this finding was based on the review of only 12 cases identified in our query, and therefore, the CI for our estimate is very wide. A previous study validation, in which the authors used a slightly different set of ICD-9-CM codes, reported a high PPV for codes representing obstetric anesthesia complications.²³ In our study, the observed inaccuracies in codes for anesthesia complications occurred because of confusion between possible or expected side effects and actual complications (category C). For example, 1 case was assigned pulmonary complications of anesthesia because a patient was intubated and sent to the ICU after developing postpartum hemorrhage in the operating room. Another case was assigned complications of anesthesia because the patient underwent an elective awake fiberoptic intubation for known difficult airway.

ICD-coding inaccuracies have been identified in other anesthesia conditions such as malignant hyperthermia, where only 25% of cases reviewed were true positives,²⁴ and obstructive sleep apnea.²⁵ If claims data are going to be used for the study of these complications, work is needed to improve the quality of documentation around anesthesia-related complications and in the training of coders about these complications. Perhaps explicit identification of whether clinical care is routine or nonroutine for billing purposes would help calibrate coding of anesthetic-related events.

Importantly, codes may be correctly applied to a patient’s hospitalization but not reflect a condition’s severity. When a condition is present, any associated morbidity is on a spectrum, from minor to catastrophic consequences. Some conditions, although clearly present during our chart review, did not pose the same threat to life as others. For example, in Disorders of coagulation, 2 cases of thrombocytopenia existed. These cases were appropriately coded, but no significant bleeding event or platelet transfusion occurred. Thus, even for the ICD-9-CM codes representing maternal morbidity that had a high PPV in this study, the morbidity identified may not be severe.

The requirement for ICU admission appears to be an effective way to verify the severity of any complication identified by ICD-9-CM diagnostic codes as the overall PPV was high at 98% (CI, 89–99%). This finding supports the principle of defining severe maternal morbidity as a combination of both a diagnosed complication and ICU admission. The National Partnership for Maternal Safety and the Joint Commission define severe maternal morbidity as “a patient safety event (not primarily related to the natural course of the patient’s illness or underlying condition) that reaches a patient and results in … either ICU admission or four or more units of erythrocytes transfused.”^26–28 Going forward, the ICD Tenth Revision, Clinical Modification codes may better capture these differences in severity. Compared with ICD-9-CM, the ICD Tenth Revision, Clinical Modification contains 5 times as many diagnosis codes and 19 times as many procedure codes to update and enhance coding description and, therefore, data quality.²⁹

Although the primary goal of this study was to assess the PPV of ICD-9-CM codes, we evaluated the level of agreement between primary physician reviewers in this study given the complexity of information often contained in medical records. The extent of disagreement between primary physician reviewers within this study (42/443, 9.5%) is consistent with previously reported ICD coding error rates, ranging from approximately 10% to 20%.^30,31 The majority of disagreements between our primary reviewers was related to data misinterpretation (27/42, 64%) and the remaining because of data simply being overlooked (Supplemental Digital Content 3, Table 3, http://links.lww.com/AA/B453). Almost half of the disagreements (20/42, 48%) related to ICD-9-CM codes in maternal morbidity categories with low PPV: Puerperal cerebrovascular disorders, Acute heart failure (includes arrest and fibrillation), Eclampsia, and Severe anesthesia complications.

This study has several limitations. First, for some categories, only a small number of cases were identified and reviewed because such clinical events are rare. As a result, CIs on the PPV estimates for these conditions are wide. Second, variances in regional or institutional coding practices may exist. Despite the high PPV at 2 separate institutions, additional studies should be conducted to ensure the generalizability of the results. For example, information technology architecture, staff resources, and case mix and volume may be different at community or rural health centers, and it is unclear how these factors may impact coding accuracy. Third, although we provide examples of miscoding in the basis of our detailed chart review, our discussion about reasons for incorrect coding are speculative and cannot be fully substantiated without further analysis, such as interviewing coders or evaluation of the coding process. Furthermore, this study assesses the PPV of codes reflecting maternal morbidity but does not evaluate the negative predictive value (NPV)—codes not assigned when a condition truly exists. Although for some studies defining conditions with high PPV (even in the face of limited sensitivity) will be adequate, for others having a high NPV will be important (ie, a low NPV for an ICD code may result in underestimation of maternal morbidity rates). Further research is needed to assess NPV of maternal morbidity ICD codes. In addition, prospective studies are needed to confirm the sensitivity and specificity of maternal morbidity codes. Of note, previous studies validating ICD-9-CM codes for rare pregnancy complications³² and nondelivery-related complications³³ had greater specificity than sensitivity. Fourth, although a restriction based on length of stay or ICU admission resulted in a high PPV for most categories, restricting the codes may have also increased the false negatives because there were fewer codes in this restricted data set. Finally, PPV depends on the prevalence of disease, and different prevalences of a condition in other patient populations could yield different results.

CONCLUSIONS

The validity of ICD-9-CM codes capturing severe maternal morbidity during delivery hospitalization depends on the amount of objective clinical data (laboratory values, pathology results, or procedure documentation) supporting the definition of the assigned code. Low PPV was observed in maternal morbidity categories when greater judgment, interpretation, and synthesis of supportive clinical information (signs and symptoms) were required to support the assigned codes, such as with the category Severe anesthesia complications. Therefore, surveillance, research, and quality improvement efforts should focus on codes defined by objective supporting data and, accordingly, maternal morbidity categories demonstrating a high PPV that are represented by these types of codes. Likewise, when a high degree of judgment, interpretation, and synthesis of the clinical scenario is required to substantiate the code, greater caution should be exercised.