Abstract
OBJECTIVE:
Measurement of obstetric hemorrhage-related morbidity is important for quality assurance purposes but presents logistical challenges in large populations. Billing codes are typically used to track severe maternal morbidity but may be of suboptimal validity. The objective of this study was to evaluate validity of billing code diagnoses for hemorrhage-related morbidity compared to data obtained from the electronic medical record.
STUDY DESIGN:
Deliveries occurring between July 2014 and July 2017 from three hospitals within a single system were analyzed. Three outcomes related to obstetric hemorrhage that are part of the Centers for Disease Control and Prevention definition of severe maternal morbidity (SMM) were evaluated: (i) transfusion, (ii) disseminated intravascular coagulation (DIC), and (iii) acute renal failure (ARF). ICD-9-CM and ICD-10-CM for these conditions were ascertained and compared to blood bank records and laboratory values. Sensitivity, specificity, positive (PPV) and negative predictive values (NPV) with 95% confidence intervals (CI) were calculated. Ancillary analyses were performed comparing codes and outcomes between hospitals and comparing ICD-9-CM to ICD-10-CM codes. Comparisons of categorical variables were performed with the chi-squared test. T-tests were used to compare continuous outcomes.
RESULTS:
35,518 deliveries were analyzed. 786 women underwent transfusion, 168 had serum creatinine ≥1.2mg/dL, and 99, 40, and 16 had fibrinogen ≤200, ≤150, and ≤100 mg/dL, respectively. Transfusion codes were 65% sensitive (95% CI 62%-69%) with a 91% PPV (89%-94%) for blood bank records of transfusion. DIC codes were 22% sensitive (95% CI 15%-32%) for a fibrinogen cutoff of ≤200mg/dL with 15% PPV (95% CI 10%-22%). Sensitivity for ARF was 33% (95% CI 26%-41%) for a creatinine of 1.2mg/dL with a PPV of 63% (95% CI 52%-73%). Sensitivity of ICD-9-CM for transfusion was significantly higher than ICD-10-CM (81%, 95% CI 76%-86% versus 56%, 95% CI 51%-60%, p<0.01). Evaluating sensitivity of codes by individual hospitals, sensitivity of diagnosis codes for transfusion varied significantly (Hospital A 47%, 95% CI 36%-58% versus Hospital B 63%, 95% CI 58%-67% versus Hospital C 80%, 95% CI 74%-86%, p<0.01).
CONCLUSION:
Use of administrative billing codes for postpartum hemorrhage complications may be appropriate for measuring trends related to disease burden and resource utilization, particularly in the case of transfusion, but may be suboptimal for measuring clinical outcomes within and between hospitals.
Keywords: Obstetric hemorrhage, patient safety, obstetric quality, maternal morbidity
INTRODUCTION
Postpartum hemorrhage (PPH) is a leading cause of maternal mortality and severe maternal morbidity (SMM) in the United States.1 Identification and review of hemorrhage-related severe morbidity on a hospital level are required to improve maternal care and safety. Measurement of hemorrhage-related severe morbidity is also important in larger populations to track trends in outcomes and safety. SMM rates may be used to compare safety and outcomes between hospitals. In assessing maternal outcomes, chart level reviews for large numbers of patients may be expensive and burdensome. As a low-cost alternative to chart reviews, administrative billing data is often used to analyze outcomes relate to PPH. As an example, the composite SMM outcome from the Centers for Disease Control and Prevention uses billing codes to identify severe morbidity conditions using billing diagnosis and procedure codes.2 While administrative data may be appropriate for evaluating temporal trends, resource utilization, and disease burden,3,4 drawbacks include lack of granularity, inability to ascertain important details, that administrative data coding is often performed by non-clinical personnel, and concerns regarding validity.5
Biomedical informatics, as defined by the American Medical Informatics Association, is an interdisciplinary field focused on effective use of biomedical data, information, and knowledge for scientific inquiry, problem solving, and clinical decision-making with the goal of improving human health.6 Querying of electronic health records (EHR) represent a means of readily ascertaining granular clinical information7 including laboratory findings, drug and device use, receipt of blood products, procedures, and other clinical information that may be particularly well suited for presenting robust, multifaceted measurement of PPH and related outcomes on a hospital or hospital system level. However, the degree to which EHR data may provide a more nuanced and valid assessment of PPH-related severe morbidity than administrative data alone has not been well characterized. Therefore, the purpose of this study was to compare hemorrhage-related morbidity based on electronic medical record data and administrative billing codes.
METHODS
Patients delivering between July 2014 and July 2017 from three hospitals that perform approximately 12,000 deliveries per year in the NewYork-Presbyterian health system were included. Data was initially collected for quality assurances purposes to manage and assess risk for PPH in the setting of practice changes and recommendations from the Safe Motherhood Initiative (SMI), a quality improvement effort led by the American Congress of Obstetrics and Gynecology District II in New York State. Starting in 2013, SMI developed and distributed bundles for leading causes of maternal mortality, including hemorrhage.8 Approval to use this data for research purposes was obtained from the Columbia University Institutional Review Board.
The primary purpose of this analysis was to determine test characteristics for Internal Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes and Internal Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes representative of significant complications related to PPH. Namely, we examined codes for: (i) transfusion, (ii) disseminated intravascular coagulation (DIC), and (iii) acute renal failure (ARF) in relation to whether evidence for these outcomes was present in the electronic medical record. Billing codes for transfusion, DIC, and ARF were based on ICD-9-CM and ICD-10-CM codes used by the Centers for Disease Control and Prevention in their severe maternal morbidity composite.2 To ascertain clinical outcomes in the EMR, data were extracted through automated query from our institution’s electronic data warehouse, which aggregates data from the EMR, billing system, and other sources. In cases of missing data, data were pulled directly from the EMR. The accuracy of ICD-9-CM and ICD-10-CM procedure codes for transfusion was analyzed based on data for transfusion from blood bank records. The accuracy of ARF ICD-9-CM and ICD-10-CM diagnosis codes was assessed based on whether laboratory values demonstrated serum creatinine ≥1.2mg/dL. The accuracy of DIC diagnosis codes was assessed based on whether laboratory values demonstrated serum fibrinogen at the following cutoffs: ≤200, ≤150, and ≤100 mg/dL. For each of these outcomes, test characteristics were calculated for ICD-9-CM and ICD-10-CM billing codes including (i) sensitivity, (ii) specificity, (iii) positive predictive value (PPV), and (iv) negative predictive value (NPV). Sensitivity was defined as the proportion of outcomes ascertained with EHR data (laboratory and blood bank records) that had an appropriate associated billing code. Specificity was defined as the proportion of patients without an outcome based on EHR data (laboratory and blood bank records) who also did not have an associated billing code indicating the relevant condition. PPV was defined as the likelihood of a condition being present based on laboratory and blood bank records if a relevant diagnosis code was present. NPV was defined as the likelihood of a condition being absent based on laboratory and blood bank records if a relevant diagnosis code was absent.
A series of three ancillary analyses were performed. First, because transfusion, DIC, and ARF can occur in the setting of other conditions, we performed two restricted analyses limited to women with large hematocrit drops, defined as the percent change between the highest pre-delivery hematocrit and the lowest post-delivery hematocrit for individual patients (as extracted from the EHR). We then repeated the primary analysis comparing billing codes to EHR data for ARF, DIC, and transfusion for (i) women with hematocrit drops of ≥30% and (ii) women with hematocrit drops of ≥40%. Second, because there is limited data regarding how the switch from ICD-9-CM to ICD-10-CM codes might have affected the relation of the codes to accurate capture of hemorrhage related outcomes, we repeated the primary analysis individually (i) for ICD-9-CM diagnosis codes, and (ii) for ICD-10-CM codes and determined whether test characteristics for DIC, ARF, and transfusion differed significantly based on the Internal Classification of Diseases iteration. Finally, billing code procedure abstraction may differ based on hospital practices and resources but data is limited regarding hemorrhage outcomes. To determine the possible role of hospitals in relation to valid billing code capture of hemorrhage related outcomes, we repeated the analysis individually for each of the three hospitals in the analysis and determined if test characteristics differed significantly for DIC, ARF, and transfusion. Comparisons of categorical variables were performed with the chi-square test. T-tests were used to compare continuous outcomes. For demographics and outcomes cell sizes <10 are suppressed to preserve patient confidentiality. SQL and R were used for data processing and analysis for this study.
RESULTS
35,518 deliveries occurred between July 2014 and July 2017 in the three hospitals that were included in the analysis. Based on EHR data (including transfusion records and blood bank records), during the study period 2.2% of women received a transfusion (n=786), 0.3% experienced DIC (n=99), and 0.5% experienced acute renal failure (n=168) (Table 1). Risk for outcomes differed significantly by the three hospitals; likelihood of transfusion and ARF were highest for hospital A (p<0.01 for both comparisons across hospitals), while hospital B had the highest rate of DIC (p<0.01). Evaluating demographic factors, women 35 and older were more likely experience DIC (p<0.01) and transfusion (p=0.02) than younger women although differences in ARF were not significantly statistically. For public compared to private insurance, transfusion rates were significantly higher (p<0.01) although risk for DIC and ART were not significantly different. For cesarean delivery, DIC, ARF, and transfusion were all more likely (p<0.01 for all).
Table 1.
Demographics and hemorrhage-related outcomes based on informatics data
| All deliveries | Transfusion | DIC | ARF | |||||
|---|---|---|---|---|---|---|---|---|
|
| ||||||||
| N | % | N | % | N | % | N | % | |
| All patients | 35518 | 100 | 786 | 100 | 99 | 100 | 168 | 100 |
| Year | ||||||||
| 2014 | 6077 | 17.1 | 125 | 15.9 | 17 | 17.2 | 28 | 16.7 |
| 2015 | 11501 | 32.4 | 210 | 26.7 | 26 | 26.3 | 46 | 27.4 |
| 2016 | 11474 | 32.3 | 277 | 35.2 | 34 | 34.3 | 67 | 39.9 |
| 2017 | 6466 | 18.2 | 174 | 22.1 | 22 | 22.2 | 27 | 16.1 |
| Age in years | ||||||||
| <18 | 206 | 0.6 | <10 | n/a | <10 | n/a | <10 | n/a |
| 18-24 | 4643 | 13.1 | 120 | 15.3 | <10 | n/a | 25 | 14.9 |
| 25-34 | 18493 | 52.1 | 356 | 45.3 | 40 | 40.4 | 71 | 42.3 |
| 35-39 | 9218 | 26.0 | 199 | 25.3 | 38 | 38.4 | 45 | 26.8 |
| ≥ 40 | 2958 | 8.3 | 107 | 13.6 | 18 | 18.2 | 25 | 14.9 |
| Race | ||||||||
| American Indian / Alaskan Native | 60 | 0.2 | <10 | n/a | <10 | n/a | <10 | n/a |
| Asian | 2935 | 8.3 | 70 | 8.9 | 18 | 18.2 | 11 | 6.5 |
| Asian Indian | 69 | 0.2 | <10 | n/a | <10 | n/a | <10 | n/a |
| Black / African American | 2307 | 6.5 | 81 | 10.3 | <10 | 9.1 | 29 | 17.3 |
| Hispanic | 10137 | 28.5 | 259 | 33.0 | 17 | 17.2 | 49 | 29.2 |
| Native Hawaiian / Pacific Islander | 110 | 0.3 | <10 | n/a | <10 | n/a | <10 | n/a |
| Other / Unknown | 7991 | 22.5 | 171 | 21.8 | 21 | 21.2 | 32 | 19.0 |
| White | 11909 | 33.5 | 199 | 25.3 | 34 | 34.3 | 46 | 27.4 |
| Payer | ||||||||
| Private | 22140 | 62.3 | 439 | 55.9 | 71 | 71.7 | 98 | 58.3 |
| Public | 13368 | 37.6 | 347 | 44.1 | 28 | 28.3 | 70 | 41.7 |
| Unknown | 10 | 0.0 | <10 | n/a | <10 | n/a | <10 | n/a |
| Mode of delivery | ||||||||
| Cesarean | 12756 | 35.9 | 580 | 73.8 | 83 | 83.8 | 101 | 60.1 |
| Vaginal | 22762 | 64.1 | 206 | 26.2 | 16 | 16.2 | 67 | 39.9 |
DIC, disseminated intravascular coagulation with cutoff of fibrinogen<=200. ARF, acute renal failure.
Overall, based on ICD-9 billing codes 143 women had DIC (0.4%), 89 women had ARF (0.3%), and 563 women had transfusion (1.6%). Evaluating ICD coding compared to clinical EHR data, transfusion had a 65.4% sensitivity (95% CI 62.0-68.7%) compared to blood bank records (that is, 65.4% of transfusion based on blood bank records had an appropriate associated billing code) with 99.9% specificity (95% CI 99.8-99.9%) (Table 2) along with a PPV of 91.3% (95% CI 88.7-93.5%) and NPV of 99.2% (95%CI 99.1-99.3%). Evaluating DIC, ICD coding sensitivity increased with lower fibrinogen lab result cutoffs; the sensitivity of ICD coding for fibrinogen ≤200mg/dL was 22.2% (95% CI 14.5-31.7%), for fibrinogen ≤150mg/dL was 35.0% (95% CI 20.6-51.7%), and 56.3% (95% CI 26.3-41.0%) for fibrinogen ≤100mg/dL; these differences were not statistically significant. With decreasing fibrinogen lab level, the PPV of ICD coding for DIC decreased non-significantly: 15.4% (95% CI 9.9-22.4%) for fibrinogen ≤200mg/dL, 9.8% (95% CI 5.5-15.9%) for fibrinogen ≤150mg/dL, and 6.3% (95% CI 2.9-11.6%) for fibrinogen ≤100mg/dL. Specificity and NPV for DIC were high at all cutoff (>99.5% for all). Evaluating ARF with a laboratory cutoff of serum creatinine of 1.2mg/dL, ICD coding was 33.3% sensitive (95% CI 26.3-41.0%) with a PPV of 62.9% (95% CI 52.0-72.9%) with high specificity and NPV (>99.5% for both).
Table 2.
Test characteristics of International Classification of Disease diagnoses codes for clinical outcomes in the electronic medical record
| ICD diagnosis | Clinical outcome | Sensitivity | 95% CI | Specificity | 95% CI |
|---|---|---|---|---|---|
| Transfusion | Transfusion | 65% | 62%-69% | 99.9% | 99.8-99.9% |
| DIC | Fibrinogen ≤200mg/dL | 22% | 15%-32% | 99.7% | 99.6-99.7% |
| DIC | Fibrinogen ≤150mg/dL | 35% | 21%-52% | 99.6% | 99.6-99.7% |
| DIC | Fibrinogen ≤100mg/dL | 56% | 30%-80% | 99.6% | 99.6-99.7% |
| ARF | Serum Cr 1.2mg/dL | 33% | 26%-41% | 99.9% | 99.9-99.9% |
| ICD diagnosis | Clinical outcome | PPV | 95% CI | NPV | 95% CI |
|
| |||||
| Transfusion | Transfusion | 91% | 89%-94% | 99.2% | 99.1-99.3% |
| DIC | Fibrinogen ≤200mg/dL | 15% | 10%-22% | 99.8% | 99.7-99.8% |
| DIC | Fibrinogen ≤150mg/dL | 10% | 6%-16% | 99.9% | 99.9-100.0% |
| DIC | Fibrinogen ≤100mg/dL | 6% | 3%-12% | 100.0% | 100.0-100.0% |
| ARF | Serum Cr 1.2mg/dL | 63% | 52%-73% | 99.7% | 99.6-99.7% |
PPV, positive predictive value. NPV, negative predictive value. Cr, creatinine. DIC, disseminated intravascular coagulation. ARF, acute renal failure. Test characteristics are for ICD billing codes predicting clinical outcomes.
Restricting the analysis to patients with hematocrit drops of ≥30% and ≥40%, diagnosis code estimates for sensitivity and specificity of transfusion, DIC, and ART were similar (Table 3). However, because of the higher prevalence of conditions in patients with hematocrit drops of ≥30% and ≥40%, PPV were higher for these outcomes than in the primary analysis. For DIC, the PPV of a DIC diagnosis code for fibrinogen ≤200mg/dL was 15% (95% CI 10%-22%) in the primary analysis compared to 50% (95% CI 32%-68%) for women with hematocrit drops of ≥30% and 62% (95% CI 32%-86%) for women with hematocrit drops of ≥40% (p<0.01).
Table 3.
Sensitivity analyses for women with hematocrit drops of ≥30% and ≥40%
| Hematocrit drop ≥30% | |||||
|---|---|---|---|---|---|
| ICD diagnosis | Clinical outcome | Sensitivity | 95% CI | Specificity | 95% CI |
| Transfusion | Transfusion | 67% | 62%, 71% | 99% | 98%, 100% |
| DIC | Fibrinogen ≤200mg/dL | 25% | 15%, 38% | 99% | 98%, 99% |
| DIC | Fibrinogen ≤150mg/dL | 42% | 22%, 63% | 99% | 98%, 0.99% |
| DIC | Fibrinogen ≤100mg/dL | 50% | 21%, 79% | 98% | 97%, 99% |
| ARF | Serum Cr 1.2mg/dL | 38% | 25%, 53% | 100% | 99%, 100% |
| ICD diagnosis | Clinical outcome | PPV | 95% CI | NPV | 95% CI |
|
| |||||
| Transfusion | Transfusion | 97% | 94%, 99% | 89% | 87%, 91% |
| DIC | Fibrinogen ≤200mg/dL | 50% | 32%, 68% | 97% | 96%, 98% |
| DIC | Fibrinogen ≤150mg/dL | 31% | 16%, 50% | 99% | 98%, 99% |
| DIC | Fibrinogen ≤100mg/dL | 19% | 7%, 36% | 100% | 99%, 100% |
| ARF | Serum Cr 1.2mg/dL | 79% | 58%, 93% | 98% | 97%, 99% |
| Hematocrit drop ≥40% | |||||
| ICD diagnosis | Clinical outcome | Sensitivity | 95% CI | Specificity | 95% CI |
|
| |||||
| Transfusion | Transfusion | 68% | 61%, 74% | 98% | 95%, 100% |
| DIC | Fibrinogen ≤200mg/dL | 21% | 9%, 36% | 99% | 97%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 40% | 16%, 68% | 98% | 96%, 99% |
| DIC | Fibrinogen ≤100mg/dL | 38% | 9%, 76% | 97% | 95%, 99% |
| ARF | Serum Cr 1.2mg/dL | 54% | 33%, 74% | 99% | 98%, 100% |
| ICD diagnosis | Clinical outcome | PPV | 95% CI | NPV | 95% CI |
|
| |||||
| Transfusion | Transfusion | 98% | 94%, 100% | 71% | 64%, 76% |
| DIC | Fibrinogen ≤200mg/dL | 62% | 32%, 86% | 91% | 88%, 94% |
| DIC | Fibrinogen ≤150mg/dL | 46% | 19%, 75% | 98% | 95%, 99% |
| DIC | Fibrinogen ≤100mg/dL | 23% | 5%, 54% | 99% | 97%, 100% |
| ARF | Serum Cr 1.2mg/dL | 87% | 60%, 98% | 97% | 95%, 98% |
PPV, positive predictive value. NPV, negative predictive value. Cr, creatinine. DIC, disseminated intravascular coagulation. ARF, acute renal failure. Test characteristics are for ICD billing codes predicting clinical outcomes.
Evaluating test characteristics of diagnosis codes dichotomized by ICD-9-CM and ICD-10-CM, sensitivity was significantly higher for transfusion with ICD-9-CM codes (81%, 95% CI 76%-86% versus 56%, 95% CI 51%-60%, p<0.01). Sensitivity estimates did not differ significantly for other diagnoses although confidence intervals were wide (Table 4). Evaluating sensitivity of codes by individual hospitals, sensitivity of diagnosis codes for transfusion varied significantly (Hospital A 47%, 95% CI 36%-58% versus Hospital B 63%, 95% CI 58%-67% versus Hospital C 80%, 95% CI 74%-86%, p<0.01). Sensitivity estimates did not differ significantly by hospital for other diagnoses although confidence intervals were wide (Table 5).
Table 4.
Sensitivity analyses for women with ICD-9-CM versus ICD-10-CM diagnosis and procedure codes
| International Classification of Diseases, 9th revision | |||||
|---|---|---|---|---|---|
| ICD diagnosis | Clinical outcome | Sensitivity | 95% CI | Specificity | 95% CI |
| Transfusion | Transfusion | 81% | 76%, 86% | 100%% | 100%, 100% |
| DIC | Fibrinogen ≤200mg/dL | 28% | 15%, 44% | 99% | 99%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 42% | 15%, 72% | 99% | 99%, 100% |
| DIC | Fibrinogen ≤100mg/dL | 100% | 16%, 100% | 99% | 99%, 100% |
| ARF | Serum Cr 1.2mg/dL | 41% | 29%, 54% | 100% | 100%, 100% |
| ICD diagnosis | Clinical outcome | PPV | 95% CI | NPV | 95% CI |
|
| |||||
| Transfusion | Transfusion | 89% | 85%, 93% | 100% | 100%, 100% |
| DIC | Fibrinogen ≤200mg/dL | 13% | 6%, 21% | 100% | 100%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 6% | 2%, 13% | 100% | 100%, 100% |
| DIC | Fibrinogen ≤100mg/dL | 2% | 0%, 08% | 100% | 100%, 100% |
| ARF | Serum Cr 1.2mg/dL | 63% | 47%, 78% | 100% | 100%, 100% |
| International Classification of Diseases, 10th revision | |||||
| ICD diagnosis | Clinical outcome | Sensitivity | 95% CI | Specificity | 95% CI |
|
| |||||
| Transfusion | Transfusion | 56% | 51%, 60% | 100% | 100%, 100% |
| DIC | Fibrinogen ≤200mg/dL | 19% | 10%, 31% | 100% | 100%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 32% | 16%, 52% | 100% | 100%, 100% |
| DIC | Fibrinogen ≤100mg/dL | 50% | 23%, 77% | 100% | 100%, 100% |
| ARF | Serum Cr 1.2mg/dL | 29% | 20%, 38% | 100% | 100%, 100% |
| ICD diagnosis | Clinical outcome | PPV | 95% CI | NPV | 95% CI |
|
| |||||
| Transfusion | Transfusion | 93% | 89%, 96% | 99% | 99%, 99% |
| DIC | Fibrinogen ≤200mg/dL | 20% | 10%, 32% | 100% | 100%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 16% | 8%, 28% | 100% | 100%, 100% |
| DIC | Fibrinogen ≤100mg/dL | 13% | 0.05, 0.24 | 100% | 100%, 100% |
| ARF | Serum Cr 1.2mg/dL | 63% | 0.47, 0.76 | 100% | 100%, 100% |
PPV, positive predictive value. NPV, negative predictive value. Cr, creatinine. DIC, disseminated intravascular coagulation. ARF, acute renal failure. Test characteristics are for ICD billing codes predicting clinical outcomes.
Table 5.
Sensitivity analyses with repeated analyses for each the three individual hospitals contributing data
| Hospital A | |||||
|---|---|---|---|---|---|
| ICD diagnosis | Clinical outcome | Sensitivity | 95% CI | Specificity | 95% CI |
| Transfusion | Transfusion | 47% | 0.36, 0.58 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤200mg/dL | 17% | 0.00, 0.64 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 33% | 0.01, 0.91 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤100mg/dL | 100% | 0.02, 1.00 | 100% | 100%, 100% |
| ARF | Serum Cr 1.2mg/dL | 31% | 0.11, 0.59 | 100% | 100%, 100% |
| ICD diagnosis | Clinical outcome | PPV | 95% CI | NPV | 95% CI |
|
| |||||
| Transfusion | Transfusion | 91% | 0.79, 0.98 | 99% | 99%, 99% |
| DIC | Fibrinogen ≤200mg/dL | 10% | 0.00, 0.45 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 10% | 0.00, 0.45 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤100mg/dL | 10% | 0.00, 0.45 | 100% | 100%, 100% |
| ARF | Serum Cr 1.2mg/dL | 56% | 0.21, 0.86 | 100% | 100%, 100% |
| Hospital B | |||||
| ICD diagnosis | Clinical outcome | Sensitivity | 95% CI | Specificity | 95% CI |
|
| |||||
| Transfusion | Transfusion | 63% | 0.58, 0.67 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤200mg/dL | 30% | 0.16, 0.49 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 38% | 0.14, 0.68 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤100mg/dL | 50% | 0.12, 0.88 | 100% | 100%, 100% |
| ARF | Serum Cr 1.2mg/dL | 37% | 0.28, 0.47 | 100% | 100%, 100% |
| ICD diagnosis | Clinical outcome | PPV | 95% CI | NPV | 95% CI |
|
| |||||
| Transfusion | Transfusion | 93% | 0.90, 0.96 | 99% | 98%, 99% |
| DIC | Fibrinogen ≤200mg/dL | 20% | 0.10, 0.33 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤150mg/dL | 10% | 0.03, 0.21 | 100% | 100%, 100% |
| DIC | Fibrinogen ≤100mg/dL | 6% | 0.01, 0.16 | 100% | 100%, 100% |
| ARF | Serum Cr 1.2mg/dL | 63% | 0.50, 0.75 | 100% | 99%, 100% |
| Hospital C | |||||
| ICD diagnosis | Clinical outcome | Sensitivity | 95% CI | Specificity | 95% CI |
|
| |||||
| Transfusion | Transfusion | 80% | 0.74, 0.86 | 99.9% | 1.00, 1.00 |
| DIC | Fibrinogen ≤200mg/dL | 18.3% | 0.10, 0.30 | 99.5% | 0.99, 1.00 |
| DIC | Fibrinogen ≤150mg/dL | 33.3% | 0.16, 0.55 | 99.5% | 0.99, 1.00 |
| DIC | Fibrinogen ≤100mg/dL | 55.6% | 0.21, 0.86 | 99.5% | 0.99, 1.00 |
| ARF | Serum Cr 1.2mg/dL | 26.5% | 0.15, 0.41 | 100.0% | 1.00, 1.00 |
| ICD diagnosis | Clinical outcome | PPV | 95% CI | NPV | 95% CI |
|
| |||||
| Transfusion | Transfusion | 87.5% | 0.82, 0.92 | 99.8% | 1.00, 1.00 |
| DIC | Fibrinogen ≤200mg/dL | 13.4% | 0.07, 0.23 | 99.7% | 1.00, 1.00 |
| DIC | Fibrinogen ≤150mg/dL | 9.8% | 0.04, 0.18 | 99.9% | 1.00, 1.00 |
| DIC | Fibrinogen ≤100mg/dL | 6.1% | 0.02, 0.14 | 100.0% | 1.00, 1.00 |
| ARF | Serum Cr 1.2mg/dL | 65.0% | 0.41, 0.85 | 99.8% | 1.00, 1.00 |
PPV, positive predictive value. NPV, negative predictive value. Cr, creatinine. DIC, disseminated intravascular coagulation. ARF, acute renal failure. Test characteristics are for ICD billing codes predicting clinical outcomes.
DISCUSSION
Main Findings
While this analysis found that ICD procedure codes are of moderate sensitivity for transfusion with high positive predictive value, sensitivity decreased with ICD-10-CM compared to ICD-9-CM and varied significantly by hospital. Sensitivities for diagnosis codes for ARF and DIC with fibrinogen ≤200mg/dL were low with lower fibrinogen cutoffs for DIC imprecise.
Clinical implications
These findings support that while it may be reasonable to track population-level trends and disease burden related to obstetric hemorrhage with billing codes, this data may be inappropriate for evaluating outcomes within a hospital or comparing risk between hospitals. SMM data based on billing codes is increasingly being publicly reported as a proxy for obstetric quality and safety (e.g., https://www.usatoday.com/maternal-mortality-harm-hospital-database/). Given that (i) test characteristics for transfusion, the primary driver of the CDC SMM composite, vary between hospitals, (ii) there are ascertainment differentials for ICD-9-CM versus ICD-10-CM iterations, and (iii) non-transfusion codes may be of low sensitivity, the validity of inter-hospital safety, quality, and outcomes comparisons made with billing data is questionable. Another option for tracking hemorrhage related morbidity includes ascertainment of receipt of four or more units of red blood cells or intensive care unit admission; however, while these criteria are appropriate for clinical care case review, such events are rare, occurring in approximately 3 out of 1,000 births, and are not necessarily representative of hospital-level hemorrhage risk.9,10 Clinical documentation of estimated or quantitative blood loss measures may also be used to track hemorrhage, but these are limited by inaccuracy and, in the case of quantitative blood loss, lack of widespread adoption.11–14 Access to EHR data, on the other hand, may be increasing due regional hospital consolidation and resultant large hospital systems and increasing market consolidation of electronic medical record systems. Access to EHR queries within hospital systems may be limited based on resources, but obtaining such data may be particularly important for analyzing obstetric safety where large numbers of patients are cared for and large samples are required to make statistically meaningful comparisons for adverse outcomes.
Strengths and limitations
In interpreting the findings of this study there are several important limitations to consider. First, this study was cross-sectional and did not presume that the severe morbidity outcomes analyzed were directly related to postpartum hemorrhage; for women hematocrit drops of ≥30% and ≥40% it is more likely that these complications were related to hemorrhage. However, other hemolytic disease processes, preeclampsia, and underlying medical conditions could have contributed to these outcomes. Second, large transfusions may have limited detection based on hematocrit drop. Third, it is possible that measurement of low fibrinogen in rare cases could be due to spurious lab values or incorrectly drawn samples; chart reviews were not performed to determine whether lab values were considered valid by clinicians. Fourth, because we did not perform chart reviews we could not determine if there were clinical cases of likely DIC that did not have low fibrinogen because of prompt transfusion of blood products. The approach in this manuscript would not be valid in capturing such cases. Fifth, we do not perform rotational thromboelastometery at our institutions for obstetric populations so this diagnostic test could be used to confirm DIC diagnoses. Sixth, we did not perform corrections for hematocrit and hemoglobin values secondary to factors such as transfusion, body mass index, and intravenous fluid administration given that the complexity of such calculations were beyond the scope of this analysis. Strengths of this study include that we were able to analyze a relatively large patient population using laboratory values and blood bank records which represent objective measures of the outcomes, that we were able to compare ICD-9-CM to ICD-10-CM codes, and that we were able to compare billing code validity across three separate hospitals.
Conclusion
Use of administrative billing codes for postpartum hemorrhage complications may be appropriate for measuring trends related to disease burden and resource utilization, particularly in the case of transfusion, but may capture a lower proportion of patients with adverse outcomes than EHR data. Furthermore, billing codes may validity may vary between hospitals with both ICD-9 and ICD-10 coding. Further research is needed to determine to what degree EHR data, available at some centers, may be preferable for measuring hospital level outcomes particularly in relation to clinical findings documented in the medical record.
Acknowledgments
Dr. Friedman is supported by a career development award (K08HD082287) from the Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health.
Financial Disclosure
Dr. D’Alton had a senior leadership role in ACOG II’s Safe Motherhood Initiative which received unrestricted funding from Merck for Mothers. The other authors did not report any potential conflicts of interest. Each author has indicated that he or she has met the journal’s requirements for authorship.
Footnotes
This study was presented at the Annual Meeting for the Society for Maternal Fetal Medicine, February, 2019, Las Vegas, Nevada
REFERENCES
- 1.Creanga AA, Syverson C, Seed K, Callaghan WM. Pregnancy-Related Mortality in the United States, 2011-2013. Obstet Gynecol 2017;130:366–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.The Centers for Disease Control and Prevention Severe Maternal Morbidity in the United States Accessed August 15, 2019. Available at: https://wwwcdcgov/reproductivehealth/maternalinfanthealth/severematernalmorbidityhtml.
- 3.Bateman BT, Berman MF, Riley LE, Leffert LR. The epidemiology of postpartum hemorrhage in a large, nationwide sample of deliveries. Anesth Analg 2010;110:1368–73. [DOI] [PubMed] [Google Scholar]
- 4.Callaghan WM, Creanga AA, Kuklina EV. Severe maternal morbidity among delivery and postpartum hospitalizations in the United States. Obstet Gynecol 2012;120:1029–36. [DOI] [PubMed] [Google Scholar]
- 5.Grimes DA. Epidemiologic research using administrative databases: garbage in, garbage out. Obstet Gynecol 2010;116:1018–9. [DOI] [PubMed] [Google Scholar]
- 6.Kulikowski CA, Shortliffe EH, Currie LM, et al. AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline. J Am Med Inform Assoc 2012;19:931–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Coorevits P, Sundgren M, Klein GO, et al. Electronic health records: new opportunities for clinical research. J Intern Med 2013;274:547–60. [DOI] [PubMed] [Google Scholar]
- 8.Fleischer A, Meirowitz N. Care bundles for management of obstetrical hemorrhage. Semin Perinatol 2016;40:99–108. [DOI] [PubMed] [Google Scholar]
- 9.Main EK, Goffman D, Scavone BM, et al. National Partnership for Maternal Safety: consensus bundle on obstetric hemorrhage. Anesth Analg 2015;121:142–8. [DOI] [PubMed] [Google Scholar]
- 10.Callaghan WM, Grobman WA, Kilpatrick SJ, Main EK, D’Alton M. Facility-based identification of women with severe maternal morbidity: it is time to start. Obstet Gynecol 2014;123:978–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Patel A, Goudar SS, Geller SE, et al. Drape estimation vs. visual assessment for estimating postpartum hemorrhage. Int J Gynaecol Obstet 2006;93:220–4. [DOI] [PubMed] [Google Scholar]
- 12.Stafford I, Dildy GA, Clark SL, Belfort MA. Visually estimated and calculated blood loss in vaginal and cesarean delivery. Am J Obstet Gynecol 2008;199:519 e1–7. [DOI] [PubMed] [Google Scholar]
- 13.Toledo P, Eosakul ST, Goetz K, Wong CA, Grobman WA. Decay in blood loss estimation skills after web-based didactic training. Simul Healthc 2012;7:18–21. [DOI] [PubMed] [Google Scholar]
- 14.Gabel KT, Weeber TA. Measuring and communicating blood loss during obstetric hemorrhage. J Obstet Gynecol Neonatal Nurs 2012;41:551–8. [DOI] [PubMed] [Google Scholar]