Abstract
Objective
To develop and validate a maternal comorbidity index to predict severe maternal morbidity, defined as the occurrence of acute maternal end-organ injury, or mortality.
Methods
Data were derived from the Medicaid Analytic eXtract for the years 2000 to 2007. The primary outcome was defined as the occurrence of maternal end-organ injury or death during the delivery hospitalization through 30 days postpartum. The dataset was randomly divided into a 2/3 development cohort and a 1/3 validation cohort. Using the development cohort, a logistic regression model predicting the primary outcome was created using a stepwise selection algorithm that included 24-candidate comorbid conditions and maternal age. Each of the conditions included in the final model was assigned a weight based on its beta coefficient, and these were used to calculate a maternal comorbidity index.
Results
The cohort included 854,823 completed pregnancies of which 9,901 (1.2%) were complicated by the primary study outcome. The derived score included 20 maternal conditions and maternal age. For each point increase in the score, the odds ratio for the primary outcome was 1.37, 95% Confidence Interval (CI) 1.35 to 1.39. The c-statistic for this model was 0.657, 95% CI 0.647 – 0.666. The derived score performed significantly better than available comorbidity indexes in predicting maternal morbidity and mortality.
Conclusion
This new maternal morbidity index provides a simple measure for summarizing the burden of maternal illness for use in the conduct of epidemiologic, health services, and comparative effectiveness research.
Introduction
In epidemiologic and health services research patients’ comorbidities must be identified and accounted for in analyses to avoid confounding bias. In certain circumstances, it is useful to have an index that summarizes the burden of comorbidity into a single numerical score.(1, 2) The most widely used indexes used for this purpose are the Charlson Comorbidity Index and the Elixhauser comorbidity classification system and their adaptations.(3–9) These indexes together have been cited over 1,000 times annually in the medical literature in recent years.(2)
These indexes have been applied in many studies in obstetrics for the purpose of describing and adjusting for comorbidity(10–19), despite having been developed for non-obstetric populations. The Charlson Comorbidity Index was developed to predict 1-year mortality in medical patients.(3) The Elixhauser comorbidity measure was developed to predict length of stay, hospital charges, and in-hospital death in explicitly non-obstetric admissions.(8) Both of these scoring systems lack obstetric conditions that are important determinants of maternal morbidity and mortality. Further, those conditions that do apply to obstetric patients are not weighted to reflect the unique contribution they make to the particular constellation of complications that present in an obstetric setting.
Recently there has been a call by leaders in the field of obstetrics to expand research into the determinants of severe maternal morbidity and mortality.(20) The development of a comorbidity score applicable to obstetric patients would provide an important tool for summarizing comorbid illness and confounding control in such research. Such an index has not, to our knowledge, been previously described.
The objective of this study was, therefore, to develop and validate a maternal comorbidity index to predict severe maternal morbidity, defined as the occurrence of acute maternal end-organ injury, or mortality.
Methods
Cohort
The study cohort was derived from the Medicaid Analytic eXtract, a healthcare utilization dataset that contains information on Medicaid enrollment and utilization claims, and included 2000–2007 data. Pregnancies were identified within this cohort as previously described.(21) The Medicaid Analytic eXtract dataset contains information regarding inpatient admissions, outpatient visits, and outpatient pharmacy dispensing claims. To allow adequate measurement of maternal comorbidities and outcomes, the cohort was restricted to women who delivered in-hospital and were eligible for Medicaid continuously from 180 days prior to the estimated last menstrual period (LMP) through either 30 days postpartum or date of death during the 30-day postpartum period. To ensure complete ascertainment of relevant claims, we further restricted our analysis to women with ≥28 days of enrollment each month, and without limited benefits, private insurance, or certain state-specific managed care programs that may underreport claims to the Medicaid Analytic eXtract.(21) The analytic cohort included 854,823 completed pregnancies. The use of this de-identified database for research was deemed not human subjects research by the Partner’s Institutional Review Board.
Study outcomes
The primary outcome for the study was defined as maternal end-organ injury or death during the delivery admission through 30 days postpartum. End-organ injury was identified by the presence of a diagnostic codes from the International Classification of Diseases, 9th revision (ICD 9) indicating acute heart failure, acute renal failure, acute liver disease, acute myocardial infarction, acute respiratory distress syndrome/respiratory failure, disseminated intravascular coagulation/coagulopathy, coma, delirium, puerperal cerebrovascular disorders, pulmonary edema, pulmonary embolism, sepsis, shock, status asthmaticus, or status epilepticus (see Appendix 1 for ICD-9-CM diagnostic codes, available online at http://links.lww.com/xxx).(22–25) Date of death was defined using the Medicaid eligibility file. The secondary outcome for the study was maternal intensive care unit admission during the delivery hospitalization through 30 days postpartum.
Appendix 1.
Diagnostic Codes Used to Define Maternal End-Organ Injury
| Complication | ICD-9 CM Codes |
|---|---|
| Acute heart failure | 415.0x, 427.5x, 428.0x, 428.1, 428.21, 428.31, 428.41, 997.1x, 669.4x, 428.23, 428.33, 428.43, 428.9x |
| Acute renal failure | 584.x, 669.3x |
| Acute liver disease | 570, 646.7x |
| Acute myocardial infarction | 410.x |
| Acute respiratory distress syndrome/respiratory failure | 518.81, 518.82, 518.84, 518.5x, 799.1x, 518.7x |
| Disseminated intravascular coagulation/coagulopathy | 666.3x, 286.6x, 286.7x, 286.9x, 287.4 |
| Coma | 780.01, 780.03, 572.2x, 250.2x, 250.3x, 251.0x |
| Delirium | 293.x |
| Puerperal cerebrovascular disorders | 671.5x, 674.0x, 430.xx–432.xx, 436, 997.01, 997.02, 433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.01, 434.11, 434.91, 325, 348.1, 348.3x, 348.5x, 437.1x, 437.2x, 437.6x, 346.6x |
| Pulmonary edema | 514.xx, 518.4x 428.1x |
| Pulmonary embolism | 673.x, 415.1x |
| Sepsis | 038.x, 995.91, 995.92, 112.5x, 659.3x, 785.52 |
| Shock | 669.1x, 785.5x, 998.0x, 995.4x, 995.0x, 995.94, 999.4x |
| Status asthmaticus | 493.01, 493.11, 493.21, 493.91 |
| Status epilepticus | 345.3x |
ICD-9-CM, International Classification of Diseases, 9th Revision, Clinical Modification.
Potential Predictors of Maternal Morbidity or Mortality
Based on a review of the relevant literature and clinical plausibility,(24) we formulated a list of maternal comorbidities that potentially confer increased risk of maternal morbidity and mortality as candidate predictors for the comorbidity index. We restricted our analysis to conditions that would likely be identifiable during the antepartum period up to the time of admission for delivery and not complications that develop during the delivery admission. We then queried maternal inpatient and outpatient claims for ICD-9 diagnoses indicating the presence of these conditions from the 180-day pre-pregnancy period through the delivery hospitalization (see Appendix 2 for diagnostic codes, available online at http://links.lww.com/xxx). We defined presence of each condition as having one or more corresponding codes during this period. We represented three pairs of conditions as hierarchies in the coding scheme. Specifically, if a patient had codes for severe preeclampsia/eclampsia, we coded her as not having mild or unspecified preeclampsia, even if she had codes for it. Similarly, we allowed a patient to be defined as having gestational diabetes only if she did not have codes for pre-existing diabetes and as having gestational hypertension only if she did not have codes for pre-existing hypertension or preeclampsia/eclampsia.
Appendix 2.
Diagnostic Codes Used to Define Maternal Comorbidities
| ICD-9-CM Definition | |
|---|---|
| Pulmonary hypertension | 416.0x, 416.8x, 416.9x |
| Placenta previa | 641.0x, 641.1x |
| Sickle cell disease | 282.4x, 282.6x |
| Gestational hypertension | 642.3x (without preeclampsia/eclampsia or pre-existing hypertension) |
| Mild preeclampsia or unspecified preeclampsia | 642.4x, 642.7x (without severe preeclampsia/eclampsia) |
| Severe preeclampsia/eclampsia | 642.5x, 642.6x |
| Chronic renal disease | 581.x–583.x, 585.x, 587.x, 588.x, 646.2x |
| Preexisting hypertension | 401.x–405.x, 642.0x–642.2x, 642.7x |
| Chronic ischemic heart disease | 412.x–414.x |
| Congenital heart disease | 745.0x–747.4x, 648.5x |
| Systemic lupus erythematosus | 710.0x |
| Human immunodeficiency virus | 042.x, V08.x |
| Multiple gestation | V27.2–V27.8, 651.x |
| Drug abuse | 304.x, 305.2x–305.9x, 648.3x |
| Alcohol abuse | 291.xx, 303.xx, 305.0x |
| Tobacco use | 305.1.x, 649.0x |
| Cardiac valvular disease | 394.x–397.x, 424.x |
| Chronic congestive heart failure | 428.22, 428.23, 428.32, 428.33, 428.42, 428.43 |
| Asthma | 493.x |
| Preexisting diabetes mellitus | 250.x, 648.0x |
| Gestational diabetes mellitus | 648.8x (without pre-existing diabetes) |
| Obesity | 278.0x, 649.1x ,V85.3, V85.4 |
| Cystic fibrosis | 277.0x |
| Previous cesarean delivery | 654.2x |
ICD-9-CM, International Classification of Diseases, 9th Revision, Clinical Modification.
Development of the maternal comorbidity index
We randomly allocated 2/3 of the cohort to a development sample (n=569,882) and 1/3 to a validation sample (n=284,941). Using the development sample, we constructed a multivariable logistic regression model using a fully stepwise selection algorithm that requires covariates to have a p-value ≤0.05 for both entry and retention in the model. The dependent variable was the presence of maternal end-organ injury or death during the delivery hospitalization through 30 days postpartum. The candidate independent variables included the 24 maternal comorbidities defined (see Appendix 2, http://links.lww.com/xxx) as well as maternal age categorized as <19, 20–34, 35–39, 40–44, and >44 years at the time of LMP.
The final model included 20 maternal conditions and maternal age. Using the beta coefficients from the final logistic regression model, we applied the weighting rule described by Schneeweiss and colleagues.(1) In this rule, conditions with a beta coefficient (which corresponds to the natural logarithm of the odds ratio) ≤ 0.15 are assigned a weight of zero and for each 0.3 increase in the beta coefficient, the weight assigned to individual conditions is increased by 1 point.
For each patient in the cohort, the presence or absence of each of the 20 comorbidities included in the final model was defined and each of these conditions was weighted, as described above. Patients’ comorbidity index was then obtained by summing the weights for all comorbidities present and adding it to the relevant weight for the patients’ age. Importantly, patients were allowed to receive weights for mild or unspecified preeclampsia only if they did not also have severe preeclampsia/eclampsia and gestational hypertension only if they did not also have pre-existing hypertension or preeclampsia/eclampsia present.
Validation of the maternal comorbidity index
The performance characteristics of the newly derived score were then assessed using the validation cohort. A logistic regression model was constructed including the primary outcome as the dependent variable and the maternal comorbidity index as a continuous independent variable. The discrimination of the model was evaluated by calculating a c-statistic (the area under the receiver operating curve). The calibration of the score was assessed by plotting the observed risk of the primary outcome by the maternal comorbidity index (divided into 7 categorizes: 0, 1–2, 3–4, 5–6, 7–8, 9–10, and >10).
To test the generalizability of the score to other measures of severe maternal morbidity, each of the performance characteristics described above were tested for the outcome of maternal intensive care unit admission during the delivery admission through 30 days post delivery using the validation cohort.
Comparative assessment of the index with existing comorbidity scores
For each patient in the validation cohort, using claims from both the inpatient and outpatient records assessed from the pre-pregnancy period through the delivery hospitalization, we calculated the Romano adaptation of the Charlson Comorbidity Index(5), the van Walraven numerical modification of the Elxihauser score,(8) and the Combined Comorbidity Score, which includes conditions from the Charlson and Elixhauser measures in a single score.(2) The discrimination of each of these scores for the study endpoints was tested by constructing a logistic regression model for each score with the outcome (primary and secondary were separately assessed) included as the dependent variable and the numerical score for each patient included as a continuous independent variable; the c-statistic for each score was then calculated.
We next compared the predictive performance of the maternal comorbidity index with the Romano/Charlson index, van Walraven/Elxihauser score, and the Combined Comorbidity Score using a reclassification measure. From the logistic regression models described above, we defined the predicted probability of the primary outcome for each patient from each scoring system. We then defined 4 categories of predicted risk, <2%, ≥2 to <5%, ≥5 to <10%, and ≥10% corresponding to thresholds that members of the study team deemed as defining low, moderate, high and very-high risk groups. We created a series of tables with the categories of predicted risk for the maternal comorbidity index plotted against predicted risk for each of the three other comorbidity indexes evaluated, noting the observed risk in each cell. We then calculated the net reclassification improvement (NRI) for the maternal comorbidity index compared with each of the alternative scores.(26) The NRI is calculated as [Pr(up|O = 1) − Pr(down|O = 1)] + [Pr(down|O = 0) − Pr(up|O = 0)], where O = 1 if the primary study outcome occurred (maternal death or end-organ injury from the delivery admission through 30 days postpartum) and O = 0 if the primary outcome did not occur; “up” and “down” are defined by whether an individual was reclassified into a higher or lower predicted risk category, by the maternal comorbidity index.(2, 26) Thus, for example, “Pr(up|O = 1)” defines the probability of being re-classified to a higher risk category if the outcome is present and “Pr(down|O = 1)” is the probability of being re-classified in a lower risk group if the outcome occurs. Stated differently, the NRI reflects the sum of correct reclassification by the new maternal comorbidity index (i.e., moving patients that had the primary outcome into higher risk strata and those that did not have the outcome into lower risk strata) minus the sum of the incorrect reclassifications.(2) A positive NRI therefore indicates better predictive ability for the new score than the score to which it is being compared. The statistical significance of the NRI for each comparison was determined using the test of the null hypothesis, NRI=0, derived by Pencina.(26)
Results
We identified 854,823 completed pregnancies for analysis. The cohort was randomly divided into a development cohort that included 569,882 pregnancies and a validation cohort that included 284,941 pregnancies. Overall, 9,901 (1.16%) of pregnancies were complicated by the primary study outcome, maternal end-organ injury or death during the delivery admission through 30 days postpartum (including 6,606 (1.16%) in the development cohort and 3,295 (1.16%) in the validation cohort). The secondary study outcome, maternal intensive care unit admission during the delivery hospitalization through 30 days postpartum, occurred in 2,451 (0.29%) of pregnancies.
Table 1 shows the frequency of the each type of maternal end-organ injury, which in composite make up the primary study outcome. The most common types of maternal end-organ injury identified included sepsis (0.26%), acute heart failure (0.23%), disseminated intravascular coagulation/coagulopathy (0.17%), acute respiratory distress syndrome/respiratory failure (0.16%), acute liver disease (0.13%), and pulmonary edema (0.12%).
Table 1.
Distribution of Primary Outcome and Its Specific Morbidity Components in the Total Cohort
| Condition | n (%) |
|---|---|
| End-organ injury or death (Primary study outcome) | 9901 (1.16) |
| Components of primary outcome | |
| Acute heart failure | 2000 (0.23) |
| Acute liver disease | 1104 (0.13) |
| Acute myocardial infarction | 65 (0.01) |
| Acute respiratory distress syndrome/respiratory failure | 1342 (0.16) |
| Acute renal failure | 339 (0.04) |
| Coma | 31 (0.00) |
| Delirium | 80 (0.01) |
| Disseminated intravascular coagulation/coagulopathy | 1417 (0.17) |
| Puerperal cerebrovascular disorders | 705 (0.08) |
| Pulmonary edema | 998 (0.12) |
| Pulmonary embolism | 397 (0.05) |
| Sepsis | 2214 (0.26) |
| Shock | 302 (0.04) |
| Status asthmaticus | 202 (0.02) |
| Status epilepticus | 35 (0.00) |
| Death | 23 (0.00) |
Table 2 shows the distribution of each of the potential predictors stratified by whether the pregnancy was complicated by the primary study endpoint. All of the antepartum maternal conditions assessed were more common in women whose pregnancies were complicated by end-organ injury or death, with the exception of tobacco use. Women who experienced the primary study outcome were slightly older than those that were unaffected.
Table 2.
Potential Conditions for Inclusion in the Maternal Comorbidity Index Stratified by the Occurrence of the Primary Study Endpoint in the Total Cohort
| With End- Organ Injury or Death n (%) (n=9,901) |
Without End- Organ Injury or Death n (%) (n=844,922) |
|
|---|---|---|
| Maternal age at LMP, years | ||
| <19 | 1526 (15.4) | 158757 (18.8) |
| 20–34 | 7289 (73.6) | 636018 (75.3) |
| 35–39 | 806 (8.1) | 39722 (4.7) |
| 40–44 | 259 (2.6) | 9814 (1.2) |
| >44 | 21 (0.2) | 611 (0.1) |
| Alcohol abuse | 230 (2.3) | 11220 (1.3) |
| Asthma | 1202 (12.1) | 72688 (8.6) |
| Cardiac valvular disease | 320 (3.2) | 7643 (0.9) |
| Chronic congestive heart failure | * | 32 (0) |
| Chronic ischemic heart disease | 115 (1.2) | 1813 (0.2) |
| Chronic renal disease | 290 (2.9) | 11017 (1.3) |
| Congenital heart disease | 432 (4.4) | 7547 (0.9) |
| Cystic fibrosis | * | 732 (0.1) |
| Drug abuse | 824 (8.3) | 40325 (4.8) |
| Gestational hypertension | 277 (2.8) | 19872 (2.4) |
| Gestational diabetes mellitus | 623 (6.3) | 48205 (5.7) |
| Human immunodeficiency virus | 107 (1.1) | 3827 (0.5) |
| Mild or unspecified preeclampsia | 671 (6.8) | 30337 (3.6) |
| Multiple gestation | 461 (4.7) | 16762 (2.0) |
| Obesity | 555 (5.6) | 32951 (3.9) |
| Placenta previa | 520 (5.3) | 28157 (3.3) |
| Previous cesarean delivery | 2441 (24.7) | 141080 (16.7) |
| Preexisting diabetes mellitus | 818 (8.3) | 38098 (4.5) |
| Preexisting hypertension | 1220 (12.3) | 41790 (5.0) |
| Pulmonary hypertension | 72 (0.7) | 507 (0.1) |
| Severe preeclampsia/eclampsia | 835 (8.4) | 12529 (1.5) |
| Sickle cell disease | 83 (0.8) | 2641 (0.3) |
| Systemic lupus erythematosus | 56 (0.6) | 1437 (0.2) |
| Tobacco use | 971 (9.8) | 82717 (9.8) |
LMP, last menstrual period.
Cell size less than 11, which cannot be disclosed in accordance with the data use agreement.
The final model from the stepwise selection algorithm used to identify predictors of the primary study outcome is shown in Table 3, along with the assigned weights that make up the maternal comorbidity index. When the maternal comorbidity index was calculated for each patient in the validation cohort and a logistic regression model predicting the primary outcome was fitted including the score as a continuous independent variable, the odds ratio per point increase in the comorbidity index was 1.37, 95% Confidence Interval (CI) 1.35 to 1.39. The c-statistic for this model was 0.657 (95% CI 0.647 – 0.666), indicating moderate discrimination.
Table 3.
Results of the Derived Model Predicting Maternal End-Organ Injury or Death and the Associated Weights for Each Condition
| Beta Coefficient | Odds Ratio (95% CI) |
Weights | |
|---|---|---|---|
| Severe preeclampsia/eclampsia | 1.63 | 5.10 (4.63–5.60) | 5 |
| Chronic congestive heart failure | 1.37 | 3.93 (1.35–11.47) | 5 |
| Congenital heart disease | 1.34 | 3.81 (3.37–4.32) | 4 |
| Pulmonary hypertension | 1.18 | 3.24 (2.31–4.56) | 4 |
| Chronic ischemic heart disease | 1.00 | 2.72 (2.13–3.46) | 3 |
| Sickle cell disease | 0.76 | 2.14 (1.63–2.81) | 3 |
| Multiple gestation | 0.74 | 2.09 (1.86–2.35) | 2 |
| Cardiac valvular disease | 0.67 | 1.95 (1.67–2.27) | 2 |
| Systemic lupus erythematosus | 0.57 | 1.77 (1.24–2.52) | 2 |
| Human immunodeficiency virus | 0.56 | 1.76 (1.37–2.27) | 2 |
| Mild or unspecified preeclampsia* | 0.50 | 1.65 (1.49–1.83) | 2 |
| Drug abuse | 0.49 | 1.63 (1.48–1.79) | 2 |
| Placenta previa | 0.48 | 1.61 (1.45–1.80) | 2 |
| Chronic renal disease | 0.43 | 1.54 (1.32–1.80) | 1 |
| Pre-existing hypertension | 0.39 | 1.48 (1.36–1.61) | 1 |
| Previous cesarean delivery | 0.37 | 1.45 (1.37–1.54) | 1 |
| Gestational hypertension‡ | 0.28 | 1.32 (1.14–1.54) | 1 |
| Alcohol abuse | 0.27 | 1.31 (1.11–1.56) | 1 |
| Asthma | 0.25 | 1.28 (1.19–1.39) | 1 |
| Pre-existing diabetes mellitus | 0.19 | 1.21 (1.1–1.33) | 1 |
| Maternal age, years | |||
| >44 | 0.81 | 2.25 (1.28–3.95) | 3 |
| 40–44 | 0.54 | 1.72 (1.47–2.02) | 2 |
| 35–39 | 0.42 | 1.52 (1.39–1.66) | 1 |
CI, confidence interval.
Patients received a weight for mild or unspecified preeclampsia only if they did not have severe preeclampsia/eclampsia present.
Patients received a weight for gestational hypertension only if they did not have preexisting hypertension or preeclampsia/eclampsia present.
The observed risk of the primary outcome by category of maternal comorbidity index is shown in Figure 1. The risk increased from 0.68% in patients with a score of 0 to 10.9% in those with a score of >10. The maximum range for the maternal comorbidity index is 0 to 45. The range for women in the validation cohort was 0 to 19. The mean score was 0.91 (s.d., 1.42) and the median score was 0 (interquartile range, 0–1).
Figure 1.
Observed incidence of primary outcome by maternal comorbidity index in the validation cohort.
The logistic regression model predicting the secondary outcome, maternal ICU admission, yielded an OR per point increase in the comorbidity index of 1.36, 95% CI 1.33 to 1.40. The c-statistic for this model was 0.651 (95% CI 0.631 – 0.670), again indicating moderate discrimination. The observed risk of the secondary outcome by category of index is shown in Figure 2. The risk increased from 0.18% in those with a score of 0 to 2.7% in those with a score of >8 (observed risk at higher levels of score cutoff cannot be displayed owing to small cell sizes).
Figure 2.
Observed incidence of maternal intensive care unit admission by maternal comorbidity index in the validation cohort.
Table 4 shows the net reclassification statistics comparing the maternal comorbidity index to the other commonly used indexes using the validation cohort. The NRI for the maternal comorbidity index was 0.118 (p<0.001) when compared to Charlson/Romano Index, 0.071 (p<0.001) when compared with the van Walraven/Elxihauser Score, and 0.027 (p=0.011) when compared with the Combined Comorbidity Score. We also calculated the c-statistic for each of the scores. For the primary outcome, the c-statistic for the Charlson/Romano Index was 0.578 (95% CI 0.570 – 0.585), for the van Walraven/Elxihauser Score was 0.586 (95% CI 0.575 – 0.597), and for the Combined Comorbidity Score was 0.617 (95% CI 0.608 – 0.627). For the secondary outcome, the c-statistic for the Charlson/Romano Index was 0.560 (95% CI 0.545 – 0.574), for the van Walraven/Elxihauser Score IHS was 0.565 (95% CI 0.544 – 0.585), and for the Combined Comorbidity Score was 0.563 (95% CI 0.545 – 0.582). These findings suggest significantly improved discriminative ability of the maternal comorbidity index for the primary and secondary study endpoints.
Table 4.
Reclassification and Discrimination Statistics Comparing the Maternal Comorbidity Index With Other Commonly Used Comorbidity Indexes and the C-statistics for Each of the Comorbidity Indexes for the Primary Outcome in the Validation Sample.
| 95% Confidence Interval |
P | ||
|---|---|---|---|
| Reclassification statistics for primary outcome | |||
| Maternal morbidity index vs. Charlson/Romano Index | |||
| Net Reclassification Index | 0.118 | 0.103–0.133 | <0.001 |
| Pr(up|O = 1) | 0.190 | 0.177–0.204 | |
| Pr(down|O = 1) | 0.039 | 0.033–0.046 | |
| Pr(up|O = 0) | 0.051 | 0.050–0.052 | |
| Pr(down|O = 0) | 0.017 | 0.017–0.018 | |
| Maternal morbidity index vs. van Walraven/Elixhauser Score | |||
| Net Reclassification Index | 0.071 | 0.054–0.088 | <0.001 |
| Pr(up|O = 1) | 0.179 | 0.167–0.193 | |
| Pr(down|O = 1) | 0.112 | 0.102–0.123 | |
| Pr(up|O = 0) | 0.051 | 0.050–0.052 | |
| Pr(down|O = 0) | 0.054 | 0.053–0.055 | |
| Maternal morbidity index vs. Combined Comorbidity Score | |||
| Net Reclassification Index | 0.027 | 0.010–0.045 | 0.0114 |
| Pr(up|O = 1) | 0.166 | 0.154–0.179 | |
| Pr(down|O = 1) | 0.140 | 0.128–0.152 | |
| Pr(up|O = 0) | 0.048 | 0.047–0.049 | |
| Pr(down|O = 0) | 0.049 | 0.048–0.050 | |
| C-statistic (95% confidence interval) for primary outcome | |||
| Maternal comorbidity index | 0.657 | 0.647–0.666 | |
| Charlson/Romano Index | 0.578 | 0.570–0.585 | |
| van Walraven/Elixhauser Score | 0.586 | 0.575–0.597 | |
| Combined Comorbidity Score | 0.617 | 0.608–0.627 | |
Pr(up|O = 1), the proportion of patients with the primary outcome who were correctly reclassified into a higher risk stratum by the maternal morbidity index; Pr(down|O = 1), the proportion of patients with the primary outcome who were incorrectly reclassified into a lower risk stratum by the maternal morbidity index; Pr(up|O = 0), the proportion of patients without the primary outcome who were incorrectly reclassified into a higher risk stratum by the maternal morbidity index; Pr(down|O = 0), the proportion of patients without the primary outcome who were correctly reclassified into a lower risk stratum by the maternal morbidity index.
We also tested the performance of more parsimonious implementations of the maternal comorbidity index. When we only included conditions with a weight of greater than one, the c-statistic for predicting the primary outcome in the validation cohort fell to 0.634 (95% CI 0.625 – 0.643), which compares to 0.657 (95% CI 0.647 – 0.666) for the full model. When we included only conditions with a weight greater than or equal to two, the c-statistic further fell to 0.607 (95% CI 0.599 – 0.615).
Discussion
Using this cohort of 854,823 completed pregnancies for which inpatient and outpatient claims were available from 6 months prior to pregnancy through 30 days postpartum, we have developed and validated a simple numerical score that summarizes obstetric and medical comorbidities and predicts severe maternal morbidity and mortality. As epidemiologic, comparative effectiveness, and health services research aimed at improving maternal outcomes increasingly becomes a priority area in obstetric research,(20) this comorbidity index promises to be an important tool for use in such work. Code to implement the index is freely available online at: [URL TBD].
Existing comorbidity scores, such as the Charlson/Romano Index, the van Walraven/Elxihauser Score, or the Combined Comorbidity Score, lack relevant obstetrical conditions and use weighting schemes that are not necessarily applicable to obstetric outcomes. Nevertheless, a large number of studies in obstetrics have applied these non-obstetric scores when describing and/or adjusting for comorbidities. Our maternal comorbidity index performed significantly better in predicting the primary study outcome, acute maternal end-organ injury or death, than did these other scores as measured by the NRI and c-statistic (measures of calibration and discrimination). For example, as compared to the commonly used Charlson Index, our maternal comorbidity index correctly reclassified 20.8% of women while incorrectly reclassifying only 9.0%, indicating improvement in classification for 11.8% of the validation cohort. More accurately measuring a strong risk factor for such a large portion of a study population can substantially reduce confounding bias in observational studies.(27)
Having such a summary score for use in obstetric research is particularly important given the relative infrequency with which severe maternal morbidity and mortality occurs in developed countries.(23, 24, 28) While ideally studies examining the effect of a risk factor or intervention on the risk of adverse maternal outcomes would measure and individually adjust for all relevant confounders using a regression model or the like, with infrequent outcomes this is not always possible without over-specifying the model. In these circumstances, an aggregate measure of the burden of relevant comorbidities, weighted in a manner that is relevant to the outcome of interest, becomes a highly useful tool for confounding control.
The scoring system described here may have clinical utility as well. It may be helpful in identifying patients that would benefit from consultation by a Maternal-Fetal Medicine Specialist before and/or during pregnancy. It may also be useful as a means of triaging patients to high-risk centers that have the infrastructure, staffing, and subspecialty expertise required to care for those whose score suggests significant risk for a complicated delivery or postpartum course. Leaders in obstetrics have recently called for the development of networks with hospitals designated by levels of maternal care.(29) This tool may find application as a screening tool for ensuring that parturients are being cared for in an appropriate setting within such a network. Trials to test these and other possible applications of a maternal comorbidity index would provide important information to improve clinical practice.
The development and validation of this maternal comorbidity index is subject to certain limitations. The weights assigned to a particular comorbidity reflect the average population effect of that comorbidity on outcome. However, there is clearly a spectrum of severity for each of the conditions included in our analysis. Thus, while an inherent part of any such index, this may result in residual confounding when the tool is used for adjustment in research. It also means that practitioners need to exercise judgment if the score is applied in a clinical context (e.g., a patient with pulmonary hypertension may have a score of 4, but if the patient’s pulmonary pressures are supra-systemic, the risk of adverse outcome may substantially exceed the average patient with a score 4). An additional limitation is that the scoring system only achieves moderate discrimination for both the primary and secondary endpoint (c-statistic of 0.66 and 0.65, respectively). This likely reflects the fact that many etiologies of severe maternal morbidity, including postpartum hemorrhage and stroke,(24, 30, 31) frequently occur in the absence of recognized risk factors. It may also reflect the fact that certain comorbidities, including maternal obesity, are not well coded in healthcare utilization data. In order to accurately identify pregnancies within the Medicaid Analytic eXtract, we require a linkage to infant records. This means that our cohort is limited to completed pregnancies; some women with severe systemic disease may elect to terminate pregnancy and would not be captured in our analysis. Additionally, maternal deaths are identified in the Medicaid Analytic eXtract using the eligibility file which may underestimate deaths. However, as deaths represent only a small fraction of the composite endpoint and generally occur in the setting of measured maternal end-organ injury, this limitation is not likely to substantially alter our results. It is important to note that while we restricted the development and validation cohorts to those women with Medicaid coverage from 6 months antepartum through 30 days postpartum to maximize our measurement of comorbid illness, this coverage criterion is not necessary when applying the score in practice. Finally, our index was developed in a Medicaid population. While we expect that the associations of maternal comorbidities to adverse outcomes defined in this study will hold in other groups, future studies will need to determine the extent to which the index needs to be tailored for optimal performance in other settings including the commercially insured population in the United States and pregnant populations in other countries. However, it is notable that the existing comorbidity scores have found wide application beyond the populations in which they were originally developed. Further, as Medicaid covers nearly half of all deliveries in the United States, a scoring system optimized for this population is of substantial interest, particularly in light of the relatively high risk of maternal morbidity in this group.(32)
With the substantial rise in the incidence of severe maternal morbidity that has occurred over the past decade,(23) there is an urgent need for research into the determinants of such morbidity and interventions aimed at preventing it. Much of this research has been, and will likely continue to be, based on administrative data. The maternal comorbidity index provides a simple approach to summarizing medical and obstetric comorbidities and can be used for confounding control in such research.
ACKNOWLEDGEMENTS
The authors thank Helen Mogun for assistance with data analysis.
The Medicaid Analytic eXtract (MAX) pregnancy cohort was supported by the Agency for Healthcare Research and Quality (AHRQ) (Grant R01HS018533). SHD has consulted for Novartis, GSK-Biologics and AstraZenaca for unrelated projects. Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute Of Child Health & Human Development of the National Institutes of Health under Award Number K08HD075831 (BTB). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Centers for Disease Control and Prevention.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Financial Disclosure
The authors did not report any potential conflicts of interest.
References
- 1.Schneeweiss S, Wang PS, Avorn J, Glynn RJ. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health services research. 2003 Aug;38(4):1103–1120. doi: 10.1111/1475-6773.00165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. Journal of clinical epidemiology. 2011 Jul;64(7):749–759. doi: 10.1016/j.jclinepi.2010.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. Journal of chronic diseases. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 4.Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. Journal of clinical epidemiology. 1992 Jun;45(6):613–619. doi: 10.1016/0895-4356(92)90133-8. [DOI] [PubMed] [Google Scholar]
- 5.Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. Journal of clinical epidemiology. 1993 Oct;46(10):1075–1079. doi: 10.1016/0895-4356(93)90103-8. discussion 81–90. [DOI] [PubMed] [Google Scholar]
- 6.D'Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson comorbidity index. Methods of information in medicine. 1993 Nov;32(5):382–387. [PubMed] [Google Scholar]
- 7.Ghali WA, Hall RE, Rosen AK, Ash AS, Moskowitz MA. Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data. Journal of clinical epidemiology. 1996 Mar;49(3):273–278. doi: 10.1016/0895-4356(95)00564-1. [DOI] [PubMed] [Google Scholar]
- 8.Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Medical care. 1998 Jan;36(1):8–27. doi: 10.1097/00005650-199801000-00004. [DOI] [PubMed] [Google Scholar]
- 9.van Walraven C, Austin PC, Jennings A, Quan H, Forster AJ. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Medical care. 2009 Jun;47(6):626–633. doi: 10.1097/MLR.0b013e31819432e5. [DOI] [PubMed] [Google Scholar]
- 10.Krieger EV, Landzberg MJ, Economy KE, Webb GD, Opotowsky AR. Comparison of risk of hypertensive complications of pregnancy among women with versus without coarctation of the aorta. The American journal of cardiology. 2011 May 15;107(10):1529–1534. doi: 10.1016/j.amjcard.2011.01.033. [DOI] [PubMed] [Google Scholar]
- 11.Opotowsky AR, Siddiqi OK, D'Souza B, Webb GD, Fernandes SM, Landzberg MJ. Maternal cardiovascular events during childbirth among women with congenital heart disease. Heart. 2012 Jan;98(2):145–151. doi: 10.1136/heartjnl-2011-300828. [DOI] [PubMed] [Google Scholar]
- 12.Nguyen GC, Boudreau H, Harris ML, Maxwell CV. Outcomes of obstetric hospitalizations among women with inflammatory bowel disease in the United States. Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association. 2009 Mar;7(3):329–334. doi: 10.1016/j.cgh.2008.10.022. [DOI] [PubMed] [Google Scholar]
- 13.Murthy SK, Heathcote EJ, Nguyen GC. Impact of cirrhosis and liver transplant on maternal health during labor and delivery. Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association. 2009 Dec;7(12):1367–1372. doi: 10.1016/j.cgh.2009.08.008. 72 e1. [DOI] [PubMed] [Google Scholar]
- 14.Wright JD, Devine P, Shah M, Gaddipati S, Lewin SN, Simpson LL, et al. Morbidity and mortality of peripartum hysterectomy. Obstetrics and gynecology. 2010 Jun;115(6):1187–1193. doi: 10.1097/AOG.0b013e3181df94fb. [DOI] [PubMed] [Google Scholar]
- 15.Panchal S, Arria AM, Harris AP. Intensive care utilization during hospital admission for delivery: prevalence, risk factors, and outcomes in a statewide population. Anesthesiology. 2000 Jun;92(6):1537–1544. doi: 10.1097/00000542-200006000-00009. [DOI] [PubMed] [Google Scholar]
- 16.Cheesman K, Brady JE, Flood P, Li G. Epidemiology of anesthesia-related complications in labor and delivery, New York State, 2002–2005. Anesthesia and analgesia. 2009 Oct;109(4):1174–1181. doi: 10.1213/ane.0b013e3181b2ef75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Jin Y, Carriere KC, Marrie TJ, Predy G, Johnson DH. The effects of community-acquired pneumonia during pregnancy ending with a live birth. American journal of obstetrics and gynecology. 2003 Mar;188(3):800–806. doi: 10.1067/mob.2003.175. [DOI] [PubMed] [Google Scholar]
- 18.Dolan GP, Myles PR, Brett SJ, Enstone JE, Read RC, Openshaw PJ, et al. The comparative clinical course of pregnant and non-pregnant women hospitalised with influenza A(H1N1)pdm09 infection. PloS one. 2012;7(8):e41638. doi: 10.1371/journal.pone.0041638. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Needleman J, Minnick AF. Anesthesia provider model, hospital resources, and maternal outcomes. Health services research. 2009 Apr;44(2 Pt 1):464–482. doi: 10.1111/j.1475-6773.2008.00919.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.D'Alton ME, Bonanno CA, Berkowitz RL, Brown HL, Copel JA, Cunningham FG, et al. Putting the "M" back in maternal-fetal medicine. American journal of obstetrics and gynecology. 2012 Dec 2; doi: 10.1016/j.ajog.2012.11.041. [DOI] [PubMed] [Google Scholar]
- 21.Palmsten K, Huybrechts KF, Mogun H, Kowal MK, Williams PL, Michels KB, et al. Harnessing the Medicaid Analytic eXtract (MAX) to Evaluate Medications in Pregnancy: Design Considerations. PloS one. 2013;8(6):e67405. doi: 10.1371/journal.pone.0067405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Callaghan WM, Mackay AP, Berg CJ. Identification of severe maternal morbidity during delivery hospitalizations, United States, 1991–2003. American journal of obstetrics and gynecology. 2008 Aug;199(2):133 e1–133 e8. doi: 10.1016/j.ajog.2007.12.020. [DOI] [PubMed] [Google Scholar]
- 23.Callaghan WM, Creanga AA, Kuklina EV. Severe maternal morbidity among delivery and postpartum hospitalizations in the United States. Obstetrics and gynecology. 2012 Nov;120(5):1029–1036. doi: 10.1097/aog.0b013e31826d60c5. [DOI] [PubMed] [Google Scholar]
- 24.Mhyre JM, Bateman BT, Leffert LR. Influence of patient comorbidities on the risk of near-miss maternal morbidity or mortality. Anesthesiology. 2011 Nov;115(5):963–972. doi: 10.1097/ALN.0b013e318233042d. [DOI] [PubMed] [Google Scholar]
- 25.Berg CJ, Mackay AP, Qin C, Callaghan WM. Overview of maternal morbidity during hospitalization for labor and delivery in the United States: 1993–1997 and 2001–2005. Obstetrics and gynecology. 2009 May;113(5):1075–1081. doi: 10.1097/AOG.0b013e3181a09fc0. [DOI] [PubMed] [Google Scholar]
- 26.Pencina MJ, D'Agostino RB, Sr, D'Agostino RB, Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in medicine. 2008 Jan 30;27(2):157–172. doi: 10.1002/sim.2929. discussion 207-12. [DOI] [PubMed] [Google Scholar]
- 27.Schneeweiss S, Seeger JD, Maclure M, Wang PS, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. American journal of epidemiology. 2001 Nov 1;154(9):854–864. doi: 10.1093/aje/154.9.854. [DOI] [PubMed] [Google Scholar]
- 28.Berg CJ, Callaghan WM, Syverson C, Henderson Z. Pregnancy-related mortality in the United States, 1998 to 2005. Obstetrics and gynecology. 2010 Dec;116(6):1302–1309. doi: 10.1097/AOG.0b013e3181fdfb11. [DOI] [PubMed] [Google Scholar]
- 29.Hankins GD, Clark SL, Pacheco LD, O'Keeffe D, D'Alton M, Saade GR. Maternal mortality, near misses, and severe morbidity: lowering rates through designated levels of maternity care. Obstetrics and gynecology. 2012 Oct;120(4):929–934. doi: 10.1097/AOG.0b013e31826af878. [DOI] [PubMed] [Google Scholar]
- 30.Bateman BT, Schumacher HC, Bushnell CD, Pile-Spellman J, Simpson LL, Sacco RL, et al. Intracerebral hemorrhage in pregnancy: frequency, risk factors, and outcome. Neurology. 2006 Aug 8;67(3):424–429. doi: 10.1212/01.wnl.0000228277.84760.a2. [DOI] [PubMed] [Google Scholar]
- 31.Bateman BT, Berman MF, Riley LE, Leffert LR. The epidemiology of postpartum hemorrhage in a large, nationwide sample of deliveries. Anesthesia and analgesia. 2010 May 1;110(5):1368–1373. doi: 10.1213/ANE.0b013e3181d74898. [DOI] [PubMed] [Google Scholar]
- 32.Markus AR, Rosenbaum S. The role of Medicaid in promoting access to high-quality, high-value maternity care. Women's health issues : official publication of the Jacobs Institute of Women's Health. 2010 Jan-Feb;20(1 Suppl):S67–S78. doi: 10.1016/j.whi.2009.11.012. [DOI] [PubMed] [Google Scholar]