Abstract
Severe maternal morbidity (SMM) is a composite outcome measure that indicates serious, potentially life-threatening maternal health problems. There is great interest in defining SMM using administrative data for surveillance and research. In the United States, one common way of defining SMM at the population level is an index developed by the Centers for Disease Control and Prevention. Modifications to this index (e.g., exclusion of maternal blood transfusion) have been proposed; some research defines SMM using an index introduced by Bateman et al. (Obstet Gynecol. 2013;122(5):957–965). Birth certificate data are also increasingly being used to define SMM. We compared commonly used US definitions of SMM among all California births (2007–2012) using the kappa (κ) statistic and other measures. We also evaluated agreement between maternal morbidity fields on the birth certificate as compared with health insurance claims data. Concordance between the 7 definitions of SMM analyzed was generally low (i.e., κ < 0.41 for 13 of 21 two-way comparisons). Low concordance was particularly driven by the presence/absence of transfusion and claims data versus birth certificate definitions. Low agreement between administrative data–based definitions of SMM highlights that results can be expected to differ between them. Further research on validity of SMM definitions, using more fine-grained data sources, is needed.
Keywords: data quality, maternal health, perinatal outcomes, quality improvement, reliability, severe maternal morbidity
Abbreviations
- CDC
Centers for Disease Control and Prevention
- ICD-9
International Classification of Diseases, Ninth Revision
- LOS
length of stay
- SMM
severe maternal morbidity
The United States has an alarmingly high rate of maternal mortality compared with other high-resource nations, and it is the only high-resource nation where maternal mortality rates have increased in recent decades. Racial/ethnic inequities in this outcome are substantial and persistent. Maternal death is still a very rare outcome in absolute terms (17.2/100,000 live births between 2011 and 2015 (1)), which highlights the need to better understand maternal health complications that precede it. In recent years, increasing emphasis has also been placed on identifying and studying women with severe maternal morbidity (SMM)—that is, women who have serious, life-threatening complications that presumably could lead to death. SMM carries significant short- and long-term physical, psychological, social, and financial consequences for women, families, and society. In the United States, one common way of defining SMM from administrative data is an index developed by the Centers for Disease Control and Prevention (CDC), which includes pregnancy-related conditions such as eclampsia and amniotic fluid embolism, severe cardiovascular complications like acute myocardial infarction and heart failure, and procedures such as blood transfusion, hysterectomy, and ventilation support (2, 3). The index is based on diagnostic and procedure codes that are typically available in administrative data, enabling population-level surveillance. SMM as measured by this index has also been increasing in the United States in recent years (4).
There is agreement that SMM is an important outcome that merits further study; however, it is measured and defined inconsistently across studies. Detailed review of medical records is currently considered the optimal way to identify true cases of SMM (5–8), but this labor-intensive approach is not feasible for tracking SMM at the population level (9). The CDC index (3) is a common approach to identifying women with SMM using administrative data, and in prior research investigators have examined potential refinements to it in attempts to increase face validity (6, 10). It is not clear that all index subcomponents truly reflect a dire maternal health situation in all circumstances (e.g., blood transfusion (6)), and other factors have been used to increase construct validity (e.g., maternal length of stay (LOS)). Accordingly, modifications to the CDC SMM index have been proposed, such as excluding maternal blood transfusion from the composite index or restricting SMM cases to those with unusually long LOS. Alternatively, some US-based research (11–14) has defined SMM using an index introduced by Bateman et al. (15) in 2013. The Bateman et al. (Bateman) definition focuses on indicators of end organ injury, but even for indicators that overlap with the CDC definition, it uses somewhat different sets of codes (see Web Table 1, available online at https://doi.org/10.1093/aje/kwab077, for details). Finally, there is also increasing interest in using fields on the birth certificate that are likely indicators of SMM (e.g., blood transfusion, uterine rupture) (16–20), despite research suggesting very low sensitivity (21, 22).
Our objective in this study was to compare commonly used US definitions of SMM within a large US population (all California births, 2007–2012). We compared the CDC SMM index and its modifications, the SMM definition developed by Bateman et al. (15), the CDC index adding in codes contained in the Bateman definition, and 2 definitions that incorporate CDC-listed conditions that are available on the birth certificate using the kappa (κ) statistic and other measures of agreement. We also evaluated how each maternal morbidity field on the birth certificate performs in comparison with health insurance claims data. This knowledge will help shed light on whether administrative data–based SMM definitions are comparable to each other. This is important for informing ongoing efforts to monitor and improve maternal health in the United States, facilitate comparisons of findings across studies using varied definitions of SMM, and support refinement of the optimal definition of SMM.
METHODS
This population-based study used the California Office of Statewide Health Planning and Development’s linked birth cohort files from the years 2007–2012. This data source includes data from birth certificates and fetal death records, which were linked with inpatient maternal health insurance claims during the period from pregnancy through 9 months postpartum. We included 3,049,774 births for which the vital record was linked with data from the maternal birth hospitalization. Hospital discharge records contain International Classification of Diseases, Ninth Revision, Clinical Modification, codes related to diagnoses and procedures. We used codes from maternal hospitalizations at birth, as well as information from vital records, to identify women with SMM occurring during the childbirth hospitalization (i.e., we did not address SMM occurring prenatally or postpartum). All study population descriptors (maternal race/ethnicity, education, and age) were derived from vital records.
We calculated the reliability between 7 administrative data–based SMM definitions, some from the literature and some that were refinements based on availability of additional data beyond International Classification of Diseases, Ninth Revision (ICD-9) codes. We describe each measure, its original intended purpose, and how it is measured (i.e., which administrative data sources are used).
Definitions of SMM
CDC index without transfusion.
Prior research has demonstrated low construct validity for SMM cases identified by blood transfusion only, partly due to the fact that the number of units of blood product transfused is not available in ICD-9 codes (i.e., in one study, Main et al. (6) reported that only 26% of these women were deemed to have true severe morbidity). Some amount of blood loss during delivery is physiologically normal, and some experts argue that transfusion of a small amount of blood (e.g., 1 unit) may reflect routine preventive care. Therefore, this first SMM definition utilized the CDC index, excluding transfusion-only cases of SMM. The intention of removing blood transfusion from the index was to exclude false-positive cases (i.e., women with SMM defined by the overall CDC index who did not actually experience a severe and life-threatening complication).
CDC index.
The CDC index of SMM is based on ICD-9 codes and was intended to be used for surveillance and monitoring of SMM across populations and over time using administrative data only (3). Notably, this definition and its derivatives were not intended to be used for comparing facility-level quality of maternity care.
CDC index—prolonged LOS required for diagnoses.
Several studies have used the CDC index but with the added criterion that women who only had diagnosis codes were included only if they had a prolonged LOS after giving birth (≥90th percentile or more than 3 standard deviations above the mean, estimated separately for women with vaginal birth and women with cesarean delivery using hospital discharge data) (2, 6, 23–27). This additional criterion is sometimes used under the rationale that true SMM reflects an acute and medically complex patient prognosis, requiring a significantly longer amount of time in the hospital before a woman’s medical condition can be sufficiently stabilized to enable discharge. We used the criterion of LOS ≥90th percentile for this definition (2 days for vaginal birth, 4 days for primary cesarean birth, and 3 days for repeat cesarean birth in our data). The intention of this added criterion also was to exclude false-positive cases.
CDC index—Bateman modification.
The CDC index with Bateman modification was defined using as a starting point the list of complications from the CDC index, other than blood transfusion (since the Bateman index did not include transfusion). For complications that appear in the Bateman index as well, this definition augments the list of ICD-9 codes to also include codes that appear in the Bateman index but not in the CDC index. The purpose of considering this SMM definition was to assess the degree to which agreement between the CDC SMM index (without transfusion) and the Bateman SMM index might be affected by differing sets of ICD-9 codes used to define the same outcome.
Birth certificate—CDC analog.
Although the standard US birth certificate is intended for registering vital events rather than research, there is interest in using this data source to study SMM because of its accessibility. This composite flags women with any of the following boxes checked on the birth certificate: blood transfusion, eclampsia, anesthetic complications, unplanned hysterectomy, and sepsis. Each of these checkboxes corresponds to an item in the CDC SMM index.
Birth certificate—CDC analog expanded.
This composite variable includes data from checkboxes in the “birth certificate—CDC analog” definition as well as other checkboxes that are also likely to identify women with a serious maternal health complication but do not correspond to specific ICD-9 codes in the CDC SMM index. These additional complications are: other excessive bleeding, admission to the intensive care unit, unplanned operating room procedure following delivery, and seizure during labor (not specified to be an eclamptic seizure, although generally presumed to be).
Bateman definition.
An index was created by Bateman et al. to identify “severe maternal morbidity, defined as the occurrence of acute maternal end-organ injury” (15, p. 958), as the dependent variable for validating an obstetrical comorbidity index. This ICD-9 code–based index has been used to define SMM in prior research (11–14) and was not designed for population-level surveillance or comparing the quality of care between hospitals.
Metrics of concordance between SMM definitions
We calculated the prevalence of each of these definitions of SMM and used several measures of concordance to establish the extent to which they agreed with one another. We calculated the κ statistic, sensitivity, specificity, positive predictive value, negative predictive value, and accuracy. To compare one SMM definition with another, we estimated these measures for all 2-way comparisons between the 7 administrative data–based SMM measures listed above. This enabled assessment of whether the specific SMM metric chosen affects how SMM is measured and reported, which could matter for evaluating studies and tracking SMM trends (especially if concordance between measures is low). We compared SMM metrics using the order above, beginning with the CDC index without transfusion as the first “true SMM” metric and the other 6 as tests, and then the CDC index as the “truth” and the other 5 as tests, and so on.
The κ statistic ranges from −1 to 1, with 0 representing chance. We used the common system to designate the degree of agreement indicated by a given κ value (<0–0.20 = poor; 0.21–0.40 = fair; 0.41–0.60 = moderate; 0.61–0.80 = good; and 0.81–1.00 = very good) (28, 29). We created a heat map to visually depict how the different SMM definitions compared with each other in terms of κ value ranges. We considered the data source (i.e., birth certificate vs. ICD-9 codes) and the presence/absence of transfusion codes (for ICD-9–based definitions) as key factors that differentiated SMM definitions from each other.
Concordance between maternal complications identified using ICD-9 codes and birth certificate checkboxes
In addition, we examined concordance measures for specific conditions listed on the birth certificate that are part of the CDC index and thus have corresponding ICD-9 codes: blood transfusion, eclampsia, anesthetic complications, unplanned hysterectomy, and sepsis. The birth certificate checkbox was considered the test and the ICD-9 code was considered the “truth.”
RESULTS
Among the 3,049,774 births included in these analyses, the frequency of SMM as defined by CDC criteria was 1.3% (Table 1). Women with SMM were more likely to be at least 35 years of age, of lower educational attainment, and non-Hispanic Black or other/missing race/ethnicity (considering “race” as a social construct that is marker for adverse exposures, including racism, that increase risk of SMM (30)).
Table 1.
SMM Status | ||||||
---|---|---|---|---|---|---|
Characteristic |
Total
(n = 3,049,774) |
No SMM
(n = 3,010,021; 98.7%) |
SMM
(n = 39,753; 1.3%) |
|||
No. | % a | No. | % b | No. | % b | |
Maternal race/ethnicity | ||||||
Non-Hispanic White | 829,085 | 27.2 | 820,206 | 98.9 | 8,879 | 1.1 |
Hispanic | 1,579,274 | 51.8 | 1,558,354 | 98.7 | 20,920 | 1.3 |
Non-Hispanic Black | 174,783 | 5.7 | 171,101 | 97.9 | 3,682 | 2.1 |
Asian | 400,422 | 13.1 | 395,136 | 98.7 | 5,286 | 1.3 |
Other/missing data | 66,210 | 2.2 | 65,224 | 98.5 | 986 | 1.5 |
Nativity | ||||||
Foreign-born | 1,313,619 | 43.1 | 1,296,310 | 98.7 | 17,309 | 1.3 |
US-born | 1,736,155 | 56.9 | 1,713,711 | 98.7 | 22,444 | 1.3 |
Mother’s age, years | ||||||
<20 | 267,981 | 8.8 | 263,949 | 98.5 | 4,032 | 1.5 |
20–34 | 2,233,754 | 73.2 | 2,207,312 | 98.8 | 26,442 | 1.2 |
≥35 | 547,838 | 18.0 | 538,579 | 98.3 | 9,259 | 1.7 |
Missing data | 201 | 0.0 | 181 | 90.0 | 20 | 10.0 |
Mother’s educational level | ||||||
High school or less | 1,512,402 | 49.6 | 1,491,329 | 98.6 | 21,073 | 1.4 |
Some college | 699,406 | 22.9 | 690,776 | 98.8 | 8,630 | 1.2 |
College completion | 731,877 | 24.0 | 723,420 | 98.8 | 8,457 | 1.2 |
Unknown/missing data | 106,089 | 3.5 | 104,496 | 98.5 | 1,593 | 1.5 |
Payment type at birthc | ||||||
Private health insurance | 1,428,773 | 46.8 | 1,411,882 | 98.8 | 16,891 | 1.2 |
Public/government plan | 1,512,196 | 49.6 | 1,490,851 | 98.6 | 21,345 | 1.4 |
Uninsured | 61,733 | 2.0 | 60,902 | 98.7 | 831 | 1.3 |
Other/missing data | 47,072 | 1.5 | 46,386 | 98.5 | 686 | 1.5 |
Parity | ||||||
0 (nulliparous) | 1,182,182 | 38.8 | 1,165,351 | 98.6 | 16,831 | 1.4 |
≥1 (multiparous) | 1,854,323 | 60.8 | 1,831,757 | 98.8 | 22,566 | 1.2 |
Missing data | 13,269 | 0.4 | 12,913 | 97.3 | 356 | 2.7 |
Abbreviations: CDC, Centers for Disease Control and Prevention; SMM, severe maternal morbidity.
a Column percentages.
b Row percentages.
c Percentages do not sum to 100 because of rounding.
The prevalence of SMM varied from 0.47% (using the birth certificate—CDC analog definition) to 1.30% (using the original CDC index) (Web Table 2). Prevalence was highest for SMM definitions that included blood transfusion (e.g., 1.13% for SMM with the LOS restriction), and adding in additional ICD-9 codes caused the prevalence to increase substantially from the CDC index—no transfusion definition (prevalence = 0.55%) to the CDC index—Bateman modification definition (0.97%). With some notable exceptions, concordance between the 7 definitions of SMM analyzed was low (i.e., poor or fair agreement; κ < 0.41 for 13 of the 21 two-way comparisons). Of the 21 two-way comparisons, κ was in the moderate range (κ = 0.41–0.60) for 4 comparisons and good or very good (κ = 0.61–1.00) for 4 comparisons (Web Table 2). High agreement was observed for SMM definitions that included similar variables (e.g., presence/absence of transfusion in the index), that used similar lists of complications (e.g., those contained in the CDC index or the Bateman index), and that used similar methods of ascertaining outcomes (i.e., ICD-9 codes from the hospital discharge record vs. the birth certificate). For example, very good agreement (κ = 0.81–1.00) was observed for the CDC index and the modified CDC index with the LOS restriction (κ = 0.927), the CDC index—Bateman modification and the Bateman index (both of which excluded transfusion; κ = 0.918), and the birth certificate analog of the CDC index and the expanded birth certificate index (κ = 0.877).
In contrast, low agreement was observed when 1 or more of the following differed: presence/absence of transfusion, inclusion of complications from the CDC index versus the Bateman index, and ICD-9 definition versus birth certificate definition. For example, agreement between the Bateman SMM index and the CDC index was in the fair range (κ = 0.388); these definitions differ based on lists of complications and the presence of transfusion in the index. Agreement between the CDC index without transfusion and the Bateman index (which also does not include transfusion) was higher (κ = 0.579) but still in the moderate agreement range. When the CDC index was compared with the CDC index—Bateman modification, agreement was moderate (κ = 0.492).
Agreement was poor (κ < 0.21) for all comparisons that compared a birth certificate-based SMM definition with any ICD-9–based SMM definition. For example, agreement between the birth certificate analog of the CDC index and the CDC index without transfusion approached the κ value of 0, which indicates agreement no better than chance (κ = 0.086).
These results were driven uniformly by low sensitivity; specificity was consistently above 99%. Sensitivity was particularly low for the birth certificate–based SMM definitions relative to ICD-9–based definitions, never exceeding 15%. Sensitivity was also low for some 2-way comparisons between ICD-9–based SMM definitions, particularly when blood transfusion was included in the variable that was considered the true measurement (sensitivity = 43.5% when CDC index—Bateman modification was the test and the CDC index was used as the truth). The sensitivity of the Bateman SMM index was low in relation to the CDC index (32.3%) but adequate for the CDC index without transfusion (73.2%).
The heat map (Figure 1) depicts the concordance between SMM definitions. All comparisons where κ values were good or very good (i.e., κ ≥ 0.61) were between definitions that shared a data source (i.e., birth certificate vs. ICD-9 codes). For ICD-9–based definitions, good/very good agreement was observed only between definitions that treated transfusion in the same way (i.e., included or excluded). For ICD-9–based definitions that differed with regard to blood transfusion, agreement was either fair or moderate.
When comparing specific outcomes flagged by birth certificate checkboxes with comparable specific outcomes identified using ICD-9 codes, concordance was also low (Table 2). Although these complications were not ascertained using identical definitions between the data sources, the prevalence of most complications was higher using ICD-9 codes—for example, 0.91% for transfusion (vs. 0.10% on the birth certificate) and 0.09% for hysterectomy (vs. 0.03% for unplanned hysterectomy on the birth certificate). Agreement ranged from κ = 0.013 (for sepsis) to κ = 0.392 (for hysterectomy/unplanned hysterectomy). Considering ICD-9–identified complications as the truth, sensitivity was always below 10%, except for unplanned hysterectomy and hysterectomy (sensitivity = 25.1%). The agreement between “any complication” (considering the 5 complications in this table) was low (κ = 0.123). There were no cases of eclampsia or anesthetic complications identified using the birth certificate checkbox in this population.
Table 2.
Concordance Metric | ||||||||
---|---|---|---|---|---|---|---|---|
Complication | Prevalence Based on Birth Certificates, % | Prevalence Based on ICD-9-CM Codes in Discharge Records, % | κ | Sensitivity, % | Specificity, % | PPV, % | NPV, % | Accuracy, % |
Transfusion | 0.10 | 0.91 | 0.143 | 8.1 | 99.9 | 70.1 | 99.2 | 99.1 |
Eclampsia | 0.00 | 0.07 | —a | 0.0 | 100.0 | — | 99.9 | 99.9 |
Anesthetic complications | 0.00 | 0.01 | — | 0.0 | 100.0 | — | 99.9 | 99.9 |
Unplanned hysterectomy | 0.03 | 0.09 | 0.392 | 25.1 | 100.0 | 89.9 | 99.9 | 99.9 |
Sepsis | 0.35 | 0.04 | 0.013 | 6.7 | 99.7 | 0.8 | 100.0 | 99.6 |
Any complication | 0.47 | 1.04 | 0.123 | 9.4 | 99.6 | 20.8 | 99.1 | 98.7 |
Abbreviations: ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; NPV, negative predictive value; PPV, positive predictive value.
a The κ value and PPV could not be calculated because no cases were identified on the basis of the birth certificate.
DISCUSSION
This analysis demonstrated that agreement between administrative data–based definitions of SMM was generally low, especially when blood transfusion was treated differently by SMM definitions (an expected finding (6)). Inclusion of different maternal complications between SMM definitions (e.g., between the CDC index and Bateman index) contributed to the low agreement between SMM definitions observed here (e.g., often below κ < 0.61). Further, different ICD-9 codes were used to define the same condition (e.g., acute myocardial infarction, shock) between some definitions (i.e., CDC and Bateman), and this also contributed to low levels of agreement. When the CDC index without transfusion was augmented to include the additional codes flagged by the Bateman index, the resulting SMM definition had moderate-to-high agreement with both the CDC index without transfusion (κ = 0.72) and the Bateman index (κ = 0.92). Together, this highlights that many factors significantly impact SMM measurement and lead to low agreement between variable definitions, with implications for surveillance and research.
Agreement between the SMM composite as captured on the birth certificate and traditional claims-based SMM definitions was universally poor (not much better than chance). Concordance between specific SMM complications (e.g., transfusion, sepsis) as measured on birth certificates and in health insurance claims was also poor. Because birth certificate fields are increasingly being used to track and study maternal health, more attention should be paid to understanding what meaningful information they provide, if any. The poor agreement was explained by universally low sensitivity; therefore, if future work demonstrates that SMM identified through this means represents true SMM cases, these fields may be used to augment claims-based SMM definitions, but they should not be understood as valid markers for SMM on their own.
These findings inform several discussions in maternal health research. Multiple federal agencies are collaborating on a careful review of SMM codes to ensure consistency across all programs and uses. Among administrative data–based definitions that are common in the United States, the CDC-based definition has been most extensively validated, and prior work has established that transfusion-only cases do not reliably identify true SMM cases (in the absence of additional data like revenue codes, permitting calculation of the number of units of blood transfused as a proxy for severity) (6). If the preferred measure is the CDC index without transfusion, then our study suggests that none of the other administrative data–based SMM metrics are sufficiently similar (the only exception would be the CDC index—Bateman modification, which was intended to check the impact of different ICD-9 codes rather than SMM-defining conditions). Efforts to refine the CDC SMM index are currently under way, and which conditions are considered SMM powerfully affects its prevalence and which women are considered to have experienced SMM. Perhaps less broadly appreciated is the discrepancy between which ICD-9 codes are used to define a given maternal complication (e.g., in the CDC index vs. the Bateman index); this too was found to affect how and for whom SMM was recorded. Moving forward, researchers and practitioners need to assess the specific codes used to flag SMM, determine which of them should be used for this purpose, and be aware of variability across studies.
Poor concordance between birth certificate- and claims-based SMM definitions is notable but is perhaps unsurprising, given the very different provenance of these data sources. Birth certificate completion varies by state, birth setting, health system, and hospital. Checkboxes are generally completed by the attending health-care provider, a nurse, or a birth certificate clerk; via worksheets given to parents; or by some combination of these methods (31, 32). In contrast, ICD-9 codes are sometimes assigned by the provider, but they are also assigned by billing coders who review the medical record for evidence supporting application of the code to that particular case. These differences probably contribute to the discrepancies observed here; more research is needed to understand how data provenance and collection affect validity and reliability.
The limitations of our study include analyzing data from only 1 state during a time period before SMM measurement and quality improvement were national priorities. Given that birth certificate completion differs between states, it will be important to conduct similar studies in other states to determine whether these findings are generalizable. We focused on a number of US-based definitions of SMM that rely on administrative data, but other definitions exist, some relying on data that are not routinely available at the population level. For example, the SMM definition of the American College of Obstetricians and Gynecologists and other definitions rely on at least 4 units of blood being transfused (33, 34), and the World Health Organization relies on vital signs and laboratory results to indicate system failures (35). The validity of SMM measures should also be examined in the current era of maternal health measurement and quality improvement. Although we used the most recent linked data available, recent maternal health quality-of-care initiatives may be associated with increased recording of maternal complications (36, 37), potentially affecting the validity of measured SMM over time. Measurement of SMM will also be affected by the transition from the ICD-9 to the International Classification of Diseases, Tenth Revision, which occurred after our study period. All SMM definitions examined here relied on some form of administrative data; there is also a need for further studies comparing SMM as flagged in these data sources with a contemporaneous clinical recording of these conditions (while acknowledging that there is no consensus as to what clinicians regard as “true” SMM when looking at the medical record, and even less is known about patients’ perspectives on this definition) (5, 6, 38). Even so, such validation studies will be critical to interpreting population-level studies on SMM. Finally, we have focused only on measurement of SMM; research on patient perspectives on what constitutes SMM and how the validity of SMM measurement might affect associations with other variables (e.g., as in a bias analysis) is needed.
SMM is an essential metric for tracking maternal health at the population level, for increasing knowledge of what causes complications during and after birth, and ultimately for preventing progression to maternal death. These are all high-priority goals in the United States and around the globe. Rigorous analysis of SMM and evidence-based improvement of maternal health requires consistent and valid measurement of SMM and its component complications.
Supplementary Material
ACKNOWLEDGMENTS
Author affiliations: School of Public Health, Oregon Health & Science University and Portland State University, Portland, Oregon, United States (Jonathan M. Snowden); Department of Obstetrics and Gynecology, School of Medicine, Oregon Health & Science University, Portland, Oregon, United States (Jonathan M. Snowden); Rory Meyers College of Nursing, New York University, New York, New York, United States (Audrey Lyndon); Department of Pediatrics, School of Medicine, Stanford University, Stanford, California, United States (Peiyi Kan, Suzan L. Carmichael); Department of Obstetrics, Gynecology and Reproductive Sciences, School of Medicine, University of California, San Francisco, San Francisco, California, United States (Alison El Ayadi); Department of Epidemiology and Biostatistics, School of Medicine, University of California, San Francisco, San Francisco, California, United States (Alison El Ayadi); California Maternal Quality Care Collaborative, Stanford, California, United States (Elliott Main); and Department of Obstetrics and Gynecology, School of Medicine, Stanford University, Stanford, California, United States (Elliott Main, Suzan L. Carmichael).
This work was funded by the National Institute of Nursing Research (grant R01 NR017020 to S.L.C.).
Conflict of interest: none declared.
REFERENCES
- 1. Petersen EE, Davis NL, Goodman D, et al. Vital signs: pregnancy-related deaths, United States, 2011–2015, and strategies for prevention, 13 states, 2013–2017. MMWR Morb Mortal Wkly Rep. 2019;68(18):423–429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Callaghan WM, Creanga AA, Kuklina EV. Severe maternal morbidity among delivery and postpartum hospitalizations in the United States. Obstet Gynecol. 2012;120(5):1029–1036. [DOI] [PubMed] [Google Scholar]
- 3. Centers for Disease Control and Prevention . How does CDC identify severe maternal morbidity? https://www.cdc.gov/reproductivehealth/maternalinfanthealth/smm/severe-morbidity-ICD.htm. Page last reviewed December 26, 2019. Accessed December 17, 2019.
- 4. Fingar KR, Hambrick MM, Heslin KC, et al. Trends and Disparities in Delivery Hospitalizations Involving Severe Maternal Morbidity, 2006–2015 . Rockville, MD: Agency for Healthcare Research and Quality; 2018. (Statistical Brief #243). www.hcup-us.ahrq.gov/reports/statbriefs/sb243-Severe-Maternal-Morbidity-Delivery-Trends-Disparities.pdf. Accessed May 9, 2020. [PubMed] [Google Scholar]
- 5. Himes KP, Bodnar LM. Validation of criteria to identify severe maternal morbidity. Paediatr Perinat Epidemiol. 2020;34(4):408–415. [DOI] [PubMed] [Google Scholar]
- 6. Main EK, Abreo A, McNulty J, et al. Measuring severe maternal morbidity: validation of potential measures. Am J Obstet Gynecol. 2016;214(5):643.e1–643.e10. [DOI] [PubMed] [Google Scholar]
- 7. Geller SE, Adams MG, Kominiarek MA, et al. Reliability of a preventability model in maternal death and morbidity. Am J Obstet Gynecol. 2007;196(1):57.e1–57.e6. [DOI] [PubMed] [Google Scholar]
- 8. Geller SE, Koch AR, Garland CE, et al. A global view of severe maternal morbidity: moving beyond maternal mortality. Reprod Health. 2018;15(suppl 1):Article 98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Knight M. Defining severe maternal morbidity—when is it time to stop? Paediatr Perinat Epidemiol. 2020;34(4):384–385. [DOI] [PubMed] [Google Scholar]
- 10. Leonard SA, Carmichael SL, Main EK, et al. Risk of severe maternal morbidity in relation to prepregnancy body mass index: roles of maternal co-morbidities and caesarean birth. Paediatr Perinat Epidemiol. 2020;34(4):460–468. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Friedman AM, Ananth CV, Huang Y, et al. Hospital delivery volume, severe obstetrical morbidity, and failure to rescue. Am J Obstet Gynecol. 2016;215(6):795.e1–795.e14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Lazariu V, Nguyen T, McNutt LA, et al. Severe maternal morbidity: a population-based study of an expanded measure and associated factors. PLoS One. 2017;12(8):e0182343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Clapp MA, James KE, Kaimal AJ. The effect of hospital acuity on severe maternal morbidity in high-risk patients. Am J Obstet Gynecol. 2018;219(1):111.e1–111.e7. [DOI] [PubMed] [Google Scholar]
- 14. Srinivas SK, Kozhimannil K, Hung P, et al. Do hospitals with a higher level of maternal care designation have better maternal outcomes? Am J Perinatol. 2019;36(6):653–658. [DOI] [PubMed] [Google Scholar]
- 15. Bateman BT, Mhyre JM, Hernandez-Diaz S, et al. Development of a comorbidity index for use in obstetric patients. Obstet Gynecol. 2013;122(5):957–965. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. De Silva DA, Thoma ME. The association between interpregnancy interval and severe maternal morbidities using revised national birth certificate data: a probabilistic bias analysis. Paediatr Perinat Epidemiol. 2020;34(4):469–480. [DOI] [PubMed] [Google Scholar]
- 17. Luke B, Brown MB, Wantman E, et al. Risk of severe maternal morbidity by maternal fertility status: a US study in 8 states. Am J Obstet Gynecol. 2019;220(2):195.e1–195.e12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Chen HY, Chauhan SP. Association between gestational weight gain adequacy and adverse maternal and neonatal outcomes. Am J Perinatol. 2019;36(6):615–623. [DOI] [PubMed] [Google Scholar]
- 19. Glance LG, Hasley S, Glantz JC, et al. Measuring childbirth outcomes using administrative and birth certificate data. Anesthesiology. 2019;131(2):238–253. [DOI] [PubMed] [Google Scholar]
- 20. Tanner LD, Chen HY, Sibai BM, et al. Racial and ethnic disparities in maternal and neonatal adverse outcomes in college-educated women. Obstet Gynecol. 2020;136(1):146–153. [DOI] [PubMed] [Google Scholar]
- 21. Luke B, Brown MB, Liu CL, et al. Validation of severe maternal morbidity on the US Certificate of Live Birth. Epidemiology. 2018;29(4):e31–e32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Schummers L. Interpregnancy interval and severe maternal morbidity—what can we learn from vital records data? Paediatr Perinat Epidemiol. 2020;34(4):388–391. [DOI] [PubMed] [Google Scholar]
- 23. Howland RE, Angley M, Won SH, et al. Estimating the hospital delivery costs associated with severe maternal morbidity in New York City, 2008–2012. Obstet Gynecol. 2018;131(2):242–252. [DOI] [PubMed] [Google Scholar]
- 24. Lyndon A, Lee HC, Gilbert WM, et al. Maternal morbidity during childbirth hospitalization in California. J Matern Fetal Neonatal Med. 2012;25(12):2529–2535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Lyndon A, Baer RJ, Gay CL, et al. A population-based study to identify the prevalence and correlates of the dual burden of severe maternal morbidity and preterm birth in California. J Matern Fetal Neonatal Med. 2019;34(8):1198–1206. [DOI] [PubMed] [Google Scholar]
- 26. Ozimek JA, Eddins RM, Greene N, et al. Opportunities for improvement in care among women with severe maternal morbidity. Am J Obstet Gynecol. 2016;215(4):509.e1–509.e6. [DOI] [PubMed] [Google Scholar]
- 27. Guglielminotti J, Landau R, Wong CA, et al. Patient-, hospital-, and neighborhood-level factors associated with severe maternal morbidity during childbirth: a cross-sectional study in New York State 2013–2014. Matern Child Health J. 2019;23(1):82–91. [DOI] [PubMed] [Google Scholar]
- 28. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. [PubMed] [Google Scholar]
- 29. Brennan P, Silman A. Statistical methods for assessing observer variability in clinical measures. BMJ. 1992;304(6840):1491–1494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Boyd RW, Lindo EG, Weeks LD, et al. On racism: a new standard for publishing on racial health inequities [Web log]. Health Affairs Blog. 10.1377/hblog20200630.939347/full. Published July 2, 2020. Accessed July 6, 2020. [DOI]
- 31. Northam S, Knapp TR. The reliability and validity of birth certificates. J Obstet Gynecol Neonatal Nurs. 2006;35(1):3–12. [DOI] [PubMed] [Google Scholar]
- 32. Kirby RS, Demetriou N. Planned home or hospital delivery: what outcomes provide valid comparisons? Am J Obstet Gynecol. 2014;210(5):488–489. [DOI] [PubMed] [Google Scholar]
- 33. American College of Obstetricians and Gynecologists . Severe maternal morbidity: screening and review. Obstetric Care Consensus number 5, September 2016. Obstet Gynecol. 2016;128:e54–e60. [DOI] [PubMed] [Google Scholar]
- 34. Zwart JJ, Richters JM, Ory F, et al. Severe maternal morbidity during pregnancy, delivery and puerperium in the Netherlands: a nationwide population-based study of 371,000 pregnancies. BJOG. 2008;115(7):842–850. [DOI] [PubMed] [Google Scholar]
- 35. Firoz T, Chou D, von Dadelszen P, et al. Measuring maternal health: focus on maternal morbidity. Bull World Health Organ. 2013;91(10):794–796. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Einerson BD, Miller ES, Grobman WA. Does a postpartum hemorrhage patient safety program result in sustained changes in management and outcomes? Am J Obstet Gynecol. 2015;212(2):140–144.e1. [DOI] [PubMed] [Google Scholar]
- 37. Skupski DW, Brady D, Lowenwirt IP, et al. Improvement in outcomes of major obstetric hemorrhage through systematic change. Obstet Gynecol. 2017;130(4):770–777. [DOI] [PubMed] [Google Scholar]
- 38. Wang E, Glazer KB, Sofaer S, et al. Racial and ethnic disparities in severe maternal morbidity: a qualitative study of women’s experiences of peripartum care. Womens Health Issues. 2021;31(1):75–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.