Abstract
Background:
Maternal mortality rates in the United States appear to be increasing. One potential reason may be increased identification of maternal deaths after the addition of a pregnancy checkbox to the death certificate. In 2016, four state health departments (Georgia, Louisiana, Michigan, Ohio) implemented a pregnancy checkbox quality assurance pilot, with technical assistance provided by the Centers for Disease Control and Prevention. The pilot aimed to improve accuracy of the pregnancy checkbox on death certificates and resultant state maternal mortality estimates.
Objective(s):
To estimate the validity of the pregnancy checkbox on the death certificate and describe characteristics associated with errors using 2016 data from a four state quality assurance pilot.
Study Design:
Potential pregnancy-associated deaths were identified by linking death certificates with birth or fetal death certificates from within a year preceding death or by pregnancy checkbox status. Death certificates which indicated the decedent was pregnant within a year of death via the pregnancy checkbox, but that did not link to a birth or fetal death certificate, were referred for active follow-up to confirm pregnancy status by either death certifier confirmation or medical record review. Descriptive statistics and 95% confidence intervals were used to examine the distributions of demographic characteristics by pregnancy confirmation category (i.e., confirmed pregnant, confirmed not pregnant, and unable to confirm). We compared the proportion confirmed pregnant and confirmed not pregnant within age, race/ethnicity, pregnancy checkbox category, and certifier type categories using a Wald test of proportions. Binomial and Poisson regression models were used to estimate prevalence ratios for having an incorrect pregnancy checkbox (false positive, false negative) by age group, race/ethnicity, pregnancy checkbox category, and certifier type.
Results:
Among 467 potential pregnancy-associated deaths, 335 (72%) were confirmed pregnant either via linkage to a birth or fetal death certificate, certifier confirmation, or review of medical records. Ninety-seven (21%) women were confirmed not pregnant (false positives) and 35 (7%) were unable to be confirmed. Women confirmed pregnant were significantly younger than women confirmed not pregnant (p<.001). Deaths certified by coroners and medical examiners were more likely to be confirmed pregnant than confirmed not pregnant (p=0.04). The association between decedent age category and false positive status followed a dose-response relationship (p<0.001), with increasing prevalence ratios for each increase in age category. Death certificates of non-Hispanic black women were more likely to be false positives, compared with non-Hispanic white women [prevalence ratio (PR) 1.41, 95% confidence interval (CI) 1.01, 1.96]. The sensitivity of the pregnancy checkbox among these four states in 2016 was 62% and the positive predictive value was 68%.
Conclusion(s):
We provide a multi-state analysis of the validity of the pregnancy checkbox and highlight a need for more accurate reporting of pregnancy status on death certificates. States and other jurisdictions may increase the accuracy of their data used to calculate maternal mortality rates by implementing quality assurance processes.
Keywords: maternal health, maternal mortality, pregnancy checkbox, quality assurance, quality improvement
Condensation
Results from a four state quality assurance pilot conducted to measure the validity of the pregnancy checkbox on the death certificate.
Background
Maternal mortality is an important public health indicator used to measure the overall health of a nation.1,2 While most of the world has reported a decrease in maternal mortality, the United States has observed an apparent increase.3 The reasons for this increase remain unclear, and accurately quantifying maternal deaths has been a challenge for public health researchers. Historically, the National Center for Health Statistics (NCHS) has published U.S. maternal mortality rate (MMR) using the World Health Organization (WHO) definition.4 Maternal deaths include deaths to women while pregnant or within 42 days of the end of pregnancy, irrespective of the duration or site of pregnancy, from any cause related to or aggravated by the pregnancy or its management but not from incidental or accidental causes.2,5 NCHS assigns International Classification of Diseases, 10th edition (ICD-10) codes for all deaths in the U.S. based on death certificate information, which includes text fields describing the cause of death and significant contributors to the death, as well as information documented by pregnancy checkbox options. The specific ICD codes (A34, O00–95, O98–99) that align with the WHO maternal death definition are included in MMR calculations; however, ICD codes are dependent on the accuracy and completeness of death certificate data. NCHS must be able to determine both causal and temporal relationships to pregnancy for a maternal death to be correctly coded.
One potential contributing factor to the increase in MMR is the pregnancy checkbox added to the U.S. standard certificate of death in 2003 (Figure 1). The pregnancy checkbox requests information on the pregnancy status of all females within the previous year. The checkbox was created in response to studies discovering a 39–93% underreporting of maternal deaths and was expected to improve identification of maternal deaths.6–8 While the pregnancy checkbox may increase the identification, it has also been shown to increase maternal death misclassification.9
Figure 1.
Pregnancy Checkbox from the United States standard certificate of death
NCHS has not published a national MMR since 2007, in part due to the staggered adoption of the revised death certificate resulting in incomparable data between states; and more recently, concerns over errors caused by the pregnancy checkbox, which may be inflating state and national maternal mortality statistics.9–11 A review of state death certificates with checkboxes indicating pregnancies within the preceding year revealed a 15% false positive rate (i.e., death certificates were not able to be linked to a birth or fetal death certificate and had no other evidence of a pregnancy in the preceding year).11 In the 23 states that adopted the standard death certificate prior to 2008, the MMR more than doubled between 1998–2002 and 2008–2012 (9.0 and 22.4, respectively).10 Nearly a third of the observed change in MMR was potentially due to misclassification among women ages ≥40 years. A recent study found no evidence of pregnancy for 50% of Texas death certificates from 2012 in which the pregnancy checkbox indicated pregnancy within the preceding year.9 Amid growing concerns over the quality of the pregnancy checkbox data, and to identify processes for improving accuracy of state maternal mortality estimates, four state health departments piloted a pregnancy checkbox quality assurance (QA) process for 2016 deaths, with technical assistance provided by staff at the CDC. Following the QA process, we measured the validity of the pregnancy checkbox using the pilot data.
Methods and Materials
Staff in four state health departments (Georgia, Louisiana, Michigan, and Ohio) linked all 2016 death certificates among women of reproductive age to 2015 and 2016 birth and fetal death certificates in order to identify deaths among women that had been pregnant within the year preceding death. States used a comparable methodology of matching, based on personal identifiers (i.e., names, social security numbers, addresses). Death certificates that indicated the woman was pregnant at the time of death or within the past year via the pregnancy checkbox that did not link to a birth or fetal death certificate were referred to as ‘checkbox only’ death certificates. Active follow-up was used to confirm the pregnancy status of the checkbox only death certificates by either contacting the death certifier (3 states) or reviewing medical records to search for evidence of pregnancy (1 state). States determined the follow-up method based on existing workflow and staff capacity (e.g., letter from state registrar’s office, phone call from maternal mortality review committee (MMRC) staff, manual medical record review).
Deaths with confirmed pregnancies included 1) deaths with a linked birth or fetal death certificate (regardless of what was entered in the pregnancy checkbox), and 2) deaths that did not link to a birth or fetal death certificate but had an indication of pregnancy in the pregnancy checkbox and the pregnancy was confirmed by either medical record review or death certifier confirmation. Certificates with no link to a birth or fetal death certificate, and where the pregnancy status was unable to be determined by the certifier or medical records, were considered unable to confirm. Deaths of residents of another state were not included in analyses due to difficulty of obtaining records to confirm pregnancy status (n=9).
False-positives were defined as deaths with a pregnancy checkbox marked as pregnant at the time of death or within the preceding year, but no evidence of pregnancy was found through linkage with birth and fetal death certificates or through active follow-up (also referred to as ‘confirmed not pregnant’). False-negatives were defined as deaths that linked to a birth or fetal death certificate within the preceding year, but had a pregnancy checkbox marked as either not pregnant within past year, unknown if pregnant within past year, or the pregnancy checkbox was blank. Proportions and 95% confidence intervals were used to examine the distributions of demographic characteristics and death certificate information by pregnancy confirmation category (i.e., confirmed pregnant, confirmed not pregnant, and unable to confirm). Wilson confidence intervals were used when proportions were 0 or 100. A Kruskal-Wallis test was performed to assess whether median age significantly differed by confirmation category. Death certificates where the pregnancy status was unable to be confirmed were excluded from additional analyses (n=35). Wald tests of proportions were used to test whether the death certificates were equally distributed among those confirmed pregnant and confirmed not pregnant, within age, race/ethnicity, death certificate certifier type (e.g. medical examiner, coroner, physician), and pregnancy checkbox category; p-values <0.05 were considered statistically significant.
Binomial regression models were used to estimate prevalence ratios for having an incorrect pregnancy checkbox (false positive, false negative) by race/ethnicity, certifier type, and pregnancy checkbox category; due to a lack of convergence, Poisson regression models with robust standard errors were used to estimate prevalence ratios for age. Each outcome of interest was coded as a dichotomous variable (false positive versus true positive, false negative versus true positive); all characteristics (age, race/ethnicity, certifier type, and pregnancy checkbox category) were coded as disjoint indicator variables to compare each index category to the referent.
We calculated sensitivity as the proportion of positive pregnancy checkboxes among all death certificates where pregnancy was confirmed. We calculated positive predictive value as the proportion confirmed pregnant among all death certificates with a positive pregnancy checkbox. We calculated specificity as the proportion of women confirmed not pregnant within the past year who had “not pregnant within the past year” recorded on their death certificate. To estimate the total number of deaths among women who were not pregnant within the preceding year (hereafter referred to as ‘true negatives’), the number of deaths among women confirmed pregnant within the preceding year (n=335) and the number of false positives (n=97) were subtracted from the total number of women of reproductive age (15–49 years) who died in 2016 in the four states (n=10,971). Due to the potential overestimation of true negatives caused by the inability to review each of the 10,539 death certificates in this category, we removed a range of 10–50% of the estimated 2016 deaths to women of reproductive age in the four states and re-calculated specificity to assess whether specificity would meaningfully differ based on the number of true negatives included. We also compared the distribution of ICD-10 codes ascribed to the true positive and false positive deaths.
Results
Death certificates with either a positive pregnancy checkbox or a linked birth certificate or fetal death certificate were included in analyses (n=467). Of these, 235 (50.3%) linked to a birth certificate, 20 (4.3%) linked to a fetal death certificate and 212 (45.4%) did not link to either (checkbox only). Among the 212 certificates that did not link but had a positive pregnancy checkbox, 97 (45.8%) of the women were confirmed not pregnant, 80 (37.7%) were confirmed pregnant and 35 (16.3%) had pregnancy status unable to be confirmed (Figure 2). Death certificates that linked to birth or fetal death certificates and death certificates where pregnancy status was confirmed via active follow up were combined into a confirmed pregnant category, which represented 71.7% (n=335) of deaths. Of these 335 confirmed pregnant death certificates, 128 (38.2%) were false negatives, that were linked to birth of fetal death certificates, but did not have a positive pregnancy checkbox. About one third of these had a checkbox value of “Not pregnant,” 63% had a checkbox value of “Unknown,” and 4% had a blank checkbox. Women confirmed pregnant were younger than women confirmed not pregnant and women whose pregnancy status could not be confirmed. The median age of women confirmed pregnant was 29 years (interquartile range (IQR) 24–33), whereas the median age for women confirmed not pregnant was 44 years (IQR 36–50), and those whose pregnancy status was unable to be confirmed was 44 years (IQR 30–52). Thirty-four percent of all women were non-Hispanic black and represented 42.3% of the confirmed not pregnant (false positives) (Table 1). While physicians represented 39.2% of the overall certifiers, they represented 76.3% of the confirmed not pregnant/false positives. Among the deaths confirmed not pregnant, 56 (57.7%) had the pregnancy checkbox value “pregnant within 42 days of death.”
Figure 2.
Classification of Deaths from Quality Assurance Pilot
Table 1.
Characteristics of women and certifiers from death certificates included in the pregnancy checkbox quality assurance pilot by confirmation status, Georgia, Louisiana, Michigan, Ohio, 2016
| Total | Confirmed Pregnanta (i.e., True Positive or False Negative) | Confirmed Not Pregnant (i.e. False Positive) | Unable to Confirm | |
|---|---|---|---|---|
| N=467 | n=335 | n=97 | n=35 | |
| (%) | % (95% CI) | % (95% CI) | % (95% CI) | |
| Age, in years | ||||
| <25 | 102 (21.8) | 27.5 (22.9, 32.5) | 4.1 (1.6, 10.5) | 17.1 (7.9, 33.3) |
| 25–29 | 105 (22.5) | 28.4 (23.8, 33.4) | 9.3 (4.9, 16.9) | 2.9 (0.4, 17.8) |
| 30–34 | 93 (19.9) | 23.3 (19.1, 28.1) | 12.4 (7.1, 20.6) | 8.6 (2.8, 23.5) |
| 35–39 | 65 (13.9) | 15.2 (11.7, 19.5) | 11.3 (6.4, 19.4) | 8.6 (2.8, 23.5) |
| 40–44 | 36 (7.7) | 4.5 (2.7, 7.3) | 16.5 (10.3, 25.3) | 14.3 (6.1, 30.1) |
| 45–49 | 28 (6.0) | 1.2 (0.4, 3.1) | 17.5 (11.2, 26.4) | 20.0 (9.8, 36.5) |
| ≥ 50 | 38 (8.1) | 0.0 (0.0,1.1) | 28.9 (20.7, 38.7) | 28.6 (16.1, 45.5) |
| Maternal race/ethnicity | ||||
| non-Hispanic white | 272 (58.2) | 61.5 (56.1, 66.6) | 52.6 (42.6, 62.3) | 42.9 (27.7, 59.5) |
| non-Hispanic black | 160 (34.3) | 30.1 (25.5, 35.3) | 42.3 (32.8, 52.3) | 51.4 (35.2, 67.3) |
| Hispanic | 23 (4.9) | 5.7 (3.6, 8.7) | 3.1 (1.0, 9.2) | 2.9 (0.4, 17.8) |
| Other | 12 (2.6) | 2.7 (1.4, 5.1) | 2.1 (0.5, 7.9) | 2.9 (0.4, 17.8) |
| Certifier Type | ||||
| Physician | 183 (39.2) | 25.1 (20.7, 30.0) | 76.3 (66.8, 83.7) | 71.4 (54.5, 83.9) |
| Coroner | 195 (41.8) | 49.6 (44.2, 54.9) | 21.6 (14.5, 31.0) | 22.9 (11.8, 39.5) |
| Medical Examiner | 89 (19.1) | 25.4 (21.0, 30.3) | 2.1 (0.5, 7.9) | 5.7 (1.4, 20.2) |
| Checkbox Value | ||||
| Pregnant at time of death | 129 (27.6) | 25.1 (20.7, 30.0) | 34.0 (25.3, 44.0) | 34.3 (20.6, 51.2) |
| Pregnant within 42 days of death | 124 (26.6) | 14.6 (11.2, 18.8) | 57.7 (47.7, 67.2) | 54.3 (37.9, 69.8) |
| Pregnant 43 days-1 year of death | 86 (18.4) | 22.1 (18.0, 26.9) | 8.2 (4.2, 15.7) | 11.4 (4.3, 26.8) |
| Not pregnant within past year | 42 (9.0) | 12.5 (9.4, 16.5) | 0.0 (0.0, 3.8) | 0.0 (0.0, 9.9) |
| Unknown if pregnant within past year | 81 (17.3) | 24.2 (19.9, 29.1) | 0.0 (0.0, 3.8) | 0.0 (0.0, 9.9) |
| Missing | 5 (1.1) | 1.5 (0.6, 3.5) | 0.0 (0.0, 3.8) | 0.0 (0.0, 9.9) |
Confirmed pregnant through linkage to a birth or fetal death certificate or through active follow-up
Comparing death certificates of women who were confirmed pregnant to those confirmed not pregnant, women under the age of 40 were significantly more likely to be confirmed pregnant than confirmed not pregnant (p<.001), whereas women ages 45 and older were more likely to be confirmed not pregnant than pregnant (p<.001). (Table 2) Death certificates certified by coroners and medical examiners were more likely to be confirmed pregnant than not pregnant (p=<.001).
Table 2.
Distribution of pregnancy checkbox verification outcome within characteristics of women and certifiers, Georgia, Louisiana, Michigan, Ohio, 2016a
| Confirmed Pregnant (i.e., True Positive or False Negative) | Confirmed Not Pregnant (i.e. False Positive) | |
|---|---|---|
| n=335 | n=97 | |
| % (95% CI) | % (95% CI) | |
| Maternal Age, in years | ||
| <25 | 95.8 (89.4, 98.4) | 4.2 (1.6, 10.6) |
| 25–29 | 91.3 (84.2, 95.4) | 8.7 (4.6, 15.8) |
| 30–34 | 86.7 (77.9, 92.3) | 13.3 (7.7, 22.1) |
| 35–39 | 82.3 (70.7, 89.9) | 17.7 (10.1, 29.3) |
| 40–44b | 48.4 (31.6, 65.5) | 51.6 (34.5, 68.4) |
| 45–49 | 19.0 (7.3, 41.3) | 81.0 (58.7, 92.7) |
| ≥ 50 | 0.0 (0.0, 12.1) | 100.0 (87.9, 100.0) |
| Maternal race/ethnicity | ||
| White, non-Hispanic | 80.2 (74.8, 84.6) | 19.8 (15.4, 25.2) |
| Black, non-Hispanic | 71.1 (63.1, 78.0) | 28.9 (22.0, 36.9) |
| Hispanic | 86.4 (65.1, 95.6) | 13.6 (4.4, 34.9) |
| Otherb | 81.8 (49.1, 95.4) | 18.2 (4.6, 50.9) |
| Certifier Type | ||
| Physicianb | 53.2 (45.3, 60.8) | 46.8 (39.2, 54.7) |
| Coroner | 88.8 (83.4, 92.6) | 11.2 (7.4, 16.6) |
| Medical Examiner | 97.7 (91.2, 99.4) | 2.3 (0.6, 8.8) |
| Checkbox Value | ||
| Pregnant at time of death | 71.8 (62.9, 79.2) | 28.2 (20.8, 37.1) |
| Pregnant within 42 days of deathb | 46.7 (37.3, 56.3) | 53.3 (43.7, 62.7) |
| Pregnant 43 days-1 year of death | 90.2 (81.6, 95.1) | 9.8 (4.9, 18.4) |
| Not pregnant within past year | 100.0 (91.6, 100.0) | 0.0 (0.0, 8.4) |
| Unknown if pregnant within past year | 100.0 (95.5, 100.0) | 0.0 (0.0, 4.5) |
| Blank | 100.0 (56.6, 100.0) | 0.0 (0.0, 43.5) |
All differences significant at P<.05 via use of the Wald tests unless otherwise noted.
Row % are presented.
Not statistically significant, P>.05
Death certificates of women in all age categories greater than 29 years were more likely to be false positives, compared with women <25 years of age (Table 3). The association between age category and false positive status followed a dose response relationship (p<0.001), with increasing prevalence ratios for each increase in age category. Death certificates of non-Hispanic black women were more likely to be false positives, compared with non-Hispanic white women [prevalence ratio (PR) 1.39, 95% confidence interval (CI) 1.00, 1.94]. Certificates certified by a coroner or physician were more likely to be false positives, compared with certificates certified by medical examiners (coroner: PR 4.59, 95% CI 1.12, 18.86; physician: PR 14.47, 95% CI 3.69, 56.78). Death certificates with a checkbox indicating pregnant within 42 days of death were more likely to be false positives compared with a checkbox indicating pregnant at the time of death (PR 1.89, 95% CI 1.35, 2.66). In contrast, death certificates with a checkbox indicating pregnant 43 days to 1 year of death were less likely to be false positives compared with a checkbox indicating pregnant at the time of death (PR 0.35, 95% CI 0.17, 0.71). There were no characteristics significantly associated with being false negative as opposed to a true positive except for women of “other” race/ethnicity, who were more likely to be a false negative compared with non-Hispanic white women.
Table 3.
Characteristics associated with having an incorrect pregnancy checkbox, Georgia, Louisiana, Michigan, Ohio, 2016
| Outcome= False Positivea (n=97) | Outcome=False Negativeb (n=128) | |||
|---|---|---|---|---|
| PR | 95% CI | PR | 95% CI | |
| Age, in yearsa,c | ||||
| <25 | ref | ref | ref | ref |
| 25–29 | 2.36 | (0.77, 7.27) | 1.31 | (0.91, 1.89) |
| 30–34 | 3.31 | (1.13, 9.69) | 1.18 | (0.79, 1.75) |
| 35–39 | 4.06 | (1.38, 11.94) | 1.05 | (0.66, 1.67) |
| 40–44 | 10.00 | (3.69, 27.10) | 0.99 | (0.46, 2.14) |
| 45–49 | 13.81 | (5.25, 36.33) | 0.74 | (0.13, 4.15) |
| ≥ 50b | 16.25 | (6.29, 41.99) | ned | ned |
| Maternal race/ethnicity | ||||
| non-Hispanic white | ref | ref | ref | ref |
| non-Hispanic black | 1.39 | (1.00, 1.94) | 1.01 | (0.75, 1.36) |
| Hispanic | 0.58 | (0.20, 1.68) | 0.55 | (0.23, 1.33) |
| Other | 1.40 | (0.47, 4.19) | 1.74 | (1.06, 2.85) |
| Certifier Type | ||||
| Medical Examiner | ref | ref | ref | ref |
| Coroner | 4.59 | (1.12, 18.86) | 0.99 | (0.73, 1.36) |
| Physician | 14.47 | (3.69, 56.78) | 0.72 | (0.48, 1.09) |
| Checkbox Value | ||||
| Pregnant at time of death | ref | ref | -- | -- |
| Pregnant within 42 days of death | 1.89 | (1.35, 2.66) | -- | -- |
| Pregnant 43 days-1 year of death | 0.35 | (0.17, 0.71) | -- | -- |
False-positives defined as deaths with a pregnancy checkbox marked as pregnant at the time of death or within the preceding year, but no evidence of pregnancy was found through linkage with birth and fetal death certificates or through active follow-up.
False-negatives defined as deaths that linked to a birth or fetal death certificate within the preceding year, but had a pregnancy checkbox marked as either not pregnant within past year, unknown if pregnant within past year, or the pregnancy checkbox was blank.
Poisson regression with robust standard errors was used to estimate prevalence ratios for age. Binomial regression was used to estimate prevalence ratios for all other variables. Each outcome of interest was coded as a dichotomous variable (false positive versus true positive, false negative versus true positive); all characteristics (age, race/ethnicity, certifier type, and pregnancy checkbox category) were coded as disjoint indicator variables to compare each index category to the referent.
ne=non-estimable due to the lack of false-negative deaths among women ≥ 50 years of age. All deaths to women ≥50 years of age were false-positives.
The sensitivity of the pregnancy checkbox was 62.0% and the positive predictive value was 68.1%. When limiting the false negatives to only those confirmed pregnant with a checkbox value of not pregnant (excluding checkbox options of unknown or blank), sensitivity was 83.1%. Specificity was 99.1%; in sensitivity analyses, specificity ranged from 98.1% when assuming a 50% overestimate of the true negatives, to 99.0% when assuming a 10% overestimate of the true negatives.
Ninety-seven (31.9%) death certificates with a positive pregnancy checkbox whose pregnancy status could be confirmed (n=304), were determined to be false-positives. Among the 97 false positive certificates, 43 (44.3%) were assigned a pregnancy-related ICD-10 code which would have been included in a MMR calculation (A34, O00–95, O98–99).12 Among these deaths, 17 (39.5%) were ascribed nonspecific codes of O99.8 (other specified diseases and conditions complicating pregnancy, childbirth and the puerperium) or O26.8 (other specified pregnancy related conditions). Within the true positive certificates, 11.6% received nonspecific codes (2.9% O99.8, 8.7% O26.8.).
Comment
Principal Findings
Our findings highlight a need for more accurate reporting of pregnancy status on death certificates and we provide the first analysis of data from a multi-state process to improve data quality of the pregnancy checkbox. The probability of having been confirmed pregnant within the preceding year among death certificates with a positive pregnancy checkbox was less than 70% (positive predictive value). Similarly, the probability of having a positive pregnancy checkbox among all death certificates where pregnancy was confirmed was approximately 60% (sensitivity).
Results
Nearly one-third (31.9%) of death certificates with a positive pregnancy checkbox, and where pregnancy status could be identified, were false-positives. Our reported percentage of false-positives is lower than that reported in a similar study (50.3%).9 The previous study was from one state and reviewed medical records for 70% of the deaths included in the analysis, which may explain the higher percentage of false positives, as medical record review may be more accurate than certifier confirmation, due to the potential for recall bias. We found a lower sensitivity (62% vs 84%), potentially due to our inclusion of death certificates with an ‘unknown’ pregnancy checkbox; excluding the ‘unknown’ category resulted in a sensitivity of 83%. Researchers have found that errors are more likely when documenting pregnancy status on death certificates among older women.10,13 Our findings support this, with deaths among women ≥45 years more likely to be confirmed not pregnant than confirmed pregnant. In addition, we identified a dose-response relationship in the data between age category and false positive status.
False positives and false negatives based on the pregnancy checkbox have different implications. While false positives cause over-estimation of the MMR, false negatives could still be identified as pregnancy-related if the cause of death listed on the death certificate included pregnancy or obstetric terms (e.g., postpartum hemorrhage) and therefore assigned an O-code — thus false negatives may have less impact on MMR. State staff corrected death certificates with an erroneous pregnancy checkbox and subsequently submitted the corrected data to NCHS. However, we were unable to compare the MMR of the four states before and after the pregnancy checkbox errors were corrected due to the recoding process.
Public Health Practice Implications
It is unclear why pregnancy checkbox errors occur. Public health researchers have offered potential options for improving the quality of death certificate data. One option is to increase or improve the training death certifiers receive, but this would require a massive effort given the large number of potential certifiers (e.g., all physicians in the U.S.). A simple pop-up message could be incorporated into electronic versions of the death certificate requesting confirmation of pregnancy status when the pregnancy checkbox is marked positive. A similar intervention has been successfully tested and utilized in electronic health records to improve data quality.14 There may also be adjustments in the specifications for the death certificate. NCHS currently recommends the appropriate pregnancy checkbox be marked if the sex of the decedent is a female and between the ages of 5 to 75 years. For a female with age less than 10 or over 54 years, NCHS recommends a query message for the certifier to verify or change the response.15 Our study suggests the upper age limit of over 44 years for querying the certifier might be more beneficial, as this group represented 46% of all false positives. Another option is to edit the coding algorithm used to assign ICD-10 codes to decrease dependence on the pregnancy checkbox. Currently, any death with the pregnancy checkbox marked as “pregnant at time of death” or “within 42 days” is assigned an ICD-10 code that signifies a maternal death, with the exception of accidental or incidental causes.4 Thus, the errors in the checkbox can potentially inflate maternal mortality calculations when only using death certificate data. Based on the four state pilot QA process, states and other jurisdictions may consider a similar quality assurance process to make corrections and help ensure accurate data for use by the states and other reporting agencies.16 Removing or modifying the pregnancy checkbox from the standard death certificate could be considered. While this may reduce false positives, it may result in a return to under-identifying maternal deaths.6–8
Strengths and Limitations
The small number of pilot states and the demographic differences of their populations may limit generalizability of the findings. However, the consistency of our findings with those from another study support that pregnancy checkbox errors are responsible for misclassification of maternal deaths.9 The difference in active follow-up approach by one state may affect the comparison of state specific results. Our findings suggest that replicating the pregnancy checkbox QA process in other states may improve death certificate accuracy.
Conclusions
Until improvement in data quality can be demonstrated, MMR calculations from death certificates alone should be interpreted with caution given the potential inaccuracies in maternal death counts due to pregnancy checkbox errors. CDC’s Pregnancy Mortality Surveillance System (PMSS) reports a national pregnancy-related mortality ratio (PRMR), or deaths caused by pregnancy or its complications which occur during pregnancy or within one year of pregnancy. PMSS is based on individual clinical review of death certificates from all states, linked birth or fetal death certificates, and other information when available.17,18 The PRMR estimates may be more accurate than MMR, given the clinical review and lack of reliance on ICD-10 codes for determination of cause of death, and use of linkage to birth and fetal death certificates to further identify potential pregnancy-related deaths. However, PMSS is also limited to the vital record data available and is likely subject to over-reporting even after clinical review. The gold standard for measurement of maternal mortality would be based on findings from maternal mortality review committees (MMRCs). MMRCs are based in cities, states, or regions, and review not only death certificates but also additional data such as medical records to determine pregnancy-relatedness, cause of death, and make recommendations for prevention. MMRCs are not currently present in all areas of the U.S., but the number is rapidly increasing.19
We found evidence of both under- and over-reporting of pregnancy status on the pregnancy checkbox. States can improve the accuracy of their maternal mortality estimates, and thereby national estimates, by performing vital statistics linkages, followed by investigating pregnancy checkbox only deaths for confirmation of pregnancy status. By reducing misclassification from death certificate information and subsequent inaccurate data, the United States can provide valid measurement and understanding of maternal mortality rates.
AJOG at a Glance.
-
Why was the study conducted?
This study was conducted amid growing concerns over the validity of the pregnancy checkbox on the death certificate and its potential impact on maternal mortality rates in the United States.
-
What are the key findings?
Nearly one-third of death certificates indicating a pregnancy at time of death or in the year prior via the pregnancy checkbox, and where pregnancy status could be identified, were false-positives (no evidence of pregnancy was found). The percentage of false positives varied by age, race/ethnicity, certifier type and pregnancy checkbox category.
-
What does this study add to what is already known?
This study further supports the limited existing evidence that errors in the pregnancy checkbox are partially contributing to the rise in maternal mortality rates in the United States and offers a potential solution for states and other jurisdictions in an effort to improve the accuracy of the pregnancy checkbox data.
Footnotes
Publisher's Disclaimer: This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest
The authors report no conflict of interest.
Author Disclosure Statement
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. No competing financial interests exist.
References
- 1.Kassebaum NJ BR, Bhutta ZA, Dandona L, Gething PW, Hay SI, et al. Global, regional, and national levels of maternal mortality, 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet (London, England). 2016;388(10053):1775–1812. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.World Health Organization. Trends in maternal mortality: 1990–2015: estimates from WHO, UNICEF, UNFPA, World Bank Group and the United Nations Population Division: executive summary. 2015. [Google Scholar]
- 3.Callaghan WM. Overview of Maternal Mortality in the United States. Seminars in Perinatology. 2012;36(1):2–6. [DOI] [PubMed] [Google Scholar]
- 4.Hoyert DL. Maternal mortality and related concepts. Vital & health statistics Series 3, Analytical and epidemiological studies. 2007(33):1–13. [PubMed] [Google Scholar]
- 5.St Pierre A, Zaharatos J, Goodman D, Callaghan WM. Challenges and Opportunities in Identifying, Reviewing, and Preventing Maternal Deaths. Obstetrics and gynecology. 2018;131(1):138–142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Horon IL. Underreporting of maternal deaths on death certificates and the magnitude of the problem of maternal mortality. American Journal of Public Health. 2005;95(3):478–482. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Deneux-Tharaux C, Berg C, Bouvier-Colle MH, et al. Underreporting of pregnancy-related mortality in the United States and Europe. Obstetrics and gynecology. 2005;106(4):684–692. [DOI] [PubMed] [Google Scholar]
- 8.MacKay AP, Rochat R, Smith JC, Berg CJ. The check box: determining pregnancy status to improve maternal mortality surveillance. American journal of preventive medicine. 2000;19(1):35–39. [DOI] [PubMed] [Google Scholar]
- 9.Baeva S, Saxton DL, Ruggiero K, et al. Identifying Maternal Deaths in Texas Using an Enhanced Method, 2012. Obstetrics and gynecology. 2018;131(5):762–769. [DOI] [PubMed] [Google Scholar]
- 10.Davis NL, Hoyert DL, Goodman DA, Hirai AH, Callaghan WM. Contribution of maternal age and pregnancy checkbox on maternal mortality ratios in the United States, 1978–2012. American journal of obstetrics and gynecology. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Building U.S. Capacity to Review and Prevent Maternal Deaths. Report from Maternal Mortality Review Committees: A View into Their Critical Role. http://reviewtoaction.org/content/report-mmrcs-view-their-critical-role2017.
- 12.World Health Organization. The WHO Application of ICD-10 to deaths during pregnancy, childbirth and the puerperium: ICD-MM. 2012.
- 13.MacDorman MF, Declercq E, Thoma ME. Trends in Maternal Mortality by Sociodemographic Characteristics and Cause of Death in 27 States and the District of Columbia. Obstetrics & Gynecology. 2017;129(5):811–818. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Maraganore DM, Frigerio R, Kazmi N, et al. Quality improvement and practice-based research in neurology using the electronic medical record. Neurology Clinical practice. 2015;5(5):419–429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.National Center for Health Statistics. Death Edit Specifications for the 2003 Revision of the U.S. Standard Certificate of Death 2005.
- 16.Daymude AEC, Catalano A, Goodman D. Checking the pregnancy checkbox: Evaluation of a four-state quality assurance pilot. Birth. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Creanga AA, Syverson C, Seed K, Callaghan WM. Pregnancy-Related Mortality in the United States, 2011–2013. Obstetrics and gynecology. 2017;130(2):366–373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Centers for Disease Control and Prevention Division of Reproductive Health. Pregnancy Mortality Surveillance System. https://www.cdc.gov/reproductivehealth/maternalinfanthealth/pregnancy-mortality-surveillance-system.htm. Accessed November 30, 2018.
- 19.Zaharatos J, St Pierre A, Cornell A, Pasalic E, Goodman D. Building U.S. Capacity to Review and Prevent Maternal Deaths. Journal of women’s health (2002). 2018;27(1):1–5. [DOI] [PubMed] [Google Scholar]