Abstract
Objective:
To evaluate the performance characteristics of existing screening tools for prediction of sepsis during antepartum and postpartum readmissions.
Methods:
Case control study using electronic health record data obtained from 67 hospitals for antepartum sepsis admissions and 71 hospitals for postpartum readmissions up to 42 days from 2016–2021. Sepsis cases were matched in a 1:4 ratio to a comparison cohort without sepsis admitted antepartum or postpartum. The following screening criteria were evaluated: California Maternal Quality Care Collaborative (CMQCC) initial sepsis screen, non-pregnancy adjusted systemic inflammatory response syndrome (SIRS), Maternal Early Warning Criteria (MEWC), United Kingdom Obstetric Surveillance System (UKOSS) obstetric SIRS, and the Maternal Early Warning Trigger Tool (MEWT). Time periods were divided into early pregnancy (<20 weeks’ gestation), > 20 weeks’ gestation, early postpartum (<3 days postpartum) and late postpartum through 42 days. False positive screening rates, C-statistics, sensitivity, and specificity were reported for each overall screening tool and each individual criterion.
Results:
We identified 525 patients with sepsis during an antepartum hospitalization and 728 patients during a postpartum hospitalization. For early pregnancy and > 3 days postpartum, non-pregnancy adjusted SIRS had the highest C-statistics, 0.78 and 0.83, respectively. For > 20 weeks’ gestation and < 3 days postpartum, pregnancy-adjusted sepsis screening tools (CMQCC and UKOSS) had the highest C-statistics (0.87 – 0.94). MEWC maintained the highest sensitivity rates during all time periods (81.9%−94.4%) but also had the highest false positive rates (30.4%−63.9%). Pregnancy-adjusted sepsis screening tools (CMQCC, UKOSS) had the lowest false positive rates in all time periods (3.9%−10.1%). All tools had the lowest C-statistics in the periods of < 20 weeks’ gestation and > 3 days postpartum.
Conclusion:
For admissions early in pregnancy and > 3 days postpartum, non-pregnancy-adjusted sepsis screening tools performed better than pregnancy adjusted tools. From 20 weeks’ gestation through up to 3 days postpartum, using a pregnancy-adjusted sepsis screening tool increased sensitivity and minimized false positive rates. The overall false positive rate remained high.
Precis:
Pregnancy-adjusted sepsis screening criteria were most accurate in identifying sepsis for non-delivery admissions after 20 weeks of gestation and before 3 days postpartum.
Introduction
Maternal sepsis remains one of the leading preventable causes for maternal mortality in the United States. Early treatment has been shown to improve outcomes; however, recognition is challenging due to similarities between physiological changes of pregnancy and developing sepsis.1,2 Until recently, little attention has focused on sepsis during antepartum and postpartum hospitalizations; however, a US population-based study found just over half of cases occurred during the postpartum hospitalization compared with the delivery hospitalization.3 Similarly, in a longitudinal study of California pregnancies, sepsis was the leading cause of severe maternal morbidity in the antepartum and postpartum periods and together accounted for more cases than delivery admissions.4
Little is known about performance of screening tools to predict sepsis during antepartum and postpartum readmission hospitalizations. Most studies reporting vital sign screening focus on the delivery hospitalization or are a combined analysis of antepartum, delivery, and postpartum readmission hospitalizations.5–7 Prior work during the delivery hospitalization has shown non-pregnancy adjusted tools to have low sensitivity or specificity leading to decreased utility.8 Modified sepsis screening tools for pregnancy have been developed that include increased thresholds for heart rate, respiratory rate, and white blood cell counts. It is unknown which screening tools perform best to identify patients at risk for sepsis during antepartum and postpartum hospitalizations.
The objective of this study was to evaluate the performance characteristics of screening tools to identify patients at risk for sepsis and report false positive screen rates in a matched cohort of patients during non-delivery antepartum and postpartum readmission hospitalizations. The secondary objective of this study is to report types of end organ injury and etiology of infection among antepartum and postpartum patients with maternal sepsis.
Methods
This was a case control study of patients receiving care in hospitals utilizing Epic Clarity (Verona, Wisconsin) for EHR data storage. Hospitals were selected as a convenience sample representing diversity in patient demographics, geographic location, types of hospitals (community, urban, rural, university hospital, teaching status), and delivery volumes. EHR data from January 1, 2016, to December 31, 2021, were obtained from all hospitals who reported patients with antepartum and postpartum sepsis hospitalizations (67 hospitals for antepartum hospitalizations and 71 hospitals for postpartum readmissions). Data including demographic data, vital signs, laboratory data, and ICD-10 (International Classification of Diseases 10th Revision) coding for admission diagnoses, procedures, and diagnoses during the encounter were obtained for the antepartum and postpartum hospitalizations. Race and Ethnicity were reported in this study and to the funder, National Institutes of Health (NIH), consistent with the Inclusion of Women, Minorities, and Children policy. The race and ethnicity were extracted from the demographic fields in Epic and were presented to describe the study population. We did not analyze the data in regard to race or ethnicity. Please see Main et al. in this issue for details on how data were extracted and eligibility of sepsis cases and controls.
Delivery dates and gestational age were identified through appropriate Epic fields. assigned using standard criteria. Patients with one of the concurrent sepsis ICD-10 codes as defined by the updated Centers for Disease Control and Prevention and Health Resources and Services Administration code set (frequently used by states to measure severe maternal morbidity) in any hospitalization during pregnancy and through 42 days postpartum were considered for designation as sepsis cases.9 Antepartum sepsis hospitalizations were defined as having sepsis while pregnant and subsequent discharge without delivery. Postpartum sepsis readmissions were defined as readmissions (after the delivery hospitalization discharge) for sepsis. Delivery hospitalizations were excluded from this study. ICD-10 codes used for this study are presented in Appendix 1, available online at http://links.lww.com/xxx.
Sepsis cases were randomly matched in a 1:4 ratio to controls who were admitted during a non-delivery antepartum or postpartum readmission hospitalization. Antepartum cases were matched on the basis of gestational age at the time of antepartum admission (<20 weeks, 20–28 weeks, >28 to 36 weeks and >36 weeks). Sepsis with end organ injury was determined using the end organ injury codes defined by both Angus and Dombrovskiy.10,11 ICD-10 codes were used to identify types of end organ injury, comorbidities, and types of infection. Although by definition, sepsis requires end organ injury based on Sequential Organ Failure Assessment (SOFA) scoring, the Centers for Medicaid and Medicare Services (CMS) Core Measure SEP-1: “Early Management Bundle for Severe Sepsis/Septic Shock” still uses the prior definition of sepsis (infection with 2 or more SIRS criteria but without needing end organ injury). We collected both ICD-10 codes indicating SMM from sepsis and sepsis with end organ injury.
Screening criteria evaluated were selected for ease of implementation in that they are threshold-based and would not require specialized integrated alert systems. The following screening criteria were evaluated: [pregnancy-adjusted sepsis screening tools: California Maternal Quality Care Collaborative (CMQCC) initial sepsis screen, United Kingdom Obstetric Surveillance System (UKOSS)], [non-pregnancy adjusted screening tool: systemic inflammatory response syndrome (SIRS)], [pregnancy-adjusted severe morbidity screening tools: Maternal Early Warning Criteria (MEWC), and the Maternal Early Warning Trigger Tool (MEWT)].12–16False positive screening rates (defined as meeting screening criteria but without sepsis diagnosis codes), C-statistics, sensitivity, and specificity were reported for each overall screening tool and each individual criterion. Details of the screening tools, handling of missing vital signs and methodology for delineation of abnormal vital sign values can be found in Main et al in this issue.
Descriptive statistics were calculated and are reported as number and percentage for categorical variables or median and interquartile range for continuous variables. Statistics are reported separately for sepsis patients, gestational age matched controls, and for sepsis patients with and without end organ injury stratified by gestational age for the following time periods: <20 weeks, 20–27 6/7 weeks, 28–36/6 weeks, ≥ 37 weeks, and combined >20 weeks of gestation. Postpartum readmission data were stratified by <72 hours after delivery and >72 hours after delivery in alignment with studies showing reduction in maternal heart rate by 48–72 hours after delivery.2,17 Characteristics of contributing hospitals are also summarized and reported as a percentage of contributing hospitals.
Sensitivity, specificity and false positive rates were calculated Negative and positive predictive values were not calculated because the overall prevalence does not reflect the national prevalence due to the case-control design. For each overall score, an exact 95% confidence interval was constructed for sensitivity and specificity using the Clopper-Pearson method. Sensitivity analyses were performed excluding sepsis cases containing ICD-10 codes for hemorrhage and chorioamnionitis/endometritis due to known overlap with sepsis physiology.1,2
Institutional Review Board (IRB) approval was obtained through the single IRB system through Duke University. This study adheres to the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE).18 All statistical analyses were performed using R, version 4.1.2..
Results
Overall 525 patients with sepsis during the antepartum hospitalization were included and matched by gestational age to 2100 controls also admitted for an antepartum hospitalization without a sepsis diagnosis. In addition, 728 patients with a postpartum sepsis readmission were included and matched to 2,912 comparison controls also readmitted postpartum without a sepsis diagnosis (See Figure 1). Characteristics of reporting hospitals are listed in Table 1. Of note, 12 (17.4%) hospitals reporting antepartum sepsis cases and 14 hospitals (19.7%) reporting postpartum readmission sepsis cases were hospitals without a labor and delivery ward.
Figure 1.
Study flowchart for antepartum and postpartum admissions. ICD-10, International Classification of Diseases 10th Revision.
Table 1.
Study Hospital Characteristics for Antepartum and Postpartum Cohorts
| Hospital Characteristics | Contributed Antepartum Patients (n=67) | Contributed Postpartum Patients (n=71) |
|---|---|---|
| Region | ||
| Northeast: Massachusetts | 5 (7.5%) | 6 (8.5%) |
| South: Arkansas, North Carolina, Texas | 8 (11.9%) | 9 (12.7%) |
| Midwest: Illinois, Michigan | 13 (19.4%) | 14 (19.7%) |
| Prairie: Iowa, Nebraska, North Dakota | 7 (10.4%) | 9 (12.7%) |
| West: California, Washington | 34 (50.7%) | 33 (46.5%) |
| Yearly Delivery Volume | ||
| < 500 | 8 (11.6%) | 9 (12.7%) |
| 500–999 | 13 (18.8%) | 13 (18.3%) |
| 1000–1999 | 13 (18.8%) | 14 (19.7%) |
| 2000–3999 | 12 (17.4%) | 12 (16.9%) |
| > 4000 | 9 (13.0%) | 9 (12.7%) |
| Hospitals without a labor and delivery ward | 12 (17.4%) | 14 (19.7%) |
| Teaching Status | ||
| Major Teaching Hospital (OB-GYN residency core site) | 12 (17.4%) | 12 (16.9%) |
| Non-teaching hospital | 50 (72.5%) | 53 (74.6%) |
| Unknown | 5 (7.2%) | 6 (8.5%) |
| City population where hospital is located | ||
| <50,000 | 18 (26.1%) | 22 (31.0%) |
| 50,000–99,999 | 16 (23.2%) | 14 (19.7%) |
| 100,000–249,999 | 11 (15.9%) | 11 (15.5%) |
| 250,000–499,999 | 7 (10.1%) | 9 (12.7%) |
| 500,000–999,999 | 11 (15.9%) | 11 (15.5%) |
| >1,000,000 | 4 (5.8%) | 4 (5.6%) |
Patient demographic information and comorbidity data are presented in Table 2. The patients with ICD-10 codes for sepsis and particularly those with end organ injury codes had longer lengths of stay and in general had higher percentages of comorbidities, with the exceptions of diabetes and hypertension, than the comparison group of antepartum admissions.
Table 2.
Patient Characteristics for Antepartum and Postpartum Cohorts
| Cohort 1: Antepartum Cases | Cohort 2: Postpartum Comparison Cohort (up to 42 days) | |||||
|---|---|---|---|---|---|---|
| Characteristics | Sepsis cases (n=525) | Sepsis cases with End organ injury (n=194) | Comparison Cohort (n=2100) (Matched for gestational age) | Sepsis cases (n=728) | Sepsis cases with End organ injury (n=203) | Comparison Cohort (n=2912) |
| Age mean [IQR] | 28 [24, 33] | 29 [25, 34] | 30 [25, 34] | 30 [26, 34] | 30 [25, 34] | 32 [27, 36] |
| Race and Ethnicity N (%) * | ||||||
| Asian/Pacific Islander | 54 (10.3%) | 21 (9.2%) | 176 (8.4%) | 93 (12.8%) | 30 (14.8%) | 217 (7.4%) |
| Non-hispanic Black | 94 (17.9%) | 27 (13.9%) | 395 (18.8%) | 130 (17.9%) | 39 (19.2%) | 638 (21.9%) |
| Non-hispanic White | 175 (33.3%) | 70 (36.1%) | 948 (45.1%) | 287 (39.4%) | 67 (33.0%) | 1367 (46.9%) |
| Hispanic | 181 (34.5%) | 66 (34.0%) | 476 (22.7%) | 184 (25.3%) | 56 (27.6%) | 541 (18.6%) |
| Year of infection N (%) | ||||||
| 2016 | 60 (11.4%) | 24 (12.4%) | 432 (20.6%) | 101 (13.9%) | 28 (13.8%) | 650 (22.3%) |
| 2017 | 72 (13.7%) | 29 (14.9%) | 385 (18.3%) | 114 (15.7%) | 34 (16.7%) | 801 (27.5%) |
| 2018 | 99 (18.9%) | 39 (20.1%) | 349 (16.6%) | 150 (20.6%) | 32 (15.8%) | 752 (25.8%) |
| 2019 | 86 (16.4%) | 31 (16.0%) | 333 (15.9%) | 126 (17.3%) | 41 (20.2%) | 534 (18.3%) |
| 2020 | 110 (21.0%) | 34 (17.5%) | 309 (14.7%) | 114 (15.7%) | 32 (15.8%) | 116 (4.0%) |
| 2021 | 98 (18.7%) | 37 (19.1%) | 292 (13.9%) | 123 (16.9%) | 36 (17.7%) | 59 (2.0%) |
| Type of insurance N (%) * | ||||||
| Commercial | 224 (43.9%) | 79 (41.8%) | 1093 (53.7%) | 394 (54.1%) | 97 (47.8%) | 1826 (62.7%) |
| Medicaid | 260 (51.0%) | 100 (52.9%) | 870 (42.8%) | 290 (39.8%) | 90 (44.3%) | 880 (30.2%) |
| Gestational age at sepsis diagnosis n (%) | ||||||
| < 20 weeks | 166 (31.6%) | 53 (27.3%) | 664 (31.6%) | --- | --- | --- |
| 20 weeks to 27/6 days | 177 (33.7%) | 68 (35.1%) | 708 (33.7%) | --- | --- | --- |
| 28 weeks to 36/6 days | 169 (32.2%) | 70 (36.1%) | 676 (32.2%) | --- | --- | --- |
| ≥ 37 weeks | 13 (2.5%) | 3 (1.5%) | 52 (2.5%) | --- | --- | --- |
| Preexisting Comorbidities N (%) | ||||||
| Diabetes mellitus | 15 (2.9%) | 8 (4.1%) | 132 (6.3%) | 31 (4.3%) | 13 (6.4%) | 80 (2.7%) |
| Systemic Lupus Erythematous | 6 (1.1%) | 3 (1.5%) | 12 (0.6%) | 12 (1.6%) | 7 (3.4%) | 17 (0.6%) |
| Obesity | 64 (12.2%) | 26 (13.4%) | 306 (14.6%) | 164 (22.5%) | 57 (28.1%) | 467 (16.0%) |
| Sickle cell disease | 10 (1.9%) | 4 (2.1%) | 9 (0.4%) | 2 (0.3%) | 1 (0.5%) | 4 (0.1%) |
| Chronic renal disease | 6 (1.1%) | 5 (2.6%) | 15 (0.7%) | 13 (1.8%) | 10 (4.9%) | 18 (0.6%) |
| Chronic heart disease | 52 (9.9%) | 37 (19.1%) | 90 (4.3%) | 64 (8.8%) | 38 (18.7%) | 197 (6.8%) |
| Hypertension | 22 (4.2%) | 11 (5.7%) | 191 (9.1%) | 66 (9.1%) | 35 (17.2%) | 648 (22.3%) |
| Asthma | 207 (39.4%) | 91 (46.9%) | 263 (12.5%) | 137 (18.8%) | 58 (28.6%) | 228 (7.8%) |
| Obstetric Comorbidities N (%) | ||||||
| Gestational Diabetes Mellitus | 33 (6.3%) | 16 (8.2%) | 141 (6.7%) | 16 (2.2%) | 7 (3.4%) | 83 (2.9%) |
| Gestational Hypertension | 15 (2.9%) | 5 (2.6%) | 161 (7.7%) | 38 (5.2%) | 13 (6.4%) | 447 (15.4%) |
| Preeclampsia | 5 (1.0%) | 2 (1.0%) | 82 (3.9%) | 62 (8.5%) | 20 (9.9%) | 1242 (42.7%) |
| Multiple gestation | 15 (2.9%) | 4 (2.1%) | 134 (6.4%) | 3 (0.4%) | 0 (0.0%) | 5 (0.2%) |
| PPROM | 1 (0.2%) | 0 (0.0%) | 20 (1.0%) | 3 (0.4%) | 0 (0.0%) | 3 (0.1%) |
| Stillbirth | 1 (0.2%) | 0 (0.0%) | 3 (0.1%) | 1 (0.1%) | 0 (0.0%) | 0 (0.0%) |
| Readmission post-delivery N (%) | ||||||
| < 7 days | --- | --- | --- | 185 (25.4%) | 46 (22.7%) | 1139 (39.1%) |
| 7–14 days | --- | --- | --- | 275 (37.8%) | 53 (26.1%) | 905 (31.1%) |
| 15–21 days | --- | --- | --- | 95 (13.0%) | 33 (16.3%) | 295 (10.1%) |
| 22–28 days | --- | --- | --- | 77 (10.6%) | 29 (14.3%) | 191 (6.6%) |
| 29–35 days | --- | --- | --- | 48 (6.6%) | 19 (9.4%) | 178 (6.1%) |
| 36–42 days | --- | --- | --- | 48 (6.6%) | 23 (11.3%) | 204 (7.0%) |
Suppressed due to small numbers: Race: American Indian/Alaska Native, Unknown, Insurance status: Medicare, Other Public, Self-pay, Insurance status missing for 34 sepsis patients and 189 comparison patients due to data use agreement at one hospital prohibiting sharing insurance status. PPROM: Preterm Premature Rupture of Membranes
Full sets of vital signs per admission, including temperature, were obtained in antepartum sepsis hospitalizations in a median frequency of 30 [IQR: 20, 45] times; in the antepartum comparison group a median frequency of 9 [IQR:7, 13] times; in postpartum sepsis readmissions in a median frequency of 27 [IQR: 19, 43] times; and in the postpartum comparison group a median frequency of 11 [IQR: 8, 14] times. Performance of screening criteria in patients with ICD-10 codes for sepsis and sepsis end organ injury are presented in Table 3. All tools had the lowest C-statistics in the periods of < 20 weeks of gestation and > 3 days postpartum. With increasing gestational age, there is increasing sensitivity for pregnancy-adjusted criteria with fewer false positive screen rates in patients without sepsis diagnosis codes. Performance characteristics for each individual criterion are presented for antepartum sepsis cases in Appendixes 2 and 3, and postpartum sepsis cases in Appendix 4 and 5, all available online at http://links.lww.com/xxx.
Table 3.
Performance of Screening Tools for Antepartum and Postpartum Sepsis and Sepsis with End Organ Injury defined by ICD-10 codes
| Antepartum admissions | ||||||
|---|---|---|---|---|---|---|
| Antepartum Admissions (<20 weeks’ gestation) | ||||||
| Sepsis by Diagnosis Codes | Sepsis with End Organ Injuryby Diagnosis Codes | |||||
| Screening System | False Positive Rate (95% CI) (In patients without sepsis codes, n= 664) | Sensitivity (95% CI)(n=166 sepsis cases) | C statistic (95% CI) | False Positive Rate (In patients without sepsis codes, n=664) | Sensitivity (95% CI)(n=53 sepsis cases with end organ injury codes) | C statistic (95% CI) |
| CMQCC | 6.3% (4.6–8.5) | 57.8% (49.9–65.4) | 0.76 (0.72 – 0.80) | 6.1% (3.3–10.3) | 75.5% (61.7–86.2) | 0.85 (0.79– 0.91) |
| SIRS | 20.9% (17.9–24.2) | 77.7% (70.6–83.8) | 0.78 (0.75–0.82) | 22.6% (17.2–28.9) | 86.8% (74.7–94.5) | 0.82 (0.77– 0.87 |
| MEWC | 30.4% (26.9–34.1) | 81.9% (75.2–87.5) | 0.76 (0.72 −0.79) | 31.6% (25.4–38.3) | 90.6% (79.3–96.9) | 0.79 (0.74 −0.85) |
| UKOSS | 10.1% (7.9–12.6) | 65.1% (57.3–72.3) | 0.77 (0.74 −0.81) | 9.0% (5.5–13.6) | 79.2% (65.9–89.2) | 0.85 (0.79 −0.91) |
| MEWT(overall) | 7.7% (5.8–10.0) | 45.2% (37.5–53.1) | 0.69 (0.65– 0.73) | 9.0% (5.5–13.6) | 66.0% (51.7–78.5) | 0.79 (0.72 −0.85) |
| Antepartum Admissions (20 weeks 0 days-27 weeks 6 days gestation) | ||||||
| Sepsis by Diagnosis Codes | Sepsis with End Organ Injuryby Diagnosis | |||||
| Screening System | False Positive Rate (95% CI) (In patients without sepsis codes, n=708) | Sensitivity (95% CI)(n=177 sepsis cases) | C statistic (95% CI) | False Positive Rate (95% CI)(In patients without sepsis codes, n=708) | Sensitivity (95% CI)(n=68 sepsis cases with end organ injury codes) | C statistic (95% CI) |
| CMQCC | 8.6% (6.7–10.9) | 79.7% (73.0–85.3) | 0.86 (0.82 – 0.89) | 10.7% (7.3–15.0) | 86.8% (76.4–93.8) | 0.88 (0.84 – 0.93) |
| SIRS | 28.5% (25.2–32.0) | 92.1% (87.1–95.6) | 0.82 (0.79 – 0.84) | 29.0% (23.7–34.8) | 92.6% (83.7–97.6) | 0.82 (0.78 – 0.86) |
| MEWC | 37.6% (34.0–41.3) | 94.4% (89.9–97.3) | 0.78 (0.76 – 0.81) | 40.4% (34.6–46.5) | 100.0% (94.7–100) | 0.80 (0.77 – 0.83) |
| UKOSS | 13.0% (10.6–15.7) | 84.7% (78.6–89.7) | 0.86 (0.83 – 0.89) | 14.0% (10.1–18.7) | 89.7% (79.9–95.8) | 0.88 (0.84 – 0.92) |
| MEWT(overall) | 10.7% (8.6–13.3) | 75.1% (68.1–81.3) | 0.82 (0.79 – 0.86) | 13.2% (9.4–17.8) | 88.2% (78.1–94.8) | 0.88 (0.83 – 0.92) |
| Antepartum Admissions (28weeks 0 days-36 weeks 6 days of gestation) | ||||||
| Sepsis by Diagnosis Codes | Sepsis with End Organ Injuryby Diagnosis Codes | |||||
| Screening System | False Positive Rate (95% CI) (In patients without sepsis codes, n=676) | Sensitivity (95% CI)(n=169 sepsis cases) | C statistic (95% CI) | False Positive Rate (95% CI) (In patients without sepsis codes, n=676) | Sensitivity (95% CI)(n=70 sepsis cases with end organ injury codes) | C statistic (95% CI) |
| CMQCC | 6.7% (4.9–8.8) | 82.8% (76.3–88.2) | 0.88 (0.85 – 0.91) | 7.1% (4.4–10.8) | 90.0% (80.5–95.9) | 0.91 (0.88 – 0.95) |
| SIRS | 19.4% (16.5–22.6) | 91.7% (86.5–95.4) | 0.86 (0.84 – 0.89) | 20.0% (15.5–25.2) | 94.3% (86.0–98.4) | 0.87 (0.84 – 0.91) |
| MEWC | 35.5% (31.9–39.2) | 94.1% (89.4–97.1) | 0.79 (0.77 – 0.82) | 38.6% (32.8–44.5) | 100.0% (94.9–100) | 0.81 (0.78 – 0.84) |
| UKOSS | 9.3% (7.2–11.8) | 87.0% (81.0–91.7) | 0.89 (0.86 – 0.92) | 9.6% (6.5–13.7) | 92.9% (84.1–97.6) | 0.92 (0.88 – 0.95) |
| MEWT(overall) | 13.2% (10.7–15.9) | 75.7% (68.6–82.0) | 0.81 (0.78 – 0.85) | 14.6% (10.7–19.3) | 85.7% (75.3–92.9) | 0.86 (0.81 – 0.90) |
| Antepartum Admissions (≥37 weeks’ gestation) | ||||||
| Sepsis by Diagnosis Codes | Sepsis with End Organ Injuryby Diagnosis | |||||
| Screening System | False Positive Rate (95% CI) (In patients without sepsis codes, n=52) | Sensitivity (95% CI)(n=13 sepsis cases) | C statistic (95% CI) | False Positive Rate (95% CI) (In patients without sepsis codes, n=52) | Sensitivity (95% CI)(n=3 sepsis cases with end organ injury codes) | C statistic (95% CI) |
| CMQCC | 3.8% (0.5–13.2) | 92.3% (64.0–99.8) | 0.94 (0.86 – 1.00) | 0% (0–26.5) | 100% (29.2–100) | 1.00 (1.00 – 1.00) |
| SIRS | 7.7% (2.1–18.5) | 100% (75.3–100) | 0.96 (0.92 – 1.00) | 8.3% (0.2–38.5) | 100% (29.2–100) | 0.96 (0.88 – 1.00) |
| MEWC | 26.9% (15.6–41.0) | 100% (75.3–100) | 0.87 (0.80 – 0.93) | 25.0% (5.5–57.2) | 100% (29.2–100) | 0.88 (0.75 – 1.00) |
| UKOSS | 3.8% (0.5–13.2) | 92.3% (64.0–99.8) | 0.94 (0.86 – 1.00) | 0% (0–26.5) | 100% (29.2–100) | 1.00 (1.00 – 1.00) |
| MEWT(overall) | 7.7% (2.1–18.5) | 76.9% (46.2–95.0) | 0.85 (0.72 – 0.97) | 8.3% (0.2–38.5) | 100% (29.2–100) | 0.96 (0.88 – 1.00) |
| Antepartum Admissions (≥20 weeks of gestation-combined) | ||||||
| Matched cohort | Sepsis by Diagnosis Codes | Sepsis with End Organ Injuryby Diagnosis | ||||
| Screening System | False Positive Rate (95% CI) (In patients without sepsis codes, n=1,436) | Sensitivity (95% CI)(n=359 sepsis cases) | C statistic (95% CI) | False Positive Rate (95% CI) (In patients without sepsis codes, n=1,436) | Sensitivity (95% CI)(n=141 sepsis cases with end organ injury codes) | C statistic (95% CI) |
| CMQCC | 7.5% (6.2–9.0) | 81.6% (77.2–85.5) | 0.87 (0.85– 0.89) | 8.7% (6.5–11.3) | 88.7% (82.2–93.4) | 0.90 (0.87– 0.93) |
| SIRS | 23.5% (21.3–25.7) | 92.2% (88.9–94.8) | 0.84 (0.83– 0.86) | 24.1% (20.6–27.9) | 93.6% (88.2–97.0) | 0.85 (0.82 −0.87) |
| MEWC | 36.2% (33.7–38.8) | 94.4% (91.5–96.6) | 0.79 (0.77– 0.81) | 39.2% (35.1–43.3) | 100% (97.4–100) | 0.80 (0.78 – 0.82) |
| UKOSS | 10.9% (9.4–12.7) | 86.1% (82.1–89.5) | 0.88 (0.86– 0.90) | 11.5% (9.0–14.5) | 91.5% (85.6–95.5) | 0.90 (0.87 −0.93) |
| MEWT(overall) | 11.8% (10.1–13.5) | 75.5% (70.7–79.9) | 0.82 (0.79– 0.84) | 13.8% (11.1–17.0) | 87.2% (80.6–92.3) | 0.87 (0.84 −0.90) |
| Postpartum Readmissions | ||||||
| Postpartum Readmissions (<72h postpartum) | ||||||
| Matched cohort (4:1) N=128 | Sepsis by Diagnosis Codes N=32 | Sepsis with End Organ Injuryby Diagnosis Codes N=9 | ||||
| Screening System | False Positive Rate (95% CI) (In patients without sepsis codes, n=128) | Sensitivity (95% CI)(n=32 sepsis cases) | C statistic (95% CI) | False Positive Rate (95% CI) (In patients without sepsis codes, n=128) | Sensitivity (95% CI)(n=9 sepsis cases with end organ injury codes) | C statistic (95% CI) |
| CMQCC | 3.9% (1.3–8.9) | 87.5% (71.0–96.5) | 0.92 (0.86 – 0.98) | 5.6% (0.7–18.7) | 100% (66.4–100) | 0.97 (0.93 – 1.00) |
| SIRS | 21.9% (15.1–30.0) | 96.9% (83.8–99.9) | 0.88 (0.83 – 0.92) | 19.4% (8.2–36.0) | 100% (66.4–100) | 0.90 (0.84 – 0.97) |
| MEWC | 37.5% (29.1–46.5) | 96.9% (83.8–99.9) | 0.80 (0.74 – 0.85) | 38.9% (23.1–56.5) | 100% (66.4–100) | 0.81 (0.72 – 0.89) |
| UKOSS | 5.5% (2.2–10.9) | 93.8% (79.2–99.2) | 0.94 (0.89 – 0.99) | 5.6% (0.7–18.7) | 100% (66.4–100) | 0.97 (0.93 – 1.00) |
| MEWT(overall) | 6.3% (2.7–11.9) | 68.8% (50.0–83.9) | 0.81 (0.72 −0.89) | 8.3% (1.8–22.5) | 88.9% (51.8–99.7) | 0.90 (0.78 – 1.00) |
| Postpartum Readmissions (>72h postpartum) | ||||||
| Sepsis by Diagnosis Codes | Sepsis with End Organ Injuryby Diagnosis | |||||
| Screening System | False Positive Rate (95% CI) (In patients without sepsis codes, n=2,784) | Sensitivity (95% CI)(n=696 sepsis cases) | C statistic (95% CI) | False Positive Rate (95% CI) (In patients without sepsis codes, n=2,784) | Sensitivity (95% CI)(n=194 sepsis cases with end organ injury codes) | C statistic (95% CI) |
| CMQCC | 6.8% (5.8–7.7) | 68.8% (65.2–72.2) | 0.81 (0.79 – 0.83) | 7.1% (5.4–9.1) | 82.5% (76.4–87.5) | 0.88 (0.85 – 0.91) |
| SIRS | 21.3% (19.8–22.9) | 87.5% (84.8–89.9) | 0.83 (0.82 – 0.85) | 23.7% (20.8–26.9) | 91.8% (87.0–95.2) | 0.84 (0.82 – 0.86) |
| MEWC | 63.9% (62.0–65.7) | 90.2% (87.8–92.3) | 0.63 (0.62 – 0.65) | 65.2% (61.7–68.6) | 92.3% (87.6–95.6) | 0.64 (0.61 – 0.66) |
| UKOSS | 10.4% (9.3–11.6) | 74.4% (71.0–77.6) | 0.82 (0.80 – 0.84) | 10.8% (8.7–13.2) | 89.2% (83.9–93.2) | 0.89 (0.87 – 0.92) |
| MEWT(overall) | 30.9% (29.2–32.7) | 55.6% (51.8–59.3) | 0.60 (0.58 – 0.63) | 31.2% (27.9–34.6) | 75.3% (68.6–81.2) | 0.71 (0.67 – 0.74) |
Concurrent end organ injury codes were present in 28.8% of patients with sepsis antepartum and 22.9% postpartum. The most common types of end organ injury for antepartum patients were cardiovascular 115 (21.9%), hematologic 45 (8.6%), and pulmonary 39 (7.4%). For postpartum patients, the most common were cardiovascular 96 (13.2%), renal 85 (11.7%), and hematologic 37 (5.1%). The types of end organ injury and etiologies for infection during antepartum and postpartum are reported in Appendix 6, available online at http://links.lww.com/xxx. The most common infections in antepartum patients with sepsis were pyelonephritis [320 (61.0%]), pneumonia [103 (19.6%)], and other viral infections [72 (13.7%)] antepartum and endometritis [187 (25.7%)], post-procedural infection [181 (24.9%)], and pyelonephritis [172 (23.6%)] postpartum.
Sensitivity analyses were performed in the postpartum readmission group to assess the effect of removing sepsis patients with a concomitant ICD-10 code indicating hemorrhage or endometritis (Appendixes 7 and 8, available online at http://links.lww.com/xxx). Although not powered to detect a difference, the sensitivity was similar (within 1 percentage point) after these exclusions for all screening tools.
Discussion
The important findings of this study are: (1) for admissions early in pregnancy (< 20 weeks’ gestation) and > 3 days postpartum, a non-pregnancy adjusted sepsis screening tool has improved predictive capability over a pregnancy-adjusted tool; and (2) from 20 weeks’ gestation through up to 3 days postpartum, using a pregnancy-adjusted sepsis screening tool increases sensitivity and minimizes false positive rates. With increasing gestational age, there is increasing sensitivity for pregnancy-adjusted criteria with fewer false positive screens. Therefore, we suggest use of a non-pregnancy-adjusted screening tool prior to 20 weeks’ gestation and after 3 days postpartum (consistent with the time period obstetric patients may be treated in emergency room settings) and use of a pregnancy-adjusted screening tool starting at 20 weeks’ gestation until 72 hours postpartum.
Prior published articles combined any admission (antepartum, delivery, and postpartum readmissions) when reporting sensitivity and specificity for criteria to identify infection or critical illness.6,7 However, there are important differences in vital sign changes within antepartum and postpartum hospitalizations. Using pregnancy-adjusted criteria once pregnancy is known and through six weeks postpartum results in reduced sensitivity to identify sepsis because pregnancy physiology changes many parameters (such as heart rate) which gradually increase during pregnancy and abruptly decrease postpartum.17,19Importantly, 548 (25.3%) patients with sepsis did not meet temperature criteria during admission underscoring that sepsis often occurs in obstetric patients without fever or hypothermia. Clinicians who care for obstetric patients should be aware that this study and prior work including patients who died from maternal sepsis have identified that fever and hypothermia are often absent in sepsis.20
Our study has limitations. Although our data collection spanned 71 hospitals and 12 states, the outcomes are identified by claims data which can overestimate or underestimate the outcome of interest. However, we used ICD-10 codes to identify cases consistent with national public health surveillance for maternal sepsis used for identification for severe maternal morbidity for sepsis, and validation studies have shown positive predictive values of 100% for these sepsis codes.21,22 Additionally, these data are retrospective, and we are unable to determine whether the use of screening criteria would improve outcomes by early identification of patients at risk for sepsis. Finally, the study includes a comparison cohort of patients without significant hemorrhage or endometritis to show the performance of screening criteria in patients due to physiological changes of pregnancy rather than conditions known to have physiologic changes similar to sepsis. However, the sensitivity analyses showed less than 1 percent differences in sensitivity indicating hemorrhage and chorioamnionitis did not have a major effect on meeting criteria and positive screening was due to the infection itself. While pregnancy-adjusted tools demonstrated improved performance over traditional sepsis screening tools from 20 weeks’ gestation through 3 days postpartum, false positive rates remained high. Positive predictive value could not be calculated given the matching used to create the cohort, which does not allow for estimation of population prevalence. Assuming a sepsis prevalence of 3 per thousand (0.3%), the estimated PPV of CMQCC and UKOSS tools in antepartum patients at 20 weeks’ gestation or greater would be 3.2% and 2.3%, respectively. Assuming similar sepsis prevalence, the PPV for postpartum patients within 3 days would be 6.3% and 4.9% for CMQCC and UKOSS, respectively. Given the anticipated PPV, positive screens for pregnancy-adjusted tools are not meant to indicate likely presence of sepsis, but rather an increased risk for sepsis. We suggest a two-step process with use of a screening tool followed by a second diagnostic step to evaluate for end organ injury.
We conclude that non-pregnancy adjusted criteria should be used for sepsis screening prior to 20 weeks’ gestation and greater than 3 days postpartum (commonly representing postpartum readmission timing). Starting at 20 weeks’ gestation, the use of pregnancy-adjusted sepsis criteria may reduce false positive screening rates in patients without sepsis. Primary screening followed by a confirmatory diagnostic step may help identify those most at risk followed by a diagnostic evaluation. All services who care for pregnant patients (emergency departments, urgent care facilities, intensive care units) should be aware of these nuances and implement screening criteria with the appropriate timing in pregnancy, which are aligned with CMS requirements for severe sepsis and septic shock early management bundle (SEP-1).
Supplementary Material
Funding Source:
This work was funded by NIH/ NICHD UG3 HD 108053 (Melissa E Bauer and Elliott Main). Vesela P Kovacheva reports funding from the NIH/NHLBI grants 1K08HL161326-01A1.
Footnotes
Financial Disclosure
Melissa Bauer disclosed receiving consulting fees from the Institute for Healthcare Innovation, and travel compensation from National AIM to present the ACOG/AIM Sepsis Bundle at the ACOG Annual Conference. Vesela Kovacheva disclosed receiving consulting fees from Avania CRO unrelated to the current study. Paloma Toledo disclosed receiving speaker fees from Pacira Biosciences, Inc. Kurt Wharton disclosed receiving consulting fees from Molnlycke. The other authors did not report any potential conflicts of interest.
Each author has confirmed compliance with the journal’s requirements for authorship.
Contributor Information
Melissa E Bauer, Department of Anesthesiology, Duke University, Durham, NC
Matt Fuller, Department of Anesthesiology, Duke University, Durham, NC
Vesela Kovacheva, Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, Boston, MA..
Rania Elkhateb, Department of Anesthesiology, University of Arkansas for Medical Sciences, Little Rock, AR..
Kristin Azar, Sutter Health Institute for Advancing Health Equity, Sutter Health, Sacramento, CA..
Morgan Caldwell, Department of Anesthesiology, Duke University, Durham, NC
Vanna Chiem, Department of Systems Clinical Informatics, Common Spirit Health, San Francisco, CA.
Mindy Foster, Common Spirit Health, San Francisco, CA.
Ronald Gibbs, Department of Obstetrics and Gynecology, Stanford University School of Medicine, Palo Alto, CA
Brenna L Hughes, Department of Obstetrics and Gynecology, Duke University, Durham, NC.
Rebecca Johnson, Department of Obstetrics and Gynecology, Baylor College of Medicine, Houston, TX
Nobin Kottukapaly, Wayne State University School of Medicine, Wayne, MI.
Melissa G. Rosenstein, Department of Obstetrics, Gynecology, and Reproductive Sciences, University of California, San Francisco, CA.
Magdalena Sanz Cortes, Department of Obstetrics and Gynecology, Baylor College of Medicine, Houston, TX.
Laurence E. Shields, Common Spirit Health, San Francisco, CA.
Sylvia Sudat, Center for Health Systems Research, Sutter Health, Sacramento, CA.
Caitlin D. Sutton, Department of Anesthesiology, Baylor College of Medicine, Houston, TX
Paloma Toledo, Department of Anesthesiology, Perioperative Medicine and Pain Management, University of Miami Miller School of Medicine, Miami, FL..
Austin Traylor, Department of Anesthesiology, Duke University, Durham, NC
Kurt Wharton, Department of Obstetrics and Gynecology, Oakland University William Beaumont School of Medicine, Royal Oak, MI..
Elliott Main, Department of Obstetrics and Gynecology, Stanford University, Palo Alto, CA.
References
- 1.Lappen JR, Keene M, Lore M, Grobman WA, Gossett DR. Existing models fail to predict sepsis in an obstetric population with intrauterine infection. Am J Obstet Gynecol. 2010;203(6):573.e571–575. doi: 10.1016/j.ajog.2010.07.040 [DOI] [PubMed] [Google Scholar]
- 2.Bauer ME, Bauer ST, Rajala B, MacEachern MP, Polley LS, Childers D, et al. Maternal physiologic parameters in relationship to systemic inflammatory response syndrome criteria: a systematic review and meta-analysis. Obstet Gynecol. 2014;124(3):535–541. doi: 10.1097/AOG.0000000000000423 [DOI] [PubMed] [Google Scholar]
- 3.Hensley MK, Bauer ME, Admon LK, Prescott HC. Incidence of Maternal Sepsis and Sepsis-Related Maternal Deaths in the United States. Jama. 2019;322(9):890–892. doi: 10.1001/jama.2019.9818 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Carmichael SL, Girsen AI, Ma C, Main EK, Gibbs RS. Using Longitudinally Linked Data to Measure Severe Maternal Morbidity Beyond the Birth Hospitalization in California. Obstet Gynecol. 2022;140(3):450–452. doi: 10.1097/AOG.0000000000004902 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Albright CM, Has P, Rouse DJ, Hughes BL. Internal Validation of the Sepsis in Obstetrics Score to Identify Risk of Morbidity From Sepsis in Pregnancy. Obstet Gynecol. 2017;130(4):747–755. doi: 10.1097/AOG.0000000000002260 [DOI] [PubMed] [Google Scholar]
- 6.Arnolds DE, Carey KA, Braginsky L, Holt R, Edelson DP, Scavone BM, et al. Comparison of early warning scores for predicting clinical deterioration and infection in obstetric patients. BMC Pregnancy Childbirth. 2022;22(1):295. doi: 10.1186/s12884-022-04631-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Stephens AJ, Lee KH, Barton JR, Chauhan SP, Baker AL, Sibai BM. Development of the Sepsis-Associated Adverse Outcomes in Pregnancy Score. Am J Perinatol. 2023;40(2):128–136. doi: 10.1055/a-1884-1001 [DOI] [PubMed] [Google Scholar]
- 8.Bauer ME, Housey M, Bauer ST, et al. Risk Factors, Etiologies, and Screening Tools for Sepsis in Pregnant Women: A Multicenter Case-Control Study. Anesth Analg. 2019;129(6):1613–1620. doi: 10.1213/ANE.0000000000003709 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Maternal and Child Health Bureau. Federally Available Data (FAD) Resource Document. Rockville, MD: Health Resources and Services Administration. Updated September 2, 2022. Accessed June 6, 2023. https://mchb.tvisdata.hrsa.gov/Admin/FileUpload/DownloadContent?fileName=FadResourceDocument.pdf&isForDownload=False [Google Scholar]
- 10.Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310. doi: 10.1097/00003246-200107000-00002 [DOI] [PubMed] [Google Scholar]
- 11.Dombrovskiy VY, Martin AA, Sunderram J, Paz HL. Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003. Crit Care Med. 2007;35(5):1244–1250. doi: 10.1097/01.CCM.0000261890.41311.E9 [DOI] [PubMed] [Google Scholar]
- 12.Gibbs R, Bauer M, Olvera L, Sakowski C, Cape V, Main E. Improving Diagnosis and Treatment of Maternal Sepsis: A Quality Improvement Toolkit. 2020. Stanford, CA: California Maternal Quality Care Collaborative. Updated July 1, 2022. Accessed May 15, 2023. https://www.cmqcc.org/content/sepsis [Google Scholar]
- 13.Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–1655. doi: 10.1378/chest.101.6.1644 [DOI] [PubMed] [Google Scholar]
- 14.Mhyre JM, D’Oria R, Hameed AB, Lappen JR, Holley SL, Hunter SK, et al. The maternal early warning criteria: a proposal from the national partnership for maternal safety. Obstet Gynecol. 2014;124(4):782–786. doi: 10.1097/AOG.0000000000000480 [DOI] [PubMed] [Google Scholar]
- 15.Acosta CD, Kurinczuk JJ, Lucas DN, Tuffnell DJ, Sellers S, Knight M. Severe maternal sepsis in the UK, 2011–2012: a national case-control study. PLoS Med. 2014;11(7):e1001672. doi: 10.1371/journal.pmed.1001672 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Shields LE, Wiesner S, Klein C, Pelletreau B, Hedriana HL. Use of Maternal Early Warning Trigger tool reduces maternal morbidity. Am J Obstet Gynecol. 2016;214(4):527.e521–527.e526. doi: 10.1016/j.ajog.2016.01.154 [DOI] [PubMed] [Google Scholar]
- 17.Green LJ, Pullon R, Mackillop LH, Gerry S, Birks J, Davidson S, et al. Postpartum-Specific Vital Sign Reference Ranges. Obstet Gynecol. 2021;137(2):295–304. doi: 10.1097/AOG.0000000000004239 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–1457. doi: 10.1016/j.jclinepi.2007.11.008 [DOI] [PubMed] [Google Scholar]
- 19.Green LJ, Mackillop LH, Salvi D, Pullon R, Loerup L, Mossop LTJ, et al. Gestation-Specific Vital Sign Reference Ranges in Pregnancy. Obstet Gynecol. 2020;135(3):653–664. doi: 10.1097/AOG.0000000000003721 [DOI] [PubMed] [Google Scholar]
- 20.Bauer ME, Lorenz RP, Bauer ST, Rao K, Anderson FWJ. Maternal Deaths Due to Sepsis in the State of Michigan, 1999–2006. Obstet Gynecol. 2015;126(4):747–752. doi: 10.1097/AOG.0000000000001028 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Sigakis MJ, Leffert LR, Mirzakhani H, Sharawi N, Rajala B, Callaghan WM, et al. The Validity of Discharge Billing Codes Reflecting Severe Maternal Morbidity. Anesth Analg. 2016;123(3):731–738. doi: 10.1213/ANE.0000000000001436 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Himes KP, Bodnar LM. Validation of criteria to identify severe maternal morbidity. Paediatr Perinat Epidemiol. 2020;34(4):408–415. doi: 10.1111/ppe.12610 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.