Development of a prediction model of postpartum hospital use using an equity-focused approach

Abstract

Background:

Racial inequities in maternal morbidity and mortality persist into the postpartum period, leading to a higher rate of postpartum hospital use (PHU) among Black and Hispanic people. Delivery hospitalizations provide an opportunity to screen and identify people at high risk to prevent adverse postpartum outcomes. Current models do not adequately incorporate social and structural determinants of health (SSDOH), and some include race, which may result in biased risk stratification.

Objective:

Our objective was to develop a risk prediction model of PHU incorporating novel SSDOH using an equity approach.

Study Design:

We conducted a retrospective cohort study using 2016–2018 linked birth certificate and hospital discharge data for liveborn infants in New York City. We included deliveries in 2016–2017 in model development, randomly assigning 70%/30% of deliveries as training/test data. We used deliveries in 2018 for temporal model validation. We defined Composite PHU as at least one readmission or emergency department visit within 30 days of the delivery discharge. We categorized diagnosis at first hospital use into 14 categories based on ICD-10 diagnosis codes. We tested 72 candidate variables, including social determinants of health, demographics, comorbidities, obstetric complications, and severe maternal morbidity. Structural determinants of health were racial-economic segregation at the zip code level using the index of concentration at the extremes (ICE) and publicly available indices of the neighborhood built/natural (BNE) and social/economic (SE) environment. We used four statistical and machine learning algorithms to predict Composite PHU, and an ensemble approach to predict Cause-specific PHU. We simulated the impact of each risk stratification method paired with an effective intervention on race-ethnic equity in PHU.

Results:

The overall incidence of PHU was 5.7%; the incidence among Black, Hispanic and White people was 8.8%, 7.4%, and 3.3%, respectively. Most common diagnoses for hospital use were general perinatal reasons (17.5%), hypertension/eclampsia (12.0%), non-gynecologic infections (10.7%), and wound infections (8.4%). Logistic regression with LASSO selection retained 22 predictor variables and achieved an auROC of 0.69 in the training, 0.69 in test, and 0.69 in validation data. Other machine learning algorithms performed similarly. Selected SSDOH features included ICE, insurance payor, depressive symptoms, and trimester entering prenatal care. The Cause-specific PHU model selected six of the 14 outcome diagnoses: acute cardiovascular disease, gastrointestinal disease, hypertension/eclampsia, psychiatric disease, sepsis, and wound infection, achieving an auROC of 0.75 in training, 0.77 in the test data, and 0.75 in the validation data using cross-validation approach. Models had slightly lower performance in Black and Hispanic subgroups. When simulating use of the risk stratification models with a postpartum intervention, identifying high-risk individuals using the Composite PHU model resulted in the greatest reduction in racial-ethnic disparities in PHU, compared to the Cause-specific PHU model or a standard approach to identifying high-risk individuals with common pregnancy complications.

Conclusion:

A Composite PHU prediction model incorporating SSDOH can be used at delivery discharge to identify persons at risk for PHU..

Introduction

Profound racial inequities in maternal mortality persist in the US. Black people are three to four times more likely to die a pregnancy-related death than White people,¹ and in some regions, including New York City (NYC),² Hispanic people are also at increased risk. Growing attention has focused on the postpartum period as important in the prevention of maternal mortality, as more than half of pregnancy-related deaths occur within one day to one year after delivery.³ Racial inequities in maternal morbidity and mortality persist into the postpartum period, leading to a higher rate of postpartum hospital use (PHU), including readmissions and emergency department (ED) visits, among Black and Hispanic birthing people.⁴ Delivery hospitalizations provide an opportunity to screen and identify people at high risk to prevent adverse postpartum outcomes. Yet little is understood how to best identify people at high risk of PHU in order to effectively target interventions and reduce inequities.

A standard approach to risk stratification is to identify birthing people as high-risk for PHU due to morbidities such as hypertensive disorders, diabetes, and mental health disorders. Recent studies have sought to improve on this approach by developing prediction models focusing on postpartum readmissions due to specific causes, including psychiatric disorders,⁵ hypertensive disorders^6,7, postpartum hemorrhage, sepsis, and wound infection.⁸ However, these algorithms have several shortcomings. Focusing on one delivery diagnosis may assist in developing interventions targeting one outcome, such as remote blood pressure monitoring, but these models are less useful for identifying an at-risk population that might benefit from more holistic supportive interventions such as patient navigators⁹or home visits.^{10 9}Moreover, these models have not used an equity approach – for example, they often inappropriately include race as a predictor variable and rarely include SSDOH. Finally, many have not tested the potential impact of these models on reducing maternal disparities in PHU.

We conducted a retrospective cohort study using 2016–2018 linked NYC birth and hospital records. Our objective was to develop a risk prediction model to identify birthing persons at high risk of PHU. Our equity-focused modeling approach included: 1) Defining race as a social construct and testing area-based structural determinants of health,¹¹ 2) Centering the modeling approach in Black and Hispanic birthing people by training and validating the model in these subgroups, and 3) Evaluating the prediction model for its potential impact on postpartum health equity compared to a standard approach to identifying high-risk individuals with common pregnancy complications.

Materials and Methods

Data Sources

We conducted a retrospective cohort study using linked NYC Statewide Planning and Research Cooperative System (SPARCS) hospital discharge and birth data for 2016–2018. We identified hospitalizations for delivery in SPARCS (n=330,945).¹² We linked SPARCS and birth data using an encrypted person ID, excluding deliveries with missing ID, resulting in n=326,632. We used delivery hospitalizations with discharge dates between January 1, 2016-November 31, 2017 for model development, and followed patients until December 31, 2017. We used discharges December 1, 2017–November 31, 2018, followed until December 31, 2018, for temporal validation. The study was approved by the Icahn School of Medicine Program for the Protection of Human Subjects and the University of Pennsylvania Human Research Protection Program.

Study Population

Eligible persons gave birth in a New York City hospital and had a zip code indicating they were a resident of the NYC Metropolitan Area (n=323,981). We randomly divided births from January 2016-November 2017 into 70% training (n=145,464) and 30% test (n=62,341) datasets. Year 2018 data was preserved as a temporal validation dataset (n= 107,999). We were not able to identify people who gave birth twice in our study sample.

Outcome

We ascertained 30-day PHU by linking the delivery hospitalization prospectively to all inpatient and emergency department hospital records in New York state. We categorized a PHU event (PHU=1) if at least one inpatient or ED event occurred within 30 days of the delivery discharge, vs. no such events (PHU=0) henceforth “Composite PHU”. We also created 14 indicator variables for the reason (i.e. “cause”) of the first occurrence of PHU based on ICD-10 diagnosis and procedure codes, henceforth “Cause-specific PHU” (Supplemental Table 1). Patients were allowed to have multiple reasons for a PHU event, such that ≥ 1 of the indicator variables could be = 1. We defined these cause indicator variables based on previous literature on reasons for postpartum readmission¹³ and ED use as well as clinical judgment from the study team.

Candidate variables

We assessed a total of 72 individual-level candidate variables from the delivery hospitalization. We included parity, multiple gestation, comorbidities, obstetric complications (including severe maternal morbidity (SMM)), length of stay, and prenatal admission (Supplemental Table 2). We created variables using the ICD-10-CM International Classification of Diseases, the birth certificate, or both. ICD-10-CM classifications of comorbidities were based on Leonard et al (Supplemental Table 2).¹⁴ We defined SMM using the Centers for Disease Control algorithm, andobstetric complications using the Joint Commission specifications for perinatal quality measures.

Social and structural determinants of health

We ascertained individual-level SSDOH from the discharge record using ICD-10-CM Z-codes. Individual-level determinants ascertained from the birth certificate included self-reported depressive symptoms, education, WIC participation, and nativity. Area-level determinants were racial-economic segregation at the zip code level using the index of concentration at the extremes (ICE)¹⁵ and the neighborhood built/natural (BNE) and social/economic (SE) environment indices of the Child Opportunity Index,¹⁶ all publicly available measures. The ICE is a proxy of structural racism and is associated with adverse maternal outcomes.^17,18 The ICE uses data from the American Community Survey and is the proportion of wealthy White households relative to poor Black households in a given zip code and ranges from −1 to 1, with −1 indicating a zip code composed completely of poor, Black households and 1 indicating a zip code composed completely of wealthy, White households. The BNE score includes: access to healthy food, green space, walkability, housing vacancy, hazardous waste dump sites, industrial pollutants in air, water, or soil, airborne microparticles, ozone concentration, extreme heat exposure, and percent of persons age 0–64 with health insurance coverage. The SE score is composed of: unemployment rate, commute duration, poverty rate, public assistance rate, homeownership rate, high-skill employment, median household income, and single-headed households. Both individual-level and area-level SSDOH were included as candidate variables simultaneously in model development described below.

Statistical Analysis

Overall missingness of covariates ranged from 0.0% - 2.9%. We excluded missing observations except for those with greater missingness (prenatal care 1.8%, depressive symptoms 2.9%) for which we included a category of missing because we viewed these categories as informative.

Composite PHU model

We developed models that predict the probability of PHU using 2016–2017 data (training data, n=145,464). “Composite PHU” was indicated by any postpartum readmission or ED visit within 30 days of discharge. We trained models to predict PHU using statistical and machine learning approaches: LASSO logistic regression¹⁹, elastic-net regression²⁰, gradient boosting machines²¹, and random forests.²² We selected these approaches after exploring many machine learning models including linear methods, tree-based, support vector machines, and neural networks, because they had the best performance in 10-cross validation while representing different types. The predictive ability of each model was assessed using standard metrics including area under the receiver operating curve (auROC) and area under the precision-recall curve (auPRC). Statistical analyses were conducted in SAS v. 9.4 and R.

Cause-specific PHU model

We developed machine learning models to predict Cause-specific PHU, using the same training data. We developed separate machine-learning models for each readmission category and measured their performance in 10-cross validation (CV). We then examined whether combining the predictions of some of these models would improve predictive performance beyond the Composite PHU model. To identify the optimal combination of models that would achieve the best performance, using the most PHU categories we performed a permutation analysis that averaged the predictions for all possible combinations of the 14 models and identified the combination that achieved the highest auROC using the most models. This ensemble model was considered our final Cause-specific model.

Validation

We performed validation on the best performing Composite PHU and Cause-specific models, by measuring predictive performance in the test data set and in the 2018 temporal validation data. We measured performance using metrics that depend on a probability decision threshold, selecting the threshold that maximizes sensitivity and specificity.²³

Equity analyses

We employed multiple analytic approaches to address the goal of our prediction model to reduce postpartum morbidity and improve maternal health equity. First, we validated our models in Black and Hispanic subgroups. Next, we centered the model selection process in Black and Hispanic birthing people by repeating model selection in each group. Finally, we used a simulation to test the potential impact on racial-ethnic disparities in PHU if three different risk stratification approaches ─ Composite PHU, Cause-specific PHU, and a standard approach to identifying high risk people ─ were paired with a hypothetical intervention of 40% efficacy (See Supplementary Methods for details of calculation). We chose 40% based on an example of a successful postpartum intervention.²⁴ Additionally, we performed sensitivity analyses. We wondered if using ICE was too close of a proxy for individual race-ethnicity and could also be considered biased, and conducted a post-hoc analysis in which we trained the LASSO logistic regression model with the SE and BNE indices only. Also, we trained the LASSO logistic regression model including ICE, SE, BNE, and individual-level race-ethnicity.

Results

Among n=145,464 births in the training data, the overall incidence of 30-day ED visits was 5.0%, 30-day readmission was 1.4%, and PHU (combined ED or readmission outcome) was 5.7%. 33.4% of birthing individuals in the training dataset were White, 28.7% were Hispanic, 19.0% were Black, 18.5% were Asian, 0.4% were of other race-ethnicities (Table 1). 60.2% had a greater than high school education, whereas 22.7% had a high school education and 17.1% had less. 59.7% of individuals had Medicaid insurance, 38.5% were privately insured, 0.9% and 1.0% were uninsured or had other insurance, respectively. Test and temporal validation datasets were similar.

Table 1.

Sample characteristics of New York City births by analytic dataset, 2016–2018.

	Training a n=145,464 2016–2017 n(%)	Testing b n=62,341 2016–2017 n(%)	Validation b n=107,999 2018 n(%)
Demographics
Maternal age in years
< 20 years	4,178 (2.9)	1,753 (2.8)	2,770 (2.6)
20–34 years	104,494(71.8)	44,745 (71.8)	75,640 (70.0)
≥ 35	36,785 (25.3)	15,843 (25.4)	29,589 (27.4)
Race/Ethnicity
Black, non-Hispanic	27,680 (19.0)	12,017 (19.3)	19,655 (18.2)
Hispanic	41,764 (28.7)	18,027 (28.9)	30,199 (28.0)
White, non-Hispanic	48,503 (33.4)	20,755 (33.3)	36,802 (34.1)
Asian, non-Hispanic	26,902 (18.5)	11,272 (18.1)	18,863 (17.5)
Other, non-Hispanic	615 (0.4)	270 (0.4)	2,480 (2.3)
Education
Less than high school	24,800 (17.1)	10,746 (17.3)	16,953 (15.8)
High school	32,966 (22.7)	14,161 (22.8)	23,988 (22.3)
More than high school	87,349 (60.2)	37,273 (59.9)	66,687 (62.0)
Insurance
Medicaid	86,827 (59.7)	37,058 (59.4)	62,099 (57.5)
Private Insurance	55,958 (38.5)	24,131 (38.7)	44,194 (40.9)
Uninsured	1,287 (0.9)	565 (0.9)	596 (0.6)
Other	1,392 (1.0)	587 (0.9)	1,110 (1.0)
Birth Country
US Born	75,632 (52.0)	32,276 (51.8)	58,360 (54.0)
Foreign Born	69,832 (48.0)	30,065 (48.2)	49,639 (46.0)
Parity
Nulliparous	63,158 (43.4)	26,923 (43.2)	46,803 (43.3)
Multiparous	82,242 (56.6)	35,400 (56.8)	61,183 (56.7)
Delivery Type
Vaginal	97,726 (67.2)	41,926 (67.3)	73,038 (67.6)
Cesarean	47,737 (32.8)	20,415 (32.8)	34,958 (32.4)
Comorbidities
Chronic Hypertension	3,995 (2.8)	1,650 (2.7)	2,985 (2.8)
Pregnancy Hypertension	10,780 (7.4)	4,649 (7.5)	8,547 (7.9)
Preeclampsia	3,483 (2.4)	1,519 (2.4)	3,354 (3.1)
Diabetes Mellitus	1,930 (1.3)	843 (1.4)	1,597 (1.5)
Gestational Diabetes	14,271 (9.8)	6,007 (9.6)	11,128 (10.3)

^aRandom 70% sample from dataset that includes deliveries from January 1, 2016-November 30, 2017.

^bRandom 30% sample of deliveries from dataset that includes deliveries from January 1, 2016-November 30, 2017.

^cAll deliveries from December 1, 2017-November 30, 2018

Composite PHU Model

The overall incidence of PHU in the training data was 5.7%; the incidence among Asian, Black, Hispanic and White birthing people was 4.2%, 8.8%, 7.4%, and 3.3%, respectively. Each of the statistical and machine learning approaches to predict Composite PHU had similar performance (Table 2). We therefore used the LASSO logistic regression model for validation analyses, due to its accessibility as a common statistical and machine learning approach. Logistic regression selection retained 22 predictor variables and achieved an AUC of 0.689 in the training data (Table 2). Unadjusted and adjusted odds ratios (aOR) for the selected variables in unadjusted analysis of the training data are displayed in Table 3. Individual-level SSDOH measures retained included Medicaid-insured (aOR=1.57, 95% Confidence Interval (CI)= (1.48, 1.67), late prenatal care (aOR=1.07, 95%CI=1.02, 1.13), and depressive symptoms during pregnancy (aOR=1.10, 95%CI=1.03, 1.18). ICE was the only selected area SSDOH measure, with an OR of 0.65 (95%CI=0.58, 0.74), representing a 0.65 unit increase in log odds of PHU for every unit increase in racial-economic segregation.

Table 2.

Model performance for composite postpartum hospital use models by statistical and machine learning approaches, training data, New York City, 2016–2017

	auROCa	auPRCb	Sensitivityc	Specificityc	Precisionc
Lasso Logistic Regression	69%	12%	66%	62%	9%
Elastic-Net Logistic Regression	69%	12%	88%	33%	7%
Gradient Boosting Machines	69%	12%	66%	62%	9%
Random Forests (Ranger)	68%	11%	59%	67%	10%

^aArea under the Receiver Operating Curve

^bArea under the Precision-Recall Curve

^cBased on threshold determined by maximizing sensitivity and specificity

Table 3.

Unadjusted and adjusted odds ratios for final selected candidate variables and composite postpartum hospital use, training data, New York City, 2016–2017 (n=145,464).

Chronic/Preexisting Factors	Odds Ratio (95% CI)	Adjusteda Odds Ratio (95% CI)
Chronic hypertension	2.53 (2.30, 2.78)	1.46 (1.32, 1.63)
Gestational hypertension	2.00 (1.87, 2.13)	1.33 (1.23, 1.43)
Preeclampsia	2.30 (2.07, 2.56)	1.23 (1.10, 1.39)
Gestational diabetes	1.35 (1.26, 1.44)	1.04 (0.97, 1.12)
Asthma	1.74 (1.61, 1.89)	1.14 (1.04, 1.24)
Digestive disease	1.62 (1.47, 1.77)	1.23 (1.11, 1.36)
Major mental health disorder	2.13 (1.94, 2.34)	1.51 (1.37, 1.68)
Neuromuscular disease	2.10 (1.81, 2.45)	1.45 (1.23, 1.7)
BMI
Underweight ( < 18.5)	0.88 (0.78, 1.0)	0.88 (0.77, 0.99)
Normal weight (18.5 - <25)	Reference	Reference
Overweight (25 - <30)	1.49 (1.41, 1.57)	1.21 (1.15, 1.29)
Obesity, Class I (30 - <40)	2.00 (1.88, 2.12)	1.37 (1.28, 1.46)
Obesity, Class II (40+)	3.00 (2.68, 3.36)	1.70 (1.51, 1.92)
Antepartum and Obstetric Factors
Induction	1.24 (1.18, 1.32)	1.09 (1.03, 1.16)
Fetal distress	1.31 (1.24, 1.38)	1.10 (1.04, 1.17)
Delivery length of stay (continuous)	1.06 (1.06, 1.07)
Antepartum ED visit or readmission	2.53 (2.42, 2.65)	1.86 (1.77, 1.96)
Severe Maternal Morbidity (SMM)	2.44 (2.21, 2.69)	1.50 (1.35, 1.68)
Parity
Nulliparous	Reference	Reference
Multiparous	0.95 (0.90, 0.99)	0.87 (0.83, 0.92)
Gestational age at birth
< 32 weeks	2.36 (2.03, 2.75)	1.14 (0.94, 1.37)
32 - < 34 weeks	2.13 (1.78, 2.55)	1.19 (0.97, 1.45)
34 - < 37 weeks	1.51 (1.39, 1.65)	1.05 (0.96, 1.15)
37 - < 39 weeks	1.21 (1.15, 1.28)	1.08 (1.02, 1.14)
39 weeks +	Reference	Reference
Delivery Mode
Vaginal	Reference	Reference
Cesarean	1.93 (1.84, 2.02)	1.58 (1.50, 1.66)
Apgar Score
0–6	1.66 (1.52, 1.82)	1.13 (1.02, 1.26)
7–10	Reference	Reference
Social-structural factors
Payor
Medicaid	2.03 (1.93, 2.14)	1.57 (1.48, 1.67)
Private	Reference	Reference
Uninsured	1.17 (0.89, 1.54)	0.96 (0.71, 1.30)
Other payor	2.09 (1.70, 2.57)	1.74 (1.39, 2.19)
Late Prenatal Care Initiation (later than first trimester or never)	1.32 (1.26, 1.39)	1.07 (1.02, 1.13)
Depressive symptoms	1.22 (1.15, 1.30)	1.10 (1.03, 1.18)
Index of Concentration at the Extremes (continuous)	0.22 (0.20, 0.24)	0.65 (0.58, 0.74)

^aMutually adjusted for all variables in Table 2.

Cause-specific Model

Most common diagnoses for PHU were general perinatal reasons (17.5%) (a category comprised of non-specific ICD-10 codes for complications in the perinatal period), hypertension/eclampsia (12.0%), non-gynecologic infections (10.7%), and wound infections (8.4%) (Figure 1). The ensemble approach found that combining prediction models of PHU for six outcomes optimized the auROC: acute cardiovascular disease, gastrointestinal disease, hypertension/eclampsia, psychiatric disease, sepsis, and wound infection, achieving an auROC of 0.750 in the temporal validation (Table 4).

Figure 1.

Reasons for Postpartum Hospital Utilization (PHU) used to select ensemble Cause-specific PHU model, Training Data, New York City, 2016–2017

Table 4.

Model performance for cause-specific postpartum hospital use ensemble (PHU) model

	Training	Testing	Validation
Ensemble of ‘Cause-specific PHU’ classifiersa	auROCb=75% auPRCc=5% Sensd=69% Spece=69% PPVf=3%	auROC=77% auPRC=5% Sens=73% Spec=69% PPV=4%	auROC=75% auPRC=5% Sens=70% Spec=67% PPV=4%

^aPHU diagnoses codes selected in ensemble model to maximize auROC were acute cardiovascular disease, gastrointestinal disease, hypertension/eclampsia, psychiatric disease, sepsis, and wound infection

^bArea under the Receiver Operating Curve

^cArea under the Precision-Recall Curve

^dSensitivity

^eSpecificity

^fPositive predictive value

Equity Analyses

When validating in racial-ethnic subpopulations, the auROC for both the Composite PHU model and Cause-specific PHU models were 65% and 69% among Black people, and 65% and 72% among Hispanic people, respectively (Supplementary Table 3). When using an alternative approach of training the LASSO logistic regression model among Black people, the auROC was similar as that of the model trained in the overall population validated in Black people (auROC=0.654). A sensitivity analysis including SE and BNE neighborhood indices instead of ICE retained the SE index and had similar auROC. A sensitivity analysis including race-ethnicity as a covariate resulted in nearly identical auROC and variable selection, including ICE.

Our final step was a simulation to test the hypothetical impact on equity comparing the Composite PHU and Cause-specific PHU models, as well as a third “standard” approach which identified high-risk people by the presence of common morbidities. Using our Composite PHU modelwith an intervention with 40% efficacy could prevent 2.8 per 100 PHU in Black people, 2.2 per 100 in Hispanic people, and 0.5 per 100 in White people(Table 5). This resulted in a reduction in the Black vs. White risk difference of 2.3 per 100 and the Hispanic vs. White risk difference of 1.7 per 100. Using the Cause-specific model, the number of PHU prevented was 1.1 per 100 in Black people, 0.6 per 100 in Hispanic people, and 0.2 per 100 in White people, resulting in a reduction in the Black vs. White risk difference of 0.9 per 100 and Hispanic vs. White risk difference of 0.4 per 100. Using a “standard” approach of targeting high-risk conditions resulted in estimates between the Composite PHU and Cause-specific PHU models.

Table 5.

Equity impact for risk-stratification approaches for hypothetical postpartum hospital use (PHU) intervention

	Composite PHU Modela	Cause-specific PHU Modelb	Standard Approachc
Prediction model classifications
Sensitivity	67.8%	70.4%	43.9%
Specificity	59.8%	67.3%	69.0%
Identified as high risk (per 100)	39.3	33.4	31.8
PHU prevented (40% of true positivesd) (per 100)
Black	2.8	1.1	1.8
Hispanic	2.2	0.6	1.3
White	0.5	0.2	0.5
Equity impact
Baseline PHU observed (per 100)
Black	9.2	9.2	9.2
Hispanic	7.5	7.5	7.5
White	3.5	3.5	3.5
Baseline PHU disparity (per 100)
Black vs. White	5.7	5.7	5.7
Hispanic vs. White	4	4	4
PHU disparity after intervention (per 100)
Black vs. White	3.4	4.8	4.4
Hispanic vs. White	2.3	3.6	3.2
Change in absolute disparity (per 100)
Black vs. White	−2.3	−0.9	−1.3
Hispanic vs. White	−1.7	−0.4	−0.8

^aLasso logistic regression model; threshold for identifying high-risk p(y)=0.055

^bEnsemble model for postpartum PHU diagnoses selected to optimize predictive ability (AUC): acute cardiovascular disease, gastrointestinal disease, hypertension/eclampsia, psychiatric disease, sepsis, and wound infection; threshold for identifying high risk p(y)=0.401

^cPersons identified as high-risk and qualified for intervention if any of the conditions were diagnoses were present at the time of delivery: pre-gestational diabetes, gestational diabetes, pre-pregnancy hypertension, hypertensive disorders of pregnancy, depressive symptoms, mental health diagnosis

^dCalculation for an intervention that is 40% effective based on example of published postpartum intervention Herschberg et al

Comment

Principal Findings

We developed a prediction model of Composite PHU incorporating individual- and area-level SSDOH for risk stratification at the delivery hospitalization. The model has good predictive ability, and coupled with an effective intervention in a simulation exercise, demonstrated the potential to reduce racial-ethnic inequities in PHU. A second model used an ensemble approach to choose Cause-specific PHU outcomes that are easier to predict, and therefore showed superior predictive ability, but because many PHU outcomes were excluded, the model had a minimal impact on racial-ethnic equity in PHU. Our Composite PHU model also compared favorably to a standard clinical approach of risk stratification by common maternal morbidities.

Results in the Context of What is Known

Our findings can be considered in the context of existing literature on prediction of PHU. A study using California linked birth and hospital data to predict 60-day PHU had a slightly lower auROC than ours (0.67 compared to 0.69 from our Composite PHU model and 0.75 for Cause-specific model).²⁵ Our analysis considered a greater number of candidate variables, including SSDOH variables. Hospital readmissions and ED visits are often difficult to predict,²⁶ in part due to the heterogeneity of the outcome, thus it is logical that limiting the ensemble model to a subset of reasons for PHU yielded a higher auROC. Our overall PHU model performed slightly worse in Black and Hispanic people. Nonetheless, because Black and Hispanic people were more likely to have high predicted probabilities, when we tested the potential equity impact of our model, we found that coupled with an effective intervention, our Composite PHU model had the greatest impact on postpartum health inequities.

Clinical implications

Our findings have implications regarding the current concern with obstetric prediction algorithms that include race as a covariate, because they could “embed” racial bias into the algorithm.²⁷ In obstetrics, algorithms using race, including the VBAC calculator^28,29 and anemia treatment guidelines³⁰ have recently been revealed as resulting in disparities in care. Because the ultimate goal of identifying people at risk of PHU is to intervene, predictive models including race may result in health care bias. Meanwhile, structural racism and neighborhood deprivation are increasingly recognized as important determinants of maternal health equity and postpartum readmissions.^31,32 We found that the ICE, a proxy for structural racism, was selected in our final model. The social-economic index, likely a downstream manifestation of structural racism, performed similarly, and adding race-ethnicity to our model did not improve performance. These results suggest incorporating structural determinants into clinical prediction models may help identify those at risk of poor health due to structural racism and disadvantage.

Research implications

Postpartum interventions currently under investigation often target high-risk groups on certain morbidities, e.g. hypertension or SMM. Although potentially effective in improving postpartum health for these subsets, they only represent a minority of the birthing population. Our whole-population approach enables us to capture birthing people who might not be targeted for high-risk postpartum support. Indeed, our Composite PHU model when compared with a “standard” risk stratification approach performed better in terms of racial-ethnic disparity reduction. Future research, however, is needed to determine the most effective way to implement our postpartum prediction model to improve postpartum care, also taking into consideration feasibility and cost. Potential multipronged interventions could address system- and hospital-level SSDOH while providing wraparound services to address clinical and individual-level SSDOH and empower patients. Additionally, our model should be externally validated in other geographic regions that may differ in the distribution of SSDOH measures, potentially using electronic medical record data. Also, alternate geographic definitions of area-based measures, e.g. census tract or buffer zones, could be tested.

Strengths and limitations

A primary strength of our analysis is our equity approach. We used a large dataset with a rich array of candidate variables.We chose area-based measures of SSDOH that are publicly available nationally. We went beyond reporting model performance to look at the potential of the risk stratification model to impact postpartum health equity. A primary limitation is lack of primary collected patient data, e.g. blood pressure measures or other biomarkers. The Cause-specific model is limited by the finding that a leading cause of readmission was non-specific ICD-10 codes making it difficult to know what types of clinical situations these category represents. We did not have data on stillbirths, which result in higher rates of maternal readmission than livebirths.³³ We were not able to ascertain obstetric triage visits. Further, many ED visits had a general ICD-10 code for general perinatal reasons – outcome data with more specific diagnosis codes may have altered model performance. We did not ascertain gender, thus it is unclear if our findings are generalizable to all pregnancy-capable genders. Also, our predictive model has yet to be paired with a real-life intervention, in order to evaluate the actual clinical impact of targeting postpartum care using a prediction algorithm.

Conclusion

A Composite PHU model incorporating SSDOH can be used to identify persons at risk for PHU with the potential to reduce racial-ethnic inequities in postpartum health.

At A Glance:

Why was this study conducted?

Racial inequities in maternal morbidity and mortality persist into the postpartum period, leading to a higher rate of postpartum hospital use (PHU) among Black and Hispanic people. Delivery hospitalizations provide an opportunity to screen and identify people at high risk to prevent adverse postpartum outcomes. Current models do not adequately incorporate social and structural determinants of health (SSDOH), and some include race, which may result in biased risk stratification.

Key findings

Our model predicting PHU had good predictive ability (auROC=0.69), and in a simulation with an effective intervention, reduced racial-ethnic inequities in PHU. SSDOHvariables were important features in the model and performed as well as a model including race. A PHU model targeting selected outcomes that are easier to predict had a superior auROC (0.75), but had a minimal impact on racial-ethnic equity in PHU.

What does this add to what is known?

Our findings provide evidence that social and structural determinants can be used in clinical algorithms in place of race or ethnicity. We also showed that risk stratification for PHU has the potential to reduce racial-ethnic inequities in postpartum PHU greater than focusing on easier to predict postpartum diagnoses, or common pregnancy complication risk groups targeted for postpartum support.