Reducing Disparities in Postpartum Care Utilization: Development of a Clinical Risk Assessment Tool

Abstract

Introduction

Patients receiving care at Federally Qualified Health Centers (FQHCs) have low postpartum care attendance. Perinatal morbidity and mortality disproportionately affects low-income patients and are potentially preventable. The purpose of this study was to develop a clinical decision support tool to identify FQHC patients less likely to return for postpartum care. To accomplish this purpose we evaluated established predictors and novel risk factors in our patient population.

Methods

This is a retrospective, secondary data analysis of 50,022 patients who received prenatal care past 24-weeks gestation in FQHCs between 2012–2017. The postpartum visit was defined using HEDIS measures as early care (birth to 21-days) and later care (21- and 84-days). Anderson’s Behavioral Model for Access to Healthcare guided inclusion of potentially predictive factors. We stratified data by postpartum care attendance and a final predictive model was selected by model fit statistics and clinical relevance.

Results

In our sample, 64% of birthing persons attended postpartum care at FQHCs. Of those who returned for care 38% returned within 21 days post birth, 62% returned between 21–84 days, with 28% returning for both early and later care. Predictors for postpartum care attendance included maternal age, parity, gestational age at first visit and number of prenatal care visits. A clinical decision support tool for identifying patients less likely to return for care was created.

Discussion

An easy to implement clinical decision support tool can help identify FQHC patients at risk for postpartum non-attendance. Future interventions to improve adequacy of prenatal care can encourage early entry into prenatal care and sufficient prenatal visits. These efforts may improve postpartum care attendance and maternal health.

Précis

An easy to implement clinical decision support tool can help identify Federally Qualified Health Center patients at risk for postpartum care non-attendance.

Introduction

Among the most pressing issues in our nation is the increase in pregnancy associated mortality. In the United States, the pregnancy related mortality ratio has steadily increased from 7.2 per 100,000 live births in 1987 to 23.8 in 2020 ^1,2. Despite spending more on healthcare services than other high-income countries, more birthing people in the United States are dying during pregnancy or within the first-year post birth ^3–5. It is estimated that roughly 83% of pregnancy related deaths are preventable ⁶. More than half of pregnancy related deaths occur after the birth of a baby ⁷, therefore the immediate postpartum period is a critical time to recognize and address health issues. The American College of Obstetricians and Gynecologists (ACOG) recommends that patients have contact with their healthcare clinicians within the first 3 weeks postpartum⁸, either through an in-person visit or via telehealth ⁹. Following this first encounter, comprehensive postpartum care is recommended within 84 days post birth. Despite receiving prenatal care, many low-income patients are disconnected from healthcare services following the birth of their infant. This is a serious health issue, which disproportionately affects those who are low-income.

Postpartum visit attendance rates are lower for low-income patients.^10,11 Fewer than 60% of patients who qualified for Medicaid returned for postpartum care within 21–56 days following the birth of their infant.¹² Perinatal morbidity and mortality, infant mortality, preterm birth rates, and poor mental health are health concerns are potentially preventable in this population.^13–16 Postpartum care thus allows clinicians to identify risk factors for poor health outcomes, assess for maternal-infant bonding and postpartum depression, and provide anticipatory guidance for reproductive health and family planning.

Several researchers have attempted to develop clinical decision support tools that identify patients at risk for postpartum care non-attendance. These tools often focused on socioeconomic variables including insurance status, which limits the utility for use with a Medicaid patient population. Significant variables for postpartum care non-attendance included age, ^17,18 marital status, ^18,19 education,²⁰ race and ethnicity, ^17–20 employment status,²⁰ transportation issues and distance from health center. ^12,21 Substandard prenatal care as measured by the Kotelchuck Adequacy of Prenatal Care Utilization is also a significant risk factor for postpartum care nonattendance.¹⁹ Depending on geographical region studied, rates of returning for postpartum care vary from a high of 91% in California,¹⁸ to a low of 50% in Virginia.²⁰ The significance of these individual risk factors differ based on the population studied. Consistently, patients of lower socioeconomic status and patients who had public health insurance or Medicaid were less likely to return as compared to patients with higher incomes or private insurance ^12,20,22. Many of these patients receive care at Federally Qualified Health Centers (FQHCs), which are community-based healthcare facilities for underserved populations. The purpose of this study was to develop a clinical decision support tool for identifying patients receiving care at FQHCs who are less likely to return for postpartum care. To accomplish this purpose, we evaluated established predictors and novel risk factors in our patient population.

Methods

Study Design

This study is a retrospective secondary analysis of electronic health record data previously collected over a 5-year period from 2012 to 2017. The study involved minimal risk to patients and was approved by the Loyola University Chicago Institutional Review Board of a large academic medical center in the Midwest. A waiver of informed consent and Health Insurance Portability and Accountability Act authorization was granted from the IRB and from our research data partner as required for access to protected health record data.

Our research data partner, AllianceChicago, is an organization that consolidates health information technology, research, and education for many of the FQHCs. They have an infrastructure to support research and targeted clinical improvement initiatives. Our research team included clinicians and data scientists from the AllianceChicago team.

Setting and Sample

The AllianceChicago Federally Qualified Health Centers in this study include 17 multisite health centers from across the United States, including both urban and rural locations. These FQHCs share an electronic health record and infrastructure for data analysis, quality improvement measures, and clinician education.

We utilized data from over 50,000 unique prenatal patients who received care at AllianceChicago FQHCs. Patients receiving care at FQHCs were primarily covered by Medicaid insurance, which provided pregnancy related services through 60 days post birth. ²³ To be included in the analysis, patients must have attended at least one prenatal care visit past the point of viability (24 weeks), as a postpartum visit would likely not have been scheduled if the patient did not complete their prenatal care within the same FQHC nor if the pregnancy did not result in a viable birth.

Postpartum Care

The Healthcare Effectiveness Data and Information Set (HEDIS) methodology defines a postpartum visit as a wellness exam before 84 days postpartum ²⁴. The data used for this analysis is from the Alliance electronic health record system, which is clinic level data and not synchronized with hospital or birth center data. In order to estimate days postpartum when a delivery date was not provided, we used the estimated due date as a proxy for actual delivery date. Our primary outcome measure was postpartum care within 84 days after birth and was dichotomized as either attending a postpartum visit or non-attendance. The HEDIS measure designates early postpartum care as a visit within 21 days after delivery, and later postpartum care as a visit between 22 and 84 days after delivery. Postpartum visits were identified with ICD-10 codes (Z39.0, Z39.1, and Z39.2) specific for postpartum care.

Candidate Predictors

The Behavioral Model for Access to Healthcare ²⁵ was used as a guide for the inclusion of variables for this analysis. Potential predictive factors included predisposing characteristics, health behaviors, and health status during pregnancy derived from the EHRS. These potential predictors reflect the variables identified in the current literature as factors related to postpartum care attendance. Predisposing characteristics included demographics and social determinants such as age, race, ethnicity, marital status, and employment. Health behaviors included gestational age at entry into prenatal care and number of prenatal visits both of which are represented in the Koltechuck index of adequacy of prenatal care ²⁶. These two indicators, entry into prenatal care and number of prenatal visits, were combined into a single summary score. The Koltechuck index categorized the summary score as inadequate, intermediate, adequate, or adequate plus care. Health status during pregnancy included co-morbid conditions or complications in pregnancy such as hypertension, diabetes, obesity, and prenatal depressive symptoms. Individuals were considered positive for hypertension, diabetes, and/or obesity based on ICD codes. Individuals were considered positive for prenatal depressive symptoms if their Edinburgh Postnatal Depressive Scale (EPDS) was greater than 10 or if the Patient Health Questionnaire (PHQ-2) was greater than 0.

Data Analysis

Predisposing characteristics, health behaviors, and health status during pregnancy were described using measures of central tendency and dispersion for continuous variables; frequencies and percent for categorical variables. Data were stratified by postpartum care as attending a postpartum visit or non-attendance within 84 days following delivery. Differences were assessed for statistical significance using logistic mixed effects models that included random intercepts to account for correlation due to patients with multiple pregnancies over the study time period. The mixed effect regression analysis approach allowed us to include one or multiple pregnancies over time for each birthing person and account for this correlation in hypothesis testing and confidence interval calculation. This analytic technique further enabled us to appropriately control for the correlation this introduces into the analysis. Continuous predictors were plotted using restricted cubic splines to assess for the direction and linearity of associations with non-attendance, and non-linear (i.e., quadratic) terms were introduced where appropriate. We assessed missing data for each variable before building our model. In some cases, we used single imputation where it was reasonable to do so ²⁷.

Among candidate mixed effects logistic regression models, receiver operating characteristic (ROC) curves were plotted, and discrimination was evaluated using the c-statistic. Model calibration was assessed with a calibration plot. A final model was chosen based on model fit statistics, discrimination, parsimony, and clinical relevance. A decision curve was constructed to evaluate the clinical value of potential implementation of the final model. We followed reporting guidelines for multivariable prediction model development as outline in the TRIPOD statement.²⁸ Analyses were performed using SAS 9.4 (SAS Institute, Cary, NC).

Results

Cohort characteristics

A total of 50,022 patients received prenatal care from 17 multisite FQHCs representing both urban and rural communities, met inclusion criteria for this study, and were included in the analysis. The sample included a majority of Hispanic (63%) and non-Hispanic-Black (21%) patients. Mean age of the sample was 27 years (SD 7.08), 40% were married or partnered, and 24% reported being employed. Patients in our sample entered into prenatal care on average at 14 (range 6 to 22) weeks with an average of 10 (range 5–15) prenatal visits. Using the Koltechuck index ref as described above ²⁹, 41% of patients received inadequate care, 17% received intermediate care, 30% received adequate care and 12% received adequate plus care. Over one-third (37%) of our sample did not return for postpartum care. Of patients that did return for postpartum care, 38% returned within 21 days, 54% returned between 21–56 days, and 8% returned between 57–84 days. Of those who returned, 10% returned only for early postpartum care (before 21 days), 62% returned only for later care (21–84 days), and 28% returned for both early and later care.

Model development

Our model development began with univariable analysis of all predictors. Continuous variables were plotted against the probability of a postpartum visit attendance as shown in Table 1. Most variables measured were statistically different in univariable analysis to predict a return visit for postpartum care. However, many of these differences were small and not likely to be clinically significant. Employment and income were removed from consideration since less than 10% of patients had data entered in these fields, and this was considered non-ignorable missing data. Maternal age and gestational age at first prenatal visit had a non-linear relationship with the likelihood of returning for a postpartum care visit. Younger as well as older patients were less likely to return compared to those ages 28–30. Additionally, patients with initial prenatal care visits in the first and third trimester were more likely to return than patients who presented for care in the second trimester. Other statistically significant continuous predictors included parity and the total number of prenatal visits. The proportion returning for postpartum care increased with higher numbers of prenatal care visits, and decreased with increasing parity. Our model does not include race or ethnicity so as not to withhold an intervention from one individual and provide to another who are otherwise similar on the important model predictors.

Table 1:

Participant characteristics by Postpartum Visit (PPV)

	No PPV N=18587	Any PPV N=31435	p-value
Mother’s age, mean (SD)	26.8 (6.2)	27.1 (6.0)	<0.001
Race/ethnicity, n (%)
Non-Hispanic White	1582 (8.5)	2691 (8.6)	<0.001
Non-Hispanic Black	3958 (21.3)	6042 (19.2)
Non-Hispanic Other	846 (4.6)	1878 (6.0)
Hispanic	11660 (62.7)	19943 (63.4)
Unknown	541 (2.9)	881 (2.8)
Marital status, n (%)
Married	5651 (30.4)	9943 (31.6)	<0.001
Partnered	1880 (10.1)	2763 (8.8)
Single	10410 (56.0)	18119 (57.6)
Not recorded	646 (3.5)	610 (1.9)
Employment, n (%)
Full time	2887 (15.5)	7773 (24.7)	<0.001
Part time	436 (2.3)	732 (2.3)
Not employed	7172 (38.6)	14205 (45.2)
Not recorded	8092 (43.5)	8725 (27.8)
Primary language, n (%)
English	9489 (51.1)	17144 (54.5)	<0.001
Spanish	8534 (45.9)	13268 (42.2)
Other	479 (2.6)	988 (3.1)
Not recorded	85 (0.5)	35 (0.1)
Prenatal care
Prenatal care initiated, n (%)
1st trimester	10435 (56.1)	20546 (65.4)	<0.001
2nd trimester	5935 (31.9)	7793 (24.8)
3rd trimester	2217 (11.9)	3096 (9.8)
Total visits, n (%)
< 5	5030 (27.1)	2309 (7.3)	<0.001
5–8	5647 (30.4)	5780 (18.4)
≥ 9	7910 (42.6)	23346 (74.3)
Gestational age (weeks) at first PNV, mean (SD)	15.1 (8.5)	13.6 (8.2)	<0.001
Parity, n (%)
0	6298 (33.9)	11217 (35.7)	<0.001
1	4913 (26.4)	8609 (27.4)
≥ 2	7376 (39.7)	11609 (36.9)
Not recorded
Maternal BMI prior to pregnancy, mean (SD) [n=21076]	27.6 (6.4)	27.6 (6.2)	0.72
ICD code for obesity, n (%)	2705 (14.6)	4497 (14.3)	0.99
ICD code for diabetes, n (%)	2443 (13.1)	3685 (11.7)	<0.001
ICD code hypertension, n (%)	881 (4.7)	1919 (6.1)	<0.001
PHQ2 > 0, n (%) [n=33714]	682 (6.0)	1310 (5.9)	0.14
EPDS > 10, n (%) [n=9881]	250 (11.4)	754 (9.8)	0.03
EPDS > 10 or PHQ9 > cutoff, n (%) [n=36067]	889 (7.5)	1917 (7.9)	0.16

Of 31,435 who had a PPV, 3245 (10.3%) only had a visit within 3 weeks post-delivery, 19,454 (61.9%) only had visit(s) 4–12 weeks post-delivery, and 8736 (27.8%) had visits both within 3 weeks and also 4–12 weeks post-delivery.

After consideration of both clinical and statistical significance in evaluating candidate models, a 6 variable model comprised of four risk factors was selected: mother’s age at first visit, gestational age at first visit, the total number of prenatal care visits, and mother’s parity (see TABLE 2). Quadratic terms for mother’s age and gestational age at first visit were included due to the non-linear relationship of these variables with the log odds of returning for postpartum care. This model had good discrimination validity (c-statistic=0.73, see FIGURE 1; and calibration).

Table 2:

Beta coefficients estimated using mixed effects logistic regression to predict postpartum care non-attendance

	$\widehat{\beta }$	SE	p-value
Intercept	1.8633	0.0461	<0.001
Mother’s age	−0.0130	0.0022	<0.001
(Mother’s age)2	0.0013	0.0003	<0.001
Gestational age at first visit, (weeks)	−0.0265	0.0021	<0.001
(Gestational age at first visit, (weeks))2	−0.0022	0.0002	<0.001
Number of prenatal care visits	−0.2595	0.0047	<0.001
Parity	0.0865	0.0092	<0.001

$\widehat{\beta }$-estimated beta coefficient; SE-standard error

Mother’s age < 15 was recoded to 15 and age > 45 recoded to age 45. Gestational age at first visit < 4 weeks was recoded to 4 weeks and > 41 weeks to 41 weeks. Age and gestational age were then mean-centered (subtract 27 years from mother’s age; subtract 14 weeks from gestational age at first PNV). Prenatal care visits > 18 were recoded to 18, and parity > 6 was recoded to 6.

Figure 1:

Receiver Operating Characteristic Curve (ROC Curve): The ROC is a measure of the discrimination of the model. The ROC curve displays the sensitivity and specificity of all potential cut-points in predicted probability of no postpartum care attendance.  

Model evaluation

A decision curve was plotted to visualize the model’s net benefit in contrast to approaches to intervene on none versus all patients. Threshold probability may be interpreted as the preference for intervention or resource limitations, where smaller values represent emphasis on intervening to prevent anyone from missing a postpartum visit, while larger values represent less frequent interventions due to limited resources or knowing not all patients need an intervention to return for care. The model has higher net benefit compared to the “intervene on all” approach for threshold probabilities of 0.2 and greater (See FIGURE 2) ³⁰.

Figure 2:

Decision curve: The decision curve evaluates the model as probability of net benefit against threshold probability. The dotted line represents the benefit if an intervention is created for patients based on our model prediction.

Discussion

Almost half (43%) of patients receiving prenatal care at FQHCs did not return for postpartum care within 84 days post-birth. This rate of non-attendance is higher than previously reported in PRAMS (11%) or Illinois Medicaid claims data (19%) ^12,31. It is possible that this discrepancy is partially due to patients returning to postpartum care at outside facilities. Unlike PRAMS self-reported data, which may be susceptible to recall bias, our dataset would not have captured postpartum visits unless they were in the FQHC system. Likewise, Medicaid claims data is not institution specific.

Although non-attendance of postpartum care was higher in our study compared to previously published research, the predictive variables for postpartum care non-attendance in our study were similar to other findings. For example, both ends of the age spectrum including older ¹⁸ and younger ¹⁷ patients were significantly less likely to attend their postpartum visit. Prior models did not use a curvilinear variable and may have missed this complex relationship. Similarly gestational age at first prenatal care visit also had a non-linear relationship, with individuals entering prenatal care in the first trimester and in the weeks before birth as most likely to return for care. This information has not been reported in the literature and may be unique to this FQHC population. A prior publication simplified beta coefficients into points, therefore losing some precision in the risk estimation.²² Finally, another advantage of our tool is the ability for easy precise calculations using an online calculator.

We identified that birthing persons with increased parity are less likely to attend a postpartum visit, which is supported by previously published studies ^32,33. Similarly, previously published research also sites inadequate prenatal care as a significant predictor for non-attendance of postpartum care ^18,20,33. We used the Koltechuck index to determine adequacy of prenatal care and our findings are also suggestive that inadequate prenatal care decreases likelihood of a postpartum visit. Furthermore, previous research has not included this variable or the number of prenatal care visits. While efforts can be made to encourage initiation of prenatal care at an earlier gestational age, the number of visits are the only potentially modifiable variable in our model. Clinicians and researchers can potentially affect this predictor by developing evidence-based strategies that encourage patients to return for scheduled prenatal visits.

Previous reports suggest patients with high-risk pregnancies are more likely to return for postpartum visits ^32,33. Our study explored individual risk factors with relationship to postpartum care attendance. Patients with diabetes were less likely to return for postpartum care (P<.001) and patients with hypertension were more likely to return for postpartum care (P<.001). While we had statistically significant data for hypertension and diabetes, these differences were small and not clinically significant. We found no difference in postpartum care attendance between patients with or without pre-pregnancy obesity or depressive symptoms. Previous studies suggest patients with depressive symptoms are more likely not to attend postpartum care ³³. Generalizing patients as high-risk based on a high-risk diagnosis alone may overestimate the likelihood for returning for postpartum care.

Postpartum health reports in marginalized populations often focus on insurance, socioeconomic status, race, and ethnicity^20,31. FQHC patients are at greatest risk for many health sequelae postpartum, yet this population has not been well studied to assess for risk factors nor has a clinical decision support tool specifically been created for this population. The majority of patients in FQHCs are low-income, non-white, and receive Medicaid insurance. Previous predictive tools for postpartum care attendance use a point scale, with the highest predictor of non-attendance being Medicaid health insurance ²². Our clinical predictive tool was designed for low-income populations receiving care at FQHCs, most of whom have Medicaid insurance. The secondary data analysis was conducted to assist clinicians with identification and prioritization of FQHC patients in need of additional support for attending postpartum care visits. We suggest future studies externally validate this tool, and to that end, we have made our model publicly available via an online application: https://lucmodels.shinyapps.io/ppv_calculator.

Our sample of FQHC patients does not reflect the general population range on income and insurance status. Additionally, 91% of patients in our sample did not have an income listed in their demographic section, it is unclear if this lack of information is patient preference or data not captured at registration. Our sample was well represented with white and non-white, with a majority Hispanic minority patients. As shown in Table 1, Hispanic patients were slightly more likely to return for postpartum care, while Black patients were slightly less likely to return for postpartum care. Although this is statistically significant due to our large sample size, it is not clinically significant and not included in our model. Furthermore, we did not want to provide or withhold an intervention from any patient based on race/ethnicity who otherwise have a similar risk profile. However, this information is important for creating culturally tailored interventions in the future.

Strengths and Limitations

The use of a large data set combined with clinic level data is a strength of this study with the power to detect small differences and minimize recall bias, however limitations were also evident. This study is a secondary data analysis of previously collected electronic health record data. As such, the creation of the decision support tool was data driven. While community partnership is helpful for creating interventions for improving patient care, it was not possible due to the nature of the study design.

The FQHCs in this sample often follow traditional care models which limits the ability to assess alternate care models, such as group prenatal care. These alternate care models may provide different predictors and lead to different interventions. Likewise, information was not available at the level of individual clinics including geographical location, number of patients seen daily, scheduling challenges, nor acuity. Each of these individual clinic characteristics may influence ease of scheduling appointments, wait times, and patient satisfaction which may be related to perinatal care. In 2018 the HEDIS measure changed and now postpartum care is defined as early or later postpartum care establishing a challenge to compare data to previous studies ²⁴. Our primary outcome of postpartum care attendance may be potentially underreported if care was obtained at a facility outside of AllianceChicago FQHC system or not coded as a postpartum visit. Electronic health record data is designed for patient care and billing purposes, not to answer specific research questions. Reporting bias may have limited which data fields were completed (e.g. income). Missing data was a challenge and data was only as good as clinician documented and patient reported. We also encountered non-ignorable missing data, like employment, where a large percentage of patients did not disclose this information. We also did not have protected health information, including distance from the health center. Although zip code concurrence between the patient and the health center was attempted for a proxy measure for distance, this was not useful as many urban areas have multiple close range zip codes and rural areas may share one zip code. Finally, this study is based on a group of similar federally qualified health care systems and results may not be generalizable outside of these clinics.

Implications for practice, policy, and research

Postpartum Care is a necessary component of full spectrum prenatal care. Two thirds of perinatal deaths occur at delivery of the infant though the first-year post birth ⁷. In this study we identified age, parity, gestational age at entry into care, and number of prenatal visits as predictors that contribute to postpartum care attendance. The 3 non-modifiable predictors (age, parity, initiation of prenatal care) increase the risk for postpartum care non-attendance and are easily identified during pregnancy. Conversely, the number of prenatal visits is a modifiable risk factor and clinicians should work closely with patients to facilitate attendance with this important standard of care.

Toward the end of pregnancy this tool is then most useful and predictive.

Interventions to improve postpartum care attendance can be addressed through public health efforts and clinic resources. Ensuring adequate prenatal care, with early entry and sufficient prenatal visits, predicts postpartum care attendance in this sample. Encouraging newly pregnant patients to initiate care at FQHCs will require public health efforts and clinic outreach. Once prenatal care is initiated, the onus is on the healthcare system to ensure patients return for scheduled, adequate prenatal care. Maternal age and parity can be identified early in pregnancy and the clinical decision support tool can be incorporated into patient centered care while discussing risk factors for postpartum care nonattendance. Tailoring interventions for patients most likely to not attend postpartum visits allocates resources wisely. Maternity care teams can create culturally tailored interventions such as phone call reminders, text messages, home visits, and telehealth to increase the likelihood of postpartum care attendance for at risk patients. Using an evidence based statistical model for individual healthcare systems can provide more reliable data for clinical decision support with limited resources.

Conclusion

Postpartum care attendance in this sample was lower than previously reported ^4,12,31. FQHCs serve a unique population of birthing persons in underserved communities. These patients are at increased risk of co-morbidities and postpartum care is crucial to optimize best healthcare outcomes. Despite lower rates of postpartum care attendance in our study, identifying prenatal predictors can benefit clinicians and researchers to utilize limited resources and improve healthcare delivery for the vulnerable postpartum patient who may not otherwise return for care. Further exploration of postpartum care attendance with broader datasets and networks beyond the FQHCs may expand the utility of the clinical decision support tool.

Quick Points:

• Perinatal morbidity and mortality disproportionately affects low-income patients and are potentially preventable

• Consistently, patients of lower socioeconomic status and patients who had public health insurance or Medicaid were less likely to return for postpartum care as compared to patients with higher incomes or private insurance

• Maternal age, parity, gestational age at first visit and number of prenatal care visits are predictors of postpartum care non-attendance for FQHC patients