Abstract
Background and aims
COVID‐19 forced healthcare systems to implement telehealth programs, facilitated in Massachusetts by a policy requiring insurers to reimburse for telehealth visits. Prior studies suggest that telehealth is effective for obstetric care, but little is known about its implementation in response to policy changes in underserved communities. We utilized the RE‐AIM framework to evaluate telehealth implementation in a large academic urban obstetric practice that serves a medically underserved population.
Methods
RE‐AIM elements were assessed through retrospective review of electronic health record (EHR) data for all obstetric encounters between March 19 and August 31, 2020 and review of clinic implementation processes. Data extracted included demographics, number and type (in‐person or telehealth) of prenatal visits, prenatal diagnoses, delivery outcomes, and number and type of postpartum visits. Data were analyzed using descriptive statistics.
Results
A total of 558 patients (60.6% Hispanic; 13.2% primary language Spanish) had 1788 prenatal visits, of which 698 (39.0%) were telehealth visits. A total of 209 patients had 230 postpartum visits, of which 101 (48.3%) were telehealth visits. The Reach of the intervention increased from 0% of patients at baseline to 69% in August. Effectiveness measures were limited but suggested potential for earlier diagnosis of some prenatal conditions. Adoption was high, with all 30 providers using telehealth, and the telehealth was found to likely be feasible and acceptable based on uptake. Increases in the percentage of telehealth visits over time and continuation post‐lockdown suggested maintenance was potentially achievable.
Conclusions
The COVID‐19 pandemic has changed traditional approaches to healthcare delivery. We demonstrate that the use of the RE‐AIM framework can be effective in facilitating implementation of telephone visits in a large academic urban obstetric practice after state‐level policy change. This may be of particular importance in settings serving patients at higher risk for maternal morbidity and poor birth outcomes.
Keywords: healthcare policy, implementation analysis, postpartum care, prenatal care, telehealth
1. INTRODUCTION
At the onset of the COVID‐19 pandemic, obstetric practices in states that experienced major surges in COVID‐19 cases needed to act quickly to develop innovative approaches to delivering high‐quality obstetric care while minimizing the potential exposure of patients to the novel virus. 1 Telehealth was quickly viewed as an important potential alternative to in‐person prenatal and postpartum services. Prior to and during the early stages of the COVID‐19 pandemic, delivering prenatal care via telehealth was found to be associated with comparable health outcomes and improved patient satisfaction and engagement when compared with in‐person care. 2 , 3 , 4 , 5 , 6 Prior studies also suggest that care delivered via telehealth is associated with improved obstetric outcomes, especially concerning the surveillance and management of gestational hypertension, 2 the reduction of smoking, as well as establishing higher rates of initiation and maintenance of breastfeeding in the postpartum period. 2 Telehealth theoretically has the potential to address some inequities in perinatal care by improving access for patients who face barriers such as limited transportation or difficulty finding child care, but it also risks introducing new inequities related to differential access to technology. 7 , 8 , 9 , 10 Ensuring equitable access to telehealth has the potential to reduce healthcare inequities worsened by the COVID‐19 pandemic. 11 Despite the potential benefits of having a telehealth option for eligible prenatal and postpartum visits, poor payer reimbursement, patient preferences, and general provider discomfort with telehealth limited the uptake of telehealth services in obstetrics before and during the early stages of the COVID‐19 pandemic. 9 , 12 , 13
The first confirmed COVID‐19 case in Massachusetts occurred in Boston on February 1, 2020. 14 As cases continued to increase throughout the commonwealth, a state of emergency was issued on March 10, and on March 17 emergency orders from the governor were issued limiting in‐person gatherings. 15 System‐wide policy changes went into effect for healthcare settings as well, including but not limited to the expansion of coverage for telehealth visits by insurers beginning with visits dated on March 12, 2020 in addition to the mandate of payment parity between insurers. 16 , 17
Healthcare organizations in many, if not all, states have implemented some version of telehealth during the pandemic, and it appears likely to remain part of the healthcare delivery infrastructure post‐COVID. 18 , 19 Understanding and disseminating best practices for implementing and sustaining such programs for individuals in populations with disparate maternal mortality and morbidity outcomes is critical to optimizing the effectiveness of telehealth for prenatal/postpartum care. We used the RE‐AIM (reach, effectiveness, adoption, implementation, and maintenance) implementation framework 20 , 21 to evaluate the rapid implementation of telehealth services for prenatal and postpartum care during the COVID‐19 pandemic in a large urban academic practice. The RE‐AIM framework was selected because of its utility in evaluating multiple elements of implementation of an evidence‐based intervention such as telehealth.
2. METHODS
2.1. Study setting and population
This study evaluated the implementation of telehealth services for patients who received their prenatal or postpartum care at a large urban academic obstetrics and gynecology practice located in Springfield, Massachusetts in the United States, between March 19, 2020, and August 31, 2020. The clinic is located in a tertiary care hospital and provides care for both low‐ and high‐risk pregnancies; high‐risk pregnancies are co‐managed by maternal‐fetal medicine specialists. The clinic serves a population comprised of 60% Latina/x identifying patients, the majority of whom had low incomes as evidenced by 89.9% being insured by Medicaid, a government‐funded health insurance plan available only to people below certain income levels. The clinic serves as the regional referral center for high‐risk obstetrical patients as well as lower‐risk patients for the surrounding communities for whom no referral is needed.
2.2. Description of implementation
Before the COVID‐19 pandemic, the clinic utilized traditional prenatal and postpartum care models, which included only in‐person visits at standard visit intervals. Patients began to be scheduled for either in‐person or telehealth appointments beginning on March 19, 2020 in response to the declaration of a state of emergency and the initiation of significant restrictions on in‐person visits to healthcare providers. Decisions about whether an appointment would be in‐person or via telehealth were made based on patient preference, medical complexity, and/or the need for ultrasound/bloodwork or other congruent in‐person services.
Implementation strategies included communication from clinic leadership about the introduction of telehealth, provider and clinic staff education about options for the provision of prenatal and postpartum care, and appropriate billing. 22 Communication with providers and staff regarding telehealth implementation took place via e‐mail, memos, technical instruction regarding use of phone systems to contact patients, and mandatory virtual meetings. Video equipment was available if patients had the technology to support video telehealth visits, otherwise visits were conducted via audio‐only telephone calls. New note templates in the electronic medical record also were made for telehealth visits, including the necessary information to meet billing guidelines. Providers were trained to properly confirm patient identity prior by confirming patient's full name and date of birth before engaging in any conversations with protected personal health information.
2.3. Data collection
We evaluated implementation of the telehealth program and selected health outcomes and processes through retrospective chart review of all pre‐ and post‐natal care visits conducted between March 19, 2020, and August 31, 2020. These dates were chosen to coincide with the mandated lockdown restrictions in the state of Massachusetts that began on March 19, 2020 and began to be lifted at the end of May and throughout the month of June. Data were collected in July and August to evaluate the maintenance of telehealth as an option for prenatal and postpartum visits after state‐mandated restrictions had been significantly eased. The following data were extracted for each prenatal and postpartum encounter using a structured data collection instrument: patient age, race/ethnicity, visit type, and pregnancy, delivery, and prenatal care outcomes as relevant to the encounter using REDCap 23 , 24 , 25 electronic data capture tools. REDCap (Research Electronic Data Capture) is a secure, web‐based application designed to support data capture for research studies.
Application of RE‐AIM framework:
Reach (the proportion of a target population that participated in the intervention). Reach was assessed based on the percentage of visits across the specified time interval for each visit type (telehealth and in‐person). Reach was further assessed based on population‐level demographic information (eg, age, primary language spoken, insurance type, race, gravida, parity, and singleton vs multiple gestations).
Effectiveness (the positive and negative outcomes of the intervention). Effectiveness was assessed by exploring various health outcomes in relation to the number of telehealth visits a patient had. Prenatal outcomes included hypertensive diseases of pregnancy (including gestational age at diagnosis), presence of pre‐existing mental health disorders, and gestational diabetes (including gestational age at the time of diagnosis). Postpartum outcomes included gestational age at delivery, induction of labor, mode of delivery (induction vs spontaneous vaginal delivery, operative [vacuum or forcep assisted] vaginal delivery or cesarean section), live birth vs intrauterine fetal demise (IUFD), the percentage of patients who received a prescription for any contraceptive methods prior to 8 weeks postpartum and for long‐acting reversible contraception (LARC), exclusive breastfeeding rate, Edinburgh postnatal depression scale (EPDS) scores, number of postpartum presentations to the emergency department or triage unit, and the percentage of patients who completed a 2‐hour glucose tolerance test before 8 weeks postpartum among patients meeting criteria for gestational diabetes.
Adoption (a measure of the temporal pattern of implementation or the proportion of settings implementing the intervention). Adoption was measured as the percentage of total telehealth visits compared with the total number of all visits over time during the study interval of March 19 to August 31, 2020. The percentage of visits conducted in‐person or via telehealth were classified by gestational age ranges (<21 weeks, 22‐25 weeks, 26‐29 weeks, 30‐34 weeks, 35‐36 weeks, and >37 weeks) and for postpartum visits conducted between 1 and 28 days postpartum (“2 week visit”) and 29 to 56 days postpartum (“6 week visit”) and was recorded to assess the changes in telehealth visit rates over time. These time intervals were chosen based off of the traditional prenatal and postpartum care model in the United States. 26
Implementation (the extent to which the intervention was delivered as intended). Implementation outcomes included the acceptability and feasibility of this intervention. 22 Acceptability was measured indirectly by comparing the number of telehealth visits over time, as the decision to proceed with telehealth at subsequent appointments was jointly made by the provider and patient at the preceding appointment. Feasibility was measured by the total initial uptake of telehealth services in the initial months of this intervention (March and April), as telehealth was not a service previously offered in this clinical setting.
Maintenance (the sustainability of the intervention). Maintenance was measured as the total number of visits conducted via telehealth after the initial lockdown restrictions were eased at the end of June, and is displayed as the number of telehealth visits in the subsequent months of July and August.
2.3.1. Statistical analysis
Data were analyzed using descriptive statistics, including means and SDs for normally distributed continuous variables or medians and percentiles for skewed continuous variables. Frequencies and percentages were utilized for describing categorical data. Graphical approaches were also used to illustrate the RE‐AIM constructs. This study was determined to be exempt by the study site's Institutional Review Board (Reference: 1535131‐2). This manuscript was created within the framework of the SQUIRE (Standards for Quality Improvement Reporting Excellence) and StaRI (Standards for Reporting Implementation Studies) guidelines. 27 , 28
3. RESULTS
A total of 558 prenatal patients and 209 postpartum patients who had 2028 outpatient visits were analyzed. Across both types of patient visits, mean age was 27 years, 60.2% identified as Hispanic, with 13.2% having a primary language of Spanish. The majority of patients were insured through Massachusetts's Medicaid program (89.0%). Additional demographic information is presented in Table 1. Delivery data were available for 110 of the prenatal patients during the study interval (Table 2). The results below are organized by RE‐AIM construct and are summaritvely organized in Table 3.
TABLE 1.
Characteristics of patients with pre‐ and post‐natal encounters (March 19‐August 31, 2020)
| Prenatal | Postpartum | |
|---|---|---|
| (N = 558) | (N = 209) | |
| Age at first contact | ||
| Mean (SD) | 27.53 (5.84) | 27.19 (5.71) |
| Gravida | ||
| Median (iqi) | 3.0 (2.0, 4.0) | 2.0 (1.0, 4.0) |
| Parity | ||
| Median (iqi) | 1.0 (0.0, 2.0) | 1.0 (1.0, 2.0) |
| Nulliparous | ||
| Multiparous | 396 (71.0%) | 168 (80.4%) |
| Nulliparous | 162 (29.0%) | 41 (19.6%) |
| Plurality | ||
| Singleton | 548 (98.2%) | 206 (98.6%) |
| Twins | 10 (1.8%) | 3 (1.4%) |
| Primary language spoken | ||
| English | 457 (81.9%) | 179 (85.6%) |
| Spanish | 77 (13.8%) | 25 (12.0%) |
| Other | 24 (4.3%) | 5 (2.4%) |
| Race | ||
| Asian/Pacific Islander | 17 (3.0%) | 5 (2.4%) |
| Black/African‐American | 104 (18.6%) | 45 (21.5%) |
| Native American | 3 (0.5%) | 0 (0.0%) |
| White/Caucasian | 297 (53.2%) | 101 (48.3%) |
| Unknown | 137 (24.6%) | 58 (27.8%) |
| Hispanic | ||
| No | 223 (40.0%) | 78 (37.3%) |
| Yes | 334 (59.9%) | 131 (62.7%) |
| Missing | 1 (0.2%) | 0 (0.0%) |
| Insurance | ||
| ACO | 384 (68.8%) | 143 (68.4%) |
| MCO | 46 (8.2%) | 20 (9.6%) |
| MassHealth | 72 (12.9%) | 25 (12.0%) |
| Private | 44 (7.9%) | 17 (8.1%) |
| Other | 12 (2.2%) | 4 (1.9%) |
| Prenatal in‐person vs telehealth visits | ||
| All in‐person | 142 (25.4%) | ‐ |
| All telehealth | 64 (11.5%) | ‐ |
| Mixed | 352 (63.1%) | ‐ |
| Postpartum visit type | ||
| In‐person | ‐ | 108 (51.7%) |
| Telehealth | ‐ | 101 (48.3%) |
Abbreviations: ACO, accountable care organization; iqi, inter‐quartile range; MCO, Managed care organization.
TABLE 2.
Delivery outcomes by number of prenatal telehealth visits
| Total | All in‐person | At least one or more telehealth visit | |
|---|---|---|---|
| (N = 110) | (N = 35) | (N = 75) | |
| Induction of labor | |||
| No | 74 (67.3%) | 25 (71.4%) | 49 (65.3%) |
| Yes | 36 (32.7%) | 10 (28.6%) | 26 (34.7%) |
| Delivery type | |||
| Cesarean section | 31 (28.2%) | 8 (22.9%) | 23 (30.7%) |
| Vaginal delivery | 76 (69.1%) | 27 (77.1%) | 49 (65.3%) |
| Forcep/vacuum delivery | 3 (2.7%) | 0 (0.0%) | 3 (4.0%) |
TABLE 3.
Summary of results based on RE‐AIM indicators
| RE‐AIM indicator | Definition | Measure | Figure/Table |
|---|---|---|---|
| Reach | Proportion of a target population that participated in the intervention. | Demographic breakdown of population characteristics and proportion of telehealth visits compared to total visits before and after telehealth implementation. | Table 1 and Figure 2 |
| Effectiveness | Positive and negative outcomes of the intervention. | Comparison of various pregnancy related outcomes in relation to the number of telehealth visits. | Tables 2, 4 and 5 |
| Adoption | Measure of the temporal pattern of implementation. | Percentage of telehealth visits compared to total visits on a month by month basis throughout the study time period. | Figure 1 |
| Implementation (acceptability and feasibility) | Extent to which the intervention was delivered as intended. | Indirect comparison of the number of telehealth visits over time, with particular focus on interval after state‐mandated lockdowns began to lift (May and June), as well as the breakdown of percentage of telehealth vs total visits broken down by gestational age to assess for whether certain prenatal/postpartum care intervals are more acceptable than others. | Figures 1 and 2 |
| Total initial uptake of telehealth services in the initial 2 months of the intervention (March and April). | Figure 2 | ||
| Maintenance | Sustainability of the intervention. | Comparison of the number of telehealth visits over time compared with the number of total visits. | Figure 2 |
Reach: Of the 2018 total prenatal and postpartum visits conducted during the study period, 698 visits (34.4%) were conducted via telehealth. Looking exclusively at postpartum visits, of the 209 patients who had at least one postpartum visit, 108 (51.7%) had only in‐person visits and 101 (48.3%) had at least one telehealth visit. A total of 8 of the 698 total telehealth visits occurred via video conferencing, with the remaining occurring via telephone only.
Effectiveness: The overall prevalence of hypertensive disease of pregnancy and gestational diabetes mellitus and estimated gestational age at the time of diagnosis by visit type is shown in Table 4. Findings included that the median gestational age at diagnosis of hypertensive disease of pregnancy was approximately 37 weeks among those receiving prenatal care via any amount of telehealth, compared with approximately 35 weeks among those receiving all of their prenatal care in person. The median gestational age at diagnosis of GDM was 31.8 weeks among patients who had only in‐person prenatal visits (n = 142), compared with 27.7 weeks among patients who had at least one telehealth prenatal visit (n = 416). Of note, blood pressure and blood sugar monitoring occurred via patient self‐report during telephone visits of self‐collected and recorded values.
TABLE 4.
Hypertension and gestational diabetes (GDM) diagnoses among prenatal patients by number of prenatal telehealth visits
| Total | All in‐person | At least one or more telehealth visit | |
|---|---|---|---|
| (N = 558) | (N = 142) | (N = 416) | |
| Hypertension | |||
| Number of patients diagnosed | 22 (3.9%) | 8 (5.6%) | 14 (3.4%) |
| Median EGA at diagnosis (range a ) | 36.8 (9.1, 39.6) | 35.3 (15.7, 39.6) | 37.0 (9.1, 39.3) |
| GDM | |||
| Patients diagnosed | 56 (10.0%) | 14 (9.9%) | 42 (10.1%) |
| Median EGA at diagnosis (range a ) | 29.6 (9.0, 37.1) | 31.8 (11.7, 36.1) | 27.6 (9.0, 37.1) |
Abbreviation: EGA, estimated gestational age.
Range = minimum, maximum.
In terms of intrapartum results, of the patients who had all in‐person prenatal visits, 28.6% (10/35) underwent induction of labor compared with 34.7% (26/75) who had at least one in‐person telehealth visit (Table 2). Vaginal delivery rates of 77.1% (27/35) among patients who had all in‐person prenatal visits compared with 66.7% (49/75) among patients who had at least one telehealth prenatal visit.
Finally, postpartum outcomes by visit type are shown in Table 5. These outcomes included patient‐reported contraception use at 6 weeks postpartum of 79.6% among patients who presented for only in‐person postpartum care visits compared with 71.3% of patients who had at least one postpartum telehealth visit. Patient‐reported breastfeeding rates at 6 weeks were 25.0% and 23.8%, respectively, and OB triage/ED visit rates were 13.0% compared with 7.9%.
TABLE 5.
Postpartum outcomes by in‐person vs telehealth postpartum visit
| Total | In‐person | Telehealth | |
|---|---|---|---|
| (N = 209) | (N = 108) | (N = 101) | |
| Hypertensive disease of pregnancy | |||
| Yes | 28 (13.4%) | 17 (15.7%) | 11 (10.9%) |
| No, never met criteria | 146 (69.9%) | 66 (61.1%) | 80 (79.2%) |
| No, was diagnosed before delivery | 35 (16.7%) | 25 (23.1%) | 10 (9.9%) |
| Days from delivery to HTN Diagnosis weeks (n = 28) | |||
| Median (range a ) | 1 (0, 6) | 1 (0, 6) | 1 (0, 2) |
| At risk for postpartum depression | |||
| EPDS < 10 | 105 (50.2%) | 77 (71.3%) | 28 (27.7%) |
| EPDS ≥ 10 | 27 (12.9%) | 20 (18.5%) | 7 (6.9%) |
| Missing | 77 (36.8%) | 11 (10.2%) | 66 (65.3%) |
| Any contraception prescribed | |||
| No | 51 (24.4%) | 22 (20.4%) | 29 (28.7%) |
| Yes | 158 (75.6%) | 86 (79.6%) | 72 (71.3%) |
| LARC placed (n = 157) | |||
| In‐hospital placement | 38 (24.1%) | 26 (30.2%) | 12 (16.7%) |
| Postpartum placement | 28 (17.7%) | 19 (22.1%) | 9 (12.5%) |
| Neither | 92 (58.2%) | 41 (47.7%) | 51 (70.8%) |
| Breastfeeding (at 6 weeks postpartum) | |||
| No | 158 (75.6%) | 81 (75.0%) | 77 (76.2%) |
| Yes | 51 (24.4%) | 27 (25.0%) | 24 (23.8%) |
| 2 h GTT (by 8 weeks postpartum) | |||
| Not indicated | 189 (90.4%) | 96 (88.9%) | 93 (92.1%) |
| No | 18 (8.6%) | 10 (9.3%) | 8 (7.9%) |
| Yes | 2 (1.0%) | 2 (1.8%) | 0 (0.0%) |
| Any triage/ED visits | |||
| No | 187 (89.5%) | 94 (87.0%) | 93 (92.1%) |
| Yes | 22 (10.5%) | 14 (13.0%) | 8 (7.9%) |
Abbreviations: ED, emergency department; GTT, glucose tolerance test; HTN, hypertension; LARC, long‐acting removable contraception; PEDS, Edinburg postnatal depression scale.
Range = minimum, maximum.
Adoption: Every provider (n = 30) participated in the initiation and maintenance of the provision of telehealth services as evidenced by universal uptake and continued participation throughout the study period of all providers in the provision of telehealth services. Additionally, Figure 1 shows the percentage of telehealth and in‐person visits by gestational age range. Two time points were noted to have a larger percentage of in‐person visits: 26 + 0 weeks to 29 + 6 weeks and 35 + 0 weeks to 36 + 6 weeks. The majority of visits between 22 + 0 and 25 + 6 weeks were via telehealth (57.6%), and the majority of visits were in‐person for all other gestational age windows. Approximately half of the patients had a telehealth postpartum visit regardless of the number of postpartum visits they had: one visit at approximately 2 weeks postpartum (53.3%), one visit at approximately 6 weeks postpartum (47.4%), or visits at both 2 and 6 weeks postpartum (52.4%). The extremely low rate of uptake of video telehealth visits was cited by providers to be due to difficulty in both providers and patients use of technology.
FIGURE 1.
Number and percentage of telehealth visit vs in‐person visit by estimated weeks of gestation and number of weeks postpartum
Implementation: Figure 2 reveals an increase in telehealth visits over time from a baseline of zero telehealth visits to 83 telehealth visits over the last 12 days of March (37% of all visits during those 12 days in March). By the end of the study period, over 300 prenatal and postpartum telehealth visits were conducted in August (69% of all visits during August) entry, resulting in subsequent near‐universal documentation of EPDS scores in the EHR.
FIGURE 2.
Total number of telehealth and in‐person prenatal outpatient appointments during the pandemic in 2020
Finally, as demonstrated in Figure 1, visits surrounding 28 weeks gestational age and 36 weeks gestational had lower proportions of telehealth visits (30.1% and 23.2% respectively) compared with other visit intervals that had higher proportions of telehealth usage ranging from 34.3% to 57.4%.
An unintended benefit of evaluating the implementation of telehealth visits in the study clinic was that it detected issues with quality of care that existed before the pandemic, including failure to provide all routine prenatal and postpartum care as intended. In the course of evaluating the implementation of telehealth in this obstetric clinic, we discovered that 36.8% of patients did not have EPDS screening documented at their postpartum visits. A root cause analysis revealed a high number of missing EPDS screens and determined that providers believed the medical assistants (MA) were entering this information into the EHR, and the MAs believed the providers were entering this information into the EHR. Adjustments were made to the MAs' workflow to standardize EPDS data.
Maintenance: Figure 2 reveals total in‐person and telehealth visit data from the months of July and August. The targeted intervention ended in June when the state of Massachusetts began to ease restrictions, and more healthcare services began to be regularly offered in person. The continued use of telehealth visits at this point is demonstrated by the even higher number of telehealth visits, 245 visits in July (46% of all visits) and 305 visits in August (69% of all visits).
4. DISCUSSION
The results of this study suggest that rapid implementation of telehealth visits for prenatal and postpartum care is feasible in a practice serving a low‐income, majority Latina/x patient population using a parsimonious set of implementation strategies. Measures of reach and adoption suggested that uptake of the telehealth intervention was ubiquitous in the practice and that implementation outcomes measured suggested acceptability and appropriateness of telehealth. Although additional evaluation post‐COVID will generate a better understanding of maintenance of telehealth, this study's early assessment suggested that maintenance of telehealth visits once in‐person restrictions were lifted in the state of Massachusetts was achievable and feasible.
The risks associated with in‐person medical encounters due to the COVID‐19 pandemic created an unprecedented need for rapid innovations in healthcare delivery. The pandemic has also illuminated and exacerbated long‐standing racial and ethnic inequities in health and healthcare. 7 Telehealth was technologically feasible before the pandemic, but the need to limit in‐person visits coupled with state‐level policies to facilitate rapid adoption of telehealth led to a dramatic increase in remote healthcare delivery. Many medical specialties responded to the COVID‐19 pandemic by implementing telehealth services. 12 , 29 Given widespread patient dissatisfaction with the pre‐pandemic model of prenatal care and recent calls by the American College of Obstetric and Gynecology to make substantial changes in the postpartum care, it may not be surprising that there has been rapid adoption of telehealth in obstetric care during the pandemic. 29 , 30 , 31 , 32 , 33 , 34 , 35 The evidence base for the effectiveness and safety of telehealth in obstetric and gynecologic care made it suitable for widespread dissemination and implementation. 4 This study addressed a gap in knowledge by using the RE‐AIM framework to evaluate the real‐world rapid implementation of telehealth services in a large, academic, urban obstetric practice that serves a lower‐income, racial/ethnic minority population.
The very low rate of video compared with telephone encounters for telehealth visits suggests that feasibility, accessibility, and acceptability may differ based on the mode of telehealth delivery. Although no formal evaluation of these visits was performed, staff and patients reported difficulty with setting up the video platform, which resulted in a rapid abandonment of this service. A growing body of literature suggests that telehealth may actually increase socioeconomic and racial/ethnic inequities in healthcare because of the greater difficulty affording devices needed to easily participate in video telehealth and higher likelihood of living in neighborhoods with lower accessibility to high‐quality internet services. 7 , 9 More studies are needed to assess and address user issues including accessibility of WiFi internet, confidence using digital technology, sufficient cellular plans, privacy, and interest in video visits.
The payment parity of telehealth services in Massachusetts during the pandemic also may have played a role in the ability to rapidly implement these services. 17 Not only did the executive order issued by Governor Baker require payment parity reimbursement for in‐person and telehealth visits during the pandemic, but it also mandated private insurers reimburse at least at Medicaid levels for telehealth services. This order had significant impact on large, urban, academic centers, such as the center in which the current study took place, whose patient population is insured almost exclusively by Medicaid for prenatal and postpartum care. This order likely played an important role in limiting, but not fully preventing, the exacerbation of healthcare disparities that a vulnerable patient population already faces and that might otherwise have been worsened by the pandemic. As previous studies have demonstrated, it is doubtful that statewide policy changes alone cannot remove all of the barriers to telehealth use in general, and specifically in diverse populations. 36 , 37
This study's strengths include the large number of patients (n = 662), assessment of both pre‐ and postpartum visits, use of detailed EHR data, and its location in a state with one of the earliest major surges of COVID‐19 cases in the country. Although the study might have benefitted from an experimental design that allowed for causal inference, it was not possible to do so given the rapid changes required by the pandemic surge and the stresses on the healthcare system at the time. Alternatively, this study illuminates implementation considerations in a real‐world scenario. Future studies that compare implementation strategies in different contexts will extend the knowledge gained from the current study. Estimations of cost or cost‐effectiveness will also be important to understand the long‐term financial impacts of telehealth and its sustainability should it continue to be used for a substantial portion of prenatal and postpartum visits. The study was undertaken at a time when healthcare organizations were in crisis in Massachusetts and healthcare providers under substantial duress, so formal surveys and interviews to evaluate provider satisfaction with telehealth were not conducted but would be important in future studies as well. Previous studies have reported high patient satisfaction with the implementation of telehealth in obstetrics care, and given that patients had an active role in deciding whether or not they had in‐person or telehealth visits in this study, suggesting that patient satisfaction might be high in this setting. 38 , 39 Ascertaining patient's satisfaction with prenatal and postpartum telehealth visits among patients in lower‐income and racial/ethnic populations will be important to be able to tailor services to patient's needs and better understand the role telehealth may play in addressing inequities in maternal health care and disparities in maternal morbidity and mortality.
The RE‐AIM framework was a useful tool for evaluating implementation and maintenance of a new intervention in the obstetric setting and should be considered for use more broadly. Given the extensive racial and ethnic disparities in obstetric care, future studies should determine the impact of telehealth specifically on these disparities. If telehealth is found to be associated with improved outcomes, it will also be important to identify the mechanisms through which it has an effect, such as removal of barriers to access to care (ie, transportation) or increased self‐efficacy through joint decision‐making.
While prior studies have addressed the initial implementation phases of telehealth during the COVID‐19 pandemic, 40 we employed a theory‐based framework to evaluate the implementation and maintenance of telehealth in response to COVID‐19 associated policy changes in an urban, low‐income, majority Latina/x patient population. This study focused primarily on practice and organizational factors influencing implementation and maintenance, but the effect of public policy on equity in obstetric care should continue to be evaluated in systematic ways to better inform policymakers and healthcare providers about policy impacts.
CONFLICT OF INTEREST
The authors declare that they have no competing interests.
AUTHOR CONTRIBUTIONS
Data Entry: Keith Reisinger‐Kindle, Shiva Niakan, Colby Cayton
Data Analysis/Table Creation: Alexander Knee
Senior Oversight: Neena Qasba, Sarah L. Goff
All authors read and approved the final manuscript. All authors contributed to manuscript writing. All authors have read and approved the final version of the manuscript.
Keith Reisinger‐Kindle had full access to all of the data in this study and takes complete responsibility for the integrity of the data and the accuracy of the data analysis.
TRANSPARENCY STATEMENT
The primary author affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
ETHICS APPROVAL AND CONSENT TO PARTICIPATE
University of Massachusetts‐Baystate's IRB granted an exemption after review of the protocol. Reference: 1535131‐2.
ACKNOWLEDGMENT
Not applicable.
Reisinger‐Kindle K, Qasba N, Cayton C, Niakan S, Knee A, Goff SL. Evaluation of rapid telehealth implementation for prenatal and postpartum care visits during the COVID‐19 pandemic in an academic clinic in Springfield, Massachusetts, United States of America. Health Sci Rep. 2021;4:e455. doi: 10.1002/hsr2.455
Funding information None of the manuscript authors have received commercial support, or specific funding related to the development of this manuscript. The project utilized a REDCap database funded by grant UL1TR002544, an Institutional Clinical and Translational Service Award (CTSA) from the NIH awarded to Tufts University, School of Medicine.
DATA AVAILABILITY STATEMENT
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
REFERENCES
- 1. Turrentine M, Ramirez M, Monga M, et al. Rapid deployment of a drive‐through prenatal care model in response to the coronavirus disease 2019 (COVID‐19) pandemic. Obstet Gynecol. 2020;136(1):29‐32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Duryea EL, Adhikari EH, Ambia A, Spong C, McIntire D, Nelson DB. Comparison between in‐person and audio‐only virtual prenatal visits and perinatal outcomes. JAMA Netw Open. 2021;4(4):e215854. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. van den Heuvel JF, Groenhof TK, Verbeek JH, et al. eHealth as the next‐generation perinatal care: an overview of the literature. J Med Internet Res. 2018;20(6):e202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. DeNicola N, Grossman D, Marko K, Sonalkar S, Butler Tobah YS, Ganju N, Witkop CT, Henderson JT, Butler JL, Lowery C Telehealth interventions to improve obstetric and gynecologic health outcomes, Obstet Gynecol: 2020;(135)2:371–382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Abraham C. Rethinking the traditional prenatal care model. Obstet Gynecol. 2020;135(5):1024‐1026. [DOI] [PubMed] [Google Scholar]
- 6. Peahl A, Smith R, Moniz M. Prenatal care redesign: creating flexible maternity care models through virtual care. Am J Obstet Gynecol. 2020;223(3):389.e1‐389.e10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Pierce R, Stevermer J. Disparities in use of telehealth at the onset of the COVID‐19 public health emergency. J Telemed Telecare. 2020;1357633X2096389. 10.1177/1357633X20963893. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Goenka A, Ma D, Teckie S, et al. Implementation of Telehealth in radiation oncology: rapid integration during COVID‐19 and its future role in our practice. Adv Radiat Oncol. 2021;6(1):100575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Weber E, Miller S, Astha V, et al. Characteristics of telehealth users in NYC for COVID‐related care during the coronavirus pandemic. J Am Med Inform Assoc. 2020;27(12):1949‐1954. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Saiyed S, Nguyen A, Singh R. Physician perspective and key satisfaction indicators with rapid Telehealth adoption during the coronavirus disease 2019 pandemic. Telemed e‐Health. 2021;27:1225‐1234. [DOI] [PubMed] [Google Scholar]
- 11. Onwuzurike C, Meadows A, Nour N. Examining inequities associated with changes in obstetric and gynecologic care delivery during the coronavirus disease 2019 (COVID‐19) pandemic. Obstet Gynecol. 2020;136(1):37‐41. [DOI] [PubMed] [Google Scholar]
- 12. Ebbert J, Ramar P, Tulledge‐Scheitel S, et al. Patient preferences for telehealth services in a large multispecialty practice. J Telemed Telecare. 2021;1357633X2098030. 10.1177/1357633X20980302 [DOI] [PubMed] [Google Scholar]
- 13. Limaye M, Lantigua‐Martinez M, Trostle M, et al. Differential uptake of Telehealth for prenatal Care in a Large New York City Academic Obstetrical Practice during the COVID‐19 pandemic. Am J Perinatol. 2020;38(03):304‐306. [DOI] [PubMed] [Google Scholar]
- 14. First case of 2019 Novel Coronavirus confirmed in Boston. Boston.gov; 2021. https://www.boston.gov/news/first-case-2019-novel-coronavirus-confirmed-boston
- 15. Coronavirus Disease (COVID‐19) timeline. Bostongov; 2021. https://www.boston.gov/departments/public-health-commission/coronavirus-timeline
- 16. MassHealth: Coronavirus Disease 2019 (COVID‐19)—Providers. Massgov; 2021. https://www.mass.gov/info-details/masshealth-coronavirus-disease-2019-covid-19-providers
- 17. Faget K. Telehealth (Massachusetts): COVID‐19 inspires relaxed telemedicine technology requirements and mandates payment parity; 2021. https://www.natlawreview.com/article/telehealth-massachusetts-covid-19-inspires-relaxed-telemedicine-technology
- 18. Miner H, Koenig K, Bozic KJ. Value‐based healthcare: not going anywhere—why Orthopaedic surgeons will continue using Telehealth in a post‐COVID‐19 world. Clin Orthop Relat Res. 2020;478(12):2717‐2719. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Rosner BI, Morozov M, Andoni A. Digital health, Telehealth, and primary care post‐COVID: a discussion with Kim Boyd and Joe Kvedar, Moderated by Benjamin Rosner. Digit Biomark. 2020;4(3):123‐127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Glasgow R, Vogt T, Boles S. Evaluating the public health impact of health promotion interventions: the RE‐AIM framework. Am J Public Health. 1999;89(9):1322‐1327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Gaglio B, Shoup J, Glasgow R. The RE‐AIM framework: a systematic review of use over time. Am J Public Health. 2013;103(6):e38‐e46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2010;38(2):65‐76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377‐381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Harris P, Taylor R, Minor B, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. The project utilized a REDCap database funded by grant UL1TR002544, an Institutional Clinical and Translational Service Award (CTSA) from the NIH awarded to Tufts University, School of Medicine.
- 26. Landon M, ed. Gabbe's Obstetrics Essentials: Normal and Problem Pregnancies. Philadelphia, PA: Elsevier; 2019. [Google Scholar]
- 27. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (standards for quality improvement reporting excellence): revised publication guidelines from a detailed consensus process: table 1. BMJ Qual Saf. 2015;25(12):986‐992. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Pinnock H, Barwick M, Carpenter C, et al. Standards for reporting implementation studies (StaRI) statement. BMJ. 2017;356:i6795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Alexander G, Tajanlangit M, Heyward J, et al. Use and content of primary care office‐based vs telemedicine care visits during the COVID‐19 pandemic in the US. JAMA Netw Open. 2020;3(10):e2021476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Peahl A, Novara A, Heisler M, et al. Patient preferences for prenatal and postpartum care delivery. Obstet Gynecol. 2020;135(5):1038‐1046. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Peahl A, Gourevitch R, Luo E, et al. Right‐sizing prenatal care to meet Patients' needs and improve maternity care value. Obstet Gynecol. 2020;135(5):1027‐1037. [DOI] [PubMed] [Google Scholar]
- 32. Geissler K, Ranchoff B, Cooper M, Attanasio L. Association of Insurance Status with Provision of recommended services during comprehensive postpartum visits. JAMA Netw Open. 2020;3(11):e2025095. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Aziz A, Zork N, Aubey J, et al. Telehealth for high‐risk pregnancies in the setting of the COVID‐19 pandemic. Am J Perinatol. 2020;37(08):800‐808. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Duzyj C, Thornburg L, Han C. Practice modification for pandemics. Obstet Gynecol. 2020;136(2):237‐251. [DOI] [PubMed] [Google Scholar]
- 35. Jeganathan S, Prasannan L, Blitz M, et al. Adherence and acceptability of telehealth appointments for high‐risk obstetrical patients during the coronavirus disease 2019 pandemic. Am J Obstet Gynecol MFM. 2020;2(4):100233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Park J, Erikson C, Han X, Iyer P. Are state Telehealth policies associated with the use of Telehealth services among underserved populations? Health Aff. 2018;37(12):2060‐2068. [DOI] [PubMed] [Google Scholar]
- 37. Hong Y‐R, Lawrence J, Williams D Jr, Mainous A III. Population‐level interest and Telehealth capacity of US hospitals in response to COVID‐19: cross‐sectional analysis of Google search and National Hospital Survey Data. JMIR Public Health Surveill. 2020;6(2):e18961. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Pflugeisen B, McCarren C, Poore S, et al. Virtual visits: managing prenatal care with modern technology. MCN Am J Matern Child Nurs. 2016;41(1):24‐30. [DOI] [PubMed] [Google Scholar]
- 39. Pflugeisen B, Mou J. Patient satisfaction with virtual obstetric care. Matern Child Health J. 2017;21(7):1544‐1551. [DOI] [PubMed] [Google Scholar]
- 40. Spiegelman J, Krenitsky N, Syeda S, Sutton D, Moroz L. Rapid development and implementation of a COVID‐19 telehealth clinic for obstetric patients. NEJM Catal Innov Care Deliv. 2020. https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0170 [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.