Abstract
Background
Nursing homes were ill-equipped for the pandemic; though facilities are required to have infection control staff, only 3% have taken a basic infection control course. Little is known about the implementation of effective practices outside of the acute care setting. We proposed an intervention utilizing Project ECHO, to connect Penn State University experts with nursing home staff and administrators to explore how infection control guidelines can be implemented effectively.
Methods
A stratified cluster randomized design was used to assign nursing homes to either AHRQ-funded COVID-19 ECHO or AHRQ-funded COVID-19 ECHO+.
Results
136 nursing homes participated. There were no significant differences in COVID-19 infection rate, hospitalization, deaths, or influenza, between ECHO or ECHO+.
Discussion
The ECHO model has significant strengths when compared to traditional training, as it allows for remote learning delivered by a multidisciplinary team of experts and utilizes case discussions that match the context of nursing homes.
Keywords: COVID-19, Nursing homes, Public health, Project ECHO
Introduction
A landmark report by the Institute of Medicine identified a 17-year translational gap in research, that is, the lag between scientific discovery and implementation of those discoveries into practice.1 Further, the report found that only 30% of emerging interventions jump this gap.1 Although there are numerous reasons for these delays in, or complete absence of, effective intervention implementation, a major reason is the lack of knowledge of such discoveries by frontline providers. The COVID-19 pandemic presented a unique opportunity to jump the translational gap given the worldwide focus on the increased pace of discovery and the necessity for translation of effective interventions in order to save lives.2 , 3
There were few locations of greater need for support early in the COVID-19 pandemic than the nursing home setting. As a result of their environment and host risk factors, nursing home residents accounted for nearly 4 of 10 COVID-19 deaths early in the pandemic.4 Although infectious outbreaks occur in these settings each year, COVID-19 presented a unique challenge given its newness and resultant uncertainty in the guidance in the early days of the pandemic. Although the Centers for Medicare and Medicaid Services (CMS) requires nursing homes to have an infection preventionist, the quality of infection control training and ability to administer infection control policies in the nursing home environment is highly variable.5 Further, barriers to infection mitigation strategies exist in addition to a lack of formal training, including challenges related to managing resident transfers, transmission prevention, and information overload.5 , 6 These challenges, coupled without an implementation science lens to understand how to effectively implement evidence-based infection control strategies in the nursing home setting, further heightened the difficulty in managing the COVID-19 pandemic.
The effective implementation of evidence-based infection control strategies in nursing homes is limited. While the Centers for Disease Control and Prevention (CDC) published guidance to assist nursing homes in addressing the pandemic,7 effective implementation requires organizational capacity, staff engagement, and problem-solving.8 We identified Project ECHO (Extensions for Community Health Outcomes), an evidence-based telehealth model,9, 10, 11 as one pathway to overcoming this translational gap. We tested an intervention utilizing Project ECHO to virtually connect academic medicine experts with nursing home staff and administrators to proactively support evidence-based infection control guideline implementation. We hypothesize that nursing homes in the ECHO+ intervention of the COVID-19 Project ECHO program will have fewer COVID-19 infections, hospitalizations, flu-like illness, and deaths, and improved quality of life (QoL) outcomes in comparison to the ECHO intervention. Engagement on multiple levels in the study's planning, design, and implementation of our diverse 16-member Stakeholder Advisory Board strengthened the patient-centeredness of the study.
Methods
Design: A stratified cluster randomized design was employed to deliver the Project ECHO intervention. Using a 1:1 ratio, we randomly assigned 136 nursing homes (with approximately 16,700 residents) to ECHO or ECHO+. Randomization was stratified by geographic location (rural vs. urban), baseline COVID-19 infection rate (some vs. none), and facility capacity (<60 beds vs. ≥60 beds). Patient-centered outcomes, including nursing home residents with COVID-19 infections, flu-like illness, COVID-19 hospitalizations, deaths, and quality of life (QoL), were assessed at baseline (intervention start date), and at 4, 6, 12, and 18 months. Our study was guided by the RE-AIM framework to critically evaluate both the effectiveness and implementation outcomes of the proposed cluster RCT. The comprehensive study protocol has been previously published.6 An overview of the study methods are outlined below.
Using the ECHO model, our primary aim was to compare the effectiveness of a 16-week AHRQ phase 1 COVID-19 Project ECHO intervention (ECHO) followed by nine weekly, optional 60 min office hour sessions (AHRQ phase 2) to ECHO plus 17 additional sessions, 60 min in length, including an 8-week flu-focused refresher series for fall 2021, focused on infection control (ECHO+) in reducing the number of nursing home residents with COVID-19. Our secondary aim was to compare the effectiveness of ECHO versus ECHO+ on other patient-centered outcomes, including QoL, flu-like symptoms, hospitalizations, and deaths.
Sample: National nursing home lists to assist recruitment efforts were obtained using Centers for Medicare & Medicaid Services (CMS) data, state agency and nursing home association contact websites, and our engaged stakeholders. Facility eligibility criteria for participation included U.S.-based, CMS-eligible skilled nursing facilities with access to a computer or electronic device for intervention participation. Facilities were ineligible if they had previously participated in a Project ECHO-led COVID-19 series. Approval for this study has been obtained from the Penn State Institutional Review Board at the Pennsylvania State University (STUDY00015883). All participants received information about the study and were asked to give consent before participating in the study. Recruitment began in the beginning of December 2020 and was extended until the end of January 2021. To support the interactive nature of the ECHO model, participants were assigned to cohorts, which allowed for smaller group discussions.
Procedure: The intervention for this study included the Agency for Healthcare Research and Quality (AHRQ) ECHO National Nursing Home COVID-19 Action Network (hereinafter, “the Network”), supported by AHRQ and in collaboration with Project ECHO at the University of New Mexico Health Sciences Center and the Institute for Healthcare Improvement (IHI).12 This network provided training and mentorship to nursing homes across the country to increase the implementation of evidence-based infection prevention and safety practices to protect residents and staff. Using the Project ECHO model of tele-mentoring, all nursing homes received the intervention in two sequential phases (Table 1 , Fig. 1 ).
Table 1.
Summary of comparators.
| Phase 1 | ECHO | ECHO+ |
|---|---|---|
| 16-week infection control ECHO | ✓ | ✓ |
| Quality Improvement component | ✓ | ✓ |
| Phase 2 | ||
| 9-week office hours | ✓ (Optional) | |
| 9-week ECHO | ✓ | |
| 8-week refresher series (Fall 2021) | ✓ | |
Fig. 1.
Summary of comparators.
Phase 1: Nursing homes in both study arms received a 16-week Network curriculum via real-time, interactive videoconferencing using Zoom at no cost to participants. The curriculum was developed specifically for this intervention in partnership between AHRQ, the University of New Mexico's ECHO Institute, and the IHI. Session recordings were available for those who were unable to participate live. Phase 1 sessions were up to 90 min in duration and held weekly for 4 months (16 sessions total) at regularly scheduled times. All sessions followed the required program format, which is standard for the ECHO model, including introductions (5 min), didactic presentations (10-15 min), case presentations (30 min), question and answer period (30 min), and close and debrief (5 min). Typically, each session included case-based discussions (1-2 cases/session) to ensure mastery of the content and skills. Participants were encouraged to ask clarifying questions and weigh in on recommendations, followed by ECHO experts who provided advice on addressing each case using best practices. Recommendations were summarized verbally during the session and distributed via email following.
Phase 2: The ECHO group was offered an optional nine weekly 60 min office hours, in which participants could drop in on an as-needed basis to ask specific questions and receive guidance from our experts on a variety of topics. The ECHO+ group received an additional 9 weeks of live 60 min ECHO sessions, following the format described for Phase 1 and covering emerging topics developed by the research team specifically for this intervention. These topics were identified as timely and important by our stakeholders, subject matter experts, and feedback from participating nursing homes. If nursing home staff were unable to attend the session live, they were offered the recording of that session. Further, ECHO+ facilities received an additional 8-session refresher series running from September to November 2021, providing an opportunity to further cover topics that were part of the CDC infection control training and prioritized by our stakeholders and nursing home participants. The ECHO group was offered an optional nine weekly 60 min office hours, in which participants could drop in on an as-needed basis to ask specific questions and receive guidance from our experts on a variety of topics.
Measures: The primary outcome, COVID-19 infection rate reduction in nursing homes and secondary outcomes were obtained using the Nursing Home COVID-19 Public File.13 Specifically, the variables assessed included the number of weekly and total resident admissions, the number of weekly and total resident COVID-19 deaths, the number of residents with new influenza, and the number of residents with acute respiratory illness symptoms excluding COVID-19 and/or influenza. Qualitative interviews of ECHO participants were conducted at 6 and 12 months. These interviews will provide informative insights into the implementation science behind the COVID-19 Project ECHO program. Formal results from these interviews will be presented elsewhere, however, representative quotes are described in the discussion.
Data Analysis: We performed cluster-level analyses using the aggregated outcomes (e.g., infection, hospitalization) originating from the CMS weekly nursing home level data (confirmed COVID cases per 1000 residents, admissions per 1000 residents, deaths per 1000 residents, and influenza cases per 1000 residents). We found 4-week totals (incidence) for the weeks preceding each study time points coinciding with baseline, month 4, month 6, month 12, and month 18. Given that all of the outcomes were continuous but skewed in distribution, comparisons were made of all of these outcomes at each study time point between the two study groups using a Wilcoxon Rank Sum test.
Our primary outcome was the number of nursing home residents with COVID-19 infection as reported in the Nursing Home COVID-19 Public File, self-reported by each facility and listed by week ending (with a 2-week lag in upload). Data periods for this study are defined below (Table 2 ). To determine baseline, we took cumulative data for the 4-week period prior to the ECHO cohort start date. To determine the 4-month time point, we took cumulative data for the 4-week period that coincided with 4 months post ECHO cohort start date, the conclusion of Phase 1 (16 weeks). To determine the 6-month time point, we took cumulative data for the 4-week period that coincided with 6 months post ECHO cohort start date, the conclusion of Phase 2 (25 weeks).
Table 2.
Timeframe for data collection.
| Cohort | Cohort Start | Cohort End | Baseline | 4-months | 6-months | 12-months | 18-months |
|---|---|---|---|---|---|---|---|
| ECHO 1 (n = 21) | 12/23/2020 | 6/29/2021 | 11/29/2020-12/20/2020 | 5/02/2021-5/23/2021 | 7/04/2021-7/25/2021 | 12/05/2021-12/26/2021 | 6/05/2022-6/26/2022 |
| ECHO+ 2 (n = 20) | |||||||
| ECHO 3 (n = 21) | 1/05/2021 | 7/01/2021 | 12/13/2021-1/03/2021 | 5/02/2021-5/23/2021 | 7/04/2021-7/25/2021 | 12/05/2021-12/26/2021 | 6/05/2022-6/26/2022 |
| ECHO+ 4 (n = 25) | |||||||
| ECHO 5 (n = 26) | 2/10/2021 | 7/28/2021 | 1/17/2021-2/07/2021 | 5/30/2021-6/20/2021 | 8/01/2021-8/22/2021 | 12/05/2021-12/26/2021 | 6/05/2022-6/26/2022 |
| ECHO+ 6 (n = 23) |
To determine the 12-month time point, we took cumulative data for the 4-week period beginning 12 months post-baseline for our original cohorts. To determine the 18-month time point, we took cumulative data for the 4-week period beginning 18 months post-baseline for our original cohorts. To normalize data, variables that were not calculated per 1000 residents were divided by the total number of occupied beds in the facility, then multiplied by 1000. These are notated in the tables below.
Results
In total, 290 nursing homes expressed interest in our study and were assessed for eligibility. Of these, 154 facilities were excluded for reasons such as not meeting inclusion criteria and declining to participate. A total of 136 nursing homes were randomized, with 68 allocated to ECHO, and 68 allocated to ECHO+ (Fig. 2 ).
Fig. 2.
Participant flow diagram. The CONSORT diagram.
Following the intention-to-treat principle, a total of 136 facilities were included in the final analysis. Summary statistics and distributions for demographic and characteristic variables are included in Table 3 . Covariates included in this study were relatively evenly split between the study arms, with the exception of three. There was an approximate 10% difference in facility payment model (51.41% for-profit in ECHO vs. 61.76% for-profit in ECHO+, 47.06% not-for-profit in ECHO vs. 38.24% not-for-profit in ECHO+), facility type (51.47% independent in ECHO vs. 42.65% independent in ECHO+, 48.53% networked in ECHO vs. 57.35% networked in ECHO+), and memory care status (16.18% no in ECHO vs. 225.00% no in ECHO+, 83.82% yes in ECHO vs. 75.00% yes in ECHO+) observed between the ECHO and ECHO+ study arms. None of these differences was statistically significant.
Table 3.
Baseline characteristics of participating facilities.
| Characteristic | ECHO (n = 68) N (%) | ECHO+ (n = 68) N (%) | P-value* |
|---|---|---|---|
| Demographic (N, %) | |||
| Location | 0.862 | ||
| Urban | 57.35) | 40 (58.82) | |
| Rural | (42.65) | 28 (41.18) | |
| Baseline Infection Rate | 0.812 | ||
| Some | 85.29) | 57 (83.82) | |
| None | 14.70) | 11 (16.18) | |
| Facility Size | 0.850 | ||
| <60 beds | 20 (29.41) | 19 (27.94) | |
| >60 beds | 48 (70.59) | 49 (72.06) | |
| Facility Payment Model | 0.298 | ||
| For-profit | 36 (52.41) | 42 (61.76) | |
| Not-for-profit | 47.06) | 26 (38.24) | |
| Facility Type | 0.303 | ||
| Independent | 35 (51.47) | 29 (42.65) | |
| Networked | 33 (48.53) | 39 (57.35) | |
| Memory Care Status | 0.203 | ||
| Yes | 11 (16.18) | 17 (25.00) | |
| No | 57 (83.82) | 51 (75.00) |
Chi-square test
Generally, attendance across both Phase 1 and 2 of the COVID-19 Project Intervention decreased over time. Average facility attendance can be found in Table 4 . There was a stark decline in facilities attending Phase 2 sessions.
Table 4.
Average facility attendance in the COVID-19 Project ECHO Program.
| Cohort | Number of Facilities per Session in Phase 1 | Number of Facilities Requesting Recording | Number of Facilities per Session in Phase 2 |
|---|---|---|---|
| ECHO 1 | 12 | 3 | 2 |
| ECHO+ 2 | 12 | 3 | 5 |
| ECHO 3 | 10 | 1 | 1 |
| ECHO+ 4 | 11 | 1 | 2 |
| ECHO 5 | 9 | 1 | 1 |
| ECHO+ 6 | 15 | 1 | 7 |
Primary aim results can be found in Table 5 . COVID-19 infections decreased over time from baseline to 6 months then increased again at the 12 and 18-month timepoints. We did not complete a subgroup analysis as there were no observed differences across study arms, sample sizes would have been greatly reduced, and results would not have been informative. Missing data was minimal, due to the initial reporting requirements for nursing homes set by CMS.14
Table 5.
Cumulative 4-week incidence at study points for confirmed COVID-19 cases per 1000 residents
| Outcome | Time | ECHO (N = 68) |
ECHO+ (N = 68) |
p-value* | ||
|---|---|---|---|---|---|---|
| Mean + SD | Median (Q1, Q3) | Mean + SD | Median (Q1, Q3) | |||
| Total confirmed COVID-19 cases per 1,000 residents in 4 weeks | Baseline | 109.02 ± 197.99 | 24.45 (0.00, 118.91) | 134.69 ± 259.89 | 23.39 (0.00, 99.65) | 0.989 |
| 4 months | 2.95 ± 13.71 | 0.00 (0.00, 0.00) | 1.45 ± 6.43 | 0.00 (0.00, 0.00) | 0.736 | |
| 6 months | 0.75 ± 4.82 | 0.00 (0.00, 0.00) | 1.96 ± 16.17 | 0.00 (0.00, 0.00) | 0.326 | |
| 12 months | 24.78 ± 48.31 | 0.00 (0.00, 26.70) | 27.92 ± 56.65 | 0.00 (0.00, 26.20) | 0.611 | |
| 18 months | 28.37 ± 71.20 | 0.00 (0.00, 30.31) | 30.67 ± 64.75 | 0.00 (0.00, 38.46) | 0.380 | |
Wilcoxon Rank Sum Test
Hospitalization and COVID-19 deaths decreased over time from baseline to 6 months but then increased slightly at 12 months. Influenza rates remained low throughout the study period.
Infection rate, hospitalizations, and deaths (Table 6 ) were all lower at 18 months compared to baseline. There was a significant difference in the Total COVID-19 deaths per 1000 residents in 4 weeks observed, with deaths higher in the ECHO+ study arm. The intervention delivered between both groups was similar over the 6-month time point, and there were no observed differences at this time point. During September-November 2021 we delivered a fall refresher series with a focus on influenza and late-breaking COVID-19 topics with the ECHO+ group. At 12 months there was still no significant difference between ECHO and ECHO+ for infection rate, hospitalization, death, and influenza rate. Missing data was minimal, due to the initial reporting requirements for nursing homes set by CMS.14
Table 6.
Cumulative 4-week incidence at study points for total COVID-19 admissions, total COVID-19 deaths, and total confirmed new influenza per 1,000 residents in 4 weeks
| Outcome | Time | ECHO (N = 62) | ECHO+ (N = 63) | p-value* | ||
|---|---|---|---|---|---|---|
| Mean + SD | Median (Q1, Q3) | Mean + SD | Median (Q1, Q3) | |||
| Total COVID-19 admissions per 1,000 residents in 4 weeks | Baseline | 36.07 ± 80.68 | 0.00 (0.00, 34.19) | 40.48 ± 85.60 | 0.00 (0.00, 38.22) | 0.743 |
| 4 months | 4.30 ± 20.40 | 0.00 (0.00, 0.00) | 2.91 ± 13.60 | 0.00 (0.00, 0.00) | 0.553 | |
| 6 months | 4.72 ± 35.78 | 0.00 (0.00, 0.00) | 9.96 ± 56.87 | 0.00 (0.00, 0.00) | 0.253 | |
| 12 months | 7.40 ± 23.68 | 0.00 (0.00, 0.00) | 32.06 ± 167.94 | 0.00 (0.00, 0.00) | 0.907 | |
| 18 months | 13.37 ± 53.50 | 0.00 (0.00, 0.00) | 7.62 ± 21.52 | 0.00 (0.00, 0.00) | 0.685 | |
| Total COVID-19 deaths per 1,000 residents in 4 weeks | Baseline | 32.96 ± 68.36 | 0.00 (0.00, 40.23) | 28.44 ± 60.15 | 0.00 (0.00, 33.03) | 0.844 |
| 4 months | 0.00 ± 0.00 | 0.00 (0.00, 0.00) | 0.77 ± 4.26 | 0.00 (0.00, 0.00) | 0.825 | |
| 6 months | 0.27 ± 2.25 | 0.00 (0.00, 0.00) | 0.43 ± 2.53 | 0.00 (0.00, 0.00) | 0.578 | |
| 12 months | 3.38 ± 10.90 | 0.00 (0.00, 0.00) | 3.39 ± 12.16 | 0.00 (0.00, 0.00) | 0.642 | |
| 18 months | 0.17 ± 1.42 | 0.00 (0.00, 0.00) | 1.85 ± 7.01 | 0.00 (0.00, 0.00) | 0.050 | |
| Total confirmed new influenza per 1,000 residents in 4 weeks | Baseline | 0.00 ± 0.00 | 0.00 (0.00, 0.00) | 0.61 ± 5.05 | 0.00 (0.00, 0.00) | 0.325 |
| 4 months | 0.00 ± 0.00 | 0.00 (0.00, 0.00) | 0.00 ± 0.00 | 0.00 (0.00, 0.00) | 1.00 | |
| 6 months | 0.77 ± 4.93 | 0.00 (0.00, 0.00) | 0.49 ± 4.04 | 0.00 (0.00, 0.00) | 0.567 | |
| 12 months | 1.24 ± 7.65 | 0.00 (0.00, 0.00) | 0.33 ± 2.69 | 0.00 (0.00, 0.00) | 0.319 | |
| 18 months | 0.00 + 0.00 | 0.00 (0.00, 0.00) | 0.00 ± 0.00 | 0.00 (0.00, 0.00) | 1.00 | |
Wilcoxon Rank Sum Test
Discussion
Overall, there were no observed differences between the ECHO and ECHO+ study arms in the primary outcome: the total number of confirmed COVID-19 cases per 1,000 residents in four weeks across study time points. We found that the COVID-19 infection rate decreased from baseline to 6 months, but then increased again at 12 and 18 months with hospitalizations and deaths mirroring this trend. However, these findings were complicated by the evolving epidemiology of the pandemic and the rollout of COVID-19 vaccines. The CDC's Advisory Committee on Immunization Practices (ACIP) recommended those living in long-term care facilities, including nursing homes, be offered the COVID-19 vaccine first in the initial phases of the COVID-19 vaccination program, along with healthcare professionals and non-institutionalized older adults. Nursing home residents and staff began receiving the COVID-19 vaccine following this in collaboration with the Pharmacy Partnership for Long-Term Care Program in January 2021. This vaccination effort contributed to a dramatic and significant decline in morbidity and mortality from COVID-19 and likely explains declines in cases and hospitalizations seen at the 4 and 6 month ECHO timepoints.15 Concerns regarding the Omicron variant began in late 2021, with the first case of the Omicron variant detected in the U.S. in December 2021.16 Omicron and its subvariants are known to circumvent vaccine derived immunity. Therefore, the observed increase in cases at the 12 and 18-month study time points aligns appropriately with the subvariant surges and loss of vaccine protection against infection.
It is unsurprising that there were minimal observed differences in infection rate, hospitalizations, and deaths between the study arms through the study period as vaccination rates were similar between the two ECHO cohorts. There was only one study time point, at 18 months, that had a statistically significant difference in total COVID-19 deaths, with deaths observed to be higher in the ECHO+ group.
Rates of flu-like illness remained low throughout the entire study period, and there was no observed difference between the ECHO and ECHO+ study arms in the number of total confirmed cases of new influenza per 1,000 residents in four weeks. This is in line with national trends of influenza virus throughout the COVID-19 pandemic, in which influenza-like illness (ILI) rates were unusually low.17 This can be attributed to the widespread preventative measures taken across the U.S., including social distancing, mask-wearing, handwashing, reduced international travel, closure of schools, and more. In fact, when observing the COVID-19 timeline, one study conducted by the CDC found that “influenza virus circulation declined sharply within two weeks of the COVID-19 emergency declaration and widespread implementation of community mitigation strategies.”17 Further, COVID-19 risk perception is associated with a decrease in ILI,18 and being that those residing in nursing homes have one of the highest risks of adverse COVID-related outcomes, it would be appropriate to surmise that this population also had a significant reduction in ILI.
Attendance through the COVID-19 Project ECHO program was suboptimal. We observed a high number of facilities attending early sessions, especially those that occurred in Phase 1. These sessions were held between December 2020 and May 2021, which demonstrate curriculum delivery during a time of high need. This is in comparison to Phase 2 for both ECHO and ECHO+ cohorts, which was held through late Summer 2021. We hypothesize that the later sessions occurring during a time where facilities were more well-prepared to handle the COVID-19 pandemic, coupled with the COVID information overload observed throughout the pandemic, as defined by an excessive amount of information dissemination that surpasses one's information processing capacity, resulted in decreasing participation through the COVID-19 Project ECHO program. It is likely that decreasing participation in Phase 2 resulted in the lack of difference found between the ECHO and ECHO+ study arms, as this is where the key differentiators occurred in the COVID-19 Project ECHO program.
Throughout the COVID-19 pandemic, there was an almost insurmountable gap between rhetoric and reality in education for long-term care professionals, contributing to the translational gap.19 Although public health professionals consistently emphasized the importance of practicing evidence-based infection control strategies in nursing homes, there were limited interventions to actually provide long-term care professionals with the tools and knowledge for implementation. Information insufficiency is defined as the gap between what one needs to know about a given topic and their current knowledge, or what one actually knows.20 During the COVID-19 pandemic, information insufficiency occurred for various reasons, including confusing and unclear guidance and everchanging knowledge about the virus, which directly resulted in policy changes. Receiving regulatory guidance from local, state, and federal agencies was reported to be confusing for nursing home administrators and staff and often contradictory.21 Several studies on the lived experiences of nursing home staff highlight that challenges in effective and reliable communication were explicitly identified as a factor that hindered the implementation of an appropriate infection control response.22, 23, 24
The ECHO model has the ability to directly address this gap between communication and practice. The Project ECHO platform promptly positioned itself at the forefront of the COVID-19 crisis and was utilized in this project to ultimately be a resource for nursing home administrators and staff, and to increase knowledge, confidence, and self-efficacy in translating evidence into best-practice care. While the majority of study primary and secondary outcomes had null results, the impact the intervention had on participants spans beyond our proposed outcomes. The qualitative interviews conducted throughout the study period emphasized that the learning community created through the Project ECHO program created a space for facilities to confirm that they were implementing evidence-based practices. One representative quote from a participant stated, “[Project ECHO] was a good opportunity to confirm the steps we had taken were in line with best practice.” Further, another participant actually detailed how Project ECHO was a more preferable format to review information in, in which they stated, “I know some colleagues were basically taking [state guidelines] or directives and just giving them to staff. Well, something that's 12 pages long with a bunch of legal verbiage is not something that is going to be digested… ECHO was just more user-friendly for the staff and easier to implement.” Another participant described that “information that we would get from the Department of Health or our local health department, they seem to be all at odds with each other. The information we received from ECHO was consistent.”
In addition to its ability to provide educational support, Project ECHO provided socializing opportunities for participating staff that may have positive emotional benefits. Long-term care professionals at the frontlines of caring for our society's most vulnerable and frail population were grappling with the heightened responsibilities and professional workload introduced by COVID-19, resulting in additional work-related strain.25 The pandemic introduced a mental health crisis in the long-term care setting, with elevated levels of burnout among staff and administrators.26, 27, 28 The Project ECHO program was ultimately a support network for these facilities and their staff, directly impacting feelings of burnout and loneliness. One participant described that the program “made it feel like we weren't on an island by ourselves” and that “it made you feel like you weren't alone.”
We believe the study had several strengths, including generalizability. First, the broad inclusion criteria and representativeness of nursing home facilities across multiple states suggests the program may apply in multiple settings. Second, the national presence of the ECHO program, highlighted by the AHRQ program, provides an opportunity for intervention utilization broadly. Lastly, the use of nationally available data, both through CMS’ COVID-19 data and the MDS, provides an efficient data source for the analysis of wide-scale studies.
The major influential limitation of this study was the minimal differences between the study arms due to contractual requirements, which resulted in both ECHO and ECHO+ participants receiving similar interventions in phase 1. This was a result of the launch of the national AHRQ program shortly following study funding and a concern of offering a different comparator. Another issue outside the study's control was the unpredictable rise and fall of COVID-19 infection rates as a result of the course of the pandemic. These rates made it difficult to understand the intervention impact, if any. Further, the stress placed upon nursing home facilities during the pandemic limited their ability in many cases for active study involvement. Research requiring engagement remained a challenge due to facility workload, particularly related to additional burdens due to the pandemic, staff turnover, and staff burnout. Active engagement from our representative stakeholder group was helpful in developing strategies for participant engagement. The short study time period further limits our ability to understand significant differences in longer-term outcomes that may result. Additionally, we were unable to prohibit additional participation in external COVID-19-related interventions, and if this participation may have impacted our findings.
In conclusion, Project ECHO provides an innovative approach to addressing the gap between healthcare guidelines and implementation. Understanding how to operationalize best practices through interaction with subject matter experts and a peer network is a unique opportunity to address leading public health challenges, like the COVID-19 pandemic. The successful adaption of the ECHO intervention to the nursing home facility setting suggests opportunity for use in a variety of healthcare conditions and topics and may be a future pathway to overcoming the translational gap.
Funding
This work was partially supported by a Patient-Centered Outcomes Research Institute® (PCORI®) Award (COVID-2020C2-10728) and partially supported by the National Center for Advancing Translational Sciences, National Institutes of Health (NIH) [Grant numbers UL1 TR002014, UL1 TR00045]. The Project ECHO intervention used in this research was partially supported by Contract No. 75Q80120C00003 from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services (HHS). The content is solely the responsibility of the authors and does not necessarily represent the official views of Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee, AHRQ, HHS, or the NIH.
Declaration of Competing Interest
None
Acknowledgments
We would like to thank all of our collaborators, including patient partners, state, and community stakeholders, for their contributions to this manuscript and their unwavering support in our research. We would also like to acknowledge the Penn State Project ECHO team for their support in developing and delivering the education intervention for this study.
References
- 1.Institute of Medicine Committee on Quality of Health Care in America . National Academies Press; 2001. Crossing the Quality Chasm: a New Health System for the 21st Century. [PubMed] [Google Scholar]
- 2.Lurie N, Saville M, Hatchett R, Halton J. Developing Covid-19 vaccines at pandemic speed. N Engl J Med. 2020;382:1969–1973. doi: 10.1056/NEJMp2005630. [DOI] [PubMed] [Google Scholar]
- 3.U.S. Government Accountability Office. COVID-19: federal efforts accelerate vaccine and therapeutic development, but more transparency needed on emergency use authorizations. 2020. https://www.gao.gov/products/gao-21-207
- 4.Gorges RJ, Konetzka RT. Factors associated with racial differences in deaths among nursing home residents with COVID-19 infection in the US. JAMA Network Open. 2021;4(2) doi: 10.1001/jamanetworkopen.2020.37431. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Herzig CTA, Stone PW, Castle N, Pogorzelska-Maziarz M, Larson EL, Dick AW. Infection prevention and control programs in US nursing homes: results of a national survey. J Am Med Dir Assoc. 2016;17(1):85–88. doi: 10.1016/j.jamda.2015.10.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Calo WA, Francis E, Kong L, et al. Implementing infection control and quality of life best practices in nursing homes with project ECHO: protocol for a patient-centered randomized controlled trial. JMIR Res Protoc. 2022;11(5) doi: 10.2196/34480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Centers for Disease Control and Prevention. 2023 Nursing homes and assisted living (Long-term Care Facilities [LTCFs]). https://www.cdc.gov/longtermcare/index.html
- 8.Mattox T, Hunter SB, Kilburn R, Wiseman SH. 2023 Determine the capacities needed for implementation. RAND Corporation. https://www.rand.org/pubs/tools/TL114/manual/step5.html
- 9.Arora S, Kalishman S, Thornton K, et al. Expanding access to HCV treatment - extension for community healthcare outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology. 2010;52(3):1124–1133. doi: 10.1002/hep.23802. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The impact of project ECHO on participant and patient outcomes: a systematic review. Acad Med. 2016;91(10):1439–1461. doi: 10.1097/ACM.0000000000001328. [DOI] [PubMed] [Google Scholar]
- 11.Puckett HM, Bossaller JS, Sheets LR. The impact of project ECHO on physician preparedness to treat opioid use disorder: a systematic review. Addict Sci Clin Pract. 2021;16 doi: 10.1186/s13722-021-00215-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Agency for Heathcare Research and Quality. Project ECHO. https://www.ahrq.gov/patient-safety/settings/multiple/project-echo/index.html
- 13.Centers for Medicare & Medicaid Services. COVID-19 Nursing Home Data. https://data.cms.gov/covid-19/covid-19-nursing-home-data [PubMed]
- 14.Centers for Medicare & Medicaid Services. Medicare and Medicaid Programs, Basic health program, and exchanges; additional policy and regulatory revisions in response to the COVID-19 public health emergency and delay of certain reporting requirements for the skilled nursing facility quality reporting program. https://www.federalregister.gov/documents/2020/05/08/2020-09608/medicare-and-medicaid-programs-basic-health-program-and-exchanges-additional-policy-and-regulatory
- 15.Domi M, Leitson M, Gifford D, Nicolaou A, Sreenivas K, Bishnoi C. The BNT162b2 vaccine is associated with lower new COVID-19 cases in nursing home residents and staff. J Am Geriatr Soc. 2021;69(8):2079–2089. doi: 10.1111/jgs.17224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Centers for Disease Control and Prevention. CDC museum COVID-19 timeline. https://www.cdc.gov/museum/timeline/covid19.html#Early-2021
- 17.Olsen SJ, Azzizz-Baumgartner E, Budd AP, et al. Decreased influenza activity during the COVID-19 pandemic — United States, Australia, Chile, and South Africa, 2020. Morb. Mortal. Wkly. Rep. 2020;69(37):1305–1309. doi: 10.15585/mmwr.mm6937a6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Zipfel CM, Bansal S. Assessing the interactions between COVID-19 and influenza in the United States. medRxiv. 2020;doi:10.1101/2020.03.30.20047993
- 19.Walsh K, Seyidov N, Wrocyznski M, Payne G, Bhagavatheeswaran L. Education and clinical decision support for healthcare professionals on emergency preparedness for extremely dangerous pathogens: report of a conference workshop. BMJ Mil Health. 2020;166(2):103–104. doi: 10.1136/jramc-2019-001328. [DOI] [PubMed] [Google Scholar]
- 20.Hwang Y, Jeong S-H. Information insufficiency and information seeking: an experiment. Sci Commun. 2016;38(6) doi: 10.1177/1075547016673200. [DOI] [Google Scholar]
- 21.White EM, Wetle TF, Reddy A, Baier RR. Front-line nursing home staff experiences during the COVID-19 pandemic. J Am Med Dir Assoc. 2021;22(1):199–203. doi: 10.1016/j.jamda.2020.11.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Yau B, Vijh R, Prairie J, McKee G, Schwandt M. Lived experiences of frontline workers and leaders during COVID-19 outbreaks in long-term care: a qualitative study. Am J Infect Control. 2021;49(8):978–984. doi: 10.1016/j.ajic.2021.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Palacios-Ceña D, Fernández-Peña R, Ortega-López A, et al. Long-term care facilities and nursing homes during the first wave of the COVID-19 pandemic: a scoping review of the perspectives of professionals, families and residents. Int J Environ Res Public Health. 2021;18(19):10099. doi: 10.3390/ijerph181910099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Fisher E, Cárdenas L, Kieffer E, Larson E. Reflections from the “Forgotten Front Line”: a qualitative study of factors affecting wellbeing among long-term care workers in New York City during the COVID-19 pandemic. Geriatr Nurs. 2021;42(6):1408–1414. doi: 10.1016/j.gerinurse.2021.09.002. (Minneap) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Snyder RL, Anderson LE, White KA, et al. A qualitative assessment of factors affecting nursing home caregiving staff experiences during the COVID-19 pandemic. PLoS One. 2021;16(11) doi: 10.1371/journal.pone.0260055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Mo S, Shi J. The psychological consequences of the COVID-19 on residents and staff in nursing homes. Work Aging Retire. 2020;6(4):254–259. doi: 10.1093/workar/waaa021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Serrano J, Hassamal S, Hassamal S, Dong F, Neeki M. Depression and anxiety prevalence in nursing staff during the COVID-19 pandemic. Nurs Manag. 2021;52(6):24–32. doi: 10.1097/01.NUMA.0000752784.86469.b9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Denny-Brown N, Stone D, Hays B, Gallagher D. COVID-19 intensifies nursing home workforce challenges. 2020.