Remote Evidence-Based Health Promotion Programs During COVID: A National Evaluation of Reach and Implementation for Older Adult Health Equity

Abstract

Introduction.

Evidence-based health promotion programs (EBPs) support older adults where they live, work, pray, play, and age. COVID-19 placed a disproportionate burden on this population, especially those with chronic conditions. In-person EBPs shifted to remote delivery via video-conferencing, phone, and mail during the pandemic, creating opportunities and challenges for older adult health equity.

Method.

In 2021–2022, we conducted a process evaluation of remote EBPs by purposively sampling diverse U.S. organizations and older adults (people of color, rural, and/or with disabilities). The Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) + Equity framework was used to understand program reach and implementation, including FRAME to describe adaptations for remote delivery. Analyses include descriptive statistics and thematic analysis of participant and provider surveys and interviews, and joint display tables to compare learnings.

Results.

Findings from 31 EBPs through 198 managers/leaders and 107 organizations suggest remote delivery increases EBP reach by improving access for older adults who are underserved. For programs requiring new software or hardware, challenges remain reaching those with limited access to—or comfort using—technology. Adaptations were to context (e.g., shorter, smaller classes with longer duration) and for equity (e.g., phone formats, autogenerated captioning); content was unchanged except where safety was concerned. Implementation is facilitated by remote delivery guidelines, distance training, and technology support; and hindered by additional time, staffing, and resources for engagement and delivery.

Conclusions.

Remote EBP delivery is promising for improving equitable access to quality health promotion. Future policies and practices must support technology access and usability for all older adults.

Background

The onset of the COVID-19 pandemic in March 2020 triggered governmental lockdown policies requiring everyone to physically and socially distance from others. This was particularly challenging for older adults living with multiple chronic conditions as they were more vulnerable to COVID-19 (Shahid et al., 2020), and the lockdown made it unsafe to participate in in-person evidence-based health promotion programs (EBPs). Persons of color (Gross et al., 2020; Mackey et al., 2021), those living in rural areas (Henning-Smith, 2020), and/or those living with disabilities (Lebrasseur et al., 2021) faced disproportionate hospitalization and COVID-related mortality, exacerbating existing inequities in health outcomes and access to care.

Prior to COVID-19, more than 1 million older persons participated in primarily in-person EBPs to support their health and well-being (National Council on Aging [NCOA] database 2022). Although criteria vary by organization, definitions converge on EBPs as standardized interventions based on sound behavior change theories that have demonstrated effectiveness in improving health outcomes through research testing. Many programs teach skills to help older people and those with disabilities to live a “new normal” in the face of multiple chronic conditions and/or disability. EBPs are relatively low-cost, high-impact community-based programs that improve equity in quality, access, and cost of care (Hoeft et al., 2018), and can address social needs and behavioral factors that support people to prevent injuries, increase activity, and reduce symptoms including depression.

Beginning in Spring 2020, EBP developers and administrators partnered with delivery organizations to adapt in-person programs for delivery through video-conferencing, phone and mailed materials, or combination thereof. The pandemic context presented both opportunities and challenges for remote delivery. Remote services can improve access to care by addressing known barriers to in-person programming (e.g., transportation, mobility, weather, and group recruitment; Elalem et al., 2018) and maintaining social connectedness. However, although older adults have better access to, and comfort using, technology than is often perceived (GSA, 2020), the digital divide is still present. A recent survey of 55 million Medicare beneficiaries found 41% lack access to a computer with high-speed internet connection or a smartphone with a wireless data plan (Roberts & Mehrotra, 2020), with one in four lacking both modes of digital access. Underserved older adults (living in poverty, aged 85+, widowed, less educated, Black or Hispanic/Latine, receiving Medicaid, or having a disability) had even worse access.

As such, there is an urgent need to understand remote EBP delivery—who is served and not served, how these programs are adapted for remote delivery, how they can be improved, and the effectiveness of remotely delivered EBPs. To this end, we conducted two studies. The first, presented here, involves understanding the reach and implementation (including adaptation) of remote-delivered, older adult health promotion EBPs. The second study, to be presented in a manuscript under preparation, looks at the effectiveness of remote delivery for this population. We designed both the implementation and effectiveness evaluations as pragmatic studies (Glasgow, 2013) to guide future policy and practice. The potential impact of this study is to improve access and quality of care for older persons through the network of social service and public health providers serving older Americans.

Method

Theoretical Framework

We used Shelton and colleagues’ (2020) expansion of RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance framework; Glasgow et al., 2019) to guide this cross-sectional evaluation. RE-AIM is a well-established implementation science framework for evaluating the public health impact of proven interventions. Shelton added equity to the original RE-AIM framework to strengthen evaluating how interventions promote and constrain health equity while attending to local context for program sustainability. This study focused on RE-AIM’s Reach and Implementation constructs, as remote delivery of in-person EBPs was novel and presented both opportunities and challenges to improving access. Further details on how RE-AIM was operationalized for this study is provided in Figure 1.

Figure 1.

Concurrent Mixed-Methods Evaluation of Remote EBP Reach and Implementation: Methods Flowchart

Note. EBP = evidence-based health promotion program.

Design

This study employed a concurrent mixed-methods evaluation design. This cross-sectional design gathered real-time quantitative and qualitative data on recently launched (within 1 year of adoption) remote EBPs. Implementation science urges using mixed methods—particularly sound qualitative methods—to describe the complexity, adaptations, and strategies to improve access to care (Mazzucca et al., 2021; Means et al., 2016). To minimize burden on all concerned, data collection was kept brief. This study was determined exempt under Category 2 for program evaluation and quality improvement by the University of Washington institutional review board (STUDY00011549).

Setting and Participants

Our study population included diverse EBP administrators/developers (the organizations who supported EBP dissemination and implementation), organizations (the agencies who delivered EBPs for their communities), and leaders (the staff and volunteers at these organizations that directly delivered remote EBPs to their clients). All EBPs recognized by the Administration for Community Living (ACL) or Centers for Disease Control and Prevention (CDC) (Arthritis-Appropriate Evidence-Based Interventions [AAEBIs]) and adapted for remote delivery as of January 1, 2021, were eligible to participate (NCOA Center for Healthy Aging, 2023; Osteoarthritis Action Alliance, 2022). Remote EBP delivery organizations were largely community based and included social services, public health, and health care agencies.

Recruitment and Sampling

In early 2021, programs, organizations, and leaders were recruited via a brief online interest survey and webinars through several networks: EBP training listservs, the CDC Arthritis Program and ACL grantees, the Evidence-Based Leadership Collaborative (EBLC), and regional EBP networks. Eligibility criteria included intent to deliver at least one ACL-recognized remote EBP from January 2021 through March 2022.

While all EBPs were represented in this evaluation, maximum variation purposive sampling (Palinkas et al., 2015) was used to identify EBP administrators that represented different remote modes (video-conferencing, phone, mail), format (1:1/group/self-administered), and health topics, and to invite organizations of diverse size and type, provider characteristics, and geographic area to aid generalizability of evaluation findings. This sampling prioritized organizations engaging low-income older populations with multiple chronic conditions.

Within each participating organization, convenience sampling was used in which all leaders of remote EBPs during the evaluation period were invited to participate via email invitation from the investigators, EBP administrators, and delivery organizations. The intent was to collect 100 surveys and conduct 20 interviews to allow for sufficient variation (Francis et al., 2010). Survey participants received US$10 gift cards and interview participants received US$25.

Data Collection

Quantitative Data Collection

Online surveys (Supplemental Appendix 1) were used to collect data from organizations and leaders. Organizations were represented by “managers”—the staff who oversee program recruitment, delivery, and monitoring. Remote EBP “leaders” were the front-line staff or volunteers who deliver remote programs. Both the organization (manager) and leader surveys gathered information about reach (new populations, underserved populations, populations not reached, and strategies for improving reach) and implementation (acceptability and feasibility of remote EBP delivery, and strategies to improve remote delivery). Both surveys collected data on remote EBP type and mode, and organization characteristics. The leader survey also collected data on demographics and EBP experience.

Qualitative Data Collection

Semistructured interviews were used with program administrators/developers, organizations (managers), and leaders. The interview guide (Supplemental Appendix 2) was derived from Rabin and colleagues’ (2018) operationalization of RE-AIM for Equity, asking about reach, adaptations (Stirman et al., 2019; what, why, how, when, for whom, impact, equity, social determinants of health), implementation outcomes (acceptability and feasibility; Proctor et al., 2011), and strategies for improving remote EBP delivery.

Data Analysis

REDCap (Harris et al., 2019) was used for survey data management and R (R Core Team, 2022) for survey data analysis. Descriptive statistics characterized the sample, and summary statistics described remote EBP modalities, reach, impact, barriers, and unintended consequences. Recorded and transcribed interview data were managed in Word and analyzed using coding matrices in Excel. A rapid analytic framework (Gale et al., 2019) was used to thematically analyze interview data. Qualitative open-ended survey questions identified key patterns and distinctions that were not captured in multiple-choice questions. After quantitative and qualitative analyses, findings were integrated in joint display tables (Guetterman, 2019). These were used to examine where findings converged, diverged, or expanded understanding of remote EBP delivery. Differences between managers and leaders, and by remote program modes, were also reported.

Results

Participation

Programs

Table 1 illustrates remote EBPs delivered by surveyed organizations. There were 17 EBPs with more than five organizational survey responses. These surveys were used for the main analysis. The remaining programs (14) had five or less survey responses. These programs were excluded from the main analysis but are included in the subanalysis of survey data by remote modes.

Table 1.

Program Type, Description, Mode, and Respondent Distribution

Program	Program type	Program description	Remote mode	Manager survey		Leader survey		Total
				n	%	n	%	n	%
PEARLS (Program to Encourage Active, Rewarding Lives)	Behavioral Health Program	This one-on-one program has six to eight 50-min sessions. It focuses on addressing late-life depression and uses brief behavioral techniques	Video-conference, Phone	2	1.2	13	5.3	15	3.7
Powerful Tools for Caregivers	Caregiver Support Program	This group-based program has six weekly sessions. It focuses on helping caregivers take better care of themselves	Video-conference, Phone	5	3.1	5	2.0	10	2.5
National Diabetes Prevention Program	Chronic Disease Prevention Program	This group-based program lasts 1 year meeting 1/week for the first 6 months and 1/month for the last 6 months. It focuses on behavior change for adults with prediabetes	Video-conference	3	1.9	11	4.5	14	3.4
Cancer: Thriving and Surviving	CDSMP	This group-based program has one 2.5-hr meeting per week for 6 weeks. It focuses on decision-making and communication skills for cancer survivors	Video-conference	4	2.5	7	2.9	11	2.7
CDSMP	CDSMP	This program is offered in two remote formats: with 6 weekly 2.5-hr video-conferencing sessions or as a mailed tool kit of materials that is used either with or without 6 weekly scripted 45- to 60-min small group phone calls. It focuses on techniques for dealing with symptoms of chronic conditions	Video-conference, mailed toolkit with or without phone	24	14.9	49	20.0	73	18.0
Chronic Pain Self-Management Program (CPSMP)	CDSMP	This program is offered in the same formats as CDSMP. It focuses on techniques for dealing with symptoms of chronic pain conditions	Video-conference, mailed toolkit with or without phone	13	8.1	30	12.2	43	10.6
DSMP	CDSMP	This program is offered in the same formats as CDSMP. It focuses on management of diabetes through healthy eating, physical activity, and medication management	Video-conference, mailed toolkit with or without phone	18	11.2	33	13.5	51	12.6
Health Coaches for Hypertension Control a	CDSMP	This is a group-based program with eight, 90-min sessions offered once or twice a week. It focuses on self-management of hypertension	Video-conference	2	1.2	5	2.0	7	1.7
HomeMeds	CDSMP	This one-on-one program uses a comprehensive assessment of a participants’ medication inventory, present adherence, and potential adverse effects to create personalized recommendations	Phone	4	2.5	6	2.4	10	2.5
Tomando Control de su Salud (Spanish CDSMP) a	CDSMP	This program is a Spanish language version of the CDSMP and is offered in the same formats, but the toolkit with phone call format lasts 12 weeks instead of 6	Video-conference, mailed toolkit with or without phone	5	3.1	3	1.2	8	2.0
Programa de Manejo Personal de la Diabetes (Spanish DSMP) a	CDSMP	This program is a Spanish language version of the DSMP and is offered in the same formats, but the toolkit with phone call format lasts 12 weeks instead of 6	Video-conference, mailed toolkit with or without phone	4	2.5	2	0.8	6	1.5
AEA Arthritis Foundation Exercise Program a	Exercise Program	This group-based program can be taken as an 11-session course or as an ongoing class. It uses exercise to improve and maintain physical health	Video-conference	4	2.5	3	1.2	7	1.7
Bingocize	Falls Prevention Program	This group-based program has 11 or more 45- to 60-min sessions. It combines the game of bingo, exercises, and health education	Video-conference	7	4.3	5	2.0	12	3.0
EnhanceFitness	Falls Prevention Program	This group-based program is offered on an ongoing basis with fitness tests at baseline, 16 weeks, and at regular intervals thereafter. It focuses on exercise to prevent falls	Video-conference	4	2.5	6	2.4	10	2.5
A Matter of Balance	Falls Prevention Program	This group-based program has eight 2-hr sessions, either once a week for 8 weeks or twice a week for 4 weeks. It focuses on fall prevention through education, group-discussion, and exercise	Video-conference	34	21.1	36	14.7	70	17.2
Tai Chi for Arthritis and Falls Prevention	Falls Prevention Program	This group-based program is offered either 1 hr a week for 16 weeks or 2 hr a week for 8 weeks. It focuses on fall prevention and arthritis management through the use of Tai Chi movements	Video-conference	8	5.0	4	1.6	12	3.0
Walk With Ease	Falls Prevention Program	This 6-week program is offered in three formats: Group-based with 1-hr per week; Self-Directed, which uses a guidebook; and Self-Directed Enhanced, where a leader coordinates communication to a group to ensure participants stay organized and motivated. It focuses on physical activity and education	Self-directed (unless otherwise specified)	6	3.7	12	4.9	18	4.4
Total b				147	100	230	100	377	100

Note. Some leaders delivered more than one program. EBP = evidence-based health promotion program CDSMP = Chronic Disease Self-Management Program; DSMP = Diabetes Self-Management Program.

^aPrograms with n ≤ 5 were excluded from the main quantitative analysis and included in the analysis by remote modes. These include Active Living Every Day (ALED), Online Better Choices, Better Health given by Canary Health (BCBH), EnhanceWellness, FallsScape and FallsTalk, HealthMatters, Healthy IDEAS, Respecting Choices, Stay Active and Independent for Life (SAIL), Stepping On, Stress Busting for Family Caregivers, Tai Ji Quan: Moving for Better Balance, Workplace Chronic Disease Self-Management Program (WCDSMP), and Wellness Recovery Action Plan (WRAP). ^bTotal N is 123 surveys with remote EBP leaders and 75 surveys with remote EBP program managers (N = 198 provider surveys representing 107 organizations). The N is 377 above because some survey respondents managed or delivered more than one remote EBP.

Administrators, organizations, managers, and leaders

Twenty-six qualitative interviews were conducted with seven EBP administrators, five managers, seven leaders, and seven individuals with both manager and leader roles. Most interview participants identified as female and worked at community or government organizations. In all, 123 surveys were conducted with remote EBP leaders and 75 surveys with managers for a total of 198 surveys from 107 EBP organizations in 33 states (Supplemental Appendix 3). Three fourths (77.7%) of both managers and leaders delivered the program using video-conferencing, 11.1% used phone + mail, 6.4% used mail + video, and 4.7% via hybrid (both in-person and remote modes). Fifteen percent of managers and leaders offered programs in languages other than English (typically Spanish).

Remote EBP leader demographics are provided in Table 2. The majority identified as female (94.2%) and 13% identified as Black/African American, 12% as Hispanic/Latine, 4% as Asian, and 1% as Biracial. One quarter of leaders lived in rural settings, one third were caregivers, and one third had two or more chronic conditions. One fourth identified as a certified health professional, and 43% as a community health worker, promotora, or other lay health provider. Leaders brought a range of experience in EBP delivery—half (53%) had delivered both in-person and remote programs prior to the survey, and 37% were conducting remote programming for the first time.

Table 2.

Provider Survey: Characteristics of Remote Evidence-Based Program Leaders (N = 123)

Program leader characteristics	n (%)
Gender
Male	7/120 (5.8%)
Female	113/120 (94.2%)
Ethnicity—Hispanic/Latine	14/120 (11.7%)
Race
Asian	5/117 (4.3%)
Black/African American	15/117 (12.8%)
Biracial a	1/117 (0.9%)
Otherb	1/117 (0.9%)
White/Caucasian	95/117 (81.2%)
Chronic conditions
M (SD)	1.17 (1.18)
Median [Min, Max]	1.00 [0, 5]
Two or more	37 (30.1%)
Rural	31/113 (27.4%)
Highest level of education
High school graduate/GED	6/121 (5.0%)
Some college	12/121 (9.9%)
College graduate	52/121 (43.0%)
Postcollege	51/121 (42.1%)
How hard to pay for basics (e.g., food, housing, medical care)c
Not hard at all	79/110 (71.8%)
Somewhat hard	29/110 (26.4%)
Very hard	2/110 (1.8%)
Live alone	20/121 (16.5%)
Caregiver role	36/120 (30%)
Certified health professional Community health worker, promotora, or other lay health provider	30/121 (24.7%) 52/120 (43.3%)
Experience delivering EBPs
First time doing program remotely, previously delivered program in person	20%
First time doing program remotely, have not previously delivered program in person	17%
Not first time doing program remotely, previously delivered program in person	53%
Not first time doing program remotely, have not previously delivered program in person	10%

Note. Some leaders delivered more than one program. EBP = evidence-based health promotion program; GED = general equivalency diploma.

^aPerson identified as White and Asian. ^bDid not specify. ^cMissing = prefer not to answer and not applicable.

Reach

Older Adults Who Are Underserved

More than half the programs reached older Black adults (58%), caregivers (57%), persons with disabilities (60%), and persons living in rural areas (59%) with remote EBPs. One third of the programs reported serving older Hispanic/Latine or Asian populations, 9% American Indian or Native Americans, and 5% Native Hawaiian or Pacific Islanders. Programs that combined mail with phone or video-conferencing delivery reported greater reach than other remote modes to Hispanic/Latine adults, caregivers, and people with disabilities; rural populations had greater reach through combined modes.

Interview findings suggested several ways remote EBP delivery may better reach underserved populations, including addressing barriers to access including transportation, having enough people to hold an in-person class, and serving wider geographic and rural areas. Interviewees also suggested that participating in remote programs may be easier for people managing multiple chronic health issues which makes it hard to consistently engage in in-person services.

Older Adults New to EBPs

Remote program delivery also offered an opportunity to support people not previously reached with in-person programming. Surveyed managers and leaders indicated new populations were those outside their usual geographic areas (68%), older older adults (48%; age not specified), rural-dwellers (46%), people with disabilities (42%), caregivers (36%), younger older adults (34%; age not specified), and men (30%; Table 3). Fifteen percent reached older adults who speak languages other than English. Programs combining mail with phone or video-conferencing delivery reported greater reach of new populations, and newly engaged participants with disabilities were best reached by phone-delivery. Managers reported lower reach outside of usual geographic areas than leaders when it came to remote EBPs delivered via a combination phone/video + mail (30% vs. 70%) and phone (27% vs. 70%); both differences were statistically significant (p < .001).

Table 3.

New Older Populations Reached by Remote Evidence-Based Programs

Population	Survey data a		Interview data b
	Leaders	Managers
People outside geographic area	70%	64%	• Able to reach people in different cities, towns, and parts of the state or region because no travel required
Older older adults	48%	45%	• Limitations attending in-person programs (e.g., health issues, mobility, transportation)
Live in rural areas	46%	45%	• Limitations filling in-person programs because small populations, isolated, lack of transportation
People with disabilities	42%	38%	• Hard to attend in-person EBPs due to caregiving duties and lack of respite care
Caregivers	36%	37%	• Hard to attend in-person EBPs due to caregiving duties and lack of respite care • Care partner could attend EBP remotely
Younger older adults	34%	29%	• Some younger seniors do not see themselves at senior serving organizations
Men	30%	31%	• Some men may not attend a place where EBPs are being offered or want to be in person in a group with others
Multiple chronic conditions	NA	NA	• Better attendance because can participate in program when not feeling well
People with limited English proficiency	NA	15%	• Lack of bilingual/bicultural leaders to deliver EBPs (e.g., organization does not have capacity to hire for all languages)
Working populations	NA	NA	• Do not have time to go to an in-person class but can attend remotely during lunch break or evening
People who are incarcerated	NA	NA	• New programming allowed with partnership in prison that may not be feasible in in-person delivery
People who would not have attended in-person	NA	NA	• Issues above with men and younger seniors • Reach people even during challenging weather (e.g., snow that limits transportation, summer heat as barrier for walking)
Weather limitations	NA	NA	• Reach people even during challenging weather (e.g., snow that limits transportation, summer heat as barrier for walking)

Note. EBP = evidence-based health promotion program.

^aSurvey findings report n (%) of leaders and managers who reported this population was reached. ^bInterview findings describe key themes and distinctions to illustrate how populations were reached.

Interviews supported the finding that remote EBP delivery reached new populations. As one provider shared,

I think it’s been a great adjustment for everyone to realize that the need is there and we certainly saw it over time, because people weren’t coming [in-person] but we didn’t have a format to really bring them in [. . .] it really resonated with me to know that there’s ways that we can reach and benefit people that we might not have otherwise thought about had we not gone to the virtual and toolkits. (Provider 115; Chronic Disease Self-Management Program [CDSMP], Chronic Pain Self-Management Program [CPSMP], Diabetes Self-Management Program [DSMP], Walk With Ease [WWE])

Likewise, an administrator shared,

We are seeing more and different types of people in the workshop that we hadn’t before, people that were more disabled and not able to get out and go. You know we’ve had some people attending while they’re in bed recuperating or because they don’t feel well, so it just makes it a little easier to reach folks that we weren’t reaching before. (Provider 119; CDCMP, DSMP, CPSMP)

Challenges to Remote EBP Reach

The most common survey response regarding who was not being reached by remote EBPs was older adults without access to (75%) or lack of comfort using (63%) computer or video technology (Table 4). In addition, one third of managers and leaders reported not reaching persons living in poverty or with low-literacy or education. Access and comfort using technology was a barrier to reach across remote program modes, even when the delivery modes were mail and/or telephone. Managers and leaders delivering EBPs that combined remote modes (mail + phone or video-conferencing) reported better reach to people living in poverty or with low-literacy or education. Interview findings suggested that challenges to reach stemmed primarily from lack of access to resources faced by older adults overall, and populations who are underserved, specifically.

Table 4.

Populations Not Reached by Remote Evidence-Based Programs

Population	Survey data		Interview data
	Leaders	Managers
Lack of access to technology	75%	80%	• Do not own a computer, laptop, tablet, or phone • Rural areas have unreliable or no broadband access • Hard to do video-conferencing EBP via small phone screen (some EBPs do not allow phones for this reason)
Lack of comfort using technology	66%	65%	• Discomfort using tech (e.g., lack of familiarity, usability, tech literacy; do not like it) • Fears about using technology (e.g., being scammed) • Do not want to be seen in homes or watched via camera • Lack of family or friend support to help with tech • “Zoomed” out by pandemic • Preference for in-person programs
Living in poverty	33%	25%	• Cannot afford required tech (e.g., cell phone minutes or • data plan, tablet with camera and internet access) • Unstable living arrangements or too small a home to allow for privacy or movement for physical activity EBPs
Low-literacy or education	33%	22%	• EBP materials too high-literacy
Not reached with in-person EBPs	NA	NA	• E.g., Hispanic/Latine, LGBTQ+
Lack of EBP awareness	NA	NA	• Harder to engage people remotely (e.g., not in physical spaces frequented by priority populations)

Note. EBP = evidence-based health promotion program; LGBTQ+ = lesbian, gay, bisexual, transgender, queer/questioning, plus (others).

Organizations offering multiple EBP delivery options felt they could “reach anyone anywhere” (Provider 115; Self-Management Programs, WWE). Although new populations were reached, remote EBPs could not solve all access problems, such as malaise or fatigue due to chronic health issues, work, or having busy schedules. Privacy was also an issue—as one organization serving tribal communities shared, some people “don’t want people in their business” (Provider 104). Finally, rapport building could be challenged by remote delivery:

It’s a barrier [with phone-delivery] because of the fact that, why would I want to give you my personal information if I can’t see your face? [. . .] that’s something that is a downside or maybe prohibits us from being able to get the assessment all the time. (Provider 106; HomeMeds)

Strategies for improving remote EBP reach

Interview participants articulated several solutions to improve remote EBP reach. To address engagement challenges, EBP delivery organizations partnered with trusted people and organizations, including faith leaders (e.g., medicine men on Indian reservations) and groups such as Black churches, senior centers, aging and long-term care networks, retirement groups, health care settings (e.g., managed care organizations, clinics in low-income regions), and other organizations engaging underserved older adults. Interview participants suggested that while these warm handoffs helped mitigate fears and concerns about remote programs, partners had limited capacity as their organizations were also overwhelmed and understaffed during the pandemic.

To improve access to technology, several organizations procured donations, grant funding, or reallocated funds to provide or loan tablets, laptops, or internet hotspots. While this facilitated access, they were sometimes logistically or technologically challenging. These issues were addressed via volunteers or staff offering one-on-one tech support for EBP participants before and during classes. Phone-based programs may have helped address some challenges to access as more people had and could use a phone; however, it may still be hard to connect and engage in remote programs when leaders and participants cannot see each other.

Implementation

Adaptation Process

For most EBPs, adaptations were made shortly after the pandemic began, while others took a year. Most adaptations were collaborative, with EBP administrators/developers working with trainers, leaders, subject matter experts, and/or technology designers to develop fidelity-consistent modifications that maintained core program content and processes while converting to remote forms (Perez Jolles et al., 2019). Adaptations were a mix of planned changes to modify an in-person program for phone, video, or mail delivery, and unplanned changes derived organically from field experience.

Adaptations for Equity

Some adaptations helped improve equity in reach and implementation (Figure 2). These included connecting older adults who speak languages other than English to bilingual/bicultural leaders, using autogenerated captioning software with video-conferencing to accommodate those hard of hearing, or delivering via phone to engage participants with limited technology access or comfort.

Figure 2.

Adaptations by Remote Format From Interview Data

Adaptations to Context

Several changes were made for in-person EBPs to be done remotely. To facilitate engagement, many phone-based programs adjusted class size, frequency, and duration. Many remote EBPs delivered via video-conferencing needed additional staff or volunteers for technology support and watching for safety concerns during exercises. Trainings for EBP leaders and trainers also moved to remote formats—They were largely viewed as successful, plus organizations could recruit and train new leaders more frequently and less expensively than in-person training. Limitations of distance training mirrored those of hosting remote EBPs, including challenges reading nonverbal cues, building rapport, igniting discussions, and managing people with varying tech abilities. Some trainings adjusted their frequency and duration to accommodate overburdened staff and busy organizations. There were differences across remote EBPs about new training requirements for remote delivery which was frustrating for some organizations and leaders.

Adaptations to Content

There were minor changes to remote EBP content. Programs using movement added safety checks and removed or altered exercises that were unsafe when unsupervised. Programs offered via multiple formats shortened scripts used for phone-delivery by adding additional sessions, removing nonessential intervention content, and/or adding new supplemental written materials. Routine data collection for phone-based programs was also streamlined. Complex protocols like medication management were simplified, and some content was made more interactive.

Acceptability

Survey findings indicated managers and leaders found remote EBPs acceptable—on a scale of 1 to 10 (1 = not at all likely, 10 = very likely), program managers (mean [SD] = 8.7 [1.8]) and leaders (9.1 [1.4]) were very likely to recommend remote programs; this difference was statistically significant (p = .047). Managers reported lower acceptability for both phone/mail programs (7.6 [2.3]) and hybrid (programs offered both in-person and remotely; 6.0 [NA]) than did leaders.

Feasibility

Looking at participation by remote mode, access, and comfort using technology was less of an issue for participants of EBPs using multiple remote modes. The most common reasons for dropout after the first session were health issues (57%) and older adults not comfortable with using the technology (40%). More than half of organizations and leaders surveyed did not know the reasons for dropout.

In interviews, some organizations shared that remote EBPs yielded more sign-ups and repeat participants, longer program participation, and more cross-referrals to other programs than had been experienced with in-person programs. They also described more consistent attendance because people could join regardless of weather, transportation, health, or location. Conversely, some organizations found no difference in dropout or attendance between in-person and remote programming. Others felt it was harder to engage people in remote programs and experienced worse attendance and dropout rates. Also, as many remote group programs had smaller class sizes, organizations had to offer more workshops to meet target numbers. In some cases, smaller class sizes could be taught by one rather than two leaders to compensate for costs.

Strategies to Improve Remote EBP Implementation

Better access to remote technology (46% of leaders, 49% of managers) and technology training for participants (38% of both leaders and managers) were the most common recommended changes for future remote EBP delivery; this was less of an issue for phone-delivered programs. More than 25% of managers and leaders recommended changes to program format, frequency, and duration—interview findings suggest possible changes such as longer program duration to build rapport or comfort using remote modes, or flexibility with what remote modalities were allowed.

Discussion

Summary of Findings

The evaluation found that remote EBPs are acceptable and feasible, reach communities who are underserved, and address barriers to accessing in-person services and support. For instance, people who face challenges with mobility, transportation, or multiple health issues can access remote EBPs from their homes; and staff, peers, and volunteers can deliver EBPs from home and engage people from wide geographic areas. Adding remote EBPs to services may also make it easier for social service and public health organizations to support older adults who are underserved.

Figure 3 calls out key take-aways from the evaluation, and Figure 4 summarizes implications for policy and practice to support remote EBP reach and implementation. There were a few surprising findings. First, program managers were more skeptical than leaders of reaching populations via phone or video + mail. At the same time, some older adults faced challenges using new technology regardless of delivery mode. It may be that some programs were not available in modes other than video-conferencing, or that managers had a bias against phone or mail delivery. In any case, these findings deserve further investigation.

Figure 3.

Key Take-Aways From the Remote EBP Implementation Evaluation

Note. EBP = evidence-based health promotion program.

Figure 4.

Implications for Policy and Practice to Support Remote EBP Delivery for Older Adult Health Equity

Note. BIPOC = Black, Indigenous, and People of Color; EBP = evidence-based health promotion program; LEP = Limited English Proficiency.

Facilitators for Remote EBP Delivery

Specific recommendations for adapting in-person programs for remote delivery include shortening program frequency or duration, and training leaders how to engage participants remotely and to problem-solve technology issues. Distance training may expand workforce capacity without extensive time or resources. While many services were limited during COVID-19, delivering EBPs remotely helped organizations continue offering health promotion and social care to reduce social isolation and loneliness. Findings suggest that while organizations and leaders were initially skeptical of whether programs could be done remotely, they can be done in these new modes. Our evaluation learnings reflect recent studies that found similar accessibility and recommendations for facilitating remote delivery (Gray et al., 2022; Li et al., 2021; Patel et al., 2022; Sheth et al., 2021).

Opportunities for Remote EBP Delivery

Future policies and practices can address the lack of access to, or comfort using, technology through training for older adults and making broadband internet a public utility for all as lack of access to resources are fundamental causes of health inequities (Phelan et al., 2010). EBP delivery by phone and mail provides another avenue for closing the digital divide (Kahlon et al., 2021; Sheth et al., 2021). Engagement of underserved older adults can be facilitated via outreach to trusted community leaders or sources (e.g., faith-based representatives, tribal leaders). In addition, as managers reported lower acceptability of some remote delivery modes than leaders, and reasons for dropout were not known by many organizations, opportunities remain for improving acceptability and participation. Finally, delivery organizations may need more staffing, funding, and other resources to establish, engage, and deliver remote programs (Gell et al., 2021).

Strengths and Limitations

The strengths of this evaluation included its collaborative nature with many partner organizations, timeliness of the issues created by COVID-19, its emphasis on how remote delivery affects health equity, and its ability to inform future health policy concerns for remote delivery of EBPs. A limitation of this study was the use of a convenience sample, which inherently biased our learnings toward people with positive experiences using remote EBPs. Lack of local population data, and limited subgroup data on race, ethnicity, tribe, rurality, and disability, made it difficult to gauge representativeness of participants and understand populations not being reached. The timing of the evaluation reflects the first 2 years of developing and delivering remote EBPs (Aarons et al., 2011) during a pandemic. It may be that over time, concerns about technology may abate for both participants and organizations. Future evaluations will yield important learnings about how to maintain remote EBPs outside the pandemic context. Furthermore, while our findings that remote EBPs are acceptable and feasible are important implementation precursors to health outcomes (Proctor et al., 2011), our second study will report on the effectiveness of remote EBP delivery by looking longitudinally at changes in key indicators of behavioral and health status.

Conclusion

Given the benefits of remote delivery, in particular, the reach of new and underserved populations, many organizations and older adults now see remote EBPs as essential on the menu of needed services and supports. For remote programs to continue to be offered outside the pandemic context, delivery and funding structures should be modified to support their implementation.