What Works and for Whom? Outcome Evaluation of an E-mail Walking Program Delivered Through Cooperative Extension

Abstract

Introduction:

Get WalkIN’ is a 12-week, e-mail-based walking promotion program. The purpose of this study was to (1) compare sociodemographics of participants who enrolled versus completed the program; and (2) evaluate program feasibility/acceptability from perspectives of program participants and county-based Extension Educators who implemented the program.

Methods:

Participants (N = 875), recruited by county-based Extension Educators, were asked Likert-scale questions (eg, ease of reading the e-mails and frequency of e-mails) to assess program acceptability and open-ended questions regarding improvements. Educators (N = 55) were asked Likert-scale and open-ended questions regarding program training, recruitment, strengths, and areas for improvement. Descriptive statistics summarized participant characteristics, acceptability, and feasibility data. Open-ended responses were analyzed using thematic analysis.

Results:

There were no significant sociodemographic differences between participants (N = 875) who started the program and completed the program (n = 438). Participants reported intervention e-mails were easy to read (mean = 4.5 ± 0.7), understand (mean = 4.5 ± 0.7), and encouraged more walking (mean = 4.1 ± 0.9). Participants would like to connect/interact with other participants/Educators, have more monitoring tools for accountability, and more visuals/videos embedded within e-mails. Educators reported program training was adequate, e-mail messages were helpful and easy to use, and requested more visuals (eg, videos) to help with recruitment. Educators thought adding a social component (eg, kick-off walk or walking group) would be helpful.

Conclusions:

This Extension-delivered walking program is acceptable to participants and feasible to deliver. However, participants and Educators reported they would like more interaction, even if virtual. Future e-mail-based programs should consider ways to incorporate social interaction among users as well as provide a wide variety of recruitment resources.

Introduction

Physical inactivity is a highly prevalent public health issue across the United States. ¹ As of January 2020, all states and territories of the U.S. had greater than 15% of adults categorized as inactive. ² The highest prevalence of inactivity is found in Hispanic adults (32%), followed closely by non-Hispanic Blacks (30%). ² Further, physical inactivity is prevalent across males and females. Twenty-eight percent of adult females are categorized as inactive, while 24% of males fall into the same category. ³ Additionally, the prevalence of inactivity increases as individuals age. Adults between the ages of 18 and 24 have a 22% rate of inactivity, while those 65 years and older have a 31% prevalence of inactivity. ³

Income and education are intertwined and considered individual-level determinants of physical inactivity. For example, 15% of individuals that obtain college degrees are categorized as inactive compared to 45% of those that complete less than a high school diploma. ³ Similar to education, income level is also related to rates of physical inactivity with 15% of adults making greater than $75 000 classified as inactive, as opposed to 41% of those making less than $25 000 per year. ³

The promotion of walking is one way to decrease physical inactivity. Walking is a highly accessible mode of physical activity since it does not require special skills, facilities, or equipment in order to be completed. For those that may have physical disabilities, assistive devices can be used, and walking can be completed in- or out-doors, whichever is preferred or accessible. ⁴ Walking is also a lower-risk physical activity when compared to other more vigorous activities. The intensity and amount of walking can be gradually increased in order to prevent injury. ⁴ Walking is an activity that can be accomplished for multiple purposes such as for transportation (eg, walking to work, school, or the store), socialization, or leisure.

Community-based physical activity programs have demonstrated impactful outcomes and cost-effectiveness. ⁵ However, consistent infrastructure to scale-up community-based programs can be challenging. In order to further promote walking to decrease physical inactivity, community-based organizations, such as Cooperative Extension, can be utilized. Extension offices and health education professionals can be found in most local communities. ⁶ Extension staff are often local residents of the counties they serve. As such, they too have a stake in the overall health and well-being of their communities. Additionally, Extension Educators are trusted individuals who are known to bring evidence-based educational interventions to address issues or needs. There are currently multiple on-going direct education focused programs through Extension, such as Eating Smart-Being Active.^6,7 Incorporating an e-mail-based program to promote physical activity is yet another way to utilize the services that Extension has to offer.

Past research has shown that internet-delivered physical activity programs have small but effective outcomes. ⁸ These small changes could prove to have a large impact on a population-level. There are also multiple benefits of internet/e-mail-based interventions for physical activity such as convenience, flexibility, and user independence.^9,10 These types of programs also allow for large numbers of participants to be reached, while avoiding high implementation costs. ¹¹

To further impact program outcomes, examining participant and program implementer feedback is crucial. In the current study, researchers partnered with the Cooperative Extension System to explore the use of an e-mail mediated walking program. The purpose of this study was to (1) compare sociodemographics of participants who enrolled in the program versus participants who completed the program; and (2) evaluate acceptability and feasibility of the Get WalkIN’ program from the perspectives of both program participants and the county-based Extension Educators who implemented the program. In addition, areas of program improvement from perspectives of both participants and program implementers will be discussed.

Methods

Program Description

The Get WalkIN’ program is e-mail-based and spans 12 weeks. A total of 16 e-mail messages are sent to participants. E-mails are sent twice a week for the first 4 weeks of the program and then weekly for the remaining 8 weeks. Get WalkIN’ is based on Bandura’s social cognitive theory which posits that individual health behavior is influenced dynamically through interactions of personal factors, environmental influences, and behavioral components. ¹² The main construct, self-efficacy, details an individual’s confidence in the ability to perform a behavior in spite of barriers. ¹² Control of the health behavior can be achieved though self-regulation and goal setting. ¹² A detailed description of the theoretical framework for this program has been previously published. ¹³ Further, an initial evaluation of the pilot of this program is also available. ¹⁰ This community-based program was deemed exempt by institutional review board, including for written informed consent. Prior to completion of program surveys, participants are presented with an online study information sheet.

Recruitment and Study Procedures

This program is offered at the county-level across the 92 counties in the state of Indiana. According to 2020 Census data, Indiana residents are 85% White, 10% Black, and approximately 3% Asian. Seven percent of Indiana residents are Hispanic or Latino. Extension has a history of serving more rural-based populations as well as those who may be considered limited resource audiences, which can influence those who participate in program offerings. Further, Indiana can be considered a mostly rural state. According to the latest USDA Economic Research Service Rural-Urban Continuum Codes, 48 of Indiana’s 92 counties (52%) are classified as Non-metro, meaning, these counties include some combination of open countryside, rural towns and urban areas with populations ranging from 2500 to 49 999 that are not part of larger labor market areas (metropolitan areas).

County-based Extension Educators from across the state of Indiana are able to offer the Get WalkIN’ program at any point in time. Educators recruited participants using a variety of strategies and resources. Extension Educators have access to a variety of pre-made, tailorable recruitment methods including social media posts, newsletters, e-mails, and flyers. The Get WalkIN’ program was advertised in local newspapers, radio ads, and other local media sources.

For the current study, participants and Extension Educators participated in Get WalkIN’ between January 2019 and January 2021. Inclusion criteria were adults at least 18 years of age who reported consistent access to e-mail and the ability to read English. There were no other limiting inclusion or exclusion criteria for program participation. Initially, 1526 community members who expressed interest in the program were e-mailed an introduction briefly describing the program and asked to complete an online, baseline survey. One week after the initial e-mail, a second reminder e-mail was sent to all participants who had not completed the survey. Program evaluation for participants occurred immediately after the last program e-mail at 12 weeks. Participants were once again e-mailed a link to an online, post-program survey. Program evaluation for Extension Educators occurred at the end of each year. All Educators who had implemented Get WalkIN’ were e-mailed a link to an online survey.

Measures

Participant sociodemographics of age, gender, race, marital status, household income, and education were assessed at 2 time points: baseline and post-program. Self-reported physical activity was also assessed at baseline and post-program using the International Physical Activity Questionnaire Short Form (IPAQ-SF). ¹⁴ Participants reported the average number of days he/she participated in walking, moderate, and vigorous activity in a typical week and the average duration in minutes per activity episode. A weekly metabolic equivalent (MET) score was calculated according to IPAQ-SF scoring protocol (walking × 3.3 METs, moderate physical activity × 4.0, vigorous physical activity × 8.0). ¹⁵ Participants were categorized as low active (<600 week MET minutes), moderately active (600-1499 weekly MET minutes), or sufficiently active (≥1500 weekly MET minutes). ¹⁵

After the 12-week program, participants were asked Likert-scale questions (1 = strongly disagree; 5 = strongly agree) about the program as a whole (eg, frequency of e-mails and encouragement provided by the e-mails), as well as specific questions about the structure and content of the program e-mails (eg, ease of reading/understanding the emails; credibility of the e-mails). Participants were also asked open-ended questions regarding ideas for program improvement and program strengths.

To assess program feasibility, county-based Extension Educators who had delivered the program at least once during the past year were asked both categorical (yes, somewhat, no) and open-ended questions regarding training, recruitment, implementation, program strengths, and limitations.

Data Analysis

Participant characteristics and outcome measures, including feasibility and usability data, were summarized with descriptive statistics. Means and standard deviations were calculated for continuous variables and frequencies and percentages for categorical variables. Chi-square and t-tests were used to assess differences between participant characteristics at baseline and post-program. Data were analyzed using SAS 9.4. ¹⁶ Open-ended responses were analyzed using thematic analysis. ¹⁷ Initially, to become familiar with the data, 2 researchers independently reviewed the open-ended responses from participants and Extension Educators. Next, the 2 researchers independently identified a list of initial codes. These codes were then compared and consensus reached. Codes were then condensed into meaningful themes. The creation of themes was guided by a 2018 evaluation of the program pilot. ¹⁰ When disagreement arose about theme categorization, discussion occurred until agreement was reached.

Results

Participant Characteristics: Pre and Post

While 1526 participants initially expressed interest in the walking program, 875 participants completed the baseline survey (57.3% attrition). At baseline, participants were on average middle aged (52.1 ± 13.4 years), White (95.8%), females (92.1%) (see Table 1). Participants were educated, with a majority (67.6%) earning at least a 2-year college degree. Further, 71% of participants reported a household income of at least $50 000 per year. At baseline, 32.7% of participants were classified as low active, 30.2% as moderate active, and 37.1% as high active. After completing the 12-week program, 438 participants completed the online survey (50.0% program completion rate). There were no significant differences in participant sociodemographic characteristics between baseline and post-program. Post-program, significantly fewer participants were classified as low active (11.4% vs 32.7%) and significantly more participants were characterized as high active (58.2% vs 37.1%; P < .001).

Table 1.

Sociodemographic and Physical Activity Characteristics of Participants at Baseline and Post-Program.

	Baseline (n = 875)		Post-program (n = 438)		P-value
Age (mean years ± SD)	52.1 ± 13.4		51.7 ± 12.9		.49
	n	%	n	%
Gender
Male	68	7.9	38	8.8	.11
Female	789	92.1	394	91.2
Race
White	817	95.8	402	96.8	.50
Black	19	2.2	4	1.0
Other	17	2.0	5	1.2
Marital status
Married/living as married	641	75.7	310	73.1	.10
Single	97	11.5	58	13.7
Divorced/separated	64	7.6	35	8.3
Widowed	45	5.3	21	5.0
Income
<$50 000	247	29.1	116	28.0	.21
$50 000-89 999	338	39.8	160	38.5
$90 000+	264	31.1	139	33.5
Education level
High school/GED	118	13.9	69	15.8	.78
Some college	158	18.6	79	18.2
2- or 4-year college degree	356	41.8	190	43.7
Masters	193	22.7	87	20.0
Doctoral degree	26	3.1	10	2.3
Physical activity
Low active	286	32.7	50	11.4	<.001
Moderately active	264	30.2	133	30.4
High active	325	37.1	255	58.2

Columns do not equal full sample size due to missing data.

Program Evaluation: Participants

Participants reported that program e-mails were easy to read (mean = 4.5 ± 0.7) and easy to understand (mean = 4.5 ± 0.7) (see Table 2). Further, participants agreed that the e-mails were delivered at adequate frequency (mean = 4.4 ± 0.7) and encouraged more walking (mean = 4.1 ± 0.9). Most participants reported reading the e-mail messages always (58%) or quite often (29%), while 10% of participants reported reading the e-mails sometimes and 3% reported rarely reading the e-mails.

Table 2.

Participant Reported Means and Standard Deviations of Intervention Acceptability (n = 438).

	Mean ± SD
The emails were easy to read	4.5 ± 0.7
The emails were easy to understand	4.5 ± 0.7
The frequency of the emails was adequate	4.4 ± 0.7
The emails encouraged me to increase my walking	4.1 ± 0.9
I found the walking tips to be credible	4.3 ± 0.8
I will continue to use the tips I received	4.1 ± 0.8
I think using emails for this intervention is a good choice	4.3 ± 0.8

When specifically asked what features of the program e-mails participants felt were most helpful (see Table 3), 75 participants stated that the e-mail content was encouraging and motivational. Participants stated “messages were just helpful to get me to walk” and “getting an email from a trusted source gave me an extra boost to get in more steps.” Participants (n = 57) also recognized that the advice, tips, and tricks were most helpful. This theme is reflected in participant quotes such as “helpful hints and success stories were good” and “the tips on how to keep motivated. . .were helpful.” Participants (n = 19) also reported that the social support and interaction he/she received from the Extension Educator was helpful and that the external/additional web links embedded in the e-mail messages were helpful (n = 16). The visuals and the tailored local information were also noted as strengths of the program.

Table 3.

Thematic Analysis of Participant Open-Ended Responses for Program Evaluation.

	N	%	Example quotes
What was helpful? (n = 182)
Encouraging and motivational	75	41.2	“Messages were just helpful to get me to walk”
			“The fact that I was getting an email prompted me to walk”
			“Getting an email from a trusted source gave me an extra boost to get in more steps”
Advice, tips, and tricks	57	31.3	“Providing reasons to walk and the impact on health was helpful”
			“Helpful hints and success stories were good”
			“What to expect when the weather starts changing and how to compact cold weather was helpful”
			“The tips on how to keep motivated and how to modify when needed were helpful”
Social support and interaction	19	10.4	“I was encouraged by the requests for my reply”
			“I liked hearing what worked for other participants”
			“Asking me to respond about my personal experiences was helpful”
External/additional links	16	8.8	“The links to other websites help to motivate me”
			“The additional links had very useful information”
			“Providing tracking apps was helpful”
Visuals	8	4.4	“The pictures kept my interest”
			“The emails were visually appealing”
Local information	7	3.9	“Including the maps of local parks was helpful”
			“The list of local places to walk”
Program improvement (n = 50)
Connect with others	12	24.5	“Add personal comments from others who are walking”
			“Have a group walking to help each other”
			“I would like to hear other’s stories about walking and overcoming barriers”
			“Set up a walking club”
More accountability or motivation	8	16.3	“Maybe include tracking or accountability checks”
			“Consider a database to log walking”
			“Have a contest for reaching goals”
More visuals or audio	7	14.3	“I am a visual person so diagrams and more pictures would be helpful”
			“Maybe include a video message summarizing the content”
			“Share photos of other participants walking routes or locations”
Additional topics	6	12.2	“Consider adding advice on how to walk in poor weather”
			“Include information on proper nutrition before walking”
			“Add tips for optimal food energy and hydration”
			“Health recipes would be nice”
Include more interaction with Educator	5	10.2	“Always include an activity or a question to respond to”
			“Maybe create an interactive quiz over the content”
			“Give me more prompts to reply to”
More local information	4	8.2	“Include where I could walk in my rural area”
			“Include information on local trails and distances”
			“I would like to know more about my local walking paths”
Too basic	4	8.2	“I would have benefitted from more unique advice”
			“Content was a little too elementary for me”
			“Some information seemed duplicated”
Text reminders	3	6.1	“I would like a text reminder to get out and walk”
			“Texts would be more convenient”
			“I would like a text group of participants”

Fewer participants (n = 50) provided feedback on areas for program improvement with most noting that the program “works great as is.” However, 12 participants specifically noted they would like more interaction or connection with other participants. For example, 1 participant stated “I would like to hear other’s stories about walking and overcoming barriers.” In addition, 8 participants recommended more aspects of accountability or motivation by suggesting “tracking or accountability checks” or “a contest for reaching goals.” Participants (n = 7) were also interested in having more visuals or audio added to the e-mails, as well as including additional topics such as nutrition and hydration (n = 6). Five participants stated that more interaction with the Extension Educator would improve the program and 4 participants thought providing more local information about walking would strengthen the program. A few (n = 4) participants stated that the e-mails seemed too basic, describing content as “a little too elementary,” while 3 participants would like text messages to be used to deliver program content.

Program Evaluation: Extension Educators

Most Extension Educators (94.2%) reported the program training was adequate, with 3 educators reporting the training was only somewhat adequate (see Table 4). A majority of Educators (76.9%) reported that the recruitment materials were sufficient and most (92.3%) reported that the pre-developed e-mail messages were easy to send. A majority of Educators (76.9%) reported that it took less than 20 min each week to implement the program.

Table 4.

Extension Educator Program Evaluation (n = 52).

Question	N (%)
Program training was adequate
Yes	49 (94.2)
Somewhat	3 (5.8)
No	0 (0)
Provided recruitment materials were sufficient
Yes	40 (76.9)
Somewhat	11 (21.2)
No	1 (1.9)
Pre-developed emails were easy to send
Yes	48 (92.3)
Somewhat	4 (7.7)
No	0 (0)
How much time did it take to implement the program each week?
<10 min	13 (25.0)
10-19 min	27 (51.9)
20-29 min	10 (19.2)
≥30 min	2 (3.8)

Despite the virtual nature of the program, 12 Educators thought adding an in-person component (eg, “kick-off walk” or “walking group”) would improve the program (See Table 5). Further, 9 Educators felt that adding more visuals or including videos in the e-mails would strengthen the program. Extension Educators reported some of their “participants requested videos to supplement the messages” or that “additional graphics would make the emails more appealing.” Educators (n = 7) also felt that using a new or different technology, such as social media or text messages, would enhance the program. Other ideas for program improvement included more opportunities to personalize the messages or increase engagement with participants. Educators clearly requested “help on personalizing [emails] more” and adding “additional activities or challenges to personalize the program and make it more engaging.” Educators (n = 5) also felt that including more external resources such as nutrition or self-monitoring information, or more local resources such as maps would enhance the program (n = 4).

Table 5.

Thematic Analysis of Program Deliverer Open-Ended Responses for Program Improvement (n = 27).

	N	%	Example quotes
In-person component	12	44.4	“My participants wanted a walking group”
			“A kick-off or celebration in-person event would be a good addition”
More visuals or videos	9	33.3	“Some of my participants requested videos to supplement the messages”
			“I am not sure the emails are always read, maybe adding videos would help”
			“Additional graphics would make the emails more appealing”
			“Videos may be helpful, especially for participants with lower reading levels”
Add new or different technology	7	25.9	“A social media group would be a good avenue to involve participants”
			“Adding text messages may help participants to walk”
			“Maybe making the program into an app would make it more trendy and participants could sync to wearable fitness trackers”
Activities to increase personalization	7	25.9	“I would like help on personalizing more because I never heard from my participants”
			“Add additional activities or challenges to personalize the program and make it more engaging”
			“Ask participants for more interaction to keep them involved and reduce drop out”
More external resources	5	18.5	“Even more resources in the emails could be helpful to participants”
			“Maybe add nutrition information as well”
			“More resources to track or self-monitor so participants can see how much they walk in a month”
More local resources	4	14.8	“Everyone should include maps of local walking areas”
			“I would like more information on where my participants can walk locally”
			“Consider providing educators with suggestions of local connections we can make”
Supplemental material	3	11.1	“I have repeat participants so having additional or new resources to offer them would be helpful”
			“Adding something for those who have already participated once”

Discussion

Recognizing the value and importance of participant and program implementer feedback in program evaluation, this study analyzed open-ended questions about program strengths and areas of improvement. Further, we examined if participant sociodemographics were related to completing a 12-week e-mail-based walking program. Study findings support that participants felt the program e-mails encouraged walking and that they will continue to use the program content to maintain walking routines. Program implementers also indicated that this community-based walking program was easy to disseminate and not time-intensive.

While there were no significant differences in the sociodemographics of participants who started and completed the program, program participants were vastly White, middle-aged females. Considering that diverse populations such as Hispanics and Blacks have higher rates of inactivity, future offerings of Get WalkIN’ should attempt to reach more diverse audiences. Further, program recruitment materials may need to be modified to be more attractive to diverse populations. It is possible that different program delivery mechanisms such as social media or text messages may be needed to reach these populations. Social media based health promotion programs tend to reach younger and more diverse audiences.^18-20 In addition, health promotion programming is becoming more rooted in internet-based mediums with social media use exponentially increasing.^20,21 Future research with the Get WalkIN’ program will consider a social media component. In general, health promotion programs should consider ways to incorporate social interaction among users as well as provide a wide variety of recruitment resources.

In line with social cognitive theory, ²² participants reported program aspects which theoretically increase self-efficacy, such as social support provided by other participants and the Extension Educators, as strengths of Get WalkIN’. In this study, both participants and Extension Educators recommended increased interaction and social support opportunities to enhance the program. In addition, participants identified that more opportunities for accountability and self-monitoring would improve the program. This feedback is especially important as both receipt of social support and the ability to self-monitor health behaviors are associated with increased self-efficacy and subsequent physical activity.^23-25

Further, participants and Extension Educators both suggested that providing more information about local resources for walking, such as information about trails and walking routes, would improve the Get WalkIN’ program. This feedback is important as environmental supports are indirectly associated (through increased self-efficacy) with increased walking behavior.^26,27 Providing participants with additional knowledge about walking locations could further improve the outcomes of this program and facilitate behavior maintenance.

Strengths and Limitations

There are several strengths and limitations to consider in the context of this study. First, participants self-selected to participate in this physical activity program. As such, selection bias is likely seen in those who chose to participate. Participants were likely motivated to change their behavior and hence, signed up for a program to help increase their walking. In addition, access to e-mail was a requirement for program participation which may have limited the number of low-resource participants who enrolled. While access to e-mail may have limited the ability to target minorities in this state, recent Census data indicated 89% of Indiana households have a computer. Further, only select program materials are currently available in Spanish, limiting current program participation to those who can read English. These sources of bias are likely highlighted by the White, educated participants this program served.

To allow for rich data collection and in-depth understanding of this program, this evaluation used open-ended questions to further understand program strengths and areas for improvement. This type of feedback allows researchers to have a more nuanced understanding of participant experiences. However the data collected, including physical activity, was self-reported which is prone to limitations such as social desirability bias. ²⁸ Of note, many community-based programs do not have the resources or infrastructure for objective monitoring of health behaviors. Future programs could explore the use of cell phone or smart watch activity tracking to monitor program outcomes. These consumer grade fitness trackers are increasing in popularity and have potential for research applications. ²⁹

Conclusions

Two years of program evaluation data suggest this e-mail-delivered, community-based walking program is effective in increasing physical activity behaviors, acceptable to participants, and feasible to deliver. However, participants and Extension Educators reported they would like more interaction, even if virtual. Future e-mail-based programs should consider ways to incorporate social interaction among users as well as provide a wide variety of recruitment resources to attract more diverse participants. Cooperative Extension is a valuable resource for health promotion programming and should continue to be utilized.