A process for converting an in-person training to increase church capacity to implement physical activity and healthy eating practices and policies to an online format

Abstract

The implementation of evidence-based public health programs into practice is critical for improving health, but trainings for organizational change agents are often not scalable.

To describe the process of converting a training that targets faith-based organizational capacity development from an in-person to an online format.

We engaged in an iterative process to convert the training delivery mode from in-person to online that included assessing stakeholder support, consulting the literature on best practices, seeking a design team, consolidating content, designing engaging lessons, and building an online site. Feedback from end-users and other audiences was incorporated throughout. Pilot participants with characteristics like intended training users were then recruited via community and faith-based partner networks. They rated their agreement with statements about the effectiveness as well as design and functionality of each lesson and the overall training (1 = strongly disagree, 5 = strongly agree) and participated in a structured follow-up interview.

Nine pilot participants (representing 9 churches in 7 states; 6 African American, 5 with health ministries) rated the online lessons favorably (all ratings ≥ 4.5). Most (90.4%) perceived the lesson duration to be “just right” and spent 52.5 ± 9.9 minutes/lesson. Participants evaluated the overall training positively (all ratings ≥ 4.7). Lesson content, resources, multimedia, and program ideas were most-liked aspects of lessons, while content, staff responsiveness, discussion board, and pace were most-liked aspects of the overall training in open-ended and interview responses.

This paper shares a replicable process for converting training modalities from in-person to online with the goal of increased scalability.

Implications.

Practice: Online trainings are important and practical for reaching and supporting large numbers of churches to create healthier environments for members.

Policy: The COVID-19 pandemic has underscored the need for more online trainings focused on helping organizations develop healthier practices and policies.

Research: Future research is needed to examine whether in-person trainings that are converted to online formats and target organizational change prove effective.

INTRODUCTION

The dissemination and implementation of evidence-based public health programs into practice is critical for impacting population health [1, 2]. Training is a commonly used and effective implementation strategy for evidence-based ­interventions [3–5], but in-person trainings are resource-intensive [6]. They also require training facilities and trainee travel time that may limit participation due to geographic availability or concerns due to issues such as COVID-19. Online trainings offer a flexible way to reach organizational change agents (e.g., teachers, practitioners, providers, ­community health workers) in a cost-effective and convenient manner [6], thereby increasing the potential for program scalability.

This paper describes how we changed the training delivery mode from in-person to online for an evidence-based, faith-based intervention called Faith, Activity, and Nutrition (FAN). FAN targets organizational practices to make church environments supportive of physical activity and healthy eating. We train church committees (organizational change agents) who then implement the core components of FAN in their churches. Church committee training in how to deliver FAN is a key implementation strategy to increase organizational capacity and skills. Increased interest in FAN from churches within and outside our state led us to convert the training to an online format. We changed the training delivery mode but not the program itself nor the key implementation strategy (i.e., training church committees).

Several examples exist in which in-person trainings for organizational change agents have been converted to an online format, with the potential for greater reach, lower cost, and greater efficiency cited as rationales for the change. For example, Saunders et al [7]. disseminated an online training (originally delivered in-person) for organizational change agents (preschool teachers) to modify institutional practices in the preschool setting to increase physical activity. They reported implementation completeness and fidelity in the statewide dissemination study comparable to their original study which relied on in-person training. Roberts et al [8]. changed training for internal medicine interns from an in-person to an online format due to the COVID-19 pandemic. The online training resulted in high levels of skill proficiency and training satisfaction. Lee et al [9]. compared an in-person train-the-trainer model to an online training model to prepare YMCA staff to implement physical activity and healthy eating practices and policies. While the in-person training resulted in significant improvements in number of practices and policies implemented relative to the control condition, the online training resulted in smaller and non-significant improvements. The online training, however, resulted in improvements in fruit and vegetable offerings, while the in-person training did not. Within the faith-based area, Santos et al [10]. compared the feasibility and implementation outcomes of a technology-based training (7 churches) with traditional classroom training (8 churches) designed to prepare volunteer community health advisors to promote cancer screening via church workshops. Both training formats were well-received and led to implementation of the workshops. However, drop-out rates, duration to complete training, and time to initiate workshops were greater in the technology-based training.

These studies [7–10], while informative, did not focus on the process and logistics of how to convert the training delivery mode from in-person to online and still retain learner engagement and interest. Further, existing models and suggested practices for developing online trainings tend to provide conceptual rather than logistical guidance and focus less on the process (e.g. [6, 11, 12],). We address this gap in the literature by describing a process for how to convert an in-person training to an online format. Beyond making the training more cost- and time-efficient, we also aimed to ensure the training was engaging to maintain the richness and interaction that naturally exists with in-person trainings.

Our work has focused on the church setting because it can play an important role in upstream and midstream public health practice [13–15]. Churches are present in nearly every community [13, 16] and offer an opportunity to reach populations with marked health disparities with messages that resonate with their beliefs [2]. For example, spiritual beliefs are particularly salient for African Americans, who report high levels of religious affiliation [15, 17]. Similar to research in other areas, few evidence-based interventions for churches are available for scale up and broad reach [2]. Our intervention, FAN, resulted from a community-based participatory research process where university and faith partners collaborated to develop the intervention and its evaluation [18]. It is guided by Cohen et al.’s structural model of health behavior [19] and aims to promote policy, systems, and environmental changes within the church setting to support physical activity and healthy eating. In the three studies conducted to date to evaluate FAN [20–22], church committees attended a full-day in-person training (training was a key implementation strategy to build organizational capacity [23]) where they were guided through an assessment and planning process to learn to provide opportunities, set policies, share messages, and enlist pastor support for physical activity and healthy eating (i.e., implementation outcomes). Additional implementation strategies to develop organizational capacity included technical assistance for program implementation and provision of media resources (e.g., bulletin inserts, newsletters, handouts).

In our first of three studies, the intervention resulted in moderate to large changes in church organizational practices (implementation outcomes) and small but significant improvements in members’ health behaviors [20, 24], leading to its inclusion in the National Cancer Institute’s Evidence-Based Cancer Control Programs [25]. The intervention was subsequently shown to produce similar outcomes in two more recent dissemination and implementation studies [21, 22], one conducted in a county with diverse religious denominations and the second in a large denomination in a single state.

We found little guidance in the literature on how to convert the training delivery mode from in-person to online, particularly for lay audiences. Thus, the three goals of this paper are to: i) describe the activities undertaken to change an in-person training for church committees to an online format; ii) present the results of our pilot study that assessed users’ experience with and evaluation of the resulting online training; and iii) provide recommendations for others who wish to undertake a similar process.

METHODS

This section describes key activities undertaken to convert the training delivery mode from in-person to online, including the feedback that various stakeholders provided and how this feedback was addressed in an iterative way. After the online training lessons were ready, the final activity was to conduct a pilot study to assess users’ experience with and evaluation of the training lessons and online training overall. Although the activities were generally done in the order presented, there were feedback loops and overlap in activities, as shown in Fig. 1. All study activities described in this paper were reviewed by our Institutional Review Board and granted exempt status.

Fig 1.

| Summary and Timeline of Activities to Convert the Program to an Online Format.

Assess stakeholder support and team capacity for converting training to an online format

We asked faith-based representatives of our research center’s Community Advisory Board (CAB) for input regarding offering the church committee training in an online format. They believed an online format would be well-received by churches, noting that many churches participate in online trainings for other faith-related topics. We also met with academic colleagues who converted a training for childcare providers to an online format [7] and a consultant who administers a national online health ministry certificate program. These discussions demonstrated stakeholder support and adequate capacity at our university to proceed.

Consult literature on best practices for online trainings and e-learning

We first searched for publications and resources that described the development and evaluation of online trainings and e-learning or provided guidance and best practices. Although we found general/conceptual recommendations, we did not find concrete, real-world applications of these general and somewhat abstract recommendations. For example, Ballew et al [6]. reviewed five disciplines and provided general recommendations to guide the development, delivery, maintenance, and evaluation of online trainings for public health. We also consulted the Guide to Improving and Measuring the Impact of Training [26], met with staff from our University’s Center for Teaching Excellence, and considered non-academic sources, including articles, and blogs (e.g., https://community.articulate.com). One useful example that resulted from non-academic sources was an extensive e-learning design and development checklist [27]. Although the categories/domains varied somewhat across these sources, there was substantial overlap in recommendations and best practices. Supplement File 1 summarizes how we applied Ballew et al.’s [6] conceptual recommendations for the delivery of web-based trainings in the domains of formative research, design and layout, content, interactivity, technical support, and feedback and follow-up; Ballew et al.’s recommendations were inclusive of content from other sources.

Seek a design team to design online lessons

Identifying professionals external to our research team who could design the training lessons from the content we provided proved most challenging. We wanted lessons that were polished, professional, and engaging for a lay audience and did not have the “look and feel” of an academic presentation. Neither campus resources nor job postings yielded viable candidates due to lack of experience developing similar trainings, inability to meet our timeline, or prohibitive costs. The design team we selected, a company with experience developing health-related online trainings for commercial groups, was identified via professional network contacts. We shared an example of a training we liked and what we liked about it (e.g., interactivity, non-academic look) with the team. The design team agreed to work with the research team to develop two lessons that would be reviewed by various stakeholders prior to moving forward with the remaining six lessons. We selected two lessons that were most ready for development by the research team and not dependent on content from other lessons. The agreement with the design team included explicit language that the university would own the lessons and that the raw files would be shared with the research team for revision over time.

Consolidate content from in-person training and develop storyboards

Based on recommendations that content should be concise, use short phrases, and avoid professional jargon [6], our team consolidated content from the in-person training slides to reflect key points. One staff member reduced the amount of text and/or created content that conveyed the information in an interactive format. A second staff member reviewed these changes and suggested additional edits. The study PI (SW) reviewed all lessons and finalized content. We ensured key learnings and content remained intact (i.e., only the mode of training change, and not the training itself). We developed storyboards in Microsoft PowerPoint to display the basic content in slides along with indications for placement of videos, audio files, scripture, or images (see later sections). We used the notes section to develop the narration script. Figure 2a provides an example of one storyboard frame.

Fig 2.

| (a) Storyboard Frame Created by Research Team. (b) Training Lesson Frame Created by Design Team.

Convert in-person training to engaging online training lessons with full narration

Design initial two lessons

The design team used our storyboards to design two lessons with Articulate Storyline software. They shared drafts with the research team, and the teams regularly interacted ­concerning lesson formatting, functionality, and interactivity. Figure 2b shows how a storyboard we created was developed into a lesson frame.

Identify, edit, and create videos and photos

After key training points were incorporated into the storyboard, we drew from the domains of design and layout, content, and interactivity (see Supplement File 1) to ensure the online training was visually appealing, used multimedia, included concrete real-world examples, and fostered learner interactivity. We used short video and audio clips (from interviews with church committee members) to provide examples of how churches implemented structural components of the program and to replicate content from our in-person trainings. For example, videos included a food demonstration, a story of a church that developed a food co-op as community outreach, and 10-minute activity breaks (from publicly available exercise videos from the National Institute on Aging and Exercise Your Faith Instant Recess videos from GramercyNC). Finally, we worked closely with the design team to choose images and scripture to enhance key points. We purchased images from professional sources (e.g., iStock), incorporated photos from earlier studies, and took new photos. These videos, audio segments, and pictures displayed diversity in people and churches.

Record and edit voice narration

Once the lessons were developed in Articulate Storyline, we recorded, edited, and integrated voice narration. The research team recorded narration separately for each frame of each lesson using Audacity (free software) and high-quality audio microphones (Samson Meteor Mic USB Studio Microphone and Blue Yeti USB Microphone) and then edited the recordings as needed for sound quality, consistency, and volume. The design team then uploaded each audio file into its corresponding Articulate Storyline frame and coordinated the timing of text and animations with the narration. One African American woman (external to our group) and one white woman (lead investigator, SW) narrated the lessons, representing the demographic characteristics of typical learners. The narration text was also available for reading in another pane on the screen. Providing the content as both audio narration and text accommodated people with lower literacy levels as well as hearing impairments.

Obtain feedback on the initial two lessons from churchgoers and additional reviewers

The research team shared the initial two lessons with a convenience sample of six adults (5 were women; ages ranged from 28 to 61 years) identified through a co-author’s (DK) social network, reflecting Baptist, Non-denominational, and Apostolic faith backgrounds. These adults attended church regularly and had lay leadership roles in their church, and thus were like typical church committee members from our previous studies. Each person reviewed one of the two lessons and completed a detailed feedback form (see Supplement File 2) where they noted what they liked and disliked about each frame of the lesson and reported on features such as ease of navigation, ability to open links and resources, technology problems, and perceived length of lesson (too long, too short, just right). They also rated their agreement (1 = strongly disagree, 5 = strongly agree) with 19 statements related to color scheme, organization, layout, navigation, photos, narration, videos, and resources. These statements came from the Knowledge for Health e-learning pilot test survey (now operated by Knowledge SUCCESS; survey no longer accessible), an online source with sample beta testing questions for online courses [28], and content created by our research team. Participants took part in a brief follow-up call to elaborate on their experiences and to clarify responses from the feedback form. Prior to the call, the interviewer flagged negative or unclear comments and developed questions to prompt elaboration. They also asked each person the one thing they would do to make the lesson more engaging or appealing. Interviews were recorded, and the interviewer summarized participants’ comments. All participants received a $25 gift card.

Thirteen additional reviewers provided feedback. These reviewers included research Co-Investigators (n = 4) and faculty (n = 2) not involved in lesson development; community partners (n = 2); staff (n = 4) with expertise in areas including online professional development trainings, graphics and design, and online technology; and an external consultant who administers an online health ministry certificate program (n = 1). The reviewers were sent links to the lessons and were asked to provide feedback regarding clarity (content, appearance, navigation) and what they liked and disliked about the lessons. They were not given a structured feedback template or statements to rate. These reviewers provided comments via the Articulate Storyline review feature or via email. They were not paid.

Mean scores and ranges were computed for the structured items used with churchgoers. Team members reviewed the transcriptions (churchgoers only) and open-ended comments and grouped recommendations in similar categories (e.g., navigation issues, photos/videos, clarity).

Churchgoers reported few problems in navigating the lessons, finding resources, opening links, and related issues. All reported that the length of the lesson was “just right.” Agreement with most items (16/19) was high (ratings >4 out of 5) regarding design features and functionality. Ratings were slightly lower for color used in lessons (M = 3.7, SD = 1.0), photo appeal (M = 3.8, SD = 1.2), and video interest (M = 3.8, SD = 0.8). Open-ended feedback on what was liked and disliked about each lesson frame included more positive than negative comments, with clarity, pace, photos, pastor videos, exercise breaks, and use of scripture cited most often as liked features. Negative feedback on specific frames tended to be easily addressed (e.g., background color in one lesson, need for better coordination of narration relative to lesson text, narration and video played at same time, sometimes unclear when to advance, suggestions for making directions clearer). The follow-up phone calls clarified issues when ratings were lower. For example, a participant who did not find the color scheme appealing thought the shade of purple was too dark. One participant suggested adding music to the start of the lesson. Another stated that the narrator’s voice was not “exciting” enough. Another thought some of the videos and images were dated.

Feedback from the 13 additional reviewers mirrored that of the 6 churchgoers. Overall, they were positive and shared many of the same suggestions, but they also included more logistical and technical recommendations. For example, two people familiar with the software suggested technical changes like adding a seek bar to the frames. Multiple people commented that the narration volume was too low.

Revise initial two lessons and design remaining six lessons

The research team organized all feedback into categories (e.g., orienting the user, navigation, design, videos, photos), discussed each category, and requested edits from the design team. Overall edits included creating orientation videos, adding a spinning arrow to indicate when to advance, incorporating music at the beginning and end of each lesson, increasing narration volume, allowing users to advance and rewind content after they listened to the frame in full, simplifying some frames, adding a zoom function for some images, adding closed captioning (text of the narration), including images of more diverse pastors (i.e., women) and churches, and better syncing the timing of narration with lesson content. Some feedback was not incorporated because it was not consistent with the program (e.g., using terms that would appeal to broader faith traditions) or because it was only raised by one person and seemed unnecessary (e.g., did not like font).

After the initial two lessons were revised based on this feedback, the final six lessons were designed, and the activities described earlier (videos/photos, voice narration, iterative process of reviews and edits) were continued. We shared lessons with our CAB members for review and incorporated minor edits based on their feedback. As shown in the timeline (Fig. 1), it took approximately 11 months from the identification of the design team to the completion of the lessons. The design team’s fee for developing the lessons was approximately $30,000. This total did not include the substantial research team time to create storyboards, create video and audio files, review and edit lessons, and record and edit lesson narration nor did it include the cost of stock images and software.

Build and modify online learning platform

While the training lessons were being completed, we developed the online learning platform to house the 8 training lessons, 12 months of media resources, and discussion board that allowed for interactions between churches and research staff. We chose to use Moodle (https://moodle.org/) as our online learning platform because it is open-sourced and we had in-house support for this platform, and thus it helped our longer-term sustainability goal. We removed all features in the Moodle platform that were not essential in order to make the training site as simple as possible for lay learners. We programmed the site to require learners to watch two brief videos that oriented them to the training site and lesson navigation. Learners were also required to introduce themselves to other churches by posting on the discussion board. In addition, we programmed for one lesson to be released per week, contingent upon the learner viewing the previous lesson in its entirety, passing the knowledge quiz (≥80% correct; could be re-taken up to three times), and completing the online survey (described below).

Conduct pilot study

Select study measures

Two comprehensive models that have guided the evaluation of online trainings were used to develop survey items for the pilot study. The first model by Ballew et al [6]. has two main categories (and recommended criteria within each) for evaluating online trainings: effectiveness (user reaction, user knowledge, user attitudes, user behavior change, and organizational change) and design and functionality (­content, design, accessibility, usability, resource availability, and technology). The second model, Multidimensional Conceptual Model for Evaluating E-learning System Success, based on approaches to evaluating e-learning systems’ success [11, 12], consisted of seven independent constructs (technical system quality, information quality, service quality, educational system quality, support system quality, learning quality, and instructor quality) and four dependent constructs (perceived satisfaction, perceived usefulness, system use, and benefits).

These two evaluation models guided the identification of a pool of items used in other studies or recommended for evaluating online trainings and e-learning. We categorized each item using the categories and criteria from [6] and the constructs from [12] and chose 15 items to assess each lesson. We also chose 14 items to assess the overall online training. We chose items that best matched our study goals and would provide useful information to guide lesson modifications. We reworded items as needed to match our study context or to provide uniformity (i.e., reworded questions to statements). For most items, the learner rated their agreement from (1) strongly disagree to (5) strongly agree. We did not assess user behavior change as it was beyond the scope of the pilot study. The items, the domains/constructs they represent, and their source are presented in Supplement File 3. To assess user knowledge (a criteria of effectiveness) at the end of the lesson, we developed 10-item quizzes consistent with the key learning objectives for each lesson. The learning management system (Moodle) captured user completion data and provided time-and date stamps when site components were accessed. We extracted lesson completion data and number of posts to the discussion board.

Recruit participants

Our Center’s CAB members and other partners shared recruitment flyers through their community and faith-based networks to help us recruit 10 people from 10 different churches to pilot test the resulting online training lessons. The recruitment flyer emphasized the goal of reaching individuals involved in church health ministry or interested in improving the health of their congregations, thus aiming to reach individuals similar to our typical church committee members.

Implement study procedures

After each lesson, participants were prompted to complete the 15-item lesson evaluation survey and the 10-item knowledge quiz. After all eight lessons were viewed, they were prompted to complete the 14-item overall training survey. Finally, each participant took part in a follow-up telephone interview about their training experiences. Using a semi-structured interview guide, the telephone interview asked what participants thought was the best part of training and suggested changes to training, as well as their views on the lesson timeline (eight lessons in eight weeks), discussion board, Moodle-generated reminder emails, and anything else they wished to share. The interviewer also asked about any technology challenges that were reported on post-lesson surveys. After pilot participants completed the lessons, evaluation surveys, and telephone interview, they received $200. Interviews were recorded and transcribed. Team members reviewed the transcriptions and open-ended comments and grouped recommendations into similar categories (e.g., navigation issues, photos/videos, clarity).

RESULTS

This section reports the results of our pilot study that assessed how each of the eight lessons and overall training were experienced and evaluated by users. Participant characteristics, lesson completion and engagement, evaluation of the lessons (Likert-style and open-ended items), and evaluation of the overall online training (Likert-style and open-ended items as well as telephone interviews) are reported.

Pilot participants

Due to health problems, one of the ten people recruited did not participate. Six of the nine pilot participants were African American (66.7%), three were white (33.3%), and seven were women (78.8%). They had a mean age of 55.5 years (range: 37-74), and all attended at least some college (six were college graduates). Their churches represented four denominations (AME, UMC, Baptist, and Church of Christ) and were from seven states. Five of the nine participants were in churches with health ministries.

Lesson completion and engagement

All pilot participants completed the eight lessons within the eight-week period and passed all knowledge quizzes. Self-reported lesson duration averaged 52.5 ± 9.9 minutes (range of means: 35 to 68). An average of 6.5 participants posted on the discussion board each week (range of means: 5 to 9).

Evaluation of lessons

Table 1 presents participants’ agreement with the post-lesson statements averaged across the eight lessons (i.e., mean of the means). Participants rated the lessons favorably; mean agreement with statements across the eight lessons ranged from 4.5 (confidence to achieve lesson goals) to 4.8 (six different items) out of 5, indicating their agreement was between “agree” and “strongly agree” for all ­criteria/constructs. Across all lessons, lesson duration was rated “just right” 90.4% and “too long” 9.6% of the time, and technical problems were reported 15% of the time (most often related to a video).

Table 1.

| Pilot Participants’ (N = 9) Ratings for Lessons

	Grand Mean (SD)	Min, Max of Means	Min, Max of Scores
Lesson ratings (1 = strongly disagree, 5 = strongly agree)
Overall satisfied with lesson	4.8 (0.1)	4.6, 4.9	4, 5
Provided useful information about [lesson content]*	4.8 (0.1)	4.6, 4.9	4, 5
Confident I can [achieve goal outlined in lesson]*	4.5 (0.2)	4.3, 4.8	3, 5
Content arranged in clear and logical way	4.8 (0.1)	4.6, 4.9	4, 5
Voice and quality of the narration appealing	4.8 (0.1)	4.6, 4.9	4, 5
Videos enhanced lesson	4.7 (0.1)	4.5, 4.8	3, 5
Photos were appealing	4.7 (0.1)	4.6, 4.9	3, 5
Layout and design clear and visually appealing	4.8 (0.1)	4.6, 4.9	3, 5
Easy to make way through lesson.	4.6 (0.3)	4.3, 4.9	2, 5
Able to access resources in lesson easily	4.7 (0.1)	4.5, 4.9	2, 5
Assessment covered and tested material presented	4.8 (0.1)	4.6, 4.9	4, 5
Perceived lesson duration
% too short	0	0, 0	n/a
% just right	90.4	78, 100	n/a
% too long	9.6	0, 22	n/a
Experienced any technical problem, % yes	15.0	0, 40	n/a

^*Questions were specific to the lesson. Note: response options were 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree.

Responses to open-ended questions about what participants liked most about each lesson included lesson content, resources provided, use of multimedia, and program ideas. Responses to what participants liked least about each lesson included primarily “nothing” or “N/A,” as well as too much content in the lesson (or too long), activity breaks were too long to implement in their setting, wanted more videos, and experienced a few technical issues (e.g., music too loud; video buffered).

Overall evaluation of the online training

Table 2 presents participants’ ratings of the survey items assessed at the completion of the eight online lessons. Participants rated the overall training very favorably; mean agreement with statements ranged from 4.7 (satisfaction with support received through the discussion board) to 5 out of 5 (would recommend to others, interactive elements were effective, mixed media was effective, material can be taught effectively in a web-based format), indicating they “strongly agreed” with statements for all criteria/constructs. In response to open-ended survey questions about the overall training, participants most-liked the content, responsiveness and helpfulness of the study staff, the discussion board (“I liked that we shared our thoughts in the discussion each week”), and the self-paced format. In response to what they liked least, responses included “nothing,” a desire for more or fewer videos, and the lack of new information for someone well-informed about health. When asked if they had anything else they wished to share, responses were positive, with statements such as, “This is a great program that I can’t wait to share with my health ministry team.”

Table 2.

| Pilot Participants’ (N = 9) Overall Rating of the Faith, Activity and Nutrition (FAN) Online Training

Item (1 = strongly disagree, 5 = strongly agree)	Mean (SD)	Min, Max
I would recommend the FAN Online Training Program to others.	5 (0)	5, 5
The interactive elements of the lessons were effective.	5 (0)	5, 5
The use of mixed media (graphics, animation, audio, and video) was effective.	5 (0)	5, 5
This material can be taught effectively in a web-based format.	5 (0)	5, 5
I could find what I was looking for on the FAN Online Training Site (e.g., Discussion Board, Assessments, Resources).	4.8 (0.4)	4, 5
I was able to use the features of the FAN Online Training Site (e.g., Discussion Board, Assessment, and Resources).	4.9 (0.3)	4, 5
I am confident that I have the skills to put the physical activity part of the FAN program in place in my church.	4.9 (0.3)	4, 5
I am confident that I have the skills to put the healthy eating part of the FAN program in place in my church.	4.8 (0.7)	3, 5
I am confident that I can work with my pastor and other church leaders to put the FAN program in place in my church.	4.8 (0.7)	3, 5
* I am satisfied with the support I received from the FAN program team.	4.9 (0.4)	4, 5
I am satisfied with the support and ideas I received through the FAN Online Training discussion board.	4.7 (0.5)	4, 5

^*One participant selected “I did not need or request support.” Thus, there were only 8 respondents for this item’s mean and range. Note: response options were 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree.

During the follow-up interviews, participants expressed a range of most-liked components of the online training, with common responses including the videos, the use of “real life” examples from churches (“Having the videos and the snippets and the information from people who’ve actually done the training, having their stories it made it more personable, to hear from the pastor or the FAN coordinators from different churches”), the inclusion of resources, the use of activity breaks, and the general comprehensiveness, usefulness, and format of the program (“It is a complete package. Nothing was lacking or missing.”). Suggestions for changes were few. Several participants reiterated that there might be too much content in some of the lessons for some people. One participant thought the activity breaks were too long and another said that the learner should not be required to view them. One participant did not like having to click to advance through the lesson.

All participants thought completing eight lessons in eight weeks was realistic and feasible. For example, one person stated, “It was perfect. You didn’t feel overwhelmed, it gave you time to really know the material and it gave you time to really look over the chapters and if you need to go back and look at things, you know, you could.” Another added that the online platform was especially convenient. Participants also liked the reminder emails sent through Moodle because they helped keep them on track. Participants found the discussion board useful, especially the opportunity to get ideas from other churches. Participants attributed most of the technology challenges to their own computer or internet connection. In response to whether there was anything else they wished to share about the online program, participants were uniformly positive. For example, one person said, “you can tell that you guys really care about this product, and really want it to succeed and really want people to learn how to do this” and “thank you for this life changing program.”

DISCUSSION

This study makes two major contributions to the field: (i) it describes a replicable process for converting in-person trainings to engaging online trainings, and (ii) it resulted in an online training that is available for broader dissemination in churches. This section describes those two contributions, notes study limitations, and provides recommendations for others seeking to embark on a similar process.

Replicable Process

Researchers have converted in-person trainings designed for organizational change agents to online trainings in clinical [8], community [7, 9], and faith-based [10] settings, but these studies have not focused on the process and logistics of converting the trainings. Changing the delivery mode to online is not simply a matter of posting training slides with voice-over narration if the goal is to create an engaging training that increases user capacity and skills. We underwent a lengthy and rigorous process that was guided by a literature review of best practices and recommendations for developing e-learning content, including Ballew’s et al.’s [6] recommendations, as well as other practical guides and checklists [26, 27]. Our process included frequent feedback loops between the research team, design team, end-users, and other stakeholders, consistent with principles of designing for dissemination [29, 30]. We also used Ballew et al.’s [6] recommendations, along with Al-Fraihat et al.’s Multidimensional Conceptual Model for Evaluating E-learning System Success [11, 12], to identify relevant domains and items to evaluate the training. This process could be replicated by other groups for a wide range of organizational settings.

Availability of an online training for churches

Churches have the potential to improve population health through the promotion of health behaviors. Indeed, faith-based settings are a sector in the National Physical Activity Plan [31] and are described as important partners in the World Health Organization’s Global Action Plan on Physical Activity [32]. Yet few evidence-based interventions for churches are available for scale up [2]. This study built on three previous studies that demonstrated that FAN, an organizational change intervention, resulted in improved organizational practices [21, 22, 24] and member health behaviors [20, 21]. We used a community-based participatory research approach [33, 34] to develop FAN, where church partners were engaged in the entire research project [18]. Thus, the intervention had good fit with the church culture. However, the modality of our primary implementation strategy (training) was in-person, which limited its scalability. In the present study, we retained the components of the evidence-based intervention and the training content that prepares church committees to implement it and focused only on changing the training modality. It was not surprising, therefore, that in the current study, users were very enthusiastic about the intervention and the training. Most of the recommended edits related to design features, logistical issues, technical issues, and user needs.

Study limitations

The primary limitation of this study is the small number of participants and churches in the pilot study. However, the purpose of the pilot study was to obtain user experiences with and evaluation of the online training since only the training modality, and not the program itself, changed. The community-engaged approach used to develop FAN, our intentional involvement of stakeholders in the process of converting the training to an online format, and the consistent positive comments from pilot participants regarding the online training led us to believe we were ready to embark on a national dissemination study. Prior to launching the national study, we made final edits to the online lessons based on the pilot results. Assessing capacity-building outcomes such as skills, self-efficacy, and motivation of church committee members is critical in this research [35], and is included in our national study. Another limitation is that we did not assess whether the training resulted in changes in organizational practices. This outcome will be the primary focus of the national implementation study. Finally, we cannot rule out that there was a social desirability/response bias that drove the favorable lesson ratings.

Lessons learned and recommendations

We conclude with lessons learned from this experience and provide recommendations for others embarking on a similar project. First, it is important to engage intended recipients of the training and intervention from the beginning. FAN was developed with and for churches, consistent with designing for dissemination [30]. Changing the training delivery from in-person to online did not require many modifications to content, allowing us to spend time on features such as design and functionality [6]. Second, one needs to allow adequate time and costs for training development. The process was time- and staff-intensive, lengthy, and expensive. Third, feedback from diverse reviewers is helpful. Through the process of converting the trainings to an online format, we involved our research team (faculty, staff, doctoral students), staff with technical skills, community members, and intended users of the training. Each type of reviewer provided unique and helpful information, and we saved time and money by obtaining feedback on two lessons prior to moving forward. Our ability to leverage existing faith-based networks and both in-house and partner resources facilitated our work substantially. Fourth, we recommend identifying a design team that understands the aspirations of the research team as well as the intended user of the training. The design team should have experience developing trainings for diverse learners, experience using the desired software, and a willingness to collaborate and accept feedback. Fifth, it is critical to be attentive to the design and layout of the training, as emphasized by Ballew et al [6]., including visual appeal, interactivity, multimedia, and consistency in formatting. Pilot participants especially liked the videos and examples from other churches. Finally, one should plan for time to learn the software and ensure that institutional technical support is in place. Articulate Storyline and Moodle had steep learning curves and would have been difficult to use without our design and support teams.

Summary

In conclusion, this paper describes an iterative set of activities to change the modality of a key implementation strategy (training) used to prepare church committees to implement an ecological faith-based program from in-person to online. Results from our pilot study indicate that users found the training informative and engaging, felt confident they could implement the program components, and would recommend the training to others. We provide logistical details of our process and make recommendations based on our experiences so that other groups might save time and encounter fewer barriers in their endeavor to increase the reach and public health impact of interventions. Our ability to now offer training for the program via a web-based platform is likely to help make the program more scalable and available to a wider target audience.

Compliance with Ethical Standards

Conflict of Interest: All authors declare that they have no conflicts of interest.

Human Rights: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent: This study was granted exempt status from the University of South Carolina Institutional Review Board. Written informed consent was not requested.

Welfare of Animals: This article does not contain any studies with animals performed by any of the authors.

Transparency Statements: This paper describes the process of converting an in-person training to an online format and the results of a pilot study to obtain study users’ experiences and evaluation of the online training. This study was not formally registered. The analysis plan was not formally pre-registered. De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author. Analytic code used to ­conduct the analyses presented in this study are not available in a public archive. They may be available by emailing the corresponding author. Materials used to conduct the study are not publicly available.