Abstract
Small Steps for Big Changes (SSBC) is a community-based diabetes prevention program (DPP). SSBC utilizes a motivational interviewing (MI) informed counseling style to deliver a structured diet and exercise curriculum to empower healthy behavioral modifications and prevent type 2 diabetes (T2D). An electronic learning (e-learning) platform to train SSBC coaches was developed to improve flexibility, reach, and accessibility. While e-learning has shown to be an effective mode of teaching content to health professionals, less is known in the context of DPP coaches. This study aimed to assess the effectiveness of the SSBC e-learning course. Twenty coaches (eleven fitness staff personnel and nine university students) were recruited through existing fitness facilities to participate in the online SSBC coach training which included completing pre- and post-training questionnaires, seven online modules of content and a mock client session. Knowledge regarding MI (Mpre = 3.30 ± 1.95, Mpost = 5.90 ± 1.29; p < 0.01), SSBC content (Mpre = 5.15 ± 2.23, Mpost = 8.60 ± 0.94; p < 0.01), and T2D (Mpre = 6.95 ± 1.57, Mpost = 8.25 ± 0.72; p < 0.01), and self-efficacy to deliver the program (Mpre = 7.93 ± 1.51, Mpost = 9.01 ± 1.00; p < 0.01) all significantly increased from pre- to post e-learning training. Participants reported positively on the user satisfaction and feedback questionnaire with a mean score of 4.58/5 (SD = 0.36). These findings suggest e-learning platforms are a promising mode to improve DPP coaches’ knowledge, counseling skills, and to bolster confidence in program delivery with high levels of satisfaction. E-learning to train DPP coaches allows for effective and feasible expansion of DPP’s, ultimately permitting more reach to adults living with prediabetes.
Supplementary Information
The online version contains supplementary material available at 10.1007/s41347-023-00316-3.
Keywords: E-learning, Motivational interviewing, Diabetes prevention, Training, Self-efficacy, User satisfaction
Introduction
More than 4 million Canadians are currently living with a diabetes diagnosis (type 1 or type 2 diabetes) while nearly 6 million Canadians have prediabetes (Diabetes Canada, 2021). A type 2 diabetes (T2D) diagnosis comes with a number of potential health risks and complications; however, more than half of individuals with prediabetes can prevent or delay T2D onset by managing their blood glucose (Center for Disease Control and Prevention, 2021; Diabetes Canada, 2021). Blood glucose levels can be managed through behavioral modification, including diet and exercise (Knowler et al., 2002; Lindström et al., 2003; Pan et al., 1997). Diabetes prevention programs (DPP) can assist individuals with prediabetes in gaining knowledge and awareness regarding their diagnosis while providing an outlet for support; however, implementation of DPPs can pose challenges (Green et al., 2021; Stokes et al., 2019) as they can be costly (Hernan et al., 2003) and commonly rely on highly educated personnel to facilitate them (Ali et al., 2012; Azzi et al., 2020; Dawes et al., 2015; Knowler et al., 2002; Liddy et al., 2013), limiting access for users.
Small Steps for Big Changes (SSBC) is a practical, evidence-based DPP (Jung et al., 2020). It is currently delivered in the community in partnership with the YMCA of Southern Interior British Columbia (Bean et al., 2021). SSBC utilizes a motivational interviewing (MI) informed counseling style, which is a collaborative, client-centered approach to strengthen a client’s commitment and motivation to change (Miller & Rollnick, 2012). MI has been shown to promote healthy behaviors and improve T2D self-management, quality of life (Chen et al., 2012) and physical activity outcomes (Soderlund, 2018). This counseling style, paired with a structured diet and exercise curriculum, aims to empower individuals with prediabetes to adopt and maintain healthy behavior modifications. The program includes six one-on-one guided exercise and counseling sessions led by a trained SSBC coach.
SSBC has been shown to be effective at eliciting and maintaining diet (Bean et al., 2021) and exercise behavior changes in people with prediabetes (Jung et al., 2020) for 1-year post-program; however, it requires specific training of coaches for it to be delivered with high fidelity. Originally, a 3-day in-person training workshop was held to train SSBC coaches to deliver the program (Dineen et al., 2021). The in-person workshop was developed by a clinical psychologist with additional training in MI and delivered by graduate students to potential SSBC coaches. This workshop included lecture-styled learning and walk-through skill practice over the course of 3 days. However, this process logistically limited training opportunities and SSBC program scale-up due to scheduling conflicts and the inability for training facilitators to travel to multiple locations. Due to these training delivery barriers an online SSBC course was developed to embody the specific qualities and techniques SSBC coaches are expected to incorporate during counseling and exercise sessions with clients (see Appendix A for an outline of course content and screenshots of the online platform).The e-learning course includes seven modules of educational content, with an approximate duration of 3h; followed by a virtual mock client session with a master SSBC coach (a research team member with highly practiced MI techniques who are experienced in delivering the program) and a post-training questionnaire. The e-learning course was developed to replace the in-person training workshop. The content included in the e-learning course was informed by focus groups with past SSBC coaches (trained by the in-person training) and YMCA management meetings. Core components of the training, coach preference of the training delivery and a realistic time commitment for the training were established.
E-learning (electronic learning) refers to any form of education accessible through an electronic device (Ruggeri et al., 2013). E-learning programs are designed with considerations including mode, learning environment, level of independence (Ruggeri et al., 2013), support provided, learning activities offered and aesthetic experience of the platform (Cook, 2005). Electronic delivery allows for standardized delivery, with the capability of updates and improvements to be made in a timely fashion. The SSBC e-learning training course uses a mixture of synchronous and asynchronous content and is delivered in a non-collaborative, off-site online environment. A variety of instructional methods including videos, readings and self-assessment opportunities are included in the learning modules. The SSBC e-learning training was co-developed with a third-party behavioral health technologies company (3C Institute), which allowed for this course to be presented in an aesthetic and professional way through the use of graphics and a modern, practical design.
Evaluation of e-learning platforms used to teach and train healthcare professionals has demonstrated improved practical knowledge, confidence, skills, and program satisfaction compared to baseline (Clancy et al., 2020; Hegerius et al., 2020; Zhang et al., 2020). E-learning platforms for healthcare practitioners have been shown to be highly satisfactory (Hegerius et al., 2020), practical, and efficient for improving knowledge (Moattari et al., 2014; Rhodes et al., 2019; Sranacharoenpong & Hanning, 2012; Wilkinson et al., 2004). Within the context of MI, e-learning platforms have demonstrated increases in MI knowledge, confidence, and skills (Edwards et al., 2015; Lukaschek et al., 2019; Schechter et al., 2021), with positive user acceptability and satisfaction (Fontaine et al., 2016; Schechter et al., 2021). Health professional’s self-efficacy to deliver care to patients has also demonstrated increases post-training (Jones & McEwen, 2015; Karvinen et al., 2017; Nørgaard et al., 2012). While studies have shown that e-learning is effective and positive for various health professionals and students, less is known within the context of DPP coaches and MI.
Evaluation of training is important as it helps measure and evaluate the impact and results of a program. There are various models used to evaluate the effectiveness of training programs; however, most expand off of the Kirkpatrick model in attempts to redefine certain stages and make them more specific to a field or purpose (Choudhury & Sharma, 2019). The Kirkpatrick model was developed for traditional in-person training programs; however, recent suggested adaptations have been made to accommodate e-learning and online training environments (Cook & Ellaway, 2015; Hamtini, 2008). The technology-enhanced learning (TEL) evaluation framework synthesizes the objectives of existing models through a process-oriented approach (Cook & Ellaway, 2015). The TEL evaluation framework (informed by the Kirkpatrick model), consists of seven evaluation areas, and offers additional considerations around training program development, implementation and results (Cook & Ellaway, 2015).
Objective
E-learning platforms have been shown to be an effective mode for teaching content and skills to health professionals; however, less is known within the context of training DPP coaches. The objective of this study was to examine the effectiveness of the online training course for SSBC coaches by assessing learning outcomes and user satisfaction, informed by evaluation activities 3 (usability), 5 (experience and satisfaction) and 6 (learning outcomes) of the TEL evaluation framework (Cook & Ellaway, 2015).
Methods
Study Design
This study was a single-arm pre-post study design with no comparison or control group. All participants were given access to the same training and completed pre- and post-study questionnaires.
Sample Size
A similar study design used to measure MI knowledge was identified in the literature; this study evaluated MI training in a group of rehabilitation professionals naïve to MI. They completed an online MI course and pre-post questionnaires to test their MI knowledge. This group showed improved MI knowledge with a mean pre-training score of 7.7/10 and mean post-training score of 8.5/10 with a pooled SD of 1.15 and Cohen’s d of 0.68 (Schechter et al., 2021). An effect size of 0.68 was used for the present sample size calculation. An a priori power analysis calculation for an effect size of 0.68, at a 2-tailed alpha of 0.05 and power of 0.80 (Cohen, 1992; Karvinen et al., 2017) produced a sample size of 19.
Participants
This study aimed to recruit participants to establish a representative sample of individuals who were seeking to become SSBC coaches. These individuals had full intentions to coach SSBC clients and complete the online training. Participants included existing staff and volunteers at fitness facilities offering the SSBC program and university students interested in volunteering. Participants were recruited through convenience sampling from November 2021 to May 2022. Eligible participants were naïve to the traditional SSBC training and MI informed counseling, over the age of 18, able to read, write, and speak in English and had access to a personal technology device with internet capabilities. Twenty participants were recruited, eleven YMCA staff, five undergraduate students and four graduate students.
Procedures
Individuals interested in becoming a SSBC coach contacted the research team or their fitness facility manager for additional information, eligibility screening and access to consent through Qualtrics. Once participants had agreed to become SSBC coaches a link with access to the e-learning course was sent by email with instructions on how to create a profile on the platform. Participant demographics were collected within the pre-study questionnaire prior to beginning e-learning training modules. The participants were instructed to complete seven asynchronous modules when convenient for them with no time requirements. Once the online modules were completed (expected to be approximately a 3-h duration), a virtual synchronous mock session with a Master Coach was scheduled to assess MI skills and knowledge demonstrated in a hypothetical SSBC session. Coaches were required to pass the mock training session by achieving a client-centered score on all 5 components (partnering, guiding, supporting autonomy & activation, evoking and expressing empathy) of the abbreviated Motivational Interviewing Competency Assessment (MICA; Jackson et al., 2015). Once this was achieved, they were given access to the post-study questionnaire to check their knowledge and satisfaction with the platform. Participants were then informed if they passed the training (scoring at least 70% on their knowledge outcomes in the post-training questionnaire). SSBC coach certification was then sent to participants informing them they had passed the training and could take on SSBC clients.
Measures
To assess effectiveness, pre- and post-training questionnaires were created to assess the content specifically covered in the modules and mock session. Content was categorized into T2D knowledge, MI knowledge, and SSBC content knowledge. Self-efficacy to deliver SSBC was also measured pre- and post-training, while user satisfaction feedback was collected only in the post-training questionnaire. Since the main focus of the training was teaching MI, MI knowledge was chosen as the primary outcome variable.
Demographics
Participants self-reported their age, sex, gender, whether they were born in Canada, whether they identify as Indigenous or as a visible minority, education level completed, occupation status, household income, previous experience in customer service, T2D, and/or counseling training. These questions were informed by the Tri-Hospital + TPH Health Equity Data Collection Research Project Report (2013), CIHR, and Bauer and colleagues (Bauer et al., 2017).
MI Knowledge
Seven multiple-choice questions (collected pre- and post-training) were created covering the MI approach to counseling, the spirit of MI and the foundational skills used during MI.
T2D Knowledge
Nine true-or-false questions (collected pre- and post-training) were used to measure T2D knowledge. Questions assessed general knowledge about prediabetes and T2D adapted from Diabetes Canada (Diabetes Canada, 2021), and SSBC specific questions created by the research team.
SSBC Knowledge
To assess SSBC knowledge, ten multiple-choice questions on the SSBC program sessions and content (collected pre- and post-training) were used. These statements were adapted from past SSBC research questionnaires.
Self-Efficacy to Deliver SSBC
Seven questions assessed participants’ confidence level to deliver SSBC, including both exercise and counseling components with an MI approach (collected pre- and post-training). For each statement, participants were asked to choose a number between 0 and 10, 0 representing not confident I can do it at all, 5 representing moderately confident I can and 10 representing highly confident I can. Study-specific statements were developed with guidance and use of theory-based recommendations for assessing self-efficacy (Blouin et al., 2019), as well as Bandura’s (Bandura, 2005) guide for constructing self-efficacy scales.
User Satisfaction and Engagement
User satisfaction, experience and engagement were assessed in the post-training questionnaire to assess stages 3 and 5 from the TEL evaluation framework (Cook & Ellaway, 2015). These statements referred to feedback on the course content but also with the participants’ experience with the e-learning platform. Participants answered statements on a 5-point Likert scale, with 1 representing totally disagree and 5 representing totally agree. Questions were study-specific and followed outlines and recommendations made by a global model of evaluation for e-learning in health (Ruggeri et al., 2013), electronic learner satisfaction (Sun et al., 2008; Wang, 2003) and Cook and Ellaway’s (2015) Evaluation of Technology-Enhanced Learning Materials: Learners Perceptions (ETELM-LP), an instrument used to assess learners’ perceptions after participating in an online course or training (Cook & Ellaway, 2015).
User engagement Likert-scale questions incorporated aspects such as aesthetic appeal, experience, mental concentration, effort in using the system, and interest in the assigned tasks. These questions followed user engagement scales and models outlined in the literature (Hwang & Thorn, 1999; O’Brien & Toms, 2013).
Additionally, two short response questions were included to allow for participants to reflect on aspects of the e-learning platform that were satisfactory and unsatisfactory. Participants were asked to provide feedback (positive or negative) on their experience with the content included in the course and their experience with the e-learning program. These questions allowed participants to give any additional feedback that may not have been covered in the multiple-choice statements and provide them with a chance to expand on their responses to the satisfaction statements.
Analysis
All demographics, outcomes, and metrics were collected on the e-learning platform, with all data accessible using the researchers’ administrative accounts. Statistical Package for Social Sciences (SPSS version 27) was used to analyze participant demographics, e-learning, and feedback outcomes. All participant demographics were analyzed using descriptive statistics.
To assess whether there were significant changes in knowledge and self-efficacy after completion of the training course, pre- and post-training questionnaires were compared through paired t-tests with a significance value of 0.05 and confidence interval of 95%. All assumptions were satisfied prior to running the paired t-tests. Cohen’s d was calculated in SPSS to report on effect size and used to measure the magnitude of relationships between variables (Cohen, 1992). The user satisfaction and engagement question responses were descriptively analyzed. The open-ended responses were assessed through content analysis by counting the number of statements provided and organizing them into categories of positive feedback, negative feedback, and recommendations/additions.
Results
Of the 20 participants who completed the pre-training questionnaire, everyone completed the mock session and post-training questionnaire. One additional individual was deemed ineligible due to their previous experience in the SSBC program and knowledge of the coach training program. The sample population lacked diversity in some categories including sex, gender, and identifying as a visible minority. There was more variability in trainer experience, education level, and household income. Twelve of the participants were under the age of 30, five of the participants were between the ages of 30 and 50 and one participant was over 55. All demographic results are reported in Table 1.
Table 1.
Sample characteristic | N (%) | M | SD |
---|---|---|---|
Age (years) | 20 (100) | 29 | 9.42 |
18–24 | 8 (40) | ||
25–29 | 4 (20) | ||
30–39 | 5 (25) | ||
40–49 | 2 (10) | ||
50–59 | 1 (5) | ||
Sex | |||
Female | 16 (80) | ||
Male | 4 (20) | ||
Gender | |||
Woman | 16 (80) | ||
Man | 4 (20) | ||
Born in Canada | |||
Yes | 7 (35) | ||
No | 10 (50) | ||
Prefer not to answer | 3 (15) | ||
Identifies as Indigenous | |||
No | 20 (100) | ||
Identifies as a visible minority | |||
Yes | 3 (15) | ||
No | 17 (85) | ||
Level of education | |||
High school | 1 (5) | ||
Apprenticeship, trades certificate or diploma | 1 (5) | ||
College, CEGEP, or other non-university certificate or diploma | 3 (15) | ||
University certificate or diploma below the bachelor level | 6 (30) | ||
Post-graduate degree | 2 (10) | ||
Currently in some university or college | 7 (35) | ||
Prefer not to answer | |||
Working status | |||
Working full time Working part-time |
6 (30) 3 (15) |
||
Student | 10 (50) | ||
Other | 1 (5) | ||
Annual income level | |||
$0 to $29,999 | 4 (20) | ||
$30,000 to $59,999 | 5 (25) | ||
$60,000 to $89,999 | 2 (10) | ||
$120,000 to $149,999 | 1 (5) | ||
$150,000 or more | 1 (5) | ||
Do not know | 7 (35) | ||
# of people income supports | 11 (55) | 2.00 | 1.34 |
Prefer not to answer | 7 (35) | ||
Do not know | 2 (10) | ||
Previous Experience | |||
Discipline in school | |||
Exercise science/kinesiology | 11 (55) | ||
Other | 4 (20) | ||
Not currently in school | 5 (25) | ||
Years in customer service | 19 (95) | 7.47 | 9.83 |
No response | 1 (5) | ||
Years in health context | 19 (95) | 3.05 | 4.64 |
No response | 1 (5) | ||
Years in chronic disease | 20 (100) | 1.60 | 3.73 |
Previous counseling training | 20 (100) | ||
Yes | 25 | ||
No | 75 |
N, number of participants who selected this response; %, percent of participants who selected this response; M, mean; SD, standard deviation
Two participants failed their initial mock session but passed on their second attempt. Two additional participants failed their initial post-training questionnaire knowledge check but passed on their second attempt. For one of these participants, a whole page of questions was blank in their responses; this may have been a technical glitch or accidental error; however, to avoid making any assumptions, all initial responses were used in the pre-post analysis. One participant took three attempts to pass the post-training questionnaire but was only one question away from a pass in both the first and second attempts. The pass rate for coaches was high with 15/20 passing on their first attempt in both the mock session and post-training questionnaire. For the post-training questionnaire analysis, participants’ scores were collected from their first attempt regardless of whether or not they passed the threshold of knowledge gained for SSBC coach certification.
Consistent with primary hypotheses, statistically significant increases were seen for MI knowledge (Mpre = 3.30 ± 1.95, Mpost = 5.90 ± 1.29; p < 0.01), SSBC content knowledge (Mpre = 5.15 ± 2.23, Mpost = 8.60 ± 0.94; p < 0.01), T2D knowledge (Mpre = 6.95 ± 1.57, Mpost = 8.25 ± 0.72; p < 0.01) and self-efficacy to deliver the program (Mpre = 7.93 ± 1.51, Mpost = 9.01 ± 1.00; p < 0.01), see Table 2. All participants reported positively on the user satisfaction and feedback questionnaire with a mean score of 4.58/5 (SD ± 0.36).
Table 2.
Pre-training | Post-training | |||||||
---|---|---|---|---|---|---|---|---|
M | SD | Range | M | SD | Range | P-value | Effect size | |
MI (0–7) |
3.30 | 1.95 | 0–6 | 5.90 | 1.29 | 2–7 | < 0.01 | 1.39 |
T2D (0–9) |
6.95 | 1.57 | 4–9 | 8.25 | 0.72 | 7–9 | < 0.01 | 0.83 |
SSBC (0–10) |
5.15 | 2.23 | 0–9 | 8.60 | 0.94 | 7–10 | < 0.01 | 1.35 |
SE (0–10) |
7.93 | 1.51 | 5–10 | 9.01 | 1.00 | 6–10 | < 0.01 | 1.03 |
M, mean; SD, standard deviation; MI, MI knowledge ranges from 0 to 7; T2D, T2D knowledge ranges from 0 to 9; SSBC, SSBC content knowledge ranges from 0 to 10; SE, self-efficacy to deliver the program ranges from 0 (not confident I can do it at all), 5 (moderately confident I can) to 10 (highly confident I can)
The user satisfaction and engagement questions were further broken down into four categories. These four categories included content satisfaction, user engagement, usability, and technical experience, see Table 3. The mean scores for these categories ranged from 4.51 to 4.69, all demonstrating high levels of satisfaction and acceptability. On average, the highest rated category was usability and the lowest rated was user engagement.
Table 3.
Questionnaire category | Number of questions | M | Range | SD |
---|---|---|---|---|
Content satisfaction | 5 | 4.62 | 3.80–5.00 | 0.45 |
User engagement | 9 | 4.51 | 3.67–5.00 | 0.44 |
Usability | 6 | 4.69 | 3.83–5.00 | 0.39 |
Technical experience | 7 | 4.53 | 3.86–5.00 | 0.36 |
Total | 27 | 4.58 | 3.85–5.00 | 0.36 |
N, 18; M, mean; SD, standard deviation; All categories range from 1 (totally disagree) to 5 (totally agree)
To be cautious, data from two participants was excluded prior to analysis of user satisfaction and engagement due to the unlikely pattern of their responses. A few statements in the satisfaction questionnaire were phrased in the negative to reduce response bias and served as attentiveness questions to see whether respondents were paying attention to the whole survey. Since both participants scored every question as a five, it is likely that they were unreliable respondents for these questions. The data from these participants was removed from analysis for the entirety of the questionnaire to ensure results were as accurate as possible.
Of the 20 participants, ten answered the open-ended question about content and eight answered the open-ended question about the e-learning platform. Within the content responses seven statements were positive, one statement was negative, and six recommendations were recorded. The e-learning platform feedback included a higher incidence of negative responses at four, with eight positive responses and one recommendation for improvement. The positive responses highlighted the flexibility and benefits of e-learning and the online resources, and there were also comments noted around ease of use and clarity. Some negative comments expressed difficulty and confusion around the mock session process while others noted their preferences around face-to-face learning as opposed to an online environment. Recommendations for improvements or changes to the learning experience included linking the corresponding resource documents to the module they were mentioned in, inclusion of additional scenarios and examples for coaches to help them prepare for a variety of situations with clients and the addition of subtitles was suggested to go alongside the videos in the modules. Some of these changes can be made through the administrative portal on the e-learning platform and others will include financial implications. All of this feedback can help inform future iterations of the platform.
Discussion
In the context of DPP coaches, these findings suggest that e-learning platforms are a promising mode of training. This e-learning course increased MI and DPP-specific knowledge, confidence to deliver the SSBC program (see Figs. S1-S4 attached in Appendix B) and demonstrated high levels of user satisfaction. These findings align with similar studies demonstrating success in e-learning programs designed for healthcare professionals (Clancy et al., 2020; Hegerius et al., 2020; Zhang et al., 2020) and for MI skills, knowledge, and confidence (Schechter et al., 2021).
Participants’ knowledge significantly increased from pre- to post-training, regardless of their starting point or level of knowledge before the start of the training. Participants demonstrated high levels of self-efficacy (perceived capability) to deliver the SSBC program post-training. These results are similar to other studies that have assessed changes in self-efficacy after healthcare professionals participated in training programs (Edwards et al., 2015; Jones & McEwen, 2015; Karvinen et al., 2017; Malan et al., 2015; Nørgaard et al., 2012). Self-efficacy is an important assessment of changes in performance and behavior (Bandura, 1997) and an effective predictor of student motivations and learning (Bandura et al., 1996; Zimmerman, 2000). The knowledge and skills that participants acquired during the online training may have contributed to the statistically significant increases in self-efficacy to deliver the program. If participants would have recorded low levels of self-efficacy to deliver the program, adjustments to the knowledge, skills, and tools provided in the training may have needed to be considered.
User satisfaction and acceptability levels were high. When broken down into categories of content satisfaction, user engagement, usability, and technical experience, all results demonstrated similar levels of ratings. The high levels in satisfaction and acceptability with the e-learning program was as hypothesized and consistent with previous research within the context of health care professionals (Clancy et al., 2020; Connan et al., 2019; Cook et al., 2008; Hegerius et al., 2020; Sranacharoenpong & Hanning, 2012). The technical experience and usability of the platform were important to gauge how well the user could navigate the course, and the content included in the training was highly satisfactory as well. The length of the modules, the amount of information offered, and the content and educational resources themselves were rated satisfactory and learning objectives were achieved by users.
Strengths and Limitations
The strengths of this study included a sufficient sample size, real-world population, and application. The findings demonstrate increases in knowledge and self-efficacy outside of a laboratory setting and with unstructured specifications. High effect sizes were reported for all pre-post outcome measures, (all above 0.8; Cohen, 1992) emphasizing the practical significance of this study and representing a large difference between the pre- and post-training mean scores. A similar study (used to inform the effect size for the sample size calculation for this thesis) resulted in a medium effect size of 0.68 for pre-post study MI knowledge scores (Schechter et al., 2021). Comparatively, our effect size was 1.39 for improvements in MI knowledge. Our large effect size may be due to the quality, amount, and delivery of MI content in the SSBC e-learning training platform.
Short study questionnaires and the flexibility offered to coaches while completing the training allowed for minimal participant burden. This study can act as a guide for rigorous evaluation methods of e-learning programs as it used a comprehensive and detailed framework to guide the evaluation (the TEL evaluation model; Cook & Ellaway, 2015). Despite the COVID-19 pandemic, this study successfully recruited a sample size adequately powered to test a priori hypotheses. All participants were planning on becoming SSBC coaches and they were only invited to be part of this study after they had signed up for the online training.
Over the course of the past 2 years, the field of e-learning has been particularly relevant for schools and universities as in-person learning was forced to pivot to an online environment. This study acts as an example of how to increase content knowledge with a mode of learning that is accessible, feasible, and flexible for learners. The delivery was also deemed satisfactory by users. Furthermore, there were no educational requirements to participate in this study, which allowed for individuals of any level of education to participate. Inequity in educational opportunities and achievement have been linked to socioeconomic status (SES; Breen and Jonsson, 2005; Gamoran and Long, 2007; James, 2001). This study breaks down the potential barriers in accessibility to e-learning training opportunities for individuals with low SES and may increase the opportunity for career development.
As identified in the literature review, e-learning research to date has focused on students in healthcare or health professionals. This study extends what is known about e-learning and who it can be applied to. To our awareness, this study represents one of the first e-learning evaluations with health and fitness professionals with little to no prior education or counseling experience.
Although this study had many strengths, additional considerations and limitations exist. The sample in this study lacked diversity and equity-owed groups were not reached. There were also no time restrictions for participants to complete the training, and thus, there was a large range in duration of training among participants. This data was not analyzed for the purpose of this study; however, future consideration of training duration and user engagement metrics such as bounce rate and completion of learning activities could offer additional insights into the training process and experience, especially if participants did not pass initial attempts of the mock session or post-training questionnaire. Although this study demonstrated effectiveness of the online training, it did not account for the relative value of the e-learning program compared to existing or alternative methods. Ruggeri and colleagues (Ruggeri et al., 2013) and Cook and Ellaway (Cook & Ellaway, 2015) mention the importance of evaluating e-learning courses and platforms through all stages, from development through to implementation and dissemination. Future considerations outside of SSBC may include exploring other study designs to account for external factors that have the potential to influence results. For example, testing effects could have been accounted for by adding a control group. Limited controls for this study could be seen as a limitation; however, this study design and pragmatic approach were chosen due to the nature of SSBC and its current delivery in the community.
Future Directions
SSBC can have confidence in this mode of training for increasing knowledge and self-efficacy to deliver the program, while feedback from the satisfaction questionnaire and open-ended responses can help make improvements to the platform. As mentioned above, it is important to evaluate all stages of course development, implementation, and results; we plan to continue the evaluation for SSBC coach training. Future research within e-learning training could also include looking at various study designs, either adding a control group, adding additional measurement time-points or dose maintenance activities needed to account for retention of knowledge, skills, and self-efficacy to deliver the program and sustainability over time. Extending beyond change in knowledge, evaluation of behavioral change in coaches and whether coaches with higher post-training knowledge in MI translate to better client outcomes could be assessed (by focusing on Kirkpatrick levels 3 and 4). The data garnered from these future studies can also be used to help improve the training further.
Future considerations around diversifying the sample and reporting ethnicity are important for data collection. It is important to report these demographics, and there are also future opportunities to highlight equity-owed communities and populations in e-learning research. There may be differences in e-learning needs across diverse populations.
Implications
This study demonstrated that the online training was effective at teaching DPP delivery, and increasing coaches’ MI, SSBC content and T2D knowledge, and self-efficacy to deliver SSBC. All the information provided in the e-learning course was appropriate to train individuals with various education and experiential backgrounds. This training was developed for individuals outside of health care specialists, and all participants successfully completed the training, highlighting the potential of this e-learning program to be integrated into a variety of community fitness facilities across Canada whose staff members and volunteers have similar backgrounds and experiences to this study’s sample. E-learning opens the delivery of training to a larger group of people rather than relying on highly qualified personnel, inclusive of the non-academic and non-healthcare population, thereby increasing the availability and reach of diabetes prevention care. These findings address a gap in the literature by looking at learners who are outside of the healthcare professionals and students who have higher levels of experience and education in the health context. By removing educational barriers for T2D prevention coaches, SSBC is able to increase the number of individuals who are eligible to provide quality diabetes prevention care to those in need.
High levels of satisfaction and acceptability were seen by coaches who engaged with the training program. SSBC can have confidence to expand to more communities in an effective and sustainable manner, while maintaining high program fidelity. Ultimately, this will allow SSBC to reach more Canadians living with prediabetes. Outside of SSBC, this study may also aid in a better understanding of e-learning programs for DPPs, chronic disease prevention, and MI. Understanding the components of a successful model for e-learning and evaluation of these programs can help guide future research, programs, and interventions. These findings can translate to e-learning training platforms for DPPs by offering insights into the effectiveness, acceptability, and features of coach training platforms. As the world continues to transition into online settings, the relevance of this study can be applied to various fields.
Supplementary Information
Below is the link to the electronic supplementary material.
Acknowledgements
We respectfully acknowledge that the work and research conducted for this study were on the traditional, ancestral, and unceded territory of the Sylix Nation. This research was possible due to the participation of Small Steps for Big Changes coaches and clients, as well as funding from CIHR and the Heart and Stroke Foundation.
Author Contribution
All authors contributed to the overall study design and conception. Material preparation, data collection, and analysis were performed by Natalie Grieve. The first draft of the manuscript was written by Natalie Grieve and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Funding
This study was funded by a Canadian Institutes of Health Research Grant (#020438) and a Heart and Stoke Foundation of Canada Grant (#011442).
Data Availability
De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.
Declarations
Ethics Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the University of British Columbia Behavioral Research Ethics Board (H21-01800).
Consent to Participate
Informed written consent to participate was obtained for this study.
Consent for Publication
Informed consent for publication was obtained for this study.
Competing Interests
The authors declare no competing interests.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- Ali MK, Echouffo-Tcheugui JB, Williamson DF. How effective were lifestyle interventions in real-world settings that were modeled on the Diabetes Prevention Program? Health Affairs. 2012;31(1):67–75. doi: 10.1377/hlthaff.2011.1009. [DOI] [PubMed] [Google Scholar]
- Azzi, J. L., Azzi, S., Lavigne-Robichaud, M., Vermeer, A., Barresi, T., Blaine, S., & Giroux, I. (2020). Participant evaluation of a prediabetes intervention program designed for rural adults. Canadian journal of dietetic practice and research: A publication of Dietitians of Canada = Revue canadienne de la pratique et de la recherche en dietetique : une publication des Dietetistes du Canada, 81(2), 80–85. 10.3148/cjdpr-2019-033 [DOI] [PubMed]
- Bandura, A. (1997). Self-efficacy: The exercise of control. W H Freeman/Times Books/Henry Holt & Co.
- Bandura, A. (2005). A guide for constructing self-efficacy scales. Self-Efficacy Beliefs of Adolescent., 307–337.
- Bandura A, Barbaranelli C, Caprara GV, Pastorelli C. Multifaceted impact of self-efficacy beliefs on academic functioning. Child Development. 1996;67(3):1206–1222. doi: 10.2307/1131888. [DOI] [PubMed] [Google Scholar]
- Bauer, G. R., Braimoh, J., Scheim, A. I., & Dharma, C. (2017). Transgender-inclusive measures of sex/gender for population surveys: Mixed-methods evaluation and recommendations. PLoS ONE, 12(5), e0178043. 10.1371/journal.pone.0178043 [DOI] [PMC free article] [PubMed]
- Bean C, Dineen T, Locke SR, Bouvier B, Jung ME. An evaluation of the reach and effectiveness of a diabetes prevention behaviour change program situated in a community site. Canadian Journal of Diabetes. 2021;45(4):360–368. doi: 10.1016/j.jcjd.2020.10.006. [DOI] [PubMed] [Google Scholar]
- Blouin JE, Cary MA, Marchant MG, Gyurcsik NC, Brittain DR, Zapski J. Understanding physiotherapists’ intention to counsel clients with chronic pain on exercise: A focus on psychosocial factors. Physiotherapy Canada. 2019;71(4):319–326. doi: 10.3138/ptc-2018-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Breen R, Jonsson JO. Inequality of opportunity in comparative perspective: Recent research on educational attainment and social mobility. Annual Review of Sociology. 2005;31:223–243. doi: 10.1146/annurev.soc.31.041304.122232. [DOI] [Google Scholar]
- Center for Disease Control and Prevention. (2021). Retrieved from: https://www.cdc.gov/diabetes/basics/prediabetes.html
- Chen SM, Creedy D, Lin H-S, Wollin J. Effects of motivational interviewing intervention on self-management, psychological and glycemic outcomes in type 2 diabetes: A randomized controlled trial. International Journal of Nursing Studies. 2012;49(6):637–644. doi: 10.1016/j.ijnurstu.2011.11.011. [DOI] [PubMed] [Google Scholar]
- Choudhury, G.B., & Sharma, V. (2019). Review and comparison of various training effectiveness evaluation models for R & D organization performance. PM World Journal, 8(2). https://pmworldlibrary.net/wp-content/uploads/2019/02/pmwj79-Feb2019-Choudhury-Sharma-comparison-of-training-effectiveness-models-for-rd.pdf
- Clancy, E. M., McIntosh, J., Booth, A. T., Sheen, J., Johnson, M., Gibson, T., Bennett, R. N., & Newman, L. (2020). Training maternal and child health nurses in early relational trauma: An evaluation of the MERTIL workforce training. Nurse Education Today, 89, 104390. 10.1016/j.nedt.2020.104390 [DOI] [PubMed]
- Cohen J. A power primer. Psychological Bulletin. 1992;112(1):155–159. doi: 10.1037/0033-2909.112.1.155. [DOI] [PubMed] [Google Scholar]
- Connan V, Marcon MA, Mahmud FH, Assor E, Martincevic I, Bandsma RH, Vresk L, Walsh CM. Online education for gluten-free diet teaching: Development and usability testing of an e-learning module for children with concurrent celiac disease and type 1 diabetes. Pediatric Diabetes. 2019;20(3):293–303. doi: 10.1111/pedi.12815. [DOI] [PubMed] [Google Scholar]
- Cook DA. The research we still are not doing: An agenda for the study of computer-based learning. Academic Medicine: Journal of the Association of American Medical Colleges. 2005;80(6):541–548. doi: 10.1097/00001888-200506000-00005. [DOI] [PubMed] [Google Scholar]
- Cook DA, Ellaway RH. Evaluating technology-enhanced learning: A comprehensive framework. Medical Teacher. 2015;37(10):961–970. doi: 10.3109/0142159X.2015.1009024. [DOI] [PubMed] [Google Scholar]
- Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: A meta-analysis. JAMA. 2008;300(10):1181–1196. doi: 10.1001/jama.300.10.1181. [DOI] [PubMed] [Google Scholar]
- Dawes D, Ashe M, Campbell K, Cave D, Elley CR, Kaczorowski J, Dawes M. Preventing diabetes in primary care: A feasibility cluster randomized trial. Canadian Journal of Diabetes. 2015;39(2):111–116. doi: 10.1016/j.jcjd.2014.08.004. [DOI] [PubMed] [Google Scholar]
- Diabetes Canada. (2021). Retrieved from: https://www.diabetes.ca
- Dineen, T. E., Banser, T., Bean, C., & Jung, M. E. (2021). Fitness facility staff demonstrate high fidelity when implementing an evidence-based diabetes prevention program. Translational Behavioral Medicine, ibab039. 10.1093/tbm/ibab039 [DOI] [PMC free article] [PubMed]
- Edwards EJ, Stapleton P, Williams K, Ball L. Building skills, knowledge and confidence in eating and exercise behavior change: Brief motivational interviewing training for healthcare providers. Patient Education and Counseling. 2015;98(5):674–676. doi: 10.1016/j.pec.2015.02.006. [DOI] [PubMed] [Google Scholar]
- Fontaine, G., Cossette, S., Heppell, S., Boyer, L., Mailhot, T., Simard, M.-J., & Tanguay, J.-F. (2016). Evaluation of a web-based e-learning platform for brief motivational interviewing by nurses in cardiovascular care: A pilot study. Journal of Medical Internet Research, 18(8), e224. 10.2196/jmir.6298 [DOI] [PMC free article] [PubMed]
- Gamoran, A., & Long, D. A. (2007). Equality of educational opportunity a 40 year retrospective. In R. Teese, S. Lamb, M. Duru-Bellat, & S. Helme (Eds.), International Studies in Educational Inequality, Theory and Policy (pp. 23–47). Springer Netherlands. 10.1007/978-1-4020-5916-2_2
- Green G, DeFosset AR, Sivashanmugam M, Mosst J, Kuo T. Current practices, facilitators, and barriers experienced by program providers implementing the National Diabetes Prevention Program in Los Angeles County. Translational Behavioral Medicine. 2021;11(2):430–440. doi: 10.1093/tbm/ibaa033. [DOI] [PubMed] [Google Scholar]
- Hamtini, T. M. (2008). Evaluating e-learning programs: An adaptation of Kirkpatrick’s model to accommodate e-learning environments. Journal of Computer Science, 4(8), 693–698.
- Hegerius A, Caduff-Janosa P, Savage R, Ellenius J. E-learning in pharmacovigilance: An evaluation of microlearning-based modules developed by uppsala monitoring centre. Drug Safety. 2020;43(11):1171–1180. doi: 10.1007/s40264-020-00981-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hernan, W. H., Brandle, M., Zhang, P., Williamson, D. F., Matulik, M. J., Ratner, R. E., Lachin, J. M., Engelgau, M. M., & Diabetes Prevention Program Research Group Costs associated with the primary prevention of type 2 diabetes mellitus in the diabetes prevention program. Diabetes Care. 2003;26(1):36–47. doi: 10.2337/diacare.26.1.36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hwang MI, Thorn RG. The effect of user engagement on system success: A meta-analytical integration of research findings. Information & Management. 1999;35(4):229–236. doi: 10.1016/S0378-7206(98)00092-5. [DOI] [Google Scholar]
- Jackson C, Butterworth S, Hall A, Gilbert J. Motivational Interviewing Competency Assessment. 2015;3:23. [Google Scholar]
- James R. Participation disadvantage in Australian higher education: An analysis of some effects of geographical location and socioeconomic status. Higher Education. 2001;42(4):455–472. doi: 10.1023/A:1012264010667. [DOI] [Google Scholar]
- Jones LL, McEwen A. Evaluating an online training module on protecting children from secondhand smoke exposure: Impact on knowledge, confidence and self-reported practice of health and social care professionals. BMC Public Health. 2015;15:1132. doi: 10.1186/s12889-015-2488-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jung ME, Locke SR, Bourne JE, Beauchamp MR, Lee T, Singer J, MacPherson M, Barry J, Jones C, Little JP. Cardiorespiratory fitness and accelerometer-determined physical activity following one year of free-living high-intensity interval training and moderate-intensity continuous training: A randomized trial. The International Journal of Behavioral Nutrition and Physical Activity. 2020;17:25. doi: 10.1186/s12966-020-00933-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Karvinen KH, Balneaves LG, Courneya KS, Perry B, Truant T, Vallance J. Evaluation of online learning modules for improving physical activity counseling skills, practices, and knowledge of oncology nurses. Oncology Nursing Forum. 2017;44(6):729–735. doi: 10.1188/17.ONF.729-738. [DOI] [PubMed] [Google Scholar]
- Knowler WC, Barrett-Connor E, Fowler SE, Hamman RF, Lachin JM, Walker EA, Nathan DM. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. New England Journal of Medicine. 2002;346(6):393–403. doi: 10.1056/NEJMoa012512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liddy CE, Cullen-Arseneau P, Merizzi S, Blazhko V. “An ounce of prevention”: A primary care based prevention program for pre-diabetic population. Canadian Journal of Diabetes. 2013;37(1):12–17. doi: 10.1016/j.jcjd.2013.01.006. [DOI] [PubMed] [Google Scholar]
- Lindström, J., Louheranta, A., Mannelin, M., Rastas, M., Salminen, V., Eriksson, J., Uusitupa, M., Tuomilehto, J., & for the Finnish Diabetes Prevention Study Group The Finnish Diabetes Prevention Study (DPS) Diabetes Care. 2003;26(12):3230–3236. doi: 10.2337/diacare.26.12.3230. [DOI] [PubMed] [Google Scholar]
- Lukaschek, K., Schneider, N., Schelle, M., Kirk, U. B., Eriksson, T., Kunnamo, I., Rochfort, A., Collins, C., & Gensichen, J. (2019). Applicability of motivational interviewing for chronic disease management in primary care following a web-based e-learning course: Cross-sectional study. JMIR Mental Health, 6(4), e12540. 10.2196/12540 [DOI] [PMC free article] [PubMed]
- Malan Z, Mash R, Everett-Murphy K. Qualitative evaluation of primary care providers experiences of a training programme to offer brief behaviour change counselling on risk factors for non-communicable diseases in South Africa. BMC Family Practice. 2015;16:101. doi: 10.1186/s12875-015-0318-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller, W. R., & Rollnick, S. (2012). Motivational interviewing: Helping People change. Guilford Press.
- Moattari M, Moosavinasab E, Dabbaghmanesh MH, ZarifSanaiey N. Validating a web-based diabetes education program in continuing nursing education: Knowledge and competency change and user perceptions on usability and quality. Journal of Diabetes & Metabolic Disorders. 2014;13(1):70. doi: 10.1186/2251-6581-13-70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nørgaard B, Ammentorp J, Ohm Kyvik K, Kofoed P-E. Communication skills training increases self-efficacy of health care professionals. Journal of Continuing Education in the Health Professions. 2012;32(2):90–97. doi: 10.1002/chp.21131. [DOI] [PubMed] [Google Scholar]
- O’Brien HL, Toms EG. Examining the generalizability of the user engagement scale (UES) in exploratory search. Information Processing & Management. 2013;49(5):1092–1107. doi: 10.1016/j.ipm.2012.08.005. [DOI] [Google Scholar]
- Pan X-R, Li G-W, Hu Y-H, Wang J-X, Yang W-Y, An Z-X, Hu Z-X, Juan-Lin X, J.-Z., Cao, H.-B., Liu, P.-A., Jiang, X.-G., Jiang, Y.-Y., Wang, J.-P., Zheng, H., Zhang, H., Bennett, P. H., & Howard, B. V. Effects of diet and exercise in preventing NIDDM in people with impaired glucose tolerance: The Da Qing IGT and diabetes study. Diabetes Care. 1997;20(4):537–544. doi: 10.2337/diacare.20.4.537. [DOI] [PubMed] [Google Scholar]
- Rhodes, D., Visker, J., Larson, K., & Cox, C. (2019). Rapid e-learning for professional development in school-based diabetes management. Nurse Education in Practice; Kidlington, 38, 84–88. http://dx.doi.org.ezproxy.library.ubc.ca/10.1016/j.nepr.2019.06.009 [DOI] [PubMed]
- Ruggeri K, Farrington C, Brayne C. A global model for effective use and evaluation of e-learning in health. Telemedicine and E-Health. 2013;19(4):312–321. doi: 10.1089/tmj.2012.0175. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schechter N, Butt L, Jacocks C, Staguhn E, Castillo R, Wegener ST. Evaluation of an online motivational interviewing training program for rehabilitation professionals: A pilot study. Clinical Rehabilitation. 2021;35(9):1266–1276. doi: 10.1177/02692155211002958. [DOI] [PubMed] [Google Scholar]
- Soderlund PD. Effectiveness of motivational interviewing for improving physical activity self-management for adults with type 2 diabetes: A review. Chronic Illness. 2018;14(1):54–68. doi: 10.1177/1742395317699449. [DOI] [PubMed] [Google Scholar]
- Sranacharoenpong K, Hanning RM. Diabetes prevention education program for community health care workers in Thailand. Journal of Community Health. 2012;37(3):610–618. doi: 10.1007/s10900-011-9491-2. [DOI] [PubMed] [Google Scholar]
- Stokes J, Gellatly J, Bower P, Meacock R, Cotterill S, Sutton M, Wilson P. Implementing a national diabetes prevention programme in England: Lessons learned. BMC Health Services Research. 2019;19(1):991. doi: 10.1186/s12913-019-4809-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sun P-C, Tsai RJ, Finger G, Chen Y-Y, Yeh D. What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education. 2008;50(4):1183–1202. doi: 10.1016/j.compedu.2006.11.007. [DOI] [Google Scholar]
- Wang Y-S. Assessment of learner satisfaction with asynchronous electronic learning systems. Information & Management. 2003;41(1):75–86. doi: 10.1016/S0378-7206(03)00028-4. [DOI] [Google Scholar]
- Wilkinson A, Forbes A, Bloomfield J, Fincham Gee C. An exploration of four web-based open and flexible learning modules in post-registration nurse education. International Journal of Nursing Studies. 2004;41(4):411–424. doi: 10.1016/j.ijnurstu.2003.11.001. [DOI] [PubMed] [Google Scholar]
- Zhang, C. X., Marshall, J., Bernard, A., & Walker-Smith, K. (2020). Development and evaluation of interprofessional e-learning for speech pathologists, interpreters and translators. Translation & Interpreting, 12(1), 142–158. 10.12807/ti.112201.2020.a09
- Zimmerman BJ. Self-efficacy: An essential motive to learn. Contemporary Educational Psychology. 2000;25(1):82–91. doi: 10.1006/ceps.1999.1016. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.