Skip to main content
NCBI home page
Preventive Medicine Reports logoLink to Preventive Medicine Reports
. 2023 Feb 11;32:102137. doi: 10.1016/j.pmedr.2023.102137

A RE-AIM analysis of the sources of strength program in a United States high school

Duke D Biber a,⁎, Gina Brandenburg b
PMCID: PMC9925958  PMID: 36798792

Highlights

  • •

    Sources of Strength can improve protective factors in high school students.

  • •

    This Sources of Strength RE-AIM analysis provides guidance for future translation.

  • •

    The reach, adoption, implementation, and maintenance of this study were high.

  • •

    Continuous fidelity training for mentors and student voice can enhance programming.

  • •

    Rigorous efficacy and maintenance metrics are needed to enhance future programming.

Keywords: Mental health, Suicide, Adolescents, Risk factors, School

Abstract

The purpose of this study was to analyze the Sources of Strength program implemented at a high school in the southeastern United States using the RE-AIM framework. Students (n = 1,250) were recruited from one high school in the southeastern United States that was implementing Sources of Strength. College undergraduate students mentored high school peer leaders to implement Sources of Strength to promote opioid and suicide protective factors for the high school. This was a cross-sectional design in which a post-implementation survey assessed program effectiveness, satisfaction, and feedback for maintenance. Analysis used the RE-AIM framework of reach, effectiveness, adoption, implementation, and maintenance. Reach: 100 % of students participated in at least one campaign. Effectiveness: Mean protective factor scores were 3.83 for happiness; 3.93 for knowledge of coping skills, and 3.38 for practice of coping skills. Adoption: One high school and 100 % of classrooms participated in at least one campaign. Implementation: 18 high school peer leader meetings were facilitated by the undergraduate mentors. Maintenance: High school students qualitatively provided input on future campaign needs, and the program was continued following this RE-AIM analysis. The Sources of Strength program promoted suicide and opioid protective factors in high school students during COVID-19. Future research should compare the efficacy of Sources of Strength at baseline and follow-up with multiple high school sites.

1. Introduction

Over the last two decades, opioid-related deaths increased by 150–250 % in adolescents. (Gaither et al., 2018) Risk factors for opioid misuse include suicidality, unsupportive social environments, mental health problems, academic issues, and bullying. (Hedegaard et al., 2018, Karch et al., 2013, Webster, 2017, Wilkins et al., 2021) Risk factors for suicide and opioid misuse were exacerbated during the COVID-19 pandemic, emphasizing the need for protective factors. (Jones et al., 2022) Protective factors include supportive family relationships, trusted adults at school, and healthy peer connections. (Conner et al., 2016, Whitlock et al., 2014) Sources of Strength is a suicide and opioid prevention mentoring program that effectively enhances protective factors in adolescent students through school-wide culture of help-seeking and connectedness between peers and caring adults. (LoMurray, 2005, Wyman et al., 2010) However, ongoing evaluation, especially in light of COVID-19 is needed. The purpose of this study was to evaluate a school-wide Sources of Strength program using the RE-AIM framework.

2. Methods

This study was implemented in compliance with guidelines for the protection of human subjects and following approval by the researchers’ Institutional Review Board.

2.1. Design

This was a cross-sectional design with post-intervention data collection.

2.2. Sample

Students and teachers from one rural, southeastern high school in the United States participated in the program. The high school had a total of 1,250 enrolled students and 85 teachers.

2.3. Measures

Participants completed self-report measures of emotionality, social support, coping skills, and suicidality via Qualtrics, all of which were measured on a 5-point Likert scale. The response options on the 5-point Likert scale ranged from 1 (strongly disagree) to 5 (strongly agree).

2.4. Intervention

The Sources of Strength program was implemented for one year. A university team consisting of faculty and undergraduate students facilitated the program. The undergraduate students recruited a diverse group of adult and peer mentors from the high school to develop and implement programming. The high school team received a two-day Sources of Strength training. The undergraduate mentors then met bi-weekly with the high school peer leader team to ensure fidelity to campaign implementation. The peer leader team implemented monthly, school-wide campaigns focused on family support, positive friends, mentors, healthy activities, generosity, spirituality, physical health, and mental health. (Sources of Strength, 2022) The goal of each campaign was to promote suicide and opioid protective factors through inclusive, school-wide messaging.

2.5. Analysis

The RE-AIM framework was used to evaluate this program, including analysis of reach, effectiveness, adoption, implementation, and maintenance. (Gaglio et al., 2013) Reach was measured as the number of students and teachers that participated in the program in relation to the total number of students and teachers in the school. The effectiveness was the impact of the program on outcomes (i.e., emotionality, social support, coping skills, and suicidality), evaluated through a year-end, post-program survey sent to all students. Adoption was the number and proportion of settings who participated in the program and included the number of sites and classrooms within the high school that participated in the program. Implementation was fidelity to the Sources of Strength program protocol and measured through frequency of training. Maintenance, the degree to which the program became part of the organization’s regular practices, was assessed as intent and preparedness to continue Sources of Strength after the end of the year. The results were analyzed using means, standard deviations, and percentages of representation.

3. Results

3.1. Reach

100 % of students (1,250 student participants/1,250 total students) participated in at least one Sources of Strength campaign. 5,912 students (including repeating students) participated in 10 campaigns (M = 591.2 students/campaign) over one year. An average of 47.30 % high school students participated in each campaign. 100 % of teachers engaged in at least one campaign (n = 85), with an average of 35.29 % of teachers per campaign.

3.2. Effectiveness

A total of 52 % of students (n = 650) completed emotionality, suicidality, social support, and coping measures after one year of implementation. On a 5-point scale, participants reported the following mean scores: happiness = 3.83 (SD = 0.96), fear = 1.67 (SD = 0.98), anger = 2.44 (SD = 1.11), hopelessness = 1.70 (SD = 1.04), self-harm = 1.12 (SD = 0.52), thoughts of death = 1.40 (SD = 0.94), thoughts of suicide = 1.25 (SD = 0.76), and suicide attempts = 1.04 (SD = 0.32). Participants reported means of 3.93 (SD = 1.29) for knowledge of coping skills and 3.38 (SD = 1.50) for practice of coping skills.

3.3. Adoption

Only one high school participated in the first year of implementation. 100 % of classrooms (n = 85) participated in at least one campaign

3.4. Implementation

A two-day, Sources of Strength training was provided to the university faculty, undergraduate mentor team, and the high school adult mentors and peer leaders. The undergraduate mentors also facilitated 18 team training meetings with the high school peer leader team to ensure fidelity of campaign implementation.

3.5. Maintenance

The high school received additional training after year one of implementation for all staff and students. A portion of the budget was allocated to the high school to purchase supplies for the following year of programming, allowing for program maintenance. Feedback from the high school students was elicited through an open-ended Qualtrics question asking how Sources of Strength could continue to support students. The key themes included 1) healthy coping strategies (“Help some of the students make friends, give journals so the kids can write down their feelings, or give them stress balls.”), 2) school culture of acceptance (“Create a positive environment that every-one feels welcome and accepted.”), and 3) adult support (“Just be there when I need to talk or need help. Always being there to check on me at least every once in a while.”). While Sources of Strength was continued by high school adult and peer leaders after the first year of implementation, reach and effectiveness data was not collected because the funding period had ended. However, the fact that the high school site continued implementation indicates maintenance of Sources of Strength.

4. Discussion

The mission of program implementation was to promote protective factors by way of a school-wide culture of help-seeking and connectedness between peers and caring adults through well-designed messaging campaigns. The reach for the program was high because 100 % of high school students participated in at least one campaign. While a strong appraisal of effectiveness was not possible without a comparison group, pre-post testing, or effect size, the post-implementation outcome results for emotionality, suicidality, social support, and coping were moderate-to-high. Participating students reported high scores for positive emotions and coping and low scores for negative emotions and suicidality after one year of program implementation. Previous RE-AIM research has utilized “low”, “moderate”, and “high” evaluations of effectiveness using a 5-point Likert scale. (Biber and Ellis, 2016) Although this was not a pre- and post-evaluation of the Sources of Strength program, subjective participant survey response can be used as a form of evaluating effectiveness to inform future programming. (RE-AIM, 2022) However, future implementation of Sources of Strength should use pre-post testing with comparison groups to assess program effectiveness more vigorously. Adoption was high as 100 % of classrooms in the school participated in the program. Implementation was high based on the frequency of trainings provided throughout the year. Maintenance was high given that extra training and resources was provided for successive years, and because a team of high school staff adult leaders and student peer leaders continued the Sources of Strength program the following year. This was the first RE-AIM analysis of Sources of Strength. Previous studies found positive programmatic impacts on student protective factors, and this analysis supports those findings during COVID-19. (Wyman et al., 2010, Petrova et al., 2015).

4.1. Limitations

There are key limitations that should be addressed for future implementation and evaluation. First, only one site participated in the program. It is necessary to use RE-AIM to evaluate this program across several high school sites in various geographic regions. Effectiveness was assessed using self-report, post-program measures only. There is a need to assess program effectiveness with multiple time points, a comparison group, and validated instruments. Student participant focus groups could also add depth to the evaluation of the program.

4.2. Significance

This is the first RE-AIM evaluation of Sources of Strength and can guide future implementation. The results highlight the successful reach, adoption, and implementation of the Sources of Strength during COVID-19, during which high school students may have needed stronger peer and adult connections. More vigorous assessment of program effectiveness is needed to understand the direct impact of Sources of Strength on emotionality, suicidality, social support, and coping skills. That said, the undergraduate university mentorship model seems an effective way to engage high school peer leaders in program implementation and campaign participation.

Clinical Trial Registration: This is not a clinical trial.

Study Approval: Study was approved by the University of West Georgia IRB (irb@westga.edu).

Funding acknowledgement: This project was funded by the Georgia Department of Behavioral Health and Developmental Disabilities.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.

References

  1. Biber D.D., Ellis R. A RE-AIM evaluation of a team-based physical activity program for university employees. Int. J. Sport Stud. 2016;6:90–94. [Google Scholar]
  2. Conner K.R., Wyman P., Goldston D.B., Bossarte R.M., Lu N., Kaukeinen K., Tu X.M., Houston R.J., Lamis D.A., Chan G., Bucholz K.K., Hesselbrock V.M. Two studies of connectedness to parents and suicidal thoughts and behavior in children and adolescents. J. Clin. Child Adolesc. Psy. 2016;45:129–140. doi: 10.1080/15374416.2014.952009. [DOI] [PubMed] [Google Scholar]
  3. Gaglio B., Shoup J.A., Glasgow R.E. The RE-AIM framework: A systematic review of use over time. Am. J. Public Health. 2013;103:e38–e46. doi: 10.2105/AJPH.2013.301299. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Gaither J.R., Shabanova V., Leventhal J.M. US national trends in pediatric deaths from prescription and illicit opioids 1999–2016. JAMA Netw. Open. 2018;1:e186558. doi: 10.1001/jamanetworkopen.2018.6558. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Hedegaard H, Curtin SC, Warner M. Suicide rates in the United States continue to increase. US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics, 2018.
  6. Jones S.E., Ethier K.A., Hertz M., DeGue S., Le V.D., Thornton J., Geda S. Mental health, suicidality, and connectedness among high school students during the COVID-19 pandemic—Adolescent Behaviors and Experiences Survey, United States, January–June 2021. MMWR Suppl. 2022;71:16. doi: 10.15585/mmwr.su7103a3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Karch DL, Logan J, McDaniel DD, Floyd CF, Vagi KJ. Precipitating circumstances of suicide among youth aged 10–17 years by sex: Data from the National Violent Death Reporting System, 16 States, 2005–2008. J. Adolesc. Health 2013;53:S51–S53. [DOI] [PubMed]
  8. LoMurray M. The North Dakota Suicide Prevention Project; Bismarck, ND: 2005. Sources of Strength Facilitators Guide: Suicide prevention peer gatekeeper training. [Google Scholar]
  9. Petrova M., Wyman P.A., Schmeelk-Cone K., Pisani A.R. Positive-themed suicide prevention messages delivered by adolescent peer leaders: Proximal impact on classmates' coping attitudes and perceptions of adult support. Suicide Life Threat. Behav. 2015;45:651–663. doi: 10.1111/sltb.12156. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. RE-AIM (2022). Assessing effectiveness. Available at: https://re-aim.org/assessing-effectiveness/ (Accessed 2 January 2023).
  11. Sources of Strength. Adult Advisor Field Guide. Available at https://sourcesofstrength.org/wp-content/uploads/FieldGuide_2019_web.pdf. Accessed September 1, 2022.
  12. Webster L.R. Risk factors for opioid-use disorder and overdose. Anesth. Analg. 2017;125:1741–1748. doi: 10.1213/ANE.0000000000002496. [DOI] [PubMed] [Google Scholar]
  13. Whitlock J., Wyman P.A., Moore S.R. Connectedness and suicide prevention in adolescents: Pathways and implications. Suicide Life Threat. Behav. 2014;44:246–272. doi: 10.1111/sltb.12071. [DOI] [PubMed] [Google Scholar]
  14. Wilkins NJ, Clayton H, Jones CM, Brown M. Current prescription opioid misuse and suicide risk behaviors among high school students. Pediatr 2021;147:e2020030601. [DOI] [PMC free article] [PubMed]
  15. Wyman P.A., Brown C.H., LoMurray M., Schmeelk-Cone K., Petrova M., Yu Q., Wang W. An outcome evaluation of the Sources of Strength suicide prevention program delivered by adolescent peer leaders in high schools. Am. J. Public Health. 2010;100:1653–1661. doi: 10.2105/AJPH.2009.190025. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Data will be made available on request.

Articles from Preventive Medicine Reports are provided here courtesy of Elsevier

RESOURCES