Assessing the Feasibility, Acceptability, and Preliminary Effectiveness of a School-Aged Program that Supports Physical Activity and Wellness

Abstract

The prevalence of anxiety symptoms in children and adolescents aged 4 to 18 years has nearly doubled after the first year of the pandemic. However, only one in five adolescents diagnosed with anxiety is treated. We R H.O.P. E. is a school-based mental health program that includes evidence-based principles designed to engage children and adolescents in anxiety treatment, including wellness and emotional regulation, and the emotional CPR method. We R H.O.P. E. augments traditional services provided by school administrators, school social workers, school teachers, and school nurses. The purpose of this study was to examine the feasibility, acceptability, and preliminary effectiveness of We R H.O.P. E.

Introduction

Before the COVID-19 pandemic, 11.6% of children and adolescents were diagnosed with generalized anxiety disorder [1]. Generalized anxiety disorder is defined by the Diagnostic and Statistical Manual V as excessive distress out of proportion with a given situation and manifested by physical symptoms [2]. However, the prevalence of children and adolescent anxiety symptoms has nearly doubled to 20.5% after the first year of the pandemic [3]. Currently, 1 in 5 children and adolescents are experiencing clinically elevated anxiety symptoms [3], likely due to disruption of their everyday lives. School closures and quarantine orders have led to social isolation, missed milestones, increased family stress, and decreased peer interactions [4]. However, only 1 in 5 children and adolescents with anxiety symptoms are treated for anxiety disorders due to lack of outward signs and difficulty of children and adolescents to identify when anxiety is not normal (2018 Children’s Mental Health Report, n.d.). As such, there is a critical need to ensure the other 80% are treated.

Childhood and adolescent anxiety disorders can lead to additional psychiatric issues such as anxiety disorders and depressive disorders, as well as suicidal ideation later in life [5]. Anxiety disorders in childhood and adolescence are also associated with less life satisfaction, poor family relationships, educational underachievement, poor adjustment at work, and alcohol use disorders [6, 7]. Additionally, adolescent anxiety predicts physical health issues such as coronary heart disease [8] and adult-onset asthma [9]. It is currently estimated there is a $5890 direct cost and $4658 indirect cost annually for families of children and adolescents with anxiety [10].

Cognitive behavior therapy (CBT) combined with antidepressant medication is considered the gold standard of treatment and is the most effective for treating separation anxiety, generalized anxiety, and social anxiety disorders in children and adolescents [11]. However, despite empirical evidence supporting specific anxiety treatments for children and adolescents stigma, negative beliefs toward mental health services and professionals, lack of knowledge, unavailability of services, and structural barriers decrease help-seeking behavior [12, 13]. As such, non-clinical school-based services provided by lay professionals or “coaches” may be an effective method to offset barriers associated with accessing evidence-based services and augment services provided by school administrators, school social workers, school teachers, and school nurses.

This study was guided by the social ecological model, which conceptualized health broadly. The social ecological model focuses on multiple factors that affect health including physical, social, mental, and social well-being (World Health Organization, 1947). The model acknowledges health to be affected by the interaction between the individual, group/community, and the physical, social, and political environments [14, 15].

The purpose of this study was to examine the feasibility, acceptability, and preliminary effectiveness of We R H.O.P. E., a school-based mental health program that includes evidence-based principles designed to engage children and adolescents in anxiety treatment, including wellness and emotional regulation, and the emotional CPR method.

Methods

A single-arm pre/post study was conducted in collaboration with We R H.O.P.E, a school-based individual coaching service. We R H.O.P. E. has created a youth mental health support program and coaching through school-based anxiety coaching services that provide daily individualized coaching sessions for children and adolescents who need support, regardless of financial resources. The school-based anxiety coaching services focus on normalizing anxiety within the school community and providing children and adolescents with skills and tools to manage anxiety. We R H.O.P. E. incorporates psychoeducation, emotional regulation, and emotional CPR (eCPR). eCPR teaches people how to assist others and work through emotional crises through connecting, empowering, and revitalizing. eCPR is based on the recovery care model of mental health and the principles of recovery: trauma-informed care, counseling after disasters, peer support, emotional intelligence, suicide prevention, and cultural attunement. eCPR was developed using an iterative design process by an expert panel of peer support specialists, nonprofit leaders, and people with a lived experience of mental health conditions. eCPR is delivered by a lay professional (non-clinically trained) using a manualized workbook. Empirically eCPR is associated with increased empowerment, hope, and quality of life and decreased incidence of psychiatric symptoms and loneliness [16].

Study instruments were administered at baseline and 90 days. Both the baseline and 90-day assessments were administered in person in a private setting. No incentives were provided. The study was approved by the (blinded for review) Institutional Review Board. A secondary data analysis was conducted.

Participants

The study included N = 191 children ages 6 through 16 years from 14 (blinded for review) schools. Of the 197 participants, 77 completed both the pre and post-instruments. Of the 56 participants who completed the pre and post–instrument, 49 identified as female, 27 identified as male, and 1 responded as “other”.

Instruments

Study instruments were administered in person at baseline and 90 days by a trained rater. A trained master-level individual entered the data into excel and then transferred to STATA for analyses. The General Anxiety Disorder-7 (GAD-7) was used to measure generalized anxiety disorder, which has shown reliability and validity in measuring anxiety in children and adolescents [17]. The GAD-7 asks participants, “Over the last two weeks, how often have you been bothered by the following problems?” Sample problems include “Feeling nervous, anxious, or on edge,” “Not being able to stop or control worrying,” “Worrying too much about different things,” and “Trouble relaxing.” Response options are on a zero to three scale, including “Not at all,” “Several days,” “More than half the days,” and “Nearly every day.” GAD-7 scores range from 0 to 21 with low scores indicating minimal anxiety and high scores indicating severe anxiety (0–4 minimal anxiety, 5–9 mild anxiety, 10–14 moderate anxiety, 15–21 severe anxiety).

Fidelity Assessment

We R H.O.P. E. research team monitored intervention fidelity through (1) biweekly discussions between coaches and supervisors; and (2) supervisors also observed a minimum of one in-person session over the 90 day intervention with each coach and provided an evaluation of their coaching work.

Informed Consent

As this is a secondary data analysis of de-identified data, consent was not requested from participants. This secondary data analysis involved no more than minimal risk to the participants, did not adversely affect the rights and welfare of the participants, and could practically be carried out without the waiver of consent.

Statistical Analyses

Descriptive statistics were used to describe the demographic characteristics of the study sample. A 2-tailed paired sample t-tests was conducted to assess the difference between pre scores and post scores for statistical significance at 90 days as well as 30 and 60 days. Repeated measure ANOVA analysis was performed to evaluate performance improvements over time. Incomplete survey responses were excluded from the analyses. STATA was used to compute descriptive statistics and analyses.

Results

Sociodemographic Characteristics of the Study Sample

Of the 197 total people who participated, 77 participants completed the GAD-7 survey at baseline and at 90-days. Among the 77 participants assessed, 64% were female (n = 49), 36% were male (n = 27), and 1 responded “other”. Participants ranged in age from 6 years old to 16 years old and had an average age of 9.9 years old. This study population heavily favored white respondents as they accounted for 92% of all races represented in the data (See Table 1).

Table 1.

Characteristics of included study participants (n = 77)

Age, years
	Mean (SD)	9.9 (2.6)
	Range	10
Sex, n (%)
	Female	49 (64)
	Male	27 (36)
	Other	1 (1.3)
Race, n (%)
	American Indian or Alaska Native	0 (0)
	Asian	1 (1.3)
	Black / African American	1 (1.3)
	Native Hawaiian or Pacific Islander	0 (0)
	White	71 (92)
	Two or more races	4 (5.2)

Feasibility and Acceptability

Of the 197 participants, 39% (n = 77) completed both the intake and 90-day Generalized Anxiety Disorder-7 (GAD-7), indicating a low level of survey responses. Low levels of pre/post measures also indicate data collection procedure was not feasible or acceptable. Data collection should be modified to ensure participants complete both pre and post-measures. Recommendations to enhance data collection involve providing incentives to the participants, making post-measures easy and fast to complete, and tracking participants to ensure they complete both the pre and post-measures.

Preliminary Effectiveness

A paired sample t-test was run on a sample of 77 participants of the We R. H.O.P.E. intervention. The purpose of this study was to determine whether there is a statistically significant mean difference between GAD-7 scores from baseline to 90 days (Table 2). Participants who went through the study had a mean intake GAD-7 score of 10.2 with a standard deviation of 5.4. At 90 days the mean score went down to 6.7 with a standard deviation of 4.7; a statistically significant decrease of 3.2 (95% CI 2.1—4.4, p < 0.001).

Table 2.

Changes in Outcomes from Baseline to 90 Days for Study Participants—Mean (SD)

Measure	Baseline	90 Days	Score Δ	pValue
GAD-7	10.2 (8.9,11.3)	6.7 (5.9,8.0)	-3.2 (-2.1,-4.4)	< 0.001

As data was collected during the study at different time points as well, change in GAD-7 from baseline was also analyzed at 30 and 60 days using paired sample t-tests. (Table 3) Statistically significant decreases in GAD-7 score also occurred at point time points with a 1.8 (95% CI 0.5–3.0, p = 0.006) and 2.4 (95%CI 1.3–3.5, p = 0.001) mean score difference at 30 and 90 days respectively. These findings were confirmed with a repeated measure ANOVA test which confirms there are statistically significant different scores at each time point, with each one showing marked improvement (F(4,68) = 9.69, p < 0.001).

Table 3.

Changes from baseline at other time intervals

Timepoint	Baseline	Score at timepoint	Score Δ	n Missing	p Value
30-Days	10.3 (9.1,11.5)	8.5 (7.4,9.6)	-1.8 (-0.5,-3.0)	1	0.006
60-Days	10.1 (8.9,11.4)	7.7 (6.6,8.8)	-2.4 (-1.3,-3.5)	2	0.001
90-Days	10.2 (8.9,11.3)	6.7 (5.9,8.0)	-3.2 (-2.1,-4.4)	0	< 0.001

Further repeated measure ANOVA test confirms there is a statistical significance in performance between time points F(4,68) = 9.69, p < 0.001

Discussion

The purpose of this study was to examine the feasibility, acceptability, and preliminary effectiveness of We R H.O.P. E., a school-aged program that supports physical activity and wellness. Preliminary effectiveness was assessed using the General Anxiety Disorder-7 (GAD-7). The study demonstrated that the school-based We R H.O.P. E. program is a promising approach to decrease anxiety in children and adolescents in everyday environments that interact in. The We R H.O.P.E. program was associated with a significant decrease in participant anxiety levels after 90 days. As such, school-based interventions that offer lay professional support services, in combination with traditional school services, may be an ideal approach for dissemination and uptake of evidence-based mental health treatment.

Feasibility and acceptability by We R H.O.P. E coaches was demonstrated through their capacity to deliver evidence-based components of the program with fidelity. The We R H.O.P. E manual and supervision sessions enabled coaches to link students' needs and preferences to standardized evidence-based intervention components. This study highlights promising findings that manualized school-based interventions provided by coaches may facilitate the delivery of evidence-based practices. Understanding the impact of We R H.O.P.E. individually and in combination with services provided by school social workers, school teachers, and school nurses may help delineate the impact of We R H.O.P.E. as a stand-alone or augmentative service.

The feasibility and acceptability of We R H.O.P. E was established through participants’ attendance. Specifically, participants met with a coach a total of 5–7 times. These findings suggest We R H.O.P. E represents a promising strategy to evidenced-based anxiety treatment outside of a clinical setting. However, feasibility and acceptability of data collection was not obtained as less than half of the participants completed the post measure at 90 days. This indicates either a low level of engagement by the participants over a sustained amount of time or infeasibility of data collection. Data collection should be modified to ensure participants complete both pre and post-measures. Providing incentives to the participants, making post-measures easy and fast to complete, and tracking participants to ensure they complete both the pre and post measures are recommended to increase student engagement and data collection.

Due to a decrease in anxiety levels associated with the We R H.O.P.E intervention, schools may provide the ideal setting to help decrease anxiety because they are already accessed daily by children and adolescents. This makes it easy for students and mental health providers to stay in continuous contact. Additionally, school-based mental health services may reduce anxiety levels by reducing stigma around mental health services [18]. When students see their classmates using mental health services, they may feel more comfortable also receiving help, normalizing the use of treatment services. When stigma is reduced, it also leads to more positive attitudes towards mental health services and professionals making students more likely to seek treatment. School-based mental health programs may also reduce anxiety levels by increasing academic outcomes, providing earlier identification and intervention for mental health issues, creating a more positive school climate, increasing psychosocial outcomes, and encouraging engagement between peers, families, and educators [19].

Limitations

This study is not without limitations and results should be interpreted with caution. Of the 197 students who participated, only 77 completed the pre and post-test. Thus, creating a potential sampling bias. Additionally, the pre/post-test design does not allow for a causal relationship between the We R H.O.P.E. intervention and reduced anxiety scores. However, as this is the first study of the We R H.O.P.E. intervention, this design is aligned with pilot studies to assess the feasibility and acceptability of our approach [20]. Also, the lack of a control group makes it impossible to conclude that the decrease in anxiety scores was due to the intervention itself and not the passage of time. However, these findings are promising, and examining We R H.OP.E. with a fully-powered sample and a rigorous design is an important next step.

Implications for Practice

This pilot study added promising evidence to the potential effectiveness of school-based programs that support physical activity and wellness. School-based interventions meet children and adolescents in their socio-cultural environments and offer additional (non-traditional) support services. School administrators, school social workers, school teachers, and school nurses can have an important role in supervising coaches delivery of services and support a comprehensive social ecological approach to support children and adolescents in schools. Further research with a fully-powered sample and a rigorous design may advance our understanding of school-based programs and their impact on systems and children and adolescents.

Biographies

Stephanie R. Lebby

is currently getting a master’s of science in physical activity and wellness at the University of Vermont. She is a graduate of Dartmouth College with a degree in psychology. Her current research focuses on improving mental health and chronic health conditions through physical activity, education, awareness, and preventative medicine.

Amanda Myers, MPH

is a researcher with skills and expertise in health policy, user-centered design, usability, user experience, mental health, and community engagement. She has 14 publications on these topics and is also a member of the Digital Peer Support Team.

Dr. Andrew R. Bohm

is an adjunct instructor in The Dartmouth Institute. He has an extensive background in Healthcare Research and received his M.S. in Healthcare Research at Geisel School of Medicine at Dartmouth.

Dr. Karen L. Fortuna

is an Assistant Professor of Psychiatry at Dartmouth College. Through the Collaborative Design for Recovery and Health, Dr. Fortuna works in equal partnership with patient partners in co-producing and empirically testing digital peer support technologies and trainings. Dr. Fortuna has over 85 peer-reviewed publications co-produced with patient partners. Dr. Fortuna serves on the American Psychiatric Association’s Expert Advisory panel on Smartphone App Development and co-Chairs PCORI’s Advisory Panel on Patient Engagement. She serves as editor of the Journal of Participatory Medicine and the Shared Wisdom feature in the American Journal of Geriatric Psychiatric.

Funding

The authors received no financial support for the research, authorship, or publication of this article.

Declarations

Conflict of Interest

Dr. Fortuna provides consulting through Social Wellness and Emissary Health, Inc.