Feasibility and Acceptability of the Youth Aware of Mental Health (YAM) Intervention in US Adolescents

Abstract

Objective:

Suicide is the second leading cause of death among US adolescents and rates of suicide among youth have been increasing for the past decade. This study assessed the feasibility and acceptability of the universal, school-based Youth Aware of Mental Health (YAM) program, a promising mental health promotion and suicidality primary prevention intervention, in US youth.

Method:

Using an uncontrolled design, the feasibility and acceptability of delivering and studying YAM were assessed in 5 Montana and 6 Texas schools.

Results:

Thirteen of 16 (81.3%) schools agreed to support YAM delivery. Facilitators delivered YAM in 78 classes (1878 students) as regular high school curriculum. Of the total number of students who received YAM, 519 (27.6%) provided parental consent and assent. 436 (84.0%) consented students participated in pre- and post-surveys. Students, parents, and school staff found YAM highly acceptable based on satisfaction surveys.

Conclusions:

This study found YAM feasible to implement in US schools. Results also suggest students, parents, and school staff supported school-based programs and were highly satisfied with the YAM program. A randomized controlled trial is warranted to test the efficacy of YAM in promoting mental health and preventing suicidal thoughts and behaviors in US adolescents.

Suicide is second only to accidental deaths in causing fatality in adolescents aged 12–18, and is a growing public health problem (Centers for Disease Control and Prevention, 2017c). While suicide attempt rates are similar across the US (8.6%), in some states, like Montana, fatal suicide rates among adolescents aged 12–18 years are well above the national average (16.5/100,000 versus 5.9/100,000) (Centers for Disease Control and Prevention, 2017a, 2017b). Texas has a higher suicide attempt rate (10.1%) according to results from the most recent Youth Risk Behavior Surveillance System survey (Centers for Disease Control and Prevention, 2017a) but a similar fatal suicide rate to the national average (5.7/100,000) (Centers for Disease Control and Prevention, 2017b). Since 2010, the suicide rate has been increasing nationwide, and represents thousands of adolescent deaths each year (Centers for Disease Control and Prevention, 2017b).

Youth suicide prevention consists of 3 tiers: universal (primary), selected (secondary), and indicated (tertiary) (D.N. Miller, Eckert, & Mazza, 2009). Primary prevention interventions, delivered to all youth before the onset of suicidality, often focus on social skill-building, reducing stigma, and increasing awareness about suicide, help-seeking and mental health knowledge (David Neil Miller, 2011; Wilcox & Wyman, 2016). Primary preventive interventions are cost effective, can be delivered to all youth, and do not require screening (Brent, 2019). The goal of primary suicide prevention is to reduce incident cases of suicidality (D.N. Miller et al., 2009). Selected interventions often focus on identifying and linking at-risk youth to services and increasing social support to prevent suicidal behaviors (D.N. Miller et al., 2009). These interventions often use strategies such as treating depression with psychotherapy, pharmacological interventions; reducing access to lethal means; and modifying media coverage of suicides (Mann et al., 2005). Finally, indicated interventions are designed for youths with current or a history of suicidal behaviors. Evidence-based interventions such as dialectical behavioral therapy or cognitive behavioral therapy can be used to reduce suicidal behaviors in indicated youth (D.N. Miller et al., 2009). Most youth-based suicide prevention interventions have focused on individuals in crisis, though a growing number of universal suicide prevention interventions have been developed and evaluated in recent years (Wilcox & Wyman, 2016).

Universal, school-based interventions are attractive candidates for reducing suicide among youth because they offer 2 important benefits as a suicide prevention strategy. First, school-based interventions can be delivered in a critical developmental period: before or early in the onset of mental illnesses, which develop in 50% of affected individuals by age 14 and 75% by age 24 (Kessler et al., 2005; Merikangas et al., 2010), and are a major risk factor for suicidal behaviors (Brent, Baugher, Bridge, Chen, & Chiappetta, 1999; Kessler, Borges, & Walters, 1999; Shaffer et al., 1996). Second, delivering in schools allows access to nearly all youth in a population, especially when delivered as part of the regular school curriculum (David Neil Miller, 2011).

Evidence supporting the efficacy of school-based suicide prevention interventions has been increasing. The Good Behavior Game (GBG), a universal primary prevention intervention delivered in early elementary school, has demonstrated reductions in suicidal thoughts and behaviors in participants through age 30 (Wilcox et al., 2008). A randomized controlled trial (RCT) of Family Check-Up, a stepped intervention, also showed a reduction in suicide risk through age 28–30 years among those receiving this selected intervention (Connell, McKillop, & Dishion, 2016). Other school-based interventions have shown positive outcomes over shorter timeframes (3–18 months). RCTs of Signs of Suicide (SOS) showed reductions in suicide attempts over 3 months (Aseltine & DeMartino, 2004; Aseltine, James, Schilling, & Glanovsky, 2007; Schilling, Aseltine, & James, 2016). Lastly, Sources of Strength showed positive perceptions of adult helpfulness particularly among youth with history of suicide and non-significant reductions in suicidal ideation 4 months post-intervention (Wyman et al., 2010).

The Youth Aware of Mental Health (YAM) intervention is a promising universal, school-based mental health promotion and suicide primary prevention intervention for adolescents (Wasserman et al., 2015). YAM is designed to raise mental health awareness about risk and protective factors associated with suicide, including knowledge about depression and anxiety, and to enhance the skills and emotional resiliency needed to deal with adverse life events, stress, and suicidal behaviors (Wasserman et al., 2010). The format of the YAM intervention, which includes student role plays, empowers youth to think about, verbalize, and discuss important stressors and mental health concerns, such as depression and suicide, in a context that is meaningful to them. YAM was recently evaluated in an RCT of ~11,000 9^th graders in 10 European countries (Wasserman et al., 2015). The YAM group experienced significantly reduced suicidality, including 55% fewer incident suicide attempts and 50% fewer cases of severe suicidal ideation, compared to control, over 1 year (Wasserman et al., 2015).

While schools receiving YAM demonstrated marked decreases in suicidal thoughts and attempts in Europe, it has not been adapted for or tested in US youth. All countries have distinct cultural and social attributes, which can affect attitudes toward mental health. Cultural adaptations of mental health interventions and treatments are needed particularly when there are specific risk and protective factors within a population (Hall, Ibaraki, Huang, Marti, & Stice, 2016). For example, individuals with mental disorders are more likely to use mental health services and seek other forms of help when information and assistance are provided in culturally-relevant contexts (Bhui & Bhugra, 2004; Carter, Read, Pyle, & Morrison, 2017; Rathod, Kingdon, Phiri, & Gobbi, 2010). In multiple studies, adapting mental health interventions and treatments so language and content are targeted for specific ethnic and cultural backgrounds resulted in improved efficacy relative to usual care or other interventions (Degnan et al., 2017; Hall et al., 2016), with a direct association between efficacy and the level of adaptation performed (Degnan et al., 2017). While the superiority of adapted over unadapted interventions and treatments has not been established definitively, evidence is growing that cultural adaptations of mental health interventions may improve outcomes (Chowdhary et al., 2014; Degnan et al., 2017; Hall et al., 2016).

Feasibility studies aim to measure factors important for intervention implementation (recruitment, data collection) and delivery (fidelity) to inform the design of future, larger-scale RCTs, which then definitively test the efficacy of the intervention (Bowen et al., 2009; Leon et al., 2011). As an initial step towards evaluating the impact of YAM in the US, the present study determined the feasibility and acceptability of the intervention when adapted for and delivered as part of the regular school curriculum in primarily 9^th grade classrooms in Montana and Texas.

METHODS

Study Design

An uncontrolled, within-subjects study was conducted to determine feasibility and acceptability of delivering and studying the YAM intervention in Montana and Texas schools.

Participants and Settings

Students were recruited from 5 public schools in Montana and 1 public and 6 charter schools in Texas (Figure S1). Students were eligible to participate in questionnaires if their principals gave permission for the school to receive YAM, they and their parents provided written informed assent/consent, and they attended a class in which YAM was delivered. Schools were approached based on convenience, but with an emphasis on inclusion of large and small schools fitting multiple geographic regions (Montana) and schools with a focus on diversity and demographics (Texas). YAM was delivered as part of the regular school curriculum primarily to students in 9^th grade. However, some classes contained students in multiple grades. The demographics of the study population are shown in Tables S1, S2, and S3.

Ethical considerations.

The Institutional Review Boards of Montana State University and the University of Texas Southwestern Medical Center approved the study.

School recruitment.

Schools were initially approached with emails describing the YAM program and research project, and a request to meet if the school had interest. Larger school systems were approached first at the administration level, then at the school level. In-person meetings were usually 45–60 minutes and typically included the state site PI or Co-I, key school administrators, teachers, and counselor(s). After schools agreed to participate, 60–90-minute community meetings were held in all communities. In all but 2 Montana schools, additional 20–30-minute meetings were held with the entire teaching faculty. Advertising targeted parents of those students who would be receiving YAM, though all were welcomed.

Consent and assent.

Informed consent and assent forms were sent home with students in participating classes, and/or mailed or emailed to parents. Research staff gave verbal and written instructions about the study and students’ rights immediately prior to survey administration. Students were provided a $5 incentive for returning signed consent and assent forms in Montana, regardless of whether they or their parent declined participation. Texas students were offered: homework pass, free dress day, breakfast to class with most returned consents, or no incentive. For parents and school staff, satisfaction surveys contained an informed consent statement, and survey return was considered granting of consent. Consent rates were calculated as the percentage of students for whom parental consent and student assent were granted.

Intervention

The YAM intervention has been described previously (Wasserman et al., 2010; Wasserman et al., 2015). Briefly, YAM consists of 5 ~hourly sessions, which include 3 role-play sessions, 2 mental health interactive lectures, an information booklet for students, and 6 posters. All students in participating classes received YAM, but only those providing consent/assent completed surveys.

US adaptation.

Because YAM was developed and tested in European students, it required some cultural adaptations for US youth. YAM facilitators were trained in a 5-day course designed by the YAM developers. The course included review of all materials, practice sessions to deliver YAM content, theory behind YAM, and practice moderating role plays with discussion and feedback. YAM helpers received 3–4 hours of training by certified YAM facilitators.

Prior to the study delivery of YAM, 2 separate groups of 5 students, comprised of 10^th and 11^th graders were recruited in Montana to aid the adaptation of YAM. Written informed consent/assent was received from parents/students for youth who participated in the cultural adaptation. Adaptation consisted of 7 steps: 1) YAM facilitators delivered an entire course of YAM to each of the 2 student groups; 2) youth provided written and verbal detailed feedback about the YAM program and materials; 3) the US study team summarized feedback and recommended adjustments to the original YAM manual and materials; 4) recommended adjustments were reviewed by the YAM originators (C. Wasserman, V. Carli), and in collaboration with the US team, initial revisions were proposed; 5) written and verbal detailed feedback was obtained from a single group of 6 students (who participated in the initial 2 student groups) on the proposed revisions; 6) a second set of recommended adjustments were developed based on the feedback from the single group of 6 students; 7) the second set of adjustments were reviewed by the YAM originators, and in collaboration with the US team, a final version was developed and approved by the YAM originators.

YAM delivery.

Facilitators trained by the YAM developers, delivered the 5 YAM sessions to individual classes over 3 to 5 weeks, guided by a detailed manual (modified for US youth) as described previously (Wasserman et al., 2010). A YAM helper assisted each facilitator. YAM delivery occurred between Oct. 2016 and May 2017.

Measures

Feasibility.

Feasibility was evaluated as follows. School participation was specified as the percentage of schools that implemented YAM delivery and assessments. Consent rates were defined as the percentage of students in participating classes (total study population) who provided written assent/consent. Youth participation was determined by the percentage of consented students who completed ≥80% of any scale on baseline and 3-month follow-up surveys. Survey completion was defined as responses to ≥ 80% of items on all survey scales. For each scale, the number of students who completed ≥ 80% of items also was quantified.

Quality control of YAM delivery.

Prior to delivery, facilitators completed a quality control questionnaire adopted from the SEYLE RCT of YAM (Wasserman et al., 2010) assessing the preparedness of each site for the intervention. It included: names of school’s principal and counselor; method of consent distribution; consent form return rates; number of baseline questionnaires distributed and completed; whether school had any prior student suicidal behavior; if another mental health/suicide educational program had been or was currently being provided; class period duration; mean number of students/classroom; and whether teachers/principals had provided students with scripted information about the YAM study.

YAM fidelity.

Fidelity to YAM included two components: intervention adherence and dose (Proctor et al., 2011). Adherence to the YAM intervention was measured by a fidelity form which facilitators completed at the close of each session (Wasserman et al., 2010). The form included the following: facilitator and helper names; rating of availability/accessibility of school contacts; date with any deviations noted; duration, content, and location of session; whether all students received the YAM booklet; YAM poster locations and availability, participation in role plays with mention of any students left out; attendance of participants and reasons for absences if known (Montana only); and adverse events with a description of how each was handled. To support fidelity to the YAM model, facilitators participated in twice-monthly and as-needed group consultation by phone (Montana) or in-person (Texas). Facilitators from both states were in contact throughout the study to discuss questions impacting delivery.

The second fidelity component, dose of the intervention, was defined by the number of sessions attended by students. In Montana, facilitators logged the attendance of consented students for each session. In Texas, facilitators recorded the total number of students who attended each session. Both states recorded attendance of consented students during survey administration.

Determination of acceptability.

Students, parents, teachers, and school staff/administrators were asked to complete anonymous questionnaires assessing their satisfaction of the YAM intervention. Student satisfaction surveys (8 items) were administered as part of the 3-month follow-up survey (Table S4). Parental surveys (12 items) were sent home with participating students or mailed at the time of the 3-month follow-up (Table S5). Teacher and staff/administrator surveys (8-items and 9-items, respectively) were delivered in-person or via mail near the end of the final semester in which YAM was delivered (Tables S6 and S7). All surveys contained 5 items (Likert-like scale with 1=strongly disagree to 5=strongly agree) asking YAM-related satisfaction questions and included additional questions specific for each group (Tables S4–S7).

Outcomes

Feasibility and acceptability of the YAM intervention were the co-primary outcomes of the study.

Data analyses

The sample of interest was defined as those students (286 from Montana, 150 from Texas) who participated in both surveys (i.e. completed at least 80% of the questions on one of the scales at each time point). Descriptive statistics were used to describe all demographic data and the primary outcomes. Tests for differences between states were completed for demographic items of interest in Table S1 as well as the acceptability questions. For continuous outcomes, either two-sample t-tests or the Mann Whitney U test were used, depending on the evaluation of model assumptions. For categorical outcomes, the Fisher’s exact test was used due to the inclusion of small counts. In all cases, two-sided tests were performed, and p values were not adjusted for multiple comparisons. For the two qualitative questions requiring written responses, word clouds were generated to identify the most frequently used words. All analyses were completed using R 3.4.3.(R Core Team, 2017).

RESULTS

Participant Characteristics

Characteristics of participants who completed both research evaluations are presented in Table S1. Proportionally more female students participated. Montana and Texas had significantly different proportions of students in each grade, with more Montana students in grades 7^th, 8^th, and 9^th, and more Texas students in 10^th and 12^th grades (p=0.002), though the numbers of youth not in 9^th grade were small. Significantly more Texas students reported not being born in the USA and being non-native English speakers (p<0.001 for both). Lastly, more Texas students lived with both parents compared to their Montana counterparts (p=0.015).

Consent, Participation, and Drop-out Rates for Montana and Texas Combined

Thirteen of 16 schools (81.3%) approached via email, agreed to meet in person for further consideration of YAM implementation. Of these, 71.4% in Montana and 66.7% in Texas participated in YAM delivery and evaluation. Two of the 7 schools approached in Montana declined participation because they were already involved in a suicide prevention grant. Of the 9 Texas schools approached, 1 declined, 1 was lost to follow-up prior to intervention implementation, and 1 participated in baseline surveys and YAM sessions, but follow-up surveys were not collected the same school year and results were therefore excluded from analyses. Tables S2 and S3 show the demographic data for participating schools and classes.

YAM was delivered as part of the school curriculum to 1878 students in 78 classes. Similar numbers of students (975 and 903) and classes (41 and 37) received YAM in Montana and Texas, respectively. Of the total students, 519 (27.6%) provided parental consent and assent to participate in research surveys. Consent rates were 37.3% (n=364) in Montana and 17.2%, (n=155) in Texas.

Ninety-three percent (n=484) of consented students participated in a baseline survey (≥80% completion of at least one instrument), and 84.0% (N=436) participated in the 3-month follow-up. A total of 345 consented students (66.5%) completed both surveys (≥80% completion of all 11 scales), with an average of 76.9% (range 65.9–80.0%) completing each of the 11 scales. In Texas, nearly all participants participated in both surveys (96.8%), while in Montana, the rate was 78.6%. Reasons for missing surveys are described in Table S8, of which school absences were the most common.

Fidelity of delivery

Intervention adherence.

Adherence to the YAM intervention by facilitators was ≥90% for all elements of the intervention except those listed in Table 1. Intervention deviations were primarily due to changes in class period duration (29.5% of sessions) or absence of posters in classrooms (13.3% of sessions). Fidelity support of facilitators was done in person in Texas and via teleconference in Montana.

TABLE 1.

YAM session protocol deviations by state

Statea	Delivery timeb N (%)	Rescheduledc N (%)	Contentd N (%)	Posterse N (%)	Facilitatorf N (%)	Helperg N (%)	Repeatedh N (%)
Montana	80 (39.0)	9 (4.4)	12 (5.9)	32 (15.6)	2 (1.0)	16 (7.8)	20 (9.8)
Texas	35 (18.9)	0 (0.0)	1 (0.5)	20 (10.8)	12 (6.5)	12 (6.5)	N/D
Total	115 (29.5)	9 (2.3)	13 (3.3)	52 (13.3)	14 (3.6)	28 (7.2)	N/D

^a205 (Montana) and 185 (Texas) total sessions were delivered. All students received the YAM booklet.

^bDue to early school/class release, 78 Montana sessions were shortened by 7.3±1.0 minutes. One session was 10 minutes longer. In Texas, 35 sessions were 20 minutes longer due to 70-minute class periods at that school.

^cSessions were rescheduled due to an unexpected school closure.

^dFull content was not delivered in 4 (2.0%) of Montana classes due to time restrictions or equipment issues. Content was delivered in the subsequent session. Other changes: repeat of self-help skills/available resources following a suicide at the school (1 session); repetition of the ice breaker/other materials (3 sessions). One Texas class missed final session due to scheduling problems.

^eLocation was changed, and YAM posters were not present in the new classroom.

^fYAM facilitator was changed due to poor travel conditions or illness.

^gYAM helper was changed due to scheduling changes or absences.

^hFour students repeated YAM due to changes in their class schedules. N/D = not done.

Completion of scheduled intervention sessions (intervention dose).

Thirteen YAM facilitator-helper teams delivered the YAM program (5 sessions) in 3 to 5 weeks (54 and 24 classes, respectively). Delivery differed by state in the number of YAM facilitators (9 in Montana, 4 in Texas). Attendance was recorded for consented students (Montana) or all students (Texas). In Montana, there were 1820 total planned YAM student sessions (number of consented students [364] x number of sessions [5]), of which 1567 (86.1%) of all possible sessions were attended, and 85.4% (n=311) of consented students received at least 4 sessions (Table S9). Sessions were primarily missed due to school absences or moving (8.8% or 4.2% of all scheduled sessions, respectively) (Table S9). Attendance specific to students participating in the research was not recorded in Texas.

Acceptability

Students.

Students reported positive satisfaction with YAM and agreed they would recommend the program to other schools (Figure 1A). Texas students had significantly higher satisfaction with the YAM program (Figure 1A). Acceptability did not differ significantly for students self-reporting as LGBTQ or as non-native English speakers except more LGBTQ students signified the need for mental health programming in schools (Question 4; Figure S2). Word clouds (Figure S3) show student written responses to two opinion questions.

FIGURE 1.

Student, parent, and school staff satisfaction with the YAM intervention. A) Total students, B) parent, and C) school staff YAM satisfaction ratings. All surveys included the following questions (with responses on a 5-point Likert-like scale: 1 = strongly disagree to 5 = strongly agree). Q1) I liked/was pleased/satisfied with the YAM project that my/my child’s school took part in; Q2) I would suggest participating in this program to other schools; Q3) I would want my/my child’s school to participate again in the YAM project; Q4) I think it is a good idea to provide young people with a mental health promotion and risk behavior prevention program in schools; and Q5) Most students/youth would find the program implemented in your/your child’s school appropriate. Data represent the mean ± SD. Bars signify Montana (gray), Texas (light gray), and combined (stippled). The minimum number of responses completed for all 5 questions is reported in the legends. Differences between state groups are denoted by *(p<0.05) and **(p<0.01).

Parents and school staff.

Parental satisfaction ratings were positive overall (N=61) (Figure 1B). Similarly, surveys completed by 50 participating teachers, principals, and relevant school staff showed favorable satisfaction with the YAM program (Figure 1C).

Safety.

In Montana, 9 adverse events occurred during YAM delivery to the 975 students who received YAM as part of the school curriculum. In 4 instances, facilitators removed students from class because of behavioral problems that impacted sessions. For 5 students, the facilitator notified the school counselor: 1 student disclosed possible parental abuse to the facilitator; 1 student became angry during class and left; 2 students, who had personal ties to a student (not in a grade receiving YAM) who recently died by suicide, became upset when hearing of the depression/suicidality focus of the third role play session were excused from the session; and 1 student gave the facilitator a note about “wanting to die”.

In Texas, 9 adverse events occurred while delivering to 903 students: a facilitator removed a student from the room due to behavioral problems that impacted the session; 4 students left the room during the session, stating they were “upset” (follow-up by the facilitator after the session indicated the students’ moods had improved); 3 students reporting significant depression or suicidal thoughts were escorted to the school counselor by the facilitator; and 1 student chose not to attend the first two sessions of YAM, but later joined the class.

DISCUSSION

Results of this study indicate that the YAM intervention was feasible to deliver and evaluate in US school settings as determined by school and student recruitment, intervention fidelity, and assessment completion rates. Surveys of students, parents, and school staff showed high satisfaction of YAM. This study provides key insights for addressing challenges of future delivery and research of YAM in urban and rural settings.

The recruitment of schools was feasible in both states, with high proportions of schools agreeing to participate. The overall school recruitment rate (81.3%) is similar to that observed in the only 3 RCTs of adolescent school-based mental health promotion/suicide prevention interventions reporting school recruitment data (50%, 75%, and 72%) (Schilling, Lawless, Buchanan, & Aseltine, 2014; Swartz et al., 2017; Wasserman et al., 2015). Future US school recruitment is unlikely to be a barrier to YAM implementation and evaluation: both states have a waiting list of schools for YAM delivery. Three Montana schools are currently paying for continued YAM delivery; a health care system is funding YAM in a Montana urban center (Bozeman Health Foundation, 2018); Montana state legislative funding will support a follow-up feasibility study of YAM (Montana State University, 2018); and Texas expanded YAM delivery to 20 schools during the 2017–2018 academic year (Ellege et al.; Hughes et al., 2018; Trivedi, Hughes, South, Lindow, & Byerly, 2018).

Optimal delivery requires rigorous intervention fidelity monitoring (Proctor et al., 2011). The current study found that YAM was delivered with a high degree of fidelity to the program’s content. Standardized fidelity assessments provided by YAM facilitators and helpers indicated all but class period duration and missing materials were ≥90% in accordance to protocol. Post-session assessments and bi-weekly “debriefing” meetings involving study leaders and facilitators likely supported adherence to the protocol. The SEYLE RCT of YAM included a series of monitoring visits and quality control questionnaires for facilitators, which also showed no or small differences among sites (Carli et al., 2013). Other analogous adolescent school-based intervention studies have reported 75%–87.6% delivery fidelity (Schilling et al., 2014; Wyman et al., 2008; Wyman et al., 2010). A future study of YAM might benefit from a quality assurance measure rating facilitator fidelity to the program; currently, Texas is piloting a measure (Trivedi, 2018).

Intervention attendance, which likely supports program efficacy, is another important fidelity measure (Proctor et al., 2011). Attendance of 86% of available sessions in Montana was considerably higher than in the only 3 similar intervention studies, involving indicated depression or universal interventions, reporting similar data (35%, 30%, and 65%) (Connell et al., 2016; Hart et al., 2018; Silverstone et al., 2015). Other universal interventions likely delivered to a majority of targeted populations, though no data were reported (Schilling et al., 2016; Schilling et al., 2014; Swartz et al., 2017; Wyman et al., 2010).

Participant recruitment is a key element of research feasibility as greater participation limits selection bias.(Ghaemi, 2009) Adolescent recruitment rates in school-based mental health promoting/suicide prevention studies have varied widely, with 2 studies not requiring consent reporting higher participation (> 90%) (Aseltine & DeMartino, 2004; Aseltine et al., 2007; Hart et al., 2018) than 3 other studies requiring consent for youth (35–69%) (Schilling et al., 2016; Schilling et al., 2014; Wyman et al., 2010), the current study (27.6%), and 1 other not requiring consent (19% and 10%) (Wyman et al., 2008). Incentivizing students may increase youth consent rates in school-based interventions (Wolfenden, Kypri, Freund, & Hodder, 2009). In a replication study of 1 adolescent school-based mental health promotion/suicide prevention intervention, higher consent rates were achieved (50% vs. 35%) when students were offered a gift card and raffle for consent form return (Schilling et al., 2016; Schilling et al., 2014). In the current study, the difference between the recruitment rates in Montana and Texas was potentially due to incentivizing strategies: Montana, which had higher recruitment rates (37% vs. 17%), used monetary incentives, while Texas offered no or non-monetary incentives. These results suggest that future research of youth school interventions requiring consent should consider incorporating material incentives to enhance participation.

Another challenge is achieving high rates of assessment completion, which is important for limiting sampling bias (Ghaemi, 2009) In this study, 93% of consented students participated in baseline surveys and 84% participated in both surveys, indicating low attrition rates. Similar studies reported a wide range of student survey participation, which depended on intervention type and consent requirements. Gatekeeper interventions had lower student rates of assessment completion (10%, 19%) (Wyman et al., 2008) relative to universal interventions (24–93%) (Aseltine & DeMartino, 2004; Aseltine et al., 2007; Hart et al., 2018; Schilling et al., 2016; Schilling et al., 2014; Swartz et al., 2017) or indicated interventions (63–79%) (Connell et al., 2016; Wyman et al., 2010). In general, universal and indicated interventions requiring consent/assent had lower student assessment participation rates (29–79%) (Connell et al., 2016; Schilling et al., 2016; Schilling et al., 2014; Swartz et al., 2017) than those using opt-out methods (58, 92, and 93%) (Aseltine & DeMartino, 2004; Aseltine et al., 2007; Hart et al., 2018). A recent meta-analysis suggested requiring active parental consent may lead to significant sample bias, and opt-out methods, which still protect minors, should be considered when possible (Liu, Cox, Washburn, Croff, & Crethar, 2017).

Positive acceptability of a school-based intervention is an additional characteristic needed for widespread implementation of such programs (Bowen et al., 2009). The single study of a similar intervention that collected quantitative satisfaction data reported high student and adult satisfaction with presentation of materials, knowledge gained, and content (Hart, Mason, Kelly, Cvetkovski, & Jorm, 2016). The positive satisfaction ratings of YAM, and for similar programs, suggest there would be continued stakeholder support for future YAM delivery and testing.

Limitations

While feasibility and acceptability outcomes of this study were promising, there were several limitations. The enrollment of students from limited regions within 2 states was relatively low (27.8%) and had proportionally more female participants than in participating classes (61% vs. 51%), which reduces the generalizability of findings. Using an uncontrolled design potentially masked the effect of randomization on recruitment rates, especially that of schools. Different measures of fidelity were used between sites. Distal outcomes of suicide (attempts and suicidal thoughts) were not measured. The quality of YAM delivery, a component of intervention fidelity often assessed by outside observers, was not measured. Finally, YAM delivery requires trained, paid facilitators rather than school staff, and fidelity monitoring, increasing cost and logistical challenges. While using teachers or other school staff would enhance generalizability, an important design element of YAM is delivery by non-school personnel. The use of outside facilitators is intended to create a “safe place” in which students can discuss stigmatized topics and stressful life situations. This study was supported by multiple funding sources, but strategies for providing widespread delivery of YAM have only been partially determined. In urban, but not rural settings, YAM can be delivered by a core group of facilitators with limited travel, as demonstrated in Texas. Using fewer facilitators increases delivery experience, thereby optimizing facilitator skill development. Providing YAM in rural regions represents a greater challenge. A model using multiple community facilitators would reduce travel, but limit YAM delivery experience, while one using a small core of facilitators would increase delivery experience but require extensive travel. Thus, optimizing methods for widespread delivery of YAM, should it prove effective, should be included in future studies.

In summary, the current study provides promising feasibility and acceptability results for YAM delivery in the US. The study identified addressable challenges to delivering and conducting research on YAM, including approaches for greater recruitment of schools and participants, and ensuring program fidelity. Future research should focus on the efficacy of YAM in promoting mental health outcomes and reducing suicidality, and related risk factors, when delivered in US school settings. If findings from a US RCT are similar to those observed in Europe,(Wasserman et al., 2015) then YAM would represent an effective method for helping to reduce one of the leading causes of death in US youth.

Abbreviations

YAM

Youth Aware of Mental Health

RCT

randomized controlled trial

SEYLE

Saving and Empowering Young Lives in Europe