Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jan 7.
Published in final edited form as: Crisis. 2015;36(4):274–280. doi: 10.1027/0227-5910/a000323

How Much Is Learned by Students Participating in Suicide Prevention Gatekeeper Training?

Christa D Labouliere 1, Sarah J Tarquini 2, Christine M W Totura 3, Krista Kutash 4, Marc S Karver 5
PMCID: PMC4703948  NIHMSID: NIHMS737498  PMID: 26440624

Abstract

Background

Although gatekeeper training is effective at increasing knowledge, some question the effectiveness of these programs due to high pretraining knowledge levels. However, knowledge scores may be artificially inflated when students guess answer options correctly but lack information needed to assist suicidal peers.

Aims

To use free-recall questions to evaluate suicide prevention knowledge and compare levels of knowledge using this methodology with established assessment methods in the literature.

Method

Free-recall knowledge questions were examined before and after participation in a student gatekeeper training program. Focus groups with students enriched interpretation of quantitative results.

Results

Unlike in studies using forced-choice assessment, students’ baseline knowledge was markedly low using free-recall questions and, despite making significant improvement from pretraining levels, posttraining knowledge barely approached passable levels. Focus group findings suggest that training sessions may need to be more engaging and interactive in order to improve knowledge transfer.

Conclusion

Free-recall questions may provide a less inflated measure of accessible knowledge learned from school-based suicide prevention curricula. Evaluators and programmatic partners should be cognizant of this methodological issue and consider using a mix of assessment methodologies to determine students’ actual levels of knowledge after participation in gatekeeper training.

Keywords: suicide prevention, adolescents, gatekeeper training, school, open-ended assessment

Suicide is the third leading cause of death for adolescents, accounting for more deaths each year than all natural causes combined (Centers for Disease Control and Prevention [CDC], 2011). Nationwide each year, 13.8% of adolescents experience suicidal ideation and 5–10% attempt suicide (CDC, 2011). The risk of death by suicide is further elevated for those not receiving adequate treatment (O’Neill, Corry, Murphy, Brady, & Bunting, 2014), an alarming fact considering insufficient numbers of suicidal persons ever have contact with the mental health system (Hwang et al., 2011). With limited treatment provided to at-risk youth, the need to address this public health crisis via prevention has been repeatedly affirmed (Office of the Surgeon General, 2012; U.S. Department of Health and Human Services, 2011).

Efforts to prevent suicidality in youth have been stymied by the difficulties inherent in identifying the youth most at risk (Husky et al., 2011). Suicidal teens often confide in peers who may be ill-equipped to respond to these disclosures (Elzy et al., 2012; Kalafat & Elias, 1994). Thus, training adolescents to recognize suicide warning signs in their peers and take responsible action could dramatically increases identification of youth at risk for suicidal behavior (Elzy et al., 2012; Katz et al., 2013). Schools have been considered one of the best locations for training efforts, since student emotional and behavioral adjustment is important to educational outcomes (Katz et al., 2013), and students in a supportive school environment are likely to have greater access to reliable mental health information and services (Husky et al., 2011). Furthermore, school-based student gatekeeper training (the purpose of which is to develop the knowledge, attitudes, and skills to identify and refer at-risk youth; Kalafat, 2003) has been found effective at increasing students’ knowledge about suicidality (Cusimano & Sameem, 2011; Robinson et al., 2013).

While increasing knowledge about suicidality is an ostensibly good outcome for prevention programs, some critics have suggested that these pre-to-posttraining gains are not particularly meaningful, as high levels of pretraining knowledge are often found (Shaffer, Garland, Vieland, Underwood, & Busner, 1991). A representative review of school-based student training program evaluations found notable ceiling effects for knowledge gain due to high levels of knowledge prior to training exposure (average of approximately 72% correct on pretraining knowledge evaluations, with a range of 49–85%) resulting in little room for growth (average gain of 8.2% with a range of 2.5–23.8%). An initial search of PsycInfo, Eric, and Med-line found 318 manuscripts using the keywords suicide prevention, youth, and school. When these manuscripts were limited to nonduplicated, empirical studies where suicide prevention knowledge was measured, the number of manuscripts dropped to 51. When the search was limited to studies where training of students (rather than staff or counselors, etc.) was conducted, the number was limited to 14. Two studies were dropped from analyses because raw means or percentages for knowledge scores were not reported (Abbey, Madsen, & Polland, 1989; Klingman & Hochdorf, 1993), leaving a final review of 12 manuscripts (Angerstein, Linfield-Spindler, & Payne, 1991; Aseltine & DeMartino, 2004; Kalafat & Elias, 1994; Overholser, Hemstreet, Spirito, & Vyse, 1989; Portzky & van Heeringen, 2006; Shaffer et al., 1991; Silbert & Berry, 1991; Spirito, Overholser, Ashworth, Morgan, & Benedict-Drew, 1988; Stuart, Waalen, & Haelstromm, 2003; Vanderbilt University, 2009; Walker, Ashby, Hoskins, & Greene, 2009). A recent meta-analysis of school-based suicide prevention curricula supported these findings (Cusimano & Sameem, 2011), suggesting that gatekeeper training leads to only modest gains in knowledge.

These findings of high pretraining knowledge are in stark contrast to the reports of adolescents who frequently express discomfort and anxiety over not knowing what to do when faced with self-harming peers (Muehlenkamp, Walsh, & McDade, 2010). One reason for the discrepancy between pretraining knowledge and confidence in assisting at-risk peers may be that all of the studies reviewed utilized multiple-choice or true/false measures. Forced-choice measures have long been criticized as only testing surface level knowledge (Bloom, 1956); that is, students may be able to guess correctly without actually accessing knowledge relevant for problem solving in real-world situations (McNulty, 1965; Simkin & Kuechler, 2005). It is possible that these high levels of knowledge are artifacts of methodology rather than true measures of students’ ability to interact effectively with suicidal peers.

It is unclear how accurately knowledge has been assessed in the current suicide prevention literature; however, the cognitive literature suggests that recognition elicits greater remembrance of information than recall, since answer choices provide cues that activate associative networks and help a person remember (McNulty, 1965; Zeidner, 1993). Despite the field’s reliance on forced-choice assessments, free recall of information may be more representative of real-world circumstances since interacting with suicidal peers comes without access to direct cues or answer choices (Bloom, 1956). Although skills-based approaches such as coached practice or in vivo instruction may be the most accurate means of assessing students’ knowledge and ability to apply it (Cross et al., 2011), these methods typically require more effort and time than is readily available in the academic setting (Whitney et al., 2011). Alternatively, free-recall questioning (i.e., essays, short answer, etc.) is an accepted methodology in many schools that does not include excessive time and effort (Neal, 2012), and may provide a more feasible analog to real-world situations where knowledge would need to be recalled.

In this study, open-ended assessment was used to examine the effectiveness of the school-based Jason Foundation “A Promise for Tomorrow” suicide prevention gatekeeper training program. This program has been widely disseminated both nationally and internationally and has evidence to support its effectiveness in increasing knowledge (Vanderbilt University, 2009). Based on past research, it was hypothesized that students’ baseline knowledge would be lower when assessed with an open-ended format (as compared with the average 72% correct established in published studies utilizing a forced-choice format) and that open-ended assessment would eliminate ceiling effects. Ultimately, if students’ levels of knowledge are not as high as implied by results from forced-choice evaluations, then students may lack sufficient knowledge to adequately implement the skills needed to interact effectively with suicidal peers. As such, an assessment that more closely parallels spontaneous generation and application of knowledge may be beneficial in determining the true effectiveness of student gatekeeper training.

Method

Participants

Students (N = 1,365; 46.4% male; 66.9% 9th graders, 16.3% 10th graders, 10.4% 11th graders, and 6.4% 12th graders; 45.1% Hispanic, 26.6% Caucasian, 13.3% of mixed race, 5.2% Native American, 3.9% African American, 2.2% Asian, and 3.8% other racial group) in a Southwestern urban school district participated in the study over 3 years. Demographic data were unavailable for one school during 1 year, so demographic analyses were based on 852 participants, whereas knowledge analyses include all 1,365 participants. No differences in knowledge were noted for students who had and had not reported demographics, F(1,1238) = 0.24, p = .63. A random sample of students (N = 58; 46.8% male; 44.7% Hispanic, 38.5% Caucasian, 10.6% mixed race, 2.1% Native American, 2.1% African American, 2.0% other racial group) also participated in focus groups upon parental consent and youth assent; 47% of focus group participants took part in gatekeeper training. Both trained and untrained students participated in focus groups to prevent the possibility of participation in suicide prevention programming biasing answers regarding the need for suicide prevention programming or knowledge of suicide risk factors. All focus group participants answered questions assessing knowledge of suicide risk, their opinions on school climate, knowledge of the existence of the suicide prevention program, and whether such programming was necessary or useful. Students trained by the suicide prevention program were also asked for recommendations for improvement.

Measure and Coding Manual

The Jason Foundation “A Promise for Tomorrow” survey was administered before and after suicide prevention program implementation. The measure consisted of four open-ended questions assessing multiple components: Q1 requested the listing of five (or more) warning signs, Q2 requested four (or more) appropriate responses if peers confided suicidal ideation, Q3 requested two appropriate responses if peers asked to keep suicidal ideation a secret, and Q4 requested two (or more) resources available to assist peers in crisis.

Questions and model correct responses were derived by the study authors from information taught in the Jason Foundation training materials and from the suicide prevention literature. Completely correct answers were assigned a value of two points, partially correct answers were assigned one point, and incorrect answers were assigned zero points, allowing for a maximum of ten raw points for Q1, eight for Q2, two for Q3, and four for Q4. The model correct responses were constructed shortly after survey generation and before data were scored. Over a 2-week period, the coding system was then pilot tested on student responses from year 1 (N = 94) to ensure that model responses covered the full breadth of potential answers, and modifications were made based on this feedback. Two independent coders per case (from a team of trained research assistants blind to the study hypotheses) then used the finalized coding manual to rate remaining responses for completeness and correctness (N = 1,359). Surveys could be coded in under 1 min and with a high level of accuracy (rater intraclass coefficient of .956). For each survey, the number of points for correct answers was divided by the total possible points to calculate an overall percentage score. Percentages were also calculated for individual items (thereby equalizing for the different ranges of raw responses possible per question).

Procedure

The Jason Foundation “A Promise for Tomorrow” gatekeeper prevention program promotes awareness of the problem of youth suicide, provides students with the knowledge and resources to interact with at-risk youth, and develops positive attitudes toward referring at-risk peers. The training sessions were conducted by teachers or counselors during health classes, were overseen by school mental health personnel, and were administered with high fidelity (95.0% student-reported and 86.9% observer- reported adherence to training protocol). Pre- and post-training tests were administered to students before and after the curriculum. Pre- and posttraining results were matched through the use of an identification number stamped on all materials. Only matched pre- and posttraining results were used in paired t tests. Unmatched pre- or posttraining results were deleted listwise for all analyses. Two independent raters coded cases (intraclass coefficient of .956), and when independent raters were discrepant, responses were discussed until consensus was achieved.

Since more emphasis on qualitative research in suicidology increases understanding in the field (Hjelmeland & Knizek, 2010), focus groups were conducted to enrich interpretation of quantitative results. Nine focus groups were run with a maximum of eight students each (range= five to eight students per group; N = 58 across all nine groups). Five groups comprised untrained students and four groups comprised students trained in the suicide prevention curriculum. Focus groups were 1–2 h in duration and were conducted either in person or over the phone using a teleconference line. Focus groups assessed participants’ knowledge of suicide risk, as well as their opinions on school climate, knowledge of the suicide prevention program, and recommendations for program improvement. Focus groups required informed student assent and parental consent for minors, and university and school district institutional review boards approved evaluation protocols. Focus groups were transcribed and responses were coded for themes by two independent research assistants. Student responses were then rated on relevance to particular themes by two independent raters per focus group (intra-class coefficient = .785). For questions in which there was rater discrepancy, responses were discussed until consensus was achieved.

Results

Student Training: Pre- and Posttraining Results

Across all four open-ended questions, students assessed before training only demonstrated an overall suicide-related knowledge score of 35%. This stands in stark contrast to the average 72% correct at baseline established in previously published studies utilizing a forced-choice format, including prior evaluations of the Jason Foundation curricula (Vanderbilt University, 2009), z = 18.75, p < .0001. When examining individual domains of suicide-related knowledge before training, only 37.6% of students were able to identify correct warning signs of suicide, 19.8% were able to provide appropriate actions to take if peers confided suicidal ideation, and 20.5% were able to recognize when to break confidentiality if a friend asked to keep suicidal ideation a secret. Students were able to demonstrate a passing score on only one question that assessed resources available to assist peers in crisis, with 71.3% of students answering this question correctly.

Next, paired-sample t tests with modified Bonferroni corrections were used to evaluate the effectiveness of the gatekeeper training curriculum. Students significantly improved their overall suicide-related knowledge by approximately 15% from pre- to posttest (see Table 1), an improvement significantly higher than the 8.2% average gains reported in previously published studies utilizing a forced-choice format, z = 6.3, p < .0001. All four open-ended questions demonstrated significant improvement (see Table 1). However, there was still substantial room for improvement in student knowledge levels, as scores only improved from an average of 35% correct at pretest to 51% correct at posttest (Cohen’s d = 0.771), a level of posttraining knowledge significantly lower than the 80.2% average reported in previously published studies utilizing a forced-choice format, z = 14.9, p < .0001.

Table 1.

Pre- and posttest student knowledge using open-ended free-recall questions

Question N Pretest Mean % Correct (SD) Posttest Mean % Correct (SD) t
Total suicide-related knowledge 1,240 35.3 (18.4) 50.77 (25.3) −19.14***
Q1: Warning signs 1,239 37.6 (24.4) 52.4 (30.7) −14.79***
Q2: Response if a peer confided suicidal ideation 1,235 19.8 (22.4) 46.1 (37.0) −23.88***
Q3: Breaking confidentiality 1,239 20.5 (26.5) 25.5 (28.0) −5.03***
Q4: Resources 1,240 71.3 (39.5) 77.0 (38.0) −3.12**
Open in a new tab

Note. N = 1,365. Paired-sample t tests were used to detect pre- to posttraining change, with participants matched by a deidentified ID number on all materials. In cases where participants did not complete both the pretraining and posttraining tests, data were deleted listwise per question so that only complete, matched data were included.

*

p < .05.

**

p < .01.

***

p < .001.

Focus Groups With Trained and Untrained Students

Focus group results demonstrated that the majority of students recognized the importance of suicide prevention programming. However, despite their participation, 64.5% of trained students reported that recognition of suicidality in their peers remained difficult. Training had positive effects on students’ confidence regarding talking with a suicidal peer, as only 3.7% of trained students reported that talking to an identified suicidal peer would be scary or difficult. Alternatively, untrained students reported more fear about speaking with a suicidal peer, with 7.4% stating that they were completely unprepared and 14.8% stating that they would have no idea what to say. No other differences existed between trained and untrained students.

Trained students were also asked their opinion of the Jason Foundation “A Promise for Tomorrow” curriculum. Whereas 45% of trained students felt the program was informative, 53% reported that aspects of the program could be improved. Common recommendations for program improvement included having trainers that were more interested in and knowledgeable about suicide prevention, increasing the clarity, concreteness, and depth of program information and materials, increasing the frequency and degree of interaction (e.g., role plays, small group discussion) with which the material is presented, and updating the material with more ethnically diverse actors and current settings and situations (see Table 2). In general, youth participants expressed a preference for program components that more closely approximate real-life settings in which they might be called upon to apply suicide prevention knowledge. This position is consistent with the use of free-recall methodology as a potentially more reliable indicator of knowledge learned and able to be applied after participation in suicide prevention training.

Table 2.

Program feedback for the Jason Foundation “A Promise for Tomorrow Program” from focus groups with students who participated

Program feedback from trained students Percent of trained students reporting recommendation
Important material was forgotten since the training, highlighting the need for increased frequency of training 71%
Aspects of the program could be improved 53%
Updated materials would improve knowledge retention 52%
Settings or events in the training materials should be changed to be more relatable 38.7%
Recommend having trainers who are more knowledgeable and interested in suicide prevention 35%
Material was too vague and did not provide concrete direction on how to help a suicidal friend outside of referral information 32%
Trainers/teachers seemed uninterested in providing training 19.4%
The program was too short, offered too infrequently, or the material was covered too quickly 19%
Training would benefit from small group discussions 18.9%*
Warning signs were not explained clearly enough to allow recognition in friends who may be trying to hide problems 13%
Training would benefit from inclusion of role-plays 10.3%*
Training materials should include more ethnically diverse actors 9.7%
Training materials should include younger actors 3.2%
Open in a new tab

Note. N = 58. All percentages reflect youth who participated in the program, unless otherwise denoted by asterisks.

Discussion

Criticisms of suicide prevention training suggest that students have favorable levels of suicide prevention knowledge prior to training and ceiling effects limit the value of trainings (Shaffer et al., 1991). However, the studies that foster these criticisms utilized forced-choice assessment of student knowledge, and did not disentangle whether this methodology may artificially inflate the level of knowledge available to students when interacting with suicidal peers. In this study, an open-ended assessment of adolescents’ ability to identify and properly respond to at-risk peers was utilized and the findings were compared with established values from forced-choice methodology in previous school-based suicide prevention studies.

Using an open-ended free recall measure in a program that was implemented with high fidelity (87–95% adherence) with a large ethnically diverse student body, students showed markedly low baseline knowledge of suicide prevention material (35%), in contrast to previous research wherein students’ baseline knowledge of suicide-related information was high (average of 72%). Likewise, posttest knowledge as assessed by an open-ended measure (51%) was much lower than the level of knowledge expected based on previous reports that used forced-choice formats (average of 80.2%). Ceiling effects were also eliminated when open-ended assessments were utilized (~16% gain vs. an average 8.2% gain, or a 45.7% vs. 11.4% improvement). However, much less knowledge was recalled by students using open-ended assessment than was traditionally demonstrated in studies using forced-choice formats (Angerstein et al., 1991; Aseltine & DeMartino, 2004; Kalafat & Elias, 1994; Overholser et al., 1989; Portzky & van Heeringen, 2006; Shaffer et al., 1991; Silbert et al., 1991; Spirito et al., 1988; Stuart et al., 2003; Walker et al., 2009), including previous evaluations of the Jason Foundation prevention curriculum specifically utilizing similar content as the program measured in this study (Vanderbilt University, 2009). Furthermore, information obtained from students in focus groups suggested that current suicide prevention programming is insufficient to provide the knowledge and skills necessary to respond confidently and appropriately to peer suicidality. Many of the suggestions proposed by students (e.g., making the curriculum more interactive, engaging, and personally relevant; increasing the frequency with which the curriculum is taught, etc.) are also strategies that are strongly supported by the cognitive psychology and education literature to increase knowledge retention (Borich, 2009).

Ultimately, there are many prevention programs (both in the field of suicide prevention and across various content areas) that rely predominantly on forced-choice knowledge assessments, which may potentially provide skewed evidence of program effectiveness. If the goal is to engage students in prevention efforts and realistically assess what knowledge is accessible, the use of open-ended questions may be preferable to the use of forced-choice questions. In a real-life situation, a trained student would need to recall information freely rather than simply recognizing the correct choice from a number of options as in forced-choice methods (Fiske & Taylor, 1984). Although other approaches, such as skills-based training or in vivo coaching, may be the most accurate option for directly observing skills learned by students (Cross et al., 2011), these types of training protocols are often deemed too costly or time-intensive compared with open-ended questions that are frequently used by educators (Neal, 2012; Whitney et al., 2011). As such, open-ended questions could be a feasible compromise that may better gauge the amount of knowledge retained by students trained by prevention programs. On the other hand, it is possible that free-recall questioning underestimates the knowledge that students can utilize in real-life situations. Future studies should compare free-recall and forced-choice knowledge assessments with skills-based approaches to better determine which methodologies predict greater confidence or proficiency in interactions with suicidal peers.

Notwithstanding its contributions, this study had several limitations. Most notably, no direct comparison between forced-choice and open-ended questions was conducted, owing to the time limitations and economic constraints of the school system. Moreover, no control group was utilized in this study. As such, it is possible that lower knowledge scores could be the result of some unmeasured difference in our sample or procedure compared with those used in other studies. For example, it is possible that students in our sample had lower baseline levels of knowledge than students in other parts of the country, or may have differed somehow in racial, ethnic, or socioeconomic composition in comparison with the predominantly Caucasian and middle-class samples used in the suicide prevention literature. It is also possible that our questionnaire covered slightly different knowledge areas or was simply more challenging than those represented in other published studies. However, our sample was large and representative of local demographics, while our procedures and measure content were designed to be as similar as possible to those utilized in other studies (Stuart et al., 2003; Walker et al., 2009), including previous evaluations of the Jason Foundation curriculum (Vanderbilt University, 2009). Similarly, not all students trained by the Jason Foundation suicide prevention curriculum consented to participate in the evaluation, so it is possible that there were differences in knowledge gained between those who did and did not consent; however, this possibility is lessened by the fact that the pattern of results did not differ between years where active consent was required and where secondary data analysis of all deidentified data was conducted. Additionally, while the suicide prevention curriculum was likely responsible for student knowledge gain, there was no measurement of more distal variables such as help seeking or suicidal behavior to determine the program’s impact on the ultimate intended goals of suicide prevention activities.

Despite these limitations, this evaluation determined that the Jason Foundation curriculum was effective in raising adolescents’ knowledge, but suggests that the level of suicide-related knowledge that students can spontaneously generate and apply to real-life situations when assessed with open-ended questions may be far lower than that previously reported. This study identified an important potential methodological confound epidemic in prevention programming, and suggests that free-recall assessments may provide a less inflated measure of knowledge in school-based suicide prevention curricula. Suicide prevention program implementers and evaluators should explore methods of increasing knowledge retention for students among a mix of assessment methodologies (i.e., free recall, skills-based approaches, etc.) when they gauge the effectiveness of school-based suicide prevention programming. Future research should also determine if suggestions offered in the focus groups – such as the addition of culturally competent components, updated materials, interactive role-playing or discussions, or more depth and breadth of training – could make suicide prevention more relevant and effective for youth. Additionally, future research should examine other variables that may mediate the effectiveness of suicide prevention programming, such as student engagement and participation, perceptions of peer and teacher engagement in training, social norms regarding suicide prevention behaviors, or relationship quality between the trainer and trainees. Lastly, as knowledge does not always predict action (e.g., Ferris, von Gunten, & Emanuel, 2001), future research should measure changes in more distal variables such as referral, help-seeking, and suicidal behavior, in addition to levels of suicide prevention knowledge.

Acknowledgments

Special thanks to Sarah Gunderson, Laurie Dempsey, and Renee Brown Hangartner for their invaluable assistance. This research was supported by Substance Abuse and Mental Health Administration grants (#SM57442-01, 02-3) to Drs. Karver, Kutash, and Totura.

Biographies

Christa D. Labouliere is a postdoctoral fellow in translational research in child psychiatry at Columbia University Medical Center, NY, USA. She received her doctorate in clinical psychology from the University of South Florida. Her research interests include suicide prevention and intervention programs, and psychosocial and neurobiological risk factors for youth self-harm behavior.

Sarah J. Tarquini is a psychologist in the pediatric psychosocial oncology program at Dana Farber Cancer Institute, Boston, MA, and a psychology instructor at Harvard Medical School, Boston, MA. She received her doctorate in clinical psychology from the University of South Florida. Her research interests include adjustment to illness, treatment adherence, and patient–provider communication.

Christine Totura is an assistant professor in clinical psychology at Auburn University, AL, USA. She received her doctorate from the University of South Florida. Her research focuses on the influence of school and community contextual factors on youth social–emotional and behavioral development and the implementation of prevention programs and policies.

Krista Kutash is Professor Emeritus at the Louis De La Parte Florida Mental Health Institute at the University of South Florida, Tampa, FL, USA. Her research interests include community-applied research methodology and the application of these methods to researching publically funded children’s mental health services, school-based mental health, and parent-support programs.

Marc S. Karver is an associate professor of clinical psychology at the University of South Florida, Tampa, FL, USA. He is interested in understanding core processes (e.g., intervention engagement) in youth mental health services research and improving prevention, assessment, and intervention services for suicidal youth and young adults.

References

  1. Abbey KJ, Madsen CH, Polland R. Short-term suicide awareness curriculum. Suicide and Life-Threatening Behavior. 1989;19:216–227. doi: 10.1111/j.1943-278x.1989.tb01034.x. Retrieved from http://www.surgeon-general.gov/library/reports/national-strategy-suicide-prevention/full_report-rev.pdf. [DOI] [PubMed] [Google Scholar]
  2. Angerstein G, Linfield-Spindler S, Payne L. Evaluation of an urban school adolescent suicide program. School Psychology International. 1991;12:25–48. [Google Scholar]
  3. Aseltine R, DeMartino R. An outcome evaluation of the SOS suicide prevention program. American Journal of Public Health. 2004;94:446–451. doi: 10.2105/ajph.94.3.446. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bloom BS. Taxonomy of educational objectives, handbook I: The cognitive domain. New York, NY: David McKay; 1956. [Google Scholar]
  5. Borich GD. Effective teaching methods: Research-based practice. 7. Cachan Cedex, France: Lavoisier; 2009. [Google Scholar]
  6. Centers for Disease Control and Prevention. Web-Based Injury Statistics Query and Reporting System. Ten leading causes of death, United States 2007. 2011 Retrieved from http://webappa.cdc.gov/cgi-bin/broker.exe.
  7. Cross WF, Seaburn D, Gibbs D, Schmeelk-Cone K, White AM, Caine ED. Does practice make perfect? A randomized control trial of behavioral rehearsal on suicide prevention gatekeeper skills. Journal of Primary Prevention. 2011;32:195–211. doi: 10.1007/s10935-011-0250-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cusimano MD, Sameem M. The effectiveness of middle and high school-based suicide prevention programmes for adolescents: A systematic review. Injury Prevention. 2011;17:43–49. doi: 10.1136/ip.2009.025502. [DOI] [PubMed] [Google Scholar]
  9. Elzy M, Labouliere C, Gryglewicz K, Romero G, Ornelas A, Brown R, Karver M. Student gatekeeper training and screening. In: Gryglewicz K, Brown R, Elzy M, Labouliere CD, Dean M, Chen J, Karver M, editors. Decreasing adolescent suicidality through a multiple-component community-based suicide prevention program: Florida Adolescent Suicide Awareness and Prevention (FL A.S.A.P.) Project. Washington, DC: Substance Abuse and Mental Health Service Administration; 2012. pp. 160–187. [Google Scholar]
  10. Ferris FD, von Gunten CF, Emanuel LL. Knowledge: Insufficient for change. Journal of Palliative Medicine. 2001;4:145–147. doi: 10.1089/109662101750290164. [DOI] [PubMed] [Google Scholar]
  11. Fiske ST, Taylor SE. Social cognition. New York, NY: Random House; 1984. [Google Scholar]
  12. Hjelmeland H, Knizek BL. Why we need qualitative research in suicidology. Suicide and Life-Threatening Behavior. 2010;40:74–80. doi: 10.1521/suli.2010.40.1.74. [DOI] [PubMed] [Google Scholar]
  13. Husky MM, Kaplan A, McGuire L, Flynn L, Chrostowski C, Olfson M. Identifying adolescents at risk through voluntary school-based mental health screening. Journal of Adolescence. 2011;34:505–511. doi: 10.1016/j.adolescence.2010.05.018. [DOI] [PubMed] [Google Scholar]
  14. Hwang I, Chiu WT, Sampson N, Kessler RC, Alonso J, Borges G, Nock M. Treatment of suicidal people around the world. British Journal of Psychiatry. 2011;199:64–70. doi: 10.1192/bjp.bp.110.084129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kalafat J. School approaches to youth suicide prevention. American Behavioral Scientist. 2003;46:1211–1223. [Google Scholar]
  16. Kalafat J, Elias M. An evaluation of a school-based suicide awareness intervention. Suicide and Life-Threatening Behavior. 1994;24:224–233. [PubMed] [Google Scholar]
  17. Katz C, Bolton SL, Katz LY, Isaak C, Tilston-Jones T, Sareen J the Swampy Cree Suicide Prevention Team. A systematic review of school-based suicide prevention programs. Depression and Anxiety. 2013;30:1030–1045. doi: 10.1002/da.22114. [DOI] [PubMed] [Google Scholar]
  18. Klingman A, Hochdorf Z. Coping with distress and self harm: The impact of a primary prevention program among adolescents. Journal of Adolescence. 1993;16:121–140. doi: 10.1006/jado.1993.1012. [DOI] [PubMed] [Google Scholar]
  19. McNulty JA. An analysis of recall and recognition processes in verbal learning. Journal of Verbal Learning & Verbal Behavior. 1965;4:430–436. [Google Scholar]
  20. Muehlenkamp JJ, Walsh BW, McDade M. Preventing non-suicidal self-injury in adolescents: The Signs of Self-Injury program. Journal of Youth and Adolescence. 2010;39:306–314. doi: 10.1007/s10964-009-9450-8. [DOI] [PubMed] [Google Scholar]
  21. Neal MA. Engaging students through effective questions. Education Canada. 2012;52 Retrieved from http://www.cea-ace.ca/education-canada/article/engaging-students-through-effective-questions. [Google Scholar]
  22. Office of the Surgeon General & National Action Alliance for Suicide Prevention. National Strategy for Suicide Prevention. 2012 Retrieved from http://www.surgeongeneral.gov/library/reports/national-strategy-suicide-prevention/full_re-port-rev.pdf.
  23. O’Neill S, Corry CV, Murphy S, Brady S, Bunting BP. Characteristics of deaths by suicide in Northern Ireland from 2005 to 2011 and use of health services prior to death. Journal of Affective Disorders. 2014;168:466–471. doi: 10.1016/j.jad.2014.07.028. [DOI] [PubMed] [Google Scholar]
  24. Overholser JC, Hemstreet AH, Spirito A, Vyse S. Suicide awareness programs in the schools: Effects of gender and personal experience. Journal of the American Academy of Child and Adolescent Psychiatry. 1989;28:925–930. doi: 10.1097/00004583-198911000-00018. [DOI] [PubMed] [Google Scholar]
  25. Portzky G, van Heeringen K. Suicide prevention in adolescents: A controlled study of the effectiveness of a school-based psycho-educational program. Journal of Child Psychology and Psychiatry. 2006;47:910–918. doi: 10.1111/j.1469-7610.2006.01595.x. [DOI] [PubMed] [Google Scholar]
  26. Robinson J, Cox G, Malone A, Williamson M, Baldwin G, Fletcher K, O’Brien M. A systematic review of school-based interventions aimed at preventing, treating, and responding to suicide-related behavior in young people. Crisis. 2013;34:164–182. doi: 10.1027/0227-5910/a000168. [DOI] [PubMed] [Google Scholar]
  27. Shaffer D, Garland A, Vieland V, Underwood M, Busner C. The impact of curriculum-based suicide prevention programs for teenagers. Journal of the American Academy of Child and Adolescent Psychiatry. 1991;30:588–596. doi: 10.1097/00004583-199107000-00010. [DOI] [PubMed] [Google Scholar]
  28. Silbert KL, Berry GL. Psychological effects of a suicide prevention unit on adolescents’ levels of stress, anxiety and hopelessness: Implications for counseling psychologists. Counseling Psychology Quarterly. 1991;4:45–58. [Google Scholar]
  29. Simkin MG, Kuechler WL. Multiple-choice tests and student understanding: What is the connection? Decision Sciences: Journal of Innovative Education. 2005;3:73–98. [Google Scholar]
  30. Spirito A, Overholser J, Ashworth S, Morgan J, Benedict-Drew C. Evaluation of a suicide awareness curriculum for high school students. Journal of the American Academy of Child and Adolescent Psychiatry. 1988;27:705–711. doi: 10.1097/00004583-198811000-00007. [DOI] [PubMed] [Google Scholar]
  31. Stuart C, Waalen J, Haelstromm E. Many helping hearts: An evaluation of peer gatekeeper training in suicide risk assessment. Death Studies. 2003;27:321–333. doi: 10.1080/07481180302906. [DOI] [PubMed] [Google Scholar]
  32. U.S. Department of Health and Human Services. Healthy people 2020 initiative – adolescent health. 2011 Retrieved from http://www.healthypeople.gov/2020/topicsobjectives2020/overview.aspx?topicid=2.
  33. Vanderbilt University. Evidence-based evaluation of “A Promise for Tomorrow”. Henderson, TN: The Jason Foundation; 2009. [Google Scholar]
  34. Walker RL, Ashby J, Hoskins OD, Greene FN. Peer-support suicide prevention in a non-metropolitan U.S. community. Adolescence. 2009;44:335–346. [PubMed] [Google Scholar]
  35. Whitney SD, Renner LM, Pate CM, Jacobs KA. Principals’ perceptions of benefits and barriers to school-based suicide prevention programs. Children and Youth Services Review. 2011;33:869–877. [Google Scholar]
  36. Zeidner M. Essay versus multiple-choice type classroom exams: The student’s perspective. In: Nevo B, Jäger RS, editors. Educational and psychological testing: The test taker’s outlook. Ashland, OH: Hogrefe & Huber Publishers; 1993. pp. 67–82. [Google Scholar]

RESOURCES