School- and Community-Based Youth Suicide Prevention Interventions: Hot Idea, Hot Air, or Sham?

Abstract

Suicide in young people is a significant health concern, with numerous community- and school-based interventions promising to prevent suicide currently being applied across Canada. Before widespread application of any one of these, it is essential to determine its effectiveness and safety. We systematically reviewed the global literature on one of the most common community suicide prevention interventions in Canada and summarized data on 2 commonly applied school-based suicide prevention programmes. None of these has demonstrated effectiveness in preventing youth suicide or safety in application. Concurrently with their widespread distribution in Canada, the suicide rate in young women has increased—the first time in over 3 decades. Policy and regulatory implications of these findings are discussed.

Youth suicide is a significant human, civic, and health concern, and the need to create and apply effective interventions that can decrease rates of suicide in young people is well recognized.^1–5 In the Canadian context, although the rate of suicide in youth is about half of the suicide rate in middle age, suicide is the second leading cause of death in young people.⁶ Suicide rates rise rapidly over the decade following the onset of puberty, in parallel with a similarly steep increase in the prevalence of mental illnesses. During this time, young people are primarily present in the school system: junior high and secondary school in particular. In non–First Nations populations, the presence of a mental illness is considered the most substantive risk factor for death by suicide.^1,7

As part of Canada’s response to the need to apply suicide prevention interventions, with the purpose of decreasing suicide rates, numerous initiatives have been put into place. Broadly speaking, these can be categorized as being applied primarily in clinical settings (e.g., targeted interventions designed to decrease suicide rates in young people who may have or are known to have a mental illness) or in the community. Non-clinical-based applications are often applied universally in schools or directed at the population as a whole.

School-Based Interventions: Do They Work?

Although there is much enthusiasm about the application of school-based interventions to decrease youth suicide rates, these largely remain unproven in their ability to do so. A recent Canadian review of systematic reviews that included school-based suicide prevention programmes was unable to identify any junior high– or secondary school–based suicide prevention programme that had been shown to decrease suicide rates or closely related proxy measures of suicide (e.g., emergency room visits for a suicide attempt or hospitalization following a suicide attempt).⁸ Additionally, an in-depth and detailed analysis of the published literature on 2 popular school-based suicide prevention programmes currently marketed in Canada (Signs of Suicide [SOS] and Yellow Ribbon) failed to find any evidence of impact on any of the above outcomes.⁹ Similar findings of lack of effectiveness were reported by the World Health Organization (WHO).¹⁰ Indeed, only 1 European school-based study has demonstrated promise in youth suicide rate reduction, and it used a curriculum-based intervention linked to gatekeeper-type training and access to mental health care provision for youth identified as at risk.¹¹ This is a very different approach from that used by SOS or Yellow Ribbon. While promising, however, this remains to be replicated. A further promising school-based intervention, the Good Behavior Game (albeit provided at primary and not junior high or secondary school levels), has also demonstrated promise in suicide prevention. Its impact on suicide-related behaviors, however, may not be specific but may be the result of its wider effect on suicide-related risk factors such as delinquency, incarceration, and substance abuse.^12,13

Community Universal Interventions: Do They Work?

The review of systematic reviews on youth suicide preventions conducted by Bennett et al.⁸ further indicated that to date, few if any community interventions have been investigated in rigorous studies (e.g., randomized controlled trails) and have clearly demonstrated an impact on reducing youth suicide rates, nor have any demonstrated a reduction in the 2 relatively robust suicide proxy measures described above. Similar results of lack of evidence for decreases in suicide rates were reported by the WHO Health Evidence Network (HEN) synthesis report.¹⁰

We further extended the search for systematic reviews on suicide prevention in the general community beyond the adolescent group. This resulted in 11 extra systematic reviews and meta-analyses, with 4 in particular including community-based universal prevention programmes.^14–17 Of these 4 reviews, 2 targeted youth,^14,15 with one finding insufficient evidence for any suicide prevention programme disseminated in postsecondary educational settings.¹⁵ The other found about half of the interventions effective in reducing suicide ideation or attempts, including one universal programme (SOS).^14,18,19 However, SOS was not recommended by other systematic reviews due to its poor quality of study designs, and a recent report on the impact of SOS has demonstrated that the intervention may actually increase suicide attempts.^9,8,20 There are 2 additional reviews of universal programmes in military or veterans.^16,17 The review by O’Neil and colleagues¹⁶ was unable to identify any well-designed study demonstrating suicide prevention in this group, while Bagley and colleagues¹⁷ identified 4 programmes purporting to reduce suicide rates. These, however, all suffered from a high risk of bias, thus raising concern about the validity of their findings.

SafeTALK: Effective and Safe?

These literature reviews of suicide interventions in both schools and the community raise serious concerns about the strong popular interest in such interventions, especially using the approach of “training” members of the public in community suicide prevention interventions in the context of aggressive marketing of these programmes by their purveyors. To our knowledge, no detailed and comprehensive analysis of the effectiveness of any such programme on decreasing youth suicide rates exists. Nor is there any evidence, to our knowledge, about the safety of programmes that encourage public discourse about suicide. Thus, we decided to analyze the data available for the most widely distributed and applied community suicide prevention programme in Canada, SafeTALK, to determine the evidence for its effectiveness and safety.

SafeTALK is a programme developed and marketed by the company Livingworks, which advertises itself as evidence based and has been disseminated across many different countries, including Canada, United States, Scotland, Norway, and Australia. It purports to help members of the public recognize individuals at risk for suicide and encourages participants to ask about suicide ideation, intent, or plans and promotes public discourse about suicide. In Canada, it has been offered at suicide prevention workshops conducted by reputable organizations such as numerous chapters of the Canadian Mental Health Association (CMHA), various universities, numerous Ontario community suicide prevention coalitions, and even the city of Ottawa. The Canadian Association for Suicide Prevention recommends SafeTALK for coaches (http://suicideprevention.ca/coaching-and-suicide-awareness) and for use in the workplace (http://suicideprevention.ca/hope-at-work/becoming-a-suicide-safer-workplace).

Methods

We applied a focused systematic review approach to investigate the quality of available evidence of the effectiveness of SafeTALK. We included research studies and programme evaluations of any type, with any outcomes, and with no limits on time, age, or population. We included both peer-reviewed publications and the grey literature published in English. We did not include studies on the suicide prevention ASIST (Applied Suicide Intervention Skills Training), a programme similar to SafeTALK that is also marketed by Livingworks.

Two researchers independently searched a number of key databases (PubMed, PsycINFO, EMBASE, CINAHL, ERIC, and Psycharticles) using the keywords safeTALK, suicide, suicidality, parasuicide, self-harm, suicidal, suicide ideation, and suicide attempt. We further “Googled” various relevant government and organization websites and searched Google Scholar. We finally checked systematic reviews and meta-analyses for relevant studies. All studies found using this methodology were included in this review. These 2 reviewers reached the consensus on their final results, and a mental health expert was available to solve the differences between the 2 researchers’ findings.

We extracted data on target population, sample size, intervention description and duration, study design, and outcomes. We further assessed the risk of bias of included studies using Office of Justice Program (OJP) What Works Repository.²¹ OJP rates studies into 6 levels of evidence of effectiveness (effective, effective with reservation, promising, inconclusive evidence, insufficient evidence, and ineffective) and 3 levels of readiness for dissemination (fully prepared for widespread dissemination, fully prepared for limited dissemination, and not ready for dissemination).

Results

We identified a total of 6 studies, including 1 peer-reviewed publication, 1 dissertation, and 4 evaluation reports.^22–27 Together, these constitute the available world literature on SafeTALK. Table 1 summarizes the studies’ characteristics and results. As indicated in the table, all studies were conducted in North America or Europe among community members. All but 1 study used feedback surveys to evaluate participant self-report on their preparedness to help others at risk of suicide.²³ All but 1 study reported results by percentages and frequencies with no statistical analyses of significance provided.²³ The study by Eynan²³ reported statistically significant knowledge increase, positive attitudes towards suicidal individuals, and self-rated improved skills to help those individuals. No study reported on the impact of SafeTALK on self-reported suicide attempts, emergency room visits for suicide attempts, or suicide rates.

Table 1.

Study Characteristics.^a

	Mellanby et al.22	Eynan23	NSPC24	McKay et al.25	McLean et al.26	Flannery and Deegan27
Sample, n	17	307	486	227	239	105
Population	Veterinary students	Toronto subway personnel	Community members	Community members	Community members	Community members
Country	United Kingdom	Canada	Canada	United States	Scotland	Ireland
Design	Survey and focus groups	Pre- and posttest survey; interviews	Pre- and posttest survey; interviews	Pre- and posttest survey; interviews	Pre- and posttest survey; interviews	Survey
Baseline	No	Yes	Yes	No	Yes	No
Comparison	No	No	No	No	No	No
Follow-up	Focus groups	Interviews	Interviews	Interviews	Interviews	No
Intervention	SafeTALK is a suicide awareness training that teaches to recognize and engage people who might have suicidal thoughts and connect them with appropriate community services.	SafeTALK is to develop positive attitudes toward suicide prevention, heighten suicide awareness, increase knowledge of suicide signs, improve assessment of immediate risk, and enhance intervention skills and appropriate referrals of at-risk individuals.	SafeTALK is a suicide awareness training delivered by certified facilitators to increase awareness and comfort in discussing topics about suicide.	SafeTALK teaches members of the community to recognize persons with thoughts of suicide and to connect them to suicide intervention resources.	SafeTALK has an awareness and training focus and teaches participants to recognise and engage persons who might be having thoughts of suicide and to connect them with community resources trained in suicide intervention.	Not reported
Duration	Half day	Full day	Half day	Half day	Half day
Measurement validity	No	Yes	No	No	No
Outcomes	Knowledge of suicide; attitudes towards suicide individuals	Knowledge of suicide; attitudes towards suicidal individuals; invention skills	Beliefs, attitudes, and behaviors about suicide	Preparedness to help suicidal individuals	Preparedness to help suicidal individuals	Preparedness to help suicidal individuals
Results (all measures are self-reports and none are validated except for Eynan23)	15-16 out of 17 participants were (much) more likely to recognize suicide signs, approach and ask a person at risk, and connect a person at risk with help	SPQ: M* = 3.61 (pre), 3.78 (post), 3.87 (follow-up); d = .41, .48 IKT-R: M* = 6.9 (pre), 9.4 (post), 8.6 (follow-up); d = .96, 1.25 SOQ-R: M* = 10.93 (pre), 12.34 (post), 12.01 (follow-up); d = .60, .70 SIRI: M = 6.4 (pre), 7.8 (post), 8.6 (follow-up); d = .89, 1.37	An increase of 6% to 30% on positive beliefs about and attitudes towards suicide	94% to 97% of participants reported that they were better able to respond to a person with suicidal thoughts; 86.2% reported they were more prepared to help others at risk of suicide.	85% participants reported they were more prepared to talk about suicides (58% at pretest). Over 80% of all respondents reported that after the course, they were (much) more likely to recognise the signs of someone at risk of suicide, to approach the person, to ask them directly whether they were having suicidal thoughts, and to be able to connect them to help.	No specific data reported except stating majority of participants felt that their level of preparedness had increased significantly.

IKT-R, Intervention Knowledge Test–Revised; M, mean; NSPC, Niagara Suicide Prevention Coalition; SIRI, Suicide Intervention Response Inventory; SOQ-R, Suicide Opinion Questionnaire–Revised; SPQ, Suicide-risk Procedural Questionnaire.

^aOnly Mellanby et al.²² is a peer-reviewed publication and is the only study addressing young people.

*P < 0.001.

Critical appraisal of risk of bias of included studies indicated that all 6 studies fell into the OJP “insufficient evidence” category due to the lack of rigor of study design (pre- and posttest or descriptive evaluation), external replication, and statistical significant effect (Figure 1). An independent application of the OJP framework demonstrates that SafeTALK is also not ready for wide dissemination due to the insufficient information on validity of the training materials, technical support, and quality control (Figure 1).

Figure 1.

Level of evidence and readiness for dissemination.

No measures of safety were reported in any of the above studies.

Discussion

The findings noted above are highly concerning. In none of the above popularly applied suicide prevention programmes, such as SOS, Yellow Ribbon, and SafeTALK, was any sufficient evidence of effectiveness or of safety found. In the peer-reviewed literature, only 1 study of Yellow Ribbon has been published, and it was negative. Only 4 studies of SOS have been published, all by the purveyors of the programme, all with high risk of bias and none showing evidence of suicide prevention.¹⁹ Furthermore, in the most recent SOS publication, which in the abstract promoted the intervention as “promising,” the total between-group difference in reported decreased suicide ideation was only in 2 subjects, while 5 subjects in the intervention group reported a suicide attempt compared to none in the control group.²⁰ This last and highly concerning information was not addressed anywhere in the article and was instead buried in a graph that minimized these important differences. Of interest is a finding from a study of a related programme conducted by Sareen et al.²⁸ that identified increased rates of “serious” suicide ideation in participants compared to controls. Taken together, these findings raise concerns about safety of these and similar interventions that must be evaluated in future research.

For SafeTALK, the entire global data set in the peer-reviewed literature on its effectiveness is 1 open study of 17 veterinary students in Glasgow, Scotland, who reported that they liked the intervention and were prepared to help suicidal individuals.²² Although an unpublished dissertation reported statistically significant improvement in knowledge and attitudes, in another sample, it provided no measures of the impact of the intervention on suicide outcomes, and this study also was rated as exhibiting a high risk of bias and cannot provide justification for the marketing and dissemination of this programme.²³

One important and perhaps related observation is that while the application of these programmes has proliferated in recent years, youth suicide rates (ages 14-19 years) in Canada have not continued to decline as they had during the 1980s and 1990s, when these programmes were not being applied (see Figure 2). Indeed, correlational to the increased application of these programmes has been an increase in youth suicide rates for the period 2000 to 2012. While correlation does not mean causality, this temporal relationship raises a question about not only the effectiveness of these programmes but also possible toxicity.

Figure 2.

Suicide rate per 100,000 residents in Canada between the years 1985 and 2012. Sources: Statistics Canada. Description for chart 8: divorce and suicide rates, per 100,000, Canada, 1950 to 2008 [Internet]. Available from: http://www.statcan.gc.ca/pub/82-624-x/2012001/article/desc/11696-08-desc-eng.htm. Statistics Canada. Suicides and suicide rate, by sex and by age group (both sexes rate) [Internet]. Available from: http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/hlth66d-eng.htm. Statistics Canada. Age-specific mortality with suicide and accident rates, per 100,000, ages 15 to 19, Canada, 1974-2009, deaths per 100,000 [Internet]. Available from: http://www.statcan.gc.ca/pub/82-624-x/2012001/article/desc/11696-06-desc-eng.htm.

Given this lack of evidence for effectiveness, it is necessary to critically consider possible reasons for the popular appeal and widespread dissemination of these programmes. Without the application of good qualitative research, this question cannot be properly answered, so our considerations on this topic remain speculative and warrant further investigation.

Perhaps their popularity is because these programmes are marketed to take advantage of public vulnerability. Suicide is a highly emotional issue, and the promise of suicide prevention, supported by slick marketing techniques, may play a role. Another speculation is that publicly responsible agencies such as the city of Ottawa, various chapters of the CMHA (e.g., Ottawa, Peel, Hamilton, Nova Scotia), universities (e.g., Toronto, York, Carlton, Manitoba), community suicide prevention organizations (e.g., SPIN Manitoba, Halton Suicide Prevention Coalition), and national organizations (e.g., Canadian Association for Suicide Prevention) may have not conducted the rigorous evaluations needed to determine if the programme that they are promoting is both safe and effective. But because of public pressure, they provide or endorse the intervention to appease criticism or to be seen to be doing something, even if doing something is not the same as doing the right thing.

A related concern is that SafeTALK is listed as an evidence-based programme on the website of a US-based national organization that promotes suicide prevention (the Suicide Prevention Resource Center, http://www.sprc.org/bpr/section-i-evidence-based-programmes), suggesting that sufficient care and evidence-driven rigor are not being taken to evaluate programmes. Such endorsement then may provide programme vendors “justification” for their marketing activities and mislead well-meaning organizations and institutions that are searching for safe and effective suicide prevention interventions.

A further speculation is that the purveyors of the programmes do not realize that they are profiting from providing interventions that do not prevent suicide and that may even be harmful.

With knowledge comes responsibility. Thus, given the data summarized above, a number of questions arise. First, why do the programme purveyors continue to market and profit from these so-called suicide prevention interventions without appropriate evidence of effectiveness or safety? Second, why, with no appropriate evidence of safety or effectiveness, do reputable organizations offer or endorse these programmes? Third, why does the Public Health Agency of Canada or Health Canada not intervene in these health-related applications as part of their role in protection of the public? Fourth, why are academics and suicide prevention researchers not forthright and clear in their communication about what has and what has not been found to effectively and safely reduce suicide rates in community-based suicide prevention? It is essential that distal proxy measures (such as improved social cohesion, self-confidence in discussing suicide, or a better knowledge about suicide) resulting from programme application that have little or no relationship to suicide rate reduction be replaced by more proximal and useful measures such as suicide rates or hospital admissions for suicide attempts to measure impact of interventions. Fifth, why do policy makers not demand good evidence for effectiveness and safety of suicide prevention before they apply or fund any suicide prevention programme in schools or in the community?

Clearly, we do not have the evidence needed to continue to apply the above so-called suicide prevention programmes. Indeed, the concern is that these programmes not only may be ineffective but also have not been shown to be safe and may lead to harm. What we do need is much better research (both qualitative and quantitative) to establish what works, what does not work, and what causes harm. There are a number of potentially effective school and community suicide prevention programmes currently under investigation.^11,29 Unlike the programmes described above, these other interventions include enhanced access to clinical care for youth with mental disorders and in-depth training of responsible stakeholders such as teachers and primary care providers in risk determination and appropriate interventions. These should be further studied.

Finally, a caution. When an intervention is being applied and popularly supported, if it is not safe or not effective, this application can often block either the application of an alternative approach that may be safe or effective (an opportunity cost) or necessary investment in finding an approach that is safe and effective (a negative development cost). Continued application of school and community suicide prevention programmes that have not been demonstrated to effectively reduce suicide rates comes with both a potential for harm and a cost.³⁰ Can we continue to afford that?