Feasibility of Screening Adolescents for Suicide Risk in “Real-World” High School Settings

Abstract

Objectives. We evaluated the feasibility of a population-based approach to preventing adolescent suicide.

Methods. A total of 1323 students in 10 high schools completed the Suicide Risk Screen. Screening results, student follow-up, staff feedback, and school responses were assessed.

Results. Overall, 29% of the participants were rated as at risk of suicide. As a result of this overwhelming percentage, school staffs chose to discontinue the screening after 2 semesters. In further analyses, about half of the students identified were deemed at high risk on the basis of high levels of depression, suicidal ideation, or suicidal behavior. Priority rankings evidenced good construct validity on correlates such as drug use, hopelessness, and perceived family support.

Conclusions. A simpler, more specific screening instrument than the Suicide Risk Screen would identify approximately 11% of urban high school youths for assessment, offering high school officials an important opportunity to identify young people at the greatest levels of need and to target scarce health resources. Our experiences from this study show that lack of feasibility testing greatly contributes to the gap between science and practice.

Suicides among youths are a source of significant and preventable loss of life. Suicide is the third leading cause of death among youths aged 15 to 19 years, accounting for approximately 2000 (12%) of the overall number of deaths in this age group each year.1^,2 Suicide death rates among 15- to 19-year-olds doubled between 1960 and 2001, peaking in 1990.3 Suicide among young women remained fairly constant over this period, with the increases occurring predominantly among young men.4^,5 In 2001, suicide rates were 12.9 per 100000 among male youths and 2.7 among female youths.1 Girls were more likely to attempt suicide, however, with an estimated 10.1% of girls and 3.8% of boys having attempted suicide by the age of 19 years.6

Suicide is a complex endpoint on a continuum ranging from ideation to gestures, plans, attempts, and completions.7 According to data from 2003, during that year 17% of US high school students had seriously considered attempting suicide, and nearly 9% had attempted suicide at least once in the preceding 12 months.8 Students in 9th (10%) and 10th (9%) grade were more likely than those in 11th (7%) and 12th (6%) grade to have attempted suicide in the past year.8 The strongest risk factors for youth suicide are depression, substance abuse, aggressive or disruptive behaviors, and previous suicide attempts.6^,9^,10 Among girls, the most important risk factor is major depression (increasing risk by up to 20-fold), followed by a previous suicide attempt; among boys, previous attempt is the most important predictor (increasing risk by more than 30-fold), followed by depression, alcohol or drug abuse, and disruptive behavior.11^,12

Current data on suicide risk factors among ethnic minority youths are scarce. The major source of such data, the Centers for Disease Control and Prevention’s biennial Youth Risk Behavior Surveillance Survey, indicates that White and Hispanic students are more likely than are Black students to consider attempting suicide or to have a plan, and Hispanic students are more likely than are White or Black students to have attempted suicide.8 Female Hispanics are particularly vulnerable to suicidal ideation and attempts,13 with the risk rate among this group estimated at 17.8%.14

Nearly half of depressed adolescents report being preoccupied with death and suicide.6 Although there is a strong association between suicidal ideation and depression, suicidal ideation may occur in the absence of depression.15 Adolescent suicide is often precipitated by a psychosocial stressor such as recent loss, rejection, or a disciplinary crisis,16 although such events are common in this age group. In fact, only a fraction of adolescents among whom these risk factors are present actually commit suicide. Strong social support networks and peer relationships are protective.17^,18

An effective population-based strategy to prevent adolescent suicide should include a school-based approach, given that such an approach is the most efficient way to reach large numbers of young people. However, in a systematic review of curriculum-based adolescent suicide prevention programs, Ploeg et al.19 found insufficient evidence to support these programs; moreover, they noted some indication of harmful effects, particularly among certain subgroups. Other research has shown that boys exposed to a curriculum-based suicide prevention program were more likely than boys not exposed to such a program to consider suicide as a possible solution to their problems and that suicide attempters find talking about suicide in class upsetting and believe it can result in some adolescents being more likely to attempt suicide.20^,21

In contrast, there is considerable support for direct case-finding strategies that consist of confidential surveys followed by more intensive assessments of young people shown to be at risk for suicide.16^,22 Only a small number of empirical studies have assessed the effectiveness or feasibility of such an approach in schools, however. The few studies that have been conducted suggest that suicide screenings in schools may face significant resistance. Surveys of both high school principals and school psychologists reveal that they view schoolwide student screening programs as less acceptable than curriculum-based or staff in-service programs.23^,24 Insufficient staffing and budgets, scheduling issues, legal concerns, and potential negative responses of administrators, parents, and students are potential “real-world” barriers to program adoption and implementation.25

In addition to demonstrating strong psychometric validity, a worthwhile screening instrument must be adopted, implemented, and maintained in field settings.26 We evaluated the feasibility of an adolescent suicide screening measure designed for use in public schools. Although clinical guidelines have advocated case finding through school-based screening,9^,16 we know of no such field test of feasibility reported in the literature. We report findings from our attempt to implement a well-known screening instrument at 10 high schools in 2 large urban districts. We also provide recommendations for further developing the instrument so as to improve its utility in school settings.

METHOD

Sample and Setting

Students in grades 9 to 11 were recruited from high schools in a large city in the southwestern (site A) and Pacific coast (site B) regions of the United States. The district ethnic composition at site A was 87% Hispanic, 9% Black, and 4% White; 90% of the district’s students qualified for the federal free/reduced-price lunch program. The ethnic composition at site B was 39% Asian/Pacific Islander, 21% Hispanic, 15% Black, 15% “other,” and 10% White; 49% of students qualified for free or reduced-price lunches.

Students were participants in a randomized, controlled trial assessing the effectiveness of Reconnecting Youth, a semester-long, indicated (i.e., targeting young people who are already experimenting with drugs or who exhibit other risk-related behaviors, as opposed to universal programs targeting all youths) prevention program taken as an elective. The curriculum integrates life skills training modules in an interactive peer group context designed to enhance high-risk adolescents’ personal (e.g., goal setting and problem solving) and social (e.g., positive social interaction) protective skills.27 Because these young people tend to have problems in multiple areas, Reconnecting Youth seeks to develop skills related to improved school performance, decreased substance use, and improved mood management.27^,28

All Reconnecting Youth study participants were included in the suicide screening. Students were identified as being at high risk if (1) they were in both the upper 25% distribution in terms of absences and the lower 50% distribution in terms of grade point average or (2) they had been referred by a teacher. A random sample of students meeting the criteria for high-risk classification were invited to take part in the study in spring and fall 2002. With the consent of their parents, those who agreed to participate completed a baseline questionnaire. Of the 1995 high-risk students eligible to be included across all schools, 930 (47%) completed the questionnaire. These high-risk participants had significantly higher mean grade point averages (1.51 vs 1.33) and rates of attendance (86% vs 84%) than did the overall pool of high-risk students; there were no differences in gender or race/ethnicity.

Subsequently, a sample of “typical” students was randomly selected to establish “normative behaviors” in the study schools and districts. Of the 608 “typical” students selected, 393 (65%) completed the survey. There were no significant differences between the “typical” students who took part in the study and the overall pool of “typical” students.

Measures

The High School Questionnaire (HSQ)27^,29 includes items assessing school deviance and school connectedness; peer bonding; family support; emotional issues such as self-esteem, stress, anxiety, hopelessness, and personal control; high-risk behaviors; substance use; suicide; and demographic variables such as age, gender, grade, ethnicity, and family structure. HSQ multi-item scales have demonstrated acceptable reliability and validity in previous studies.29^,30 In this study, the HSQ was adapted to an audio-computer-assisted self-interviewing format intended to assist students with limited reading ability, decrease missing data and data processing times,31 and improve veracity in terms of reporting sensitive information.32 Only data collected at baseline were used in the present analyses.

The Suicide Risk Screen (SRS),33 a screening instrument embedded in the HSQ, focuses on sets of rank-ordered criteria based on confirmed suicide risk factors (Table 1 ▶). The SRS is unique in assessing the full range of factors found to predict suicide in psychological autopsy studies.34 The developers have established preliminary construct, discriminant, and predictive validity in independent samples.35 Validity against clinician rating scales and other standard instruments has been reported in a diverse sample of high-risk youths, with sensitivity rates ranging from 87% to 100% and specificity rates from 54% to 60%.33

TABLE 1—

Criteria of the Suicide Risk Screen for Identifying At-Risk Youths

Criteria Set 1 (Evidence of Any 1)	Criteria Set 2 (Evidence of Any 2)
High suicidal ideation (≥ 2 on a 0 to 3 scalea; ≥ 3 times in past month owing to alcohol or drug use, on a 0 to 6 scaleb )	Moderate suicidal ideation (≥ 1 on a 0 to 3 scalea; ≥ 1 time in past month owing to alcohol or drug use, on a 0 to 6 scalec )
Serious depression (≥ 1.7 on a 0 to 3 scalea )	Moderate depression (≥ 1 on a 0 to 3 scalea)
Previous suicide attempts (≥ 1 in past year)	Indirect/direct threats of suicide (≥ 1 on a 0 to 3 scalea; ≥ 1 in past year on a 0 to 6 scaleb)
	High drug use, polydrug use, and/or drug use problems (on a 0 to 6 scalec: alcohol use ≥ 4, marijuana use ≥ 3, hard drug use ≥ 2, polydrug use ≥ 2, drug use problems ≥ 2)

^a 0 = never, 1 = sometimes, 2 = usually, 3 = always.

^b 0 = not at all, 1 = once, 2 = twice, 3 = 3 times, 4 = 4 times, 5 = 5 times, 6 = 6 or more times.

^c 0 = not at all, 1 = once, 2 = 2 or 3 times, 3 = about once per week, 4 = several times per week, 5 = almost every day, 6 = every day during the past month.

Procedure

Study personnel examined extant suicide prevention policies in both school districts and negotiated necessary adaptations for screening youths. All of the schools contacted voluntarily agreed to participate in the study. HSQ data were collected from study participants and analyzed by researchers within 48 hours. Lists of students meeting SRS criteria were then sent to the respective schools. A primary school contact was designated to verify that each student had been assessed within 1 week of list receipt. The contact was asked to interview the student or delegate the interview to another qualified staff member in accordance with all applicable school policies. If there was cause for concern and a referral was necessary, parents were notified and provided a list of community resources. Site research coordinators monitored protocol compliance.

Feasibility Evaluation

Initially, we evaluated feasibility by examining the percentages of young people rated as at risk of suicide. We then conducted bivariate analyses to examine associations between suicide risk and demographic factors. Subsequently, we documented the availability and performance of school personnel in follow-up interviews of young people classified as at risk and solicited staff and school feedback to assess support for continuing implementation.

Additional Analyses

We conducted additional analyses to determine whether students rated as at risk could be further classified according to degree of risk. Using the available literature as a guide,6^,12^,16 we developed 5 mutually exclusive risk level categories: (1) highest (past year suicide attempt in combination with a high level of current suicidal ideation or serious depression), (2) very high (past year suicide attempt), (3) high (no past year suicide attempt but high current suicidal ideation or serious depression with any ideation), (4) moderate (other SRS criteria met), and (5) low (SRS criteria not met). We examined risk factor levels according to demographic characteristics, screening items, and additional questionnaire items related to suicide (belief that suicide is an acceptable personal option, hopelessness, drug use, school connectedness, and family support). Finally, we compared rates of follow-up according to risk level.

RESULTS

Screening

Overall, 389 students (29%) were rated as at risk for suicide. The female-to-male ratio among these students was nearly 2:1 (38% vs 20%). Screening rates were similar across sites: 30% at site A and 29% at site B. High-risk students were more likely than “typical” students to be rated as at risk (33% vs 21%).

Follow-Up Interviews

Of the 389 students rated as at risk, 120 (31%) did not complete a follow-up interview. The protocol directed school staff to make at least 3 attempts to reach students at school or home. Seventy-three students were not located after the requisite number of attempts; no attempt was made to interview the remaining 43 students. Schools varied greatly in terms of quality and timing of follow-up interviews at each of the sites. Protocol compliance was best when the designated contact or team conducted all of the interviews. Compliance was poorest when the primary contact assigned interviews to the students’ academic counselors, a common deviation from the protocol. No school in either area was able to complete all of the interviews within the 1-week time frame.

At 3 site A schools, the primary contact was a master’s-level social worker or counselor who personally completed all follow-up interviews within 2 to 4 weeks of the initial screening. This individual also personally contacted all parents through home visits, school meetings, or telephone conferences. At a fourth school, no follow-ups were conducted during the first semester, but counselors completed most as assigned during the second semester. At a fifth school, no follow-up interviews were conducted. Rather, the assistant principal sent a letter to the parents of all youths rated as at risk, notifying them about the availability of mental health services.

Site B had a federal grant to establish school and community partnerships to promote student health. Of the site’s 5 schools, 4 had grant-funded wellness coordinators (who served as the primary contacts in this study), a community health outreach worker, a half-time substance abuse counselor, and a half-time mental health counselor. All of the site’s schools had a nurse and academic counselors. These staff members participated in follow-up interviews in varying combinations in the different schools. The fifth school had a school-based clinic staffed by doctors, nurses, therapists, and health educators, all of whom conducted follow-up interviews.

Although site B schools appeared to have adequate staff, the numbers of students identified for follow-up overwhelmed school personnel. Two schools used a designated team to assess all identified students. The remaining schools required persistent monitoring and reminders, and one school ultimately needed staffing assistance to complete the follow-ups.

Staff Feedback

Even some of the most ardent supporters of the screening process at site A questioned the feasibility of conducting follow-up interviews given the numbers of students identified as at risk. An administrator at the school that sent letters to parents rather than conducting student interviews reported that counselors there had not believed that some of the students identified were really at risk, and they were concerned about unduly alarming students and parents. Another administrator raised concerns about liability if an identified student who had not been followed up attempted suicide.

Personnel at each of the site B schools wanted to discontinue screening at the end of the second semester. Because of a heavy regular workload, they considered it unrealistic to assess 10 to 25 students during a given week. Staff did not perceive great value in the screening, in that most of the students assessed were found to be at low to moderate risk and were difficult to locate as a result of truancy. Two comments from school staff members provide insights into their perspectives as well as those of others:

The process is totally unrealistic given the numbers we are getting. I really believe the people who should be doing the assessment are the counselors, since they are responsible for the students, but . . . they already are overloaded with 600 students apiece. They cannot drop everything to follow up on the numbers we are getting. . . . Although we have wellness staff, they have students on their caseloads that they need to see. It is getting into the holiday season, which can be very stressful for students; staff needs to be available to help them prepare for the holidays. . . . Out of all the [assessments] we have done, we have only had one that was serious enough to get [the student] into counseling right away.

I spoke with [the district coordinator] this afternoon and she thinks we need to call another meeting to discuss pulling the suicide questions [from the survey]. The school staff is “totally overwhelmed” and she’s meeting with some resistance as she’s giving them the lists of students who are screening in during Q2 [the survey] this week. It’s not so much that they don’t want to do it . . . as it is feeling like they can’t. They’ve just finished dealing with all the kids who screened in during recruitment and are not happy about having to deal with more kids now, and are worried about the ever-growing list of names.

At the end of the 2 semesters, both school districts decided that they no longer wanted to participate in the suicide screening, and it was dropped from the HSQ and the study.

Additional Analyses

Given staff feedback, we conducted additional analyses to determine whether students could be prioritized so that counselors could assess those in greatest need first. Applying the study criteria (as outlined in the Methods section) to students rated as at risk, we found that 41 (11%) were in the highest risk factor category, 81 (20%) were in the very high category, 76 (20%) were in the high category, and 191 (49%) were in the moderate category (Table 2 ▶). A comparison of the 2 study groups (high risk and “typical”) revealed that high-risk students exhibited greater risk, with twice the percentage of these students falling in the highest risk category.

TABLE 2—

Sample Characteristics by Suicide Risk Screen Status and Suicide Risk Factor Level

	Suicide Risk Screen Status/Risk Factor Level
	Low, No. (%)	Moderate, No. (%)	High, No. (%)	Very High, No. (%)	Highest, No. (%)	Total, No.
Overall	934 (71)	191 (14)	76 (6)	81 (6)	41 (3)	1323
Gender
Male	508 (80)	77 (12)	27 (4)	18 (3)	7 (1)	637
Female	426 (62)	114 (17)	49 (7)	63 (9)	34 (5)	686
Site
Site A	414 (70)	79 (13)	37 (6)	35 (6)	25 (4)	590
Site B	520 (71)	112 (15)	39 (5)	46 (6)	16 (2)	733
Study group
High risk	623 (67)	148 (16)	64 (7)	62 (7)	33 (4)	930
Typical	311 (79)	43 (11)	12 (3)	19 (5)	8 (2)	393
Ethnicity
Hispanic/Latino	420 (69)	86 (14)	34 (6)	39 (6)	26 (4)	605
Asian/Pacific Islander	260 (70)	58 (16)	21 (6)	25 (7)	7 (2)	371
Black	155 (78)	26 (13)	10 (5)	8 (4)	1(0.5)	200
White	57 (63)	15 (16)	8 (9)	7 (7)	4 (4)	91
Mixed or other	33 (79)	3 (7)	1 (2)	2 (5)	3 (7)	42
Missing	9 (64)	3 (21)	2 (14)	0 (0)	0 (0)	14

As the number of suicide risk factors rose, students were increasingly likely to engage in more drug use, to have less family support, to be less connected to school, to have more feelings of hopelessness, and to more frequently endorse suicide as an acceptable solution to life’s problems (Table 3 ▶). Because the SRS protocol eschewed prioritization in terms of student follow-up, however, each group showed similar follow-up rates, ranging from 68% to 72%.

TABLE 3—

Mean High School Questionnaire (HSQ) Item Scores, by Suicide Risk Factor Level

	Suicide Risk Factor Level, Mean (SD)
Item	Low	Moderate	High	Very High	Highest
Belief in suicide as a personal option	0.04 (0.27)	0.28 (0.59)	0.86 (0.90)	0.87 (0.74)	2.05 (1.02)
Hopelessness	0.27 (0.56)	0.96 (0.86)	1.55 (0.89)	1.21 (0.82)	2.02 (0.95)
School connectedness	10.39 (1.96)	9.57 (2.09)	8.87 (2.11)	9.20 (2.21)	9.01 (2.25)
Perceived family support	1.94 (0.82)	1.52 (0.89)	1.20 (0.84)	1.09 (0.72)	1.09 (0.81)
Attempt in past month	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.52 (0.81)	2.12 (2.06)
Attempt in past year	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	1.80 (1.33)	3.15 (2.21)
Suicidal ideation	0.06 (0.25)	0.49 (0.50)	1.45 (0.77)	1.09 (0.75)	2.34 (0.62)
Ideation because of alcohol or drug use	0.003 (0.06)	0.07 (0.32)	0.65 (1.53)	0.36 (0.80)	2.80 (2.67)
Depression	0.57 (0.43)	1.34 (0.66)	1.91 (0.57)	1.39 (0.62)	2.46 (0.36)
Suicide threat in past year	0.01 (0.13)	0.73 (1.37)	1.28 (1.84)	2.00 (1.99)	3.95 (2.13)
Substance use in past month	0.15 (0.57)	0.36 (0.87)	0.42 (1.04)	0.56 (1.12)	1.39 (2.27)

DISCUSSION

Our findings suggest that there is a great need to address suicide-related mental health problems in large urban high schools. Similar to national estimates,8 9% of screened students reported that they had attempted suicide during the past year. During the period in which these data were collected, 1 young person in each of the 2 school districts had committed suicide. Neither had been participants in the screening program, but their deaths underscored the seriousness of the problem. Despite the promise of the approach, we found that a recognized screening instrument developed for use in high schools was not, in its present form, practical for adoption by schools.

Although our analyses support prioritization of students rated as at risk of suicide, the SRS developers have argued that a high level of tolerance of false positives is necessary because of the seriousness of missing a single suicidal individual.33 Assessments of screening instruments involve tests of their ability to predict risk—that is, their positive predictive value (PPV)—but in the case of suicide, base rates affect PPV.36 The PPV for Thompson and Eggert’s high-risk sample (n = 576) was 37% (i.e., roughly one of each 3 students rated as at risk according to the SRS was deemed at risk by clinicians), for example, but this assumed a 22% base rate.33 In contrast, the developers of the Columbia Suicide Screen assumed base rates of 4% to 10% in calculating PPV.34 Using published SRS data,33 we recalculated PPVs with base rates of 9% (Youth Risk Behavior Surveillance Survey suicide attempt rate) and 11% (high-risk suicide attempt rate from the present study). This analysis yielded PPVs of 18% and 21%, respectively, corroborating the feedback from school staff members.

We believe that the SRS’s low specificity results in too many false positives for the instrument to be practical in schools. We recommend testing a simplified instrument that identifies only students who have attempted suicide in the past year, who are at high current levels of suicidal ideation, or who are at high current levels of depression and exhibit any degree of suicidal ideation. Our analyses showed a dramatic jump in most risk and protective factors between students at moderate and at high risk.

Because high-risk youths were oversampled in this study, more accurate estimates can be derived through weighting of the data. (We adjusted the data by weighting as follows: We divided the desired proportion [0.8] by the actual proportion [0.3] to obtain the weight for the “typical” group [2.67] and engaged in a similar process to obtain the weight for the high-risk group [0.2/0.7 = 0.286].) High-risk youths actually represented about 20%, rather than 70%, of all students in the sample schools. Assuming 80% “typical” and 20% high-risk students, 11% of students would be expected to be classified in the high, very high, or highest group (4%, 5%, and 2%, respectively), and 12% would be expected to be classified in the moderate group. Thus, in a school with 1000 students, the SRS would indicate 23%, or 230 students, as being at risk; limiting the screening to the high, very high, and highest groups would decrease the number to 110. Even this reduced number, however, would require schools to systematically screen students over the full school year (perhaps 100 students per month) so as not to overwhelm assessment staff.

We found other problems in addition to the screening instrument itself. School personnel were reluctant to participate in training, which was offered but often declined. Most believed they had sufficient professional training, but, on the basis of our experience, we suggest that training on a standard assessment protocol be provided to all involved staff. Recent budget crises37 and the increased academic accountability standards of the No Child Left Behind legislation (Pub L No. 107–110), however, may leave schools unable to take on a screening program alone. An alternative might be for schools to partner with local health departments, adolescent health clinics, or mental health agencies to conduct periodic screenings.38

Another problem was school readiness. Appropriate personnel were available in most of our sample schools to conduct follow-up interviews, and most supported and expressed great interest in doing so. Nevertheless, not all staff and administrators were enthusiastic. Diffusion theory39 suggests that although some organizations embrace innovation, others prefer to wait and observe as others attempt new programs, while still others prefer to wait a significant amount of time before changing their ways. Excessive staff turnover and changes in administrative personnel can further hamper successful implementation. In each of the 2 districts, we found that only a few schools were able to comply with the protocols and act as partners in evaluating the program. On the basis of our experience, we recommend identifying schools that are both stable and innovative and limiting feasibility research to such schools. After sufficient testing and refinement, the program can then be tested in other schools.

Several limitations of this study should be noted. We took advantage of the opportunity afforded by a large evaluation study—one that involved the use of an existing questionnaire with an embedded suicide prevention screening instrument—to test the feasibility of a screening program. All of the study schools were located in 2 large urban districts. The researchers, rather than the schools themselves, collected and analyzed the screening data. The study design was observational, and there was no attempt to test predictive validity. The researchers subsequently analyzed the data according to hypothesized, literature-based levels of risk, but further rigorous testing of the recommended revised instrument is required. Although other promising adolescent suicide screening instruments are available, a recent comprehensive review showed that thorough, systematic evaluations of properties and utility are generally lacking.40

Our experience indicates that feasibility testing is essential to narrow the gap between science and practice. We wish to underscore the seriousness of the public health problem of suicide and the important opportunity to identify young people with serious, suicide-related mental health problems through school-based programs. Students exhibiting high levels of suicide risk factors are also more likely to have health problems related to drug abuse and risky sexual behavior.41 Arguably, students in this group are most in need of scarce school-based health resources, and identifying them for services should be a high priority. Nevertheless, additional population-based studies are needed to confirm whether this approach actually leads to improved health outcomes and reductions in suicide behavior and depression among high-school populations.