Predicting adolescent suicidality: Comparing multiple informants and assessment techniques

Abstract

Adolescent suicidality is a serious problem among American youth. Common risk factors for adolescent suicidality include depression and conduct problems but there is little agreement on the best means to assess these factors. We compared multiple informants (mothers, fathers, the adolescent and a sibling) and multiple assessment techniques using a sample of more than 460 families. Assessment techniques included paper-pencil instruments, observer ratings, and diagnostic interviews. Suicidality was assessed concurrently and two years after the risk assessment. Adolescent-reported paper-pencil instruments and diagnostic interviews were strongly associated with concurrent and future suicidality. Parents’ report of adolescent feelings and behaviors were also useful. Observed behaviors were not useful in assessing suicidality risk factors. Clinical recommendations include utilizing paper-pencil and diagnostic adolescent risk factor assessment and focusing on emotions.

Suicide rates among US adolescents continue to alarm mental health therapists, policy makers, and families. Approximately 8% of US adolescents attempt and 17% consider suicide annually (Eaton et al., 2006). Accurate suicidal assessment is needed to reduce these rates (Pfeffer, 2001; Sommers-Flanagan & Sommers-Flanagan, 1995). However, accurate assessment is challenging as many adolescents are not forthcoming with their suicidal inclinations; therefore risk factor assessment is necessary.

Risk factors most consistently linked to adolescent suicidality include internalizing risks such as depressive symptoms and disorder and behavior problems including delinquency and conduct disorder (e.g., Boergers, Spirito, & Donaldson, 1998; Brent, Kolko, Allan, & Brown, 1990; Fleischmann, Bertolote, Belfer, & Beautrais, 2005). Various depression and conduct disorder assessment techniques exist such as paper-pencil symptom inventories, diagnostic interview, and behavioral observation. Multiple informants including adolescents, parents, or other family members can also be used.

To date, there is no consensus on the most accurate means of evaluating adolescent suicidal risk factors. Most suicidal risk factor assessments are designed for adult populations (Guitterez, 2006; Velting, Rathus, & Asnis, 1998). Also, adolescents report suicidal behaviors differently depending on how information is gathered, e.g. paper-pencil inventory versus diagnostic interview (Prinstein, Nock, Spirito, & Grapentine, 2001), and informant usefulness varies (Breton, Tousignant, Bergeron, and Berthiaume, 2002). The professional setting can also affect the outcome. For example, psychiatric risk assessments are more predictive of future self-harm than risk assessments done by emergency room staff (Kapur et al., 2005).

Informant usefulness may vary because informants tap into different symptoms (Achenbach, McConaughy, & Howell, 1987). For example, diagnoses vary by informant because informants endorse different symptoms (Grills & Ollendick, 2002; Jensen et al., 1999). Within families, parents agree more with other informants on externalizing symptoms (Achenbach et al., 1987), but adolescents more accurately report depression (Rubio-Stipec, Fitzmaurice, Murphy, & Walker, 2003). Siblings often agree on depression symptoms but disagree on externalizing (e.g. aggression) (Epkins & Dedmon, 1999). However, this research gives little information about the specific link between risk factor informant and adolescent suicidality.

There is also no agreement on the best technique for assessing suicidal risk factors. Techniques differ conceptually and can provide unique information (Kasius, Ferdinand, van den Berg, & Verhulst, 1997). For example, paper-pencil inventories often produce normed quantitative data whereas diagnoses are categorical (Kasius et al., 1997). Diagnostic interviews and observer ratings typically focus on behavior whereas paper pencil instruments assess global interactions and emotions (Darling & Steinberg, 1993). Paper-pencil and diagnoses are commonly used, but observation can allow a naturalistic, unobtrusive assessment of adolescent behavior (Kaminer, Feinstein, & Seifer, 1995), especially when suicidal individuals are withholding their true intentions (Brent, 2001).

Given the lack of consensus on the most useful means of evaluating adolescent suicidality risk factors, we examined the utility of various techniques and informants. We defined suicidality as suicidal ideation, planning, and attempts. We assessed adolescent internalizing risk (depression) using adolescent, parent, and sibling reports from paper-pencil surveys, independent observer ratings and a diagnostic structured interview. The same informants and mix of techniques were used to assess externalizing risk factors (behavioral problems).

Usually clinicians want to know if the adolescent is currently suicidal, making a concurrent analysis useful. However, understanding future risk can also be beneficial. Therefore, we examined the usefulness of each informant and technique for predicting suicidality concurrently and two years later. We posed two research questions; (1) Which assessment technique is most associated with adolescent suicidality concurrently and two years in the future? and (2) Which informant’s responses are most associated with adolescent suicidality concurrently, and two years in the future?

Method

Sample families participated in the Iowa Youth and Families Project (IYFP) (Conger & Elder, 1994), and the Single Parent Project (SPP) (Simons, Conger, Elder, Jr., Lorenz, & Whitbeck, 1996). Both studies were designed to examine effects of economic difficulties on families and used nearly identical procedures and variables.

Study families (IYFP N = 451, SPP N = 108) were white, primarily lower-middle and middle class and included at least two children. The “adolescent” was in seventh grade (12 – 13 years of age) in 1989. A second child, the “sibling”, was within four years of the adolescent’s age, either older or younger. IYFP families included the children’s two biological parents. SPP families included the children’s biological mother. A single sample with a mix of two- and single-parent families more representative of today’s families was created in 1994 by combining the IYFP and SPP into the Family Transitions Project.

Suicidality measures were first collected in 1992 (adolescent M age = 15.6 years, SD = .56) and again in 1994. Therefore, this study used data collected in those two years. Henceforth, 1992 is referred to as Time 1 and 1994 as Time 2.

By Time 2, two adolescents had died, one by suicide. 525 adolescents remained in the study (retention rate = 94%). 25 adolescents were excluded from the present study because they did not complete a diagnostic interview. Adolescents who had missing data at both times 1 and 2 were also excluded. As described in the Analysis Plan, we estimated an internalizing risk model and an externalizing model. The final internalizing model sample included 464 adolescents (255 girls, 209 boys); 469 adolescents (257 girls, 212 boys) were included in the externalizing risk model.

Six percent of mothers and 9.2% of fathers had missing data. Siblings missing data was higher (25.4%) due to non-participation in Time 1 observational tasks. We used expectation maximization (EM) to recover data missing at random, a reliable imputation method preferred to case deletion (Schafer & Graham, 2002). T-test comparisons of adolescents and families with complete data to those with missing data showed no statistically significant differences on income, education, age, or any study variable. Missing data were, thus, considered missing at random and eligible for data imputation. We used EM (SPSS 14.0, SPSS Inc, 2004) to estimate values for missing data.

Procedure

At Times 1 and 2, families received two home visits that were two weeks apart from a trained interviewer. During the Time 1 and Time 2 first home visit, participating family members independently completed surveys covering, among other things, family member characteristics and family interaction. Participants engaged in four observational tasks during the Time 1 second home visit; tasks were video-recorded and later rated by trained observers. The adolescent participated in a diagnostic interview at Time 2.

Measures

Adolescent-report: Depression

At Time 1, adolescents completed the depression SCL-90-R subscale (Derogatis, 1983). This paper-pencil instrument asks adolescents to report how much they had been bothered by depression symptoms during the past week (1 = not at all to 5 = extremely). One item in the SCL-90-R depression subscale refers to suicidal thoughts (“bothered by thoughts of death or dying”) and was removed from the subscale to avoid artificially inflating the association between self-reported depression and suicidality measures. Responses to the remaining 12 items were summed to produce the Adolescent-report: Depression measure (α =.91)

Diagnosed depression

At Time 2, adolescents completed the University of Michigan modified CIDI (World Health Organization, 1990), a fully structured diagnostic interview that generates lifetime DSM-III-R (American Psychiatric Association, 1987) diagnoses and onset dates with good reliability and validity (Wittchen, 1994). To further assure reliability, all study interviewers, who were blind to diagnoses and other risk factor assessments, underwent a five-day training workshop, and all interviews were audio-taped. Counseling psychology graduate students duplicated the interview schedule using 10% of the audiotapes. There was 100% agreement between field interviewers and students for symptom counts and diagnoses.

Diagnosed depression was defined as the presence of an affective disorder (major depressive episode or dysthymia) by age 16 years (Time 1). Adolescents who experienced at least one affective disorder by age 16 received a score of 1 for the diagnosed depression measure. All others received a score of 0.

Parent-report: Internalizing

At Time 1, parents independently completed 19 paper-pencil NEO-PI (Costa & McCrae, 1985) items adapted to assess their child’s depression and anxiety symptoms. For example, parents were asked how much they agreed that their child is happy, depressed, tense, or irritable, etc. (1 = strongly agree to 5 = strongly disagree). As needed, responses were reverse coded so higher scores indicated higher risk. Responses were summed to create the Parent-report: Internalizing measure (α = .93). Twenty-three percent of the families were mother-headed. In these families, only mother’s report was used. In families with two participating parents, we used the mean of mother’s and father’s responses. (Correlation between mother and father report, r = .56.)

Sibling-report: Internalizing

At Time 1, siblings completed three paper-pencil survey items assessing the adolescent’s depression and anxiety symptoms. Siblings were asked how much they agreed that “he/she is a happy person, “he/she is always sad”, “he/she is always worried”, etc (1 = strongly agreed to 5 = strongly disagree). As necessary, responses were reverse coded so that higher scores indicated higher risk. All responses were summed (α = .55).

Observer-report: Internalizing

Observed internalizing risk was assessed at Time 1. Family members engaged in 4 observational tasks. Data for this study were taken from the first 3 tasks. Trained interviewers began the tasks by asking family members to sit around a table, usually the dining room table. For task one, family members discussed questions about family life designed to elicit typical parent-child interactions. Task two began shortly after task one. During this task, parents and children discussed and attempted to resolve salient family problems. Task three included only the adolescent and sibling who were asked to discuss questions about the sibling relationship and family life designed to elicit typical sibling interactions. For all tasks, a trained interviewer first explained task procedures and then left the room while the family engaged in their discussion.

Trained observers, who were not the interviewers, rated family member behavior using the Iowa Family Interaction Rating Scales (IFIRS, Melby et al., 1998). Observers globally assessed behavior using a scale ranging from 1 = not at all characteristic of the person to 9 = mainly characteristic of the person. Each observer received approximately 200 training hours and demonstrated reliability by passing written and observational examinations prior to rating videotaped interactions. Observer reliability was further assessed by randomly assigning 25% of all tapes to be rated by a second observer. Primary and secondary observer ratings were compared using intraclass correlations (ICC). ICCs ranged from .55 to .85, an acceptable reliability level (Kenny, 1991).

Observed internalizing risk was assessed across all three observational tasks. Each task was independently rated by a different observer. Observers used two scales to assess internalizing symptoms: sadness/depression and anxiety. The sadness/depression scale assessed listless and/or social withdraw verbal and nonverbal behavior. The anxiety scale assessed worry, tenseness, and/or fidgeting. The two ratings were summed across the three tasks to create a 6-item Observer-report: Internalizing measure (α = .66).

Adolescent-report: Delinquency

At Time 1, adolescents completed a delinquency checklist adapted from the National Youth Survey (Elliott, Huizinga, & Ageton 1985). This paper-pencil instrument assessed involvement in aggressive or delinquent behavior. For example, adolescents reported how often during the prior year (1 = never to 5 = 6 or more times) they had beaten up or attacked someone, stolen something, been placed in jail, or driven while drunk. Responses to the 22 items were summed to create the Adolescent-report: Delinquency measure (α =.84).

Adolescent-report: Hostility

At Time 1, adolescents were asked to complete the hostility SCL-90-R subscale (Derogatis, 1983). Using this paper-pencil instrument, adolescents report how much they had been bothered by hostility symptoms during the past week (1 = not at all to 5 = extremely). Responses were summed to produce the Adolescent-report: Hostility measure (α =.89).

Diagnosed conduct disorder

Diagnosed conduct disorder was defined as the presence of a conduct disorder (CD) diagnosis. In accordance with the DSM-III-R, the UM-CIDI (described above) assesses CD based on behavior occurring prior to age 15 years. Adolescents given a CD diagnosis received a score of 1 on the Diagnosed conduct disorder measure. All others received a score of 0.

Parent-report: Externalizing

At Time 1, mothers and fathers independently completed the Revised Behavior Problem Checklist conduct disorder subscale (Quay & Peterson, 1987). Using a 4-point scale (1 = no problem to 4 = severe problem), each parent rated the adolescent’s problem behavior on 22 items. For example, parents reported how much picking fights with others, being difficult to control, and being disobedient were problems for the adolescent. Responses were summed to create the Parent-report: Externalizing measure (α = .95). For mother-headed families, mother’s responses were summed. In two-parent families, the mean of the mother’s and father’s summed responses was used. (Correlation between mother and father report, r = .49).

Sibling-report: Externalizing

At Time 1, siblings completed eleven paper-pencil survey items that asked about the adolescent’s hostile and aggressive behavior. Siblings reported how much they agreed with statements like “he/she gets into a lot of fights”, “he/she sometimes breaks the law”, “he/she enjoys making my life miserable”, etc (1 = strongly agreed to 5 = strongly disagree). Responses were summed (α = .89) to create the Sibling-report: Externalizing measure.

Observer-report: Externalizing

Using the IFIRS, described above, observers assessed adolescent externalizing risk using four scales. The hostility scale assessed angry, demeaning or critical verbal and nonverbal behavior. The antisocial scale assessed out-of-control, uncooperative, or immature behavior. The externalize negative scale assessed expressions of anger, hostility, or criticism towards others, including teachers, schools, or peers. The angry coercion scale assessed verbal and nonverbal attempts to control others through hostility, resistance, or threats. All ratings were summed across tasks 1 through 3 to create the Observer-report: Externalizing measure (α = .93).

Adolescent suicidal ideation, plans, and attempts

At Times 1 and 2, adolescents completed 3 questions taken from the Youth Risk Behavior Survey (Centers for Disease Control, 1991). All three questions used the same response scale (1 = never to 4 = 3 or more times). They asked if during the previous year the adolescent had (1) “seriously thought about committing suicide”, (2) “made a plan for committing suicide”, (3) “attempted suicide”. At Time 1, adolescents who reported having suicidal thoughts received a 1 for the Concurrent suicidal ideation measure. All others received a 0. Adolescents who reported making a suicide plan at Time 1 received a 1 for the Concurrent suicidal plans measure. All others received a 0. Adolescents who reported planning a suicide at time 1 received a 1 for the Concurrent suicidal attempts measure; all others received a 0. Future suicidality measures were created using the same questions and coding procedures at Time 2 questions.

Analysis Plan

Preliminary analyses showed strong correlations between several internalizing and externalizing risk assessments. For example, the correlation between adolescent depression and hostility reports was r = .75 (p <.01). To avoid problems with multicolinearity, we estimated the associations between form of assessment and suicidality separately for internalizing and externalizing risk.

As described above, adolescent suicidal ideation, plans, and attempts were coded as dichotomous variables. Therefore, associations between measures of concurrent and future suicidality and each risk factor assessment were examined using logistic regression. For both internalizing and externalizing risk, we ran six logistic regression models, a suicidal ideation model, a plans model, and an attempts model for concurrent and for future risk. Each model included a product term to estimate the interaction between sex and risk factor assessment. Few and inconsistent interactions were found and are not reported. Also, the conduct disorder variable was not added to the externalizing risk future attempts model because no adolescents with a CD diagnosis reported a future attempt.

Results

Table 1 displays suicidal ideation, planning, and attempts rates at each time point. Ideation rates were higher than planning or attempts rates. Females reported more suicidal ideation than males at Time 1 (χ² = 12.15, p < .001) but were similar to males on plans and attempts at both times, and ideation at Time 2.

Table 1.

Frequency of Suicidality in Entire Sample

Variable	Boys	Girls	Total
Time 1 Ideation	30 (14%)	73 (28%)	103 (22%)
Time 1 Plan	13 (6%)	29 (11%)	42 (9%)
Time 1 Attempt	3 (1%)	11 (4%)	14 (3%)
Time 2 Ideation	38 (18%)	56 (22%)	94 (20%)
Time 2 Plan	15 (7%)	21 (8%)	36 (8%)
Time 2 Attempt	4 (2%)	7 (3%)	11 (2%)

Table 2 presents associations between internalizing risk assessments and concurrent and future suicidality. Controlling other assessment techniques, adolescent-reported depression was significantly associated with each type of concurrent suicidality but only future ideation. Parent report was associated with concurrent ideation and each type of future suicidality. A depression diagnosis produced statistically significant odds of suicidality for all models except future attempts, ranging from 1.80 (p < .05) for concurrent ideation to 5.27 (p < .05) for concurrent attempts. Neither observer reports nor sibling reports were statistically significant associates of any suicidality measure.

Table 2.

Logistic Regression Estimates of Associations between Internalizing Risk Assessments and Suicidality

	Concurrent Internalizing Risk Models									Future Internalizing Risk Models
	Ideation			Planning			Attempts			Ideation			Planning			Attempts
Assessment Technique
	b	Exp(b)	SE	b	Exp(b)	SE	b	Exp(b)	SE	b	Exp(b)	SE	b	Exp(b)	SE	b	Exp(b)	SE
Adolescent Reported Depression	.14*	1.15*	.02	.10*	1.10*	.02	.07*	1.07*	.03	.04*	1.04*	.02	.00	1.00	.02	.00	1.00	.04
Diagnosed Depression	.59*	1.80*	.44	1.24*	3.46*	.48	1.66*	5.27*	.68	1.32*	3.73*	.40	1.46*	4.31*	.46	.55	1.73	.97
Parent Reported Internalizing	.03*	1.03*	.01	.00	1.00	.02	.02	1.02	.03	.01*	1.01*	.01	.04*	1.04*	.02	.11*	1.11*	.03
Sibling Reported Internalizing	.01	1.01	.05	.07	1.07	.07	-.10	.90	.13	.00	1.00	.05	.00	1.00	.07	-.02	.82	.16
Observer Reported Internalizing	.01	1.01	.04	.03	1.03	.05	-.02	.98	.08	.03	1.03	.03	.04	1.04	.05	-.05	.95	.09

N = 464

^*≤.05

Table 3 presents the findings for the externalizing risk assessments. Adolescent-reported hostility was associated with each type of concurrent suicidality but only future ideation. Parent report predicted only future suicidality. Observer reports, adolescent-reported delinquency, sibling reports, and a CD diagnosis were not significantly associated with any suicidality measure.

Table 3.

Logistic Regression Estimates of Associations between Externalizing Risk Assessments and Suicidality

	Concurrent Internalizing Risk Models									Future Internalizing Risk Models
	Ideation			Planning			Attempts			Ideation			Planning			Attempts
Assessment Technique
	b	Exp(b)	SE	b	Exp(b)	SE	b	Exp(b)	SE	b	Exp(b)	SE	b	Exp(b)	SE	b	Exp(b)	SE
Adolescent Reported Hostility	.16*	1.17*	.30	.11*	1.11*	.04	.15*	1.16*	.05	.06*	1.06*	.03	.04	1.04	.04	.08	1.08	.08
Adolescent. Reported Delinquency	.00	1.00	.02	.02	1.02	.02	.05	1.06	.03	.01	1.01	.02	.01	1.01	.02	.01	1.01	.05
Diagnosed CD	-.07	.93	.37	.23	1.25	.48	-.07	.93	.79	-.40	.67	.39	-.18	.84	.52	NA	NA	NA
Parent Reported Externalizing	.02	1.02	.01	.01	1.01	.02	.04	1.04	.03	.03*	1.03*	.01	.04*	1.04*	.02	.09*	1.10*	.03
Sibling Reported Externalizing	.01	1.01	.01	.01	1.01	.01	-.04	.96	.03	.00	1.00	.01	.02	1.02	.01	.00	1.00	.03
Observer Reported Externalizing	.01	1.01	.00	.00	1.00	.01	.01	1.01	.01	.00	1.00	.00	.00	1.00	.01	-.01	.99	.01

N = 469

^*≤.05

CD = Conduct Disorder, NA = Not Analyzed because no adolescent with a CD diagnosis reported a future attempt.

Discussion

Multiple techniques and informants are available to assess risk factors for adolescent suicidality. To our knowledge, this study is the first to directly compare several assessment techniques. Findings indicate that some assessment techniques are more useful than others.

Having an affective disorder diagnosis was a particularly useful indicator of suicidality, as others have found (Fleischmann et al., 2005; Pfeffer, 2001). Adolescents with an affective diagnosis were three times more likely to report planning suicide, four times more likely to think about or plan suicide in the future, and five times more likely to have attempted suicide than those without a diagnosis. A CD diagnosis was not associated with suicidality. We speculate that this difference may be due to a focus on emotions in the affective disorder interview whereas the CD diagnostic interview focuses on behavior (WHO, 1990).

This speculation is supported by our finding that adolescent-reported depression and hostility, which focus on emotions, predicted concurrent suicidality and future ideation. Adolescent-reported delinquency, a behavioral measure, did not predict suicidality. Thus, clinicians may find that including an affective diagnosis and adolescent report of depressed or hostile emotions in their suicidality assessment adds useful information regarding potential suicidality, particularly current ideation, plans, and attempts.

Clinicians often rely on observation to ascertain adolescent feelings, as adolescents may not be forthcoming (Velting et al., 1998). Our findings indicate that clinicians should supplement observational assessment with assessments directed towards the adolescent and possibly parents to avoid erroneous conclusions. However, this recommendation is given cautiously based on two limitations to the current study. One, the observers, although highly trained, were not clinicians. Trained clinicians may be more attuned to observable signs of suicidality. Second, clinicians often observe adolescents in a clinical setting whereas the observers rated behavior occurring in the adolescent’s home. This difference in settings may affect how well our findings generalize to the typical clinician’s practice.

Parents proved to be the most useful family informants. Parent-reported internalizing was associated with concurrent and future suicidal ideation. Surprisingly, parents were more attuned to internalizing and externalizing risk associated with future planning and attempts than concurrent planning and attempts whereas adolescent reports were most predictive of current suicidality. Sibling reports were not predictive of adolescent suicidality. Possibly the siblings in our sample did not accurately understand their siblings’ emotional state or did not accurately report what they knew. This interpretation is tempered by the low reliability of sibling-reported internalizing and the possibility that some siblings were too young to adequately assess another’s mood.

As reported above, internalizing risk assessment techniques more consistently predicted suicidality than a CD diagnosis or adolescent reported delinquent behaviors. However, we caution against entirely excluding externalizing symptom assessments. Other studies have linked CD to completed suicide (Brent, Baugher, Bridge, Chen, & Chiappetta, 1999; Brent et al., 1993; Shaffer et al., 1996) or, in samples of inpatient or incarcerated adolescents, to suicidal ideation and attempts (Ko et al., 2004; Ruchkin, Schwab-Stone, Kaposov, Vermeiren, & King, 2003). We studied ideation, planning, and attempts in a community sample. Possibly, delinquent behaviors and suicidality are more strongly associated in more problematic samples. However, community based clinicians may find focusing on hostile feelings over delinquent behaviors more useful.

Several considerations are necessary when interpreting these results. First, the sample included rural Midwestern Caucasian families with adolescent children. Our results may not apply to children of other ages, ethnicities, and geographic groups. We used adolescent report to measure suicidality, but parents occasionally report suicide attempts when adolescents deny them, indicating parents may add unique information (Ko et al., 2004). To further validate our findings, future research should use multiple informants of adolescent suicidality. Finally, this study could measure only a portion of known suicidality risk factors. Others, like problem solving and impulsivity, should be considered in future research.

Overall, our results endorse the use of a diagnostic interview and paper-pencil measures completed by the adolescent and his or her parent(s) to assess adolescent suicidality risk factors. In community samples, a focus on emotions rather than behaviors in both the diagnostic interview and paper-pencil inventories appears warranted. Finally, adolescent reports appear to be most associated with concurrent suicidality. Parent-reported risk is most predictive of future suicidality.