An Examination of the Validity of Retrospective Measures of Suicide Attempts in Youth

Abstract

Purpose

This study used prospective data to investigate the validity of a retrospective measure of suicide attempts from 4 different perspectives.

Methods

Data were from 883 participants in the Raising Healthy Children project, a longitudinal study of youth recruited from a Pacific Northwest school district. The retrospective measure was collected when participants were 18 to 19 years old and results were compared to measures of depressive symptoms collected prospectively.

Results

Results showed strong corroboration between retrospective reports of first suicide attempt and prospective measures of depression, with attempters experiencing significantly more depression than their nonattempting peers, t (df = 853) = 10.26, p < .001. In addition, within the attempter group, depression scores during the year of their reported first attempt were significantly higher than the average depression score across prior years, t (df = 67) = 3.01, p < .01.

Conclusions

Results from this study suggest that the reports of older adolescents regarding their suicide attempts are corroborated by their prospective reports of depression in childhood and earlier adolescence. Thus, there is support that retrospective measures of suicidal behavior, namely suicide attempts, may be a valid method of assessment.

Suicide-related behaviors are exhibited by many children and adolescents in the United States [1-3]. According to the Centers for Disease Control and Prevention [4], suicide ranked as the third leading cause of death in the age group 10 - 14 (middle school) and 15 - 19 (high school). Suicides are part of a broader spectrum of suicide-related thoughts and behaviors [3]. Results from the recent Youth Risk Behavior Survey [4] found that 16.9% of high school students seriously thought about suicide in the prior year and 8.4% made an actual suicide attempt during the past 12 months. With a significant amount of youth engaging in suicide-related thoughts and behaviors, it is important to accurately identify the timing of suicide-related thoughts and behavior to allow for more in-depth research into the longitudinal and concurrent risk factors related to their suicidal behavior.

One of the current gaps in the field of suicidology research is the onset of suicide-related thoughts and behaviors and the risk factors during childhood or adolescence that lead to initial engagement in suicide-related thoughts and behavior. In 2001, the U.S. Department of Health and Human Services identified this gap as a major focus in their National Strategy for Suicide Prevention: “Increasing the understanding of how individual and environmental risk and protective factors interact with each other to affect an individual’s risk for suicidal behavior…” [5, p. 14]. Yet this gap remains because there is a lack of longitudinal datasets containing prospective information regarding suicide-related thoughts and behaviors from early childhood through young adulthood. One remedy to this gap is to include retrospective measures in longitudinal studies once youth are 18 years old. This paper focuses on examining the validity and reliability of retrospective reporting of suicide attempts (type I and type II – not differentiated in this paper [6]) in an existing longitudinal dataset.

Identification of Previous Suicide Attempts

The identification of suicide attempts in youth, whether assessed prospectively or retrospectively, presents validity challenges. In using prospective reports, respondents may fail to report such an event due to embarrassment and/or fear of having parents contacted [7]. An advantage of retrospective reports (post 18 years old) is that they may reduce embarrassment because the event may be more distant and the need to contact parents is removed. Conversely, young adults may suffer from selective memory distortion [8, 9] and inaccuracies in the timing of the event which makes it difficult to use prospective data to investigate proximal predictors of youth suicide attempts. Finally, both methodologies suffer from the possibility of individuals’ over reporting their suicide-related thoughts and behavior in hopes of getting attention [10]. The present study provides information on the validity and reliability of a retrospective measure of suicide attempts, matching prospective longitudinal data on depressive symptoms with retrospective reports of suicide attempts in a community sample.

Retrospective Measures in Related Fields

Investigations of the validity of retrospective measures of other adverse childhood experiences, such as physical and sexual abuse, have shown mixed results [7, 11, 12]. Hardt and Rutter [13] reviewed the literature on adult retrospective measures of childhood experiences. They found a substantial rate of false negatives, suggesting that adverse events such as physical or sexual abuse are likely to be underreported using retrospective measures [13]. Tajima and colleagues [7] compared retrospective parent reports of physical abuse to prospective adolescent measures and found that both prospective and retrospective measures were significantly predictive of key outcomes in adolescence.

Validity Components of a Retrospective Measure of Suicide Attempt/s

We examined several aspects of the validity of a retrospective measure of suicide attempts. First, we examined concurrent validity. To accomplish this, a retrospective measure of suicide attempt/s was compared with a prospective measure of depressive symptoms. [3, 14]. Research examining the relationship between depression and suicide-related thoughts and behaviors among young adults and adolescents reports moderate to strong concurrent correlations ranging from .50 to .60 [15, 16].

The second type of validity evidence was obtained by matching the timing of the prospective data on depression with the retrospective measure of suicide attempts. This type of analysis examined the amount of agreement or lack thereof between the retrospective and prospective measures, which only a few research studies have collected [17]. In related fields where prospective and retrospective measures have been used, such as child abuse and drug use [7, 11], research has found agreement between retrospective and prospective measures ranging from .60 to .70, but few have examined the agreement of the timing of depression to retrospective measures of past suicide-related behavior.

Third, we investigated whether individuals who have made multiple suicide attempts had elevated levels of depressive symptoms at the time of their first reported suicide attempt compared to those who only made a single suicide attempt. Although Kessler and colleagues [18] were able to differentiate single versus multiple attempters, they did not have prospective data to compare the agreement of their retrospective measure between the two groups.

Reliability

Reliability, in the form of test-retest reliability, was the final component of the retrospective measure that was evaluated. Specifically, we assessed the level of agreement for the same person across two administrations (approximately 6 months apart) using the same retrospective measure of suicide attempt.

Need for Valid and Reliable Retrospective Measures of Suicide Attempts

The need for determining if retrospective measures of suicide attempts have high degrees of validity and reliability is important because most research that analyzes suicide-related behaviors uses retrospective rather than prospective questions [15, 17, 19]. If the retrospective method of measuring suicide attempts is validated through prospective measures, then adding retrospective measures of suicide-related behavior in existing longitudinal datasets would contribute significantly to our understanding of why youth engage in suicide-related thoughts and behaviors.

From the practitioner/clinician viewpoint, youth who engage in suicide-related thoughts and behaviors are often not identified in middle or high school [3, 20]. Having measures that may be indicative of suicide-related thoughts and behavior validated with the use of a retrospective measure of suicide attempts will be useful for mental health professionals working with high-risk youth and young adults.

Purpose of This Study

The purpose of this study was fourfold: 1) to examine the concurrent validity of the retrospective measure and the prospective measure of depressive symptoms during the year the adolescent reported his/her first suicide attempt. 2) to examine the element of time using 5 years as a cutoff marker, comparing the relationship of the retrospective measure of suicide attempts with the prospective measure of depression within 5 years versus more than 5 years. 3) to examine if adolescents who reported making multiple attempts were more or less in agreement with their reporting of their first suicide attempt as measured by their prospective depressive symptom scores than adolescents who had made only one attempt. Finally, 4) to examine the test-retest reliability of youth’s retrospective report of timing of suicide attempt over a 6-month period.

Methods

Participants

In 1993 and 1994, 1,040 students and their parents (76% of those eligible) from 10 suburban public elementary schools in a Pacific Northwest school district consented to participate in a longitudinal study examining the effectiveness of the Raising Healthy Children (RHC) preventive program in which schools were randomly assigned into a treatment or control condition [21, 22]. At recruitment, 52% were in first grade. Prior to baseline data collection, parents provided written consent for their children’s participation. After age 18, youth participants provided written consent for subsequent data collection. All procedures were approved by a University of Washington Institutional Review Board. Of the 1,040 initial participants, 883 (85% retention rate) completed post-high school surveys 12 years later.

The average age of the sample during the first year was 6.21 years (SD = .62), and 12 years later, 18.94 years (SD = .33). There were 53.6% males in the longitudinal sample and the ethnicity included 81.4% European Americans, 7% Asian Americans, 4.8% Hispanic Americans, 4.1% African Americans, and 2.8% Native Americans. Twenty-nine percent of the sample qualified for public assistance and/or the free or reduced-price lunch program at baseline.

Data collection

Data analyzed in this study were collected annually starting in 1994 when participants were in second or third grade (ages 7 to 8 years). Although early data collection included parent- and teacher-reported information, only the self-reported data were used for this study. The retrospective questions regarding past suicide attempt/s were collected during the fall in 2005 when participants had just completed high school or were 1 year post high school (ages 18 to 19 years). Annual data were collected by interviewers who received training in assessing childhood and adolescent issues, including suicide-related thoughts and behaviors. The questions about suicide were self-administered by participants during a Computer Assisted Personal Interview (CAPI) with sensitive questions, including the suicide-related behavior questions, thus providing a higher level of privacy [23].

Measures

Retrospective measure of suicide attempt

The retrospective measure of a past suicide attempt was administered when subjects were 18 and 19 years old. This measure contained 4 questions, including: “Have you ever attempted suicide?”, “If so how many times?”, “When was your first suicide attempt?”, and “Did you receive medical treatment for your suicide attempt/s?” Questions regarding attempt history and medical treatment had a Yes/No response format. Students were categorized as having attempted suicide if they answered “Yes” to the question “Have you ever attempted suicide?” regardless of whether they completed the follow-up questions. These questions were repeated approximately 6 months after the fall data collection.

Depressive symptom measure

Students completed a self-report depressive symptoms measure composed of 7 selected items from the Seattle Personality Questionnaire (SPQ) [24]. These items assessed depressive symptoms such as sleep difficulties, wanting to be alone, feeling tired, and feeling like crying. Items were scored on 4-point Likert scale with item response options being NO/no/yes/YES, with 1 representing “NO” and 4 representing “YES.” This scoring system was introduced by Clifford [25], and has been validated and used extensively over the years in other studies [26-28]. The mean depression scores for the depression measure were computed by summing items and dividing by the number of items. Depressive symptom scores were included from when participants were in grades 3 or 4 to grade 12. The internal consistency reliability for the 7-item scale ranged from .69 to .86. Content validity was established by comparing measured items to similar items on other gold-standard child depression self-report measures such as the Child Depression Inventory [29] and the Reynolds Child Depression Scale [30]. Strong concurrent validity was established for this scale by comparing it with the larger 10-item depression scale on the SPQ; the resulting correlation coefficient was r = .90 (N. Riggs, personal communication, 2003).

Data analyses

Data analyses were conducted using SPSS 15.0 (SPSS©). Descriptive statistics were used to examine the demographic characteristics of the sample. We used listwise deletion of cases.

Results

Descriptive results

At the fall 2005 data collection time point, 78 (8.8%) participants reported having made a suicide attempt, with 38 students having made a single attempt and 40 having made multiple attempts. There were significantly more females (50) attempting suicide than males (28), χ² (df = 1) = 10.18, p < .001. Of the 78, 1 skipped the month and year question and 1 answered “don’t know.” Both of these were single attempters. Among the 36 single attempters, 19 reported making their attempt in high school compared to 10 making their attempt in elementary or middle school and 7 making their attempt in young adulthood, χ² (df =2) = 6.50, p = .039. In comparing single versus multiple attempters regarding when they made their first suicide attempt, there was a greater proportion of multiple attempters making their first attempt in elementary/middle school (χ² (df =2) = 6.70, p = .035). The results are presented in Table 1.

Table 1.

A comparison of nonattempting peers, single attempters, and multiple attempters across depression symptom mean scores and when the first suicide attempt occurred

Variable	Nonattempting peers (n = 805)	Single attempters (n = 36)	Multiple attempters (n = 39)	Test statistic1	p – value
School category at time  of first attempt				χ2 (df = 2) = 6.70	.035*
Elementary/middle	0	10	18
High school	0	19	20
Post high school	0	7	1
Depressive symptom  mean scores
Overall mean score	1.95 (SD = .39)
Year of first suicide  attempt	NA	2.44 (SD = .65)	2.55 (SD = .70)	.78	.437
All prior years of first  suicide attempt	NA	2.17 (SD = .53)	2.37 (SD = .56)	1.54	.129

Note:

^*p < .05

¹A comparison between single and multiple suicide attempters

3 suicide attempters did not provide data on when they attempted suicide, thus the number of attempters does not add up to 78.

Concurrent validity

This analysis examined the 67 students who had reported their first suicide attempt during elementary/middle or high school and who had a depressive symptom score during the year of their first attempt and at least one year prior to their first suicide attempt. There was a significant difference in average depression scores across all time points prior to the suicide attempt (M = 2.30) versus during the year of the first reported attempt (M= 2.52, t (df = 853) = 3.01, p < .01.

Retrospective report agreement with prospective measure of depression

Analyses were conducted to examine the level of agreement between the retrospective measure among attempters who had made their first suicide attempt within the 5 years prior to their retrospective report (meaning their first attempt was most likely in high school) and those who made their first attempt more than 5 years ago (their first attempt most likely in middle/elementary school). Results showed that of the 42 who reported their first suicide attempt within the past 5 years, there was a significant difference between their mean depressive symptom score at the time of their reported attempt versus all prior years, t (df = 41) = 3.36, p < .01. However, this was not the case for those who attempted more than 5 years ago (t (df = 24) = .56, p = .58).

A comparison of retrospective reports for single versus multiple attempters

Analyses were also conducted examining the level of agreement between the retrospective reports for single attempters and those who made multiple attempts. For those who had made only one suicide attempt, the difference between their depressive symptom scores during the time of their attempt versus all prior years was significant (t (df = 32) = 2.80, p < .01). This finding was not found among multiple attempters (t (df = 33) = 1.48, p = ns). See Table 1 for results. Interestingly, there were no differences in depressive symptom scores between single and multiple attempters during the year of their first suicide attempt, t (df = 66) = .78, p = ns), nor in multiple attempters’ scores for all the prior years, t (df = 70) = 1.54, p = ns. This finding suggests that multiple attempters were more likely to be experiencing elevated levels of ongoing or chronic depressive symptoms, while single attempters appeared to have a significantly higher level of depressive symptoms in the year in which they reported their suicide attempt, compared to their own levels in the years prior.

Agreement of retrospective reports for attempters versus nonattempters

In examining the results for attempters versus nonattempters, 67 attempters reported their first attempt was during elementary/middle school or high school and had a depressive symptom score during the year of their reported first attempt. The mean depressive symptom scores for the attempters during the year of their first attempt was significantly higher (2.50, SD = .67) compared to the mean depressive symptom score for nonattempters across all time points (1.95, SD = .39) (t (df = 870) = 10.26, p < .001).

Reliability

In trying to establish the consistency of young adults reporting their past suicide attempts, we examined the amount of agreement in the retrospective reports of suicide attempts across 2 different time points, fall of 2005 and spring of 2006. The analyses showed that 95.4% (809/850) of the adolescents were consistent in their report of ever having a suicide attempt across the 2 time points (kappa = .70). In fall of 2005, among those reporting having made a suicide attempt, 71% reported similarly in the spring of 2006.

Discussion

This study examined the validity of using a retrospective measure of past suicide attempt/s completed by older adolescents using a longitudinal dataset in which participants annually completed self-reports of depressive symptoms beginning when they were 8 to 9 years old. This study examined 4 aspects of the validity and reliability in assessing the use of a retrospective measure of past suicide attempts. The implications of the findings are discussed below.

The prevalence of suicide attempts in this sample was 8.5%, which is consistent with what has been reported by the CDC. [1]. The number of multiple attempters and single attempters were similar, 39 and 36, respectively. Multiple attempters were more likely to have made their first suicide attempt during elementary/middle school compared to single attempters. These findings are consistent with the prospective literature on chronic suicide attempters compared to youth who have made one suicide attempt [31]. The youngest age of reported suicide attempt in our sample was 9 years, similar to what Bolger et al. [32] reported in using retrospective data among 364 college students. Kessler and colleagues [18] reported that in using retrospective data, the age of the first suicide attempt in their sample was around 9 years, with a sharp increase around age 12 (sixth grade), the same pattern that we found in our dataset. These findings have implications for the timing of suicide prevention programs.

This study showed that adolescents reported higher depressive symptom scores during the year that they made their first suicide attempt, providing support for the validity of the retrospective reports of suicide attempts. Depressive symptom scores were significantly higher during the year in which the first attempt occurred, compared to: 1) depressive symptom scores for the same person for all prior years, and 2) depressive symptom scores of nonattempting peers across all years. These findings suggest that young adults may be reliable reporters of their first suicide attempt. Assessments of young adults may provide important information regarding the timing of past suicide attempts, and retrospective reports can be used to examine the etiology of suicide-related thoughts and behaviors.

However, there are several caveats to these findings. Although the findings suggest that retrospective reports of suicide attempts within the last 5 years are more in agreement with depressive scores than reports made prior to 5 years ago (i.e., recency effect), this may not be the case. One explanation for this finding may be that those who attempted suicide more than 5 years ago were more likely to be multiple attempters and may represent a subgroup of high-risk youth who show more symptoms and behaviors consistent with ongoing elevated levels of depressive symptoms and/or chronic mental health problems in general. This group would be less likely to have marked changes in their depressive symptom scores across the different years prior to their first suicide attempt. Thus, it should not be concluded that youth who report making their first suicide attempt more than 5 years ago are not valid reporters. Further research into this area is warranted.

Important notes of caution

There are several important notes of caution. First, research in other fields, such as physical and sexual abuse, have reported only a moderate degree of retrospective and prospective agreement, ranging from .60 to .70 [7, 10], suggesting caution be used in generalizing these findings. Second, although depressive symptoms appear to be a good proxy for suicide attempts, the two different constructs and should be assessed separately [3]. Mazza and Reynolds [3] discussed the important differences between the two constructs and highlight their earlier findings [33] that showed 33% of adolescents who were at risk for suicidal ideation were not at risk for depression. Therefore, we caution readers that the results of this study do not support substituting a measure of depression or depressive symptoms for a measure of suicide-related thoughts and behaviors, or that taking a history of depression would be synonymous with a history of suicide-related thoughts and behaviors.

Limitations

Several limitations in this study should be mentioned. First, the validation and reliability study relied on youth prospective self-reports of depressive symptomatology and retrospective self-reports of suicide attempt history. Given both suicide-related behavior and depression are internalizing disorders, research has supported using self-report measures to obtain this data [3, 34, 35], however, multiple methods and multiple sources would have been ideal. Second, there were approximately twice as many females as males who attempted suicide, which was expected, and thus generalizations to males must be made cautiously. Third, we did not ask about suicide-related thoughts (suicidal ideation) retrospectively, and some participants may have been thinking about suicide for several years before making their first attempt. Fourth, what constituted a suicide attempt was not defined on the questionnaire, but consistent with other research, was left up to the individual to determine if they ever attempted suicide Thus, it is possible for one adolescent to interpret, for example, taking 5 aspirin as a suicide attempt, while another adolescent taking the same 5 aspirin was using them for a bad headache. Furthermore, there is no agreed upon standard definition of what constitutes a suicide attempt [6]. Fifth, as discussed above, the constructs of suicide-related behaviors and depressive symptoms are moderately correlated, but they also have some important differences, and thus depressive symptoms should not be a substitute for suicide-related behavior and vice-versa. Finally, the sample size of suicide attempters was 78, which limited the different types of statistical analyses that could be conducted, especially being able to examine gender differences. However, it should be noted that this was a universal sample of over 850 individuals that have been assessed annually and sometimes twice a year, with a greater than 86% retention rate during a 12-year time period. We are unaware of any universal sample that has this many participants with high retention and annual prospective measures throughout the participants’ formal schooling, making this sample appropriate to answer the questions posed.

Summary and implications

The results of this study provide empirical support for using a retrospective measure to obtain past history of suicide attempts among young adults. The implications of these results are particularly important for mental health professionals working with young adults. This study provides support that responses to questions about past suicide attempts appear valid. These findings are also important for researchers who are using longitudinal data and may wish to add suicide attempt history to samples of young adults. Given the empirical evidence for the reliability and validity of using a retrospective measure of suicide attempt/s provided in this study, asking questions about past suicide-related thoughts and behaviors, particularly suicide attempts, allows mental health professionals the ability to understand their patient’s history of suicide attempts. This validation study also found that nearly 40% of youth who first attempted suicide were in elementary/middle school at the time, implying that suicide prevention programs should perhaps also focus on elementary/middle school populations as well as high school populations.