Abstract
Objectives
This study was designed to examine escalation in repeat suicide attempts from adolescence through adulthood, as predicted by sensitization models (and reflected in increasing intent and lethality with repeat attempts, decreasing amount of time between attempts, and decreasing stress to trigger attempts)
Method
In a prospective study of 180 adolescents followed through adulthood after a psychiatric hospitalization, suicide attempts and antecedent life events were repeatedly assessed (M = 12.6 assessments, SD = 5.1) over an average of 13 years, 6 months (SD = 4 years, 5 months). Multivariate logistic, multiple linear, and negative binomial regression models were used to examine patterns over time.
Results
After age 17-18, the majority of suicide attempts were repeat attempts (i.e., made by individuals with prior suicidal behavior). Intent increased both with increasing age, and with number of prior attempts. Medical lethality increased as a function of age but not recurrent attempts. The time between successive suicide attempts decreased as a function of number of attempts. The amount of precipitating life stress was not related to attempts.
Conclusions
Adolescents and young adults show evidence of escalation of recurrent suicidal behavior, with increasing suicidal intent and decreasing time between successive attempts. However, evidence that sensitization processes account for this escalation was inconclusive. Effective prevention programs that reduce the likelihood of individuals attempting suicide for the first time (and entering this cycle of escalation), and relapse prevention interventions that interrupt the cycle of escalating suicidal behavior among individuals who already have made attempts are critically needed.
Keywords: suicide attempts, adolescence, sensitization, escalation, developmental trends
The rate of suicide attempts varies over the lifespan. For example, in community and population-based studies, the transition to adolescence has been found to be associated with a marked increase in the rates of suicide attempts (Boeninger, Masyn, Feldman, & Conger, 2010; Joffe, Offord, & Boyle, 1988; Kessler, Borges, & Walters, 1999; Lewinsohn, Rohde, Seeley, & Baldwin, 2001; Velez & Cohen, 1988; Wunderlich et al., 2001). Females generally attempt suicide at higher rates than males (Lewinsohn et al., 2001; Nock et al., 2013), and in one study, the increase in suicide attempts during adolescence was found primarily among females (Lewinsohn et al., 2001). In community samples, the rate of suicide attempts has been observed to decline during the transition to young adulthood (Kessler et al., 1999; Lewinsohn et al., 2001). Fewer studies have examined age trends in suicide attempts in clinical or high-risk samples, but results suggest similar patterns in the prevalence of suicidal behavior (Angle, O'Brien, & McIntire, 1983; Kovacs, Goldston, & Gatsonis, 1993).
Despite decreasing rates of suicide attempts from adolescence through early adulthood, a greater proportion of the attempts that do occur may be made by individuals who have attempted suicide on more than one occasion. For example, within 3 to 5 years of previous suicidal behavior, rates of repeat suicide attempts among adolescents and adults who have presented in treatment settings range from 25% to 31% (Christiansen & Borge, 2007; Goldston et al, 1999; Tejedor, Diaz, Castillon, & Pericay, 1999). Moreover, past suicide attempts have been found to be strongly associated with increased risk for future attempts (Goldston et al., 1999; Leon, Friedman, Sweeny, Brown, & Mann, 1990). Hence, adolescents who have attempted suicide may be at particularly high risk for repeat attempts even as they transition into adulthood.
The increased risk for suicidal behavior among individuals with prior suicide attempts may be in part due to sensitization processes. If such processes were operative, individuals would become more sensitive to and show increased reactivity to the triggers for behaviors or illness with repeated exposures to those triggers (Post, Rubinow, & Ballenger, 1986; Post, 2007). Sensitization processes are reflected in three patterns of response. First, when there is sensitization, individuals become more reactive with repeated exposures, and the magnitude of the behavioral or physiological response to the stress increases in intensity. In the example of suicide attempts, the severity of the suicide attempts (e.g., intent and/or lethality of attempts) would increase with repetition. Second, with increased sensitivity to triggers or precipitants, there are more rapid recurrences of the behavioral response or episode of disorder. For suicide attempts, this would be reflected in a decreasing amount of time between repeated suicide attempts. Third, with repeated exposure to a provocative stimulus or stress, it takes progressively less severe stress to trigger or provoke the reaction. In the case of a suicidal person, the amount of life stress that could trigger a suicide attempt would decrease as the individual made an increasing number of attempts.
Sensitization processes have been posited to be associated with different psychiatric disorders, including affective disorders (Bender & Alloy, 2011; Monroe & Harkness, 2005; Post et al., 1986; Post, 1992). For example, earlier episodes of affective disorder in adults are often less severe than later episodes (Kessing, 2008; Lewinsohn, Zeiss, & Duncan, 1989; Maj, Veltro, Pirozzi, Lobrace, & Magliono, 1992). Several (but not all) studies have found that among adults, there is a decreasing amount of time between affective disorder episodes as the number of episodes increases (Kessing, 1998). Furthermore, in adults and older adolescents, recurrent episodes of affective illness are sometimes precipitated by less severe stresses than initial or earlier episodes (Bender & Alloy, 2011; Monroe & Harkness, 2005; Post, 1992; Stroud, Davila, Hammen, & Vrshek-Schallhorn, 2011).
Despite these findings with affective disorders, research findings have been inconsistent regarding whether recurrent suicide attempts conform to a pattern that would be consistent with a sensitization model. Intent and medical lethality, for example, are often considered to be indices of severity of suicide attempts. To date, there have been mixed findings from studies of adults regarding associations between these indices and patterns of suicide attempts. Specifically, some studies have found that repeat suicide attempts are associated with greater intent to die than first-time suicide attempts (Kaslow et al., 2006; Reynolds & Eaton, 1986), whereas other studies have not found this pattern (Forman, Berk, Henriques, Brown, & Beck, 2004; Michaelis et al., 2003; Ojehagen, Danielsson, & Traskman-Bendz, 1992). Similarly, there have been mixed findings regarding a possible association between higher medical lethality and increasing number of suicide attempts (Forman et al., 2004; Kaslow et al., 2006; Michaelis et al., 2003; Pettit, Joiner, & Rudd, 2004; Reynolds & Eaton, 1986). No studies to our knowledge have examined the possibility that there may be decreasing amounts of time between successive suicide attempts. Last, the studies of life stress among individuals with differing numbers of prior suicide attempts have yielded mixed findings. For example, the amount of life stress preceding a suicide attempt has variously been found to not differ between individuals making their first attempts and individuals making repeat attempts (Crane et al., 2007; Joiner & Rudd, 2000; Kaslow, Jacobs, Young, & Cook, 2006; Pompili et al., 2011), to be positively related to the number of past attempts (Pettit et al., 2004), and to be related to severity of suicidal episode among individuals making first but not repeat attempts (Crane et al., 2007; Joiner & Rudd, 2000). Putting these mixed findings in context, it is worth noting that with few exceptions (e.g., Joiner & Rudd, 2000; Ojehagen et al., 1992), the majority of studies pertinent to sensitization models of suicide attempts have been cross-sectional in nature and focused on individuals at treatment entry. Cross-sectional comparisons of suicide attempts that precede initiation of treatment may be biased because these attempts may not be typical of all attempts. Prospective studies of recurrent suicide attempts within the same individuals over significant periods of time may be less biased, and more likely to reflect escalation of suicidal behavior consistent with sensitization models.
Sensitization processes are prominently described in theoretical conceptualizations of suicidal behavior. For example, Joiner (2005), in describing his interpersonal theory of suicide, referred to the process of cognitive sensitization. This occurs when “[an individual] undergoes a provocative experience, and subsequently, images and thoughts about that experience become more accessible and easily triggered . . . As suicidal experience accumulates, suicide-related cognitions and behaviors may become more accessible and active. The more accessible and active these thoughts and behaviors become, the more easily they are triggered (even in the absence of negative events), and the more severe are the subsequent suicidal episodes” (pp. 82-83). To this point, Beck (1996) theorized that the cognitive schemas underlying information processing become integrated with motivational, behavioral, and affective response systems. With repeated exposure to relevant experiences, these “modes” of responding, including a hopeless-suicidal mode, become more accessible and more easily activated. As a result, more severe reactions can result from less serious precipitants. Sensitization conceptualizations have been highly influential in our current thinking about suicidal behavior and in the development of interventions for suicidal individuals (e.g., Brown et al., 2005). However, the suggestions that sensitization processes might account in part for patterns of recurrence of suicidal behavior have not previously been tested among individuals followed over long periods.
If sensitization processes contribute to the recurrence of suicidal behavior, a sensitization model would provide a framework for understanding the course and repetition of suicidal behavior (Post et al., 1986, 1992). It also would provide a framework for understanding the high-risk group of individuals who have made multiple suicide attempts, and whose suicidal behavior has become increasingly more severe over time (Post et al., 1986). A sensitization model additionally would have implications for relapse prevention approaches for working with suicidal individuals, and for the theoretical conceptualizations that provide the basis for these interventions (Segal et al., 1996).
In 1991, we began conducting a naturalistic, prospective study of the risk for suicidal behaviors among adolescents who were psychiatrically hospitalized and then followed through young adulthood. With repeated assessments, we examined patterns in suicide attempts in adolescence and through adulthood following hospitalization, and also retrospectively assessed suicide attempts prior to hospitalization. This continuous record of suicidal behavior allowed us a rare opportunity to examine the degree to which a sensitization model might account for patterns in recurrent attempts across two developmental periods (adolescence and young adulthood). We hypothesized that (a) the severity of suicidal behavior, as reflected in intent to die and in the medical lethality of suicide attempts, would increase as the number of suicide attempts by an individual increases; (b) there would be decreasing amounts of time between successive pairs of suicide attempts as the number of suicide attempts made by an individual increases, and (c) the degree of association between severe life stresses and suicide attempts would decrease as the number of suicide attempts by an individual increases.
Method
Participants and Overview of Procedures
The 180 participants in this study were followed prospectively from adolescence, when they were psychiatrically hospitalized, through young adulthood. To be eligible for the study, youths needed to be: (a) 12 to 19 years old at index hospitalization, (b) hospitalized for 10 or more days, (c) able to cooperate with and complete the assessments in the hospital, and (d) a resident of North Carolina or Virginia at time of recruitment. Adolescents were excluded from the study if they (a) had a serious physical disease, (b) had intellectual disability, or (c) if their sibling was already enrolled in the study. At the time the study was initiated, the average length of stay in hospitals was 23.6 days (National Association of Psychiatry Health Systems, 2002). Hence, the stipulation of hospital stays of 10 or more days was made because patients with shorter hospital stays were often considered by clinical staff to have less severe problems or to be inappropriate for hospitalization. For example, adolescents with shorter lengths of stays had lower scores on the Beck Depression Inventory than individuals with longer stays (Goldston et al., 1999).
Patients on the inpatient unit participated in a comprehensive intake assessment as part of their psychiatric evaluations, including psychiatric diagnostic interviews and interviews about prior suicidal behavior. To recruit the longitudinal sample, we attempted to contact individuals (and their parents / guardians) who met inclusion and exclusion criteria approximately six to eight months following discharge from the hospital. Adolescents and their parents or guardians were contacted in the order of their discharge from the hospital. The total eligible sample consisted of 225 adolescents and their parents or guardians. One adolescent died of cardiac problems before we were able to contact him. We contacted 96% of the remaining sample and of these, 84% (n = 180) agreed to participate. The final sample consisted of 91 girls and 89 boys; 80% were European American, 16.7% were African American, and the other participants were Hispanic American, Native American, or Asian American. The average age of participants was 14 years, 10 months (SD = 1 year, 7 months; range = 12 years, 0 months to 18 years, 5 months) at their index hospitalization. Sixteen percent of youths were in the custody of the Department of Social Services at study entry. For the remaining families, the socioeconomic status as classified by the Hollingshead (1957) index from highest to lowest was as follows: I = 3.3%, II = 12.6%, III = 21.9%, IV = 29.8%, and V = 32.4%. At the time of their index hospitalization, 41.7% (n = 75) of the youths had histories of suicide attempts and another 33.3% (n = 60) reported current suicide ideation (Goldston et al., 1999). Psychiatric disorders at the index hospitalization and over the course of the longitudinal study, and the relationship of these psychiatric disorders to risk for suicide attempts have previously been described (Goldston et al., 1999, 2009).
The design of the follow-up study called for the participants to have their first follow-up assessment six to eight months following hospitalization. After their initial assessment in the study, this schedule was tapered so that assessments were subsequently scheduled every 10 to 12 months, and then annually. The longitudinal methods for this study were modeled after successful longitudinal studies by Kovacs and colleagues, in which the time between follow-up assessments after the initial assessments was also tapered (Kovacs, Feinberg, Crouse-Novak, Paulauskas, & Finkelstein, 1984; Kovacs, Goldston, Obrosky, & Drash, 1997). The more frequent assessments at the beginning of the study allowed us to more closely track the course of psychiatric problems after the hospitalization. The amount of time between assessments was tapered as a practical consideration to reduce burden on participants and to reduce study costs.
The median amount of time preceding the first three follow-up assessments ranged from 8.2 to 10.1 months, whereas the median time preceding assessments 8, 9, and 10 ranged from 10.9 to 11.4 months. The number of assessments and the amount of time between assessments varied both within and across participants due to scheduling conflicts, subject requests, staff shortages, funding lapses, and difficulties locating or contacting participants. These assessments occurred primarily in the homes of participants, but also at a university or medical center, in jails and prisons, or in other settings convenient to participants. A variety of methods were used to maintain contact with the sample including phone calls and correspondence, maintenance of information regarding ancillary contacts, use of publicly available databases to help locate participants, and scheduling of assessments in participants’ homes and communities.
As of June 30, 2009, participants had been followed for a maximum of 17.5 years (M = 13 years, 6 months; SD = 4 years, 5 months), and participated in a total of 2,270 assessments, including the baseline hospital assessments (M = 12.6 assessments, SD = 5.1, range = 2 to 26). The mean age of participants at the last assessment was 28 years, 5 months (SD = 4 years, 10 months; range = 13 years, 0 months to 34 years, 7 month). By the cutoff date for this paper, 20 individuals had dropped out of the study, six participants had been administratively withdrawn from the study because of lost contact, and eight participants had died (none due to suicide). Six of the individuals who were no longer active in the study made post-hospitalization attempts.
The subsamples of participants used in analyses of developmental trends and to test the different hypotheses are described in Table 1. For developmental trends, we focused on the 109 participants who attempted suicide at least once in their lives, either prior to hospitalization or during the follow-up study. Of note, 34 of the 105 (32.3%) participants who had not attempted suicide by the time of their index hospitalization subsequently attempted suicide (total attempts = 65, M = 1.9, SD = 1.4, range = 1 to 6) over the follow-up. To test the hypothesis regarding intent and lethality as a function of number of suicide attempts, we focused on the 41 participants who made more than one suicide attempt at any point over the follow-up or during the two weeks prior to hospitalization. The decision to not examine data regarding intent and lethality of suicide attempts before the two weeks that preceded the index hospitalization was made in an effort to reduce potential bias in retrospective reports of clinical characteristics. For the hypothesis regarding the amount of time between suicide attempts, we focused on the 63 participants who had a lifetime history of repeat attempts. Last, to test the hypothesis regarding the association between life events and suicide attempts, we focused on the 36 individuals who made more than one suicide attempt after their discharge from the hospital. This strategy was used because life events were assessed only post-hospitalization.
Table 1.
Purpose of Sample | Participants | Length of Follow-Up | Assessments | Number of Attempts |
---|---|---|---|---|
Developmental Trends | n = 109 (45 males, 64 females) with at least one attempt | M = 14 years, 0 months (SD = 3 years, 10 months); range: 0 year, 6 months - 17 years, 6 months) | 1511 (M = 13.2, SD = 4.9; range: 2 - 26) | 286 (M = 2.6, SD = 2.5; range: 1 - 15) |
Intent and Lethality | n = 41 (17 males, 24 female) with at least two attempts during the time immediately prior to hospitalization or over the follow-up | M = 14 years, 10 months (SD = 2 years, 10 months; range: 1 year, 9 months - 17 years, 6 months) | 713 (M = 13.0, SD = 4.6; range: 3 - 26) | 147 (M = 3.6, SD = 2.7; range: 1-13) |
Time Between Successive Attempts | n = 63 (23 males, 40 females) who made at least two attempts at any time in their lives | M = 14 years, 1 month (SD = 3 years, 10 months; range: 0 years, 6 months - 17 years, 6 months) | 986 (M = 13.8, SD = 4.9; range: 2 - 26) | 240 (M = 3.8, SD = 2.7; range: 2 - 15) |
Life Events and Suicide Attempts | n = 36 (16 males, 20 females) who made at least two attempts over the follow-up | M = 15 years, 2 months (SD = 2 years, 0 month; range: 8 years, 0 month - 17 years, 6 months) | 686 (M = 14.6, SD = 4.5; range: 4 - 26) | 129 (M = 3.6, SD = 2.7; range: 2 - 12) |
Research interviewers were master's and doctoral level mental health professionals. The interviewers were extensively trained (e.g., with role plays, calibration of symptom ratings, observed interviews) and supervised by the principal investigators for the study (D.G., S.D.).
The institutional review boards of the participating institutions provided approval for this ongoing study, and for use of clinical data from the baseline hospitalization for research purposes. Participants provided assent and their parents or legal guardians provided consent at the time of the hospitalization. Participants who turned 18 while participating in the study provided consent at the time of their 18th birthday. Participants were re-consented an additional time at the beginning of the last funding period for the grant.
Instruments
Assessment of suicide attempts
The Interview Schedule for Children and Adolescents (ISCA; Kovacs, Pollock, & Krol, 1997; Sherrill & Kovacs, 2000) and the Follow-Up Interview Schedule for Adults (FISA; Kovacs, Pollock, & Krol, 1995; Sherrill & Kovacs, 2000) are semi-structured clinical interviews developed for longitudinal studies used to assess symptoms of psychiatric disorders. Psychiatric diagnoses obtained with these instruments have been shown to be reliable and to have predictive validity as summarized by Sherrill and Kovacs (2000). In the current investigation, these instruments were used to assess suicide attempts. To aid in this assessment, the ISCA and FISA have standardized questions about the presence/absence of thoughts of death, suicide ideation, and suicide attempts, plans and methods, circumstances and suicidal motivations, and psychological intent (e.g., “Have you ever thought about killing yourself?” “Have you ever done anything to try to kill yourself?” “What did you do?” “What did you think would happen when you ____?”). In these instruments, suicide attempts were defined operationally as potentially self-injurious behaviors associated with some (i.e., non-zero) intent to end one's life; this definition is consistent with current approaches to classification of suicide-related terms (Crosby, Ortega, & Melanson, 2011; Posner, Oquendo, Gould, Stanley, & Davies, 2007; Silverman, Berman, Sanddal, O'Carroll, & Joiner, 2007). Self-injurious behaviors not associated with at least some intent to kill oneself (e.g., cutting to relieve tension) were not considered as suicide attempts. If reports of self-harm were vague or indefinite (e.g., “I honestly can’t remember what was going through my head,” “she took a bunch of pills but I have no idea if she was trying to kill herself or just get high”), the behavior conservatively was not counted as a suicide attempt.
At the index hospitalization and over the follow-up period, all available information was used to make determinations of the dates of attempts. Sources of information included the semi-structured interviews; treatment, legal, and school records; and parent interviews. At the index hospitalization, we obtained information about all previous suicide attempts. In subsequent assessments, we assessed all suicide attempts since last contact. The information obtained at the index hospitalization and follow-up assessments was combined to yield continuous (lifetime) records of participants’ suicidal behavior. The ISCA was used in interviews with adolescents at hospitalization, and in interviews with parents or guardians and adolescents over the follow-up until participants reached the age of 18 or began living independently. After that point, the participants were administered the FISA, but parents and guardians were not interviewed.
When participants could not provide precise dates for suicide attempts, but could describe a likely window of time during which the attempt occurred, the dates were estimated as the midpoint of the defined period of time (Kovacs, Feinberg, Crouse-Novak, Paulauskas, & Finkelstein, 1984). Prior to the age of 18, suicide attempts (meeting our operational definition of this behavior) were considered to be present when reported by either adolescent or parent. The strategy of counting suicide attempts as present when reported by either adult informants or adolescent participants was used in light of the findings from multiple studies that parents are often not aware of adolescents’ suicide attempts (e.g., Breton, Tousignant, Bergeron, & Berthiaume, 2002; Foley, Goldston, Costello, & Angold, 2006; Walker, Moreau, & Weissman, 1990).
We have conducted two inter-rater reliability trials of our classifications of suicidal thoughts and behavior in this sample using all information, including interviews with the ISCA and FISA, and treatment records. In the first trial of 40 cases, there was 95.0% agreement in the classification of suicide ideation and suicide attempts (Goldston et al., 2001). In a second trial, 500 cases were classified as to whether there was presence of (a) no suicide ideation, (b) suicide ideation without means envisioned, (c) suicide ideation with means envisioned, (d) a single suicide attempt, or (e) multiple attempts since the last assessment. In this trial, there was excellent agreement between previously determined consensus ratings and the ratings of an independent coder (96.4% agreement; κ = 0.92). In all cases, discrepancies in ratings were discussed and resolved by consensus.
Assessment of suicide intent
The subjective intent of suicide attempts during the follow-up period was assessed on the basis of all available information, using the 4-point Subjective Intent Rating Scale developed by our research group (SIRS; Sapyta et al., 2012). This scale was developed to assess suicide intent independently of related constructs such as impulsivity or factors potentially related to medical lethality such as isolation at the time of the attempt. The construct validity of the SIRS has been demonstrated by the higher correlation with the Subjective index than with the Objective index of the Beck Suicide Intent Scale (Beck, Schuyler, & Herman, 1974). Based on all available information including responses to the ISCA and FISA (Sherrill & Kovacs, 2000), intent was rated from “Mild” (respondent acknowledges a wish to die, but mainly wants to live) to “Very High” (respondent expresses very little ambivalence about wanting to die). There was not a point on this scale corresponding to no intent, since by definition, suicide attempts were associated with at least some intent to die. Two independent coders rated intent, and discrepancies were resolved by consensus. SIRS ratings in this study have been found to have high inter-rater reliability (ICC = 0.99; p < .05), and the maximum intent of past suicide attempts has been found to be predictive of future attempts (Sapyta et al., 2012). The average unweighted intent score for suicide attempts among participants that made more than one attempt was 2.51 (SD = 0.93).
Assessment of medical lethality
Medical lethality of all suicide attempts during the follow-up was rated on the basis of all available information using the Lethality of Suicide Attempt Rating Scale (Berman, Shepherd, Silverman, 2003; Smith, Conroy, & Eller, 1984). Using this scale, the suicide attempts were rated in severity of potential medical consequences from 0 (“death is an impossibility”) to 10 (“death is almost a certainty”) by two independent raters, with discrepancies resolved by consensus. This scale has been shown to have high inter-rater reliability and concurrent validity among adolescents as well as adults (Lewinsohn, Rohde, & Seeley, 1996; Nasser & Overholser, 1999; Sapyta et al., 2012) and the maximum lethality of past suicide attempts was found to be predictive of future suicidal behavior (Sapyta et al., 2012). In this sample, there was high inter-rater reliability in ratings from this scale (ICC = 0.95, p < .05; Sapyta et al., 2012). Similar to other clinical and epidemiologic samples of young people (e.g., Diamond et al., 2005; Lewinsohn, Rohde, & Seeley, 1994), most of the suicide attempts were in the mild to moderate range of lethality (Sapyta et al., 2012). The average medical lethality score of suicide attempts (unweighted for number of observations per participant) among individuals who made repeat attempts was 2.88 (SD = 1.92).
Assessment of life events
Life events prior to suicide attempts were assessed using all available information. Sources included (but were not limited to) a modified version of the Life Events Checklist (Johnson & McCutcheon, 1980), the symptom timelines that we developed in conjunction with the semi-structured clinical interviews (ISCA and FISA; Sherrill & Kovacs, 2000), the queries regarding legal involvement of the Follow-Up Information Sheet, and precipitant section of the Suicide Circumstances Schedule (Brent et al., 1988). Negative life events in the three months prior to each suicide attempt were coded independently by at least two coders and discrepancies were resolved by consensus between the reviewers. If a participant explicitly described a life event as a precipitant, but was vague about the timing, we counted the life event as though it occurred within the three-month period. In an inter-rater reliability trial, agreement between two independent coders regarding the presence/absence of a subset of major life events (loss and legal events) was 92.5% (κ = 0.85). For the events agreed upon by the two coders, there was 97.9% agreement as to the date (within a two-week period of time). The total severity of life stress preceding suicide attempts was assessed in two different ways. First, we examined the unweighted total number of negative life events in the three months prior to suicide attempts. Second, the magnitude of social adjustment required by different life events (“life change units”) was estimated using the standardized table of life change unit values provided by Miller and Rahe (1997). The table of life change units provided by Miller and Rahe (1997) was derived in a scaling study, and represented a revision of the life change values originally described by Holmes and Rahe (1967). In previous studies, both the unweighted number of life events and life change units have been linked to poorer health outcomes (e.g., De Benedittis, Lorenzetti, & Pieri, 2013; Lantz, House, Mero, & Williams, 2005). The average number of life events and life change units in the three months prior to suicide attempts were 3.78 (SD = 2.61) and 184.55 (SD = 125.81), respectively.
Statistical Method
General approach and covariates
Given the number of observations over time and the multiple suicide attempts, we used longitudinal statistical models that can accommodate different numbers of observations per participant, varying amounts of time between observations, and missing data. The data were not analyzed as a panel study with specific “waves” of data and missing values when a scheduled assessment was delayed or missed. Rather, the data set was organized so that assessments for participants were consecutively numbered, regardless of when they occurred.
There were some missing data that the analyses could not accommodate. Specifically, there were five suicide attempts, all occurring prior to the index hospitalization, for which precise dates could not be estimated. These suicide attempts were not included in analyses of developmental trends, and of the intervals between consecutive suicide attempts. There were no missing life events or lethality data. There were eleven missing values (for 7.1% of suicide attempts at hospitalization or over the follow-up) regarding intent; in these cases, the participants reported enough information to indicate that there was at least some intent to die, but gave vague or inconsistent reports about the degree of intent or ambivalence. These data were viewed conservatively as missing at random (MAR) rather than being imputed, given that we did not know the mechanisms associated with the missing data (Little & Rubin, 1987; Rubin, 1987, 1996). The statistical models implemented in SAS were able to use full information available from the data, under the assumption of MAR.
Because gender (e.g., Lewinsohn et al., 2001) and race/ethnicity (see Goldston et al., 2008) have been found in previous studies to be related to suicide attempts, they were included as covariates in all analyses to reduce variance attributable to potentially confounding or background variables. Because age also has been noted to be related to the clinical characteristics of suicide attempts (Conwell et al., 1998; Hamdi, Amin, & Mattar, 1991; O'Brien et al., 1987), age was included as a time-varying covariate in models of intent and lethality to disentangle age effects from effects associated with increasing number of suicide attempts.
As preliminary analyses, we used linear regression to evaluate whether demographic variables (age at hospitalization, gender, race/ethnicity), or number of suicide attempts at baseline hospitalization were related to either number of assessments completed, or the length of time in the study (log transformed to improve normality of distribution). We also used linear regression to examine whether variability in the timing of assessments (i.e., time between the assessment when a suicide attempt was reported, and the prior assessment) was related to four of the outcomes of the study (intent of suicide attempts, lethality of suicide attempts, number of life events prior to suicide attempts, life change prior to suicide attempts). The timing of assessments was not examined in relation to the time between successive suicide attempts because many of the reported attempts occurred prior to initiation of the follow-up study.
Developmental model
A cubic polynomial logistic regression was used to examine suicide attempts as a function of age (z-transformed for numerical stability). This model was chosen over a linear or quadratic model because of the sharp rise in attempts in adolescence, followed by a tapering off in adulthood. This model was fitted in PROC GLIMMIX with variance-components covariance structure, which was assumed to be different for males and females. This model is equivalent to a generalized estimating equations (GEE) based approach except that GEE models do not have an option for a heterogeneous variance-covariance structure. As a conservative approach to modeling, we used sandwich (robust) variance estimates in the analyses, which provided additional protection against heterogeneity and departures from assumptions. Interactions with gender were explored, but eventually were not included in models because they were not reliably related to suicide attempts, and because of multicollinarity.
For descriptive purposes, the actual proportions of individuals with attempts as a function of age (in two year intervals) and gender were graphed in Figure 1. The curves were smoothed using the lowess function (Cleveland, 1981) in R (R Development Core Team, 2010). This Figure was not generated from the polynomial logistic regression model, but was based on aggregate data for clarity of presentation.
Sensitization models
To examine whether intent and medical lethality of attempts increased as a function of number of prior suicide attempts, we used GEE implemented in SAS 9.2 (SAS Institute, Inc., 2008). This approach allowed us to account for the within-subject correlations from multiple observations. For both intent and lethality analyses, we used multivariate ordinal logistic models (with a cumulative logit link for the Likert scales).
To address the question of whether the amount of time between suicide attempts decreases as a function of number of suicide attempts, linear regression models using GEE were utilized. These models adjusted for within-subject correlations. The amount of time between successive suicide attempts was transformed to a logarithmic scale because of the non-normal distribution of these times.
Generalized estimating equations were used to examine the relationship between the number of suicide attempts and life change in the three months prior to the most recent suicide attempt. The primary predictor in these analyses was the number of suicide attempts. In separate analyses, the total number of life events and life change scores in the three months prior to each suicide attempt were dependent variables. For the analyses regarding sum of life events as a dependent variable, we used negative binomial regression (with log link). For the analyses regarding life change scores as a dependent variable, we used a multiple linear regression.
In all analyses examining sensitization, robust (sandwich) variance-covariance estimates were used to adjust for heterogeneity and departures from assumptions. Results with model-based estimates of standard errors were also examined and the pattern of results was nearly identical to results with robust estimates. Results with model-based estimates are not presented, but are available upon request.
Results
Preliminary Analyses
Demographic variables and history of suicide attempts at baseline hospitalization were not related to the amount of time that participants were followed in this study (p values > 0.05). History of suicide attempts, age at hospitalization, and race/ethnicity were not related to the number of follow-up assessments (p values > 0.05). However, females participated in more assessments than males (b = 1.546, se = 0.763, t = 2.03, p = 0.044). The amount of time between a follow-up assessment when a suicide attempt was reported and the previous follow-up assessment was not related to intent or lethality of suicide attempts, number of life events prior to attempts, or life change prior to attempts (p values > 0.05).
Developmental Trends in Suicide Attempts from Adolescence to Young Adulthood
As seen in Table 2, results from the cubic polynomial regression indicated that females made more suicide attempts than males and that the proportion of the sample with suicide attempts varied as a quadratic function of age. Specifically, as seen in the smoothed curve in Figure 1 using aggregate data, the rates of suicide attempts increased from early adolescence through mid-adolescence, peaked in mid-adolescence, and decreased again until the early 20's, whereupon the rates stabilized.
Table 2.
Variable | b | se | 95% CI | df | t | p |
---|---|---|---|---|---|---|
Age | −2.014 | 0.574 | −3.162, −0.866 | 166 | −3.51 | <0.001 |
Age2 | −1.522 | 0.734 | −2.990, −0.054 | 166 | −2.07 | <0.040 |
Age3 | 0.572 | 0.335 | −0.098, 1.242 | 166 | 1.71 | 0.089 |
Gender | −2.940 | 0.913 | −4.766, −1.114 | 177 | −3.22 | 0.002 |
Gender * Age | 0.139 | 0.722 | −1.305, 1.583 | 166 | 0.19 | 0.847 |
Gender * Age2 | 1.768 | 0.951 | −0.134, 3.670 | 166 | 1.86 | 0.065 |
Gender * Age3 | −0.543 | 0.426 | −1.395, 0.309 | 166 | −1.28 | 0.204 |
Race/Ethnicity | 0.117 | 0.641 | −1.165, 1.399 | 177 | 0.18 | 0.855 |
The proportion of suicide attempts at each age that was made by individuals with prior attempts increased from ages 9-10 through adulthood. For example, between ages 9-10 and 15-16, the proportion of suicide attempts in any two-year period that were repeated attempts ranged between 0.25 and 0.57. From ages 17-18 through 31-32, the proportion of attempts that were repeated attempts ranged from 0.67 to 0.90.
Suicide Intent and Medical Lethality as a Function of Number of Attempts and Age
Suicide intent was positively related both to number of prior suicide attempts and to increasing age in separate models. In contrast, medical lethality of suicide attempts increased with participants’ age (see Table 3), but was not related to number of suicide attempts. Gender and race/ethnicity were not related to intent or lethality.
Table 3.
Suicide Intent – Model with Past Attempts but not Age (ordinal logistic regression) | ||||||
---|---|---|---|---|---|---|
Variable | b | se | OR | 95% CI | z | p |
Attempt Number | 0.110 | 0.042 | 1.12 | 0.028, 0.191 | 2.64 | 0.008 |
Gender | −0.328 | 0.365 | 0.72 | −1.044, 0.388 | −0.90 | 0.370 |
Race/Ethnicity | −0.131 | 0.343 | 0.88 | −0.802, 0.541 | −0.38 | 0.703 |
Suicide Intent – Model with Age but not Past Attempts (ordinal logistic regression) | ||||||
Age | 0.066 | 0.027 | 1.07 | 0.013, 0.120 | 2.42 | 0.015 |
Gender | −0.257 | 0.375 | 0.78 | −0.991, 0.478 | −0.68 | 0.494 |
Race/Ethnicity | −0.101 | 0.382 | 0.91 | −0.850, 0.648 | −0.26 | 0.791 |
Suicide Intent – Model with Both Past Attempts and Age (ordinal logistic regression) | ||||||
Attempt Number | 0.086 | 0.047 | 1.09 | −0.007, 0.178 | 1.82 | 0.069 |
Age | 0.052 | 0.032 | 1.05 | −0.010, 0.114 | 1.65 | 0.100 |
Gender | −0.265 | 0.362 | 0.77 | −0.973, 0.444 | −0.73 | 0.464 |
Race/Ethnicity | −0.145 | 0.353 | 0.87 | −0.838, 0.547 | −0.41 | 0.681 |
Lethality – Model with Past Attempts but not Age (ordinal logistic regression) | ||||||
Attempt Number | 0.092 | 0.054 | 1.09 | −0.015, 0.198 | 1.69 | 0.091 |
Gender | 0.170 | 0.351 | 1.19 | −0.857, 0.518 | −0.48 | 0.629 |
Race/Ethnicity | 0.001 | 0.535 | 0.99 | −1.048, 1.049 | 0.00 | 0.999 |
Lethality – Model with Age but not Past Attempts (ordinal logistic regression) | ||||||
Age | 0.124 | 0.038 | 1.13 | 0.050, 0.197 | 3.30 | 0.001 |
Gender | −0.034 | 0.349 | 0.71 | −0.718, 0.650 | −0.10 | 0.923 |
Race/Ethnicity | −0.124 | 0.422 | 0.88 | −0.951, 0.703 | −0.29 | 0.769 |
Lethality – Model with both Past Attempts and Age (ordinal logistic regression) | ||||||
Attempt Number | 0.043 | 0.050 | 1.05 | −0.055, 0.142 | 0.86 | 0.388 |
Age | 0.117 | 0.039 | 1.12 | 0.041, 0.193 | 3.02 | 0.003 |
Gender | 0.031 | 0.333 | 1.03 | −0.622, 0.683 | 0.09 | 0.927 |
Race/Ethnicity | 0.155 | 0.423 | 1.17 | −0.675, 0.984 | 0.36 | 0.715 |
Log Time Between Successive Attempts (multiple linear regression) | ||||||
Attempt Number | −0.326 | 0.015 | -- | −0.356, −0.297 | −21.60 | <0.001 |
Gender | −0.134 | 0.282 | -- | 0.687, 0.419 | −0.47 | 0.653 |
Race/Ethnicity | 0.026 | 0.424 | -- | −0.806, 0.857 | 0.06 | 0.952 |
Number of Life Events (negative binomial regression) | ||||||
Attempt Number | −0.026 | 0.024 | 0.974 | −0.074, 0.022 | −1.07 | 0.286 |
Gender | −0.159 | 0.145 | 0.853 | −0.443, 0.124 | −1.10 | 0.271 |
Race/Ethnicity | −0.005 | 0.139 | 0.995 | −0.278, 0.268 | −0.03 | 0.972 |
Sum of Life Change (multiple linear regression) | ||||||
Attempt Number | −1.856 | 4.124 | -- | −9.938, 6.227 | −0.45 | 0.653 |
Gender | −19.725 | 27.020 | -- | −72.684, 33.233 | −0.73 | 0.465 |
Race/Ethnicity | −8.407 | 26.109 | -- | −59.579, 42.764 | −0.32 | 0.747 |
Note: Suicide Intent = ratings from the Subjective Intent Rating Scale (Sapyta et al., 2012) for suicide attempts. Lethality = ratings of medical lethality of suicide attempts, using the Lethality of Suicide Attempt Rating Scale (Berman et al., 2003; Smith et al., 1984). Dates of suicide attempts were estimated using analogous methods to those used for dating onset and offset of symptoms on Interview Schedule for Children and Adolescents and the Follow-Up Interview Schedule for Adults (Sherrill & Kovacs, 2000). Number of life events = sum of life events listed in the tables by Miller and Rahe (1997) in the three months prior to the suicide attempt. Sum of life change = sum of life change units from life events listed in the tables by Miller and Rahe (1997) in the three months prior to the suicide attempt.
Inter-suicide Attempt Intervals as a Function of Number of Attempts
As reflected in Table 3, the log-transformed inter-suicide attempt intervals decreased in length as a function of the number of past suicide attempts. Converting to the original duration scale by exponentiation, the associated “hazard ratio” was 0.722 per unit increase in the number of past attempts. That is, the more suicide attempts an individual made, the shorter the period of time before the next repeat suicide attempt on average. Neither gender nor race/ethnicity was related to the amount of time between repeat suicide attempts.
Life Events and Suicide Attempts
As seen in Table 3, neither total number of life events nor the sum of life change scores from the three months preceding suicide attempts was related to the number of suicide attempts an individual had made.
Discussion
In this prospective, naturalistic study, we examined patterns in recurrent suicidal behavior among adolescents and young adults, and the degree to which these patterns were consistent with a sensitization model. In this high-risk sample, the rate of suicide attempts among both males and females increased through mid-adolescence, and then decreased during young adulthood, stabilizing by the mid-20's. These developmental patterns are similar to those noted in epidemiological research (Boeninger et al., 2010; Joffe et al., 1988; Kessler et al., 1999; Lewinsohn et al., 2001; Velez & Cohen, 1988; Wunderlich et al., 2001). However, in an extension of these previous studies, we found that by the transition to adulthood, the majority of suicide attempts were made by individuals who already had a history of attempts. Future studies are needed to establish whether a similar pattern would be observed in larger-scale epidemiologic samples.
Given the high rate of repeat suicidal behavior in this sample, it is critical to examine whether suicidal behavior escalates with recurrences, and whether sensitization processes might account for any escalation. As predicted from a sensitization model, the intent of suicide attempts did increase as individuals made a greater number of attempts. The intent of suicide attempts also increased as participants got older, and the effects of increasing age and the number of prior suicide attempts were confounded to a degree. Previous findings regarding the clinical characteristics of earlier versus subsequent suicide attempts (or first-time as contrasted with repeat attempts) have yielded contradictory findings (Forman et al., 2004; Kaslow et al., 2006; Michaelis et al., 2003; Ojehagen et al., 1992; Reynolds & Eaton, 1986). However, these studies generally have been cross-sectional and focused on suicide attempts that precipitated treatment entry, which may not be a “representative” period of time in the natural history of suicidal behavior. The finding that intent increases with number of attempts contradicts the common clinical myth that individuals who make multiple attempts “are not serious” about killing themselves. To the contrary, these individuals seem to become more determined and have less ambivalence about dying with successive attempts.
Another index of severity, the medical lethality of suicide attempts, increased as a function of age, but was not related to the cumulative number of attempts. This finding was not consistent with what would have been predicted by a sensitization model. On the other hand, the finding of increased lethality with increasing age dovetails with other findings that lethality of suicide attempts in some adult patient populations is positively correlated with age (Shearer et al., 1988), and that individuals at older ages are more likely than individuals at younger ages to die by suicide when they engage in suicidal acts (Friedman & Kohn, 2008). Sapyta et al. (2012) found that intent and lethality are not strongly correlated among adolescents and young adults, although both maximum intent and maximum lethality of past attempts were predictive of future suicidal behavior. This finding could be due in part to restricted access to more lethal methods at younger ages, the fact that adolescents feel constrained in choice of methods because they live with parents, the lack of knowledge about the medical consequences associated with different methods at younger ages (Brown, Henriques, Sosdjan, & Beck, 2004), or greater planning and premeditation among older individuals (Conwell et al., 1998).
The second prediction from a sensitization model was that there would be decreasing intervals of time between successive suicide attempts. Although the amount of time between suicide attempts was quite variable, overall, there was a decreasing length of time between suicide attempts as the number of suicide attempts increased. This possibility, to our knowledge, has not been evaluated previously. The prospective, repeated assessments design of the current study made it particularly well suited for examining the length of time between attempts. The pattern of decreasing amounts of time between successive attempts highlights the possibility of increasing vulnerability associated with repeated occurrences of suicidal behavior.
Last, the sensitization model is predicated on the notion that individuals become more reactive or sensitive to stress through repeated exposure. In this study, life stress measured in two different ways was unrelated to the number of prior suicide attempts. Although these results are consistent with several findings from cross-sectional studies with adults (Crane et al., 2007; Joiner & Rudd, 2000; Kaslow et al., 2006), it should be noted that the sample size for the life stress analyses (n = 36 with 129 suicide attempts) was smaller than for the other samples used for tests of sensitization hypotheses. Therefore, although there was very little indication of an effect, it could be the case that the sample size was not sufficiently large to detect patterns in reactivity to life stress across individuals. In addition, it is possible that approaches or measures for assessing the relationship between life stress and suicidal behavior to date have not been sufficiently sensitive for detecting patterns or reactivity to stress. It also is possible that the heterogeneity among individuals who attempt suicide is so great that any evidence of a sensitization process is obscured in-group analyses. For example, some suicidal individuals may become more reactive to life stress, whereas others, rather than being more reactive, simply experience an inordinate number of negative life stresses, some of which could even be related to their own mental health difficulties (Conway, Hammen, & Brennan, 2012). Another possibility is that some individuals become especially reactive to only certain types of stresses (e.g., losses or difficulties in relationships), and the sensitization process is not apparent across the whole spectrum of major life stresses. Joiner (2005), for instance, has emphasized life circumstances associated with thwarted belongingness and perceived burdensomeness, and Shneidman (1998) emphasized the importance of circumstances associated with unmet psychological needs in the etiology of suicide.
In sum, the results from this longitudinal study revealed a pattern of escalation of suicidal behavior, with increasing intent and decreasing amounts of time between successive attempts. The data were inconclusive as to whether a sensitization model might account for this escalation. It is possible that repeated exposures to stresses or situations that provoked earlier suicidal behavior change the way individuals think or respond affectively to future situations, and additional research is needed to examine this possibility. Nonetheless, the prospective findings of this study importantly underscore the observation that there is an escalation in suicidal behavior that occurs as individuals make a greater number of attempts. If not due to sensitization, there are multiple possibilities for this escalation of suicidal behavior. For example, increasing suicidal behavior may be reflective of increasing distress with persisting difficulties, or of increasing severity of psychopathology. In addition, Linehan (1993) suggested that individuals who have vulnerabilities with emotion regulation may respond to invalidating and inconsistent responses from others with escalating self-destructive behavior. It also is possible that there is other “scarring” that occurs with prior suicide attempts, which renders individuals more vulnerable for future episodes of suicidal behavior. Kessing, Mansen, Andersen, and Angst (2004) have demonstrated that scarring or episode sensitization is one mechanism that may account for recurrent episodes of affective disorder.
The findings from this study have multiple implications for mental health professionals. In working with suicidal clients, clinicians need to be aware that the intent associated with suicidal behavior may increase with repeated attempts, and that both intent and lethality of suicidal behavior may increase as individuals get older. Clinicians sometimes consider recurrent suicidal behavior with less urgency than they should, mistakenly assuming that individuals who make multiple attempts may not be serious about killing themselves. In fact, the opposite appears to be true. As individuals make repeated attempts, they are on average more intent on dying by suicide.
The finding, that for some individuals there is an escalation of suicidal behavior after an initial attempt, underscores the importance of developing effective prevention programs for at-risk populations before individuals have made their first attempt, and potentially entered into this pattern of escalation. For individuals who already have made attempts, there is a strong need for relapse prevention interventions that can interrupt the cycle of recurrent suicidal behavior before there is further escalation. The use of chain analysis and the focus on development of coping skills in dialectical behavior therapy (Linehan, 1993) and the complementary use of the relapse prevention task in cognitive behavior therapy for suicide prevention (Brown et al., 2005) are two promising approaches to relapse prevention. Nonetheless, suicidal individuals often terminate treatment prematurely or fail to initiate treatment following referrals (Dahlsgaard, Beck, & Brown, 1998; Rudd, Joiner, & Rajab, 1995). When individuals terminate treatment prematurely, they may not learn the skills or alternatives to suicidal behavior they need for forestalling future episodes and interrupting this cycle of escalation. Therefore, it is important that effective approaches (e.g., drawing from motivational enhancement strategies) be developed for facilitating treatment engagement and follow-through so that suicidal individuals maximally benefit from relapse prevention activities (Britton, Patrick, Wenzel, & Williams, 2011).
Several caveats regarding the findings from this study should be acknowledged. First, the sample was recruited from an inpatient adolescent psychiatric unit. The findings from the study may not be generalizable to other populations, including individuals who have not been hospitalized or older populations. Second, there was variability both within and between participants in the intent and lethality of suicide attempts, the number and type of stressful life events preceding suicide attempts, and the length of inter-suicide attempt intervals. As such, the effects of sensitization may not always be detectable or apparent on an individual basis. Third, although we had a large number of observations over a relatively long period, which contributed to the power of our analyses and provided opportunities to see unfolding patterns, the actual number of individuals on which some results are based is still smaller than would be ideal. In particular, the analyses regarding life stress focused on only 36 individuals who made 129 attempts over the follow-up. Hence, it will be important for these findings to be replicated. Fourth, no one in this study had died by suicide so the degree to which these processes are applicable to suicide deaths is not clear. Fifth, although severity of life stress preceding suicide attempts was measured in two different ways, using all available information, we did not use specific interviews for assessing life events, which would have provided more contextual information about the life stresses. Sixth, this study did not examine potential psychiatric factors including treatment history, increasing severity of psychiatric and substance use problems, or exposure to childhood adversity that could have shed light on the mechanisms associated with escalation or sensitization. These caveats notwithstanding, the pattern of results from this prospective study provided evidence of escalating suicidal behavior among individuals who make repeat suicide attempts in adolescence and young adulthood, even if the tests of a sensitization model per se were not fully supported. A better understanding of the processes underlying this escalation will be important to inform the design of more effective relapse prevention interventions and intervene in patterns that culminate in repeat suicidal behaviors.
Public Health Significance Statement.
Some individuals attempt suicide on multiple occasions during adolescence and young adulthood. As they make repeated attempts, the severity of their intention to die increases, and the amount of time between their suicide attempts decreases on average. These findings underscore the need for effective interventions to prevent and interrupt this cycle of escalation in suicidal behavior.
Acknowledgments
Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health (R01MH048762 and K24MH066252). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Contributor Information
David B. Goldston, Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine
Stephanie S. Daniel, Center for Youth, Family, and Community Partnerships, University of North Carolina at Greensboro
Alaattin Erkanli, Department of Biostatistics and Bioinformatics, Duke University.
Nicole Heilbron, Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine.
Otima Doyle, Jane Addams College of Social Work, University of Illinois, Chicago.
Bridget Weller, Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine.
Jeffrey Sapyta, Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine.
References
- Angle CA, O'Brian TP, McIntire MS. Adolescent self-poisoning: A nine-year follow-up. Journal of Developmental and Behavioral Pediatrics. 1983;4:83–87. doi: 10.1097/00004703-198306000-00001. doi: 10.1097/00004703-198306000-00001. [DOI] [PubMed] [Google Scholar]
- Beck AT. Beyond belief: A theory of modes, personality, and psychopathology. In: Salkovskis P, editor. Frontiers of cognitive therapy. Guilford; New York: 1996. pp. 1–25. [Google Scholar]
- Bender RT, Alloy LB. Life stress and kindling in bipolar disorder: Review of the evidence and integration with emerging psychosocial theories. Clinical Psychology Review. 2011;31:383–398. doi: 10.1016/j.cpr.2011.01.004. doi:10.1016/j.cpr.2011.01.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berman AL, Shepherd G, Silverman MM. The LSARS-II: Lethality of Suicide Attempt Rating Scale – updated. Suicide and Life-Threatening Behavior. 2003;33(3):261–276. doi: 10.1521/suli.33.3.261.23211. doi: 10.1521/suli.33.3.261.23211. [DOI] [PubMed] [Google Scholar]
- Boeninger DK, Masyn KE, Feldman BJ, Conger RD. Sex differences in developmental trends in suicide ideation, plans, and attempts among European American adolescents. Suicide & Life-Threatening Behavior. 2010;40(5):451–464. doi: 10.1521/suli.2010.40.5.451. doi:10.1521/suli.2010.40.5.451. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brent DA, Perper JA, Goldstein CE, Kolko DJ, Allan MJ, Allman CJ, Zelenak JP. Risk factors for adolescent suicide: A comparison of adolescent suicide victims with suicidal inpatients. Archives of General Psychiatry. 1988;45:581–588. doi: 10.1001/archpsyc.1988.01800300079011. doi:10.1001/archpsyc.1988.01800300079011. [DOI] [PubMed] [Google Scholar]
- Breton JJ, Tousignant M, Bergeron L, Berthiaume C. Informant-specific correlates of suicidal behavior in a community survey of 12- to 14-year-olds. Journal of the American Academy of Child Adolescent Psychiatry. 2002;41(6):723–730. doi: 10.1097/00004583-200206000-00012. doi:10.1097/00004583-200206000-00012. [DOI] [PubMed] [Google Scholar]
- Britton PC, Patrick H, Wenzel A, Williams GC. Integrating motivational interviewing and self-determination theory with cognitive behavioral therapy to prevent suicide. Cognitive and Behavioral Practice. 2011;18:16–27. doi: 10.1016/j.cbpra.2009.06.004. doi: 10.1016/j.cbpra.2009.06.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brown GK, Henriques GR, Sosdjan D, Beck AT. Suicide intent and accurate expectations of lethality: Predictors of medical lethality of suicide attempts. Journal of Consulting and Clinical Psychology. 2004;72:1170–1174. doi: 10.1037/0022-006X.72.6.1170. doi: 10.1037/0022-006X.72.6.1170. [DOI] [PubMed] [Google Scholar]
- Brown GK, Ten Have T, Henriques GR, Xie SX, Hollander JE, Beck AT. Cognitive therapy for the prevention of suicide attempts: A randomized controlled trial. JAMA. 2005;294(5):563–570. doi: 10.1001/jama.294.5.563. doi: 10.1001/jama.294.5.563. [DOI] [PubMed] [Google Scholar]
- Christiansen E, Borge FJ. Risk of repetition of suicide attempt, suicide, or all deaths after an episode of attempted suicide: A register-based survival analysis. Australian and New Zealand Journal of Psychiatry. 2007;41:257–265. doi: 10.1080/00048670601172749. doi: 10.1080/00048670601172749. [DOI] [PubMed] [Google Scholar]
- Cleveland SS. LOWESS: A program for smoothing scatterplots by robust locally weighted regression. The American Statistician. 1981;35:54. doi: 10.2307/2683591. [Google Scholar]
- Conwell Y, Duberstein PR, Cox C, Herrman J, Forbes N, Caine ED. Age differences in behaviors leading to completed suicide. American Journal of Geriatric Psychiatry. 1998;6:122–126. doi: 10.1097/00019442-199805000-00005. [PubMed] [Google Scholar]
- Conway CC, Hammen C, Brennan PA. Expanding stress generation theory: Test of a transdiagnostic model. Journal of Abnormal Psychology. 2012;121(3):754–766. doi: 10.1037/a0027457. doi: 10.1037/a0027457. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Crane C, Williams JMG, Hawton K, Arensman E, Hjelmeland H, Bille-Brahe U, Wasserman D. The association between life events and suicide intent in self-poisoners with and without a history of deliberate self-harm: A preliminary study. Suicide and Life-Threatening Behavior. 2007;37(4):367–378. doi: 10.1521/suli.2007.37.4.367. doi: 10.1521/suli.2007.37.4.367. [DOI] [PubMed] [Google Scholar]
- Crosby AE, Ortega L, Melanson C. Self-directed violence surveillance: Uniform definitions and recommended data elements, Version 1. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control; 2011. Downloaded from the internet: http://www.cdc.gov/violenceprevention/pdf/Self-Directed-Violence-a.pdf. [Google Scholar]
- Dahlsgaard KK, Beck AT, Brown GK. Inadequate response to therapy as a predictor of suicide. Suicide & Life-Threatening Behavior. 1998;28(2):197–204. [PubMed] [Google Scholar]
- De Benedittis G, Lorenzetti A, Pieri A. The role of stressful life events in the onset of chronic primary headache. Pain. 1990;40:65–75. doi: 10.1016/0304-3959(90)91052-K. doi:10.1016/0304-3959(90)91052-K. [DOI] [PubMed] [Google Scholar]
- Diamond GM, Didner H, Waniel A, Priel B, Asherov J, Arbel S. Perceived parental care and control among Israeli female adolescents presenting to emergency rooms after self-poisoning. Adolescence. 2005;40(158):257–272. [PubMed] [Google Scholar]
- Foley DL, Goldston DB, Costello EJ, Angold A. Proximal psychiatric risk factors for suicidality in youth: The Great Smoky Mountains Study. Archives of General Psychiatry. 2006;63:1017–1024. doi: 10.1001/archpsyc.63.9.1017. doi:10.1001/archpsyc.63.9.1017. [DOI] [PubMed] [Google Scholar]
- Forman EM, Berk MS, Henriques GR, Brown GK, Beck AT. History of multiple suicide attempts as a behavioral marker of severe psychopathology. American Journal of Psychiatry. 2004;161(3):437–443. doi: 10.1176/appi.ajp.161.3.437. doi: 10.1176/appi.ajp.161.3.437. [DOI] [PubMed] [Google Scholar]
- Friedman H, Kohn R. Mortality, or probability of death, from a suicidal act in the United States. Suicide and Life-Threatening Behavior. 2008;38(3):287–301. doi: 10.1521/suli.2008.38.3.287. doi: 10.1521/suli.2008.38.3.287. [DOI] [PubMed] [Google Scholar]
- Goldston DB, Daniel SS, Reboussin BA, Reboussin DM, Frazier PH, Harris AE. Cognitive risk factors and suicide attempts among formerly hospitalized adolescents: A prospective naturalistic study. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40(1):91–99. doi: 10.1097/00004583-200101000-00021. [DOI] [PubMed] [Google Scholar]
- Goldston DB, Daniel SS, Reboussin DM, Reboussin BA, Frazier PH, Kelley AE. Suicide attempts among formerly hospitalized adolescents: A prospective naturalistic study of risk during the first 5 years after discharge. Journal of the American Academy of Child and Adolescent Psychiatry. 1999;38(6):660–671. doi: 10.1097/00004583-199906000-00012. [DOI] [PubMed] [Google Scholar]
- Goldston DB, Daniel SS, Erkanli A, Reboussin BA, Mayfield A, Frazier PH, Treadway SL. Psychiatric diagnoses as contemporaneous risk factors for suicide attempts among adolescents and young adults: Developmental changes. Journal of Consulting and Clinical Psychology. 2009;77(2):281–190. doi: 10.1037/a0014732. doi: 10.1037/a0014732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goldston DB, Molock SD, Whitbeck LB, Murakami JL, Zayas LH, Hall GCN. Cultural considerations in adolescent suicide prevention and psychosocial treatment. American Psychologist. 2008;63(1):14–31. doi: 10.1037/0003-066X.63.1.14. doi: 10.1037/0003-066X.63.1.14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hamdi E, Amin Y, Mattar T. Clinical correlates of intent in attempted suicide. Acta Psychiatrica Scandinavica. 1991;83:406–411. doi: 10.1111/j.1600-0447.1991.tb05565.x. doi: 10.1111/j.1600-0447.1991.tb05565.x. [DOI] [PubMed] [Google Scholar]
- Hollingshead A. Two factor index of social position. Yale University Department of Sociology; New Haven, CT: 1957. [Google Scholar]
- Holmes TH, Rahe RH. The Social Readjustment Rating Scale. Journal of Psychosomatic Research. 1967;11:213–218. doi: 10.1016/0022-3999(67)90010-4. doi: 10.1016/0022-3999(67)90010-4. [DOI] [PubMed] [Google Scholar]
- Joffe RT, Offord DR, Boyle MH. Ontario Child Health Study: Suicidal behavior in youth age 12-16 years. American Journal of Psychiatry. 1988;145(1):1420–1423. doi: 10.1176/ajp.145.11.1420. [DOI] [PubMed] [Google Scholar]
- Johnson JH, McCutcheon S. Assessing life stress in older children and adolescents: Preliminary findings with the Life Events Checklist. In: Sarason IG, Spielberger CD, editors. Stress and anxiety. Vol. 7. Hemisphere Publishing Corporation; New York: 1980. [Google Scholar]
- Joiner TE. Why people die by suicide. Harvard University Press; Cambridge, Mass.: 2005. [Google Scholar]
- Joiner TE, Rudd MD. Intensity and duration of suicidal crises vary as a function of previous suicide attempts and negative life events. Journal of Consulting and Clinical Psychology. 2000;68(5):909–916. doi: 10.1037/0022-006X.68.5.909. doi: 10.1037/0022-006X.68.5.909. [DOI] [PubMed] [Google Scholar]
- Kaslow NJ, Jacobs CH, Young SL, Cook S. Suicidal behavior among low-income African American women: A comparison of first-time and repeat suicide attempters. Journal of Black Psychology. 2006;32(3):349–365. doi: 10.1177/0095798406290459. [Google Scholar]
- Kessing LV. Severity of depressive episodes during the course of depressive disorder. British Journal of Psychiatry. 2008;192:290–293. doi: 10.1192/bjp.bp.107.038935. doi: 10.1192/bjp.bp.107.038935. [DOI] [PubMed] [Google Scholar]
- Kessing LV. Recurrence in affective disorder. II. Effects of age and gender. British Journal of Psychiatry. 1998;172:29–34. doi: 10.1192/bjp.172.1.29. doi: 10.1192/bjp.172.1.29. [DOI] [PubMed] [Google Scholar]
- Kessing LV, Mansen MG, Andersen PK, Angst J. The predictive effect of episodes on the risk of recurrence in depressive and bipolar disorders – a life-long perspective. Acta Psychiatrica Scandinavica. 2004;109:339–344. doi: 10.1046/j.1600-0447.2003.00266.x. doi: 10.1046/j.1600-0447.2003.00266.x. [DOI] [PubMed] [Google Scholar]
- Kessler RC, Borges G, Walters EE. Prevalence of and risk factors for lifetime suicide attempts in the National Comorbidity Study. Archives of General Psychiatry. 1999;56:617–626. doi: 10.1001/archpsyc.56.7.617. doi: 10.1001/archpsyc.56.7.617. [DOI] [PubMed] [Google Scholar]
- Kovacs M, Feinberg TL, Crouse-Novak MA, Paulauskas SL, Finkelstein R. Depressive disorders in childhood: A longitudinal prospective study of characteristics and recovery. Archives of General Psychiatry. 1984;41:229–237. doi: 10.1001/archpsyc.1984.01790140019002. doi:10.1001/archpsyc.1984.01790140019002. [DOI] [PubMed] [Google Scholar]
- Kovacs M, Goldston D, Gatsonis C. Suicidal behaviors and childhood-onset depressive disorders: A longitudinal study. Journal of the American Academy of Child and Adolescent Psychiatry. 1993;32(1):8–20. doi: 10.1097/00004583-199301000-00003. doi:10.1097/00004583-199301000-00003. [DOI] [PubMed] [Google Scholar]
- Kovacs M, Goldston D, Obrosky DS, Drash A. Major depressive disorder in youths with IDDM: A controlled prospective study of course and outcome. Diabetes Care. 1997;20(1):45–51. doi: 10.2337/diacare.20.1.45. [DOI] [PubMed] [Google Scholar]
- Kovacs M, Pollock M, Krol R. Follow-Up Interview Schedule for Adults (FISA) University of Pittsburgh School of Medicine, Western Psychiatric Institute and Clinic; Pittsburgh, PA: 1995. [Google Scholar]
- Kovacs M, Pollock M, Krol R. Interview Schedule for Children and Adolescents: Current and Interim (ISCA–C&I) University of Pittsburgh School of Medicine, Western Psychiatric Institute and Clinic; Pittsburgh, PA: 1997. [Google Scholar]
- Lantz PM, House JS, Mero RP, Williams DR. Stress, life events, and socioeconomic disparities in health: Results from the Americans’ Changing Lives Study. Journal of Health and Social Behavior. 2005;46:274–288. doi: 10.1177/002214650504600305. doi: 10.1177/002214650504600305. [DOI] [PubMed] [Google Scholar]
- Lewinsohn PM, Rohde P, Seeley JR. Psychosocial risk factors for future suicide attempts. Journal of Consulting and Clinical Psychology. 1994;62(2):297–305. doi: 10.1037//0022-006x.62.2.297. doi:10.1037/0022-006X.62.2.297. [DOI] [PubMed] [Google Scholar]
- Lewinsohn PM, Rohde P, Seeley JR. Adolescent suicidal ideation and attempts: Prevalence, risk factors, and clinical implications. Clinical Psychology: Science and Practice. 1996;3(1):25–46. doi:10.1111/j.1468-2850.1996.tb00056.x. [Google Scholar]
- Lewinsohn PM, Rohde P, Seeley JR, Baldwin CL. Gender differences in suicide attempts from adolescence to young adulthood. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40(4):427–434. doi: 10.1097/00004583-200104000-00011. doi: 10.1097/00004583-200104000-00011. [DOI] [PubMed] [Google Scholar]
- Lewinsohn PM, Zeiss AM, Duncan EM. Probability of relapse after recovery from an episode of depression. Journal of Abnormal Psychology. 1989;98(2):107–116. doi: 10.1037//0021-843x.98.2.107. doi: 10.1037/0021-843X.98.2.107. [DOI] [PubMed] [Google Scholar]
- Linehan M. Cognitive-behavioral treatment of borderline personality disorder. Guilford Press; New York: 1993. [Google Scholar]
- Little RJA, Rubin DB. Statistical analysis with missing data. John Wiley & Sons, Inc.; New York: 1987. [Google Scholar]
- Maj M, Veltro F, Pirozzi R, Lobrace S, Magliono L. Pattern of recurrence of illness after recovery from an episode of major depression: A prospective study. American Journal of Psychiatry. 1992;149(6):795–800. doi: 10.1176/ajp.149.6.795. [DOI] [PubMed] [Google Scholar]
- Michaelis BH, Goldberg JF, Singer TM, Garno JL, Ernst CL, Davis GP. Characteristics of first suicide attempts in first versus multiple suicide attempters with bipolar disorder. Comprehensive Psychiatry. 2003;44(1):15–20. doi: 10.1053/comp.2003.50004. doi:10.1053/comp.2003.50004. [DOI] [PubMed] [Google Scholar]
- Miller MA, Rahe RH. Life changes scaling for the 1990s. Journal of Psychometric Research. 1997;43(3):279–292. doi: 10.1016/s0022-3999(97)00118-9. doi: 10.1016/S0022-3999(97)00118-9. [DOI] [PubMed] [Google Scholar]
- Monroe SM, Harkness KL. Life stress, the “kindling hypothesis,” and the recurrence of depression: Considerations from a life stress perspective. Psychological Review. 2005;112(2):417–445. doi: 10.1037/0033-295X.112.2.417. doi: 10.1037/0033-295X.112.2.417. [DOI] [PubMed] [Google Scholar]
- National Association of Psychiatric Health Systems . Annual survey report. Author; Washington, DC: 2002. [Google Scholar]
- Nasser EH, Overholser JC. Assessing varying degrees of lethality in depressed adolescent suicide attempters. Acta Psychiatrica Scandinavica. 1999;99:423–431. doi: 10.1111/j.1600-0447.1999.tb00988.x. doi:10.1111/j.1600-0447.1999.tb00988.x. [DOI] [PubMed] [Google Scholar]
- Nock MK, Green JG, Hwang I, McLaughlin KA, Sampson NA, Zaslavsky AM, Kessler RC. Prevalence, correlates, and treatment of lifetime suicidal behavior among adolescents: Results from the National Comorbidity Scale Replication Adolescent Supplement. JAMA Psychiatry. 2013;70(3):300–310. doi: 10.1001/2013.jamapsychiatry.55. doi: 10.1001/2013.jamapsychiatry.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O'Brien G, Holton AR, Hurren K, Watt L, Hassanyeh F. Deliberate self harm – correlates of suicide intent and severity of depression. Acta Psychiatrica Scandinavica. 1987;75:474–477. doi: 10.1111/j.1600-0447.1987.tb02820.x. doi: 10.1111/j.1600-0447.1987.tb02820.x. [DOI] [PubMed] [Google Scholar]
- Ojehagen A, Danielsson M, Traskman-Bendz L. Deliberate self-poisoning: Treatment follow-up of repeaters and nonrepeaters. Acta Psychiatrica Scandinavica. 1992;85:370–375. doi: 10.1111/j.1600-0447.1992.tb10321.x. doi:10.1111/j.1600-0447.1992.tb10321.x. [DOI] [PubMed] [Google Scholar]
- Pettit JW, Joiner TE, Rudd MD. Kindling and behavioral sensitization: are they relevant to recurrent suicide attempts? Journal of Affective Disorders. 2004;83:249–252. doi: 10.1016/j.jad.2004.08.010. doi:10.1016/j.jad.2004.08.010. [DOI] [PubMed] [Google Scholar]
- Pompili M, Innamorati M, Szanto K, Di Vittorio C, Conwell Y, Lester D, Amore M. Life events as precipitants of suicide attempts among first-time suicide attempters, repeaters, and non-attempters. Psychiatry Research. 2011;186:300–305. doi: 10.1016/j.psychres.2010.09.003. doi: 10.1016/j.psychres.2010.09.003. [DOI] [PubMed] [Google Scholar]
- Posner K, Oquendo MA, Gould M, Stanley B, Davies M. Columbia Classification Algorithm of Suicide Assessment (C-CASA): Classification of suicidal events in the FDA's pediatric suicidal risk analysis of antidepressants. American Journal of Psychiatry. 2007;164:1035–1043. doi: 10.1176/appi.ajp.164.7.1035. doi:10.1176/appi.ajp.164.7.1035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Post RM. Transduction of psychosocial stress into the neurobiology of recurrent affective disorder. American Journal of Psychiatry. 1992;149(8):999–1010. doi: 10.1176/ajp.149.8.999. [DOI] [PubMed] [Google Scholar]
- Post RM. Kindling and sensitization as models for affective episode recurrence, cyclicity, and tolerance phenomena. Neuroscience and Biobehavioral Reviews. 2007;31:858–873. doi: 10.1016/j.neubiorev.2007.04.003. doi: 10.1016/j.neubiorev.2007.04.003. [DOI] [PubMed] [Google Scholar]
- Post RM, Rubinow DR, Ballenger JC. Conditioning and sensitization in the longitudinal course of affective illness. British Journal of Psychiatry. 1986;149:191–201. doi: 10.1192/bjp.149.2.191. doi: 10.1192/bjp.149.2.191. [DOI] [PubMed] [Google Scholar]
- R Development Core Team . R: A language and environment for statistical computing. R Foundation for Statistical Computing; Vienna, Austria: 2010. Retrieved from http://www.R-project.org. [Google Scholar]
- Reynolds P, Eaton P. Multiple attempters of suicide presenting at an emergency department. Canadian Journal of Psychiatry. 1986;31(4):328–330. doi: 10.1177/070674378603100408. [DOI] [PubMed] [Google Scholar]
- Rubin DB. Multiple imputation for nonresponse in surveys. John Wiley & Sons, Inc.; New York: 1987. [Google Scholar]
- Rubin DB. Multiple imputation after 18+ years. Journal of the American Statistical Association. 1996;91:473–489. [Google Scholar]
- Rudd MD, Joiner TE, Rajab MH. Help negation after acute suicidal crisis. Journal of Consulting and Clinical Psychology. 1995;63(3):499–503. doi: 10.1037//0022-006x.63.3.499. doi:10.1037/0022-006X.63.3.499. [DOI] [PubMed] [Google Scholar]
- Sapyta J, Goldston D, Erkanli A, Daniel S, Heilbron N, Mayfield A, Treadway S. Evaluating the predictive validity of suicidal intent and medical lethality in youth. Journal of Consulting and Clinical Psychology. 2012;80:222–231. doi: 10.1037/a0026870. doi:10.1037/a0026870. [DOI] [PMC free article] [PubMed] [Google Scholar]
- SAS Institute, Inc. SAS/STAT 9.2 User's Guide. SAS Institute, Inc.; Cary, NC: 2008. [Google Scholar]
- Segal ZV, Williams JM, Teasdale JD, Gemar M. A cognitive science perspective on kindling and episode sensitization in recurrent affective disorder. Psychological Medicine. 1996;26:371–380. doi: 10.1017/s0033291700034760. [DOI] [PubMed] [Google Scholar]
- Shearer SL, Peters CP, Quaytman MS, Wadman BE. Intent and lethality of suicide attempts among female borderline patients. American Journal of Psychiatry. 1988;145:1424–1427. doi: 10.1176/ajp.145.11.1424. [DOI] [PubMed] [Google Scholar]
- Sherrill JT, Kovacs M. Interview Schedule for Children and Adolescents (ISCA). Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39:67–75. doi: 10.1097/00004583-200001000-00018. doi:10.1097/00004583-200001000-00018. [DOI] [PubMed] [Google Scholar]
- Silverman MM, Berman AL, Sanddal ND, O'Carroll P W, Joiner TE. Rebuilding the tower of Babel: A revised nomenclature for the study of suicide and suicidal behaviors. Part 1: Background, rationale, and methodology. Suicide and Life-Threatening Behavior. 2007;37(3):248–263. doi: 10.1521/suli.2007.37.3.248. doi:10.1521/suli.2007.37.3.248. [DOI] [PubMed] [Google Scholar]
- Smith K, Conroy RW, Ehler BD. Lethality of Suicide Attempt Rating Scale. Suicide and Life-Threatening Behavior. 1984;14:215–242. doi: 10.1111/j.1943-278x.1984.tb00678.x. doi: 10.1111/j.1943-278X.1984.tb00678.x. [DOI] [PubMed] [Google Scholar]
- Stroud CB, Davila J, Hammen C, Vrshek-Schallhorn S. Severe and nonsevere events in first onsets versus recurrences of depression: Evidence for stress sensitization. Journal of Abnormal Psychology. 2011;120(1):142–154. doi: 10.1037/a0021659. doi: 10.1037/a0021659. [DOI] [PubMed] [Google Scholar]
- Tejedor MC, Diaz A, Castillon JJ, Pericay JM. Attempted suicide: Repetition and survival –Findings of a follow-up study. Acta Psychiatrica Scandinavica. 1999;100:205–211. doi: 10.1111/j.1600-0447.1999.tb10847.x. doi: 10.1111/j.1600-0447.1999.tb10847.x. [DOI] [PubMed] [Google Scholar]
- Velez CN, Cohen P. Suicidal behavior and ideation in a community sample of children: Maternal and youth reports. Journal of the American Academy of Child and Adolescent Psychiatry. 1988;27(3):349–356. doi: 10.1097/00004583-198805000-00014. doi:10.1097/00004583-198805000-00014. [DOI] [PubMed] [Google Scholar]
- Walker M, Moreau D, Weissman MM. Parents' awareness of children's suicide attempts. American Journal of Psychiatry. 1990;147:1364–1366. doi: 10.1176/ajp.147.10.1364. [DOI] [PubMed] [Google Scholar]
- Walker RA, Joiner TE, Rudd MD. The course of post-crisis suicidal symptoms: How and for whom is suicide “cathartic”? Suicide and Life-Threatening Behavior. 2001;31:144–152. doi: 10.1521/suli.31.2.144.21514. doi: 10.1521/suli.31.2.144.21514. [DOI] [PubMed] [Google Scholar]
- Wunderlich U, Bronisch T, Wittchen H-U, Carter R. Gender differences in adolescents and young adults with suicidal behavior. Acta Psychiatrica Scandinavica. 2001;104:332–339. doi: 10.1034/j.1600-0447.2001.00432.x. doi: 10.1111/j.1600-0447.2001.00432.x. [DOI] [PubMed] [Google Scholar]