Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Dec 1.
Published in final edited form as: J Child Psychol Psychiatry. 2017 Jul 4;58(12):1319–1329. doi: 10.1111/jcpp.12769

Implicit Identification with Death Predicts Change in Suicide Ideation during Psychiatric Treatment in Adolescents

Catherine R Glenn 1, Evan M Kleiman 2, Daniel D L Coppersmith 2, Angela C Santee 1, Erika C Esposito 3, Christine B Cha 4, Matthew K Nock 2, Randy P Auerbach 3
PMCID: PMC5693711  NIHMSID: NIHMS883523  PMID: 28675456

Abstract

Background

Suicidal thoughts and behaviors are major public health concerns in youth. Unfortunately, knowledge of reliable predictors of suicide risk in adolescents is limited. Promising research using a death stimuli version of the Implicit Association Test (Death IAT) indicates that stronger identification with death differs between adults with and without a history of suicidal thoughts and behaviors and uniquely predicts suicide ideation and behavior. However, research in adolescents is lacking and existing findings have been mixed. The current study extends previous research by testing whether implicit identification with death predicts changes in suicide ideation during psychiatric treatment in adolescents.

Methods

Participants included 276 adolescents, ages 13–19, admitted to a short-term residential treatment program. At hospital admission and discharge, adolescents completed the Death IAT and measures of recent suicidal thoughts.

Results

At admission, implicit identification with death was associated with recent suicide ideation, but did not differ between those who engaged in prior suicidal behavior and those who did not. Prospectively, adolescents’ implicit identification with death at admission significantly predicted their suicide ideation severity at discharge, above and beyond explicit suicide ideation. However, this effect only was significant for adolescents with longer treatment stays (i.e., more than 13 days).

Conclusions

Implicit identification with death predicts suicidal thinking among adolescents in psychiatric treatment. Findings clarify over what period of time implicit cognition about death may predict suicide risk in adolescents.

Keywords: suicide, suicidal behavior, adolescence, information processing, prediction

Introduction

Suicide and non-fatal suicidal behaviors (i.e., suicide attempts) are significant public health concerns among adolescents. Suicide is the second leading cause of death among youth ages 10–19 (CDC, 2015). In addition to suicide deaths, approximately 16% of US high school students report seriously considering suicide and 8% make one or more suicide attempts annually (CDC, 2015). Moreover, an estimated 51,518 US adolescents are hospitalized each year for self-inflicted injuries resulting in total annual costs of approximately $477,580,000 (CDC, 2010). Taken together, suicidal thoughts and behaviors in youth are prevalent, impairing, and costly.

To prevent suicide, a better understanding of the psychological processes that lead youth to become suicidal is needed. Despite decades of research on risk and protective factors, our ability to accurately predict suicidal behavior is limited (Franklin et al., 2017; Glenn & Nock, 2014; Nock, 2010). Prior research has been hampered by conceptual and methodological factors. Conceptually, over the past several decades, the vast majority of studies examining suicide risk factors have continued to test the same distal and nonspecific risk factors, despite their limited utility for understanding and predicting suicide risk (Franklin et al., 2017). Methodologically, a major limitation of prior research is the reliance on self-report and subjective measurement tools, which are problematic as: (a) individuals may be motivated to conceal their suicidal thoughts or plans (Busch, Fawcett, & Jacobs, 2003); (b) self-injurious and suicidal thoughts are variable, such that they can be absent one moment and present the next (Nock, Prinstein, & Sterba, 2009); and (c) people have limited access to the processes that may be influencing their behavior (Nisbett & Wilson, 1977). Thus, it is critical to develop and validate more objective methods for understanding and predicting risk of suicidal behavior (Glenn & Nock, 2014).

Implicit identification with death

A growing body of research suggests that implicit identification with death may be a meaningful marker of suicide risk and, importantly, tools assessing implicit cognition have the potential to address many of the aforementioned conceptual and methodological limitations. Implicit self-identification with death is hypothesized to develop, and strengthen, over time as an individual experiences more distress and considers more extreme solutions to end this pain (Nock, 2010). Notably, implicit identification with death is thought to be related but distinct from most explicit self-report measures of suicide ideation and risk (i.e., measures that require introspective reporting of suicidal thinking). Given that these implicit associations can occur outside of conscious awareness and are captured by measures other than self-report, they may signal when an individual is at greater risk of attempting suicide even if individuals are unwilling or unable to report their potential risk of self-injury. Therefore, the assessment of implicit identification with death may provide critical insight about risk of suicidal behavior.

Prior research testing a death stimuli version of the Implicit Association Test (Death IAT)1 in adults demonstrated that implicit identification with death increases with repeated thoughts of suicide and more engagement in suicidal behavior, and prospectively predicts suicidal thoughts and behaviors over time. The seminal Death IAT paper (Nock et al., 2010) found that implicit identification with death significantly distinguished adults who presented to the emergency department for a suicide attempt from adults presenting for other psychiatric reasons, and predicted suicide attempts over the subsequent six months, above and beyond prior suicide attempts and patient and clinician prediction of future risk. Notably, the robustness of these initial results has been demonstrated across multiple samples and settings. In another emergency department sample of adults, the Death IAT significantly predicted (nonsuicidal and suicidal) self-injurious behaviors over the following three months (Randall, Rowe, Dong, Nock, & Colman, 2013). Moreover, in a sample of psychiatrically hospitalized veterans, implicit identification with death uniquely predicted suicide attempts over the subsequent six months (Barnes et al., 2016). The nature of implicit death-related cognition also has been examined among nonclinical groups. In the largest study to date, 2,042 adults completed the Death IAT online. Implicit death-related cognition was strongest among adults with more frequent, recent, and medically severe suicide attempts (Glenn, Werntz, et al., 2016). Finally, in college students, stronger implicit identification with death significantly related to greater self-reported suicide ideation (Harrison, Stritzke, Fay, Ellison, & Hudaib, 2014; Vannoy et al., 2016), suggesting that implicit death-related cognition tracks severity of suicidal thinking in addition to behavior.

To date, only one published study has examined implicit identification with death in youth (Dickstein et al., 2015).2 Dickstein and colleagues (2015) compared psychiatric inpatients with a recent suicide attempt (SA only), psychiatric inpatients with previous nonsuicidal self-injury (NSSI only), and psychiatrically healthy controls (HC). Contrary to hypotheses, the NSSI only group exhibited stronger implicit identification with death relative to the SA and HC groups. These findings may reflect Berkson’s bias—namely, adolescents reporting NSSI may need to have greater clinical severity to be hospitalized relative to recent suicide attempters—thus, making these findings difficult to interpret. Taken together with extant adult research, this study suggests patterns of implicit death-related cognition may be different in adolescents. However, given that there is only one adolescent study, further investigation in younger clinical populations is warranted.

Recent expansions in this line of research have tested whether implicit identification with death is sensitive to changes over the course of treatment. In a series of studies, researchers administered the Death IAT two hours before and 24 hours after a one-dose treatment of ketamine (Price et al., 2014; Price, Nock, Charney, & Mathew, 2009). Results indicated no significant changes in implicit identification with death from pre- to post-treatment despite changes in explicitly reported suicide ideation. In contrast, a recent study in adults receiving inpatient psychiatric treatment (average treatment length was six weeks) found that implicit identification with death significantly decreased from admission to discharge, and moreover, implicit identification at admission predicted suicide ideation at discharge (Ellis, Rufino, & Green, 2016). However, this study did not control for change in suicide ideation over the course of treatment (i.e., they used different measures of suicide ideation at pre- and post-treatment); therefore, it is unclear whether the implicit identification with death uniquely predicted changes in suicide ideation beyond suicide ideation at admission (on that measure—a more conservative test). Moreover, no studies have examined the predictive utility of the Death IAT in youth, or how implicit identification with death may change over the course of treatment in younger populations.

Previous studies offer differing conclusions about the extent to which implicit identification with death may change over the course of treatment. However, one possibility that may help to explain the seemingly disparate results is that implicit identification with death may change over weeks, but not hours. This would be consistent with prior research in anxiety: Implicit associations with panic weakened over the course of cognitive behavioral therapy, but were not evident until week 6 of treatment (Teachman, Marker, & Smith-Janik, 2008). Thus, prior research suggests that implicit identification with death changes may occur over longer time periods. However, no study has explicitly tested this hypothesis. Even more importantly, prior research has not examined how the predictive utility of the Death IAT may vary by treatment length.

Current study

The current study tested the prospective utility of implicit identification with death in psychiatric inpatient youth (N=276), and extended prior research in five key ways. First, using a sample of high-risk adolescents receiving short-term residential treatment, we tested whether implicit identification with death is associated with severity and recency of suicidal thoughts and behaviors. Consistent with prior research, we hypothesized that stronger implicit identification with death is related to greater suicide ideation severity (Harrison et al., 2014; Vannoy et al., 2016), engagement in lifetime suicide attempts (Nock et al., 2010), and more recent suicide attempts (Nock et al., 2010). Second, in line with research in adults (Ellis et al., 2016), we hypothesized that implicit identification with death decreases over the course of short-term psychiatric treatment, with the assumption that adolescents would be more acutely suicidal at admission than at discharge. Third, we tested whether treatment length moderated the change in Death IAT from pre- to post-treatment, as length of treatment varied widely across the sample. We predicted greater change in Death IAT performance over longer (Ellis et al., 2016) as compared to shorter treatment lengths (Price et al., 2009; 2014). Fourth, we tested whether implicit identification with death at admission predicted suicide ideation at hospital discharge. We hypothesized that stronger implicit identification with death at admission would predict greater suicide ideation at discharge. Fifth, and finally, we tested whether treatment length moderated the predictive utility of implicit death-related cognition. Given prior research indicating that implicit associations may fluctuate in conjunction with explicit symptoms reports over the span of a few weeks in adults (Ellis et al., 2006; Price et al., 2009; 2014; Teachman et al., 2008), we hypothesized that the Death IAT would be more predictive of suicide ideation among adolescents with longer treatment stays.

Method

Participants

This study included two waves of data collection3 from a short-term residential psychiatric treatment unit for adolescents recruited from a larger study aimed at identifying psychobiological processes associated with adolescent self-injury and suicide (the samples are referred to as Wave 1 [W1] and Wave 2 [W2]). Eligible participants were adolescents aged 13–19 years with a history of suicidal thoughts or behaviors admitted to the treatment program from June 2012–December 2013 (W1=183) and December 2013–April 2015 (W2=210). Adolescents were excluded for the following reasons: psychiatric disorder with a potential medical/organic cause (W1=2, W2=9), unusable Death IAT data (W1=13, W2=29), missing suicide history data (W1=1, W2=4), and missing post-treatment (discharge) data (W1=18, W2=37). In W2, four adolescents were excluded because their treatment stay was less than one day (i.e., admission and discharge occurred on the same day) or more than 50 days (a significant outlier).

There were no significant differences between the two waves regarding age, gender, or ethnicity (all ps>.20). The two groups of adolescents also had similar diagnostic presentations. There were no significant differences in anxiety, behavior, bipolar, eating, or substance use disorders (all ps>.10). However, there were significantly fewer adolescents with a current depressive disorder in W2 (34.6%) than W1 (86.6%; χ2[1,N=276]=79.05, p<.001),2 more psychotic disorders in W2 (10.2%) than W1 (0%; χ2[1,N=276]=16.01, p<.001), and more attention-deficit hyperactivity disorder in W2 (22.8%) than W1 (12.1%; χ2[1,N=276]=5.61, p=.018). Finally, and most relevant to the present study, there were no significant differences between the two waves in terms of suicidal thoughts and attempts, Death IAT scores, or days of hospitalization (ps>.10). As the two waves were similar in recruitment, treatment received, and overall demographic and psychiatric features, and given that larger samples are preferable to replication with two underpowered samples (Bakker, van Dijk, & Wicherts, 2012), the two waves were combined to increase statistical power.

The final combined sample included 276 adolescents (W1=149, W2=127). The combined sample was 71.0% female, average age was 15.53 years (SD=1.34) and the self-identified racial/ethnic group distribution was 79.3% European American, 7.6% Asian, 2.2% African American, and 9.4% multiple racial/ethnic groups. Table 1 summarizes the prevalence and characteristics of psychiatric diagnoses and self-injurious and suicidal thoughts and behaviors in the total sample. As noted in the inclusion criteria, all adolescents reported lifetime suicide ideation. As no adolescents attempted suicide during their hospitalization, suicide attempts during hospitalization were not examined as an outcome variable.

Table 1.

Prevalence and frequency of self-injurious and suicidal thoughts and behaviors1 and major diagnoses at hospital admission.2

Prevalence Frequency3
% (n) M SD n4
Suicide Ideation
 Lifetime 100 (276) 308.56 529.82 255
 Past year 96.7 (267) 104.14 115.10 259
 Past month 89.9 (248) 13.42 10.04 247
 Past week 62.7 (173) 3.68 2.34 173
Suicide Attempts
 Lifetime 39.9 (110) 3.31 6.03 108
 Past year 33.7 (93) 2.97 6.10 92
 Past month 20.7 (57) 1.63 2.60 57
 Past week 2.5 (7) 1.29 0.49 7
Nonsuicidal Self-Injury
 Lifetime 81.2 (224) 249.92 792.46 213
 Past year 76.3 (203) 119.51 731.65 203
 Past month 62.7 (173) 8.56 12.76 173
 Past week 24.6 (68) 2.47 2.93 68
Major Psychiatric Disorder Categories
 Anxiety Disorder 65.6 (181)
 ADHD 17.0 (47)
 Behavior Disorder 0.7 (2)
 Bipolar Disorder 5.8 (16)
 Depressive Disorder 62.7 (173)
 Eating Disorder 5.1 (14)
 OCD 9.1 (25)
 PTSD 18.1 (50)
 Psychotic Disorder 4.7 (13)
 SUD 7.6 (21)
 Number of major psychiatric disorder categories 1.96 1.15 276
Open in a new tab
1

Measured with the Self-Injurious Thoughts and Behaviors Interview (SITBI).

2

Measured with the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-Kid). Anxiety disorder includes any of the following current disorders: panic disorder, agoraphobia, social phobia, specific phobia, or generalized anxiety disorder; ADHD = Attention-Deficit Hyperactivity Disorder includes any of the following current subtypes: combined, inattentive, or hyperactive/impulsive; Behavior Disorder includes current conduct disorder or oppositional defiant disorder; Bipolar Disorder includes current bipolar I or II disorder; Depressive Disorder includes current major depressive disorder or dysthymia; Eating Disorder includes current anorexia nervosa or bulimia nervosa; OCD = Obsessive Compulsive Disorder; PTSD = Posttraumatic Stress Disorder; SUD = Substance Use Disorder includes current alcohol abuse/dependence or substance (drug) abuse/dependence.

3

Frequency measured in days for suicide ideation and NSSI and episodes for suicide attempts.

4

Number of adolescents reporting frequency data.

Measures

Self-injurious and suicidal thoughts and behaviors (SITBs)

Explicit reports of SITBs were assessed with the Self-Injurious Thoughts and Behaviors Interview (SITBI), a structured interview that measures the presence and frequency of self-injurious and suicidal thoughts, plans, and behaviors over the individual’s lifetime, past year, past month, and past week (Nock, Holmberg, Photos, & Michel, 2007). The SITBI has good to excellent reliability and validity (Nock et al., 2007). Past week suicide ideation was also measured with the first 19 items of the Beck Scale for Suicide Ideation (SSI; i.e., excluding items 20 and 21 which measure history of suicidal behavior) (Beck & Steer, 1991).4 Suicide intent during the most recent suicide attempt was measured with item 21 on the SSI.

Major psychiatric disorders

Major psychiatric disorders were assessed with the Mini-International Neuropsychiatric Interview for Children and Adolescents, Child Version (Sheehan et al., 2010)—a brief diagnostic structured interview that assesses the major DSM-IV disorders diagnosed during childhood and adolescence. The MINI-Kid has good to excellent reliability and validity (Sheehan et al., 2010) and has been utilized in numerous studies to assess Axis I psychopathology in psychiatric inpatient samples of children and adolescents (Glenn & Klonsky, 2013; Stewart et al., 2015).

Implicit identification with death

The Death Implicit Association Test (Death IAT) is a brief, computer-based behavioral task that uses reaction times when classifying stimuli to measure implicit associations between death and the self (Nock et al., 2010). Words related to “death” (e.g., die, suicide) and “life” (e.g., alive, survive) are paired with either “me” or “not me” stimuli. Using standard IAT scoring procedures (Greenwald, Nosek, & Banaji, 2003), associations between “death” and “me” are measured by calculating a difference (D) score for each participant; positive D scores indicate a stronger association between death and the self (i.e., faster response latencies on death & me paired trials than on life & me paired trials), whereas negative D scores represent a stronger association between life and the self (see Nock et al., 2010 for scoring details). Importantly, iatrogenic effects of self-injury IATs (i.e., Self-Injury IAT, Death IAT, Suicide IAT) have been examined in adults, undergraduate students, and adolescent psychiatric inpatients: minimal change in desire for death or self-injury and only a slight mood decline has been observed from pre- to post-IAT administration (Cha, Glenn, et al., 2016). Therefore, self-injury IATs, such as the Death IAT, are considered appropriate to use with adolescent clinical populations.

Procedure

Study procedures were similar for both waves.3 All patients presenting to the acute residential treatment program completed admission and discharge assessments as part of quality assurance approved by the Institutional Review Board. During the brief hospitalization, adolescents received a combination of individual and group psychotherapy (e.g., Cognitive Behavior Therapy, Dialectical Behavior Therapy), family consultation, and pharmacotherapy. Consent was obtained during intake from all participants before study initiation: adolescent consent for participants 18–19 years-old and adolescent assent and parental permission for participants ages 13–17 years. The study protocol was administered by post-baccalaureate research assistants and masters-level and doctoral-level graduate students, who all completed 40 hours of didactic and practical training with a licensed clinical psychologist. The initial study session took place within the first 48 hours of admission to the treatment program. At admission, adolescents completed all study measures, including the MINI-Kid, SITBI, SSI, and Death IAT. At discharge, adolescents completed the SSI and Death IAT a second time. Consistent with prior studies (Cha, Augenstein, et al., 2016; Ellis et al., 2016), we selected assessment time points of admission and discharge for all participants (i.e., two assessments for all adolescents). This design yielded variable durations of time between assessments, allowing us to test the moderating effect of length of treatment stay.

Data analytic plan

Cross-sectional associations between the Death IAT and continuous suicide ideation measures were examined with Pearson product-moment correlations. Between-groups differences on the Death IAT based on history and recency of suicidal behavior were examined with independent samples t tests. Change in Death IAT scores and suicide ideation (SSI) from hospital admission to discharge were compared with paired samples t tests.

To identify relevant covariates, we examined associations between (a) key study variables: predictors (Death IAT and SSI at admission), outcomes (Death IAT and SSI at discharge), and moderators (treatment length) and (b) major sociodemographic (age, gender, race/ethnicity) and clinical variables (given the high rates of diagnostic comorbidity, number of psychiatric disorders was utilized). Key study variables were not significantly related to adolescent race/ethnicity (ps>.40). However, age, female gender, and number of major psychiatric disorders were related to at least one major study variable. Age was significantly negatively correlated with Death IAT scores at discharge, r(276)=−.20, p=.001. Female gender was significantly related to higher SSI scores at admission, t(274)=3.16, p=.002. Number of major psychiatric disorders was also (marginally) positively correlated with SSI scores at admission (r[276]=.12, p=.051). These covariates were included in all major cross-lagged analyses (described below).

To examine the unique prospective predictive ability of the Death IAT, we explored the reciprocal (i.e., cross-lagged) associations between SSI and Death IAT scores from hospital admission to discharge to see if baseline SSI predicted changes in Death IAT over treatment and if baseline Death IAT predicted changes in SSI over treatment. To conduct the cross-lagged panel design, we regressed SSI and Death IAT scores at discharge onto SSI and Death IAT scores at admission. We controlled for the identified covariates (age, female gender, number of major psychiatric disorders) in these analyses.

Adolescents’ treatment stays ranged from 3 to 42 days (M=14.06 days, SD=5.34), which led to great variability in the timing between assessments that could impact the cross-lagged effects. As such, we examined whether length of treatment stay moderated the cross-lagged analyses. To do so, we tested the same cross-lagged panel model, but also regressed onto suicide ideation (SSI) and Death IAT: (a) the main effect of treatment length, (b) the interaction between treatment length and SSI at admission, and (c) the interaction between treatment length and Death IAT at admission. To avoid multicollinearity, we centered all predictors prior to the calculation of the interaction term (Aiken, West, & Reno, 1991). Finally, we probed, plotted, and analyzed regions of significance for all significant interactions. Typical cutoffs (i.e., +/- 1 SD) for plotting an interaction may not be informative, and thus, we utilized a, region of significance approach, which determines the exact point where values of the moderator affect the relationship between the independent and the dependent variable (Preacher, Curran, & Bauer, 2006). By using this approach, we then determined the precise number of days between measurements when the nature of the relationship between SSI and Death IAT scores changed. We tested all cross-lagged models using R (R Core Team, 2016) with the laavan package (Rosseel, 2012). We controlled for the identified covariates (age, female gender, number of major psychiatric disorders) in these analyses.

Results

Implicit identification with death and explicit suicide severity

At hospital admission, there were small-to-medium associations between Death IAT scores and: (a) days of suicide ideation in the past month, r(274)=.20, p=.001, (b) days of suicide ideation in the past week, r(275)=.22, p<.001, and (c) suicide severity in the past week, r(275)=.14, p=.02. Contrary to our hypotheses, adolescents who had attempted suicide (M=−.29, SD=.33) did not exhibit significantly stronger implicit identification with death compared to adolescents who had thought about suicide but never attempted (M=−.28, SD=.40), t(273)=.30, p=.76, d=.04. There also were no significant differences in implicit identification with death between adolescents who attempted suicide in the past year (n=36; M=−.24, SD=.37) and those who attempted suicide in the past month (n=57; M=−.31, SD=.31), t(91)=1.03, p=.31, d=.21.

We further explored factors that may account for the nonsignificant performance differences on the Death IAT between suicide attempters and nonattempters. Group differences on the Death IAT remained nonsignificant based on: (1) number of lifetime suicide attempts: comparing nonattempters (n=166) to single suicide attempters (n=43) and multiple suicide attempters (n=65), F(2,272)=.732, p=.48, (2) intent of attempts: comparing nonattempters (n=166) to suicide attempts with low to moderate intent (n=40), and high intent (n=70) reported on the SSI, F(2,274)=.10, p=.90, and (3) controlling for differences in past-week suicide ideation severity (SSI) between the attempter group (n=110; M=15.52, SD=9.51) and the nonattempter group (n=166; M=9.61, SD=8.57), F(1,272)=1.11, p=.29. Although the majority of the sample reported a lifetime history of NSSI (81.2%; see Table 1), adolescents in the attempter group (88.2%) reported significantly more lifetime NSSI than the nonattempter group (76.5%), χ2(1, N=276)=5.90, p=.02. The breakdown of the sample by engagement in NSSI and SA was as follows: (1) No NSSI or SA (ideation only): 14.1% (n=39), (2) NSSI only: 46.0% (n=127), (3) SA only: 4.7% (n=13), (4) NSSI + SA: 35.1% (n=97). The SA only group was fairly small and excluding these individuals did not significantly impact the analyses comparing attempters and nonattempters on the Death IAT, t(260)=.42, p=.68. Results were similar when we compared the NSSI only group and NSSI + SA groups, t(222)=.39, p=.70. Finally, the number of major psychiatric diagnoses was not significantly different between the attempter (M=2.09, SD=1.25) and nonattempter groups (M=1.88, SD=1.08; t[274]=, p=.14). Moreover, the specific categories of disorders were relatively similar across groups. There were no differences between groups in terms of anxiety disorders, mood disorders (depression or bipolar), obsessive-compulsive disorder (OCD), attention-deficit hyperactivity disorder (ADHD), disruptive behavior disorders, psychotic disorders, or eating disorders (ps ranged from .12 for ADHD to .99 for OCD). However, compared to the nonattempter group (NA), the attempter group (A) did have significantly more adolescents who met criteria for: posttraumatic stress disorder (PTSD; A: 25.5%; NA: 13.3%), χ2(1, N=276)=6.64, p=.01 and a substance use disorder (SUD; A: 12.7%; NA: 4.2%), χ2(1, N=276)=6.82, p=.01. Although not ideal for accounting for inherent group differences (Miller & Chapman, 2001), statistically controlling for these diagnoses did not change between-groups differences on the Death IAT (PTSD: F(1,272)=.08, p=.77; SUD: F(1,272)=.09, p=.76).

Change in implicit identification with death during treatment

Descriptive statistics and bivariate associations for suicide ideation (SSI), Death IAT scores, and treatment length (in days) are provided in Table 2. Both Death IAT (t[249]=2.22, p=.03, d=.19) and SSI (t[275]=13.57, p<.001, d=.74) scores decreased significantly from hospital admission to discharge. The reciprocal cross-lagged model showed that there were significant direct (stability) paths from SSI at admission to SSI at discharge and from Death IAT at admission to Death IAT at discharge (ps<.001; Figure 1). The significant stability effects (i.e., relationships between constructs over time) indicated test-retest reliability for both constructs, controlling for identified covariates (age, female gender, number of major psychiatric disorders).

Table 2.

Bivariate correlations among major study variables at hospital admission and discharge.

SSI
Admission		 Death IAT
Admission		 SSI
Discharge		 Death IAT
Discharge		 Treatment
Length
(days)
 SSI Admission ----- .136*   .617*** .036       .281***
Death IAT Admission ----- .176**       .328*** .043
SSI Discharge ----- .071       .217***
Death IAT Discharge ---- .094
Treatment Length ----
     n 276 275 276 250 276
     M 11.97 −.28 5.74 −.35 14.06
     SD 9.39 .38 7.28 .37 5.34
Open in a new tab

IAT= Implicit Association Test; SSI=Beck Scale for Suicide Ideation

***

p<.001,

**

p<.01,

*

p<.05

Figure 1. Cross-lagged panel (i.e., reciprocal associations) between implicit identification with death (Death IAT) and suicide ideation (SSI) at hospital admission and discharge.

Figure 1

Open in a new tab

***p<.001, **p<.01, *p<.05.

Note. Dashed lines show regression paths to discharge Death IAT, solid lines show regression paths to discharge SSI; all regression paths controlling for identified covariates (i.e., age, (female) gender, and number of major psychiatric disorders). Unstandardized regression weights shown. Curved lines are covariances. Regression paths from covariates at admission predicting main discharge variables are not shown for clarity purposes: Number of diagnoses → Discharge Death IAT (b=0.005, p=.79), Number of diagnoses → Discharge SSI (b=0.03, p=.92), Age → Discharge Death IAT (b=−0.05, p=.003), Age → Discharge SSI (b=−0.06, p=.83), (Female) gender ↔ Discharge Death IAT (b=0.05, p=.29), and (Female) gender ↔ Discharge SSI (b=−0.76, p=.29). The following covariances not shown for clarity purposes: Number of diagnoses ↔ Admission Death IAT (cov=0.04, p=.09), Number of diagnoses ↔ Admission SSI (cov=0.93, p=.15), Number of diagnoses ↔ Age (cov=−0.03, p=.78), and Number of diagnoses ↔ (Female) gender (cov=.07, p=.02).

Prospective utility of implicit identification with death

As shown in Figure 1, there was a significant cross-lagged effect of Death IAT at admission to SSI at discharge, but not of SSI at admission to Death IAT at discharge. This suggests that admission Death IAT significantly predicted discharge SSI (controlling for admission SSI), but admission SSI did not predict discharge Death IAT (controlling for admission Death IAT). Effects held when controlling for the identified covariates.

Moderation by treatment length

We tested whether the days of treatment moderated the relationship between admission Death IAT and SSI and discharge Death IAT and SSI (see Figure 2). In this moderated model, there was a significant interaction between treatment length and Death IAT at admission predicting suicide ideation at discharge (p<.05). This effect held when controlling for the identified covariates. We plotted and probed this significant interaction. Admission Death IAT did not predict discharge suicide ideation for those who had shorter treatment stays (−1SD; <~9 days; b=−0.04, t=−0.03, p=.96) treatment stays. However, it did for those with average (b=2.66, t=2.03, p=.04) or longer (+1SD; >~20 days; b=5.50, t=3.99, p<.001) treatment stays. Region of significance testing (Preacher et al., 2006) indicated that the relationship between admission Death IAT and discharge suicide ideation became significant at approximately 13.22 days of treatment (see Figure 3). Of note, treatment length did not moderate the stability of Death IAT over the course of treatment; that is, change in Death IAT from admission to discharge did not significantly differ based on treatment length (see Figure 2).

Figure 2. Moderated cross-lagged panel between implicit identification with death (Death IAT) at hospital admission, suicide ideation (SSI) at admission, and length of treatment (in days) predicting Death IAT and SSI at hospital discharge.

Figure 2

Open in a new tab

***p<.001, **p<.01, *p<.05

Note. Dashed lines show regression paths to discharge Death IAT, solid lines show regression paths to discharge SSI; all regression paths controlling for identified covariates (i.e., age, female gender, and number of major psychiatric disorders). Unstandardized regression weights shown. Curved lines are covariances. Regression paths from covariates at admission predicting main discharge variables are not shown for clarity purposes: Number of diagnoses → Discharge Death IAT (b=−0.003, p=.89), Number of diagnoses → Discharge SSI (b=−0.10, p=.76), Age → Discharge Death IAT (b=−0.06, p=.001), Age → Discharge SSI (b=−0.05, p=.19), (Female) gender ↔ Discharge Death IAT (b=−0.67, p=.16), and (Female) gender ↔ Discharge SSI (b=0.04, p=.32). The following covariances not shown for clarity purposes: Number of diagnoses ↔ Admission Death IAT (cov=0.04, p=.09), Number of diagnoses ↔ Admission SSI (cov=1.91, p=.07), Number of diagnoses ↔ Age (cov=−0.09, p=.35), and Number of diagnoses ↔ (Female) gender (cov=.08, p=.01).

Figure 3. Implicit identification with death (Death IAT) predicting discharge suicide ideation (SSI) moderated by length of residential treatment (in days).

Figure 3

Open in a new tab

Note. Analysis controls for identified covariates (i.e., age, female gender, and number of major psychiatric disorders).

Discussion

This study reports the largest prospective examination to date of adolescents’ implicit death-related cognition. In adolescent psychiatric inpatients, findings partially supported prior research indicating that implicit identification with death is correlated with severity of suicidal thoughts (Harrison et al., 2014; Vannoy et al., 2016) and decreases over the course of treatment (Ellis et al., 2016). Notably, this is the first study in adolescents to demonstrate that implicit self-identification with death prospectively predicts suicide ideation in adolescents. Several of these findings warrant further comment.

First, implicit identification with death was modestly related to explicit reports of suicidal thinking at hospital admission, but there were no differences based on previous engagement in suicidal behavior. Although contrary to our hypotheses, our results are less surprising given the previous mixed cross-sectional findings. Some studies have found stronger implicit associations among more severely suicidal individuals (Glenn, Werntz, et al., 2016; Nock et al., 2010), whereas others have not found cross-sectional differences (Barnes et al., 2016; Dickstein et al., 2015; Price et al., 2014). Moreover, the modest cross-sectional associations between implicit and explicit suicidal thinking may be due to the restricted range on suicide measures in more clinically severe samples (Barnes et al., 2016; Price et al., 2014), the difference in the constructs being assessed (i.e., explicit suicide ideation and implicit self-identification with death), as well as the low to moderate correspondence generally found between explicit and implicit measures (Nosek, 2005; Werntz, Steinman, Glenn, Nock, & Teachman, 2016).

Second, this study found reciprocal changes in implicit identification with death and suicide ideation over the course of psychiatric treatment among adolescents. Just as these implicit associations can strengthen with increased suicidal thinking, they are also hypothesized to weaken over time as suicidal thinking decreases. Consistent with research in adults (Ellis et al., 2016), there was a small decrease in implicit identification with death, as well as a large decrease in explicit suicide ideation over this period. Our findings suggest that explicit suicide ideation did not account for the changes in implicit identification with death observed from pre- to post-treatment. Implicit identification with death may be targeted (directly or indirectly) during psychosocial treatment by strengthening individuals’ coping skills for dealing with distress (e.g., emotion regulation, distress tolerance, interpersonal effectiveness), thereby making suicide a less desirable option, as well as by increasing salience of individuals’ reasons for living (for a review of efficacious treatment components for self-injurious and suicidal youth, see: Glenn, Franklin, & Nock 2015). In this particular study, we were not able to directly examine how the specific treatment received, or engagement with treatment, may have moderated the study findings. This is an interesting direction for future research.

Third, we found that stronger implicit identification with death at hospital admission significantly predicted greater suicide ideation at hospital discharge, a finding that is consistent with the results from a recent study of hospitalized adults (Ellis et al., 2016). Extending those earlier findings (Ellis et al., 2016), we observed that this effect held even when controlling for suicide ideation at admission (using the same measure). This means that implicit identification with death at pre-treatment uniquely predicted suicide ideation post-treatment, over and above adolescents’ history of self-reported suicide ideation.

Finally, findings indicated that this predictive effect was significantly moderated by treatment length, which varied widely across the sample (3–42 days). Implicit identification with death at admission did not predict suicide ideation at discharge among adolescents with relatively short treatment stays (≤13 days). Of note, this group reported the lowest suicide ideation at both admission and discharge. One possibility is that adolescents with shorter treatment stays underreported their suicide ideation to facilitate a more rapid hospital discharge. Alternatively, as this group was also likely less psychiatrically complex and reported lower severity of suicide ideation, the lack of implicit-explicit suicidal thinking may reflect a floor effect (i.e., lower limit and restricted range). However, the Death IAT at admission did significantly predict suicide ideation at discharge among adolescents with treatment stays of at least 13 days. These results may help to clarify the time point at which the Death IAT provides predictive utility and may explain previous mixed findings examining change in the Death IAT during treatment (Ellis et al., 2016; Price et al., 2009; 2014).

Implications

Consistent with research using the self-injury version of the IAT among adolescents (Cha, Augenstein, et al., 2016; Glenn, Kleiman, Cha, Nock, & Prinstein, 2016; Nock & Banaji, 2007), current findings indicate that implicit identification with death is present early on in the development of suicidal thoughts and behaviors—rather than after decades of chronic suicidal thoughts, urges, and behaviors. Moreover, like research in adults (Barnes et al., 2016; Ellis et al., 2016; Nock et al., 2010), implicit death-related cognition correlates with suicidal states: among a group of suicidal adolescents, implicit identification with death significantly predicted persistent suicidal thinking at hospital discharge. Integrating the current study with prior work, the Death IAT appears to assess suicidal thinking (and potentially suicide risk) that is not consciously accessible to the individual or measurable via explicit self-report measures. Finally, this study may help integrate mixed findings from previous studies about the treatment length needed to observe change in implicit death-related cognition. An important next step is for research to examine whether implicit identification with death may predate initial onset of suicide ideation.

Study findings also have the potential to inform clinical practice with suicidal youth. First, implicit identification with death significantly predicted suicide ideation at hospital discharge, above and beyond adolescents’ suicide ideation at admission. These results add to growing research indicating that objective tools provide unique predictive information and may be useful for augmenting explicit reports of suicide risk (Barnes et al., 2016; Nock et al., 2010). For instance, these types of tools could potentially be useful, in combination with other risk assessment tools, for informing clinical decision-making (e.g., when to discharge from clinical care) (Mann et al., 2008; Oquendo & Bernanke, 2017). However, before making this translation to clinical practice, it will be important for future research to clarify how to integrate implicit and explicit measures of suicidal thinking, how to utilize these tools effectively at the individual level, and which measures optimally predict outcomes (e.g., Cha, Augenstein, et al., 2016).

This research also may have implications for interventions with suicidal individuals. If malleable, implicit identification with death could be a novel target for treatment. Promising results have been found for an evaluative conditioning procedure that increases aversion to self-harm and decreases aversion to the self (Franklin et al., 2016). However, it is unclear whether these types of brief interventions will lead to long-term behavioral change.

Limitations and future directions

These results should be interpreted in the context of several important limitations. First, our admission-discharge design resulted in varied intervals between assessments. Although we examined how treatment length moderated associations between implicit and explicit suicidal thinking, differences in treatment length could have impacted the change in suicidal thinking during treatment. Second, although length of treatment was found to be a significant moderator, the current study had limited information about the treatment received (e.g., type, engagement). Future research would benefit from clarifying the type and dose of treatment needed to change implicit identification with death. Third, the main outcome—suicide ideation at hospital discharge—may have been biased by adolescents’ motivations to underreport (e.g., to ensure discharge) or overreport suicide severity (e.g., to delay discharge). Future studies should examine how the Death IAT relates to less biased measures of suicidal thoughts and behaviors, such as reports of suicide ideation during hospital stay that would not directly impact hospital discharge and/or suicidal behaviors that can be confirmed from outside sources (e.g., hospital records). In addition, given growing research indicating that the same risk factors that predict suicide ideation do not predict suicide attempts (Kessler et al., 1999; Nock et al., 2009), future research would benefit from examining how implicit identification with death prospectively predicts suicidal behavior in youth. Fourth, the null Death IAT findings between attempters and nonattempters could be due to the self-reported assessment of attempts. Although this strategy has been used in other studies that have found significant between-group effects (Glenn, Werntz, et al., 2016; Nock et al., 2010), future research would benefit from incorporating informant reports of attempts (e.g., clinician reports or medical records). Finally, although many key results were statistically significant, effect sizes were small-to-moderate in most cases. Additional research is needed to identify other potential moderators of the cross-lagged relationship (e.g., sociodemographic factors).

Summary

Extending research in adults, the current study indicates that implicit death-related cognition may improve prediction of suicide risk in adolescents. Specifically, in short-term psychiatric settings, implicit identification with death may uniquely predict continued suicidal thinking above and beyond self-reported suicide ideation. The next step in this line of research is to determine whether reducing implicit identification with death leads to sustained reductions in suicidal thoughts and behaviors in youth.

Key Points.

  • Research in adults suggests that stronger implicit identification with death indicates increased risk for suicide.

  • However, research in adolescents is limited and findings are mixed.

  • We tested whether implicit identification with death predicts suicide ideation during psychiatric treatment in adolescents.

  • Implicit identification with death predicted suicide ideation at discharge among adolescents receiving 13+ days of treatment.

  • Findings clarify over what period of time implicit identification with death may predict suicide risk in adolescents.

Acknowledgments

The research was partially supported by grants from the National Institute of Mental Health (F32 MH097354 [CRG]; K23MH097786 [RPA]), the Klingenstein Third Generation Foundation Adolescent Depression Fellowship (RPA), the Tommy Fuss Fund (RPA), the Simches Fund (RPA), and with support from the John D. and Catherine T. MacArthur Foundation (MKN). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or NIMH.

Footnotes

The authors have declared that they have no competing or potential conflicts of interest.

Conflict of interest statement: No conflicts declared.

1

The “Death IAT” was formerly referred to as the “Death/Suicide IAT” (Nock et al., 2010) and primarily includes words related to death as well as the word “suicide”. Since the seminal study by Nock et al. (2010), a suicide-specific version of the IAT has been created (the “Suicide IAT”), which includes words related to specific suicide methods (e.g., hanging). For clarity, and consistency with other recent publications (e.g., Glenn, Werntz, et al., 2016), we refer to the version of the IAT that displays words related to death including the word “suicide” as the Death IAT.

2

Nock and Banaji (2007) utilized a Self-Injury (cutting) IAT to predict suicide ideation and attempts in youth, but not the Death IAT. Cha, Glenn, et al. (2016) examined iatrogenic effects of a series of IATs (including the Self-Injury, Suicide, and Death IATs) among adolescent psychiatric inpatients, but did not report specifically on IAT performance. Cha, Augenstein, et al. (2016) examined the Self-Injury IAT to predict nonsuicidal self-injury among adolescent psychiatric inpatients, but did not assess the Suicide or Death IAT.

3

Demarcations between waves were made because in W2 some adolescents with major depression were recruited for another behavioral task (instead of the Death IAT), which explains the lower rate of major depression among adolescents in this period of data collection. This was the only change to the assessment battery.

4

Suicide ideation assessed via questionnaire was confirmed via interview. Specifically, SSI item #4 endorsed at either 1=“I have a weak desire to kill myself” or 2=“I have a moderate to strong desire to kill myself” in the past week was confirmed with ≥1 day of suicide ideation in the past week on the SITBI.

References

  1. Aiken LS, West SG, Reno RR. Multiple regression: Testing and interpreting interactions. Thousand Oaks, CA: SAGE Publications, Inc; 1991. [Google Scholar]
  2. Bakker M, van Dijk A, Wicherts JM. The rules of the game called psychological science. Perspectives on Psychological Science. 2012;7(6):543–554. doi: 10.1177/1745691612459060. [DOI] [PubMed] [Google Scholar]
  3. Barnes SM, Bahraini NH, Forster JE, Stearns-Yoder KA, Hostetter TA, Smith G, Nock MK. Moving beyond self-report: Implicit associations about death/life prospectively predict suicidal behavior among veterans. Suicide and Life-Threatening Behavior. 2016 doi: 10.1111/sltb.12265. [DOI] [PubMed] [Google Scholar]
  4. Beck AT, Steer RA. Manual for the Beck scale for suicide ideation. San Antonio, TX: Psychological Corporation; 1991. [Google Scholar]
  5. Busch KA, Fawcett J, Jacobs DG. Clinical correlates of inpatient suicide. Journal of Clinical Psychiatry. 2003;64(1):14–19. doi: 10.4088/jcp.v64n0105. [DOI] [PubMed] [Google Scholar]
  6. Centers for Disease Control and Prevention. Injury Center: Violence Prevention—Suicide Prevention: Youth Suicide. 2015 Retrieved from http://www.cdc.gov/violenceprevention/suicide/youth_suicide.html.
  7. Centers for Disease Control and Prevention. Web-based Injury Statistics Query and Reporting System (WISQARS) National Center for Injury Prevention and Control, Center for Disease Control and Prevention; 2010. Available from http://www.cdc.gov/injury/wisqars/index.html. [Google Scholar]
  8. Cha CB, Augenstein TM, Frost KH, Gallagher K, D’Angelo EJ, Nock MK. Using implicit and explicit measures to predict nonsuicidal self-injury among adolescent inpatients. Journal of the American Academy of Child & Adolescent Psychiatry. 2016;55(1):62–68. doi: 10.1016/j.jaac.2015.10.008. [DOI] [PubMed] [Google Scholar]
  9. Cha CB, Glenn JJ, Deming CA, D’Angelo EJ, Hooley JM, Teachman BA, Nock MK. Examining potential iatrogenic effects of viewing suicide and self-injury stimuli. Psychological Assessment. 2016;28:1510–1515. doi: 10.1037/pas0000280. [DOI] [PubMed] [Google Scholar]
  10. Dickstein DP, Puzia ME, Cushman GK, Weissman AB, Wegbreit E, Kim KL, Spirito A. Self-injurious implicit attitudes among adolescent suicide attempters versus those engaged in nonsuicidal self-injury. Journal of Child Psychology and Psychiatry. 2015;56(10):1127–1136. doi: 10.1111/jcpp.12385. [DOI] [PubMed] [Google Scholar]
  11. Ellis TE, Rufino KA, Green KL. Implicit measure of life/death orientation predicts response of suicidal ideation to treatment in psychiatric inpatients. Archives of Suicide Research. 2016;20(1):59–68. doi: 10.1080/13811118.2015.1004483. [DOI] [PubMed] [Google Scholar]
  12. Franklin JC, Fox KR, Franklin CR, Kleiman EM, Ribeiro JD, Jaroszewski AC, Nock MK. A brief mobile app reduces nonsuicidal and suicidal self-injury: Evidence from three randomized controlled trials. Journal of Consulting and Clinical Psychology. 2016;84:544–557. doi: 10.1037/ccp0000093. [DOI] [PubMed] [Google Scholar]
  13. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Jaroszewsi AC, Nock MK. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin. 2017 doi: 10.1037/bul0000084. [DOI] [PubMed] [Google Scholar]
  14. Glenn CR, Franklin JC, Nock MK. Evidence-based psychosocial treatments for self-injurious thoughts and behaviors in youth. Journal of Clinical Child & Adolescent Psychology. 2015;44(1):1–29. doi: 10.1080/15374416.2014.945211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Glenn CR, Kleiman EM, Cha CB, Nock MK, Prinstein MJ. Implicit cognition about self-injury predicts actual self-injurious behavior: Results from a longitudinal study of adolescents. Journal of Child Psychology and Psychiatry. 2016;57(7):805–813. doi: 10.1111/jcpp.12500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Glenn CR, Klonsky ED. Nonsuicidal self-injury disorder: An empirical investigation in adolescent psychiatric patients. Journal of Clinical Child & Adolescent Psychology. 2013;42(4):496–507. doi: 10.1080/15374416.2013.794699. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Glenn CR, Nock MK. Improving the short-term prediction of suicidal behavior. American Journal of Preventive Medicine. 2014;47(3):S176–S180. doi: 10.1016/j.amepre.2014.06.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Glenn JJ, Werntz AJ, Slama SJ, Steinman SA, Teachman BA, Nock MK. Suicide and self-injury-related implicit cognition: A large-scale examination and replication. Journal of Abnormal Psychology. 2016 doi: 10.1037/abn0000230. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Greenwald AG, Nosek BA, Banaji MR. Understanding and using the implicit association test: I. An improved scoring algorithm. Journal of Personality and Social Psychology. 2003;85(2):197–216. doi: 10.1037/0022-3514.85.2.197. [DOI] [PubMed] [Google Scholar]
  20. Harrison DP, Stritzke WGK, Fay N, Ellison TM, Hudaib AR. Probing the implicit suicidal mind: Does the death/suicide implicit association test reveal a desire to die, or a diminished desire to live? Psychological Assessment. 2014;26(3):831–840. doi: 10.1037/pas0000001. [DOI] [PubMed] [Google Scholar]
  21. Mann JJ, Ellis SP, Waternaux CM, Liu X, Oquendo MA, Malone KM, Currier D. Classification trees distinguish suicide attempters in major psychiatric disorders: A model of clinical decision making. Journal of Clinical Psychiatry. 2008;69(1):23–31. doi: 10.4088/jcp.v69n0104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Miller GA, Chapman JP. Misunderstanding analysis of covariance. Journal of Abnormal Psychology. 2001;110(1):40–48. doi: 10.1037//0021-843x.110.1.40. [DOI] [PubMed] [Google Scholar]
  23. Nisbett RE, Wilson TD. Telling more than we can know: Verbal reports on mental processes. Psychological Review. 1977;84(3):231–259. doi: 10.1037/0033-295x.84.3.231. [DOI] [Google Scholar]
  24. Nock MK. Self-Injury. Annual Review of Clinical Psychology. 2010;6:339–363. doi: 10.1146/annurev.clinpsy.121208.131258. [DOI] [PubMed] [Google Scholar]
  25. Nock MK, Banaji MR. Prediction of suicide ideation and attempts among adolescents using a brief performance-based test. Journal of Consulting and Clinical Psychology. 2007;75(5):707–715. doi: 10.1037/0022-006X.75.5.707. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Nock MK, Holmberg EB, Photos VI, Michel BD. Self-injurious thoughts and behaviors interview: Development, reliability, and validity in an adolescent sample. Psychological Assessment. 2007;19(3):309–317. doi: 10.1037/1040-3590.19.3.309. [DOI] [PubMed] [Google Scholar]
  27. Nock MK, Park JM, Finn CT, Deliberto TL, Dour HJ, Banaji MR. Measuring the suicidal mind: Implicit cognition predicts suicidal behavior. Psychological Science. 2010;21(4):511–517. doi: 10.1177/0956797610364762. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Nock MK, Prinstein MJ, Sterba SK. Revealing the form and function of self-injurious thoughts and behaviors: A real-time ecological assessment study among adolescents and young adults. Journal of Abnormal Psychology. 2009;118(4):816–827. doi: 10.1037/a0016948. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Nosek BA. Moderators of the relationship between implicit and explicit evaluation. Journal of Experimental Psychology: General. 2005;134(4):565e584. doi: 10.1037/0096-3445.134.4.565. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Oquendo MA, Bernanke JA. Suicide risk assessment: tools and challenges. World Psychiatry. 2017;16:28–29. doi: 10.1002/wps.20396. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Preacher KJ, Curran PJ, Bauer DJ. Computational tools for probing interactions in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics. 2006;31(4):437–448. [Google Scholar]
  32. Price RB, Iosifescu DV, Murrough JW, Chang LC, Al Jurdi RK, Iqbal SZ, Mathew SJ. Effects of ketamine on explicit and implicit suicidal cognition: A randomized controlled trial in treatment-resistant depression. Depression and Anxiety. 2014;31(4):335–343. doi: 10.1002/da.22253. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Price RB, Nock MK, Charney DS, Mathew SJ. Effects of intravenous ketamine on explicit and implicit measures of suicidality in treatment-resistant depression. Biological Psychiatry. 2009;66(5):522–526. doi: 10.1016/j.biopsych.2009.04.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2016. Retrieved from http://www.R-project.org. [Google Scholar]
  35. Randall JR, Rowe BH, Dong KA, Nock MK, Colman I. Assessment of self-harm risk using implicit thoughts. Psychological Assessment. 2013;25(3):714–721. doi: 10.1037/a0032391. [DOI] [PubMed] [Google Scholar]
  36. Rosseel Y. lavaan: An R package for structural equation modeling. Journal of Statistical Software. 2012;48(2):1–36. [Google Scholar]
  37. Sheehan DV, Sheehan KH, Shytle RD, Janavs J, Bannon Y, Rogers JE, Wilkinson B. Reliability and validity of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID) Journal of Clinical Psychiatry. 2010;71(3):313–326. doi: 10.4088/JCP.09m05305whi. [DOI] [PubMed] [Google Scholar]
  38. Stewart JG, Kim JC, Esposito EC, Gold J, Nock MK, Auerbach RP. Predicting suicide attempts in depressed adolescents: Clarifying the role of disinhibition and child sexual abuse. Journal of Affective Disorders. 2015;187:27–34. doi: 10.1016/j.jad.2015.08.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Teachman BA, Marker CD, Smith-Janik SB. Automatic associations and panic disorder: Trajectories of change over the course of treatment. Journal of Consulting and Clinical Psychology. 2008;76(6):988–1002. doi: 10.1037/a0013113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Vannoy S, Gable S, Brodt M, Cadet M, Andrews B, Maloney M. An application of eye tracking technology to detect attention bias for suicide related stimuli 2016 [Google Scholar]
  41. Werntz AJ, Steinman SA, Glenn JJ, Nock MK, Teachman BA. Characterizing implicit mental health associations across clinical domains. Journal of Behavior Therapy and Experimental Psychiatry. 2016;52:17–28. doi: 10.1016/j.jbtep.2016.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES