Assessing Suicide Risk Among Callers to Crisis Hotlines: A Confirmatory Factor Analysis

Abstract

Our goal was to investigate the factor structure of a risk assessment tool utilized by suicide hotlines and to determine the predictive validity of the obtained factors in predicting subsequent suicidal behavior. 1,085 suicidal callers to crisis hotlines were divided into three sub-samples, which allowed us to conduct an independent Exploratory Factor Analysis (EFA), EFA in a Confirmatory Factor Analysis (EFA/CFA) framework, and CFA. Similar to previous factor analytic studies (Beck et al., 1997; Holden & DeLisle, 2005; Joiner, Rudd, & Rajab, 1997; Witte et al., 2006), we found consistent evidence for a two-factor solution, with one factor representing a more pernicious form of suicide risk (i.e., Resolved Plans and Preparations) and one factor representing more mild suicidal ideation (i.e., Suicidal Desire and Ideation). Using structural equation modeling techniques, we found preliminary evidence that the Resolved Plans and Preparations factor trended toward being more predictive of suicidal ideation than the Suicidal Desire and Ideation factor. This factor analytic study is the first longitudinal study of the obtained factors.

Fewer than one-third of the approximately 32,000 people who die by suicide in the United States each year (Centers for Disease Control and Prevention, 2004) were seen by a mental health provider in the year prior to death (Luoma, Martin, & Pearson, 2002). Thus, although psychotherapy and pharmacotherapy have value in treating suicidal behavior, the vast majority of potential suicide decedents will not be reached if we completely rely on these intervention measures. Public health intervention strategies offer a good alternative or conjunctive way of preventing and treating health conditions. There are two general categories of public health strategies. Universal strategies aim to prevent the onset of a disease or condition in the entire population, whereas high-risk prevention strategies only target individuals believed to be at higher risk than the general population (Rose, 1992). Although universal prevention strategies (e.g., limiting access to lethal means) have shown great promise in preventing suicide (Hawton et al., 2001; Kreitman & Platt, 1984), these types of strategies are often met with resistance in the United States, where the thought of limiting the individual freedoms of everyone for the sake of the relatively few goes against the American sensibility of individual liberty (Center for Consumer Freedom, 2007; Duberstein & Witte, 2009).

High-risk prevention strategies are less controversial than universal prevention strategies and can still be an effective complement to psychotherapeutic and psychopharmacological treatments for suicidal behavior. One type of high-risk prevention strategy that has a long history in the United States is the 24-hour suicide prevention hotline. Suicide hotlines are widely available, as nearly all major cities in the United States have them (Dew, Bromet, & Greenhouse, 1987). Virtually everyone, including people with financial and mobility restrictions, has free, easy access to this type of service.

It is not enough, however, to be non-controversial, widely available, and inexpensive – suicide prevention strategies must also be effective. Fortunately, several recent studies have provided promising empirical evidence for the effectiveness of suicide hotlines. Kalafat, Gould, Munfakh, and Kleinman (2007) examined the effectiveness of hotlines in non-suicidal callers and found significant reductions in negative affect (e.g., depression, anger, anxiety) and hopelessness from the beginning to the end of the calls. Gould, Kalafat, Munfakh, and Kleinman (2007) focused on suicidal callers. Similar to the non-suicidal callers, they exhibited significant reductions in negative affect and intent to die from the beginning to the end of the calls. However, intent to die did not significantly change between the end of the call and follow-up. Furthermore, intent to die at the end of the call (as assessed by the items How much do you really want to die? And How likely are you to carry out your thoughts/plans to kill yourself?) was a significant predictor of subsequent suicidal behavior.

Unlike traditional counseling sessions, in which suicidal individuals are physically in the presence of the mental health provider, callers to crisis hotlines may choose to terminate the call at any time without warning. Although thoroughness is important when conducting a suicide risk assessment, in this type of time-limited situation, it is perhaps even more important to appropriately prioritize the most crucial information so that it can be collected toward the beginning of the call. Information that is important, but not necessarily crucial, can be collected later on in the call if time permits. Thus, although Shea (2002) suggested that engaging the suicidal person and establishing rapport are necessary before being able to accurately assess suicide risk, it is crucial for phone counselors to obtain as much information as possible early in the call, should it end unexpectedly.

Although suicide hotlines provide confidential crisis counseling, it is very common for hotlines to have the capability of tracing calls and to use this capability in order to implement emergency services when callers are determined to be at high risk (Personal Communication, Thomas Joiner, Ph.D.; Chair of the National Suicide Prevention Lifeline Standards, Practice, and Training Subcommittee; January 17, 2010). Thus, determining which callers are at high risk for suicide and require this level of intervention is an important task, as it is crucial to strike a balance between honoring the privacy of callers and ensuring the safety of those who are at imminent risk for making a suicide attempt.

Although the presence of any form of suicidal ideation should be cause for concern, not all forms of it are equally pernicious. Given the time-limited nature of calls to suicide hotlines, it behooves crisis workers to assess for those aspects of suicidal ideation that are most pernicious as soon as is viable. Previous exploratory factor analytic studies (Beck, G. Brown, & Steer, 1997; Holden & DeLisle, 2005; Holden, Mendonca, & Mazmanian, 1985; Joiner, Rudd, & Rajab, 1997; Witte et al., 2006) have demonstrated that there are two facets, or factors, of suicidal ideation, as measured by the Scale for Suicide Ideation (Beck, Kovacs, & Weissman, 1979) and the Modified Scale for Suicide Ideation (Miller, Norman, Bishop, & Dow, 1986). Specifically, thoughts that are related to specific plans and intent for suicide have been designated as Resolved Plans and Preparations (RPP) whereas thoughts related to desire for death, frequency of suicidal ideation, and lacking deterrents for suicide have been designated as Suicidal Desire and Ideation (SDI).

Three of the aforementioned factor-analytic studies have examined the two factors and their relationship with past suicidal behavior. Holden and DeLisle (2005) found that the relationship between the RPP factor and past suicide attempts was stronger than the relationship between SDI and past suicide attempts. Similarly, Joiner et al. (1997) found that the RPP factor was significantly more predictive of attempter versus ideator history than the SDI Factor. They suggested that this indicates that RPP is more pernicious than SDI. In contrast, Witte et al. (2006) did not find a relationship between past attempts and RPP in their sample of suicidal older adults. Although all three studies demonstrated similar factor structure, none of them examined whether the factors actually predict subsequent suicidal thoughts or behaviors. Furthermore, the evidence that RPP is more pernicious than SDI was somewhat inconsistent.

In the previous factor analytic studies, the researchers were focused on determining the factor structure of suicidal ideation (i.e. thoughts about suicide). As such, they did not include an indicator of past suicide attempts in their factor analyses. In the current study, we focused on the factor structure of current suicide risk, which includes but is not limited to suicidal ideation. Past suicidal behavior is one of the strongest predictors of future suicidal ideation (Joiner et al., 2005), subsequent suicide attempts (Maser et al., 2002; Putnins, 2005), and death by suicide (G. Brown, Beck, Steer, & Grisham, 2000; Maser et al., 2002; Tidemalm, Elofsson, Stefansson, Waern, & Runeson, 2005; Zonda, 2006). Accordingly, empirically informed suicide risk assessment frameworks recommend assessing for past suicidal behavior as part of a comprehensive suicide risk assessment (e.g., Joiner, Van Orden, Witte, & Rudd, 2009). Thus, in the current study, we included past suicidal behavior as an indicator in our factor analyses. Hopelessness is an additional predictor of suicidal ideation (Beck, Steer, Beck, & Newman, 1993), and both fatal and non-fatal suicide attempts (McMillan, Gilbody, Beresford, & Neilly, 2007). As such, it was also included in our risk assessment approach.

Despite the fact that there have been several studies that have examined the factor structure of suicidal ideation and have determined there to be two factors, there are a number of issues that have not been addressed in prior work. The current study addresses these gaps in the literature. First, we utilized past suicide attempts in our risk assessment measure rather than using them as a proxy for subsequent risk for suicide. Second, our key criterion variables were suicidal thoughts and suicide attempts at follow-up; to our knowledge, this study thus represents the first longitudinal examination of the factor structure of suicide risk. Third, as will become evident, our factor-analytic approach possesses advantages as compared to past approaches. Fourth, our sample consists of a large sample of callers to crisis hotlines – this is the first time, to our knowledge, that the factor structure of suicide risk has been studied in this type of sample.

We predicted that there would be a similar factor structure to that found in prior studies (i.e., there would be two factors, one representing Suicidal Desire and Ideation and one representing Resolved Plans and Preparations). We also planned to determine if these factors differentially predicted subsequent suicidal thoughts and suicide attempts with the use of structural equation modeling (SEM) techniques. We predicted that the RPP factor would more strongly predict subsequent suicidal behavior than the SDI factor. Confirmation of this hypothesis would imply that the items comprising the RPP factor should be assessed earlier in the call.

Method

This project’s protocol was approved by the Institutional Review Boards of New York State Psychiatric Institute/Columbia University and Rutgers Graduate School of Applied and Professional Psychology.

Participants

Baseline Cohort

Our sample (N = 1,085) is identical to the sample described by Gould et al. (2007), though our study aims were distinct from Gould et al. (2007). We examined suicidal, adult callers to one of eight crisis lines between March 2003 and July 2004. The majority (72%; n = 781) of the callers placed their calls to local crisis suicide hotline numbers, whereas the remainder called 1–800-SUICIDE, a national network of crisis centers. Sixty percent (n = 651) of the callers were female, and they ranged in age from 18–72 (Mean = 36.1 years). Sixty-six percent (n = 716) were Caucasian, 15% (n = 163) were African American, 10% (n = 108) were Hispanic, 4% (n = 43) were Native American, 3% (n = 33) were Asian, 2% (n = 22) were other, and six callers chose not to provide information about ethnicity.

Some suicidal callers (n = 654) were not assessed because they were believed by the crisis workers, using their own clinical judgment, to be at a risk level that was too high, and as such, the crisis workers opted to attempt to reduce suicidal symptoms and/or initiate rescue procedures rather than conduct the standardized risk assessment. Therefore, some of the callers who were the most intensely suicidal were not included in our study, making our sample less severe overall. There were also some callers in addition to those 654 who were not assessed at all because of high call volume and other similar practical factors; it is therefore impossible to know whether they were suicidal or not. If we assume that none of the un-assessed callers were suicidal, our participation rate among suicidal callers is 62.4% (1,085 out 1,739). If some of the un-assessed callers were suicidal, our participation rate would be lower than that.

Follow-up Cohort

Follow-up assessments were conducted with 380 (35%) of the eligible callers who completed the initial assessment. Of those who did not complete a follow-up assessment, 44% (n = 310) refused follow-up initially, 39% (n = 275) were not asked if they would permit follow-up, 9% (n = 64) initially agreed to follow-up but subsequently refused when re-contacted, and 8% (n = 56) were unable to be reached because they had given incorrect contact information to the initial crisis worker. On average, the follow-up occurred approximately two weeks after the initial call, with a range of 1 to 48 days between the two calls. By the end of the first week, 22% (n = 87) of the follow-ups had occurred; by the end of the second week, 58% (n = 220) of the follow-ups had occurred; by the end of the third week, 82% (n = 312) of the follow-ups had occurred; and by end of the fourth week, 93% (n = 353) of the follow-ups had occurred. This range in follow-up time occurred because callers were not always available during the initial follow-up call.

Comparisons on the study variables between those with and without follow-up data revealed several differences. Those with follow-up data were significantly more likely to be female than those without follow-up data (69.7% vs. 55.6%, χ² = 19.98, df = 1, p <.001). They were also significantly less likely to have hurt themselves before calling (5.5% vs. 9.7%, χ² = 5.66, df = 1, p = .02) and were significantly less likely to believe that suicide is the only option to solve their problems (25.7% vs. 34.4%, χ² = 8.48, df = 1, p = .004) than those without follow-up data. Those with follow-up data had significantly lower scores on how much they really wanted to die (F [1, 1066] = 8.0, p = .01), their self-reported likelihood of killing themselves (F [1, 1019] = 22.05, p = .00), the degree to which they felt hopeless about the future (F [1, 1021] = 6.87, p = .01), and the degree to which they believed they cannot go on (F [1, 992] = 16.78, p < .0001). The only variable for which those with follow-up data appeared to be more severe than those without was in their likelihood of past suicide attempts (64.0% vs. 55.7%, χ² = 6.84, df = 1, p = .01). In sum, the follow-up sample seems to be less clinically severe than the sample without follow-up data.

Measures

Suicide Risk Status

This measure is described in Gould et al. (2007). Briefly, it was influenced by empirical and clinical work describing suicide risk assessments and risk factors for suicide (i.e., Chiles & Strosahl, 1995; Goldsmith, Pellmar, Kleinman, & Bunney, 2002; Joiner et al., 1997; Joiner, Walker, Rudd, & Jobes, 1999). Crisis center directors and counselors also provided feedback on the practical feasibility of the measure. The final Suicide Risk Status measure consisted of 12 items that were designed to measure psychological pain, hopelessness, suicidal ideation, suicidal behavior, and intent to die. Some of the items were binary and some were on a likert scale from 1 to 5. See Tables 1 and 2 for the items included in this measure. Internal consistency in our sample was adequate (Cronbach’s α = .80).

Table 1.

Descriptive statistics for binary variables.

	Sample 1 (N = 355)	Sample 2 (N = 350)	Sample 3 (N = 380)
How much of each day do you think about suicide?	Fleeting thoughts (41%)	Fleeting thoughts (40%)	Fleeting thoughts (45%)
	Persistent thoughts (55%)	Persistent thoughts (57%)	Persistent thoughts (53%)
	Missing (4%)	Missing (3%)	Missing (2%)
Are you able to control your suicidal thoughts?	Yes (61%); No (32%)	Yes (58%); No (35%)	Yes (65%); No (30%)
	Missing (7%)	Missing (7%)	Missing (5%)
Did you do something to hurt yourself now or before you called?	No (87%); Yes (10%)	No (90%); Yes (9%)	No (94%); Yes (6%)
	Missing (3%)	Missing (1%)	Missing (0.3%)
Have you made any plans to hurt or kill yourself?	No (44%); Yes (53%)	No (39%); Yes (57%)	No (46%); Yes (52%)
	Missing (3%)	Missing (4%)	Missing (2%)
Do you believe there are other ways to solve your problems besides suicide?	Yes (63%); No (33%)	Yes (61%); No (32%)	Yes (72%); No (25%)
	Missing (5%)	Missing (7%)	Missing (3%)
Have you ever attempted suicide in your lifetime?	No (43%); Yes (54%)	No (44%); Yes (55%)	No (36%); Yes (63%)
	Missing (3%)	Missing (2%)	Missing (1%)

Table 2.

Descriptive statistics for continuous variables (All on a likert scale from 1–5)

	Sample 1 (N = 355)	Sample 2 (N = 350)	Sample 3 (N = 380)
When you think about killing yourself, how much do you really want to die?	M = 2.98 (SD = 1.26), Median = 3, missing = 1%	M = 3.11 (SD = 1.22), Median = 3, missing = 2%	M = 2.82 (SD = 1.18), Median = 3, missing = 1%
How likely are you to carry out your thoughts or plans?	M = 2.80 (SD = 1.27), Median = 3, missing = 7%	M = 2.91 (SD = 1.28), Median = 3, missing = 9%	M = 2.48 (SD = 1.12), Median = 3, missing = 3%
How much hurt, anguish, or misery are you feeling right now?	M = 4.22 (SD = 0.99), Median = 5, missing = 6%	M = 4.30 (SD = 0.96), Median = 5, missing = 8%	M = 4.29 (SD = 0.96), Median = 5, missing = 2%
If your current situation didn’t change, could you tolerate the way you feel?	M = 3.95 (SD = 1.18), Median = 4, missing = 6%	M = 4.06 (SD = 1.13), Median = 4, missing = 9%	M = 3.87 (SD = 1.16), Median = 4, missing = 2%
To what degree do you feel that there is no hope for improvement in your situation?	M = 3.66 (SD = 1.18), Median = 4, missing = 8%	M = 3.76 (SD = 1.13), Median = 4, missing = 9%	M = 3.51 (SD = 1.18), Median = 4, missing = 1%
To what extent does the belief I don’t think I can go on describe how you are feeling right now?	M = 3.27 (SD = 1.18), Median = 3, missing = 10%	M = 3.41 (SD = 1.19), Median = 3, missing = 12%	M = 3.03 (SD = 1.15), Median = 3, missing = 3%

Follow-Up Assessment

At follow-up, each caller was asked the following: I need to ask you whether you have had any thoughts about killing yourself since you called the crisis line. This was a binary, observed variable; 43% (n = 163) acknowledged experiencing such thoughts. Callers were also asked, Since you called the crisis center, have you tried to kill yourself?; 3% of our sample had (n = 11).

Procedure

The Institutional Review Boards that reviewed the study protocol granted a waiver of consent for use of de-identified information that was collected at the baseline assessment because the procedure did not involve greater than minimal risk and was similar to risk assessments that are typically administered by phone counselors at suicide hotlines. Callers to the hotlines were not offered compensation for their participation in the baseline assessment because this was included as part of the intervention normally provided to callers; however, the callers were given a $25 money voucher for the follow-up assessment. The baseline assessments were conducted by phone counselors at the crisis centers. The follow-up assessments were conducted by paid members of the project staff and not crisis center staff in order to ensure independent assessments. They had either telephone crisis counseling experience or equivalent clinical training and experience.

Phone counselors used an IRB-approved re-contact consent script to ask all callers who completed the baseline assessment if they were willing to receive a follow-up call one to two weeks after the initial call. Steps were taken to protect the confidentiality of the callers, as they were allowed to choose when and how they wished to be contacted and whether the counselor could leave a message on an answering machine or with the person answering the telephone.

Analytic Strategy

We were interested in determining the underlying factor structure of the suicide risk status questionnaire and ultimately, to determine whether certain latent factors were more predictive of subsequent suicidal behavior than others. To accomplish this aim, we divided our sample of 1,085 participants into three sub-samples. We first separated out the 705 participants who had not completed the follow-up and randomly divided them into two sub-samples. Sample 1 (N = 355) was used to conduct an exploratory factor analysis (EFA). Sample 2 (N = 350) was used to conduct an EFA in a confirmatory factor analysis (CFA) framework (EFA/CFA). EFA/CFA was first described by Joreskog (1969), but has been relatively under-utilized. To conduct an EFA/CFA, one must run a CFA with all items loading on to all factors while applying m² (m = the number of latent factors) restrictions, which makes it equivalent to an EFA model. This is accomplished by fixing the factor variance to 1 for all factors and by choosing anchor items for each factor and fixing their loadings to 0 for the factors onto which they load weakly (T. A. Brown, 2006; Muthen & Muthen, 2008). EFA/CFA can be considered an intermediate step between the initial EFA and subsequent CFA’s, as it has two distinct advantages over EFA. First, EFA/CFA provides significance tests for the factor loadings that indicate which items, if any, significantly load onto more than one factor. Second, EFA/CFA provides modification indices, which indicate possible areas of strain in the model that can be adjusted before moving on to a confirmatory factor analysis. Sample 3 (n = 380) was utilized to conduct a CFA on the suicide risk status questionnaire and also to model the relationship between the latent factors and subsequent suicidal behavior (i.e., ideation, suicide attempts) measured at follow-up in a structural equation model (SEM) framework.

Some of the items on the suicide risk questionnaire were continuous, whereas others were binary. As such, it was necessary to utilize an estimation procedure that would accommodate categorical, non-normal data, as the use of standard maximum likelihood estimation procedures with this type of data can be problematic. According to T. A. Brown (2006), the Mplus software package offers the best options to conduct factor analyses with categorical, non-normal data, as it is currently the only statistical software package that offers the weighted least squares mean- and variance-adjusted (WLSMV) estimator. WLSMV uses a diagonal weight matrix to provide weighted least square parameter estimates, robust standard errors, and a mean- and variance-adjusted χ² statistic (Muthen & Muthen, 1998–2007). This estimator has been recommended for models that involve binary indicators (e.g., T. A. Brown, 2006; Flora & Curran, 2004). Because of this, we used Mplus version 5.1 for our analyses.

Missing Data

The covariance coverage matrix, which indicates the proportion of cases with complete data for each pair-wise combination of variables, ranged in magnitude from .83–.99. Missing data were handled using pair-wise deletion for all of our analyses, which is the default method of handling missing data in Mplus when using the WLSMV estimator. Although pair-wise deletion can result in bias when data are missing at random (MAR; T. A. Brown, 2006), Mplus does not provide modification indices for models estimated using multiple imputation. Thus, it would have been difficult to modify subsequent models without this information.

Evaluating Model Fit

For all models, we used a variety of indices to evaluate model fit. We utilized χ², weighted root-mean-square residual (WRMR; Muthen & Muthen 1998–2007), Root Mean Square Error of Approximation (RMSEA; Steiger, 1990), Comparative Fit Index (CFI; Bentler, 1990), and Tucker Lewis Index (TLI; Tucker & Lewis, 1973). We used the following cutoff values for our fit indices, based upon the results of various Monte Carlo studies: χ² p value > 0.05 (Yu, 2002), WRMR < 1.0 (Yu, 2002), RMSEA close to < 0.06 (Hu & Bentler, 1999), and CFI and TLI close to > .95 (Hu & Bentler, 1999).

Results

Preliminary Results

Descriptive statistics for the binary variables can be found in Table 1 for all three samples; descriptive statistics for the continuous variables can be found in Table 2. Univariate outliers were defined as scores that were plus or minus three standard deviations from the mean; there were no such outliers with any of the continuous variables. We examined scatter-plots of bivariate correlations between variables and determined that there were no bivariate outliers.

Exploratory Factor Analysis

First, we conducted an EFA with Sample 1 (N = 355) using the WLSMV estimator and pair-wise deletion. We opted to use an oblique rotation (i. e. promax), as we anticipated the factors to be correlated with one another. However, we ran the analyses using two other types of oblique rotations (i. e. geomin and quartimin) and obtained nearly identical results. See Table 3 for bivariate correlations between all study variables in Sample 1.

Table 3.

Bivariate correlations for all variables in Sample 1 (N = 355)

	1	2	3	4	5	6	7	8	9	10	11	12
1. How much do you think about suicide?	1
2. Able to control your suicidal thoughts?	.31**	1
3. Done something to hurt self now or before called?	.10	.16**	1
4. Made plans to hurt/kill self?	.27**	.23**	.31**	1
5. Ever tried to kill self?	.18**	.13*	.21**	.18**	1
6. How much do you want to die?	.31**	.24**	.12*	.29**	.16**	1
7. Other ways to solve problems other than suicide?	.20**	.32**	.24**	.28**	.11	.47**	1
8. How likely are you to kill self?	.33**	.34**	.24**	.37**	.11*	.54**	.52**	1
9. Degree feels no hope for improvement in future?	.24**	.32**	.04	.14*	.04	.33**	.46**	.40**	1
10. Degree of belief that you can’t go on?	.31**	.26**	.12*	.20**	.06	.41**	.41**	.58**	.52**	1
11. Amount of psychological pain	.24**	.24**	.05	.22**	−.02	.28**	.26**	.41**	.41**	.45**	1
12. How much could you tolerate if situation didn’t change?	.25**	.26**	.07	.28**	.06	.33**	.36**	.42**	.39**	.47**	.51**	1

Note.

^*p < 0.05;

^**p < 0.01

To interpret the EFA, we used the strategy recommended by Muthen and Muthen (2008). First, we examined the eigenvalues and found that only the one-factor and two-factor solutions had eigenvalues greater than one (4.99 and 1.70, respectively). We then examined the fit indices for these two solutions and found that the two-factor solution had a more favorable fit to the data (χ² = 58.47, df = 29, p = 0.002; RMSEA = 0.05) than the one-factor solution (χ² = 126.10, df= 33, p < .0001; RMSEA = .09). Furthermore, examination of the factor loadings revealed that the loadings for the two-factor solution were similar to those that have been found using other measures of suicidal ideation (Beck et al., 1997; Joiner et al., 1997; Holden & DeLisle, 2005; Witte et al., 2006). As seen in Table 4, the items loading onto factor 1 appeared consistent with the RPP factor discussed previously, whereas the items loading onto factor 2 seemed to measure SDI. The correlation between the RPP and SDI factor was moderate (0.44) and indicated that (as expected) these are two related but separable constructs. This correlation is highly similar to the factor correlations found by previous factor analytic studies (Beck et al., 1997; Holden & DeLisle, 2005; Witte et al., 2006).

Table 4.

Latent Factor Structure of the Suicide Risk Status Questionnaire in Samples 1, 2, and 3.

	EFA		EFA/CFA		CFA
	Sample 1 (N = 355)		Sample 2 (N = 350)		Sample 3 (N = 380)
Item	Factor 1	Factor 2	Factor 1	Factor 2	Factor 1	Factor 2
Done anything to hurt self before calling?	0.92	−0.10	0.87**	0.00	0.61*
Made any specific plans to hurt/kill self?	0.61	0.21	0.51**	0.32*	0.89**
Ever in your whole life tried to kill self?	0.54	−0.09	0.31*	0.02	0.23*
To what extent does the belief I don’t think I can go on describe how you are feeling right now?	−0.10	0.75	0.00	0.75**		0.62**
No hope for improvement in your situation	−0.19	0.75	−0.10	0.77**		0.49**
Any other ways to solve problems other than suicide?	0.19	0.72	−0.09	0.91**		0.64**
How much hurt, anguish, or misery are you feeling?	−0.16	0.68	0.31**	0.37**		0.46**
If your current situation didn’t change, could you tolerate the way you feel?	−0.06	0.68	0.22*	0.57**		0.47**
How likely are you to kill yourself?	0.19	0.64	0.02	0.98**	0.72**
How much do you really want to die?	0.17	0.50	−0.07	0.88**	0.57**
Can you control your thoughts about killing yourself?	0.22	0.49	0.27*	0.49**	0.39**
How much of each day do you think about suicide?	0.27	0.41	0.15	0.47**	0.50**

Note. EFA = Exploratory Factor Analysis; CFA = Confirmatory Factor Analysis; EFA/CFA = Exploratory Factor Analysis in a Confirmatory Factor Analysis Framework; WLSMV estimator and pairwise deletion; factor loadings ≥ 0.30 are in bold.

^*p < 0.05;

^**p < 0.01

Exploratory Factor Analysis in a Confirmatory Factor Analytic Framework (EFA/CFA)

Next, we conducted an EFA/CFA to test the fit of the two-factor model in an independent sample, Sample 2 (N = 350). See Table 5 for bivariate correlations between all study variables in Sample 2. The two-factor model appeared to fit the data adequately in Sample 1; thus, we tested a two-factor model in our EFA/CFA, again using the WLSMV estimator and pair-wise deletion for missing data. We chose our anchor items based upon their performance in Sample 1. For Factor 1 (RPP), the item with the strongest loading was Have you done anything to hurt yourself before you called us, or right now; its factor loading was thus fixed to zero for Factor 2 (SDI). The anchor item for the SDI factor was To what extent does the belief “I don’t think I can go on” describe how you are feeling right now; its loading was fixed to zero for RPP. The factor variances were both fixed to one. The two-factor model also provided an adequate fit to the data in this sample (χ² = 59.14, df= 28, p < 0.001; RMSEA = .06; CFI = 0.92; TLI = 0.96; WRMR = 0.66). Examination of the individual items revealed that the factor loadings were relatively consistent across samples. In general, the items that had appeared to have primary loadings on the various factors in Sample 1 had the same primary loadings in Sample 2 (and these loadings were statistically significant). The correlation between the SDI and RPP factor was 0.46 (p < 0.05), which is similar to what we found in Sample 1.

Table 5.

Bivariate correlations for all variables in Sample 2 (N = 350)

	1	2	3	4	5	6	7	8	9	10	11	12
1. How much do you think about suicide?	1
2. Able to control your suicidal thoughts?	.29**	1
3. Done something to hurt self now or before called?	.10	.23**
4. Made plans to hurt/kill self?	.25**	.24**	.26**	1
5. Ever tried to kill self?	.09	.09	.10	.15**	1
6. How much do you want to die?	.31**	.37**	.16**	.27**	.07	1
7. Other ways to solve problems other than suicide?	.29**	.33**	.17**	.29**	−.02	.49**	1
8. How likely are you to kill self?	.32**	.38**	.17**	.46**	.09	.60**	.56**	1
9. Degree feels no hope for improvement in future?	.31**	.28**	.07	.23**	.12*	.43*	.45**	.46**	1
10. Degree of belief that you can’t go on?	.20**	.31**	.12*	.25**	.15*	.42**	.43**	.48**	.48**	1
11. Amount of psychological pain	.27**	.26**	.13*	.30**	.14*	.31**	.29**	.37**	.35**	.36**	1
12. How much could you tolerate if situation didn’t change?	.19**	.27**	.10	.33**	.11	.35**	.39**	.40*	.35**	.48**	.59**	1

Note.

^*p < 0.05;

^**p < 0.01

Two exceptions are worth mentioning, however. First, the item Have you made any specific plans to hurt yourself? had statistically significant loadings on both the RPP and SDI factors in Sample 2, whereas in the EFA it had appeared to load more clearly on the RPP Factor. Nevertheless, in Sample 2, this item’s primary loading still appeared to be on the RPP factor. Furthermore, in previous factor analytic studies on measures of suicidal ideation (Beck et al., 1997; Holden & DeLisle, 2005; Joiner et al., 1997; Witte et al., 2006), a similar item assessing specificity of planning clearly loaded onto the RPP Factor. Thus, in the subsequent CFA, we began under the assumption that this item would have its primary loading on the RPP Factor, keeping in mind that this might be an area in need of modification if the CFA model had a poor fit. Another exception was found with the item assessing how much pain, hurt, or misery was being experienced by the callers. This item had clearly loaded on the SDI Factor in Sample 1 (and had a near 0 loading on RPP), but in Sample 2 the loadings were approximately equal (although the loading for SDI was still higher in magnitude). Again, prior factor analytic studies (Joiner et al., 1997; Witte et al., 2006) have shown that the SDI factor is differentially correlated with measures of depression and hopelessness. Although this item is arguably more similar to Shneidman’s (1999) psychache (i. e. psychological pain that is unbearable) than it is to depression and hopelessness, psychache has been shown to be highly correlated with measures of depression and hopelessness (e. g., Mills, Green, & Reddon, 2005) and would reasonably be expected to have a similar relationship with SDI. Thus, in the subsequent CFA, we proposed that this item loaded only onto the SDI factor. In sum, these two items did not perform entirely consistently across the two samples. In both cases, their performance in Sample 1 was consistent with prior research and theory (and was more parsimonious than the solution suggested in Sample 2); thus, when constructing the final CFA model, we began with the assumption that the items’ performance in Sample 1 was most consistent with prior work and constructed the model accordingly.

Although the current model suggested an adequate fit to the data, there was one large MI (MI = 19.19) suggesting that the items, If your current situation didn’t change, could you tolerate the way you feel? and How much hurt, anguish, or misery are you feeling right now? may have correlated residual variance. We kept this in mind when conducting the CFA.

Confirmatory Factor Analysis

Based upon the results of the EFA, EFA/CFA, and prior work (Beck et al., 1997; Holden & DeLisle, 2005; Joiner et al., 1997; Witte et al., 2006), we conducted a CFA (WLSMV estimator; pair-wise deletion) using a third sample (N = 380; see Table 4). See Table 6 for bivariate correlations between all variables in Study 3. The proposed model was congeneric; that is, we did not allow any cross-loadings of the indicators. Again, the fit indices suggested adequate fit (χ² = 60.27, df= 36, p = .01; RMSEA = .04; CFI = 0.93; TLI = 0.95; WRMR = 0.81). With regard to the χ² value possibly being indicative of poor model fit, we used a procedure recommended by L. Muthen (2008, personal communication) to determine if the χ² test was simply too sensitive due to relatively large sample size. We freed up parameters until the χ² reached an acceptable value (χ² = 41.61, df = 35, p = .21). Then, we compared the loadings of this model to the loadings of the more restricted model. They were highly similar, indicating that the poor χ² values were likely due to sensitivity from large sample size rather than poor model fit. Because the EFA and EFA/CFA were saturated, we were unable to use this technique (i.e. add other paths). Nevertheless, it seems reasonable to assume the same to be true of the prior models (i.e., the χ² values were inflated because of sample size), given the similarity in sample size among the three models.

Table 6.

Bivariate correlations for all variables in Sample 3 (N = 380)

	1	2	3	4	5	6	7	8	9	10	11	12	13	14
1. How much do you think about suicide?	1
2. Able to control your suicidal thoughts?	.16**	1
3. Done something to hurt self now or before called?	.06	.12*	1
4. Made plans to hurt/kill self?	.25**	.16**	.23**	1
5. Ever tried to kill self?	.00	.09	.01	.06	1
6. How much do you want to die?	.21**	.11*	.04	.26**	.14**	1
7. Ways to solve problems other than suicide?	.15**	.08	.04	.12*	.09	.37**	1
8. How likely are you to kill self?	.25**	.21**	.11*	.24**	.13*	.43**	.39**	1
9. Degree feels no hope for improvement in future?	.21**	.16**	.06	.15**	.06	.30**	.28**	.29**	1
10. Degree of belief that you can’t go on?	.19**	.18**	.09	.22**	.04	.37**	.29**	.48**	.28**	1
11. Amount of psychological pain	.22**	.15**	.00	.13*	.08	.24**	.17**	.34**	.25**	.30**	1
12. How much could you tolerate if situation didn’t change?	.22**	.15**	.13*	.19**	.04	.16**	.17**	.38**	.26**	.32**	.39**	1
13. Thought about suicide (follow-up)	.12*	.07	.00	.11*	.08	.18**	−.07	.01	.12*	.07	.00	.04	1
14. Attempted suicide (follow-up)	−.06	−.05	.03	.04	.00	−.04	−.02	.11*	.03	−.02	−.07	.00	.20**	1

Note.

^*p < 0.05;

^**p < 0.01

Although both our EFA and EFA/CFA demonstrated compelling evidence that the two-factor solution had a favorable fit to the data, we were interested in empirically comparing the two-factor solution to a one-factor solution. To do this, we fixed the covariance between the RPP and SDI latent factors to 1, thereby creating a model that is empirically identical to a one-factor solution. We then compared this nested model to our two-factor solution, which freely estimates the covariance between these factors. The χ² difference test between our models (χ² = 13.52, df = 1, p < 0.001), using the procedure described by T. A. Brown (2006) and Muthen and Muthen (1998–2007) for use with the WLSMV estimator, suggested that our model fit was significantly worsened by constraining the covariance between the latent factors to 1. In other words, the fit of the two-factor solution is superior to that of the one-factor solution.

Our next task was to look for consistencies between the MI’s of the CFA and the EFA/CFA. The only MI that was consistent across Samples 2 and 3 indicated that we should allow the residuals for the items, If your current situation didn’t change, could you tolerate the way you feel? and How much hurt, anguish, or misery are you feeling right now? to correlate with one another. The χ² difference test (T. A. Brown, 2006; Muthen & Muthen, 1998–2007) between our models (χ² = 18.6, df = 1, p < 0.001 suggested that our model fit was degraded by not allowing these residual variances to correlate (r = 0.22, p < 0.001). The other fit indices for this modified model were also more favorable than the original model (χ² = 51.67, df= 35, p = .03; RMSEA = .04; CFI = 0.95; TLI = 0.97; WRMR = 0.75). Given that this MI appeared consistently across two samples and that both items are related in some way to emotional pain being experienced, we have empirical and conceptual justification for this. Therefore, in all subsequent analyses and discussion, we refer to this modified model.

Next, we examined the individual items in the hopes of resolving the two major discrepancies between Sample 1 and Sample 2. Regarding the item, Have you made any specific plans to hurt yourself? the modification indices for Sample 3 did not indicate that model fit would have been significantly improved by allowing this item to cross-load on both the SDI and RPP factors. This is consistent with the results found in the initial EFA (and prior research), suggesting that this item is a solid indicator of the RPP factor. Similarly, the item assessing how much pain, hurt, or misery was being experienced by the callers loaded moderately onto the SDI factor in Sample 3, and the modification indices did not indicate that fit would have been significantly improved by allowing this item to cross-load onto the RPP factor.

Examination of the individual items revealed similar loadings to previous models (see Table 4). All of the loadings for each factor were statistically significant at the p < 0.05 level. It is important to point out, however, that the item assessing whether the caller had ever previously attempted suicide had a somewhat weak loading onto the RPP factor (0.23, p < 0.05). Its loading was higher in both the EFA and EFA/CFA. Still, it does have a statistically significant loading. Furthermore, although previous suicide attempts are crucial indicators of risk for serious suicidal behavior, it seems reasonable that this somewhat distal risk factor would have a smaller loading than more proximal risk factors (e.g., current plans for suicide).

In sum, the results of the EFA, EFA/CFA, and CFA provide support for the proposed measurement model. Our results demonstrated two separate but inter-related factors (i.e., RPP, SDI; r = 0.53) with congeneric indicators and one residual covariance between the indicators.

Structural Equation Models (SEM)

Our next aim was to determine if the latent factors differentially predicted subsequent suicidal ideation and suicide attempts. This model is depicted in Figure 1. To do this, we regressed both observed, dichotomous variables assessing whether the participant had experienced suicidal ideation or suicide attempt since the initial call onto the latent variables (i.e. RPP and SDI), using theta parameterization. We also estimated the path between suicidal ideation at follow-up and suicide attempt at follow-up, since it makes logical and empirical sense to do so. Again, we used the WLSMV estimator in our analysis. The fit indices suggested an adequate fit to the data (χ² = 69.13, df= 43, p = .01; RMSEA = .04; CFI = 0.93; TLI = 0.95; WRMR = 0.83).

Figure 1.

Sample 3 (N = 380) Structural Equations Model with standardized loadings predicting suicidal ideation and suicide attempts at Time 2.

As expected, suicidal ideation at follow-up significantly predicted suicide attempt at follow-up (standardized loading of .59, p = .00). Contrary to expectations, neither the RPP nor the SDI factor significantly predicted subsequent suicidal ideation (standardized loadings of 0.14 and 0.07 respectively, p’s > .20) or suicide attempts (standardized loadings of .12 and −.16 respectively, p’s > .20). In this model, the predictor variables accounted for 3.5% of the variance in suicidal ideation at follow-up and 36.0% of the variance in suicide attempts at follow-up. Although neither RPP nor SDI were significant predictors of our dependent variables, examination of the loadings reveals that there is a trend toward RPP being a stronger predictor than SDI of suicidal ideation at follow-up. Similarly, the relationship between SDI and suicide attempts at follow-up trends toward negative, whereas there is a trend toward a positive relationship for RPP. Of course, these trends should be viewed with caution given the lack of statistical significance of these findings.

Given the relatively high intercorrelation between our latent factors, and the potential for multicollinearity impacting the estimation of the paths between individual predictors and our outcome variables, we were interested in determining whether the latent factors predicted the follow-up variables individually. Thus, we ran two additional SEM models. In the first, we simply deleted the paths between SDI and the follow-up variables so that we could examine the relationship between these variables and RPP. The fit indices for this model suggested an adequate fit to the data (χ² = 66.64, df= 43, p = .01; RMSEA = .04; CFI = 0.94; TLI = 0.95; WRMR = 0.84). We found a significant relationship between RPP and suicidal ideation at follow-up (standardized loading of .23, p = .01) but not between RPP and suicide attempt at follow-up (standardized loading of −.08, p = .70). This model accounted for 5.0% of the variance in the suicidal ideation follow-up variable, which is greater than the variance accounted for when both RPP and SDI were predictors. The model accounted for 34.5% of the variance in the suicide attempt follow-up variable, which is slightly less than the variance accounted for in the previous model.

Next, we ran a model with only SDI predicting the follow-up variables. This model also had an adequate fit to the data (χ² = 68.61, df= 43, p = .01; RMSEA = .04; CFI = 0.93; TLI = 0.95; WRMR = 0.84). SDI was a significant predictor of suicidal ideation at follow-up (standardized loading of .16, p = .03), although this relationship was slightly weaker than the relationship between RPP and suicidal ideation at follow-up. The relationship between SDI and suicide attempt at follow-up was not statistically significant (standardized loading of −.09, p = .59). This model accounted for 2.5% of the variance in suicidal ideation at follow-up, which is less than both of the previous models. It accounted for 35.0% of the variance in suicide attempt, which is similar to both previous models.

In sum, there was a very strong relationship between suicidal ideation and suicide attempt at follow-up. Neither of our factors significantly predicted subsequent suicide attempts when examined jointly or separately (not too surprisingly given the power implications of the dependent variable, attempts, which less than 3% of the sample endorsed). The story is different with regard to subsequent suicidal ideation. When examined jointly, neither factor was a significant predictor. Examined separately, both factors were significant predictors of subsequent suicidal ideation. However, the magnitude of the relationship between RPP and suicidal ideation was larger than the magnitude of the relationship between SDI and suicidal ideation. Additionally, the amount of variance accounted for in suicidal ideation was larger when only RPP is used as a predictor than it is when both latent variables are included and when only SDI is a predictor. This pattern of results suggests that there is something common to both factors that predicts subsequent suicidal ideation; however, there is at least some reason to believe that RPP may be a more important predictor than SDI. Perhaps with more power, we would be able to detect a difference in the strength of the relationship between the factors and the dependent variables.

Discussion

In the current study, we utilized three independent, samples of suicide ideators and found consistent evidence for a two-factor model for suicidal ideation, with the first factor consisting of resolved plans and preparations for suicide (RPP) and the second factor consisting of suicidal desire and ideation (SDI). This is consistent with prior work (Beck et al., 1997; Holden & DeLisle, 2005; Joiner et al., 1997; Witte et al., 2006). Our study builds upon these prior studies in that a different measure of suicidal ideation was utilized, a different population was studied (i.e. suicidal individuals who had not necessarily presented in a clinical setting), and our factor-analytic approach had advantages over past approaches. The compelling results of the measurement model alone are an important contribution to the suicide risk assessment literature.

This study also represents the first longitudinal examination of the SDI and RPP factors in the prediction of suicidal thoughts and behavior. We found evidence that both factors predict subsequent suicidal ideation when entered singly into the model; we also found a trend toward RPP having a stronger relationship with subsequent suicidal thoughts than SDI did, as the model with only RPP predicting subsequent suicidal ideation accounted for more of the variance in subsequent suicidal ideation than when both RPP and SDI were included in the model. This is consistent with our hypothesis that RPP represents a more pernicious form of suicide risk than SDI. As stated above, this trend should be viewed with caution, as it did not reach statistical significance. Nevertheless, our results provide some evidence that future studies that address some of the limitations noted below (e.g., low power, relatively crude follow-up measures) could find a similar effect that is statistically significant. This study therefore is an important step in the development of scientifically-informed risk assessment models, as we have provided convincing evidence for the factor structure of suicide risk and a preliminary examination of its performance in longitudinal suicide prediction.

In contrast, neither of our factors predicted subsequent suicide attempts when entered into the model either singly or jointly. This could be due to a number of reasons. As previously mentioned, only 3% of our sample had attempted suicide at follow-up, which severely reduced the power we had to detect a relationship (Baldessarini, Finklestein, & Arana, 1988). Our prediction is that a study with greater power would be able to demonstrate a stronger relationship between RPP and suicide attempt. Aside from power issues, the relationship between suicidal ideation at follow-up and suicide attempt at follow-up was relatively strong and could have detracted from our ability to detect a relationship between our baseline measure and follow-up suicide attempts. Previous longitudinal research (e.g., Prinstein et al., 2008) has found an important relationship between changes in suicidal ideation levels and suicide attempts. Indeed, some (Borges et al., 2006; Kessler, Borges, & Walters, 1999; Prinstein et al., 2008) have suggested that the majority of studies examining risk factors for suicide attempts actually only predict suicidal ideation at follow-up and are not able to differentially predict suicide attempt at follow-up. Nevertheless, given the clear relationship in the literature and in the current study between suicidal ideation and suicide attempts, the ability to predict future suicidal ideation (and to intervene at this point) could help prevent future suicide attempts. Furthermore, in the model with both latent variables predicting subsequent suicidal behavior, the relationship between RPP and suicide attempts trended in the positive direction despite controlling for this powerful variable of subsequent suicidal ideation; in contrast, the relationship between SDI and suicide attempt trended toward negative. Again, this is tentative evidence that RPP is a more pernicious form of suicide risk than SDI.

When both latent variables were included in the model, neither latent factor predicted subsequent suicidal ideation or suicide attempts. There are several possible explanations for this unexpected result. First, we demonstrated that each factor did predict subsequent suicidal ideation when entered in to the model singly; thus, it seems as though there is something common to both factors that is predictive, and when they are entered into the model together, multicollinearity obscured the predictive power of the individual variables.

An additional explanation for our null results is that suicidal symptoms at Time 1 alone may not be sufficient to predict subsequent suicidal thoughts. Rather, they could represent a baseline level of suicide risk that only gets activated in the presence of a life stressor. For example, the RPP factor may have interacted with life stressors to predict subsequent suicidal behavior; unfortunately, data on life stressors between calls were not available. Finally, the RPP factor only had three indicators, one of which (past attempts) loaded rather weakly onto it. Because of this, we may not have had adequate coverage of this construct, which could limit its predictive ability. Ideally, we would have had additional items to ensure that this construct was measured reliably.

An additional point worth mentioning is that one could argue that the current study represents an examination of the stability of suicidal crises over time rather than an examination of the predictive validity of the SDI and RPP factors. That is, we have no way of knowing whether suicidal ideation experienced at follow-up represents a newly initiated suicidal crisis or a continuation of the symptoms experienced at baseline. If it is the case that our study is an examination of the stability of suicidal crises rather than an examination of the initiation of new crises, the fact that the relationship between these factors and subsequent suicidal thoughts and behaviors is relatively weak is somewhat surprising. However, prior studies have demonstrated that day-to-day variability (rather than stability) of suicidal ideation is associated with a past history of suicidal behavior (Witte et al., 2006). Given the relatively high proportion of individuals in our sample with past suicide attempts (i. e. a range of 54–63% across our subsamples), the apparent instability of suicidal symptoms in our sample over time is less surprising than at first glance. In addition, callers to the hotline received support from the phone counselor, which may have resulted in the resolution of their initial suicidal crisis and have weakened the relationship between initial and subsequent suicidal symptoms. Finally, the variable length of follow-up time could have weakened the relationship, although the majority of follow-up calls occurred within two weeks after the initial call. Regardless of whether our study represents an examination of the stability of suicidal crises over time versus the prediction of subsequent crises, we propose that the identification of risk assessment items that predict suicidal ideation beyond the baseline assessment is an important task.

It should be noted that in Gould et al. (2007), an intent to die scale created by expert consensus did significantly predict subsequent suicidal thoughts and behaviors. The items that were included in this scale loaded on to the SDI factor in the current study. It is interesting that this scale seemingly outperformed ones that were derived empirically in terms of being able to predict subsequent suicidal thoughts and behaviors. Future research in an independent sample should be conducted that compares the predictive validity of the two items included on the Gould et al. (2007) intent to die scale to those contained in SDI and RPP factors.

Our results have meaningful clinical implications. Given the converging evidence that suicide risk does have two factors, it is important that clinical risk assessments aim to cover both of these latent factors in order to provide a comprehensive risk assessment. Our results also have important theoretical implications. Despite the fact that differential perniciousness of these factors was not entirely confirmed in our study, prior empirical work is consistent with our measurement model of suicidal ideation (e.g., Beck et al., 1997; Holden & DeLisle, 2005; Joiner et al., 1997; Witte et al., 2006). Additionally, Joiner’s (2005) interpersonal theory of suicidal behavior suggests that desire for suicide, which is similar to the SDI factor, is a separate but related construct from acquired capability for suicide (i. e. fearlessness about enacting suicidal behavior and physical pain tolerance). As discussed by Van Orden et al. (in press), the RPP factor is similar to fearlessness about enacting suicidal behavior but does not necessarily indicate physical pain tolerance. Thus, RPP can be considered a sub-component of the acquired capability for suicide construct. According to the interpersonal theory, although desire for suicide is a necessary condition for serious suicidal behavior, fearlessness and resolve about suicide must occur in order for an individual to engage in actual self-harm. Our finding of two separable latent variables is thus consistent with predictions of Joiner’s theory. Additional empirical work with more optimal follow-up data is needed, however, in order to determine what, if anything, distinguishes these factors from one another in terms of outcome.

Aside from the aforementioned limitations, there are others that are important to mention and demonstrate the need to view our results with some degree of caution. First, the follow-up participation rate was low. Given our evidence that those who participated in the follow-up were less clinically severe than those who did not participate, the generalizability of our results to all suicidal callers to crisis hotlines may be limited. Among those who did not complete follow-up, the proportion who refused follow-up was approximately equal to the proportion who were not provided the opportunity to participate because the phone counselor did not ask them for consent. The former reason points to the difficulty of conducting outreach with suicidal patients, and the latter points to the need for additional training of phone counselors to increase their comfort in this area. Although certainly a limitation of the current study, we feel that our results still provide important information regarding the latent structure of suicide risk among hotline callers – it is important to keep in mind that the latent structure of the factors was similar across all three sub-samples. If anything, the use of a less clinically severe sample at follow-up may have attenuated our ability to find significant relationships between our latent factors and subsequent suicidal ideation or attempts. We look forward to future studies with increased follow-up rates. Determining ways to increase follow-up rates among callers to suicide hotlines is an important area of research in its own right, as increasing the willingness of suicidal individuals to accept follow-up assessment and care would be beneficial both clinically and scientifically.

Relatedly, not all eligible callers participated in the baseline assessment, as a large portion were designated by phone counselors, using their own clinical judgment, as being at too high of risk to participate. This is one of the difficulties inherent in conducting research with callers to suicide hotlines. Our experience on this front can be used to inform future researchers as to the need for additional training for phone counselors, emphasizing the need for structured suicide risk assessments even with high risk callers. Arguably, these are the callers most in need of suicide risk assessments. Although clearly a limitation, our hope is that as evidence accrues regarding the clinical utility of structured suicide risk assessments for callers to suicide hotlines, phone counselors will be less reluctant to follow study protocol.

There are additional limitations related to our data analytic strategy. First, there are equivalent models that provide an identical fit to the data as those presented. Another limitation that should be noted is the possibility that using pair-wise deletion to handle missing data may have resulted in biased parameters. We feel fairly confident, however, that our model parameters are trustworthy because of our relatively consistent findings across three independent samples and our study’s convergence with prior work.

In conclusion, we have provided compelling evidence that our measure of suicide risk assesses two separable, latent constructs. This is consistent with prior empirical and theoretical work. We have also provided preliminary evidence that RPP may be a stronger predictor of subsequent suicidal ideation than SDI, although further research is needed to demonstrate this definitively.