The Interpersonal Needs Questionnaire: Statistical Considerations for Improved Clinical Application

Abstract

The interpersonal theory of suicide (ITS) has accumulated empirical support; however, less research has investigated the clinical utility of ITS constructs in suicide risk assessment. The current study sought to increase the clinical utility of the Interpersonal Needs Questionnaire (INQ), an assessment of thwarted belongingness (TB) and perceived burdensomeness (PB), among 318 adult psychiatric outpatients while considering statistical methodology (i.e., multicollinearity, partialling). Results emphasized PB in the prediction of concurrent desire for death/suicide when TB was simultaneously considered. The interaction between TB and PB did not enhance prediction of concurrent desire for death/suicide. Independently, PB was a stronger predictor than TB of concurrent desire for death/suicide in the total sample and gender subsamples. Estimated probabilities of concurrent desire for death/suicide across INQ scores and preliminary INQ clinical cutoff scores are provided to enhance clinical application. These findings suggest the INQ could provide valuable information for suicide risk assessment and conceptualization.

Alarmingly, suicide rates have increased by 24% between 1999 and 2014, and suicide remains the 10^th leading cause of death in the United States (Centers for Disease Control and Prevention [CDC], 2018; Curtin, Warner, & Hedegaard, 2016). Theoretical models of suicide, including the interpersonal theory of suicide (ITS; Joiner, 2005; Van Orden et al., 2010), have been proposed to explain the development of suicidal thoughts and behaviors. The ITS suggests that thwarted belongingness (TB; indicated by loneliness and a perceived lack of reciprocal care relationships) and perceived burdensomeness (PB; indicated by feelings of self-hate and perceived liability on others) are necessary and sufficient risk factors for passive suicide ideation (i.e., desire for death), such that passive suicide ideation develops when TB or PB are experienced independently. Furthermore, TB, PB, and hopelessness are posited to be necessary and sufficient risk factors for active suicide ideation (i.e., desire for suicide), such that active suicide ideation develops when TB and PB are experienced simultaneously and are perceived to be intractable or unchanging. The Interpersonal Needs Questionnaire (Van Orden, Cukrowicz, Witte, & Joiner, 2012), a self-report assessment of TB and PB, has demonstrated strong psychometric properties (e.g., Hill et al., 2015; Van Orden et al., 2012). Extensive research has been conducted using the INQ (see Chu et al., 2017a and Ma et al., 2016), but there has been a dearth of literature examining its clinical utility by providing guidance on how to use the INQ in clinical practice and apply research findings related to the ITS.

Notably, literature lacks consensus on the definition of clinical utility, which may hold different meanings to different stakeholders (e.g., Lesko, Zineh, & Huang, 2010). In the current study, clinical utility refers to 1) an assessment’s practical use for clinicians to accurately identify desire for death and desire for suicide to ensure patients receive optimal care based on an empirically-supported assessment, and 2) an assessment’s practical use for informing clinical case conceptualization and treatment planning. Although, Bryan (2011) derived clinical cutoff scores for the 10-item version of the INQ to best detect suicide ideation among military personnel. The cutoff scores for the 10-item version of the INQ were presented prior to the validation of the most recent 15-item version of the INQ (Van Orden et al., 2012). Given that the psychometric properties of the 15-item version of the INQ are most consistently strong across various samples (see Hill et al., 2015 and Van Orden et al., 2012), it was used in the current study. The aim of our study was to empirically examine the clinical utility of the 15-item version of the INQ (Van Orden et al., 2012) among psychiatric outpatients by considering the implications of the statistical methodology of the existing ITS literature. We also provided estimated probabilities and clinical cutoffs to guide clinical practice.

The 15-item INQ has several potential advantages in clinical settings. The INQ has low face validity as a suicide risk assessment. Van Orden et al. (2012) suggest that individuals may be more willing to disclosure TB and PB than suicidal thoughts, which has been supported by research (e.g., Cukrowicz, Jahn, Graham, Poindexter, & Williams, 2013). Others have suggested that concerns about stigma, being hospitalized or medicated, and others being informed of their suicidal thoughts are deterrents to disclosure of suicidal thoughts (Hom, Stanley, Podlogar, & Joiner, 2017b). The INQ allows clinicians to present items about social relationships, which patients may perceive as less threatening, and then use this information to gain insight into the patient’s current suicidal thoughts. In situations in which a patient explicitly denies suicidal thoughts but provides elevated TB and PB scores, clinicians may consider the possibility that the patient may be under-reporting suicide or death ideation. Clinicians could use this information in suicide risk assessment but also in case conceptualization and clinical intervention planning. For example, using the INQ to provide insight into patients’ social functioning, beyond its use as a suicide risk assessment, may aid clinicians in identifying specific treatment targets and tailored interventions. Therefore, the INQ appears to hold promise for clinical use, but its clinical utility has not been sufficiently tested while considering statistical issues that may impact findings in the ITS literature.

Although research spanning a decade has generally supported the ITS (Chu et al., 2017a; Ma, Batterham, Calear, & Han, 2016), the pattern of findings are likely due to methodological and statistical issues, which warrant consideration when translating empirical findings to clinical application. A recent review by Ma et al. (2016) revealed 40% of studies examining the main effects of TB, 82.6% of studies examining the main effects of PB, and 66.6% of studies examining the interaction between TB and PB produced statistically significant predictions of suicide ideation. It should be noted that Ma et al. (2016) did not account for publication biases. Additionally, some of the studies reviewed by Ma et al. (2016) did not use the INQ and instead relied on proxy measures of TB and PB. Similarly, a meta-analysis testing the predictions of the ITS found that TB and PB were related to suicide ideation with moderate effect sizes, but PB was more strongly associated with suicide ideation than TB (Chu et al., 2017a). Furthermore, the interaction between TB and PB was associated with suicide ideation with a small effect size, and results suggest the variability accounted for by the interaction may be attributed to chance (Chu et al., 2017a). Taken together, these findings provide some support for the tenets of the ITS related to suicide ideation. Although some have concluded that TB is not as clinically meaningful as PB or that PB should be the primary focus of clinical assessment related to suicide ideation (e.g., Ma et al., 2016), this postulation requires greater empirical scrutiny. Ma et al. (2016) and Chu et al. (2017a) did not fully consider important statistical explanations for their findings, particularly the influence of partialling, as they relate to the clinical utility of INQ scores of TB and PB.

As reviewed by Lynam, Hoyle, and Newman (2006), partialling predictor variables can be useful when testing theoretical models; however, the Achilles heel to this approach is multicollinearity (i.e., a strong association between predictor variables that are regressed on a criterion variable). Lynam et al. (2006) suggest the constructs assessed are no longer intact after partialling; thus, when attempting to understand the clinical implications of partial coefficients, it becomes practically impossible to know how to interpret raw assessment scores clinically. For example, depressive symptom scores are often included as a covariate in analyses predicting suicide ideation; however, after partialling the variance associated with depressive symptoms, the remaining suicide ideation construct that is being predicted may no longer represent a typical or meaningful experience of suicide ideation. For example, when depressive symptoms are covaried out of suicide ideation, what remains in suicide ideation is characterized by fearlessness about death, self-sacrifice, and externalizing pathology (Rogers et al., 2018). Therefore, in statistical analyses that covary depressive symptoms out of suicide ideation, TB and PB may not be predicting suicide ideation in its entirety but rather the other remaining facets (Rogers et al., 2018). It is important to note that TB and PB are consistently strongly associated with each other (Chu et al., 2017a), and Ma et al. (2016) suggest that 60% of the tests of the main effects of TB were not significant due to “the stronger effects of [PB] and other covariates” as predictors of suicide ideation (p. 40). Therefore, multicollinearity, due to the strong correlation between TB and PB, may camouflage the clinical benefits of measuring TB and PB in risk assessments. In sum, it is possible that TB and PB are important in clinical assessment when predicting suicide ideation and as treatment targets, but common statistical approaches (i.e., partialling when predictors are multicollinear) may mask the clinical importance of these constructs.

To test this postulation, Mitchell et al. (2017) examined TB and PB as predictors of concurrent distress due to suicide ideation, desire for death, and desire for suicide using two samples of psychiatric inpatients. As simultaneous predictors, PB, but not TB, significantly predicted concurrent distress due to suicide ideation, desire for death, and desire for suicide; however, the interaction between TB and PB was not significant. Notably, there were strong correlations between TB and PB in both samples; thus, multicollinearity may have masked the clinical importance of TB as a predictor of concurrent distress due to suicide ideation, desire for death, and desire for suicide. TB and PB were also examined as separate predictors of concurrent distress due to suicide ideation, desire for death, and desire for suicide. Results indicated the maximum TB and PB score corresponded with a 93% and 95% chance of some level of distress due to suicide ideation, a 91% and 92% chance of some level of desire for death, and a 79% and 84% chance of desire for suicide, respectively. In addition, Mitchell et al. (2017) proposed clinical cutoff scores for the 15-item version of the INQ when predicting the dichotomized concurrent distress due to suicide ideation, desire for death, and desire for suicide, which had previously been absent in the literature among psychiatric inpatients. Thus, TB and PB appear to similarly predict concurrent distress due to suicide ideation, desire for death, and desire for suicide, which would have been overlooked by solely relying on results from the simultaneous predictor models and p-values.

Although Mitchell et al. (2017) provided initial evidence of the clinical utility of the INQ among psychiatric inpatients, there are several limitations that warrant replication of these findings to further clarify multicollinearity concerns and highlight how the INQ could guide clinical assessment and practice. First, Mitchell et al. (2017) used two samples of psychiatric inpatients, one of which only included individuals who were hospitalized for elevated suicide risk. Considering this study only included acute, high-risk samples, these results may not generalize to less clinically severe psychiatric outpatients. For example, mean suicide ideation scores, using the same assessment, are higher among psychiatric inpatients (e.g., Cero, Zuromski, Witte, Ribeiro, & Joiner, 2015; Jahn, Cukrowicz, Mitchell, Poindexter, & Guidry, 2015) than psychiatric outpatients (e.g., Rogers et al., 2018). This is an important limitation given that many clinicians are interested in tools that may facilitate risk decisions in outpatient settings. To better understand how the INQ can be used in outpatient settings, replication is required. Second, the proposed clinical cutoff scores in Mitchell et al. (2017) were limited by small samples that may produce skewed or sample-specific clinical cutoff scores; thus, clinical cutoff scores should be identified for larger samples to produce more stable and generalizable cutoff scores. Third, Mitchell et al. (2017) did not examine gender differences in the relation between the INQ scores and concurrent desire for death and desire for suicide. Considering research indicating gender differences in suicide ideation (Canetto & Sakinofsky, 1998; Drapeau & McIntosh, 2015) and in the associations between TB, PB, and suicide ideation (Chu et al., 2017a; Hill, Hatkvich, Pettit, & Shart, 2017), gender differences should be considered. It is possible gender may impact associations, the estimated probabilities of concurrent desire for death and desire for suicide, and the clinical cutoff scores in the current study. In sum, replication is needed to provide preliminary support for the clinical utility of the INQ in psychiatric outpatient settings.

The Current Study

The current study sought to establish the clinical utility of the INQ by replicating the findings of Mitchell et al. (2017) utilizing a larger sample of psychiatric outpatients. To directly address issues related to multicollinearity and clinical interpretability, TB and PB were examined as simultaneous predictors and as separate predictors of concurrent desire for death and desire for suicide to allow for direct comparisons. We also tested the interaction between TB and PB to demonstrate that TB and PB, when experienced simultaneously, are posited to predict the suicide ideation as it has been traditionally tested (Van Orden et al., 2010). We hypothesized that PB, but not TB, would be a significant predictor of elevated concurrent desire for death and desire for suicide in simultaneous predictor models. Conversely, we hypothesized that TB and PB would be significant predictors of elevated concurrent desire for death and desire for suicide when examined as separate predictors. Although the interaction between TB and PB was not significant in Mitchell et al. (2017), the interaction was significant in meta-analytic research (Chu et al., 2017a). Therefore, we hypothesized a significant interaction between TB and PB, such that the association between TB and concurrent desire for death and desire for suicide would be stronger when PB was higher.

We sought to establish the clinical utility of the INQ by examining TB and PB as separate predictors of dichotomized concurrent desire for death and desire for suicide. To accomplish this, we aimed to: 1) provide clinicians with estimated probabilities of concurrent desire for death and desire for suicide given a specific INQ score for TB or PB, 2) provide empirically identified clinical cutoff scores that maximize sensitivity and specificity of INQ scores predicting concurrent desire for death and desire for suicide, 3) extend the findings of Mitchell et al. (2017) by examining potential gender differences that may influence clinical assessment (i.e., estimated probabilities and INQ cutoff scores), and 4) provide clinicians with preliminary guidance on how to utilize the INQ in a psychiatric outpatient setting, which could provide insight into suicide risk and potential points of psychotherapeutic interventions.

Method

Participants

Participants included 318 adult psychiatric outpatients seeking services from a psychology department university-based training clinic for doctoral clinical psychology students (this was not a student counseling center). Participants were between the ages of 18 and 73 (M = 27.16, SD = 11.29) who mostly identified as women (n = 214, 67.3%). Most participants identified as White (n = 231, 72.6%), followed by Black (n = 28, 8.8%), Asian Pacific Islander (n = 7, 2.2%), American Indian/Alaskan Native (n = 2, 0.6%), and “other” (n = 5, 1.6%). Furthermore, 42 (13.2%) participants identified as Hispanic (n = 42, 13.2%). Racial/ethnicity data were missing for 3 participants (0.9%). The majority of participants completed high school (n = 312, 98.11%). Furthermore, 161 (50.5%) participants were undergraduate students, 38 (11.9%) were graduate students, and 119 (37.4%) were community adults. Of those who were students, 176 (88.4%) were completing their coursework at the university that housed the psychology department university-based training clinic from which the current study collected data. Regarding employment, 93 (29.2%) participants were unemployed, 87 (27.4%) were employed, and employment data were missing for 180 (56.6 %). The average household income of the individual(s) paying for the clinic services was $41,712.31 (SD = $63,502.59), and the average number of individuals supported by the household income was 2.04 (SD = 1.40). At intake, most participants’ primary diagnosis was a depressive disorder (n = 72, 22.6%) or an anxiety disorder (n = 55, 17.3%), and 48 (15.1%) had another primary diagnosis (e.g., a bipolar disorder, substance use disorder, eating disorder, adjustment disorder, schizophrenia spectrum disorder, post-traumatic stress disorder, and attention-deficit/hyperactivity disorder); 143 (45%) participants were not diagnosed for various reasons (e.g., they did not return to the clinic after their initial screening). There were no statistically significant differences in gender, age, race/ethnicity, education level, or employment status between participants who received a psychiatric diagnosis and those who did not. Regarding psychiatric symptoms severity at intake, the average Global Assessment of Functioning (American Psychiatric Association, 2000) score was 61.25 (SD = 12.01) where scores ranged from 20 to 95 with a mode of 60. Additionally, 113 (35.5%) and 62 (19.5%) participants reported some level of desire for death or desire for suicide, respectively. In the entire sample, 240 (75.5%) participants denied a suicide attempt history, 54 (17%) previously attempted suicide once, and 24 (7.5%) reported multiple previous suicide attempts. Of the women participants, 39 (18.2%) reported one previous suicide attempt and 18 (8.4%) reported multiple suicide attempts. Of the men participants, 15 (14.4%) reported one previous suicide attempt and 6 (5.8%) reported multiple suicide attempts.

The data used in the current study were collected between July 2012 through September 2017 as part of a large ongoing data collection at the Florida State University Psychology Clinic; however, we used a subset of these data from participants who completed the measures needed for the current study. Therefore, some of the participants we included in the current study overlap with other studies. Although some of these INQ data were used in 12 other publications (Anestis et al., 2018; Chu et al., 2017b; Hagan, Podlogar, Chu, & Joiner, 2015; Hames et al., 2015; Hawkins et al., 2014; Hom et al., 2017a; Rogers & Joiner, 2018, 2016; Rogers et al., 2017a; Rogers et al., 2017,b; Rogers et al., 2018; Silva, Ribeiro, & Joiner, 2015), it should be noted that our participants’ data do not overlap with those used in primary 15-item INQ validation paper (Van Orden et al., 2012). Furthermore, the analyses that we conducted are unique to the current study and provide important clinical utility information with the most up-to-date data.

Measures

INQ.

The INQ (Van Orden et al., 2012) is a 15-item assessment of TB (9 items; scores range from 9 to 63) and PB (6 items; scores range from 6 to 42). Individuals provide a self-report response to each item on a 7-point response metric ranging from 1 (Not at all true for me) to 7 (Very true for me). The appropriate items are reverse coded, and items are summed to calculate the TB and PB subscale scores with higher scores indicating greater TB and PB. The INQ has demonstrated strong psychometric properties, including construct validity among psychiatric outpatients (Hill et al., 2015; Van Orden et al., 2012). In the current study, the Cronbach’s alpha coefficients were strong for TB (.91) and PB (.94). Bivariate and descriptive statistics for TB and PB are presented in Tables 1 and 2.

Table 1.

Descriptive Statistics and Frequencies Among Total Sample, Men Subsample, and Women Subsample

	Total Sample	Men Subsample	Women Subsample
TB	M = 32.28 (SD = 13.30) Range = 9 to 63	M = 31.73 (SD = 13.43) Range = 9 to 59	M = 32.55 (SD = 13.26) Range = 9 to 63
PB	M = 13.48 (SD = 8.88) Range = 6 to 42	M = 12.14 (SD = 7.75) Range = 6 to 40	M = 14.14 (SD = 9.33) Range = 6 to 42
BSS item 2: No desire for death	n = 205 (64.5%)	n = 80 (76.9%)	n = 125 (58.4%)
BSS item 2: Weak desire for death	n = 84 (26.4%)	n = 16 (15.4%)	n = 68 (31.8%)
BSS item 2: Moderate to strong desire for death	n = 29 (9.1%)	n = 8 (7.7%)	n = 21 (9.8%)
BSS item 4: No desire for suicide	n = 256 (80.5%)	n = 91 (87.5%)	n = 165 (77.1%)
BSS item 4: Weak desire for suicide	n = 51 (16%)	n = 10 (9.6%)	n = 41 (19.2%)
BSS item 4: Moderate to strong desire for suicide	n = 11 (3.5%)	n = 3 (2.9%)	n = 8 (3.7%)

Note. TB = Interpersonal Needs Questionnaire Thwarted Belongingness Score (possible range of 9 to 63); PB = Interpersonal Needs Questionnaire Perceived Burdensomeness Score (possible range of 6 to 42); BSS item 2 = Beck Scale for Suicide Ideation item 2; BSS item 4 = Beck Scale for Suicide Ideation item 4.

Table 2.

Bivariate Correlations Among the Total Sample, Men Subsample, and Women Subsample

Sample	Variable	1	2	3	4	5
Total	1. TB	---
	2. PB	.57**	---
	3. Desire for Death	.40**	.57**	---
	4. Desire for Suicide	.34**	.47**	.63**	---
	5. Gender	−.03	−.11	−.18**	-.12*	---
Men	1. TB	---
	2. PB	.57**	---
	3. Desire for Death	.33**	.69**	---
	4. Desire for Suicide	.27**	.45**	.69**	---
Women	1. TB	---
	2. PB	.58**	---
	3. Desire for Death	.44**	.52**	---
	4. Desire for Suicide	.37**	.46**	.60**	---

Note. TB = Interpersonal Needs Questionnaire Thwarted Belongingness Score; PB = Interpersonal Needs Questionnaire Perceived Burdensomeness Score; Desire for Death = Beck Scale for Suicide Ideation item 2 Dichotomized (0 = no desire for death; 1 = a weak or moderate to strong desire for death); BSS item 4 = Beck Scale for Suicide Ideation item 4 Dichotomized (0 = no desire for suicide; 1 = a weak or moderate to strong desire for suicide); Gender = Dichotomous variable (0 = women, 1 = men); Correlations between continuous variables are Pearson’s r coefficients, between continuous and categorical variables are point-biserial coefficients, and between dichotomous variables are phi coefficients;

^*p < .05;

^**p < .01.

Beck Scale for Suicide Ideation (BSS).

The BSS (Beck & Steer, 1991) is an assessment of suicide ideation, including plans and intent for suicide, during the past week. The BSS has demonstrated strong internal consistency, test-retest reliability, and convergent validity among psychiatric inpatients (Pinninti, Steer, Rissmiller, Nelson, & Beck, 2002). In the current study, desire for death and desire for suicide were assessed using single items from the BSS (Beck & Steer, 1991). Participants responded to these items using an ordinal response metric; therefore, these items were tested as categorical criterion variables, which is also necessary to determine clinical cutoff scores. Notably, the BSS total score does not have established clinical cutoff scores (Beck & Steer, 1991) and could not be used for clinical utility analyses because it is a continuous score. Furthermore, examining these BSS items as categorical criterion variables is consistent with the methods of Mitchell et al. (2017). Previous research has also indicated that a single-item assessing suicide ideation is significantly associated with a suicide ideation total score from a multi-item assessment (Desseilles et al., 2012), which is consistent with our data. In the current study, point-biserial correlations indicate the dichotomized BSS item 2 (r_pb = .66, p < .001) and BSS item 4 (r_pb = .71, p < .001) were significantly associated with the BSS total score (dichotomization discussed further below). Bivariate and descriptive statistics for BSS items 2 and 4 are presented in Tables 1 and 2.

BSS items 2 and 4.

Desire for death (BSS item 2) and desire for suicide (BSS item 4) were assessed using two items from the BSS (Beck & Steer, 1991) in which individuals indicate the degree of their desire for death or suicide during the past week using a three-point ordinal response metric. Individuals indicate no desire for death or suicide (coded 0), a weak desire for death or suicide (coded 1), or a moderate/strong desire for death or suicide (coded 2). Desire for death and desire for suicide are congruent with passive suicide ideation and active suicide ideation as described by the ITS, respectively (Van Orden et al., 2010). In the current study, BSS items 2 and 4 were dichotomized into no desire for death or suicide (coded 0) and a weak or moderate to strong desire for death/or suicide (coded 1), which is consistent with the approach of Mitchell et al. (2017) and the pattern of endorsement of this item in the current study.¹

Procedures

All procedures were approved by the university institutional review board. Participants were seeking psychological services at a university-based psychology training clinic in the Southeast United States. The therapists in this clinic were clinical psychology doctoral students from an American Psychological Association accredited program who were supervised by clinical psychologists. Therapist utilized the Structured Clinical Interview for DSM-IV-TR Axis I and Axis II Disorders (SCID-I/II; First, Gibbon, Spitzer, Williams, & Benjamin, 1997; First, Spitzer, Gibbon, & Williams, 2007) to inform psychiatric diagnoses. The therapists also used Diagnostic and the Statistical Manual of Mental Disorders, Fifth Edition (American Psychiatric Association, 2013) because the SCID-5 was not available during data collection. Furthermore, patients were administered a battery of self-report psychological assessments at intake. The assessments included in the current study were administered as part of this large assessment battery, and individuals who completed the variables used in the current study were included. Per clinic policy, individuals diagnosed with schizophrenia- or bipolar-spectrum disorders who were not stabilized on psychotropic medication and those at imminent risk of harm to themselves or others were referred to other mental health agencies for services. These individuals would have been screened and referred elsewhere prior to completing the large intake assessment packet. All participants provided informed consent for their assessments to be used for research purposes.

Data Analysis Plan

Unless otherwise noted, the data analysis plan mirrors that of Mitchell et al. (2017) for replication purposes. As an extension of Mitchell et al. (2017), the current study examined gender differences. Analyses were conducted using SAS 9.4 software. Raw scores for TB and PB from the INQ were used in the analyses considering these scores would be used in clinical settings. However, to allow for effect size comparisons between TB and PB, additional analyses were conducted with standardized TB and PB scores (z-score transformed) to determine standardized odds ratios (SOR) and confidence intervals. There were no missing data points for the INQ and BSS items, and there were no univariate outliers. There were two multivariate outliers identified using Mahalanobis Distance statistics; these cases were removed prior to analysis, which produced our final sample of 318 participants.

Binary logistic regression analyses were conducted.² To examine the implications of multicollinearity emphasized by Lynam et al. (2006) and Mitchell et al. (2017), TB and PB were examined as simultaneous and separate predictors of concurrent desire for death and desire for suicide (the criterion variables). First, TB and PB were examined as simultaneous predictors of concurrent desire for death and desire for suicide. Second, the interaction between TB and PB (TBxPB) was added to the model with the conditional main effects. Third, TB and PB were examined as separate predictors of concurrent desire for death and desire for suicide. To determine gender differences in the association between TB or PB and concurrent desire for death and desire for suicide, the separate predictor model analyses for TB and PB were repeated including gender (coded 1 = men, 0 = women) with the conditional main effects (TB and Gender, or PB and Gender) and interactions (TBxGender or PBxGender). If an interaction was significant, the association between TB or PB and concurrent desire for death or desire for suicide was examined among men and women to identify clinical cutoff scores for TB and PB for each gender. To improve the interpretability and clinical utility of the INQ, graphs were developed (Figure 1) to depict the estimated probability of concurrent desire for death and desire for suicide across TB and PB scores.

Figure 1.

The lines in the graphs indicate the estimated probability of a non-zero level of desire for death (Beck Scale for Suicide Ideation item 2) or desire for suicide (Beck Scale for Suicide Ideation item 4) across unstandardized thwarted belongingness scores (top graph; using the total sample) and perceived burdensomeness scores (bottom graph; using the total sample, men only, and women only).

Receiver operating characteristic (ROC) curve analyses were conducted to provide further information regarding the effect sizes of these models.³ Furthermore, to provide insight into the clinical unity of the INQ score, ROC curve analyses were conducted with TB and PB tested as separate predictors of concurrent desire for death and desire for suicide. Youden’s J Statistic (i.e., Youden Index), which equally minimizes false positive and false negative rates, was calculated to determine the clinical cutoff scores for TB and PB that maximize sensitivity and specificity when predicting concurrent desire for death and desire for suicide.⁴

Results

Simultaneous Predictor Models with TB, PB, and TBxPB

Criterion variable: Desire for death (BSS item 2).

PB (SOR = 3.69, 95% Wald CI = [2.49, 5.31]), but not TB (SOR = 1.39, 95% Wald CI = [0.98, 1.98]), was significantly associated with concurrent desire for death (Table 3). Additionally, TBxPB was not significantly associated with concurrent desire for death (SOR = 0.70, 95% Wald CI = [0.48, 1.03]; Table 3). ROC curve analyses indicate the area under the curve (AUC) statistic for the model including the main effects was .85, and when the interaction was included in the model, the AUC remained .85.⁵ The AUC statistic of .85 corresponds with a Cohen’s d effect size of 1.47 (i.e., a large effect size; see Ruscio, 2008 for conversion equations). Therefore, TBxPB did not contribute additional predictive value in concurrent desire for death beyond the main effects of TB and PB. Monte Carlo simulation power analyses were conducted using Mplus 8 (Muthén & Muthén, 1998–2017) to determine the power to detect the TBxPB effect in the current study, which indicated power of .68. A sample of 410 or 1,210 would be needed to achieve power of .80 and > .999 to detect the TBxPB effect, respectively. This indicates that a very large number of participants would be required to detect the TBxPB effect because it is very small; however, increasing the sample size to detect a small effect would also increase the risk for Type I error. Consistent with Chu et al. (2017a), the small effect of TBxPB suggests it may not be clinically significant.

Table 3.

Unstandardized Thwarted Belongingness, Perceived Burdensomeness, and the Interaction Between Thwarted Belongingness and Perceived Burdensomeness Scores as Simultaneous Predictors of Desire for Death and Desire for Suicide

Parameter	Logit	SE	Wald χ2	p	OR	OR 95% CI
Outcome: BSS item 2 (Desire for Death)
Intercept	−3.49	0.45	60.82	< .001
TB	0.02	0.01	3.40	.065	1.03	1.00, 1.05
PB	0.15	0.02	44.48	< .001	1.16	1.11, 1.21
Intercept	−4.81	0.91	27.78	< .001
TB	0.06	0.02	5.99	.014	1.06	1.01, 1.12
PB	0.26	0.07	13.63	< .001	1.30	1.13, 1.50
TBxPB	−0.003	0.002	3.21	.073	1.00	0.99, 1.00
Outcome: BSS item 4 (Desire for Suicide)
Intercept	−4.39	0.59	56.31	< .001
TB	0.04	0.02	5.66	.017	1.04	1.01, 1.07
PB	0.10	0.02	25.18	< .001	1.10	1.06, 1.15
Intercept	−6.09	1.23	24.47	< .001
TB	0.08	0.03	6.84	.009	1.09	1.02, 1.16
PB	0.21	0.07	9.24	.002	1.24	1.08, 1.42
TBxPB	−0.003	0.002	3.00	.083	1.00	0.99, 1.00

Note. TB = Interpersonal Needs Questionnaire Thwarted Belongingness Score; PB = Interpersonal Needs Questionnaire Perceived Burdensomeness Score; TBxPB = the multiplied effect the Interpersonal Needs Questionnaire Thwarted belongingness and Perceived Burdensomeness scores; BSS item 2 = Beck Scale for Suicide Ideation item 2 assessing desire for death; BSS item 4 = Beck Scale for Suicide Ideation item 4 assessing desire for suicide.

Criterion variable: Desire for suicide (BSS item 4).

TB (SOR = 1.66, 95% Wald CI = [1.09, 2.53]) and PB (SOR = 2.40, 95% Wald CI = [1.71, 3.38]) were significantly associated with concurrent desire for suicide (Table 3). In addition, TBxPB was not significantly associated with concurrent desire for suicide (SOR = 0.71, 95% Wald CI = [0.49, 1.05]; Table 3). The AUC statistic for the model including the main effects was .83, and when the interaction was included in the model, the AUC remained .83. The AUC statistic of .83 corresponds with a Cohen’s d effect size of 1.35 (i.e., a large effect size). Again, these results indicate that TBxPB did not contribute additional predictive value in concurrent desire for suicide beyond the main effects of TB and PB. Monte Carlo simulation power analyses indicated power to detect the effect of TBxPB was .49. A sample of 660 or 1,860 would be needed to achieve power of .80 and > .999, respectively. Again, this indicates that a very large number of participants would be required to detect the TBxPB effect because it is so small and likely not clinically meaningful.

Separate Predictor Models with TB and PB, and Gender Differences

Criterion variable: Desire for death (BSS item 2).

On a bivariate level, TB was significantly associated with concurrent desire for death (SOR = 2.67, 95% Wald CI = [2.00, 3.57]; Table 4). The AUC statistic for TB predicting concurrent desire for death was .74 (SE = .03, 95% CI = [.69, .80]), which was converted to a Cohen’s d effect size of 0.91 (i.e., a large effect size). Similarly, on a bivariate level, PB was significantly associated with concurrent desire for death (SOR = 4.39, 95% Wald CI = [3.13, 6.14]; Table 4). The AUC statistic for PB predicting concurrent desire for death was .85 (SE = .02, 95% CI = [.81, .89]), which was converted to a Cohen’s d effect size of approximately 1.47 (i.e., a large effect size). When the AUC statistics were compared, PB was a significantly stronger predictor of concurrent desire for death than TB (χ² [1, N = 318] = 18.03, p < .001). See Table 5 for clinical cutoff scores and related statistics.

Table 4.

Unstandardized Thwarted Belongingness and Perceived Burdensomeness Scores as Separate Predictors of Desire for Death and Desire for Suicide

Parameter	Logit	SE	Wald χ2	p	OR	OR 95% CI
Outcome: BSS item 2 (Desire for Death)
Intercept	−3.11	0.42	55.69	< .001
TB	0.07	0.01	43.99	< .001	1.08	1.05, 1.10
Intercept	−2.93	0.30	93.14	< .001
PB	0.17	0.02	73.83	< .001	1.18	1.14, 1.23
Outcome: BSS item 4 (Desire for Suicide)
Intercept	−4.17	0.55	56.79	< .001
TB	0.08	0.01	31.58	< .001	1.08	1.05, 1.11
Intercept	−3.41	0.35	94.58	< .001
PB	0.12	0.02	51.24	< .001	1.13	1.09, 1.17

Note. TB = Interpersonal Needs Questionnaire Thwarted Belongingness Score; PB = Interpersonal Needs Questionnaire Perceived Burdensomeness Score; BSS item 2 = Beck Scale for Suicide Ideation item 2 assessing desire for death; BSS item 4 = Beck Scale for Suicide Ideation item 4 assessing desire for suicide.

Table 5.

Receiver Operating Characteristic Curve Analyses Statistics and the Clinical Cutoff Scores of Unstandardized Thwarted Belongingness and Perceived Burdensomeness Scores as Separate Predictors of Desire for Death and Desire for Suicide

	Total Sample: Desire for Death (BSS item 2)	Men Subsample: Desire for Death (BSS item 2)	Women Subsample: Desire for Death (BSS item 2)	Total Sample: Desire for Suicide (BSS item 4)
TB Maximum Youden Index	0.37	----	----	0.41
TB Sensitivity	0.88	----	----	0.95
TB Specificity	0.49	----	----	0.46
TB Area Under the Curve	0.74	----	----	0.74
TB Cutoff Score	35	----	----	50
PB Maximum Youden Index	0.56	0.72	0.51	0.57
PB Sensitivity	0.79	0.96	0.87	0.89
PB Specificity	0.77	0.76	0.65	0.68
PB Area Under the Curve	0.85	0.93	0.82	0.83
PB Cutoff Score	19	22	17	30

Note. TB = Interpersonal Needs Questionnaire Thwarted Belongingness Score; PB = Interpersonal Needs Questionnaire Perceived Burdensomeness Score; BSS item 2 = Beck Scale for Suicide Ideation item 2 assessing desire for death; BSS item 4 = Beck Scale for Suicide Ideation item 4 assessing desire for suicide.

Subsequently, gender was examined as a moderator of the relation between TB or PB (as separate predictors) and concurrent desire for death. TBxGender (SOR = 0.81, 95% Wald CI = [0.43, 1.53], p = .513) was not significant in relation to concurrent desire for death. Therefore, the association between TB and concurrent desire for death did not significantly vary by gender. The AUC statistic for TB was .73 (SE = .05, 95% CI = [.63, .83]) for men and .75 (SE = .03, 95% CI = [.69, .81]) for women when predicting concurrent desire for death. However, PBxGender (SOR = 3.58, 95% Wald CI = [1.26, 10.15], p = .017) was statistically significant, which indicated that the association between PB and concurrent desire for death differs by gender. Specifically, among men, PB was significantly associated with concurrent desire for death (SOR = 8.96, 95% Wald CI = [3.82, 21.04]; Table 6). Among women, PB was also significantly associated with concurrent desire for death (SOR = 3.67, 95% Wald CI = [2.51, 5.37]; Table 6). The AUC statistic for PB was .93 (SE = .02, 95% CI = [.89, .98]) for men and .82 (SE = .02, 95% CI = [.77, .88]) for women, which were converted to Cohen’s d effect size of approximately 2.09 and 1.29 (i.e., large effect sizes), respectively. These results indicate the association between PB and concurrent desire for death is stronger among men than women (though this association was significant among both men and women). When the AUC statistics were compared, PB was a stronger predictor of concurrent desire for death than TB among men (χ² [1, N = 318] = 19.77, p < .001) and women (χ² [1, N = 318] = 5.73, p = .017). See Table 5 for cutoff scores and related statistics for PB predicting concurrent desire for death for men and women, separately.

Table 6.

Unstandardized Perceived Burdensomeness Score as a Separate Predictor of Desire for Death Among the Men and Women Subsamples.

Parameter	Logit	SE	Wald χ2	p	OR	OR 95% CI
Men
Intercept	−5.26	0.93	32.35	< .001
PB	0.28	0.06	25.34	< .001	1.33	1.19, 1.48
Women
Intercept	−2.31	0.33	50.28	< .001
PB	0.14	0.02	44.87	< .001	1.15	1.10, 1.20

Note. Desire for Death = Assessed using the Beck Scale for Suicide Ideation item 2; PB = Interpersonal Needs Questionnaire Perceived Burdensomeness Score.

Criterion variable: Desire for suicide (BSS item 4).

On a bivariate level, TB was significantly associated with concurrent desire for suicide (SOR = 2.76, 95% Wald CI = [1.94, 3.94]; Table 4). The AUC statistics for TB predicting concurrent desire for suicide was .74 (SE = .04, 95% CI = [.68, .80]), which was converted to a Cohen’s d effect size of 0.91 (i.e., a large effect size). Similarly, on a bivariate level, PB was significantly associated with concurrent desire for suicide (SOR = 3.01, 95% Wald CI = [2.23, 4.07]; Table 4). The AUC statistic for PB predicting concurrent desire for suicide was .83 (SE = .03, 95% CI = [.78, .88]), which was converted to a Cohen’s d effect size of 1.35 (i.e., large effect sizes). When the AUC statistics were compared, PB was a significantly stronger predictor of concurrent desire for suicide than TB (χ² [1, N = 318] = 8.77, p = .003).

Subsequently, gender was examined as a moderator of the relation between TB or PB (as separate predictors) and concurrent desire for suicide. TBxGender (SOR = 0.80, 95% Wald CI = [0.37, 1.76], p = .581) and PBxGender (SOR = 1.32, 95% Wald CI = [0.62, 2.83], p = .473) interactions were not significant in relation to concurrent desire for suicide. Therefore, the association between TB or PB (as separate predictors) and concurrent desire for suicide did not significantly vary by gender, and ROC curve analyses were not examined separately among men and women.

Using TB and PB Scores to Estimate the Probability of Concurrent Desire for Death or Suicide

To increase the clinical utility of the findings, estimated probabilities of concurrent desire for death and suicide across INQ TB and PB scores are provided in Figure 1. For example, Figure 1 (top graph) indicates a TB score of 9 (lowest score) corresponds with an 8% and 3% chance of concurrent desire for death and desire for suicide, respectively. A TB score of 63 (highest score) corresponds with an 82% and 66% chance of concurrent desire for death and desire for suicide, respectively.

See Figure 1 (bottom graph) for graphed estimated probabilities of concurrent desire for death across PB scores for the total sample, men only, and women only, as well as estimated probabilities of concurrent desire for suicide across PB scores for the total sample. Figure 1 (bottom graph) indicates a PB score of 6 (lowest score) corresponds with a 13% (total sample), 3% (men), and 17% (women) chance of concurrent desire for death. A PB score of 42 (highest score) corresponds with a 98% (total sample), 99% (men), and 97% (women) chance of desire for death. Furthermore, a PB score of 6 corresponds with a 7% chance of concurrent desire for suicide, whereas a PB score of 42 corresponds with an 86% chance of concurrent desire for suicide. Table 7 displays the estimated probabilities of concurrent desire for death and desire for suicide when TB and PB are 1 SD above and below the mean.

Table 7.

Estimated Probabilities of Desire for Death and Desire for Suicide at ± 1 SD on the Unstandardized Thwarted Belongingness and Perceived Burdensomeness Scores

Total Sample: Desire for Death (BSS item 2)
	BSS 0	BSS Non-0
TB +1 SD	0.44	0.56
TB −1 SD	0.85	0.15
PB +1 SD	0.65	0.35
PB −1 SD	0.94	0.06
Men Subsample: Desire for Death (BSS item 2)
	BSS 0	BSS Non-0
PB +1 SD	0.41	0.59
PB −1 SD	0.98	0.02
Women Subsample: Desire for Death (BSS item 2)
	BSS 0	BSS Non-0
PB +1 SD	0.28	0.72
PB −1 SD	0.84	0.16
Total Sample: Desire for Suicide (BSS item 4)
	BSS 0	BSS Non-0
TB +1 SD	0.67	0.33
TB −1 SD	0.94	0.06
PB +1 SD	0.65	0.35
PB −1 SD	0.94	0.06

Note. TB = Interpersonal Needs Questionnaire Thwarted Belongingness Score; PB = Interpersonal Needs Questionnaire Perceived Burdensomeness Score; BSS item 2 = Beck Scale for Suicide Ideation item 2 assessing desire for death; BSS item 4 = Beck Scale for Suicide Ideation item 4 assessing desire for suicide.

Discussion

Although theoretical models of suicide should enhance suicide prevention efforts, not all theory-driven research directly informs clinical application or improves suicide risk assessment. With accumulating evidence generally supporting the ITS (Chu et al., 2017a; Ma et al., 2016), there is a need to establish the clinical utility of the 15-item version of the INQ, a psychometrically-sound self-report assessment of TB and PB that was initially validated using psychiatric outpatient data from the same psychology clinic used in the current study (but not the same participants; Van Orden et al., 2012). The current study examined the possible influence of multicollinearity between TB and PB in association with concurrent desire for death and desire for suicide, which has implications for the clinical application of the ITS. The current study also 1) provided clinicians with estimated probabilities of concurrent desire for death and desire for suicide given a specific INQ score for TB or PB, 2) provided empirically identified clinical cutoff scores that maximize sensitivity and specificity of INQ scores predicting concurrent desire for death and desire for suicide, 3) replicated and extended the findings of Mitchell et al. (2017) by examining potential gender differences that may influence clinical assessment (i.e., estimated probabilities and INQ cutoff scores), and 4) provided clinicians with preliminary guidance on how to utilize the INQ in a psychiatric outpatient setting, which could provide insight into suicide risk and potential points of psychotherapeutic interventions.

Our hypothesis that PB, but not TB, would be significantly associated with elevated concurrent desire for death and desire for suicide in simultaneous predictor models was partially supported. Results indicated TB and PB, as simultaneous predictors, were significantly associated with concurrent desire for suicide; however, only PB was significantly associated with concurrent desire for death when TB was included as a simultaneous predictor. Although the AUC effect sizes were similar in magnitude, these findings suggest that TB and PB predicted unique variance in concurrent desire for suicide, but only PB predicted unique variance in concurrent desire for death. Therefore, TB appears to play a greater role in concurrent desire for suicide than desire for death when tested as a simultaneous predictor with PB; conversely, PB is consistently a strong predictor of concurrent desire for death and desire for suicide when considered simultaneously with TB. The ITS posits that passive suicide ideation (i.e., desire for death) occurs when TB or PB is experienced, whereas active suicide ideation (i.e., desire for suicide) develops when TB and PB are experienced simultaneously, and an individual feels hopeless about these states changing (Joiner, 2005; Van Orden et al., 2010). Our findings support the ITS in that TB and PB each predicted unique variance in concurrent desire for suicide, whereas this combination of TB and PB did not appear to improve prediction of concurrent desire for death.

Notably, our findings were not completely consistent with the findings of Mitchell et al. (2017), which indicated significant main effects of PB, but not TB, when tested as simultaneous predictors of concurrent desire for death and desire for suicide among psychiatric inpatients. This discrepancy may have occurred because the psychiatric inpatient samples examined in Mitchell et al. (2017) were more clinically severe, as evidenced by approximately 63% (sample 1) and 74% (sample 2) of participants reporting suicide attempt history. However, in the current study of psychiatric outpatients, only 24.5% reported a suicide attempt history. Furthermore, TB and PB were more strongly correlated (r = .69) in the Mitchell et al. (2017) study that utilized psychiatric inpatients admitted due to elevated suicide risk; however, in the current study, TB and PB were less strongly correlated (r = .57) among community psychiatric outpatients. It appears that multicollinearity between TB and PB did not mask the significance of TB among a less clinically severe sample but may be impacting statistical significance trends across samples. This is congruent with the issues in interpreting the clinical meaning when multicollinearity among predictors is high, as demonstrated by Lynam et al. (2006).

Our hypothesis that there would be a significant interaction between TB and PB was not supported. Additionally, the interaction effect between TB and PB was small and would have required a very large sample to detect, but this would also increase chances of a Type I error. This is consistent with Mitchell et al. (2017) who also did not find a significant interaction between TB and PB predicting concurrent desire for death or desire for suicide. The interaction between TB and PB has demonstrated statistical significance in other studies (see Chu et al., 2017,a and Ma et al., 2016 for review); however, meta-analytic results indicate this interaction is statistically significant, but the effect size is small, may lack clinical significance, and may be due to chance (Chu et al., 2017a). This is consistent with Mitchell et al. (2017) who suggest that the ITS literature may be overemphasizing the importance of the interaction between TB and PB. It may also be possible that the interaction between TB and PB was not significant in the current study due to the measures we used (e.g., dichotomized concurrent desire for death and desire for suicide). Another consideration is that the INQ is not a perfect assessment of TB and PB, and the lack of support for the interaction between TB and PB and the strong correlation between TB and PB could be a product of the INQ measure itself. Therefore, future research should consider continued refinement and improvement of the INQ or alternative assessments of TB and PB.

The findings from the simultaneous predictor models within the current study and Mitchell et al. (2017), coupled with the inconsistent findings in the literature regarding the predictive ability of TB (Chu et al., 2017a; Ma et al., 2016), may suggest PB offers the most clinical utility. However, it is crucial to consider the impact of multicollinearity on the clinical interpretation of assessment scores (Lynam et al., 2006) considering, the multicollinearity between TB and PB may be masking the predictive and clinical value of TB. To determine the clinical interpretability of the INQ scores for clinicians, TB and PB were tested as separate predictors of concurrent desire for death and desire for suicide in the current study. Consistent with our hypothesis, TB and PB, independently, were each significantly associated with concurrent desire for death and desire for suicide. However, the difference in the AUC statistics indicated PB was a statistically stronger predictor of concurrent desire for death and desire for suicide compared to TB among the total sample, men subsample, and women subsample. Furthermore, the association between PB and concurrent desire for death was stronger among men than women. Conversely, the associations between TB and concurrent desire for death and desire for suicide, and PB and concurrent desire for suicide did not vary by gender. PB alone predicted concurrent desire for death (AUC = .85) and desire for suicide (AUC = .83) equivalently to TB and PB simultaneously predicting concurrent desire for death (AUC = .85) and desire for suicide (AUC = .83). However, this does not necessarily indicate that TB lacks clinical value. TB was still a significant independent predictor of concurrent desire for death and desire for suicide. Moreover, TB could still provide clinical information that can be used to generate unique clinical hypotheses and targets for psychotherapeutic interventions; therefore, clinicians should consider both TB and PB when conceptualizing patients’ suicide risk.

Given the findings from the current study, Mitchell et al. (2017), Chu et al. (2017a), and Ma et al. (2016), it is possible that it may not be necessary, theory-consistent, or meaningful to test the interaction between TB and PB. Van Orden et al. (2010), the seminal ITS article, hypothesized, “…the simultaneous presence of thwarted belongingness, perceived burdensomeness, and hopelessness about one’s interpersonal connections is a proximal and sufficient cause of suicidal desire” (p. 589). This hypothesis is traditionally tested by examining the main effects of TB and PB, and then including the interaction (i.e., TBxPB) as predictors of suicide ideation (at times with additional covariates), but hopelessness is not always included (e.g., Cero et al., 2015). However, the phrasing of this hypothesis by Van Orden et al. (2010) may be more accurately tested by examining the additive effects of TB, PB, and hopelessness as predictors of suicide ideation, rather than focusing on interactions that have small effect sizes. Further, the addition of interpersonal hopelessness to simultaneous predictor models of TB and PB predicting desire for suicide would be more consistent with the ITS and possibly enhance the prediction of desire for suicide compared to models that include the interaction between TB and PB, which is consistent with recent research (i.e., Tucker et al., 2018).

The current findings also emphasize important considerations when attempting to translate empirical findings into clinical application. Although PB predicted more variability in concurrent desire for death and desire for suicide than TB, TB and PB may relate more to distinct conceptual aspects of desire for death and desire for suicide (e.g., frequency, duration, or intensity of desire) or may not be predicting the same variance in desire for death and desire for suicide. This could have implications for assessment and treatment, such that TB and PB may provide distinct clinical information that is not well-reflected in statistical models. Furthermore, the strength of the associations between TB and PB, and concurrent desire for death and desire for suicide appear to have similar strength among higher-risk samples (i.e., psychiatric inpatients admitted for elevated suicide risk; Mitchell et al., 2017), whereas PB appears strongly associated with concurrent desire for death and desire for suicide among psychiatric outpatients in the current study. Additionally, the gender differences found in the current study, the differences between the findings of the current study and previous literature (i.e., Mitchell et al., 2017), and findings suggesting possible moderators of the ITS postulations (i.e., Chu et al., 2017a), suggest idiographic or population-specific factors (e.g., severity of psychiatric distress, gender) should be considered when administering and interpreting results of the INQ.

Clinical and Assessment Implications

The findings of this study have clinical implications and provide preliminary support for the clinical use of the INQ, a theory-driven suicide risk assessment instrument, as part of a comprehensive suicide risk assessment. However, future research should compare the INQ to other risk assessment instruments to determine if the INQ (or other risk assessment instruments) hold incremental value in detecting suicide ideation. To improve the clinical utility of the INQ, estimated probabilities of concurrent desire for death and desire for suicide across INQ scores were provided in Figure 1 and Table 2. Examining partial relations between variables is incredibly difficult and impractical in clinical settings where only raw scores are readily available (Lynam et al., 2006). As such, our findings allow clinicians to more easily estimate the probability of concurrent desire for death and desire for suicide based on INQ scores independently. For example, if a clinician identified that a male patient in an outpatient setting obtained a score of 61 on TB and 28 on PB, using Figure 1 they could estimate that their patient has 80% to 93% chance of experiencing concurrent desire for death and a 52% to 62% chance of experiencing concurrent desire for suicide.

We also provided clinical cutoff scores for TB and PB with an optimal balance between sensitivity (true positive rate) and specificity (true negative rate) when predicting concurrent desire for death and desire for suicide, as well as cutoff scores for PB predicting concurrent desire for death among men and women separately due to the significant moderating effect of gender (see Table 5). For example, if a female patient in a psychiatric outpatient setting were administered the 15-item version of the INQ and obtained a score of 51 for TB and 32 for PB, we would anticipate that this individual is at elevated risk for experiencing some level of concurrent desire for death and desire for suicide with optimal confidence. It should be noted that our data are consistent with meta-analytic findings (Chu et al., 2017a) and indicate TB and PB are strongly correlated; thus, it is likely that an individual reporting elevated TB will also report elevated PB. However, if the case were to arise where an individual reported elevated TB, but not PB, or vice versa, the TB and PB scores and clinical cutoff scores can be used independently to estimate concurrent desire for death or desire for suicide. This is possible because the estimated probabilities and clinical cutoff scores were derived from models where TB and PB were tested as independent predictors of concurrent desire for death or desire for suicide. The use of clinical cutoff scores is consistent with suicide risk decision trees (e.g., Chu et al., 2015) where elevated suicide risk is determined by identifying the presence of clinically significant risk factors. The cutoff scores for the INQ provided in the current study, can guide clinicians on what could be considered elevated TB and PB among psychiatric outpatients.

Although we hope that our findings provide clinicians with enhanced guidance when using the 15-item version of the INQ, it is important to note that the estimated probabilities and clinical cutoff scores should be considered as preliminary. Continued replication with larger samples is necessary to firmly establish probability estimates and cutoff scores. It should be recognized that the cutoff scores are specific to concurrent desire for death and desire for suicide and should be considered within the sample characteristics (e.g., psychiatric outpatient sample). Thus, our results may not be generalizable to suicidal behaviors (e.g., suicide attempts) or other populations. For example, our cutoff scores for concurrent desire for death and desire for suicide are higher than those presented by Mitchell et al. (2017) who utilized the 15-item version of the INQ among psychiatric inpatients (with the exception of gender-specific cutoff scores, which were not examined). This difference may be due to the elevated severity of the suicide risk among psychiatric inpatient samples compared to psychiatric outpatients. With this in mind, evidence of elevated desire for death or desire for suicide should be further clinically investigated, and the population should be strongly considered.

Using the INQ to understand the severity of TB and PB, in tandem with a comprehensive suicide risk assessment, could help inform a suicide risk mitigation plan. Clinicians should carefully consider the use of evidence-based suicide risk management practices if a patient scores above the recommended cutoff scores for TB and/or PB, especially when the severity of TB and PB converges with other clinical information that may indicate heightened risk for suicide (e.g., presence of additional risk factors, absence of protective factors). For instance, clinicians may refer the patient to more intensive clinical services, adjust the treatment plan to include the use of suicide-specific treatments (e.g., Safety Planning Intervention [Stanley & Brown, 2012], Cognitive Behavior Therapy for Suicide Prevention [Stanley et al., 2009]), request permission to obtain collateral information from family or friends, and provide the patient with a crisis line number. In situations in which a patient explicitly denies suicide or death ideation but provides scores for TB and PB that are above the recommended clinical cutoffs, clinicians should consider the possibility that the patient may be under-reporting suicide or death ideation. Additionally, clinicians may wish to conduct a reassessment of suicide and death ideation given the possible fluctuations that may occur (e.g., Kleiman et al., 2017). Overall, we advocate that a patient’s score for TB and PB on the INQ should be considered along with other information collected during a comprehensive suicide risk assessment, especially due to the preliminary nature of the estimated probabilities and cutoff scores provided in this study.

The use of the INQ in clinical settings is also supported by its low face validity as a measure of suicide risk. Van Orden et al. (2012) suggest some individuals may be less willing to directly disclose desire for death or desire for suicide due to the perceived potential of negative consequences. Research suggests that older adults in primary care may feel more inclined to disclose TB and PB than suicide ideation (Cukrowicz et al., 2013); however, further research is needed to test this in other populations such as psychiatric outpatients. Similarly, stigma-related concerns are a primary barrier to disclosure of past suicide ideation among undergraduate students, and concerns about being hospitalized, medicated, and others being informed of their suicidal thoughts are additional deterrents to disclosure (Hom et al., 2017b). Direct clinical inquiry about suicide or other mental health-related risk factors (e.g., depressive symptom inventories that may include questions about suicide ideation) could produce non-disclosure of suicide ideation due to mental health-related stigma or fear of consequences of disclosure among psychiatric outpatients. Because the INQ is an assessment of social experiences, not symptoms of a psychiatric diagnosis, patients may be more willing to disclose TB and PB, and in turn, provide clinicians insight into their risk for concurrent desire for death and desire for suicide; however, these postulations require further empirical investigations. The clinician and the patient could then discuss discrepancies between their INQ scores and disclosure of suicidal thoughts. Such a discussion could lead to more in-depth information about the patient’s reasons for living and reasons for dying, and they may then elect to disclose experiences of suicide ideation.

The incorporation of the INQ into suicide risk assessment is consistent with the Therapeutic Risk Management Model for the Suicidal Patient (Wortzel, Matarazzo, & Homaifar, 2013), which highlights the importance of integrating multiple sources of information when conducting a comprehensive suicide risk assessment to yield the most accurate risk severity determination and provide the most appropriate intervention. This model acknowledges that suicide risk assessment clinical interviews are important assessment strategies, but that clinicians should also incorporate standardized assessment tools. While a patient may not be comfortable answering a direct question about suicide ideation from a clinician, they may answer these questions in other formats (e.g., self-report questionnaires) and questions about TB and PB. This provides an additional avenue to obtain vital information about a patient’s suicide risk and discuss discrepancies. Furthermore, the Therapeutic Risk Management Model for the Suicidal Patient encourages discussion of reasons for living during a suicide risk assessment (Homaifar, Matarazzo, & Wortzel, 2013). The INQ could be used to identify reasons for living among patients with high INQ scores but who denied, and are genuinely not experiencing, suicidal thoughts. For example, if the individual feels like a burden, but has many close relationships, discussing these relationships as potential protective factors against suicide could provide a more holistic conceptualization of the patient’s risk. Another possibility is that the patient has high INQ scores, but due to other protective factors, does not consider suicide. Therefore, the INQ scores can be used to initiate further discussion about suicide risk even with patients who may deny or not disclose suicidal thoughts. Future research should explore this more thoroughly.

The INQ has other potential advantages for use in clinical settings. In addition to its contribution to suicide risk assessment, the INQ may also provide important information about patients’ social functioning and interpersonal resources more generally, which inform patient-tailored intervention. Moreover, the INQ is a relatively brief self-report suicide risk assessment instrument to administer and is unlikely to be excessively demanding on existing risk assessment and management protocols in many outpatient settings. The INQ is also freely available in the public domain, which makes it a cost-effective resource.

A thorough understanding of the theory-driven factors influencing a patient’s desire for death and desire for suicide (e.g., TB and PB) may inform treatment targets for suicide-specific psychotherapeutic interventions. The current study suggests that PB may be particularly useful when estimating the probability of concurrent desire for death and desire for suicide; however, assessing TB could still provide clinical insight that may inform treatment planning. Therefore, assessing TB and PB may allow for more targeted intervention at several stages of care. For example, a clinician may recognize a patient reported elevated TB and PB and subsequently discuss this with the patient further, which may increase the therapeutic alliance and provide additional points of intervention within a collaborative treatment planning session. TB and PB may also serve as recognizable warning signs of concurrent desire for death and desire for suicide that can be collaboratively explored within a safety planning intervention (Stanley & Brown, 2012). Moreover, TB and PB may provide meaningful targets for cognitive restructuring in cognitive behavior therapy (Joiner, Van Orden, Witte, & Rudd, 2009) or mindfulness and cognitive defusion techniques in acceptance and commitment therapy (Roush, Brown, Mitchell, & Cukrowicz, 2017; Roush, Mitchell, Brown, & Cukrowicz, 2018).

Limitations

There are several limitations that may inform the interpretation of the current findings and future efforts aimed at improving the clinical utility of the INQ. The findings of the current study may not generalize to other clinical settings or all psychiatric outpatient settings due to it being predominately comprised of university student clientele, and readers should refer to Mitchell et al. (2017) for information regarding the preliminary clinical utility evidence of the INQ in psychiatric inpatient settings. Further impacting the generalizability of our findings, the data used in the current study included participants’ data that were used in several other studies cited in the methods section of this paper. The current findings should not be considered additional or independent support for the INQ or the ITS beyond what has been presented in previous papers using these data. However, it should be noted that our analyses and clinical utility analyses are unique to this paper and have not been published elsewhere with these data. Additionally, our participants do not overlap with those used in the Van Orden et al. (2012) INQ validation paper. Moreover, our data were collected from self-report assessments, which may be vulnerable to response biases; thus, future studies should consider behavioral indicators of desire for death and desire for suicide. The cross-sectional design of the current study also precludes any determination of possible temporal associations between TB, PB, and concurrent desire for death and desire for suicide, including the predictive validity of clinical cutoffs. Therefore, future studies should consider the use of longitudinal designs and ecological momentary assessment designs, which has become increasingly feasible with technological advancements (Kleiman & Nock, 2017). These methodologies may allow for the development of clinical cutoff scores for the INQ that predict future suicide and death ideation. The current study used the 15-item version of the INQ, but future studies should consider continuing to test the clinical utility of the 10-item version in clinical populations (see Bryan, 2011), which has demonstrated good psychometric properties (Hill et al., 2015). Furthermore, the interpersonal hopelessness component of the ITS was not assessed because the current study focused on the application of the INQ, and a psychometrically established measure of interpersonal hopelessness was not available at the time of data collection. Fortunately, an assessment of interpersonal hopelessness has recently been developed (Tucker et al., 2018). Future studies could include an assessment of interpersonal hopelessness and establish its clinical utility. However, it is important to remain cognizant of multicollinearity issues when examining interpersonal hopelessness as a predictor of concurrent desire for death and desire for suicide.

Another potential limitation of the current study may be the use of single-item assessments of concurrent desire for death and desire for suicide. Single-item assessments of suicide ideation and attempts are more prone to imprecise language (Hom, Joiner, & Bernert, 2016) and to misclassification of participants when limited to yes/no responses of lifetime incidence of suicide ideation (Millner, Lee, & Nock, 2015); however, our use of single-items for desire for death and desire for suicide may be less susceptible to these limitations. For instance, the current study utilized single items for desire for death and desire for suicide that used clear language, allowed for more variability in responses rather than asking a yes/no question, and specified a specific and more immediate time frame than the items examined in Millner et al. (2015). Future research should consider replicating these findings using composite scores from validated assessments to ensure the validity of desire for death and desire for suicide and improve the prediction ability of assessment batteries that may include a wide range of assessments. The current study did not include a measure of suicide-related behavior; thus, our results do not generalize to suicide attempts or deaths. Instead, our study focused on the INQ given the clear theoretical relation between TB, PB, and concurrent desire for death and desire for suicide. Future studies should test the clinical utility of assessments of the capability for suicide as a predictor of suicidal behaviors. Furthermore, meta-analytic studies demonstrate that a small set of factors may not serve as meaningful predictors of suicide (Franklin et al., 2017), and large sets of available data and machine learning algorithms may offer the most predictive value (McCarthy et al., 2015; Walsh, Ribeiro, & Franklin, 2017). Therefore, we do not suggest the sole use of the INQ in place of a comprehensive suicide risk assessment for prediction purposes.

Conclusions

The current study sought to extend the clinical utility of the ITS, specifically the 15-item version of the INQ, an assessment of TB and PB, to better understand risk for concurrent desire for death and desire for suicide in outpatient clinical populations. Results of the study provide clear recommendations for the use of the INQ in psychiatric outpatient settings. Specifically, this article demonstrates the particular importance of assessing PB – in comparison to TB – in determining risk for concurrent desire for death and desire for suicide among psychiatric outpatients, whereas in previous studies TB and PB appear to have similar predictive strength among higher-risk samples (i.e., psychiatric inpatients admitted for elevated suicide risk; Mitchell et al., 2017). Data also depict how the interpretation of the INQ may be altered as a function of a patient’s reported gender. Additionally, this study provides clinically-relevant and tangible results in the form of estimated probability graphs and clinical cutoff scores. This work is integral to help translate over a decade of empirical support for the ITS to meaningful clinical practices that enhance suicide prevention.