Abstract
The present study examines the psychometric properties and construct validity of scores derived from the Interpersonal Needs Questionnaire (INQ) using latent variable modeling with five independent samples varying in age and level of psychopathology. The INQ was derived from the Interpersonal Theory of Suicide and was developed to measure thwarted belongingness and perceived burdensomeness—both proximal causes of desire for suicide. Results support that thwarted belongingness and perceived burdensomeness are distinct, but related constructs and that they can be reliably measured. Further, multiple group analyses were consistent with invariance for younger vs. older adults and non-clinical versus clinical populations thereby supporting the relevance of these constructs to diverse populations. Finally, both constructs demonstrated convergent associations with related interpersonal constructs—including loneliness and social support for belongingness and social worth and death ideation for burdensomeness—as well as prospective associations with suicidal ideation.
Keywords: suicide, thwarted belongingness, perceived burdensomeness, need to belong, interpersonal theory of suicide, Interpersonal Needs Questionnaire
One of the most striking and consistent epidemiological findings regarding suicide is that only a small subset of those who think about suicide go on to attempt, and even fewer will die by suicide—this finding has been replicated worldwide and over time (World Health Organization, 1998). Mental disorders dramatically elevate risk for suicide (Cavanagh, Carson, Sharpe, & Lawrie, 2003), yet the vast majority of individuals with mental disorders will not attempt or die by suicide, including those with major depression (Bostwick & Pankratz, 2000). The National Institute of Mental Health has proposed that shifting from the current categorical classification system of psychopathology (i.e., DSM-IV) to a neuroscience-based emphasis on pathophysiology will improve treatment outcome and prevention studies research (Insel et al., 2010). Consistent with this proposal, we suggest that a better understanding of suicide will emerge from a perspective emphasizing interpersonal and behavioral processes in addition to discrete diagnoses, such as depression. Specifically, in the framework of the interpersonal theory of suicide (T. Joiner, 2005; Van Orden et al., 2010), we propose that desire for suicide results from unmet interpersonal needs: an unmet need to belong (Baumeister & Leary, 1995; Cacioppo & Patrick, 2008) results in thwarted belongingness, and an unmet need for social competence (Ryan & Deci, 2000) results in perceived burdensomeness. When both thwarted belongingness and perceived burdensomeness are present, according to the theory, desire for suicide develops, which manifests behaviorally as active (versus passive) suicidal ideation.
Empirical tests of the theory’s hypotheses regarding suicidal desire have supported the associations between these constructs and suicidal ideation and behavior (Davidson, Wingate, Rasmussen, & Slish, 2009; T. Joiner et al., 2002; T. E. Joiner, Jr., Hollar, & Van Orden, 2006; Thomas E Joiner et al., 2009; Nademin et al., 2008; Van Orden, Lynam, Hollar, & Joiner, 2006; Van Orden, Witte, Gordon, Bender, & Joiner, 2008; Van Orden et al., 2008). The current series of studies focus on the development of a self-report instrument designed to measure the constructs of thwarted belongingness and perceived burdensomeness. This instrument—the Interpersonal Needs Questionnaire (INQ)—was developed by the authors for use by researchers in the investigation of the etiology of suicidal desire/behavior, as well as by clinicians as part of a risk assessment framework grounded in the theory (Thomas E. Joiner, Van Orden, Witte, & Rudd, 2009). By assessing thwarted belongingness and perceived burdensomeness, clinicians are able to use crisis intervention strategies to directly target these risk factors and thereby potentially decrease suicidal desire. Further, when thwarted belongingness and perceived burdensomeness are absent, this gives clinicians information about protective factors, such as social support, which are important elements of risk assessment frameworks (Gutierrez & Osman, 2008).
According to the theory, thwarted belongingness is a psychologically-painful mental state that results when the fundamental need for connectedness—described by Baumeister and Leary (1995) as the “need to belong” (p.1)—is unmet (see also, Cacioppo & Patrick, 2008). The theory proposes that the various indices of social isolation that are associated with suicide—living alone (Heikkinen, Aro, & Lönnqvist, 1994), loneliness (Koivumaa-Honkanen et al., 2001), and low social support (Qin & Nordentoft, 2005; Sourander et al., 2009; Turvey et al., 2002)—are associated with suicide across the lifespan because they are indicators that the need to belong has been thwarted.
Perceived burdensomeness is a mental state characterized by apperceptions that others would “be better off if I were gone,” which manifests when the need for social competence that is posited by frameworks including Self-Determination Theory (Ryan & Deci, 2000) is unmet. The theory proposes that family discord (Duberstein, Conwell, Conner, Eberly, & Caine, 2004; Heikkinen, et al., 1994), unemployment (G. K. Brown, Beck, Steer, & Grisham, 2000; Heikkinen, et al., 1994), and functional impairment (Conwell et al., 2010; Conwell et al., 2000) are associated with suicide across the lifespan because these factors are likely to engender perceptions of burdensomeness on others.
Thwarted belongingness and perceived burdensomeness are proposed to be the most proximal mental states that precede the development of thoughts of suicide—stressful life events, mental disorders, and other risk factors for suicide are relatively more distal in the causal chain of risk factors for suicide. Further, thwarted belongingness and perceived burdensomeness are posited to be dynamic and amenable to therapeutic change.
Implicit in our discussion thus far is the interpersonal theory’s assumption that thwarted belongingness and perceived burdensomeness are distinct, but related constructs – that they occupy neighboring, but separate spaces in the nomological net for suicidal behavior (cf., Cronbach & Meehl, 1955), and thus, that they are non-redundant constructs. Empirical findings are supportive, with prior studies indicating a significant correlation of moderate magnitude between the two constructs as measured by the INQ (e.g., zero order correlation coefficient of .58; Van Orden, Witte, Gordon, et al., 2008). However, this assumption is in need of more rigorous testing given the limited number of samples that have investigated psychometric properties of the INQ, problems with multicollinearity that necessitated use of a subset of the original item pool (Van Orden, Witte, Gordon, Bender, & Joiner, 2008), and problems with administration length that necessitated administering a subset of items (Bryan et al., 2010).
As mentioned above, empirical tests of the theory have also supported the theory’s hypothesized relations between suicidal desire, thwarted belongingness, and perceived burdensomeness among diverse samples. In studies using the INQ, with samples of both young adults (Van Orden, Witte, Gordon, et al., 2008; Van Orden, Witte, James, et al., 2008) and older adults (Cukrowicz, Cheavens, Van Orden, Ragain, & Cook, 2010), thwarted belongingness and perceived burdensomeness have been shown to be associated with severity of concurrent suicidal ideation, thus supporting the concurrent validity of the INQ. Second, in a sample of psychiatric outpatients, higher levels of perceived burdensomeness have been shown to be concurrently associated with higher levels of clinician-rated risk for suicide (thwarted belongingness data were unavailable; (Van Orden, Witte, Gordon, et al., 2008). Third, the constructs as measured by a shorter version of the INQ (i.e. items were dropped to decrease the administration length) were found to have high sensitivity and specificity in the detection of current suicidal ideation among active duty military personnel (Bryan, Cukrowicz, Joiner, & Cornette, 2010). Prior to the current publication, psychometric problems with the INQ, such as multicollinearity and administration length, were handled on an individual study basis, thus potentially relying on “local” rather than generalizable solutions. The current series of studies examining the psychometric properties of the INQ with multiple samples is designed to address these problems, and to provide a revised and psychometrically sound version of the INQ. The psychometric methods used in the current paper have not previously been used with the INQ.
The current series of studies tests the theory’s definitions of—and hypotheses for—thwarted belongingness and perceived burdensomeness by examining the construct validity of scores derived from the INQ. The INQ measures beliefs about the extent to which individuals believe their need to belong is met or unmet (i.e., thwarted belongingness) and the extent to which they perceive themselves to be a burden on the people in their lives (i.e., perceived burdensomeness). Given that the theory proposes that these constructs are related, but distinct, our first aim was to develop a measurement model for the underlying latent structure of the INQ – do two factors best capture this latent structure (as posited by theory) and if so, which items are the most reliable and relatively pure indicators of these latent constructs? This aim involves examining the theory’s definitions of thwarted belongingness and perceived burdensomeness.
A related aim involves examining the reliability and generalizability of the constructs – do the same definitions of thwarted belongingness and perceived burdensomeness hold across variables such as age and clinical severity? We predict measurement invariance across age and clinical severity because the theory includes the falsifiable assumption that thwarted belongingness and perceived burdensomeness are the proximal causes of all forms of suicidal desire regardless of the population being studied. Thus, examining the number and nature of the latent factors underlying the INQ, as well as the degree to which measurement invariance holds, will constitute tests of the theory’s hypotheses and assumptions regarding the etiology of suicidal desire.
Finally, a third aim involves examining convergent, divergent and criterion (both concurrent and predictive) validity. This aim involves empirically examining the degree to which thwarted belongingness and perceived burdensomeness relate to similar constructs (and do not relate to other constructs), as hypothesized by the theory’s definitions of the constructs of thwarted belongingness and perceived burdensomeness (i.e. a pattern of inter-relations supporting convergent and divergent validity), and the degree to which the constructs are associated with and predict the primary outcome of interest – desire for suicide (i.e. concurrent and predictive validity). Each of these three aims is addressed in a separate section below, followed by implications for our understanding of the etiology of suicide.
Aim 1: Development of a Measurement Model with EFA
In this section, a series of exploratory factor analyses (with confirmatory elements) are described that involve examining the construct validity of the theory’s definitions of thwarted belongingness and perceived burdensomeness—a process operationalized as an investigation of the latent structure of the INQ. Doing so involves creating a latent measurement model of the original set of 25 INQ items – these items function as observed indicators of two latent variables, thwarted belongingness and perceived burdensomeness. This initial stage – the development of a latent variable measurement model – involves the examination of the following question: Can two latent variables, thwarted belongingness and perceived burdensomeness, be uncovered and measured using the theory’s definitions of the constructs (i.e., the INQ items)? Further, previous studies have demonstrated the need for refinement of the INQ as the full set of 25 items has been found to result in a high degree of multicollinearity between the two subscales (Van Orden, Witte, Gordon, et al., 2008). Thus, the aim for this study is to further refine the INQ by determining which items (out of the original 25-item pool) are the most reliable and relatively pure indicators of the constructs of belongingness and perceived burdensomeness.
The research strategy used in this section follows recommendations by Muthén and Muthén (2008) and Brown (2006) for the use of latent variable modeling in the process of instrument development. Both sets of recommendations include a sequential process starting with a series of exploratory factor analyses (EFA) and ending with a confirmatory factor analysis (CFA). Although scale development of the INQ has been thoroughly grounded in theory such that a measurement model could be hypothesized and examined within a confirmatory framework, at this stage in construct validation and measurement development, the emphasis rests more heavily on the investigation of the construct validity of the individual INQ items, a task most suitable for an empirical exploratory framework, rather than formal hypothesis testing using a confirmatory framework. However, given that this analysis is theory driven and utilizes hypothesis testing, the current analysis also utilizes elements of a confirmatory framework including examination of standard errors for loadings and the use of modification indices (Muthén & Muthén, 2008). The current section includes results from two EFA’s using two independent samples of young adults.
Methods
Measures
Interpersonal Needs Questionnaire (INQ)
The INQ was developed by the authors for use by researchers in the investigation of the etiology of suicidal desire/behavior, as well as by clinicians as part of a risk assessment framework grounded in the interpersonal theory of suicide (Thomas E. Joiner, et al., 2009). The original version of the INQ used in the current analysis contains 25 items: 10 items measure thwarted belongingness (e.g., “These days other people care about me,” reversed) and 15 items measure perceived burdensomeness (e.g., “These days I feel like a burden on the people in my life”). To complete the INQ, participants indicate the degree to which each item is true for them recently (on a 7 point Likert scale). Scores are coded such that higher numbers reflect higher levels of thwarted belongingness and perceived burdensomeness. In a previous study using a subset of the total item pool in order to manage multicollinearity (Van Orden, Witte, Gordon, et al., 2008), comparable internal consistency coefficients were found for the belongingness items (α= .85) and the perceived burdensomeness items (α = .89).
Items were rationally derived from the hypotheses of the interpersonal theory. For the belongingness subscale, item content also heavily drew from Baumeister and Leary’s (1995) definition of the need to belong, including their proposal that to meet this need, individuals must have frequent, proximal, and positive social interactions. Additional items were adapted from a self-report scale that measures a related construct, Inclusionary Status (Leary, Terdal, Tambor, & Downs, 1995). In particular, the items “These days, I rarely interact with people who care about me,” (reversed) “These days, I am fortunate to have many caring and supportive friends,” and “These days, I feel unwelcome in most social situations” (reversed) were modified only slightly for use in the INQ. The content for a perceived burdensomeness item, “These days, I matter to the people in my life” (reversed), was adapted from the content of items in the Mattering to Others Questionnaire (Marshall, 2001).
Study Procedures
These studies were approved by the Florida State University Institutional Review Board. Responses to the questions about suicide were screened by the experimenters for severe and imminent suicide risk according to routinized procedures described by Joiner, Walker, Rudd, and Jobes (1999). Procedures were in place to escort students to the FSU Psychology Clinic and/or utilize emergency mental health services in the event of imminent risk; however, no participants presented with imminent risk. All participants were debriefed and given phone numbers for local mental health services.
EFA Model Specification
An exploratory factor analysis with the 25 INQ items as observed variables was conducted using MPlus Version 5.1. The items on the INQ are continuous variables, but the majority of the perceived burdensomeness items are not normally distributed. As these items were designed to measure constructs that are not presumed to be normally distributed in the population (i.e., measuring a rare phenomenon), transformations were not used. Rather, Robust Maximum Likelihood (MLR) was used as this is an estimation procedure that is robust to violations of non-normality (T. A. Brown, 2006). EFA conducted with maximum likelihood estimation provides advantages over EFA conducted with estimators that are not full information estimators (e.g., Principal Factors) because it provides standard errors for factor loadings, as well as overall fit indices, characteristics typically associated with CFA. MLR with a Geomin oblique rotation was used as it is recommended when factor indicators demonstrate substantial loadings on more than one factor (Muthén & Muthén, (1998–2007)). As one of the primary aims of this analysis is to handle the problem of presumed cross-loadings (i.e., identify “pure” indicators), this rotation was deemed appropriate.
As each index of model fit has unique properties that yield strengths and weaknesses, several fit indices will be considered—two indices of absolute fit, chi-square (χ2) and standardized root mean square residual (SRMR); two indices of comparative fit, the Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI); and a parsimony corrected fit index, the root mean squared error of approximation (RMSEA). When cutoff criteria are available, the following recommendations are used: CFI and TLI values between .90–.95 represent the lower bound of potentially acceptable fit (Bentler, 1990) and SRMR and RMSEA values less than .08 suggest adequate fit (Browne & Cudeck, 1992; Hu & Bentler, 1999). The fit statistics reported are those associated with the MLR estimator in MPlus. The chi square statistic reported in these analyses is the Yuan-Bentler scaled chi-square, which is a chi-square test of overall model fit for continuous non-normal outcomes that involves dividing the standard chi-square value by a scaling correction factor to adjust for non-normality.
Participants and Input Data
Sample One Characteristics
Participants were 312 undergraduates recruited through General Psychology courses who received course credit for participation. The majority of the sample was female (74%). Mean age was 19 years (range: 17 to 51 years). Data on the race and ethnicity of participants were not available; however, this sample was likely representative of the FSU Psychology Department subject pool which has the following racial breakdown: 64% White, 28% Black or African American, 5% American Indian/Alaska native, and 3% Asian. The subject pool is 10% Hispanic/Latino. Regarding socioeconomic status, approximately 26% of the undergraduate population meets federal criteria for financial need status. Data from these participants have been previously published (Van Orden, Witte, Gordon, et al., 2008); however, the analysis below is novel and does not appear in prior publications.
Sample Two Characteristics
This new sample is a randomly selected half of a larger sample of 912 undergraduates at FSU; the sample was randomly split in half in order to better examine the reliability of the INQ measurement model. The first half of the sample was used for the current EFA, and the second half was used for a CFA (described below). The first half of the sample consists of 456 undergraduates at Florida State University who received course credit for participation. The majority of the sample was female (64%). Mean age was 18.83 years (std. 1.274; range: 18 to 29 years). Data on the self-identified race of participants are as follows: 80% Caucasian, 9% Black or African American, 5% Asian, and 6% other race. Regarding ethnicity, 20% of the sample self-identified as Hispanic/Latino.
Description of Data
Descriptive statistics (i.e., n, mean, standard error, median, mode, standard deviation, variance, skew, kurtosis, range) for the individual INQ items for Sample One, as well as the inter-correlations among them (as EFA analyzes a correlation matrix) appear in Table 1. As these data are highly similar for Sample Two, they are not reproduced here; however, complete tables are available upon request. For all samples, all available data were used in estimation, with missing data handled by direct maximum likelihood (ML) using the robust maximum likelihood estimator (MLR). For all samples, examination of the missing data patterns indicated lowest covariance coverages well above the proposed minimum accepted value (Muthén & Muthén, (1998–2007)).
Table 1.
Descriptive Statistics and Inter-correlations among INQ Items for Sample 1
| Bel1 | Bel2 | Bel3 | Bel4 | Bel5 | Bel6 | Bel7 | Bel8 | Bel9 | Bl10 | Br1 | Br2 | Br3 | Br4 | Br5 | Br6 | Br7 | Br8 | Br9 | Br10 | Br11 | Br12 | Br13 | Br14 | Br15 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Bel1 | 1.00 | ||||||||||||||||||||||||
| Bel2 | 0.58 | 1.00 | |||||||||||||||||||||||
| Bel3 | 0.35 | 0.35 | 1.00 | ||||||||||||||||||||||
| Bel4 | 0.61 | 0.65 | 0.30 | 1.00 | |||||||||||||||||||||
| Bel5 | 0.54 | 0.70 | 0.37 | 0.54 | 1.00 | ||||||||||||||||||||
| Bel6 | 0.52 | 0.64 | 0.30 | 0.50 | 0.60 | 1.00 | |||||||||||||||||||
| Bel7 | 0.58 | 0.54 | 0.30 | 0.61 | 0.47 | 0.42 | 1.00 | ||||||||||||||||||
| Bel8 | 0.47 | 0.56 | 0.27 | 0.44 | 0.47 | 0.63 | 0.37 | 1.00 | |||||||||||||||||
| Bel9 | 0.64 | 0.57 | 0.40 | 0.64 | 0.51 | 0.49 | 0.64 | 0.44 | 1.00 | ||||||||||||||||
| Bl10 | 0.40 | 0.39 | 0.36 | 0.39 | 0.38 | 0.38 | 0.46 | 0.31 | 0.45 | 1.00 | |||||||||||||||
| Br1 | 0.48 | 0.38 | 0.23 | 0.33 | 0.37 | 0.39 | 0.37 | 0.53 | 0.43 | 0.30 | 1.00 | ||||||||||||||
| Br2 | 0.40 | 0.42 | 0.25 | 0.42 | 0.35 | 0.36 | 0.45 | 0.29 | 0.46 | 0.32 | 0.27 | 1.00 | |||||||||||||
| Br3 | 0.47 | 0.42 | 0.23 | 0.39 | 0.37 | 0.42 | 0.40 | 0.54 | 0.43 | 0.32 | 0.88 | 0.29 | 1.00 | ||||||||||||
| Br4 | 0.35 | 0.39 | 0.28 | 0.28 | 0.43 | 0.38 | 0.30 | 0.36 | 0.31 | 0.28 | 0.48 | 0.30 | 0.48 | 1.00 | |||||||||||
| Br5 | 0.38 | 0.22 | 0.25 | 0.24 | 0.25 | 0.21 | 0.32 | 0.38 | 0.34 | 0.31 | 0.46 | 0.22 | 0.45 | 0.28 | 1.00 | ||||||||||
| Br6 | 0.35 | 0.30 | 0.19 | 0.21 | 0.27 | 0.32 | 0.22 | 0.41 | 0.30 | 0.31 | 0.49 | 0.24 | 0.47 | 0.37 | 0.33 | 1.00 | |||||||||
| Br7 | 0.51 | 0.44 | 0.28 | 0.52 | 0.45 | 0.37 | 0.51 | 0.34 | 0.57 | 0.40 | 0.46 | 0.44 | 0.51 | 0.42 | 0.36 | 0.38 | 1.00 | ||||||||
| Br8 | 0.48 | 0.50 | 0.27 | 0.49 | 0.44 | 0.42 | 0.45 | 0.38 | 0.50 | 0.44 | 0.36 | 0.49 | 0.45 | 0.40 | 0.32 | 0.34 | 0.59 | 1.00 | |||||||
| Br9 | 0.26 | 0.21 | 0.15 | 0.15 | 0.24 | 0.25 | 0.23 | 0.37 | 0.20 | 0.14 | 0.65 | 0.14 | 0.65 | 0.41 | 0.36 | 0.42 | 0.26 | 0.32 | 1.00 | ||||||
| Br10 | 0.58 | 0.58 | 0.33 | 0.59 | 0.51 | 0.47 | 0.55 | 0.45 | 0.61 | 0.47 | 0.51 | 0.50 | 0.53 | 0.50 | 0.39 | 0.38 | 0.74 | 0.67 | 0.30 | 1.00 | |||||
| Br11 | 0.35 | 0.44 | 0.31 | 0.35 | 0.40 | 0.42 | 0.34 | 0.37 | 0.41 | 0.33 | 0.54 | 0.36 | 0.52 | 0.55 | 0.32 | 0.47 | 0.48 | 0.41 | 0.35 | 0.57 | 1.00 | ||||
| Br12 | 0.36 | 0.38 | 0.20 | 0.30 | 0.30 | 0.33 | 0.32 | 0.41 | 0.30 | 0.21 | 0.55 | 0.21 | 0.56 | 0.55 | 0.34 | 0.42 | 0.36 | 0.37 | 0.45 | 0.42 | 0.56 | 1.00 | |||
| Br13 | 0.40 | 0.45 | 0.27 | 0.41 | 0.37 | 0.38 | 0.39 | 0.34 | 0.47 | 0.31 | 0.29 | 0.73 | 0.33 | 0.35 | 0.25 | 0.30 | 0.46 | 0.58 | 0.29 | 0.53 | 0.40 | 0.28 | 1.00 | ||
| Br14 | 0.42 | 0.42 | 0.31 | 0.37 | 0.41 | 0.38 | 0.39 | 0.43 | 0.44 | 0.37 | 0.65 | 0.32 | 0.64 | 0.65 | 0.36 | 0.54 | 0.51 | 0.45 | 0.45 | 0.57 | 0.69 | 0.59 | 0.41 | 1.00 | |
| Br15 | 0.56 | 0.48 | 0.32 | 0.49 | 0.43 | 0.41 | 0.52 | 0.47 | 0.61 | 0.48 | 0.61 | 0.28 | 0.62 | 0.42 | 0.52 | 0.40 | 0.62 | 0.54 | 0.40 | 0.64 | 0.51 | 0.52 | 0.38 | 0.58 | 1.00 |
| N | 312 | 312 | 312 | 312 | 312 | 312 | 312 | 312 | 312 | 312 | 312 | 312 | 312 | 312 | 311 | 311 | 310 | 310 | 312 | 311 | 312 | 312 | 311 | 311 | 311 |
| Mean | 2.26 | 3.06 | 2.08 | 2.21 | 3.23 | 3.18 | 1.86 | 2.25 | 2.45 | 2.12 | 1.50 | 3.38 | 1.53 | 2.27 | 1.86 | 1.54 | 2.47 | 2.57 | 1.26 | 2.36 | 1.94 | 1.44 | 3.40 | 1.74 | 1.74 |
| SE | 0.07 | 0.10 | 0.08 | 0.09 | 0.10 | 0.10 | 0.08 | 0.08 | 0.09 | 0.09 | 0.06 | 0.09 | 0.06 | 0.09 | 0.08 | 0.06 | 0.08 | 0.09 | 0.05 | 0.08 | 0.08 | 0.06 | 0.10 | 0.07 | 0.07 |
| SD | 1.27 | 1.70 | 1.42 | 1.52 | 1.80 | 1.84 | 1.34 | 1.46 | 1.58 | 1.50 | 1.08 | 1.55 | 1.12 | 1.61 | 1.42 | 1.09 | 1.41 | 1.54 | 0.86 | 1.39 | 1.41 | 1.04 | 1.68 | 1.26 | 1.15 |
| Var | 1.61 | 2.90 | 2.02 | 2.32 | 3.22 | 3.38 | 1.79 | 2.14 | 2.49 | 2.26 | 1.16 | 2.40 | 1.25 | 2.59 | 2.01 | 1.20 | 2.00 | 2.36 | 0.74 | 1.93 | 2.00 | 1.09 | 2.81 | 1.60 | 1.31 |
| Skew | 0.82 | 0.68 | 1.51 | 1.27 | 0.50 | 0.55 | 1.85 | 1.41 | 0.95 | 1.47 | 2.88 | 0.37 | 2.70 | 1.34 | 2.20 | 2.72 | 1.15 | 1.03 | 4.53 | 1.19 | 1.72 | 3.29 | 0.27 | 2.11 | 2.02 |
| Kurt | 0.02 | −0.4 | 1.87 | 0.86 | −0.7 | −0.8 | 3.11 | 1.48 | 0.04 | 1.65 | 9.25 | −0.5 | 7.72 | 0.98 | 4.67 | 8.06 | 1.22 | 0.55 | 22.78 | 1.24 | 2.34 | 12.17 | −0.7 | 4.28 | 4.56 |
| Range | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 | 6.00 |
Results & Discussion
First, we examined eigenvalues, difference in eigenvalues using parallel analysis, fit statistics, and the pattern of factor loadings to determine the number of factors that best describe the latent structure of the INQ. For both samples, results yielded four eigenvalues greater than one (see Table 2) suggesting the presence of a multidimensional latent structure, with up to a 4-factor model potentially providing a good fit to the data. Parallel analysis of eigenvalues using the “fapara” program for STATA (UCLA: Academic Technology Services, 2011) indicated retention of two factors in Sample One and two or three factors in Sample Two (see Table 2 and Figure 1). Tests of model fit are also presented in Table 2 and, in line with the eigenvalues and parallel analysis, suggest that the one-factor model should be rejected. Model fit statistics indicate the most consistent support in both samples for a four-factor model, with some evidence of adequate fit from both the standardized root mean square residual (SRMR) and root mean square error of approximation (RMSEA) for the two and three factor models. Examination of factor loadings for both samples indicated that the three-factor models were uninterpretable, and for the four-factor model, three of the four factors had fewer indicators per factor than is recommended to ensure the stability and viability of the factor (i.e., 4 indicators) (Muthén & Muthén, 2008). Further, three and four factor models were not predicted theoretically. Thus, only the two-factor model was retained as viable for both samples. Loadings for the three and four factor models are available in the supplemental material (Tables S1–S4).
Table 2.
Fit statistics for EFA/CFA models
| Chi-square | CFI | TLI | RMSEA (90% CI) | SRMR | Eigen | PAdif | ||
|---|---|---|---|---|---|---|---|---|
| Sample One (n = 312) | ||||||||
| Original Model: 25 Items | 1 | 1223.250 (df 275), p<0.001 | 0.709 | 0.683 | 0.105 (0.099 0.111) | 0.083 | 11.163 | 9.276 |
| 2 | 755.827 (df 251), p < 0.001 | 0.845 | 0.815 | 0.080 (0.074 0.087) | 0.052 | 2.273 | 1.502 | |
| 3 | 622.750 (df 228), p < 0.001 | 0.879 | 0.841 | 0.074 (0.068 0.082) | 0.043 | 1.263 | 0.398 | |
| 4 | 428.696 (df 206), p < 0.001 | 0.932 | 0.901 | 0.059 (0.051 0.067) | 0.033 | 1.135 | 0.321 | |
| Revised Model: 15 Items | 1 | 618.216 (df 90), p < 0.001 | 0.667 | 0.611 | 0.137 (0.127 0.147) | 0.098 | 6.864 | -- |
| 2 | 1.68.883 (df 76), p < 0.001 | 0.941 | 0.919 | 0.063 (0.050 0.075) | 0.037 | 1.996 | -- | |
| 3 | 105.889 (df 63), p < 0.001 | 0.973 | 0.955 | 0.047 (0.031 0.062) | 0.029 | <1.000 | -- | |
| 4 | 72.384 (df 51), p < 0.05 | 0.987 | 0.972 | 0.037 (0.013 0.055) | 0.020 | <1.000 | -- | |
| Sample Two (n = 456) | ||||||||
| Original Model: 25 Items | 1 | 1212.079 (df 275), p<0.001 | 0.704 | 0.677 | 0.086 (0.082 0.091) | 0.081 | 10.100 | 8.609 |
| 2 | 936.184 (df 251), p<0.001 | 0.784 | 0.784 | 0.784 (0.072 0.083) | 0.059 | 2.059 | 1.060 | |
| 3 | 634.578 (df 228), p<0.001 | 0.872 | 0.872 | 0.063 (0.057 0.068) | 0.044 | 1.610 | 0.947 | |
| 4 | 444.011 (df 206), p<0.001 | 0.925 | 0.891 | 0.050 (0.044 0.057) | 0.031 | 1.249 | 0.398 | |
| Revised Model: 15 Items | 1 | 385.172 (df 90), p<0.001 | 0.761 | 0.721 | 0.085 (0.076 0.094) | 0.084 | 6.489 | -- |
| 2 | 202.074 (df 76), p<0.001 | 0.898 | 0.859 | 0.060 (0.050 0.070) | 0.042 | 1.746 | -- | |
| 3 | 144. 434 (df 63), p < 0.001 | 0.934 | 0.890 | 0.053 (0.042 0.065) | 0.031 | <1.000 | -- | |
| 4 | 68.483 (df 51), p < 0.10 | 0.986 | 0.971 | 0.027 (0.000 0.043) | 0.019 | <1.000 | -- | |
| Sample Three (n = 456) | ||||||||
| 211.422 (df 88), p < 0.001 | 0.912 | 0.895 | 0.055 (0.046 0.065) | 0.052 | -- | -- | ||
| Sample Four | ||||||||
| 283.064 (df 88), p < 0.001 | 0.915 | 0.898 | 0.075 (0.065 0.085) | 0.060 | -- | -- | ||
| Sample Five (n = 265) | ||||||||
| 181.847 (df 88), p < 0.001 | 0.864 | 0.838 | 0.063 (0.050 0.076) | 0.072 | -- | -- | ||
Note. Eigen = eigenvalue; PAdiff = Difference in eigenvalues using Parallel Analysis
Figure 1.
Difference in Eigenvalues from Parallel Analysis
The pattern of loadings for this model in Sample One (see Table 3) indicates that nine of the ten items written to measure thwarted belongingness clearly loaded onto a “belongingness” factor and that seven of the 15 items written to measure perceived burdensomeness loaded onto the “burdensomeness” factor, supporting the content validity of the INQ’s latent structure. Several items did not perform well: four of the items written to measure perceived burdensomeness loaded onto the “belongingness” factor and four items did not clearly load onto either factor using criteria proposed by Muthén and Muthén (2008) that loadings should be two times greater than loadings on another factor (as well as statistically significant). In Sample Two, all ten of the items written to measure belongingness loaded on a “belongingness” factor, and eight of the 15 items written to measure perceived burdensomeness loaded onto the “burdensomeness” factor. As with Sample One, several items written to measure burdensomeness loaded on the “belongingness” factor, however all items clearly loaded onto one of the factors. Items were considered poor factor indicators if they did not load onto the a priori specified factor or did not clearly load onto the specified factor in both samples. For both samples, fit indices fell outside established cutoff values. Table 3 indicates which items were dropped before re-running the EFA in both samples to examine the reliability of the revised model.
Table 3.
Standardized EFA Factor Loadings for Two-factor Models for Samples One and Two
| Sample One | Sample Two | |||||
|---|---|---|---|---|---|---|
| Item # | Item Content | Belong | Burden | Belong | Burden | Action |
| BEL1 | Others care about me | 0.675 | 0.110 | 0.706 | 0.071 | |
| BEL2 | I feel like I belong | 0.786 | −0.030 | 0.817 | −0.003 | |
| BEL3 | Rarely interact w/others who care | 0.447 | 0.008 | 0.231 | 0.132 | |
| BEL4 | Many supportive friends | 0.826 | −0.112 | 0.703 | 0.026 | |
| BEL5 | Disconnected from others | 0.693 | 0.004 | 0.563 | 0.120 | |
| BEL6 | Outsider at gatherings | 0.608 | 0.083 | 0.560 | 0.057 | |
| BEL7 | People to turn to | 0.725 | 0.014 | 0.683 | 0.001 | |
| BEL8 | Unwelcome in social situations | 0.374 | 0.352 | 0.515 | 0.104 | Drop |
| BEL9 | Close to others | 0.785 | −0.001 | 0.854 | −0.216 | |
| BEL10 | Daily satisfying interaction | 0.555 | 0.024 | 0.653 | −0.100 | |
| BUR1 | Better off if I were gone | −0.037 | 0.948 | 0.103 | 0.743 | |
| BUR2 | Give back to society | 0.647 | −0.078 | 0.597 | −0.086 | Drop |
| BUR3 | Happier without me | 0.024 | 0.901 | −0.044 | 0.837 | |
| BUR4 | Failed people in my life | 0.265 | 0.420 | 0.101 | 0.617 | Drop |
| BUR5 | Miss me if I went away | 0.174 | 0.408 | 0.391 | 0.116 | Drop |
| BUR6 | Burden on society | 0.139 | 0.478 | −0.002 | 0.634 | |
| BUR7 | Asset to people in my life | 0.618 | 0.173 | 0.547 | 0.094 | Drop |
| BUR8 | I make a difference | 0.664 | 0.068 | 0.705 | −0.007 | Drop |
| BUR9 | My death a relief to others | −0.172 | 0.803 | 0.072 | 0.576 | |
| BUR10 | Contribute to others' well-being | 0.717 | 0.161 | 0.605 | 0.073 | Drop |
| BUR11 | Burden on the people in my life | 0.330 | 0.422 | 0.068 | 0.668 | Drop |
| BUR12 | Wish they could be rid of me | 0.131 | 0.563 | 0.087 | 0.580 | |
| BUR13 | Contribute to my community | 0.627 | −0.005 | 0.697 | −0.100 | Drop |
| BUR14 | Make things worse for others | 0.257 | 0.580 | −0.035 | 0.727 | |
| BUR15 | I matter to people in my life | 0.457 | 0.413 | 0.685 | 0.076 | Drop |
Note. Underlined loadings are statistically significant (critical value >1.96). Bolded loadings are > 2x’s larger than loadings on the other factor
After dropping poor indicators, the two-factor structure was recovered in both samples and the pattern of factor loadings did not change (see Supplemental Online material); further, model fit improved after dropping these items, with fit statistics falling within (or close to) established cutoff values. These results indicate that a two-factor latent structure with 15 items provides reliable and acceptable fit to the data. Finally, inspection of modification indices indicated that the fit of the model could be improved (drop in chi-square of 19.99) by allowing one pair of belongingness items to covary (i.e. “I feel disconnected from other people” and “I often feel like an outsider at social gatherings”). These items share content regarding social anxiety/exclusion above and beyond shared content of belongingness; thus in the CFA a residual covariance will be estimated for these items in the following section. These items were kept in the model because they both clearly load onto the belongingness, but not burdensomeness factor.
These results from two EFA’s suggest a measurement model for the INQ that consists of 15 INQ items, with 9 items loading exclusively on a “thwarted belongingness” factor and six items loading exclusively on a “perceived burdensomeness” factor. Further, these data suggest that the resultant measurement model consists of indicators of thwarted belongingness and perceived burdensomeness that should provide an adequate fit to the data when cross-loadings are removed in confirmatory factor analysis. These data support the hypothesized definition of these constructs as distinct but related constructs.
Aim 1: Examination of a Measurement Model with CFA
The next step to support construct validity involved using the results from the exploratory factor analyses described above to formulate and test the latent model in a confirmatory analysis framework using an additional independent sample of young adults (Sample Three). Unlike in EFA in which all indicators freely load onto both factors, in CFA, the parameterization of the model is fully specified a priori. Thus, the items deemed to be good indicators of thwarted belongingness in the above analyses were specified to only load onto this factor in the CFA, and similarly for perceived burdensomeness. This specification allows for an empirical test of the interpersonal theory’s assumption that thwarted belongingness and perceived burdensomeness are distinct (though related) constructs by examining the fit of a model with two latent constructs (allowed to co-vary) and the absence of cross-loadings (i.e., each INQ item is an indicator of either the thwarted belongingness latent variable or perceived burdensomeness latent variable). In this section, the model is initially fit with an undergraduate sample.
Then, the generalizability of the model is examined by fitting the model in a clinical sample and an older adult primary care sample, a key component in testing the construct validity of the INQ as the INQ was developed for use in investigating suicidal behavior, which is relatively uncommon in non-treatment-seeking young adults (who were used largely for convenience purposes). The vast majority of people who die by suicide (i.e., approximately 95%) have diagnosable mental disorders (Cavanagh, et al., 2003) – and it is quite possible that the remaining 5% have subclinical variants of mental disorders. Thus, it is crucial that the structure of the INQ be examined in the population of individuals with mental disorders (i.e., a clinical sample; Sample Four, psychiatric outpatients). Further, the age range of the participants in the young adult samples used to develop the measurement model is another notable characteristic that may limit generalizability of the psychometric properties. Suicide rates vary by age, with the highest suicide rate, in most areas of the world, among older adults (Gould, Shaffer, & Greenberg, 2003; Heron et al., 2009). Thus it is crucial that the structure of the INQ be examined with the population of older adults. In addition, the majority of older adults who die by suicide present for services in primary care settings (i.e., rather than mental health clinics; Luoma, Martin, & Pearson, 2002). Thus, an investigation of the psychometric properties of the INQ among older adults in a primary care setting (Sample Five) is highly relevant for suicide prevention efforts.
Method
Participants and Procedures
Sample Three Characteristics
This sample is the other half of the randomly divided sample used to form Sample Two. This sample consists of 456 undergraduates at Florida State University who received course credit for participation. The majority of the sample was female (67%). Mean age was 18.79 years (std. 1.47; range: 18 to 35 years). Data on the self-identified race of participants are as follows: 83% Caucasian, 9% Black or African American, 2% Asian, and 6% other race. Regarding ethnicity, 17% of the sample self-identified as Hispanic/Latino. Procedures for Sample Three are identical to that for the second EFA described above, as these participants are the randomly selected second half of that split sample.
Sample Four Characteristics
Sample Four consists of 397 adult clients from the Florida State University (FSU) Psychology Clinic, an outpatient community mental health center. This sample represented all new adult therapy cases admitted over the course of May 2007 to December 2008 who completed intake questionnaire packets including the INQ (missing data for individual items were allowed). The majority of the sample was female (57%). Mean age was 26.57 years (std. 9.85; range: 18 to 65 years). Data on the self-identified race/ethnicity of participants are as follows: 76% Caucasian, 10% Black or African American, 10% Hispanic/Latino, 3% Asian, and 1% other race. The vast majority of the sample was diagnosed with at least one Axis I mental disorder (93.2%). The Global Assessment of Functioning (GAF) ratings for the sample ranged from 35 to 90, spanning the “major impairment” to “minimal symptoms” ranges respectively, with the modal score (i.e., 60) falling within the “moderate” symptoms/functional impairment range. These data indicate that the current sample consists of individuals who evidence significant psychopathology of a moderate severity.
Sample Four Procedures
Upon application to the clinic, all clients were informed of the research and training nature of the clinic and signed a form consenting to their inclusion in research and to limits of confidentiality (including imminent risk for suicide). All measures were administered to clients with their application materials prior to treatment. All items from the 25 original item pool of the INQ were administered. All participants were debriefed and given phone numbers for local mental health services. The FSU Psychology Clinic primarily serves patients who present with clinical disorders typical of a community mental health outpatient clinic and employs minimal exclusionary criteria, excluding from treatment only people with psychotic or bipolar-spectrum disorders who do not take actions to become stabilized on medications.
Sample Five Characteristics
Sample Five consists of 265 healthy older adults. These data were collected over the course of several years (May, 2006 – December, 2009). Participants were recruited from one of two sources: a participant registry of older adults willing to participate in research through the Duke University Center for Aging and Human Development (n =57) or primary care settings within the Texas Tech University Health Sciences Center (n = 208). No significant differences on means for INQ items were observed as a function of data collection site. The majority of the sample was female (60%). Mean age was 72.83 years with a range of 57 to 93 that was roughly normally distributed in the sample (std. 7.49). The majority of the sample was married at the time of data collection (64%) and a significant minority were either widowed (12%) or divorced (15%); the remaining participants were living with a relationship partner but not married (2%), in a relationship but not living together (2%), separated (3%), or never married (3%). Data on the self-identified race/ethnicity of participants are as follows: 88% Caucasian, 2% Black or African American, 5% Hispanic, and 5% other race/ethnicity. The sample was, on the whole, highly educated, with 21% obtaining a high school diploma, over a quarter completing some college (26%), a quarter obtained a college degree (i.e., AA, BA, BS; 25%), and nearly a quarter obtained an advanced degree (i.e., MA, MS, Ph.D., MD; 22%). Exclusionary criteria included current manic symptoms or substance abuse, as well as cognitive/memory impairment or psychotic symptoms that impaired the participants’ ability to complete the study. A total of seven potential participants were excluded. Within included participants, 26% reported having been diagnosed with a mental condition or psychological disorder in the past. Eleven percent reported having been diagnosed within the last 12 months. Six participants reported at least one previous suicide attempt.
Sample Five Study Procedures
Participants recruited from the Duke University Center for Aging and Human Development were contacted by phone and provided a description of the study; participants were then scheduled to complete a one-hour research session in a research clinic. Participants recruited from primary care settings within the Texas Tech University Health Sciences Center were identified by a review of scheduled medical appointments for physicians within this setting. Identified patients 60 years and older were either approached during their medical appointment to determine interest in the study or were sent a letter describing the study that indicated that a member of the research staff would contact them to tell them more about the study and answer any questions. All participants consented to participate in the research. All participants were assessed for imminent suicide risk and procedures were followed to ensure immediate evaluation in a medical facility in the event of imminent risk. All participants were debriefed and given phone numbers for local mental health services. As with other samples, all items from the 25 original item pool of the INQ were administered.
Model Specification & Description of Data
Three separate confirmatory factor analyses were run using MLR (Mplus Version 5.1 software). These models were run with the 15 INQ items that were deemed to be the best indicators—through EFA procedures described above—of one of two latent variables, one representing thwarted belongingness and one representing perceived burdensomeness. Thus, the model includes 15 observed variables. Each observed variable is only an indicator of a single latent variable (i.e., either thwarted belongingness or perceived burdensomeness). The first factor loading for each latent variable was fixed to one to manage scale dependency. Two covariances were estimated (i.e., between the belongingness and perceived burdensomeness factors; between residual terms of two belongingness items).
Description of Data
Descriptive statistics for all three samples (i.e. mean, standard error, standard deviation, skew, variance) are provided for each INQ item in Table 4. As before, missing data were handled with direct maximum likelihood and for all samples missing data patterns indicated lowest covariance coverages well above the proposed minimum accepted value. As expected, greater variances in INQ items were observed for the clinical sample and to a lesser extent, the older adult sample, compared to young adult samples.
Table 4.
Descriptive Statistics for Samples Three, Four, Five
| Young Adults (Sample 3) | Clinical Outpatients (Sample 4) | Older Adults (Sample 5) | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Item # | Mean | SD | SE | Skew | Var | Mean | SD | SE | Skew | Var | Mean | SD | SE | Skew | Var |
| BEL1 | 1.656 | 1.047 | 0.049 | 2.132 | 1.097 | 2.652 | 1.735 | 0.087 | 0.813 | 3.010 | 1.773 | 1.154 | 0.070 | 1.611 | 1.333 |
| BEL2 | 2.169 | 1.331 | 0.062 | 1.356 | 1.771 | 3.907 | 2.082 | 0.105 | 0.034 | 4.333 | 2.003 | 1.320 | 0.080 | 1.513 | 1.743 |
| BEL3 | 1.717 | 1.326 | 0.062 | 2.386 | 1.759 | 2.713 | 1.91 | 0.096 | 0.837 | 3.650 | 2.249 | 1.839 | 0.112 | 1.442 | 3.382 |
| BEL4 | 1.746 | 1.110 | 0.052 | 1.804 | 1.232 | 3.177 | 2.051 | 0.103 | 0.497 | 4.206 | 1.926 | 1.494 | 0.091 | 1.831 | 2.233 |
| BEL5 | 2.228 | 1.474 | 0.069 | 1.182 | 2.172 | 3.718 | 2.165 | 0.109 | 0.132 | 4.688 | 2.074 | 1.624 | 0.099 | 1.513 | 2.636 |
| BEL6 | 2.418 | 1.540 | 0.072 | 1.001 | 2.371 | 3.708 | 2.177 | 0.109 | 0.157 | 4.738 | 2.572 | 1.710 | 0.104 | 0.789 | 2.925 |
| BEL7 | 1.697 | 1.070 | 0.05 | 2.143 | 1.146 | 3.114 | 2.031 | 0.102 | 0.566 | 4.126 | 1.788 | 1.303 | 0.079 | 2.214 | 1.697 |
| BEL9 | 2.054 | 1.311 | 0.061 | 1.518 | 1.718 | 3.592 | 1.96 | 0.099 | 0.133 | 3.843 | 2.238 | 1.558 | 0.095 | 1.399 | 2.428 |
| BEL10 | 1.874 | 1.198 | 0.056 | 1.712 | 1.435 | 3.470 | 2.063 | 0.104 | 0.244 | 4.255 | 2.156 | 1.566 | 0.095 | 1.447 | 2.453 |
| BUR1 | 1.298 | 0.811 | 0.038 | 3.994 | 0.658 | 1.823 | 1.446 | 0.073 | 1.853 | 2.090 | 1.361 | 1.055 | 0.064 | 3.392 | 1.112 |
| BUR3 | 1.268 | 0.755 | 0.035 | 4.099 | 0.570 | 1.768 | 1.347 | 0.068 | 1.985 | 1.815 | 1.275 | 0.942 | 0.057 | 4.152 | 0.887 |
| BUR6 | 1.379 | 0.861 | 0.040 | 3.209 | 0.741 | 2.106 | 1.584 | 0.80 | 1.434 | 2.509 | 1.375 | 1.035 | 0.063 | 3.531 | 1.071 |
| BUR9 | 1.123 | 0.516 | 0.024 | 6.727 | 0.266 | 1.574 | 1.258 | 0.063 | 2.472 | 1.583 | 1.268 | 0.899 | 0.055 | 4.315 | 0.809 |
| BUR12 | 1.276 | 0.803 | 0.038 | 3.941 | 0.644 | 1.748 | 1.419 | 0.071 | 1.999 | 2.012 | 1.338 | 1.065 | 0.065 | 4.009 | 1.135 |
| BUR14 | 1.371 | 0.865 | 0.041 | 3.239 | 0.748 | 2.401 | 1.827 | 0.092 | 1.163 | 3.337 | 1.401 | 0.944 | 0.058 | 2.904 | 0.890 |
Note. SD = standard deviation; SE = standard error of the mean; Var = variance
Results & Discussion
To examine the viability of the CFA model in each of the samples, fit statistics appear in the bottom of Table 2. Values for both the SRMR and RMSEA indicate adequate-to-good fit in all three samples (i.e. <.08). Values for both the comparative fit index (CFI) and Tucker-Lewis Index (TLI) approach or exceed cut-off values for acceptable fit (i.e., .90) in all samples, though these values are lowest for the older adult sample. While the chi-square test of model fit in all samples is inconsistent with adequate fit, this index is overly sensitive to sample size, and given the large samples used, chi-square may underestimate the degree of model fit (Brown, 2006).
Parameter estimates for the model – the estimated (unstandardized and standardized) factor loadings, covariances, residual covariances, and R-square values (i.e. communalities) are displayed in Table 5. In all three samples, all items significantly loaded onto the specified latent variable and r-square values range from .12 to .80, with most values falling somewhere in between, indicating moderate magnitudes (i.e., in the .30 – .60 range). These data suggest that both of the latent variables are adequately measured by the indicators included in the current model, and that all of these indicators are reasonably good indicators of the latent constructs of thwarted belongingness and perceived burdensomeness. Thus, in line with the model fit statistics, the results of model estimation suggest that a 15-item model provides a viable representation of the latent structure of the INQ across diverse samples, with two distinct, but related constructs, as posited by the interpersonal theory. The appendix lists these 15 items, as well as which items require re-coding so that higher summed scores on the INQ reflect greater levels of thwarted belongingness and perceived burdensomeness.
Table 5.
Model Estimated Factor Loadings, Covariances, Residual Covariances, R-square for Samples Three, Four, Five
| Young Adults (Sample 3) | Clinical Outpatients (Sample 4) | Older Adults (Sample 5) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Estimate | S.E. | Stand | R-sq | Estimate | S.E. | Stand | R-sq | Estimate | S.E. | Stand | R-sq | ||
| BELONG | 0.508 | 0.080 | -- | -- | 1.844 | 0.187 | -- | -- | 0.736 | 0.174 | -- | -- | |
| Bel1 | Others care | 1.000 | 0.000 | 0.681 | 0.464 | 1.000 | 0.000 | 0.784 | 0.614 | 1.000 | 0.000 | 0.745 | 0.555 |
| Bel2 | I belong | 1.488 | 0.124 | 0.797 | 0.636 | 1.242 | 0.075 | 0.812 | 0.659 | 1.272 | 0.108 | 0.828 | 0.686 |
| Bel3 | Rarely interact | 0.669 | 0.111 | 0.360 | 0.130 | 0.722 | 0.081 | 0.514 | 0.264 | 0.676 | 0.131 | 0.316 | 0.100 |
| Bel4 | Friends | 1.113 | 0.103 | 0.715 | 0.512 | 1.161 | 0.057 | 0.769 | 0.592 | 1.060 | 0.220 | 0.610 | 0.372 |
| Bel5 | Disconnected | 1.215 | 0.149 | 0.588 | 0.346 | 1.087 | 0.089 | 0.682 | 0.466 | 0.944 | 0.130 | 0.500 | 0.250 |
| Bel6 | Outsider | 1.221 | 0.146 | 0.565 | 0.319 | 0.899 | 0.095 | 0.562 | 0.315 | 0.867 | 0.125 | 0.436 | 0.190 |
| Bel7 | Turn to | 1.032 | 0.092 | 0.687 | 0.472 | 1.217 | 0.052 | 0.815 | 0.664 | 1.095 | 0.148 | 0.723 | 0.522 |
| Bel9 | Close to others | 1.461 | 0.120 | 0.795 | 0.633 | 1.265 | 0.063 | 0.877 | 0.770 | 1.340 | 0.165 | 0.739 | 0.547 |
| Bel10 | Daily interact | 1.034 | 0.131 | 0.616 | 0.379 | 1.092 | 0.072 | 0.720 | 0.519 | 1.233 | 0.154 | 0.677 | 0.458 |
| BURDEN | 0.469 | 0.124 | -- | -- | 1.575 | 0.220 | -- | -- | 0.608 | 0.200 | -- | -- | |
| Bur1 | Better off | 1.000 | 0.000 | 0.845 | 0.714 | 1.000 | 0.000 | 0.869 | 0.755 | 1.000 | 0.000 | 0.741 | 0.549 |
| Bur3 | Happier out me | 0.985 | 0.063 | 0.894 | 0.800 | 0.941 | 0.058 | 0.878 | 0.771 | 0.865 | 0.105 | 0.717 | 0.514 |
| Bur6 | Burden society | 0.509 | 0.118 | 0.405 | 0.164 | 0.864 | 0.078 | 0.685 | 0.470 | 0.701 | 0.134 | 0.529 | 0.280 |
| Bur9 | Death as relief | 0.428 | 0.105 | 0.569 | 0.323 | 0.740 | 0.081 | 0.739 | 0.546 | 0.866 | 0.175 | 0.752 | 0.565 |
| Bur12 | Rid of me | 0.801 | 0.076 | 0.684 | 0.468 | 0.898 | 0.071 | 0.796 | 0.633 | 0.839 | 0.175 | 0.615 | 0.379 |
| Bur14 | Makes worse | 1.000 | 0.073 | 0.792 | 0.628 | 1.014 | 0.089 | 0.698 | 0.487 | 0.580 | 0.240 | 0.480 | 0.231 |
| Covariances | |||||||||||||
| Bel w/ Burd | 0.327 | 0.066 | 0.670 | -- | 1.028 | 0.113 | 0.603 | -- | 0.341 | 0.091 | 0.510 | -- | |
| bel5 w/ bel 6 | 0.736 | 0.107 | 0.486 | -- | 1.554 | 0.213 | 0.547 | -- | 0.864 | 0.173 | 0.401 | -- | |
Note. The first item for each latent variable was set to 1.000 to handle scale dependency.
Aim 2: Reliability & Generalizability with Multiple Group Analysis
In this section, the three measurement models developed above are used in two multiple-group CFA models, with the young adult sample as the comparison group. These multiple group analyses allow an examination of whether older adults and those evidencing a greater degree of psychopathology (i.e. clinical outpatients) respond to items on the INQ in a manner that yields a comparable factor structure to responses from young adults. These analyses also attempt to answer the following: do the individual INQ items perform equally well as indicators of the latent constructs of thwarted belongingness and perceived burdensomeness in clinical outpatients and older adults as in a relatively healthier sample of young adults? In other words, do the theory’s definitions of thwarted belongingness and perceived burdensomeness work equally well across diverse populations?
Method
The first multiple-group CFA involves an investigation of the equivalence of the measurement model across levels of psychopathology by comparing undergraduates (Sample Three) and clinical outpatients (Sample Four). The second multiple-group CFA involves an investigation of the equivalence of the measurement model across age groups by comparing undergraduates (Sample Three) and older adults (Sample Five). For many forms of invariance/heterogeneity, the viability of subsequent models rests on the assumption of ‘less strict’ forms of invariance/homogeneity. Thus, the models were run sequentially until invariance/homogeneity no longer held. Examining invariance using multiple-group analysis involves running a series of CFA models, specifying multiple groups, with equality constraints specified to model increasingly strict forms of measurement invariance/heterogeneity, thereby forming nested models for which model fit can be compared. The primary test used for model comparison in the current project is chi square difference testing with the Yuan-Bentler scaled chi-square.
The presence of invariance/homogeneity is supported by a non-significant increase in the chi-square value with the addition of an equality constraint(s). However, chi-square difference testing is affected by sample size and thus unequal sample sizes between two groups used in multiple group CFA could bias results (T. A. Brown, 2006). Thus, the difference between the CFI values of the nested and comparison model will also be evaluated, as simulation studies with multiple-groups CFA suggest that the performance of this fit index for multiple group CFA is superior to that of chi-square as it appears to be more robust to differences in sample size and model complexity (Cheung & Rensvold, 2002), with a reduction of .01 or less consistent with presence of invariance/homogeneity.
Results & Discussion
Results appear in Table 6. Results for the model that impose the same factor structure for both groups—test of equal form—are consistent with adequate fit for both the young adults vs. outpatients model and the young vs. older adult model, supporting equivalent factor structure in populations of young adults, clinical outpatients, and older adults. The test of equivalent factor loadings involves imposing equality constraints on the unstandardized factor loadings across the model such that within the model, loadings are allowed to vary, but across groups, loadings are held to equivalent values. These models examine whether the INQ items perform equivalently across the populations. If indicators load more strongly onto the latent variable in one group, for example, this would indicate a lack of equivalent factor loadings. For both the young adults vs. outpatient model and the young vs. older adult model, the YB chi-square different test was non-significant, supporting equality of factor loadings. The CFI difference supported equality of factor loadings for clinical outpatients, but was equivocal for older adults. Finally, the test of equivalent intercepts involves the inclusion of mean structures. Data did not support equivalent intercepts for the young adults vs. clinical outpatients model, as these equality constraints significantly reduced model fit according to both the YB chi square difference test and change in CFI. Intercepts for some of the items differed for both the clinical sample and older adult sample compared to those for the undergraduates.
Table 6.
Tests of Measurement Invariance
| χ2 | df | MLR Scaling |
YB χ2diff | Δdf | CFI | Δ CFI |
TLI | RMSEA | SRMR | |
|---|---|---|---|---|---|---|---|---|---|---|
| Young Adults vs. Outpatients | ||||||||||
| Equal Form | 508.336* | 178 | 1.543 | -- | -- | 0.907 | -- | 0.891 | 0.066 (0.059 0.073) | 0.057 |
| Equal factor loadings | 550.330* | 191 | 1.581 | 19.985 | 13 | 0.899 | 0.008 | 0.889 | 0.066 (0.060 0.073) | 0.068 |
| Equal factor intercepts | 611.470* | 203 | 1.557 | 52.034* | 12 | 0.885 | 0.044 | 0.882 | 0.069 (0.062 0.075) | 0.070 |
| Young vs. Elder Adults | ||||||||||
| Equal Form | 394.636* | 178 | 1.635 | -- | -- | 0.898 | -- | 0.879 | 0.058 (0.050 0.066) | 0.060 |
| Equal factor loadings | 437.843* | 191 | 1.687 | 18.010 | 13 | 0.883 | 0.015 | 0.872 | 0.060 (0.052 0.067) | 0.075 |
| Equal factor intercepts | 486.222* | 203 | 1.644 | 50.417* | 12 | 0.866 | 0.017 | 0.862 | 0.062 (0.055 0.069) | 0.076 |
Note. YB χ2diff = Yuan-Bentler Scaled χ2 difference
p < .001
The multiple-group analyses described in this section were conducted to examine the generalizability of the theory’s definitions of thwarted belongingness and perceived burdensomeness. This was accomplished by examining the generalizability of the psychometric properties and latent structure of the INQ across populations varying in clinical severity and age, as both population parameters are known to impact suicide rates. Results of tests of equal form indicate equivalent latent structures of the INQ across populations of clinical outpatients and elder adults, suggesting that the measurement model developed in samples of undergraduates is also viable for populations demonstrating greater psychopathology and older ages. Further, both analyses provided support for equivalent factor loadings (though this evidence was less strong for the older adult sample), indicating that the INQ items demonstrate comparable relationships with the latent constructs of thwarted belongingness and perceived burdensomeness across clinical severity and older age. These results support our hypothesis of measurement invariance and suggest that the INQ items are appropriate indicators of these constructs for both younger and older adults and across levels of clinical severity. However, these conclusions should be tempered somewhat for the older adult sample given that one index of equivalence of factor loadings was consistent with equivalence while the other was equivocal; we suggest that future investigations with larger samples should seek to replicate the finding of measurement invariance in older adults. Further, older adults comprise a heterogeneous group; thus, future investigations should also examine invariance among the ‘younger old’ and ‘oldest old,’ as well among samples varying in parameters such as medical burden, functional impairment, and residence type (i.e., home in the community, assisted living, nursing home).
Finally, neither multiple group analysis indicated equivalence of intercepts; average values for several of the INQ items are greater in the clinical sample compared to the undergraduate sample and values are greater for some items in the older adult sample (compared to undergraduates) and lower for other values. These results indicate differential item functioning for some items, such that some items may artificially inflate or underestimate “true scores” for clinical and older adult populations. Thus, future research should investigate whether item weighting improves the precision with which the INQ items measure the latent constructs of thwarted belongingness and perceived burdensomeness items.
Aim 3: Convergent and Divergent Validity
In this section, the measurement model developed in the first section and examined across populations in the second section, was examined with regards to both convergent and divergent validity: two structural models were used to examine the pattern of relationships between these latent variables and related constructs. These structural equation models—one using younger adults and another with older adults—use the INQ measurement model as well as several outcome variables with definitions that, according to the interpersonal theory of suicide, should demonstrate discriminant relations with thwarted belongingness and perceived burdensomeness.
According to the theory, thwarted belongingness, loneliness, the self-determination theory concept of relatedness (Deci & Ryan, 2000; Ryan & Deci, 2000)1, and social support should occupy neighboring, but distinct positions in the nomological net for suicidal desire (i.e., related, but distinct constructs). These constructs can be conceptualized as measuring the higher order construct of social connectedness at varying levels of analysis (Berkman, Glass, Brissette, & Seeman, 2000), including intermediate levels (i.e., loneliness and social support) and the most micro level that presumably measures inner needs or states (i.e., belongingness and relatedness). Similarly perceived burdensomeness, responsibility to family (as a reason for living), competence, and autonomy should occupy neighboring, but distinct positions in the nomological net, as responsibility to family involves beliefs that one is an important contributor to the family (i.e., rather than a burden), and competence and autonomy—both self-determination theory constructs—indicate a sense of self-efficacy and mastery across many domains, including academics, work, etc.
Self-esteem is another construct that may occupy a neighboring position to thwarted belongingness and perceived burdensomeness in the suicidal desire nomological net. Sociometer theory (Leary & Baumeister, 2000) posits that fluctuations in global self-esteem serve as an internal monitoring system of interpersonal inclusion or exclusion (c.f., thwarted belongingness). The theory posits that individuals feel good about themselves when they perceive inclusion and feel bad about themselves when they perceive exclusion. Tafarodi and Swann (1995) suggest that global self-esteem consists of two dimensions: self-liking, which taps a sense of social worth, and self-competence, which taps a sense of social efficacy. It may be that self-liking functions as a monitor for the need to belong, while self-competence functions as a monitor for the need to contribute socially (i.e., level of perceived burdensomeness). Regarding the nomological net in question, the above analysis suggests the following discriminant relations: belongingness may be located near self-liking (but not self-competence), while perceived burdensomeness may be located near self-competence (but not self-liking). As social motivations may change as we age (Carstensen, Isaacowitz, & Charles, 1999), we tested these relations with self-esteem in the young adult sample only.
For older adults, we measured constructs designed to tap Carstensen and colleagues’ (Carstensen, et al., 1999) concept of “emotion motives”—the process of regulating emotions through interpersonal contact—including finding meaning in life and defining one’s personal and social worth. A robust literature has linked meaning in life among older adults with both social support (Charles & Carstensen, 2010; Krause, 2007) and loneliness (Golden et al., 2009), thus we proposed convergent relations between thwarted belongingness and meaning in life. A large literature has also linked perceptions of burden on others with loss of dignity and desire for death among older adults nearing the ends of their lives (McPherson, Wilson, & Murray, 2007), thus we proposed convergent relations between perceived burdensomeness and both death ideation and loss of personal and social worth, as the latter construct beliefs involves beliefs that one no longer contributes in a positive way to the well-being of oneself and others.
Method
Measures
Revised UCLA Loneliness Scale (Russell, Peplau, & Cutrona, 1980)
The UCLA Loneliness Scale is a 20-item self-report measure of perceptions of loneliness. Participants indicate the frequency with which they experience satisfaction and dissatisfaction with social relationships. For example, “I feel part of a group of friends,” and “I feel isolated from others.” Russell, Peplau, and Cutrona (1980) report high internal consistency for the scale (α = .94), as well as support for concurrent and divergent validity. This scale’s total score is used in the model below as an observed, continuous indicator of the “loneliness” construct. Higher numbers represent higher levels of loneliness, with scores ranging from 1–4. Internal consistency was high in both the young adult sample (α = .91) and older adult sample (α = .90).
Responsibility to Family subscale of the Reasons for Living Inventory (Linehan, Goodstein, Nielsen, & Chiles, 1983)
Seven items comprise the Responsibility to Family subscale and measure the degree of importance of reasons not to die by suicide. Osman and colleagues (1999) provide construct validity for the scale as well as internal consistency data (α = .93). This scale’s total score is used in the model below as an observed, continuous indicator of the “responsibility to family” construct. Higher numbers represent higher levels of responsibility tied to greater importance for staying alive, with scores ranging from 1 (not at all important) to 6 (very important). Internal consistency was high in both the young adult sample (α = .81) and older adult sample (α = .86).
Self-Liking/Self-Competence Scale (SLSC; Tafarodi & Swann, 1995)
The SLSC measures two facets of self-esteem, self-liking and self-competence. Participants rate how much they agree with 20 statements about self-feelings such that higher scores indicate greater levels of self-liking and self-competence. Scores range from 1–5. Comparable internal consistency coefficients were found in the current study for the self-liking items (α = .94) and the self-competence items (α = .91). Total scores for the subscales of the SLSC are used in the model as observed, continuous indicators of the constructs of “self-liking” and “self-competence.”
The Basic Need Satisfaction in Life Scale (Gagné, 2003)
This scale was designed to measure the three basic needs as posited by Self-Determination Theory—relatedness, competence, and autonomy—and was adapted from a domain-specific scale measuring these needs in the workplace (Ilardi, Leone, Kasser, & Ryan, 1993). For the present version, participants indicate the degree to which the abovementioned needs are “satisfied in their life.” Gagné (2003) reported adequate internal consistency coefficients for all three subscales. Total scores for the relatedness, competence, and autonomy subscales of this measure are used as observed, continuous indicators of the constructs of “relatedness” (α in current sample = .81), “competence” (α = .74) and “autonomy” (α = .60). Higher scores represent greater need fulfillment, with scores ranging from 1–7.
Belonging subscale of the Interpersonal Support Evaluation List (Cohen, Mermelstein, Kamarck, & Hoberman, 1985)
This scale was designed to measure perceptions of the availability of various facets of social support. The authors define social support as resources provided by others. The belonging support subscale consists of 10 items constructed to measure the degree to which individuals perceived that others are available and willing to do things with them. Sample items include, “If I decide on a Friday afternoon that I would like to go to a movie that evening, I could find someone to go with me,” “I regularly meet or talk with members of my family or friends,” and “When I feel lonely, there are several people I could call and talk to.” Cohen and colleagues (1985) provide data indicating adequate internal consistency, test-retest reliability, and correlation magnitudes with related measures. The total score for this measure is used as an observed, continuous indicator of the construct of “social support,” with higher scores indicating greater perceived social support and scores ranging from 1–7 (younger adults α = .83; older adults α = .80).
Geriatric Suicide Ideation Scale (Heisel & Flett, 2006)
The GSIS was designed to measure suicidal ideation and closely related constructs among older adults. Heisel and Flett (2006) describe data supporting a four-factor structure of the scale, as well as reliability and construct validity data supporting scores derived from these subscales. The meaning in life subscale total score is used as an observed, continuous indicator of the construct of “meaning in life,” with higher scores indicating lower meaning in life (α = .97). The perceived social worth subscale total score is used as an observed continuous indicator of the construct of “social worth,” with higher scores indicating lower perceived social worth (α = .82). The death ideation subscale is used as an observed, continuous indicator of the construct of “death ideation,” with higher scores indicating stronger and more frequent thoughts of one’s own death (α = .72).
Model Specifications
For the first model with young adults, a structural equation model with eight observed variables regressed onto the INQ measurement model was run using MLR: the first four were posited to relate to belongingness—loneliness, social support, self-liking, relatedness—while the remaining were posited to relate to burdensomeness—competence, autonomy, responsibility to family, self-competence. For the second model with older adults, six observed variables were regressed onto the INQ measurement model: the first three were posited to relate to belongingness—loneliness, social support, and (lower) meaning in life—while the remaining were posited to relate to burdensomeness—responsibility to family, (lower) social worth, and death ideation. All observed variables were regressed onto both latent variables in order to examine the magnitude and direction of regression coefficients, rather than an approach in which paths are added/deleted and model fit is compared. In addition, all observed variables in the structural part of the model were allowed to covary.
Input Data
Sample Characteristics & Procedures
Two separate samples are used in this section. The first sample consists of 602 undergraduates at Florida State University who received course credit for participation. These participants represent a subsample of participants from Samples Two and Three (i.e., the samples that were created by randomly splitting the overall sample) who completed the battery of measures described below. The other sample used in this section is the sample of older adults (i.e., Sample Five).
Results & Discussion
For the young adults model, standardized regression coefficients for the simultaneous regressions of the eight observed variables on the latent variables of thwarted belongingness and perceived burdensomeness appear in Figure 2. Supportive of convergent validity for thwarted belongingness, the relations between thwarted belongingness and the four constructs posited to be conceptually similar were statistically significant and in the expected directions. However, results did not support divergent validity as thwarted belongingness was also significantly associated with the four constructs posited to be conceptually similar to perceived burdensomeness. Perceived burdensomeness was significantly associated with three out of the four conceptually similar constructs and associated with only one of the constructs posited to be similar to thwarted belongingness, thereby largely supporting both convergent and divergent validity. However, with regards to the two types of self-esteem, self-liking and self-competence, discriminant relations were not observed: both thwarted belongingness and perceived burdensomeness were associated with both types of self-esteem.
Figure 2.
Structural Equation Model Results Examining Discriminant Relations in Younger Adults
Lonely = UCLA Loneliness Scale; Social Support = Belong support subscale of the Interpersonal Support Evaluation List; Self-Liking = Self-Liking subscale of the Self-Liking/Self-Competence scale; Relatedness, Competence, Autonomy = same-named subscales of the Basic Need Satisfaction in Life Scale; Responsibility to Family = subscale of the Reasons for Living Scale; Self-Competence = Self-Liking subscale of the Self-Liking/Self-Competence scale; standardized loadings appear on the lines; Dotted lines indicate statistically insignificant loadings, all other loadings are significant at p<.05; grey-shaded boxes (observed variables) were posited to be more strongly related to perceived burdensomeness, while the unshaded boxes (excluding the measurement indicators) were posited to be more strongly related to belongingness.
For the older adults model, standardized regression coefficients for the simultaneous regressions of the six observed variables on the latent variables of thwarted belongingness and perceived burdensomeness appear in Figure 3. As expected, thwarted belongingness, but not perceived burdensomeness was associated with loneliness, social support, and (lower) meaning in life, supporting discriminant relations with these constructs. Discriminant relations were also observed for one of the constructs predicted to be associated with perceived burdensomeness: death ideation was associated with perceived burdensomeness, but not thwarted belongingness. Discriminant relations were not observed for loss of social worth or responsibility to family, as both thwarted belongingness and perceived burdensomeness were associated with loss of social worth, while neither were associated with responsibility to family. The finding regarding loss of social worth parallels the finding with younger adults above with self-esteem: these results suggest that both thwarted belongingness and perceived burdensomeness are associated with decrements in self-worth. Regarding responsibility to family, it may be that the items used in this scale were not relevant to older adults and the adaptation of the scale for older adults—Reasons for Living Older Adults Scale (Edelstein et al., 2009)—would be a more appropriate test of discriminant relations with this construct.
Figure 3.
Structural Equation Model Results Examining Discriminant Relations in Older Adults
Lonely = UCLA Loneliness Scale; Social Support = Belong support subscale of the Interpersonal Support Evaluation List; (lower) Meaning in Life = Meaning in Life subscale of the Geriatric Suicide Ideation scale; Responsibility to Family = subscale of the Reasons for Living Scale; (lower) Social Worth = Loss of Personal and Social Worth subscale of the Geriatric Suicide Ideation Scale; standardized loadings appear on the lines; Dotted lines indicate statistically insignificant loadings, all other loadings are significant at p<.05; grey-shaded boxes (observed variables) were posited to be more strongly related to perceived burdensomeness, while the unshaded boxes (excluding the measurement indicators) were posited to be more strongly related to belongingness.
Taken together, these data indicate support for convergent validity for thwarted belongingness and perceived burdensomeness in both older and younger adults, as well as some evidence of divergent validity for both subscales among older adults. Additional research needed to establish discriminant validity for the belongingness subscale; this issue is addressed in the General Discussion.
Aim 3: Criterion Validity
The interpersonal theory of suicide proposes that thwarted belongingness and perceived burdensomeness are proximal causes of suicidal desire. Thus, the key outcome with regards to the criterion validity of the thwarted belongingness and perceived burdensomeness is whether the operationalized forms of these constructs—responses on the INQ—predict the presence of thoughts of suicide. This section examines the association between scores on the INQ and the presence and severity of suicidal ideation (as measured by the Beck Suicide Scale; BSS; Beck & Steer, 1991).
In the first set of analyses, two models were run to examine concurrent validity. First, a structural equation model was constructed using the INQ measurement model developed above in which suicidal ideation (present versus absent) was regressed upon the belongingness and burdensomeness latent variables, using data from Sample Four (i.e., the clinical sample). A poisson regression model was also constructed, with suicidal ideation severity regressed upon thwarted belongingness and perceived burdensomeness; in this model, observed summed scores for belongingness and burdensomeness were used due to the computational complexity of the models.
Second, follow-up data from Sample One was used to examine whether elevations on thwarted belongingness and perceived burdensomeness predict the presence (versus absence) of suicidal thoughts one month later. Similar to the analyses above, two models were run. First, using SEM, suicidal ideation (present versus absent) was simultaneously regressed on the latent variables of thwarted belongingness and perceived burdensomeness. In a second model, using poisson regression, total suicidal ideation scores (i.e., BSS scores not dichotomized) at Time Two were regressed on Time One BSS scores (to create residualized change scores in suicidal ideation) and Time One thwarted belongingness and perceived burdensomeness observed sum scores.
Method
Measures
Beck Scale for Suicide Ideation (BSS; Beck & Steer 1991)
The BSS is a 21 item self report inventory designed for the assessment of suicidal ideation and behavior. Items 1–19 are used in the following analyses as an index of current suicidal ideation to tap the construct of suicidal desire; items 20 and 21 assess past suicide attempts and are not used in the present analyses. The BSS is the most frequently used self-report scale of suicidal ideation/behavior and numerous studies suggest that it possesses sound psychometric properties (e.g., Beck, Steer, & Ranieri, 1988; Beck, Brown, & Steer, 1997). Due to a low prevalence of suicidal ideation at Time Two (7%), this variable was dichotomized for one set of analyses, with any endorsement on the scale representing the presence of suicidal ideation. For the poisson model, the full range of observed scores was used (0–17).
INQ
For two models below, observed total scores were created for the thwarted belongingness and perceived burdensomeness subscales by summing the items in each scale. The average inter-item correlations for the burdensomeness and belongingness subscales for the clinical sample were .61 and .54 respectively, and were .56 and .49 for the young adult sample.
Sample Characteristics
Regarding concurrent validity analyses, Sample Four characteristics are described in an earlier section. For the predictive validity analyses, all participants in Sample One were invited to participate in a follow-up one month later. This sample consists 254 participants who completed the one-month follow-up (i.e., 82% of the total sample). The demographic profile of these participants is comparable to that of the complete sample (mean age 19.10, std 2.43; 74% female).
Results & Discussion
Examining concurrent validity with the clinical sample, greater odds of reporting suicidal ideation were associated with higher levels of both the thwarted belongingness latent variable (OR = 1.59, p < .01) and the perceived burdensomeness latent variable (OR = 2.21, p < .01). Examining severity of suicidal ideation, higher Beck Suicide Scale scores were concurrently associated with greater scores on the thwarted belongingness subscale (95% confidence interval [CI] of incidence rate ratio [IRR] = 1.03 – 1.07, p < .01) and perceived burdensomeness subscale (95% CI of IRR = 1.04 – 1.90).
Examining predictive validity, greater odds of reporting suicidal ideation one month later were associated with higher levels of both the thwarted belongingness latent variable (OR = 1.83, p < .05) and the perceived burdensomeness latent variable (OR = 1.64, p < .01). Examining severity of suicidal ideation one month later while covarying out Time 1 BSS scores, higher summed scores on the thwarted belongingness and perceived burdensomeness subscales were both associated with higher BSS scores one month later, as evidenced by incidence rate ratios that were greater than one (95% CI of belong IRR = 1.007 – 1.009; 95% CI of burden IRR = 1.050 – 1.051). The interaction was not examined due to the low prevalence of suicidal ideation that reduced statistical power. These results indicate that thwarted belongingness and perceived burdensomeness were significant predictors of the presence and severity of suicidal ideation concurrently and one month later, supporting the concurrent and predictive validity of scores derived from the INQ. These data provide support for the theory’s hypothesis that thwarted belongingness and perceived burdensomeness are proximal causes of suicidal desire.
General Discussion
The first aim of the current project was to examine the theory’s definitions of thwarted belongingness and perceived burdensomeness through the development of a viable latent variable measurement model of the INQ: results from two EFA’s and a single-group CFA using three undergraduate samples support the viability of a latent variable measurement model of the INQ with nine indicators of the construct of thwarted belongingness and six indicators of the construct of perceived burdensomeness (see Appendix for items). These data support the assumption of the interpersonal theory of suicide that thwarted belongingness and perceived burdensomeness are related, but distinct constructs. The measurement model does not include cross-loadings and provides an adequate fit to the data, suggesting that the 15 INQ items retained from the original 25 items represent relatively pure indicators of their respective construct (i.e. either thwarted belongingness or perceived burdensomeness), suggesting that these constructs, are in fact, distinct. The use of 15 items (versus 25) also reduces the administration length of the INQ, improving the practicality of using the tool in research and clinical settings.
The second aim of the project was to examine the generalizability of the measurement model by testing it with populations at elevated risk for suicide – clinical outpatients and older adults. Multiple-group CFA procedures were used to examine measurement invariance and population heterogeneity between undergraduates and both clinical outpatients and older adults. Results indicated that the measurement model developed using undergraduates is a viable model for both clinical outpatients and older adults. Results from the multiple-group CFA are consistent with the interpersonal theory’s proposal that the simultaneous experience of thwarted belongingness and perceived burdensomeness is a proximal cause of suicidal desire, regardless of age, clinical status, etc. Tests of the invariance of intercepts indicated that the values of intercepts for the INQ items vary across these populations, indicating that normative and risky levels of individual INQ items may also vary across populations.
The third aim of the project was to empirically examine the degree to which thwarted belongingness and perceived burdensomeness relate to similar constructs, as hypothesized by the theory’s definitions of the constructs (i.e., convergent and divergent validity), and the degree to which they predict the primary outcome of interest – desire for suicide (i.e., criterion validity). Evidence for discriminant relations was found for the perceived burdensomeness latent variable in both younger and older adults. Discriminant relations were not found for thwarted belongingness: convergent, but not divergent, validity was found for thwarted belongingness in the younger and older adult samples. Thwarted belongingness was significantly associated with most measured constructs (with the exception of death ideation in older adults, and responsibility to family in older adults), including those that are interpersonal in nature but posited to be associated with perceived burdensomeness (i.e. responsibility to family in younger adults), as well as intrapersonal constructs posited to be related to perceptions of burdensomeness (i.e. competence and autonomy). Why might this be the case? Recall that the thwarted belongingness subscale was created to measure the degree to which one’s “need to belong” (Baumeister & Leary, 1995) is unmet. This need to belong—the presence of meaningful and positive social connections—has been posited to be a “fundamental human motivation” (Baumeister & Leary, 1995, p. 1). Among other criteria for what constitutes a fundamental motivation, Baumeister and Leary propose that there must be evidence that the motivation is universal (i.e. applies to all individuals) and has a broad impact on human behaviors and functioning—both of which they posit have been demonstrated for the need to belong. Thus, our data indicating low divergent validity for the belongingness subscale may not be due to poor scale construction or less than ideal psychometric properties; rather, it may be a reflection of the fact that belongingness impacts such a broad range of psychological experiences and behaviors that there are few psychological constructs with which it would be unrelated. In other words, our findings converge with the notion of belongingness as a fundamental human need. However, future studies should attempt to identify constructs that are unrelated to belongingness to delineate the boundaries of its nomological net. For example, some traits, such as attachment style (Green, Furrer, & McAllister, 2011) or rejection sensitivity (Downey & Feldman, 1996), may be associated with belongingness, whereas other traits such as grit (Duckworth, Peterson, Matthews, & Kelly, 2007), or self-regulatory style (Higgins, 1997) may be less strongly associated.
The role of perceptions of burdensomeness in impacting the degree to which relationships can satisfy the need to belong is an area of research that merits investigation. It may be the case that relationships cannot meet the need to belong when the interactions are characterized by perceptions of burdensomeness, as the interactions cease to be positively valenced and emotionally supportive. Further, the presence of a desire for suicide may itself be an indication that a fundamental need is unmet: just as life threatening physical illnesses or injuries involve disruption of life sustaining biological functions, life threatening mental illnesses or processes—in this case suicidal desire—may involve disruptions of life sustaining psychological functioning. In this regard, both thwarted belongingnesss and perceived burdensomeness may indicate the presence of unmet psychological needs because both constructs have demonstrated concurrent and predictive associations with suicidal desire. Most relevant for clinical utility, both constructs were significant predictors of the presence of suicidal ideation one month later. These results should be placed in the context of limitations of the current study, however.
First, not all model fit statistics for the retained models exceeded optimal cut-off values. The non-normal nature of our data may have contributed to sub-optimal model fit. Simulation studies examining the performance of fit indices for severely non-normal data such as suicidal ideation could be useful with regards to improving model fit and increasing the precision with which the latent structure of the INQ and other constructs in the nomological net of suicide can be modeled and predicted. Second, research on the current measurement model is needed using samples with higher clinical severity and increased diversity in terms of race/ethnicity and socioeconomic status. Before the INQ is used in other countries, the content validity of the INQ items for the specific culture should be examined. Future research also needs to consider clinically relevant variables such as depression. Research is also needed to examine structural models in clinical populations. It is possible that our relatively small effect size in the prediction of suicidal ideation at follow-up is due to the fact that the presence of suicidal ideation in our undergraduate sample was so rare. In populations in which suicidal ideation is more common, we would expect a stronger relationship. Finally, convergent and divergent validity were only examined with self-report methodology. Additional support for the construct validity of scores derived from the INQ could be gained by utilizing non self-report cognitive tasks, such as the Implicit Associations Task (Nock et al., 2010).
Another direction for future research, as suggested by the presence of invariant intercepts, includes the use of Item Response Theory methods to examine whether the endorsement of certain items indicates a more or less “severe” level of thwarted belongingness or perceived burdensomeness, which could suggest differential weighting of items when scoring the INQ; these procedures could also more rigorously evaluate the measure’s potential clinical utility in screening for suicide risk factors. Additionally, replication of the latent structure of the final 15-item version of the INQ is needed, and psychometric studies could also investigate if the responses to the INQ differ when belongingness and burdensomeness items are interspersed or administered individually. Finally, a higher order factor model with thwarted belongingness and perceived burdensomeness each loading onto a general “unmet needs” factor was not examined because the theory does not specify this relationship between the constructs. However, this model could be examined in future studies as a form of theory testing.
These data have implications for clinical work with suicidal patients. The INQ has the potential to be an especially valuable instrument for use in suicide risk assessment because in contrast to assessment measures that specifically and exclusively assess the quality and severity of desire for suicide (i.e., suicidal ideation), the INQ assess proximal causes for suicidal desire that indicate targeted areas for intervention (e.g., assisting patients in increasing social support, calling crisis hotlines, etc.). In addition, some patients may be initially reluctant to endorse suicidal ideation for fear of involuntary hospitalization, shame, etc. These patients may be more likely to accurately report their thoughts and feelings on the INQ, thus alerting clinicians to the presence of proximal causes of suicidal desire that may indicate the presence of suicidal ideation and unwillingness to discuss the ideation.
Preventing deaths by suicide, through such means as crisis intervention targeting elevations in thwarted belongingness and/or perceived burdensomeness, is the ultimate goal underlying the development of the INQ – and data from the current project suggest that the INQ measures two constructs especially relevant for suicide prevention, thereby warranting future research on the clinical utility of the INQ as an addition to risk assessment protocols.
Supplementary Material
Acknowledgments
This research was supported in part by grants T32MH20061 and T32AG000029 from the National Institute of Mental Health and National Institute on Aging, respectively, as well as a grant titled, “Interpersonal Vulnerability in Older Caucasian Males: An Examination of the Role of Perceived Burdensomeness” from the American Foundation for Suicide Prevention.
Appendix
INQ
The following questions ask you to think about yourself and other people. Please respond to each question by using your own current beliefs and experiences, NOT what you think is true in general, or what might be true for other people. Please base your responses on how you’ve been feeling recently. Use the rating scale to find the number that best matches how you feel and circle that number. There are no right or wrong answers: we are interested in what you think and feel.
1 1 2 3 4 5 7
Not at all Somewhat Very True
true for me true for me for me
________1 These days the people in my life would be better off if I were gone
________2 These days the people in my life would be happier without me
________3 These days I think I am a burden on society
________4 These days I think my death would be a relief to the people in my life
________5 These days I think the people in my life wish they could be rid of me
________6 These days I think I make things worse for the people in my life
________7 These days, other people care about me
________8 These days, I feel like I belong
________9 These days, I rarely interact with people who care about me
________10 These days, I am fortunate to have many caring and supportive friends
________11 These days, I feel disconnected from other people
________12 These days, I often feel like an outsider in social gatherings
________13 These days, I feel that there are people I can turn to in times of need
________14 These days, I am close to other people
________15 These days, I have at least one satisfying interaction every day
Note: Items 7, 8, 10, 13, 14, and 15 are reverse coded.
Footnotes
The need for relatedness is identical to the need to belong; while the needs for competence and autonomy share similarities with a lack of perceived burdensomeness, they are distinct in that the latter construct is concerned exclusively with interpersonal effectiveness. While the constructs of relatedness and belongingness are identical—and thus should occupy the same place in the nomological net—an examination of the items used to measure relatedness in a SDT-derived self-report measure (i.e., the Basic Need Satisfaction in Life Scale; Gagné, 2003), includes a preponderance of items targeting moderate levels of unmet relatedness needs (e.g., “There are not many people that I am close to” and “The people I interact with regularly do not seem to like me much”). These items presume some level of contact and not a complete lack of positivity in social relations. The thwarted belongingness items of the INQ were designed to have a higher “ceiling” (i.e., to measure higher levels of thwarted belongingness), with items such as “I feel like there are people I can turn to in times of need”). Thus, it is hypothesized that these two measures will be highly related, but non-redundant.
Contributor Information
Kimberly A. Van Orden, Department of Psychiatry, University of Rochester Medical Center
Kelly C. Cukrowicz, Department of Psychology, Texas Tech University
Tracy K. Witte, Department of Psychology, Auburn University
Thomas E. Joiner, Jr, Department of Psychology, Florida State University
References
- Baumeister RF, Leary MR. The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin. 1995;117(3):497–529. [PubMed] [Google Scholar]
- Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107(2):238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
- Berkman LF, Glass T, Brissette I, Seeman TE. From social integration to health: Durkheim in the new millennium. Social Science & Medicine. 2000;51:843–857. doi: 10.1016/s0277-9536(00)00065-4. [DOI] [PubMed] [Google Scholar]
- Bostwick JM, Pankratz VS. Affective disorders and suicide risk: A reexamination. American Journal of Psychiatry. 2000;157:1925–1932. doi: 10.1176/appi.ajp.157.12.1925. [DOI] [PubMed] [Google Scholar]
- Brown GK, Beck AT, Steer RA, Grisham JR. Risk factors for suicide in psychiatric outpatients: A 20-year prospective study. Journal of Consulting and Clinical Psychology. 2000;68(3):371–377. [PubMed] [Google Scholar]
- Brown TA. Confirmatory factor analysis for applied research. New York, NY, US: Guilford Press; 2006. [Google Scholar]
- Browne MW, Cudeck R. Alternative ways of assessing model fit. Sociological Methods & Research. 1992;21(2):230–258. [Google Scholar]
- Bryan CJ, Cukrowicz KC, Joiner TE, Cornette MM. Military suicide and the interpersonal-psychological theory of suicide: Preliminary evidence. Paper presented at the Suicide 2010: Proceedings of the 42nd Annual Conference of the American Association of Suicidology; Orlando, Florida. 2010. [Google Scholar]
- Cacioppo JT, Patrick W. Loneliness: Human Nature and the Need for Social Connection. New York: W.W.Norton & Company; 2008. [Google Scholar]
- Carstensen LL, Isaacowitz DM, Charles ST. Taking time seriously -A theory of socioemotional selectivity. American Psychologist. 1999;54(3):165–181. doi: 10.1037//0003-066x.54.3.165. [DOI] [PubMed] [Google Scholar]
- Cavanagh JT, Carson AJ, Sharpe M, Lawrie SM. Psychological autopsy studies of suicide: a systematic review. Psychol Med. 2003;33(3):395–405. doi: 10.1017/s0033291702006943. [DOI] [PubMed] [Google Scholar]
- Charles ST, Carstensen LL. Social and Emotional Aging. Annual Review of Psychology. 2010;61:383–409. doi: 10.1146/annurev.psych.093008.100448. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cheung GW, Rensvold RB. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling. 2002;9(2):233–255. [Google Scholar]
- Cohen S, Mermelstein R, Kamarck T, Hoberman HH. Measuring the functional components of social support. In: Sarason IG, Sarason BR, editors. Social Support: Theory, Research, and Application. Boston: Martinus Nijhoff; 1985. pp. 73–94. [Google Scholar]
- Conwell Y, Duberstein PR, Hirsch JK, Conner KR, Eberly S, Caine ED. Health status and suicide in the second half of life. Int J Geriatr Psychiatry. 2010;25(4):371–379. doi: 10.1002/gps.2348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Conwell Y, Lyness JM, Duberstein P, Cox C, Seidlitz L, DiGiorgio A, Caine ED. Completed suicide among older patients in primary care practices: a controlled study. Journal of the American Geriatrics Society. 2000;48(1):23–29. doi: 10.1111/j.1532-5415.2000.tb03024.x. [DOI] [PubMed] [Google Scholar]
- Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychological Bulletin. 1955;52(4):281–302. doi: 10.1037/h0040957. [DOI] [PubMed] [Google Scholar]
- Cukrowicz KC, Cheavens JS, Van Orden KA, Ragain RM, Cook RL. Suicide ideation in older adults: perceived burdensomeness is critical. 2010 doi: 10.1037/a0021836. Under Review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Davidson CL, Wingate LR, Rasmussen KA, Slish ML. Hope as a Predictor of Interpersonal Suicide Risk. Suicide and Life-Threatening Behavior. 2009;39(5):499–507. doi: 10.1521/suli.2009.39.5.499. [DOI] [PubMed] [Google Scholar]
- Deci EL, Ryan RM. The "what" and "why" of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry. 2000;11(4):227–268. [Review]. [Google Scholar]
- Downey G, Feldman SI. Implications of rejection sensitivity for intimate relationships. Journal of Personality and Social Psychology. 1996;70(6):1327–1343. doi: 10.1037//0022-3514.70.6.1327. [DOI] [PubMed] [Google Scholar]
- Duberstein PR, Conwell Y, Conner KR, Eberly S, Caine ED. Suicide at 50 years of age and older: perceived physical illness, family discord and financial strain. Psychological Medicine. 2004;34(1):137–146. doi: 10.1017/s0033291703008584. [DOI] [PubMed] [Google Scholar]
- Duckworth AL, Peterson C, Matthews MD, Kelly DR. Grit: perseverance and passion for long-term goals. J Pers Soc Psychol. 2007;92(6):1087–1101. doi: 10.1037/0022-3514.92.6.1087. [DOI] [PubMed] [Google Scholar]
- Edelstein BA, Heisel MJ, McKee DR, Martin RR, Koven LP, Duberstein PR, Britton PC. Development and psychometric evaluation of the reasons for living--older adults scale: a suicide risk assessment inventory. Gerontologist. 2009;49(6):736–745. doi: 10.1093/geront/gnp052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Golden J, Conroy RM, Bruce I, Denihan A, Greene E, Kirby M, Lawlor BA. Loneliness, social support networks, mood and wellbeing in community-dwelling elderly. Int J Geriatr Psychiatry. 2009;24(7):694–700. doi: 10.1002/gps.2181. [DOI] [PubMed] [Google Scholar]
- Gould MS, Shaffer D, Greenberg T. The epidemiology of youth suicide. King, Robert A. 2003 [Google Scholar]
- Green BL, Furrer CJ, McAllister CL. Does attachment style influence social support or the other way around? A longitudinal study of Early Head Start mothers. Attach Hum Dev. 2011;13(1):27–47. doi: 10.1080/14616734.2010.488121. [DOI] [PubMed] [Google Scholar]
- Gutierrez PM, Osman A. Adolescent suicide: An integrated approach to the assessment of risk and protective factors. DeKalb, IL: Northern Illinois University Press; US; 2008. [Google Scholar]
- Heikkinen M, Aro H, Lönnqvist J. Recent life events, social support and suicide. Acta psychiatrica Scandinavica. Supplementum. 1994;377:65–72. doi: 10.1111/j.1600-0447.1994.tb05805.x. [DOI] [PubMed] [Google Scholar]
- Heisel MJ, Flett GL. The Development and Initial Validation of the Geriatric Suicide Ideation Scale. The American Journal of Geriatric Psychiatry. 2006;14(9):742–751. doi: 10.1097/01.JGP.0000218699.27899.f9. [DOI] [PubMed] [Google Scholar]
- Heron MP, Hoyert DL, Murphy SL, Jiaquan X, Kochanek KD, Tejada-Vera B. Deaths: Final Data for 2009. National Vital Statistics Reports. 2006;57(14):1–135. [PubMed] [Google Scholar]
- Higgins ET. Beyond pleasure and pain. Am Psychol. 1997;52(12):1280–1300. doi: 10.1037//0003-066x.52.12.1280. [DOI] [PubMed] [Google Scholar]
- Hu L-t, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6(1):1–55. [Google Scholar]
- Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, Wang P. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. American Journal of Psychiatry. 2010;167(7):748–751. doi: 10.1176/appi.ajp.2010.09091379. [DOI] [PubMed] [Google Scholar]
- Joiner T. Why people die by suicide. Cambridge, MA, US: Harvard University Press; 2005. [Google Scholar]
- Joiner T, Pettit JW, Walker RL, Voelz ZR, Cruz J, Rudd MD, Lester D. Perceived burdensomeness and suicidality: Two studies on the suicide notes of those attempting and those completing suicide. Journal of Social & Clinical Psychology. 2002;21(5):531–545. [Google Scholar]
- Joiner TE, Jr, Hollar D, Van Orden K. On Buckeyes, Gators, Super Bowl Sunday, and the Miracle on Ice: "Pulling together" is associated with lower suicide rates. Journal of Social & Clinical Psychology. 2006;25(2):179–195. [Google Scholar]
- Joiner TE, Jr, Walker RL, Rudd MD, Jobes DA. Scientizing and routinizing the assessment of suicidality in outpatient practice. Professional Psychology: Research and Practice. 1999;30(5):447–453. [Google Scholar]
- Joiner TE, Van Orden KA, Witte TK, Rudd MD. The Interpersonal Theory of Suicide: Guidance for Working with Suicidal Clients. Washington, DC: American Psychological Association; 2009. [Google Scholar]
- Joiner TE, Van Orden KA, Witte TK, Selby EA, Ribeiro J, Lewis R, Rudd MD. Main Predictions of the Interpersonal-Psychological Theory of Suicidal Behavior: Empirical Tests in Two Samples of Young Adults. Journal of Abnormal Psychology. 2009;118(3):634–646. doi: 10.1037/a0016500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koivumaa-Honkanen H, Honkanen R, Viinam ki H, Heikkil K, Kaprio J, Koskenvuo M. Life satisfaction and suicide: a 20-year follow-up study. The American journal of psychiatry. 2001;158(3):433–439. doi: 10.1176/appi.ajp.158.3.433. [DOI] [PubMed] [Google Scholar]
- Krause N. Longitudinal study of social support and meaning in life. Psychology and Aging. 2007;22(3):456–469. doi: 10.1037/0882-7974.22.3.456. [DOI] [PubMed] [Google Scholar]
- Leary MR, Terdal SK, Tambor ES, Downs DL. Self-esteem as an interpersonal monitor: the sociometer hypothesis Journal of Personality and Social Psychology. 1995;68:518–530. [Google Scholar]
- Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: A review of the evidence. American Journal of Psychiatry. 2002;159(6):909–916. doi: 10.1176/appi.ajp.159.6.909. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marshall SK. Do I matter? Construct validation of adolescents’ perceived mattering to parents and friends. Journal of Adolescence. 2001;24:473–490. doi: 10.1006/jado.2001.0384. [DOI] [PubMed] [Google Scholar]
- McPherson CJ, Wilson KG, Murray MA. Feeling like a burden to others: a systematic review focusing on the end of life. Palliative Medicine. 2007;21(2):115–128. doi: 10.1177/0269216307076345. [DOI] [PubMed] [Google Scholar]
- Muthén LK, Muthén BO. Mplus Short Courses Topic 1: Exploratory Factor Analysis, Confirmatory Factor Analysis, and Structural Equation Modeling for Continuous Outcomes; Paper presented at the Workshop on Statistical Analysis with Latent Variables Using MPlus on March 18, 2008, at Johns Hopkins University, Baltimore, MD; Johns Hopkins University, Baltimore, MD. 2008. [Google Scholar]
- Muthén LK, Muthén BO. Mplus User's Guide. Fifth Edition. Los Angeles, CA: Muthén & Muthén; 1998–2007. [Google Scholar]
- Nademin E, Jobes DA, Pflanz SE, Jacoby AM, Ghahramanlou-Holloway M, Campise R, Johnson L. An Investigation of Interpersonal-Psychological Variables in Air Force Suicides: A Controlled-Comparison Study. Archives of Suicide Research. 2008;12(4):309–326. doi: 10.1080/13811110802324847. [DOI] [PubMed] [Google Scholar]
- Nock MK, Park JM, Finn CT, Deliberto TL, Dour HJ, Banaji MR. Measuring the suicidal mind: implicit cognition predicts suicidal behavior. Psychol Sci. 2010;21(4):511–517. doi: 10.1177/0956797610364762. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Qin P, Nordentoft M. Suicide Risk in Relation to Psychiatric Hospitalization. Archives of General Psychiatry. 2005;62(4):427–432. doi: 10.1001/archpsyc.62.4.427. [DOI] [PubMed] [Google Scholar]
- Ryan RM, Deci EL. The darker and brighter sides of human existence: Basic psychological needs as a unifying concept. Psychological Inquiry. 2000;11(4):319–338. [Google Scholar]
- Sourander A, Klomek AB, Niemela S, Haavisto A, Gyllenberg D, Helenius H, Gould MS. Childhood Predictors of Completed and Severe Suicide Attempts: Findings From the Finnish 1981 Birth Cohort Study. Archives of General Psychiatry. 2009;66(4):398–406. doi: 10.1001/archgenpsychiatry.2009.21. [DOI] [PubMed] [Google Scholar]
- Turvey CL, Conwell Y, Jones MP, Phillips C, Simonsick E, Pearson JL, Wallace R. Risk factors for late-life suicide: A prospective community-based study. American Journal of Geriatric Psychiatry. Special Issue: Suicidal behaviors in older adults. 2002;10(4):398–406. [PubMed] [Google Scholar]
- UCLA: Academic Technology Services, S. C. G. How to do parallel analysis for pca or factor analysis in Stata? [Retrieved January 24, 2011];2011 from http://www.ats.ucla.edu/stat/stata/faq/parallel.htm.
- Van Orden KA, Lynam ME, Hollar D, Joiner TE., Jr Perceived Burdensomeness as an Indicator of Suicidal Symptoms. Cognitive Therapy and Research. 2006;30(4):457–467. [Google Scholar]
- Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, Joiner TE., Jr The Interpersonal Theory of Suicide. Psychological Review. 2010 doi: 10.1037/a0018697. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Van Orden KA, Witte TK, Gordon KH, Bender TW, Joiner TE., Jr Suicidal desire and the capability for suicide: Tests of the interpersonal-psychological theory of suicidal behavior among adults. Journal of Consulting and Clinical Psychology. 2008;76(1):72–83. doi: 10.1037/0022-006X.76.1.72. [DOI] [PubMed] [Google Scholar]
- Van Orden KA, Witte TK, James LM, Castro Y, Gordon KH, Braithwaite SR, Joiner TE., Jr Suicidal ideation in college students varies across semesters: The mediating role of belongingness. Suicide and Life-Threatening Behavior. 2008;38(4):427–435. doi: 10.1521/suli.2008.38.4.427. [DOI] [PubMed] [Google Scholar]
- World Health Organization. Geneva: World Health Organization; 1998. Primary prevention of mental, neurological and psychosocial disorders. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.