Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Sep 1.
Published in final edited form as: Int J Offender Ther Comp Criminol. 2019 Apr 8;63(12):2138–2156. doi: 10.1177/0306624X19842027

Thwarted Interpersonal Needs and Suicide Ideation Distress Among Psychiatric Inpatients: The Moderating Role of Criminal Associates

Sean M Mitchell a, Kelly C Cukrowicz b, Jared F Roush b, Sarah L Brown b, Jessica L Alquist b, Angelea D Bolaños b, Robert D Morgan b,*, Erin K Poindexter c,d
PMCID: PMC6679915  NIHMSID: NIHMS1043966  PMID: 30957603

Abstract

Psychiatric inpatients are at elevated risk for suicide, and approximately half are criminal justice-involved. Their involvement with criminal associates may be linked to increased suicide ideation distress; however, this has not been examined. This study tested main effects of, and interactions between, thwarted belongingness (TB) or perceived burdensomeness (PB), time spent with associates, and associates’ criminal involvement predicting suicide ideation distress. In our study, psychiatric inpatients (N= 139) completed assessments cross-sectionally. Results indicated that TB, PB, and associates’ criminal involvement were significantly related to greater suicide ideation distress. A significant three-way interaction indicated participants who endorsed high TB, spent more time with associates, and had associates high in criminal involvement had the greatest probability of “Extreme” suicide ideation distress. These findings suggest that spending time with criminal associates may increase suicide ideation distress more than not having social interactions. Implications and limitations of this study are discussed.

Keywords: thwarted belongingness, perceived burdensomeness, criminal associates, time spent with associates, suicide ideation

Suicide is the 10th leading cause of death in the United States (Centers for Disease Control and Prevention, 2018); however, specific populations, such as psychiatric inpatients, demonstrate even higher risk. Research has indicated that 52.2% of psychiatric inpatients reported suicide ideation at intake and 19.7% had attempted suicide before hospitalization (Oiesvold, Bakkejord, Hansen, Nivison, & Sorgaard, 2012). There is a need for research that examines theory-driven risk factors for suicide ideation among psychiatric inpatients that also considers experiences relevant for this population, such as interactions with criminal associates (i.e., individuals one knows who participate in crime; Logan, Crosby, & Hamburger, 2011; Marcus, 1996; Mills & Kroner, 2001). Therefore, in the current study, we tested how the amount of time spent with criminal associates impacted the relation between theory-driven interpersonal risk factors and suicide ideation distress among psychiatric inpatients.

The interpersonal theory of suicide (Joiner, 2005; Van Orden et al., 2010) provides a theoretical framework for conceptualizing suicide ideation and places a strong emphasis on social distress, which may be particularly salient for psychiatric inpatients. This theory describes thwarted belongingness (TB) and perceived burdensomeness (PB) as two proximal risk factors for suicide ideation. TB is proposed to develop when one experiences loneliness and a lack of reciprocal caring relationships, whereas PB develops when one experiences feelings of self-hate and holds the belief that one is a liability on others to the extent that one’s death would be a relief to others. When TB or PB is experienced, this theory posits that desire for death may occur. However, when TB and PB are experienced concurrently and perceived as unchangeable, it is postulated that suicide ideation may develop. Meta-analytic results have indicated that TB and PB are associated with suicide ideation to a moderate degree, but PB is a stronger predictor of suicide ideation than TB (Chu et al., 2017). This meta-analysis noted that merely 14.8% of the studies testing this theory utilized a clinical sample (i.e., psychiatric inpatients or outpatients). Therefore, greater attention to how this theory applies to clinical samples is needed.

Previous research supports the positive associations between TB and PB, and suicide ideation among psychiatric inpatients after adjusting for various covariates. For example, bivariate correlations indicated that both TB and PB were both significantly associated with suicide ideation, but after adjusting for depressive symptoms, only PB remained significant among psychiatric inpatients (Cero, Zuromski, Witte, Ribeiro, & Joiner, 2015). Furthermore, among psychiatric inpatients admitted for suicide ideation or a suicide attempt, PB was significantly associated with suicide ideation after adjusting for variables related to cognitive functioning, depressive symptoms, hopelessness, age, and gender (TB was not examined; Jahn, Cukrowicz, Mitchell, Poindexter, & Guidry, 2015). In a sample of psychiatric inpatients diagnosed with a mood disorder who were admitted for suicide ideation or a suicide attempt, PB, but not TB was significantly associated with suicide ideation after adjusting for depressive symptoms, hopelessness, and mood disorder diagnosis (Taylor et al., 2016). Furthermore, in a series of three studies with psychiatric inpatient samples, TB and PB in combination mediated the relation between most of the variables related to the psychological flexibility model and suicide ideation (Roush, Brown, Mitchell, & Cukrowicz, 2017; Roush, Cukrowicz, Mitchell, Brown, & Seymour, 2018; Roush, Mitchell, Brown, & Cukrowicz, 2018). Mitchell et al. (2017) also demonstrated that among two psychiatric inpatient samples, TB and PB were bivariate correlates of death ideation, suicide ideation, and suicide ideation distress, but when adjusting for the other, only PB remained significantly associated with death ideation, suicide ideation, and suicide ideation distress. Notably, this research indicated that the associations of TB and PB in relation to suicide ideation and suicide ideation distress were similar in direction and magnitude (Mitchell et al., 2017). These studies indicate that TB and PB are associated with suicide ideation and suicide ideation distress (to different degrees depending on the covariates included); however, it is less clear what variables may moderate the association between TB and PB and suicide ideation distress among psychiatric inpatients. Consistent with the suggestions of Chu et al. (2017), the current study tested moderators of the relation between TB and PB, and suicide ideation distress to elucidate how variables that may be particularly relevant to psychiatric inpatients impact these associations.

In terms of suicide risk, psychiatric inpatients’ involvement in the United States criminal justice system may be necessary to consider given that approximately half of psychiatric inpatients reported a history of criminal justice involvement (e.g., Gross & Morgan, 2013). Additionally, previous research suggests suicide risk is elevated among other individuals involved in the United States criminal justice system, such as inmates and those on community supervision (e.g., Mumola, 2005; Yu & Sung, 2015). However, there is a dearth of research examining how social relationships may be negatively impacted by criminal justice involvement and how these negative social relationships may be related to increased suicide risk. We posit that time spent with criminal associates (i.e., negative social relationships) may change the relation between TB and PB, and suicide ideation distress among psychiatric inpatients. Previous research has suggested that involvement with criminal associates is one of the “Big Four” risk factors for engaging in criminal activity (see Andrews & Bonta, 2010). For example, time spent with criminal associates has been linked to increased criminogenic thinking (Whited, Wagar, Mandracchia, & Morgan, 2017) and general and violent recidivism (Mills, Kroner, & Toni, 2004) among incarcerated men. Despite the importance of criminal associates when considering an individual’s risk for criminal activity, there is a lack of literature directly examining how time spent with criminal associates may be related to interpersonal suicide risk factors and suicide ideation distress among adults.

Given the lack of literature in this area utilizing adult samples, we must rely on literature related to child and adolescent social relationships. Literature supports the link between interpersonal stressors and TB and PB among adolescent psychiatric inpatients (e.g., Buitron et al., 2016), and one such stressor could be time spent with criminal associates. Previous research has indicated that among adolescents, peers’ antisocial behavior (e.g., physical aggression, violating home rules, threatening others, truancy) was significantly associated with participants’ antisocial behavior (Criss et al., 2016), which may influence the quality and quantity of their relationships with others. In fact, children and young adolescents who engaged in delinquent behavior reported poor relationship quality (Poulin, Dishion, & Haas, 1999). Marcus (1996) also indicated that delinquent friendships among youth are associated with greater conflict, poor attachment quality, deficits in their ability to repair relationships, more cognitive distortions, and less social-cognitive problem-solving skills. Moreover, among youth, having delinquent friends was associated with greater suicide ideation (Logan et al., 2011), which may be explained by the exacerbation of TB and PB due to the quality of relationships with criminal associates. Taken together, research has indicated that the frequency of engagement with criminal associates and the criminal involvement of one’s associates may magnify the associations between TB and PB and suicide ideation distress.

Therefore, it is possible that individuals who have criminal associates and experience TB and PB may be at elevated risk for suicide ideation distress; however, this has not been examined among adult psychiatric inpatients, a population with high rates of criminal justice involvement and suicide risk (e.g., Gross & Morgan, 2013; Oiesvold et al., 2012). For example, it is possible that adult psychiatric inpatients who spend more time with criminal associates may also perceive less reciprocal care or support from those relationships (i.e., TB) if the focus of the relationship is engagement in crime or the relationship is more conflictual when compared to individuals who spend more time in prosocial relationships. Similarly, adult psychiatric inpatients who spend more time with criminal associates may also be exposed to greater interpersonal stressors and may also feel like they become a burden in these stressful relationships with criminal others (i.e., PB). In addition, it is possible that individuals who engage more with criminal associates may also damage their prosocial relationships and experience greater TB and PB regarding these relationships when compared to individuals who do not spend time with criminal associates. The current study aims to clarify the impact of relationships with criminal associates on the association between TB, PB, and suicide ideation distress among adult psychiatric inpatients.

Summary and The Current Study

The interpersonal theory of suicide suggests that TB and PB are associated with increased suicide ideation (Van Orden et al., 2010), which has been supported among psychiatric inpatients when examining suicide ideation and suicide ideation distress (e.g., Mitchell et al., 2017). This research may be further extended by examining moderators of these associations that are particularly relevant for high-risk psychiatric inpatients. Approximately half of psychiatric inpatients are criminal justice-involved (e.g., Gross & Morgan, 2013), and individuals involved in the criminal justice system have elevated suicide risk (e.g., Mumola, 2005; Yu & Sung, 2015). Furthermore, involvement with criminal associates is associated with poor relationship quality (Poulin et al., 1999; Marcus, 1996) and may be contributing to elevated suicide ideation (e.g., Logan et al., 2011). Thus, involvement with criminal associates may be a relevant moderator of the relations between TB and PB, and suicide ideation distress.

First, we hypothesized that TB would be positively associated with suicide ideation distress, and this association would be moderated by the amount of time spent with associates and associates’ criminal involvement. Second, we hypothesized that PB would be positively associated with suicide ideation distress, and this association would also be moderated by the amount of time spent with associates and associates’ criminal involvement. Specifically, for both hypotheses, we expected that individuals who spent more time with criminal associates would have a stronger association between TB/PB and suicide ideation distress than those who spent more time with non-criminal associates.

Method

Participants

Participants were 139 patients (Mage = 37.53, SDage = 13.78) from an acute psychiatric inpatient unit (typical stay of three to five days) within a large private hospital in the southwest United States. Participants identified as men (n = 76, 54.7%) or women (n = 63, 45.3%). Most participants identified as Caucasian (n = 86, 61.9%), followed by Hispanic (n =31, 22.3%), African American (n =11, 7.9%), Native American (n = 6, 4.3%), and “other” (n = 5, 3.6%). On average, participants reported 12.96 years of education (SD = 2.68). Additionally, 51.8% (n = 72) reported a history of criminal justice involvement (i.e., a misdemeanor and/or felony). Furthermore, 36.7% (n = 51) reported previous non-suicidal self-injury, and 62.6% (n = 86) reported a previous suicide attempt. Per medical records, most participants were admitted for suicide ideation (n = 87, 62.6%), followed by substance detoxification (n = 28, 20.1%), depressive symptoms (n = 21, 15.1%), a suicide attempt (n = 16, 11.5%), and/or psychosis (n = 14, 10.1%); it was possible that they had multiple reasons for admission. Additionally, 58.6% of individuals who were approached agreed to participate in the study.

Measures

Interpersonal Needs Questionnaire (INQ).

The INQ (Van Orden, Cukrowicz, Witte, & Joiner, 2012) is a 15-item self-report assessment of TB (9 items) and PB (6 items) rated from 1 (Not at all true for me) to 7 (Very true for me). Higher scores indicate greater TB and PB. The INQ has demonstrated acceptable construct validity and reliability across multiple samples (Van Orden et al., 2012). Among psychiatric inpatients, the INQ has evidenced strong internal consistency for TB (Cronbach’s α = .85) and PB (Cronbach’s α = .93; Cero et al., 2015). In the current study, the Cronbach’s α was .81 and .93 for TB and PB, respectively.

Measures of Criminal Attitudes and Associates (MCAA).

The MCAA is a self-report assessment of criminal attitudes and criminal associates (Mills & Kroner, 2001); the criminal attitudes scales were not examined in the current study. Regarding criminal associates, participants consider four adults with whom they spend most of their time with while in the community and rate the percentage of time they spend with each associate (1 = <25%, 2 = 25–50%, 3 = 50–75%, 4 = 75–100%), as well as the level of criminal involvement of each associate (assessed using four yes/no questions per associate). The amount of time spent with any associate, not only criminal associates (i.e., MCAA Time; participants selected one of the following categories: 1 = <25%, 2 = 25–50%, 3 = 50–75%, 4 = 75–100%), was summed for each of the four possible associates. The time spent with associates score ranges from 1 (indicating that the participant spent < 25% of their time with only one associate) to 16 (indicating that the participant spent 75% to 100% of their time with each of their four associates). The associates’ criminal involvement (i.e., MCAA Crime), or the degree of criminal involvement of participants’ associates, was assessed using four crime-related questions for each of the four associates, for a total of 16 items. These 16 items were coded as 0 = no and 1 = yes and then summed; thus, the associates’ criminal involvement score ranges from 0 (indicating all four of the participant’s associates are not criminally involved) to 16 (indicating all four of the participant’s associates are highly criminally involved).

Suicide ideation distress.

Suicide ideation distress was assessed using the Brief Symptom Inventory (BSI; Derogatis, 1993; Derogatis & Melisaratos, 1983) item 9. The BSI assesses the severity of a range of psychiatric symptoms during the past week. Item 9 asks, “How much were you distressed by thoughts of ending your life?” Participants responded to this item using a five-point ordinal response metric by indicating 0 = Not at all, 1 = A little bit, 2 = Moderately, 3 = Quite a bit, 4 = Extremely. On BSI item 9 in the current study, 41 (28.9%) participants endorsed the Not at all category, followed by 18 (12.7%) who endorsed the A little bit category, 14 (9.9%) who endorsed the Moderately category, 27 (19%) who endorsed the Quite a bit category, and 39 (27.5%) who endorsed the Extremely category. Notably, this item was treated as a categorical, rather than a continuous, criterion variable given the ordinal discrete response options. Therefore, categorical data analysis methods were used, which allows us to provide clinically-useful estimated probabilities. The BSI item 9 has also been used as a categorical criterion variable in previous research and has been associated with suicide risk factors (i.e., Jahn et al., 2018; Mitchell et al., 2017; Mitchell et al., 2018), and TB and PB are significantly associated with suicide ideation distress and suicide ideation (Mitchell et al., 2017).

Procedures

All procedures were approved by the university and hospital Institutional Review Boards. Psychiatric inpatients were not eligible to participate if they could not provide consent due to active psychotic symptoms, and/or they were unable to speak English. After giving informed consent, participants completed the assessments in a counter-balanced order to address sequence effects. Participants’ responses were confidential and were not shared with the hospital staff. Also, participants were not compensated for their participation.

Data Analysis Plan

Given that BSI item 9 (suicide ideation distress) is on an ordinal response metric ranging from 0 to 5, this variable was treated as a categorical criterion variable. Therefore, ordinal logistic regression was used to test the hypotheses when the proportional odds assumption was met; however, when this assumption was violated, multinomial logistic regression was used. All predictor variables were mean centered prior to analyses. TB (hypothesis one) or PB (hypothesis two), the amount of time spent with associates (not accounting for associates’ criminal involvement), and the associates’ criminal involvement (not accounting for the amount of time spent with associates) were entered as predictor variables. Subsequently, for each hypothesis, the two-way interactions were added as predictors to the model using TB or PB (e.g., TB × time spent with associates, TB × associates’ criminal involvement, time spent with associates × associates’ criminal involvement) prior to the three-way interaction being added to the model as a predictor using TB or PB (e.g., TB × time spent with associates × associates’ criminal involvement). Simple slopes were examined for significant interactions.1,2

Results Hypothesis One

SAS version 9.4 was used to test the assumptions of the analyses and conduct the primary analyses. See Table 1 for bivariate correlations and descriptive statistics. Because the proportional odds assumption was satisfied, ordinal logistic regression was used to test TB, time spent with associates, associates’ criminal involvement scores, and the two-way interactions, as predictors of the category of suicide ideation distress. As seen in Table 2, ordinal logistic regression analyses indicated significant main effects of TB (OR = 1.05, p < .001) and associates’ criminal involvement (OR = 1.10, p = .014) but not for amount of time spent with associates (OR = 0.97,p = .573). There were no significant two-way interactions. Multinomial logistic regression analyses (utilized because the proportional odds assumption was not met) indicated a significant three-way interaction between TB, time spent with associates, and associates’ criminal involvement (OR = 1.01, p = .031).

Table 1.

Bivariate and Descriptive Statistics

1 2 3 4
1. TB ---
2. PB .61** ---
3. MCAA Time −.22* −.09 ---
4. MCAA Crime .02 −.01 .14 ---
M 32.39 19.60 8.20 3.65
SD 12.84 11.67 3.17 4.21
Observed Range 9–63 6–42 2–16 0–16
Open in a new tab

Note. TB = Interpersonal Needs Questionnaire, Thwarted Belongingness Score; PB = Interpersonal Needs Questionnaire, Perceived Burdensomeness Score; MCAA Time = Measure of Criminal Attitudes and Associates, Amount of Time Spent with Associates Score; MCAA Crime = Measure of Criminal Attitudes and Associates, Associates’ Criminal Involvement Score;

**

p < .001;

*

p < .01.

Table 2.

Hypothesis I: Main effects of and two- and three-way interactions between Thwarted Belongingness, Amount of Time Spent with Associates, and Associates’ Criminal Involvement Predicting Suicide Ideation Distress

Parameter BSI 9 Category Logit SE Wald χ2 P OR OR 95% CI
Intercept 4 −1.05 0.20 26.06 < .001
Intercept 3 −0.10 0.18 0.27 .602
Intercept 2 0.39 0.19 4.30 .038
Intercept 1 0.96 0.20 22.77 < .001
TB 0.05 0.01 14.34 < .001 1.05 1.03 1.08
MCAA Time −0.03 0.05 0.32 .573 0.97 0.88 1.08
MCAA Crime 0.10 0.04 6.11 .014 1.10 1.02 1.19
Intercept 4 −1.14 0.22 27.97 < .001
Intercept 3 −0.17 0.19 0.79 .375
Intercept 2 0.31 0.19 2.72 .099
Intercept 1 0.89 0.20 19.19 < .001
TB 0.05 0.01 15.19 < .001 1.06 1.03 1.09
MCAA Time −0.02 0.06 0.12 .732 0.98 0.88 1.09
MCAA Crime 0.10 0.04 6.47 .011 1.11 1.02 1.20
MCAA Time × TB 0.00 0.00 1.08 .299 1.00 0.99 1.00
MCAA Crime × TB 0.01 0.00 1.63 .202 1.01 1.00 1.01
MCAA Time × MCAA Crime 0.02 0.01 2.10 .148 1.02 0.99 1.05
Intercept 4 −0.29 0.28 1.03 .309
Intercept 3 −0.49 0.29 2.76 .097
Intercept 2 −1.03 0.35 8.45 .004
Intercept 1 −1.15 0.41 7.98 .005
TB 4 0.08 0.02 11.64 .001 1.09 1.04 1.14
TB 3 0.02 0.02 0.76 .383 1.02 0.97 1.07
TB 2 0.00 0.03 0.01 .943 1.00 0.94 1.06
TB 1 0.01 0.03 0.14 .704 1.01 0.95 1.09
MCAA Time 4 −0.02 0.09 0.06 .813 0.98 0.82 1.17
MCAA Time 3 −0.03 0.10 0.08 .778 0.97 0.80 1.18
MCAA Time 2 −0.07 0.12 0.34 .561 0.93 0.73 1.18
MCAA Time 1 0.01 0.14 0.00 .970 1.01 0.77 1.31
MCAA Crime 4 0.16 0.08 4.32 .038 1.17 1.01 1.37
MCAA Crime 3 0.11 0.08 1.98 .159 1.12 0.96 1.31
MCAA Crime 2 0.06 0.10 0.35 .554 1.06 0.87 1.29
MCAA Crime 1 −0.09 0.13 0.45 .502 0.92 0.71 1.19
MCAA Time × TB 4 0.01 0.01 0.37 .541 1.01 0.99 1.03
MCAA Time × TB 3 0.00 0.01 0.03 .874 1.00 0.98 1.02
MCAA Time × TB 2 0.01 0.01 0.74 .390 1.01 0.99 1.04
MCAA Time × TB 1 0.01 0.02 0.21 .647 1.01 0.98 1.04
MCAA Crime × TB 4 0.01 0.01 1.00 .317 1.01 0.99 1.02
MCAA Crime × TB 3 −0.01 0.01 1.46 .227 0.99 0.98 1.01
MCAA Crime × TB 2 0.00 0.01 0.12 .728 1.00 0.98 1.02
MCAA Crime × TB 1 0.01 0.01 0.37 .541 1.01 0.99 1.03
MCAA Time × MCAA Crime 4 0.05 0.02 3.89 .049 1.05 1.00 1.10
MCAA Time × MCAA Crime 3 0.02 0.03 0.33 .564 1.02 0.96 1.07
MCAA Time × MCAA Crime 2 0.04 0.03 1.61 .205 1.04 0.98 1.11
MCAA Time × MCAA Crime 1 0.06 0.04 2.25 .133 1.06 0.98 1.15
MCAA Time × MCAA Crime × TB 4 0.01 0.00 4.63 .031 1.01 1.001 1.01
MCAA Time × MCAA Crime × TB 3 0.00 0.00 2.46 .117 1.01 1.00 1.01
MCAA Time × MCAA Crime × TB 2 0.00 0.00 0.99 .320 1.00 1.00 1.01
MCAA Time × MCAA Crime × TB 1 0.01 0.01 1.84 .175 1.01 1.00 1.02
Open in a new tab

Note. BSI Item 9 = Suicide ideation distress (BSI 9 Category: 1 = A little bit, 2 = Moderately, 3 = Quite a bit, 4 = Extremely; the reference group is 0 = Not at all); TB = Interpersonal Needs Questionnaire, Thwarted Belongingness Score; MCAA Time = Measure of Criminal Attitudes and Associates, Amount of Time Spent with Associates Score; MCAA Crime = Measure of Criminal Attitudes and Associates, Associates’ Criminal Involvement Score.

Simple slopes were examined to further probe this significant three-way interaction as a predictor of the category of suicide ideation distress using multinomial logistic regression by examining the associations between TB and category of suicide ideation distress at varying levels (± 1 SD) of the amount of time spent with associates and associates’ criminal involvement. As seen in Table 3, when comparing individuals with no suicide ideation distress to those with extreme suicide ideation distress, there were significant positive associations between TB and suicide ideation distress only when individuals spend a high amount of time with associates high in criminal involvement or when they spend a low amount of time with associates low in criminal involvement. To more clearly depict the simple slopes, the significant simple slopes were plotted to illustrate the estimated probability of suicide ideation distress (see Figure 1) across TB scores at varying levels of time spent with associates and associates’ criminal involvement. As a sample interpretation of Figure 1, individuals with the highest TB score (63) who spend a high amount of time with associates high in criminal involvement have a 96% chance of being Extremely distressed due to suicide ideation. However, individuals with the highest TB score (63) only have a 53% chance of being Extremely distressed due to suicide ideation when they spend a low amount of time with associates low in criminal involvement. Therefore, estimates indicate that individuals who are high in TB and spend more time with criminal associates have the highest probability of being Extremely distressed due to suicide ideation.

Table 3.

The Simple Slopes of the Relation Between Thwarted Belongingness and Suicide Ideation Distress (Brief Symptom Inventory Item 9) at Varying Levels of Amount of Time Spent with Associates and Associates’ Criminal Involvement

MCAA Associates’ Criminal Involvement (+1 SD) MCAA Associates’ Criminal Involvement (−1 SD)
MCAA Time Spent with Associates Variable BSI 9 Category Logit SE OR p Variable BSI 9 Category Logit SE OR p
MCAA Time (+1 SD) INQ TB 4 0.23 0.08 1.25 .007 INQ TB 4 −0.02 0.05 0.98 .643
3 0.04 0.07 1.05 .532 3 −0.01 0.05 0.99 .816
2 0.08 0.10 1.08 .408 2 −0.01 0.07 0.99 .931
1 0.16 0.12 1.17 .182 1 −0.09 0.06 0.91 .126
MCAA Time (−1 SD) INQ TB 4 0.00 0.07 1.00 .963 INQ TB 4 0.13 0.05 1.14 .010
3 −0.07 0.08 0.93 .353 3 0.13 0.05 1.14 .014
2 −0.10 0.11 0.90 .367 2 0.04 0.05 1.04 .451
1 −0.08 0.15 0.93 .618 1 0.06 0.05 1.06 .270
Open in a new tab

Note. Brief Symptom Inventory Item 9 = Suicide ideation distress (BSI 9 Category: 1 = A little bit, 2 = Moderately, 3 = Quite a bit, 4 = Extremely; the reference group is 0 = Not at all); TB = Interpersonal Needs Questionnaire, Thwarted Belongingness Score; MCAA Time = Measure of Criminal Attitudes and Associates, Amount of Time Spent with Associates Score; MCAA Crime = Measure of Criminal Attitudes and Associates, Associates’ Criminal Involvement Score.

Figure 1.

Figure 1.

Open in a new tab

Each line in the graphs depicts the significant simple slopes based on the significant three-way interaction. Each line indicates the change in the predicted probability of being in a given category of suicide ideation distress during the last seven days including today (Brief Symptom Inventory [BSI] Item 9; BSI 0 = Not at all distressed due to suicide ideation, BSI 1 = A little bit distressed due to suicide ideation, BSI 2 = Moderately distressed due to suicide ideation, BSI 3 = Quite a bit distressed due to suicide ideation, BSI 4 = Extremely distressed due to suicide ideation) across thwarted belongingness scores at high (+1 SD), average (M), and low (−1 SD) levels of time spent with associates (MCAA Time) and associates’ criminal involvement (MCAA Crime). Top graph: MCAA Time high, MCAA Crime high. Middle graph: MCAA Time average, MCAA Crime average. Bottom graph: MCAA Time low, MCAA Crime low. Notably, the mean centered thwarted belongingness scores were used to calculate the predicted probabilities, but the raw scores are presented on the x-axes in the graph.

Hypothesis Two

Because the proportional odds assumption was satisfied, ordinal logistic regression was used to test PB, time spent with associates, and associates’ criminal involvement scores, and the two- and three-way interactions as predictors of the category of suicide ideation distress. As seen in Table 4, there were significant main effects of PB (OR = 1.09, p < .001) and associates’ criminal involvement (OR = 1.11, p = .009) but not of the amount of time spent with associates (OR = 0.95, p = .342). There were no significant two- or three-way interactions; therefore, simple slopes were not examined.

Table 4.

Hypothesis 2: Main effects of and two- and three-way interactions between Perceived Burdensomeness, Amount of Time Spent with Associates, and Associates’ Criminal Involvement Predicting Suicide Ideation Distress

Parameter BSI 9 Category Logit SE Wald χ2 p OR OR 95% CI
Intercept 4 −1.15 0.22 28.25 < .001
Intercept 3 −0.11 0.19 0.35 .552
Intercept 2 0.42 0.19 4.66 .031
Intercept 1 1.05 0.21 25.10 < .001
PB 0.08 0.02 28.75 < .001 1.09 1.05 1.12
MCAA Time −0.05 0.05 0.90 .342 0.95 0.86 1.05
MCAA Crime 0.11 0.04 6.82 .009 1.11 1.03 1.21
Intercept 4 −1.19 0.22 28.69 < .001
Intercept 3 −0.14 0.20 0.51 .475
Intercept 2 0.39 0.20 4.02 .045
Intercept 1 1.03 0.21 23.77 < .001
PB 0.09 0.02 28.94 < .001 1.09 1.06 1.13
MCAA Time −0.03 0.06 0.24 .627 0.97 0.87 1.09
MCAA Crime 0.12 0.04 7.30 .007 1.12 1.03 1.22
MCAA Time × PB 0.00 0.00 0.01 .903 1.00 0.99 1.01
MCAA Crime × PB 0.01 0.00 1.49 .223 1.01 1.00 1.01
MCAA Time × MCAA Crime 0.02 0.02 2.06 .152 1.02 0.99 1.05
Intercept 4 −1.20 0.22 28.82 < .001
Intercept 3 −0.14 0.20 0.51 .474
Intercept 2 0.40 0.20 4.06 .044
Intercept 1 1.03 0.21 23.93 < .001
PB 0.09 0.02 28.63 < .001 1.09 1.06 1.13
MCAA Time −0.02 0.06 0.14 .706 0.98 0.88 1.09
MCAA Crime 0.12 0.04 7.49 .006 1.13 1.04 1.23
MCAA Time × PB 0.00 0.01 0.08 .781 1.00 0.99 1.01
MCAA Crime × PB 0.01 0.00 1.68 .194 1.01 1.00 1.01
MCAA Time × MCAA Crime 0.02 0.02 2.29 .130 1.02 0.99 1.06
MCAA Time × MCAA Crime × PB 0.00 0.00 0.15 .701 1.00 1.00 1.00
Open in a new tab

Note. BSI Item 9 = Suicide ideation distress (BSI 9 Category: 1 = A little bit, 2 = Moderately, 3 = Quite a bit, 4 = Extremely; the reference group is 0 = Not at all); PB = Interpersonal Needs Questionnaire, Perceived Burdensomeness Score; MCAA Time = Measure of Criminal Attitudes and Associates, Amount of Time Spent with Associates Score; MCAA Crime = Measure of Criminal Attitudes and Associates, Associates’ Criminal Involvement Score.

Discussion

The current study aimed to test time spent with associates and associates’ criminal involvement as moderators of the relation between TB and PB, and suicide ideation distress among psychiatric inpatients. Main effects results indicated that greater TB, PB, and associates’ criminal involvement, but not the amount of time spent with associates, were associated with higher odds of being in a more severe category of suicide ideation distress. Overall, the findings of the current study are consistent with the interpersonal theory of suicide (Joiner, 2005; Van Orden et al., 2010), such that TB and PB were significantly associated with suicide ideation distress among psychiatric inpatients. This is consistent with other studies that examined the interpersonal theory of suicide among psychiatric inpatients (e.g., Cero et al., 2015) and meta-analytic results (Chu et al., 2017). Moreover, consistent with the literature on the adverse impact of antisocial or delinquent relationships (e.g., Logan et al., 2011; Marcus, 1996), this finding suggests that psychiatric inpatients with more criminal associates may experience greater suicide ideation distress. This may reflect poorer quality relationships that provide minimal social support and may contribute to psychiatric inpatients’ distress. Another possibility is that psychiatric inpatients may lack interpersonal skills to manage stressful relationships effectively. Future research should further consider these possibilities and examine the reasons why having criminal associates could increase suicide ideation. Moreover, the non-significant main effect of the amount of time spent with associates may also suggest that individuals’ suicide ideation distress is less sensitive to the amount of time spent with others than TB and PB, and associates’ criminal involvement.

Our hypotheses related to the moderating effects of time spent with associates and associates’ criminal involvement on the relations between TB and PB, and suicide ideation distress were partially supported. A significant three-way interaction was found between TB, associates’ criminal involvement, and the amount of time spent with associates predicting the category of suicide ideation distress. Specifically, when comparing individuals with no suicide ideation distress to those with Extreme suicide ideation distress, psychiatric inpatients who spend more time with associates high in criminal involvement and those who spend less time with associates low in criminal involvement evidenced a significant positive association between TB and category of suicide ideation distress. Furthermore, individuals highest in TB and who spend more time with associates high in criminal involvement have the highest estimated probability of being Extremely distressed due to suicide ideation, whereas those highest in TB who spend a little amount of time with associates lower in criminal involvement had a lower probability of being Extremely distressed due to suicide ideation (see Figure 1). These results highlight the potential adverse impact of spending time with criminal associates. The estimated probabilities indicate that spending more time with criminal associates may be more detrimental and related to greater suicide ideation distress than a lack of social interactions. In other words, it appears that it would be less detrimental for psychiatric inpatients to spend time alone rather than with criminal associates. The interaction between PB, associates’ criminal involvement, and the amount of time spent with associates was not significantly related to suicide ideation distress. This finding may be due to the more objective and relational nature of TB (i.e., feeling disconnected from others, the absence of reciprocal care relationships; Van Orden et al., 2010). On the other hand, PB may be less influenced by one’s social interactions, such that one may perceive themselves to be a liability on others despite their objective level of interaction or with whom they are interacting. This discrepancy between TB and PB indicates that these are distinct, yet related constructs, as suggested by the interpersonal theory of suicide (Van Orden et al., 2010). Furthermore, consistent with the suggestions of Chu et al. (2017), our findings demonstrate the importance of considering population-specific moderators within the interpersonal theory of suicide. Future research should continue to consider moderators that could be particularly salient for high-risk populations and contribute to the refinement of suicide risk assessment and intervention.

The results of this study have important implications for suicide risk assessment among psychiatric inpatients. The MCAA is an assessment of criminal associates and attitudes, which was designed to predict criminal recidivism for offender populations (Mills & Kroner, 2001); however, the current study provides preliminary support for the use of the MCAA in suicide risk assessment among psychiatric inpatients. Given that approximately half of psychiatric inpatients have a history of criminal justice involvement (Gross & Morgan, 2013), which is consistent with the current study (52.1%), psychiatric inpatients likely engage with criminal associates. This engagement increases their risk of criminal behavior (Andrews & Bonta, 2010; Mills & Kroner, 2001) and also increases their risk of suicide ideation distress. Thus, using the MCAA among psychiatric inpatients could provide some insight into risk for criminal behaviors and suicide ideation; however, this requires further research.

The current study also supports the use of the INQ as part of a comprehensive assessment of suicide risk among psychiatric inpatients, as TB and PB were significantly associated with a more severe category of suicide ideation distress. Van Orden et al. (2012) suggest that the INQ has potential as a clinical assessment for suicide ideation because it is brief and grounded in theory. Additionally, patients may be more willing to endorse experiences of TB and PB than suicide ideation (Van Orden et al., 2012), which has been supported by previous research (Cukrowicz, Jahn, Graham, Poindexter, & Williams, 2013). Previous research has provided preliminary clinical cutoff scores for the INQ when predicting several concurrent suicide ideation-related outcomes among psychiatric inpatients, which can be used during suicide risk assessment (Mitchell et al., 2017).

The current study also provides support for involvement with criminal associates and TB and PB as potential targets for clinical intervention. Previous research suggests that involvement with criminal associates is one of the “Big Four” risk factors for engaging in criminal activity, which makes it an important treatment target to reduce an individual’s risk for engaging in crime (see Andrews & Bonta, 2010); however, previous studies have not considered targeting involvement with criminal associates to reduce suicide risk. The current findings suggest that psychotherapeutic interventions aimed at systematically encouraging both a reduction in involvement with criminal associates and increasing the amount of time spent with non-criminal associates in prosocial relationships could reduce the risk for suicide ideation distress in addition to lowering criminal risk. It may be beneficial to enhance prosocial relationships and provide community resources that allow individuals to connect with prosocial peers. Additionally, cognitive behavioral therapy techniques (e.g., cognitive restructuring) may be used to reduce TB and PB with the goal of ameliorating suicide ideation and related distress by targeting negative automatic thoughts related to one’s belief about not belonging or being a burden on others (Joiner, Van Orden, Witte, & Rudd, 2009).

Although the current study extends our understanding of the role of criminal associates in suicide risk among psychiatric inpatients, there are limitations. The present study utilized self-report assessments, which are subject to possible reporting biases and the effects of monomethod assessment. Additionally, we assessed suicide ideation distress, but future studies should replicate our findings with other assessments of suicide ideation. It is possible that the predictor variables included in the current study might be related to suicide ideation differently than suicide ideation distress. Future research should directly compare reports of suicide ideation distress to suicide ideation to determine if these concepts are genuinely distinct among psychiatric inpatients. It should be noted that although it was not a direct comparison, previous research among psychiatric inpatients has indicated the associations of TB and PB with suicide ideation and suicide ideation distress are similar in direction and magnitude (see Mitchell et al., 2017).

Another possible limitation of the current study was the use of a single-item assessment of suicide ideation distress as the criterion variable. Research has indicated that single-item assessments of suicide ideation and attempts can be problematic when they include imprecise language (Horn, Joiner, & Bernert, 2016) and could increase the likelihood of misclassification of participants when limited to yes/no response options of lifetime incidence of suicide ideation (Millner, Lee, & Nock, 2015). It should be noted, however, that the suicide ideation distress item (BSI item 9) may have been less impacted by these limitations. For example, the suicide ideation distress item used clear language, included more response options than only yes or no, and indicated a specific and recent timeframe (the past week) than the items noted in the studies above. Future research should replicate the findings of the current study using more comprehensive assessments of suicidal thoughts.

The cross-sectional design also precludes the determination of causality or directionality between the variables of interest. Future studies should consider longitudinal methodology (e.g., ecological momentary assessment), which could improve recall and allow for the testing of temporal associations (Davidson, Anestis, & Gutierrez, 2016). Similarly, this may allow for a greater understanding of the mechanisms (e.g., social problems solving, self-esteem) involved in the development of TB and PB among high-risk populations who engage with criminal associates across the lifespan. Moreover, there may be additional moderators (e.g., other social support, the severity of criminal behavior, antisocial personality traits) between TB/PB and suicide ideation distress, which should be explored in future studies. Furthermore, we did not test the suicide capability component of the interpersonal theory of suicide, which is posited to be associated with increased risk for suicidal behaviors (Joiner, 2005; Van Orden et al., 2010); therefore, further research may examine the association between involvement with criminal associates and suicide capability.

The present study provides evidence that TB and PB and associates’ criminal involvement may play a significant role in suicide ideation distress. Furthermore, these findings indicate that when psychiatric inpatients report elevated TB and spend a high amount of time with criminal associates, they are most likely to be Extremely distressed due to suicide ideation.

These results highlight the potential adverse impact of spending time with criminal associates for psychiatric inpatients. As it relates to suicide ideation distress, it appears that it may be less detrimental for psychiatric inpatients to spend time alone than to spend time with criminal associates. Interventions aimed at reducing time spent with criminal associates and TB and PB, while also increasing prosocial relationships, may reduce suicide risk among high-risk psychiatric inpatients.

Acknowledgments

This research was supported, in part, by a grant from the National Institute of Mental Health (T32 MH020061).

Footnotes

1.

Gender was not significantly associated with suicide ideation distress (p-values > .05); therefore, it was not included as a covariate in the model. Although, on average, men reported lower PB (F[l,131] = 4.51,p = .034) and higher associates’ criminal involvement scores than women (F[l,131] = 8.73,P = .004), there were no significant interactions between gender and the other predictor variables in relation to suicide ideation distress (p-values > .05). Given this, gender was not included in the hypothesized models.

2.

Participants’ own history of involvement in the criminal justice system was not significantly associated with suicide ideation distress (p-values > .05). On average, those with a criminal justice history reported significantly higher associates’ criminal involvement scores (F[1,131] = 13.24, p < .001); however, there were no significant interactions between criminal justice history and the other predictor variables in relation to suicide ideation distress (p-values > .05). Therefore, criminal justice history was not included in the hypothesized models.

References

  1. Andrews DA, & Bonta J (2010). Rehabilitating criminal justice policy and practice. Psychology, Public Policy, and Law, 16, 39–55. doi: 10.1037/a0018362 [DOI] [Google Scholar]
  2. Buitron V, Hill RM, Pettit JW, Green KL, Hatkevich C, & Sharp C (2016). Interpersonal stress and suicidal ideation in adolescence: An indirect association through perceived burdensomeness toward others. Journal of Affective Disorders, 190, 143–149. doi: 10.1016/j.jad.2015.09.077 [DOI] [PubMed] [Google Scholar]
  3. Centers for Disease Control and Prevention. (2018). Web-based injury statistics query and reporting system. Retrieved from http://webappa.cdc.gov/sasweb/ncipc/leadcauslO_us.html
  4. Cero I, Zuromski KL, Witte TK, Ribeiro JD, & Joiner TE (2015). Perceived burdensomeness, thwarted belongingness, and suicide ideation: Re-examination of the Interpersonal-Psychological Theory in two samples. Psychiatry Research, 228, 544–550. doi: 10.1016/j.psychres.2015.05.055 [DOI] [PubMed] [Google Scholar]
  5. Chu C, Buchman-Schmitt JM, Stanley IH, Horn MA, Tucker RP, Hagan CR, & … Joiner TE (2017). The Interpersonal Theory of Suicide: A Systematic Review and Meta-Analysis of a Decade of Cross-National Research. Psychological Bulletin, 143, 1313–1345. doi: 10.1037/bul0000124 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Criss MM, Houltberg BJ, Cui L, Bosler CD, Morris AS, & Silk JS (2016). Direct and indirect links between peer factors and adolescent adjustment difficulties. Journal of Applied Developmental Psychology, 43, 83–90. doi: 10.1016/j.appdev.2016.01.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cukrowicz KC, Jahn DR, Graham RD, Poindexter EK, & Williams RB (2013). Suicide risk in older adults: Evaluating models of risk and predicting excess zeros in a primary care sample. Journal of Abnormal Psychology, 122, 1021–1030. doi: 10.1037/a0034953 [DOI] [PubMed] [Google Scholar]
  8. Davidson CL, Anestis MD, & Gutierrez PM (2016). Ecological momentary assessment is a neglected methodology in suicidology. Archives of Suicide Research. Advance online publication, doi: 10.1080/13811118.2015.1004482 [DOI] [PubMed] [Google Scholar]
  9. Derogatis LR (1993). Brief Symptom Inventory: Administration, scoring and procedures manual - II. Minneapolis, MN: National Computer Systems. [Google Scholar]
  10. Derogatis LR, & Melisaratos N (1983). The Brief Symptom Inventory: An introductory report. Psychological Medicine, 13, 595–605. doi: 10.1017/S003329170004801 [DOI] [PubMed] [Google Scholar]
  11. Gross NR, & Morgan RD (2013). Understanding persons with mental illness who are and are not criminal justice involved: A comparison of criminal thinking and psychiatric symptoms. Law and Human Behavior, 37, 175–186. doi: 10.1037/lhb0000013. [DOI] [PubMed] [Google Scholar]
  12. Horn MA, Joiner TJ, & Bernert RA (2016). Limitations of a single-item assessment of suicide attempt history: Implications for standardized suicide risk assessment. Psychological Assessment, 28, 1026–1030. doi: 10.1037/pas0000241 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Jahn DR, Cukrowicz KC, Mitchell SM, Poindexter EK, & Guidry ET (2015). The mediating role of perceived burdensomeness in relations between domains of cognitive functioning and indicators of suicide risk. Journal of Clinical Psychology, 71, 908–919. doi: 10.1002/jclp.22190 [DOI] [PubMed] [Google Scholar]
  14. Jahn DR, Muralidharan A, Drapalski AL, Brown CH, Fang LJ, & Lucksted A(2018). Differences in suicide and death ideation among veterans and nonveterans with serious mental illness. Psychological Services, 15, 31–39. doi: 10.1037/ser0000127 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Joiner TE (2005). Why people die by suicide. Cambridge, MA: Harvard University Press. [Google Scholar]
  16. Joiner TE, Van Orden KA, Witte TK, & Rudd MD (2009). The interpersonal theory of suicide: Guidance for working with suicidal clients. Washington, DC: American Psychological Association. [DOI] [PubMed] [Google Scholar]
  17. Logan JE, Crosby AE, & Hamburger ME (2011). Suicidal ideation, friendships with delinquents, social and parental connectedness, and differential associations by sex: Findings among high-risk pre/early adolescent population. Crisis: The Journal of Crisis Intervention and Suicide Prevention, 32, 299–309. doi: 10.1027/0227-5910/a000091 [DOI] [PubMed] [Google Scholar]
  18. Marcus RF (1996). The friendships of delinquents. Adolescence, 31, 145–158. [PubMed] [Google Scholar]
  19. Millner AJ, Lee MD, & Nock MK (2015). Single-item measurement of suicidal behaviors: Validity and consequences of misclassification. Plos ONE, 10. doi: 10.1371/journal.pone.0141606 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Mills JF, & Kroner DG (2001). Measure of criminal attitudes and associates: User guide. Unpublished manuscript, Carleton University, Ottawa, Canada.
  21. Mills JF, Kroner DG, & Hemmati T (2004). The Measures of Criminal Attitudes and Associates (MCAA): The prediction of general and violent recidivism. Criminal Justice and Behavior, 31, 717–733. doi: 10.1177/0093854804268755 [DOI] [Google Scholar]
  22. Mitchell SM, Brown SL, Bolanos AD, Rose B, Delgado D, Morgan RD,… & Cukrowicz KC (2018). Psychiatric symptoms, criminal risk, and suicidal ideation and attempts among not guilty by reason of insanity state hospital inpatients. Psychological Services, 15, 340–348. doi: 10.1037/ser0000209 [DOI] [PubMed] [Google Scholar]
  23. Mitchell SM, Brown SL, Roush JF, Bolanos AD, Littlefield AK, Marshall AJ, … & Cukrowicz KC (2017). The clinical application of suicide risk assessment: A theory-driven approach. Clinical Psychology & Psychotherapy, 24, 1406–1420. doi: 10.1002/cpp.2086 [DOI] [PubMed] [Google Scholar]
  24. Mumola CJ (2005). Suicide and homicide in state prisons and local jails. Retrieved from: http://bjs.ojp.usdoj.gov/index.cfm?ty=pbdetail&iid=1126
  25. Oiesvold T, Bakkejord T, Hansen V, Nivison M, & Sorgaard KW (2012). Suicidality related to first-time admissions to psychiatric hospital. Social Psychiatry and Psychiatric Epidemiology, 47, 419–425. doi: 10.1007/s00127-011-0343-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Poulin F, Dishion TJ, & Haas E (1999). The peer influence paradox: Friendship quality and deviancy training within male adolescent friendships. Merrill-Palmer Quarterly, 45, 42–61. [Google Scholar]
  27. Roush JF, Brown SL, Mitchell SM, & Cukrowicz KC (2017). Experiential avoidance, cognitive fusion, and suicide ideation among psychiatric inpatients: The role of thwarted interpersonal needs. Psychotherapy Research. doi: 10.1080/10503307.2017.1395923 [DOI] [PubMed] [Google Scholar]
  28. Roush JF, Cukrowicz KC, Mitchell SM, Brown SL, & Seymour NE (2018). Valued living, life fulfillment, and suicide ideation among psychiatric inpatients: The mediating role of thwarted interpersonal needs. Journal of Contextual Behavioral Science, 7, 8–14 doi: 10.1016/j.jcbs.2017.11.001 [DOI] [Google Scholar]
  29. Roush JF, Mitchell SM, Brown SL, & Cukrowicz KC (2018). Thwarted interpersonal needs mediate the relation between facets of mindfulness and suicide ideation among psychiatric inpatients. Psychiatry Research, 265, 167–173. doi: 10.1016/j.psychres.2018.04.026 [DOI] [PubMed] [Google Scholar]
  30. Taylor NJ, Mitchell SM, Roush JF, Brown SL, Jahn DR, & Cukrowicz KC (2016). Thwarted interpersonal needs and suicide ideation: Comparing psychiatric inpatients with bipolar and non-bipolar mood disorders. Psychiatry Research, 246, 161–165. doi: 10.1016/j.psychres.2016.09.025 [DOI] [PubMed] [Google Scholar]
  31. Van Orden KA, Cukrowicz KC, Witte TK, & Joiner TE (2012). Thwarted belongingness and perceived burdensomeness: Construct validity and psychometric properties of the interpersonal needs questionnaire. Psychological Assessment, 24, 197–215. doi: 10.1037/a0025358 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, & Joiner TE (2010). The interpersonal theory of suicide. Psychological Review, 117, 575–600. doi: 10.1037/a0018697 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Whited WH, Wagar L, Mandracchia JT, & Morgan RD (2017). Partners or partners in crime? The relationship between criminal associates and criminogenic thinking. International Journal of Offender Therapy and Comparative Criminology, 61, 491–507. doi: 10.1177/0306624X15599605 [DOI] [PubMed] [Google Scholar]
  34. Yu S-SV, & Sung H-E (2015). Suicidal ideation of probationers: Gender differences. Crisis: The Journal of Crisis Intervention and Suicide Prevention, 36, 424–432. doi: 10.1027/0227-5910/a000336 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES