Does Assessing Suicidality Frequently and Repeatedly Cause Harm? A Randomized Control Study

Abstract

Assessing suicidality is common in mental health practice and is fundamental to suicide research. Although necessary, there is significant concern that such assessments have unintended harmful consequences. Using a longitudinal randomized control design, we evaluated whether repeated and frequent assessments of suicide-related thoughts and behaviors negatively affected individuals, including those at-risk for suicide-related outcomes. Adults (N = 282), including many diagnosed with Borderline Personality Disorder (BPD), were recruited through psychiatric outpatient clinics and from the community at large, and were randomly assigned to assessment groups. A Control Assessment group responded to questions regarding negative psychological experiences several times each day during a 2-week Main Observation phase. During the same observation period, an Intensive Suicide Assessment group responded to the same questions, along with questions regarding suicidal behavior and ideation. Negative psychological outcomes were measured during the Main Observation phase (for BPD symptoms unrelated to suicide and for BPD-relevant emotions) and/or at the end of each week during the Main Observation phase and monthly for 6 months thereafter (for all outcomes, including suicidal ideation and behavior). Results revealed little evidence that intensive suicide assessment triggered negative outcomes, including suicidal ideation or behavior, even among people with BPD. A handful of effects did reach or approach significance, though these were temporary and non-robust. However, given the seriousness of some outcomes, we recommend that researchers or clinicians who implement experience sampling methods including suicide-related items carefully consider the benefits of asking about suicide and to inform participants about possible risks.

Assessing suicidal thoughts and actions is an integral part of healthcare and research in preventing suicide (American Psychiatric Association, 2003). As suicide is a leading cause of death in the United States and worldwide, healthcare professionals and researchers must assess suicidal thoughts, feelings, and behaviors to improve risk assessment, understand causal mechanisms, generate and evaluate effective prevention strategies, and implement and monitor these strategies. Although assessing suicidal ideation and behavior is crucial for these purposes, such assessments might have unintended, but dangerous, consequences. Indeed, the possibility that asking about suicide could increase suicidal thoughts and behavior has alarmed healthcare providers and research reviewers (Bajaj et al., 2008; Lakeman & FitzGerald, 2009). The purpose of the current study is to reveal any risks associated with assessing suicidality intensively and repeatedly using a longitudinal randomized control design.

Identifying risks of intensive suicide assessment is important for at least three reasons. First, if any assessment of suicidal thoughts and behaviors increases suicidality, then frequent and repeated assessment would likely have particularly strong effects. Frequent assessments may bring suicidal thoughts into consciousness more often, potentially magnifying any effects caused by one or even a few assessments. Intensive assessments may even produce harmful effects not seen with less frequent schedules. However, if intensive assessments produce no harmful effects, then it seems reasonable to infer that less frequent assessment of suicidal thoughts and behaviors in clinical research or practice would produce no harm.

Second, suicide assessment commonly occurs in mental health settings. For individuals at-risk for suicide, ongoing tracking of suicidal thoughts and behavior is integrated into treatment (American Psychiatric Association, 2001). In practice, mental healthcare professionals often assess suicidality before each session (e.g., Resick, Monson, & Chard, 2008). Moreover, inventories assessing psychopathology and mental health are regularly used to track therapy progress, and widely used inventories include suicide items (i.e., Beck Depression Inventory, Symptom Checklist-90-Revised, and Outcome Questionnaire 45.2, Hatfield & Ogles, 2004). Assessment of psychopathology daily, or even several times per day, is uncommon in current clinical practice, but there are emerging calls to integrate such assessments into clinical practice (e.g., Kramer et al., 2014; Telford, McCarthy-Jones, & Rowse, 2012) and some experts predict sharp increases in their use (Shiffman, Stone, & Hufford, 2008). These methods have been recommended for clinical practice largely because of their potential for improving the accuracy of diagnosis, effectiveness of treatment (e.g., through gains in insight), and assessment of treatment outcome. Evaluating the potential harmful impact of repeatedly assessing suicide is crucial prior to any integration into practice.

Third, in a research context, intensive suicide assessment has methodological benefits and is essential for answering critical questions about suicide. Advantages of experience sampling methodology (ESM) or ecological momentary assessment include assessing individuals in their natural setting, limiting recall errors, detecting infrequent events, and capturing within-person variability (Conner, Tennen, Fleeson, & Barrett, 2009; Furr, 2009; Myin-Germeys et al., 2009). To understand day-to-day events or contexts that trigger suicidal thoughts and behaviors, the optimal design would include frequent, repeated assessments of such thoughts and behaviors across days or weeks. Considering their benefits, research agendas for suicide have called for and adopted the increased use of such methods (e.g., Ben-Zeev, Young, & Depp, 2012; Nock, Prinstein, & Sterba, 2009). Responsible conduct of such work requires understanding its risks.

There have been six studies directly investigating whether assessing suicide increases suicidal thoughts and/or behavior, and results generally indicate no effects (Crawford et al., 2011; Cukrowicz, Smith, & Poindexter, 2010; Gould et al., 2005; Lang, Uttaro, Caine, Carpinello, & Felton, 2009; Mathias et al., 2012). Interestingly, ambiguous results were obtained in a study of participants diagnosed with Borderline Personality Disorder (BPD) who also reported current and chronic suicidality (Reynolds, Lindenboim, Comtois, Murray, & Linehan, 2006). In this study, one analysis revealed no difference in the number of participants reporting increases versus decreases in suicidality after an in-depth suicide assessment; however, another analysis revealed a marginally significant average increase in suicidality after suicide assessment.

The previous research has important strengths. It includes a range of ages, participants with and without a pre-existing risk of suicide, and diverse assessments of suicidality and negative events. Moreover, several studies included randomized control groups, and several included longitudinal designs spanning months or years.

Unfortunately, this work does not directly or robustly evaluate whether intensive suicide assessment is harmful. Two studies included only a single “post-assessment” (Crawford et al., 2011; Gould et al., 2005); thus, they cannot reveal effects of repeated assessment. The other four studies included multiple assessments but not randomized control groups. Although such studies are valuable, they do not clearly reveal causality or natural patterns of suicidality.

Researchers have also examined whether assessing suicide affects psychological distress more generally. Again, few effects have been reported. Although two studies revealed significant relationships between suicidal assessment and subjective distress (Langhinrichsen-Rohling, Arata, O’Brien, Bowers, & Klibert, 2006; Robinson et al., 2011), these effects were small and limited to individuals already at-risk for suicide. Other studies revealed no effects, even among those at-risk (Biddle et al., 2013; Deeley & Love, 2010; Gould et al., 2005). Several studies have also examined repeated assessment of non-suicidal negative events (e.g., marital stress, depressed mood, bullying, pain). Results reveal some harmful effects, (e.g., Merrilees et al., 2008), some beneficial effects (e.g., Nishina, 2012), and no effects (e.g., Cruise et al., 1996).

Overview of Current Study

The current study addresses important issues not previously addressed conclusively. First and most broadly, we examined whether frequent and repeated suicide assessment has harmful consequences in general. Second, we examined when such effects might occur - only in the short-term or only after a period of time; and if they do arise in the short term, do they have a sustained influence? Previous studies might underestimate or miss important effects, because they were too brief to reveal longer-term effects or too intermittent (in terms of the frequency of suicide inquiry) to reveal short-term effects. Third, we examined whether repeated suicide assessment affects some people more than others. Individuals with pathologies associated with a risk for suicidal thoughts and behaviors might be particularly vulnerable to these effects. Fourth, we examined a variety of effects potentially triggered by repeated suicide assessment. Of primary concern are suicide attempts, suicidal ideation, and non-suicidal self-harming behaviors, as these can directly increase the likelihood of suicide and death. Of secondary interest, though still important, are potentially distressing cognitive, affective, social, and behavioral effects.

We conducted a randomized control, longitudinal study of a sample that included adults at-risk for suicidal thoughts and behaviors. Our sample included many participants diagnosed with BPD. These participants are at heightened risk for suicide, with as many as 73% of individuals with BPD attempting suicide (Soloff, Lis, Kelly, Cornelius, & Ulrich, 1994).

Method

Participants

As part of a larger project investigating BPD, two methods were used to recruit participants with a range of symptom-levels. Some participants were recruited from five mental health clinics, a human services agency, and via flyers describing BPD-related problems, and they were required to endorse ≥7 items on the McLean Screening Instrument for BPD (MSI-BPD; Zanarini, Vujanovic, Parachini, Boulanger, Frankenburg, & Hennen, 2003). Other participants were recruited via advertisements (e.g., direct mailings via newspaper; flyers at restaurants and grocery stores). These ads referred to “a study on personality change over time” and described methods (e.g., “you will be interviewed and complete questionnaires,” “you will also carry a palm pilot for two weeks”). These participants were included regardless of MSI-BPD scores. This was not a treatment study.

Inclusion criteria included: 1) age 18 to 65 years old; 2) residing within 50-miles of the study’s location; and 3) self-reported ability to participate in electronic data collection. Exclusion criteria included: 1) scoring <24 on the Mini Mental State Examination (MMSE; Sheehan et al., 1998); 2) being actively suicidal (i.e., current suicidal ideation with intent) or psychotic at screening; 3) meeting criteria for alcohol or substance dependence disorder and used substances within 30 days; 4) reporting a history of arrest for violent crime; or 5) having a court-appointed guardian. Individuals received a $20 gift card for participating in screening.

A total of 282 participants enrolled in the study (N’s = 187 and 95, from the two recruitment methods respectively). As described below, based on valid responses on relevant assessments, the final sample included 248 participants. After receiving a description of the study, participants provided written informed consent. When concluding participation, participants received a $125 gift card for completing a “Main Observation” set of assessments, along with entries into gift card drawings for Follow-up assessments.

As assessed via the Structured Interview for DSM-IV Personality (SIDP-IV; Pfhol, Blum, & Zimmerman, 1997), 75 of the 248 participants (30%) met criteria for current BPD. All subsequent analyses of diagnostic groups (BPD and Non-BPD) are based on this diagnosis, regardless of recruitment origin or MSI-BPD score. Participants met criteria for a variety of Diagnostic and Statistical Manual Fourth Edition (DSM-IV) axis I disorders as assessed by the Mini International Neuropsychiatric Interview (MINI; Sheehan et al., 1998), most commonly anxiety disorder (48.2%) or mood disorder (44.7%). About one-fourth (22.3%) had a lifetime history of at least one psychiatric hospitalization.

The sample was diverse in sex (67.3% female) and ethnicity (59.7% white). Mean age was 43.9 years (SD=11.2), and mean education was 14.7 years (SD = 2.8). Approximately 35% of the participants were in therapy and/or taking psychiatric medication at screen-in.

Measures

Participants completed two phases of assessment. The Main Observation phase was a 2-week period following enrollment in the study and completion of intake assessments (e.g., the SIDP-IV and MINI). During this phase, participants completed “temporally-layered” assessments of BPD symptoms and BPD-relevant emotions, including ESM assessments (5 times/day, which allows for detailed assessments of BPD symptoms as they occur in a clinical sample and allows for intensive analysis of within-person personality processes not relevant to the current study), daily assessments (once/day), and weekly assessments (once/week). The Follow-up phase was the 6-month period starting two weeks after the Main Observation phase. During follow-up, participants completed monthly assessments of the same outcomes.

The use of temporal layering has at least two important strengths. First, it allows us to capture occurrences of both frequent and rare events. This is important, considering the variety of outcomes that were assessed. Second, it balances concern about recall bias, which can particularly affect assessments based on broad retrospective time periods, against the possibility that people’s reconstructed memories can have important implications (even if they might not be fully accurate memories). Indeed, research shows that immediate and recalled reports can diverge meaningfully, particularly among people with BPD (e.g., Ebner-Priemer et al., 2006).

Table 1 summarizes the temporally layered assessments of BPD symptoms and BPD-relevant emotions. Assessments paralleled each other closely in content, differing mainly in frequency, time-frame, and rating scales. All assessments included 23 relevant questions – 17 for non-suicide BPD symptoms and 6 for BPD-relevant emotions. Weekly and monthly assessments included two additional questions referring to suicidal thoughts and behaviors (see “Experimental Manipulation” below). Questions referred to a symptom or emotion as it occurred in a given timeframe (e.g., “in the last 60 minutes,” for ESM assessments).

Table 1.

Summary of Assessment Schedule

Phase & Assessment	Frequency (Timing)	Time Frame	Total Possible Assessments (Average # completed)
Main Observation Phase (First two weeks of study)
ESM	5 times/day (10AM, 1PM, 4PM, 7PM, 10PM)	Last 60 minutes	70 (44.0)
Daily	1 time/day (End of each day)	During the entire day	14 (12.6)
Weekly	1 time/week (End of each week)	Over the past week	2 (1.6)
Follow-up Phase (6 months following the Main Observation phase)
Monthly	1 time/month (End of each month)	Over the last 30 days	6 (3.6)

BPD symptom questions reflected the eight non-suicide BPD symptoms,with one or two questions each (e.g., “I couldn’t stop myself from overdoing something bad in the last 60 minutes,” “Today, I felt like I didn’t know who I am or like I had no identity”). Participants responded to each symptom item on 6-point scales. The ESM scale ranged from 0=”Does not describe me at all” to 5=”Describes me very well.” Daily, weekly, and monthly scales ranged from 0 = “Never” to 5 = “50+ times.”

BPD-relevant emotion questions were taken from the Positive and Negative Affective Schedule-Expanded form (Watson & Clark, 1994) – guilty, ashamed, irritable, angry, excited, and happy (e.g. “I felt guilty in the last 60 minutes”). Participants responded to each on a 6-point scale, from 0=”Very slightly or not at all,” to 5=”Extremely.”

Responses were aggregated to create composite (non-suicide) symptom scores and emotion scores at each assessment occasion. Of main interest among these is a Total BPD symptom score, which is aggregated across all BPD symptom items. Additional symptom composite scores included Unstable Relationships, Efforts to Avoid Abandonment, Unstable Affect, Emptiness/Hollow, Uncertain sense of Self/Identity, Uncontrollable Anger, Unreal Experiences (i.e., paranoia, dissociation), and Impulsive Behavior. In addition, one item reflected non-suicidal self-harm – “I hurt myself on purpose (e.g. punched myself, cut myself, or burned myself) in the last 60 minutes [or “today”, etc.].” Previous research indicates that these symptom aggregates are reliable and converge with other methods of symptom assessment (Hawkins et al., 2014). Three emotion composite scores were created by averaging pairs of emotions responses: Anger (irritable and angry), Self-conscious (ashamed and guilty), and Unhappy (excited and happy, reverse-scored). Cronbach’s alphas for the pairs of emotions ranged from .73 to .92.

At the beginning of the study, participants completed baseline assessments of all BPD symptoms and BPD-relevant emotions (including the two questions about suicidal behavior and thoughts) as experienced in the 18 months immediately prior to the study.

Experimental Manipulation

Participants were randomly assigned to groups differing only in the ESM and daily assessments. The Control Assessment group received the 23 core questions related to non-suicide BPD symptoms and BPD-relevant emotions. The Intensive Suicide Assessment group received those same questions, with two additional questions about suicidal behavior and thoughts - “I tried to kill myself in the last 60 minutes [or, “today”]” and “I thought about committing suicide in the last 60 minutes [or today”]”¹. Although these suicide-relevant items were received only by participants in the experimental condition during ESM and daily assessments, they were received by all participants during weekly and monthly assessments (i.e., representing crucial outcomes at those temporal layers).

Procedure

After a screening-in, participants engaged in a clinical interview (SIDP-IV), completed the Baseline assessments, and completed other questionnaires not relevant to the current study. Participants were randomly assigned to Control Assessment or Intensive Suicide Assessment group prior to the clinical interview.

Immediately following the interview meeting, participants began the 2-week Main Observation phase. ESM assessments were conducted via PDA’s. For Daily, Weekly, and (later) Monthly assessments, participants chose an online format or a take-home paper format. Participants were told that assessments would not effective for communicating crises, as staff would not see responses in the near future, and they received contact information for resources in case crises emerged. For safety reasons, each ESM assessment occasion began by encouraging participants to contact mental health providers or the emergency room in case of crisis.

After the Main Observation phase, participants met with study staff to return PDAs and for a general assessment of well-being and safety. Potential safety procedures were based upon existing risk assessments (i.e., Linehan, 2009 Reynolds et al., 2006) and included questions about present intent to commit suicide, suicide risk factors (e.g., plan, severe hopelessness, suicide history, recent losses), and protective factors (e.g., family responsibility, protective social network). No participant required implementation of these protocols.

Stringent inclusion criteria were used for subsequent analyses. ESM reports and individual items were excluded from analyses if they were: completed too quickly (i.e., 500 ms or faster), completed more than three hours after a designated start time or five minutes prior to a designated start time, or if there was an insufficient number of reports (i.e., less than 13 valid reports, to ensure reasonable exposure to the study’s manipulation). In addition, participants were required to have completed at least one Weekly or Monthly assessment, which included measurement of the most crucial outcomes – suicide attempts and suicidal thoughts.

Initial analyses of attrition reveal no clear negative response effects of intensive suicide assessment. Of the 282 participants enrolled, 270 returned any ESM data (97% of participants originally assigned to Control Assessment, 95% of those assigned to Intensive Suicide Assessment, p > .05). Of those, 255 provided sufficient valid ESM responses (92% of participants originally assigned to the Control Assessment, 89% of those originally assigned to Intensive Suicide Assessment, p > .05). Seven of those 255 participants completed no Weekly or Monthly assessments and were excluded (5 assigned to Control Assessment, 2 assigned to Intensive suicide Assessment, p > .05).

Thus, the total sample size for analyses was N = 248 (Control N=119, Intensive Assessment N=129; 89% and 87% of participants originally enrolled in the two groups, respectively, X²=.18, p=.67), though sample sizes varied by specific time points, due to attrition². Across the 14 days of the Main Observation phase, participants completed a mean of 44.0 valid ESM reports (Min=13, Max=68)³, a mean of 12.6 Daily reports (Min=1, Max=14), a mean of 1.6 Weekly reports (Min=0, Max=2), and a mean of 3.6 monthly reports (Min=0, Max=6). The Control and Intensive Suicide Assessment groups did not differ in mean number of reports for any of these assessments (all, p’s>.40).

Analyses

The three key outcomes (suicide attempts, suicidal ideation, and self-harming behaviors) had limited variability and extreme non-normality, with most participants reporting no suicidal thoughts or suicide attempts. These low base rates violate key assumptions underlying familiar parametric procedures. Thus, we dichotomized these outcomes, where a rating higher than “never” (i.e., 0 on the 0-to-5 scale) was treated as an occurrence of the outcome.

For main analyses of the three key outcomes, we used generalized linear mixed models via SAS proc glimmix. Outcomes were modeled as a binary distribution with a logit link function, with occasions of assessment (i.e., 2 Weeklies and 6 Monthlies) nested within participant. Focusing on effects of intensive suicide assessment, we calculated 1-df contrasts between the Control and Intensive Suicide Assessment groups. All tests were directional, evaluating the hypothesis that the Intensive Suicide Assessment group had higher scores on the outcomes (with all scored so that higher scores reflected greater problems). We examined effects across the entire study (Week 1 of Main Observation through Month 6 of Follow-up), and then focused on each phase. We conducted analyses separately for the entire sample, the Non-BPD group, and the BPD group. These analyses produce Odds Ratios (ORs) reflecting the degree to which the odds of an event (e.g., suicide attempt) are greater among participants in the Intensive Suicide Assessment group than in the Control Assessment group. Directionally, ORs >1.0 indicate more events in the Intensive Suicide Assessment group than in the Control Assessment group, while ORs <1.0 indicate fewer events within the Intensive Suicide Assessment group. Significance tests were directional, testing the hypothesis that ORs would be greater than 1.0.

For outcomes of secondary interest (e.g., Total BPD symptom composite), we similarly used generalized linear mixed models and directional contrasts to evaluate the manipulation. Dependent variables were checked for homogeneity of variance, with very little evidence of problems. Shapiro-Wilks tests suggested departures from normality; however, this test may over-identify non-normality. Most variables had skew < |2| and kurtosis < |7|, suggesting that non-normality was not too severe (Curran, West, & Finch, 1996). Thus, we assumed normality⁴. We focused on several temporal layers: a) the entire study (as reflected by Weekly and Monthly assessments), b) the Main Observation phase (as reflected, separately, by Weeklies, Dailies, and ESM assessments, and c) the Follow-up phase (as reflected by Monthly assessments). Assessment occasions (e.g., 14 Daily assessments, or 70 ESM assessments) were nested within individuals, requiring the use of mixed modeling.

Given the nature of the outcomes we examine, our analytic strategy errs on the side of avoiding Type II errors (i.e., missing a truly harmful effect) at the risk of committing Type I errors (i.e., erroneously concluding that an effect does exist). We examine many outcomes across and within diagnostic groups, and we use one-tailed tests without adjustments for Type I error. We believe that this approach is appropriate, considering the implications of failing to identify effects with potential life-or-death consequences.

Results

Demographics and Baseline Differences

Table 2 presents demographic information for each group. Groups did not differ significantly on any demographic variables.

Table 2.

Demographic Information, Baseline Differences, and Response Rates

	Group
	Intensive Suicide Assessment (N = 129)	Control Assessment (N = 119)
Diagnosed with BPD	30.2%	30.2%
Sex (% Female)	61.2%	73.9%
Ethnicity
Black or African American	28.7%	37.0%
White	62.0%	57.1%
Other	6.3%	4.2%
Ethnicity (Hispanic)	3.1%	.8%
Therapy / Medication (At screen-in)	32.8%	37.2%
Age (Mean)	43.2	44.7
Education (Mean # of years)	14.7	14.8
Endorsement (18 months preceding study)
Suicide Attempts	8.3%	3.8%
Suicidal Ideation	31.1%	30.6%
Self-harm	13.8%	9.3%
Response Rates
Week 1	83.7%	85.7%
Week 2	77.5%	79.8%
Month 1	65.9%	71.4%
Month 2	62.8%	66.4%
Month 3	59.7%	63.0%
Month 4	56.6%	58.0%
Month 5	55.0%	52.9%
Month 6	45.7%	52.9%

Note. N for specific variables varies due to missing data.

Not surprisingly, participants with BPD reported higher baseline levels of suicidal thoughts, attempts, and all other BPD-relevant symptoms and emotions (Median r=.41, all p’s<.01, two-tailed). As shown in Table 2, the experimental groups (i.e., Control and Intensive Suicide Assessment) did not differ significantly in baseline suicide attempts, suicidal thoughts, and self-harm, nor were there baseline differences in other BPD symptoms and emotions.

We examined baseline differences between experimental groups separately by diagnostic group. Results revealed one marginally significant difference – among participants with BPD, the Intensive Suicide Assessment group reported more suicide attempts than the Control Assessment group (Odds Ratio=7.84, p=.06, two-tailed). Specifically, 7 of 32 in the Intensive Suicide Assessment group reported a recent attempt, but only 1 of 29 in the Control group reported a recent attempt. Thus, among participants with BPD, despite random assignment, there was a potentially meaningful pre-existing group difference in suicidal behavior.

Differential Attrition and Response Rates

Table 2 presents response rates across Weekly and Monthly assessments, revealing some attrition. Importantly, the experimental groups did not differ significantly in responses at any time point (all p>.25, two-tailed) and did not differ in the average number of weekly and monthly responses (M_Control=5.3, M_{IntensiveSuicide}=5.1, d=.11, t=.81, df=246, p=.42, two-tailed). Participants with BPD provided fewer weekly and monthly responses, on average, than participants without BPD (M_BPD=4.7, M_Non-BPD=5.4, d=.30, t=2.13, df=246, p=.03), and at Months 2 and 4 (p’s<.05, two-tailed). Participants who reported recent experiences of self-harm at baseline provided fewer weekly and monthly reports than participants reporting no such experiences (M_Self-harm=5.5, M_No-Self-harm=4.4, d=.49, t=2.27, df=212, p=.02, two-tailed). The number of weekly and monthly responses was not significantly related to any other baseline measure of study outcomes.

General BPD Effects

Not surprisingly, the BPD group differed significantly from the Non-BPD group on all outcomes at all time-points. For example, generalized linear mixed modeling revealed significant differences in the number of participants reporting suicide attempts, suicidal ideation, and self-harm across the entire study (Odds Ratios=6.51, 4.29, and 8.85, respectively, all p<.01, one-tailed). Similarly, analyses of Weekly and Monthly assessments revealed a significant difference in the Total BPD symptom aggregate (M_BPD=1.83, M_Non-BPD=.82, t=9.58, p<.0001, one-tailed).

Key Outcomes: Suicide Attempts, Suicidal Ideation, and Self-harm

Importantly, no participants died during the study, as indicated by direct contact after follow up, indirect contact, or an online search covering nationwide death records. Thus, the most extreme possible consequence of intensive suicide assessment – actual suicide completion – did not occur in this sample.

Table 3 presents the raw percent of participants reporting at least one occurrence of a suicide attempt, suicidal ideation, or self-harm, across the entire study and within each phase. Over the entire study, approximately 5% of the sample reported a suicide attempt, 29% reported suicidal ideation, and 13% reported self-harm. Table 3 also presents Odds Ratios (ORs) reflecting contrasts between experimental groups.

Table 3.

Participants Reporting at Least One Occurrence of Suicide Attempts, Suicidal Ideation, and Self-harm: Percent (By Experimental Group), Odds Ratio, and 95% Confidence Interval

	Suicide Attempts			Suicidal Ideation			Self-Harm
	Int Suic Cont Assess. Assess.		OR (95% CI)	Int Suic Assess.	Cont Assess.	OR (95% CI)	Int Suic Assess.	Cont Assess.	OR (95% CI)
All Participants
Entire Study	3.1%	7.6%	.55(.14,2.18)	31.0%	26.1%	1.35 (.78,2.36)	12.4%	12.6%	1.00 (.42,2.36)
Main Obs.	.01%	5.5%	.13(.01,1.40)	18.3%	15.6%	1.37 (.65,2.89)	4.2%	9.2%	.47 (.14,1.57)
Follow-up	3.8%	2.8%	1.61(.31,8.29)	32.7%	24.3%	1.36 (.74,2.47)	13.5%	10.3%	1.43 (.56,3.66)
Non-BPD Group
Entire Study	0%	6.0%	.00 (.00,--)	21.1%	20.5%	.88 (.43,1.80)	6.7%	9.6%	.51 (.16,1.66)
Main Obs.	0%	3.8%	.00 (.00,--)	7.1%	11.4%	.87 (.29,2.62)	1.1%	5.1%	.25 (.03,2.49)
Follow-up	0%	2.8%	.00 (.00,--)	20.8%	20.5%	.87 (.40,1.87)	11.1%	20%	.66 (.18,2.47)
BPD Group
Entire Study	10.3%	11.1%	1.13 (.23,5.63)	53.8%	38.9%	2.09 (.79,5.50)†	25.6%	19.4%	1.24 (.41,3.75)
Main Obs.	2.8%	10.0%	.21 (.02,2.54)	44.4%	26.7%	1.74 (.54,5.65)	6.9%	8.2%	.50 (.11,2.15)
Follow-up	12.5%	2.9% 5.14 (.52,50.84)†		59.4%	32.3%	2.35 (.83,6.63)†	28.1%	14.7%	2.02 (.60,6.83)

Note.

^†p < .10,

^*p < .05,

^**p < .01.

OR = Odds Ratio, 95% CI = 95%, -- Value is undefined. Confidence Interval. All significance tests are one-tailed, CIs are two-tailed. Percentages are raw, based on the 2 Weekly and 6 Monthly assessments. OR and CI values are based on modeled estimates.

Across all participants, there were no significant effects of intensive suicide assessment on any of the key outcomes. This was true across the entire study and within each phase. Of primary interest is suicide attempts, which was, if anything, somewhat lower in the Intensive Suicide Assessment group during the Main Observation phase of the study (though our directional contrasts are not intended to detect significant effects in this direction). Many of the 95% Confidence Intervals around the ORs are fairly wide, reflecting the low base rates.

Within the Non-BPD group, there were again no significant effects. Indeed, no Non-BPD participant in the Intensive Suicide Assessment group reported a single suicide attempt. The relevant Odds Ratios are below 1.0, in the opposite direction of a harmful effect of intensive suicide assessment, and the Confidence Intervals are undefined on the upper end.

Within the BPD group, no effects were significant at p < .05, but some were marginally significant at p < .10 (one-tailed). During the Main Observation phase, the direction of the Odds Ratios (<1.0) are opposite of what would be expected if intensive suicide assessment had harmful effects on suicide attempts and self-harm. However, this pattern shifts during Follow-up, with ORs for all three outcomes directionally indicating more events in the Intensive Suicide Assessment group, and with two being marginally significant. For example, during the Follow-up phase, 4 of the 32 BPD participants in the Intensive Suicide Assessment group reported a suicide attempt, while only 1 of the 34 BPD participants in the control group reported an attempt. Additional analyses revealed that the two marginally significant effects for suicidal ideation were themselves marginally significantly larger than the corresponding effects within the non-BPD group; however, the marginally significant effect of “Follow-up” suicide attempts was not significantly larger than the corresponding effect within the non-BPD group.

Given the marginally significant pre-existing group difference in baseline reports of suicide attempts (see Table 2), we conducted additional analysis of suicide attempts at Follow-up in the BPD group. Controlling for baseline differences, the effect was non-significant (adjusted odds for Control Assessment = 0, adjusted odds for Intensive Suicide Assessment = .08, t = .13, p = 89). This suggests that the apparent group difference in suicide attempts at Follow-up may reflect pre-existing group differences in risk for suicidal behavior.

Total BPD Composite

We next examined the Total BPD symptom composite, representing the aggregate of 8 non-suicide/self-harm BPD symptoms. Table 4 presents the estimated means, mean differences, and 95% confidence intervals for the BPD composite across the entire study (i.e., averaged across the Weekly and Monthly assessments) and at each phase of the study.

Table 4.

Total BPD Symptom Composite Estimated Means and Difference (95% Confidence Interval)

Timeframe (Assessment)	Intensive Suicide Assessment	Control Assessment	Difference (95% CI)
All Participants
Entire Study (W&M)	1.17 (1.02,1.33)	1.08 (.92,1.24)	.09 (−.13,.31)
Main Obs. (W)	1.11 (.95,1.27)	.96 (.80,1.13)	.15 (−.09,.38)
Main Obs. (D)	.53 (.44, .67)	.46 (.36,.55)	.08 (−.06,.21)
Main Obs. (ESM)	.51 (.41, .60)	.47 (.37, .57)	.04 (−.11,.18)
Follow-up (M)	1.21 (1.05,1.37)	1.15 (.99,1.31)	.06 (−.16,.29)
Non-BPD
Entire Study (W&M)	.87 (.71, 1.03)	.77 (.61, .94)	.10 (−.13,.33)
Main Obs. (W)	.79 (.62, .96)	.65 (.47, .82)	.14 (−.10,.38)
Main Obs. (D)	.37 (.27, .47)	.29 (.19, .39)	.08 (−.06,.22)
Main Obs. (ESM)	.33 (.23, .43)	.28 (.17, .38)	.05 (−.09,.20)
Follow-up (M)	.93 (.77, 1.09)	.85 (.68, 1.02)	.08 (−.15,.32)
BPD
Entire Study (W&M)	1.86 (1.62,2.11)	1.80 (1.55,2.05)	.06 (−.29,.41)
Main Obs. (W)	1.86 (1.60,2.11)	1.72 (1.45, 1.99)	.14 (−.24,.51)
Main Obs. (D)	.91 (.76, 1.05)	.86 (.71, 1.02)	.05 (−.17,.26)
Main Obs. (ESM)	.90 (.75, 1.06)	.89 (.73, 1.05)	.01 (−.21,.24)
Follow-up (M)	1.87 (1.62,2.12)	1.84 (1.59, 2.10)	.03 (−.33,.39)

Note: Significance tests were one-tailed, CIs are two-tailed. None of the differences reached significance at p < .10 or p < .05. W = Weekly assessments, M = Monthly assessments, D = Daily assessments, ESM = ESM assessments

Results reveal no significant effects of intensive suicide assessment on the BPD composite, either for the sample as a whole, or for each diagnostic group. Although the Intensive Suicide Assessment group’s means are somewhat larger than the Control groups’ means, these differences are quite small, being well-within 95% confidence intervals. Thus, there is no evidence that intensive suicide assessment increases the constellation of BPD symptoms as a whole. The BPD and Non-BPD groups did not differ on the magnitude of these effects.

Specific BPD Symptoms and BPD-relevant emotions

To explore the possibility that intensive suicide assessment might affect specific symptoms or emotions, we probed for effects on 8 specific BPD symptoms and 3 BPD-relevant emotions. For each, we examined five assessment periods/strategies, paralleling those reported in Table 4. We conducted 165 one-tailed contrasts – 55 (11 symptoms & emotions X 5 assessment periods/strategies) separately for the entire sample and for each diagnostic group. We summarize these exploratory findings here, with full results available in an online supplement.

Results reveal little, though some, evidence of effects of intensive suicide assessment on specific symptoms and emotions. Of the 165 contrasts, only 11 were significant at p < .05 (6.7%), with another 18 marginally significant at p < .10 (11%). These potential effects were observed mostly during the Main Observation phase of the study, with only 2 appearing (marginally significantly) during Follow Up. Across all participants, 11 significant or marginally significant effects were primarily found with four outcomes – Emptiness, Impulsivity, Unreality (i.e., paranoia and dissociation), and Self-conscious emotions (i.e., guilt and shame). Within the Non-BPD group, 9 significant or marginally significant effects were concentrated on Emptiness and Unreality. Finally, within the BPD group, 8 significant or marginally significant effects were concentrated on Efforts to Avoid Abandonment, Impulsivity, and Self-conscious emotions⁵. Additional analyses revealed that seven of the significant (or marginally significant) effects in the BPD group were significantly or marginally significantly larger than the corresponding effects in the Non-BPD group (the exception being the Daily assessment’s Impulsivity).

Discussion

Overall, this study provides little evidence that repeatedly assessing suicidal thoughts and behaviors has widespread or robustly harmful effects, even among a group of people at heightened risk for such effects. Very few effects were observed across all outcomes, all time points, and all methods of assessment.

Though there is some indication that people with BPD might experience effects related to suicidal ideation and behavior, these results are weak at best. They are only marginally significant, and the effect on suicide attempts disappears when controlling for pre-existing group differences in suicidality. This pattern of weak effects parallels findings by Reynolds et al. (2006), who reported both a null effect and a marginally-significant effect of suicide assessment on suicidality among people with BPD (all of whom we also selected based on current and chronic suicidality). Moreover, to the degree that the effects found in the current study are real, most appear to be temporary. In sum, the current results lead us to conclude that repeatedly assessing suicide is unlikely to cause the most detrimental outcomes, and any other outcomes, if they arise at all, are likely to be limited in scope, severity, and duration.

Although we see these results as raising little cause for concern about intensive suicide assessment, a cautious interpretation would note the few effects that approached or reached significance. In terms of the most serious outcomes – suicide attempts, suicidal ideation, and self-harm – several findings were in the direction of a problematic effect among people with BPD during the follow-up phase (though it is likely that the apparent effect on suicide attempts was due to small pre-existing difference in the experimental condition, despite random assignment to condition). Although none of these effects reached conventional levels of statistical significance, a cautious approach would be to adopt a less stringent criterion and to consider the possibility that intensive suicide assessment may affect these variables among people with BPD in a way that is delayed, rather than immediate or concurrent with assessment.

In terms of the secondary outcomes (i.e., non-suicidal BPD symptoms and BPD-relevant emotions), the BPD symptom composite revealed no effects in any group at any time point. However, exploratory analyses of specific symptoms and emotions suggested that several effects were significant during the Main Observation phase. Again, these potential effects should be considered in the context of the number of tests and the timeframe of occurrence. Across all relevant tests, approximately 5% reached the conventional level of statistical significance, in line with what might be expected by chance. Thus, it is quite possible that the few significant effects would not replicate. In addition, the significant effects generally were found only during the Main Observation phase. Thus, the apparent effects, even if real and replicable, generally seem to be temporary. However, a cautious approach would entertain the possibility that such effects – particularly those replicating across assessment methods – reflect real effects. Of perhaps most potential concern would be the potential effects among people with BPD – increased impulsivity, self-conscious emotions, and efforts to avoid abandonment.

Caveats and limitations

Though this is arguably one of the broadest studies of suicidal assessment, in terms of participants, outcomes, and timeframe, other populations might be more strongly affected by intensive suicide assessment than those in the current study, or affected in ways not assessed in this study. For example, our sample did not include participants below the age of 18 or above 65, and it is possible that people in those age ranges respond differently than our participants. Second, participants were not given explicit definitions of suicide attempts or any other variable. Thus, participants might have interpreted “I thought about committing suicide” or “I tried to kill myself” in different ways. Third, this study focused only on the effect of intensive suicide assessment, over and above any effects of assessing potentially distressing experiences more generally. This study was not designed to evaluate those potential effects, be they harmful (e.g., Merrilees et al., 2008) or beneficial (e.g., Nishina, 2012). Fourth, our analytic strategy was “conservative” in that we minimized Type II errors at the risk of committing Type I errors. Given the importance of our outcomes, we wanted to avoid missing a truly harmful effect, at the risk of identifying an effect that might not truly exist. Thus, we conducted many tests (>200), examining many outcomes (15 in all) across and within diagnostic groups, and we used one-tailed tests without adjustments for Type I error. Although this approach is not “conservative” in the typical statistical sense, we believe it is appropriately conservative in the sense of reducing the likelihood of failing to identify effects with potential life-or-death consequences.

Finally, some outcomes – particularly suicide attempts – were rare. This is, of course, “good news” at the most fundamental level. The fewer participants who endure such experiences the better, and “none” would be ideal. However, low base rates limit effect sizes, power, and precision of analyses (Furr & Bacharach, 2014). For example, given our base rates of suicide attempts (5% across the entire sample, 11% in the BPD group), the largest effect sizes we could have obtained were small, at Φ = .23 (for the entire sample) and Φ = .33 (for the BPD subgroup)⁶. Our sample sizes (N=248 for the entire sample, N=75 for the BPD group) produced power of approximately .90 or higher for effects of this size, which are considered small (for the entire sample) to moderate (for the BPD group). However, the effects actually produced by intensive suicide assessment were generally very small (i.e., smaller than what our sample sizes are powered to detect) or in even the opposite direction of a hypothesized harmful effect. For example, the effect of intensive suicide assessment on Follow-up suicide attempts among people with BPD was one of the “larger” effects, at only Φ = .18, producing power of .43. Considering this, and considering similar effects on the precision of confidence intervals, despite a sample larger than in many similar studies, the current estimates in Table 3 are relatively rough estimates of the true effects, particularly within diagnostic groups.

Implications

The current results speak to clinical care and research in the use of repeated suicide assessment. Repeated assessment of suicidal ideation and behavior offers potential benefits, including providing insight into patients’ self-harm or suicide potential and allowing researchers to examine important novel scientific questions with relative precision and rigor. The benefits of implementing such a methodology must be weighed against any potential costs.

At least two issues merit consideration in any potential implementation of repeated suicide assessment. The first is whether any use of such assessments is warranted. The current study indicates that the most serious potential effects are unlikely to occur (see also Crawford et al., 2011; Cukrowicz, Smith, & Poindexter, 2010; Gould et al., 2005; Lang, Uttaro, Caine, Carpinello, & Felton, 2009; Mathias et al., 2012; Reynolds et al., 2006), particularly for people not already at risk for suicidal behavior/ideation. Going further, current findings provide highly tentative indications that the effects of intensive suicide assessment may include limited and generally temporary cognitive, emotional, or behavioral consequences, which might be distressing and disruptive for some people (e.g., feelings of abandonment or emptiness). However, given the serious nature of the relevant outcomes, there is some merit to a cautious interpretation of these findings. Thus, taking a cautious approach, mental healthcare providers, researchers, and institutional review boards should balance these potential costs (particularly for people with BPD) against the potential benefits to patients or to people whose lives can be improved through advanced understanding of suicidality. The potential for psychological distress (albeit temporary) suggests that potential benefits should be meaningful and real. In a research context, the information gained from a study must have the capacity to advance theory or practice. The current study and similar studies indicate minimal, or no, significant or enduring psychological costs for many people. Thus, we believe that the use of intensive suicide assessment represents an ethically warranted and potentially informative tool in psychological research and, potentially, in clinical practice. In a clinical context, providers might find repeated assessment of suicidality to be beneficial, particularly where the risk of suicide is high and where close monitoring is needed. However, additional research is needed to assess the utility of such ongoing assessments as part of clinical practice, which was not the focus of the current study.

A second issue is whether safeguards should accompany repeated suicide assessment. Again, we believe that the current research indicates minimal, if any, risks associated with intensive suicide assessment, even among a group of participants at high risk of suicide. However, given the importance of potential outcomes, we suggest some caution, again particularly for people with BPD. For example, in our own work, we implemented several safeguards, including providing emergency contact information for mental health organizations and suicide hotlines, beginning each ESM assessment with reminders to contact a provider or an emergency room if participants intended to harm themselves, and assessing suicidal risk in-person following the repeated suicide assessment period. Until additional research further verifies the lack of robust and pervasive effects, we encourage other researchers and providers to similarly monitor potential distress during and following repeated suicide assessment and to ensure that individuals in healthcare settings and in research studies are provided with resources in case they experience distress. Given the marginally significant effects in our Follow-up phase, a cautious approach would also recommend monitoring patients or participants for some period (e.g., some months) after concluding intensive suicide assessment.

In conclusion, we believe that important advances can be made in understanding and treating suicide, and the need for such advances is urgent. Moreover, we believe that intensive assessment of suicidal thoughts and behaviors may have an important role to play in these advances. The current research suggests that these assessments are unlikely to be harmful; in contrast, intensively assessing suicide may ultimate help save many troubled lives, either indirectly through advances in research or directly through advances in treatment.