Effect of Offering Care Management or Online Dialectical Behavior Therapy Skills Training vs Usual Care on Self-harm Among Adult Outpatients With Suicidal Ideation: A Randomized Clinical Trial

Gregory E Simon; Susan M Shortreed; Rebecca C Rossom; Arne Beck; Gregory N Clarke; Ursula Whiteside; Julie E Richards; Robert B Penfold; Jennifer M Boggs; Julia Smith

doi:10.1001/jama.2022.0423

. 2022 Feb 15;327(7):1–10. doi: 10.1001/jama.2022.0423

Effect of Offering Care Management or Online Dialectical Behavior Therapy Skills Training vs Usual Care on Self-harm Among Adult Outpatients With Suicidal Ideation

A Randomized Clinical Trial

Gregory E Simon ^1,^✉, Susan M Shortreed ¹, Rebecca C Rossom ², Arne Beck ³, Gregory N Clarke ⁴, Ursula Whiteside ⁵, Julie E Richards ¹, Robert B Penfold ¹, Jennifer M Boggs ³, Julia Smith ¹

¹Kaiser Permanente Washington Health Research Institute, Seattle

²HealthPartners Institute, Minneapolis, Minnesota

³Kaiser Permanente Colorado Institute for Health Research, Denver

⁴Kaiser Permanente Northwest Center for Health Research, Portland, Oregon

⁵NowMattersNow.org, Seattle, Washington

^✉

Corresponding Author: Gregory E. Simon, MD, MPH, Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave #1600, Seattle, WA 98112 (gregory.e.simon@kp.org).

Accepted for Publication: January 12, 2022.

Author Contributions: Dr Shortreed and Ms Smith had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Simon, Shortreed, Rossom, Beck, Whiteside, Penfold.

Acquisition, analysis, or interpretation of data: Simon, Shortreed, Rossom, Beck, Clarke, Richards, Boggs, Smith.

Drafting of the manuscript: Simon, Shortreed, Whiteside, Smith.

Critical revision of the manuscript for important intellectual content: Simon, Shortreed, Rossom, Beck, Clarke, Whiteside, Richards, Penfold, Boggs.

Statistical analysis: Shortreed, Smith.

Obtained funding: Simon, Rossom, Whiteside.

Administrative, technical, or material support: Simon, Rossom, Beck, Clarke, Whiteside, Richards, Penfold, Boggs.

Supervision: Simon, Shortreed, Beck, Clarke, Whiteside.

Conflict of Interest Disclosures: Drs Simon, Shortreed, and Richards and Ms Smith are employees of Kaiser Permanente Washington; Dr Rossom is an employee of HealthPartners Institute; Dr Beck is an employee of Kaiser Permanente Colorado; Dr Clarke is an employee of Kaiser Permanente Northwest; and Dr Whiteside is the principal of NowMattersNow LLC. Dr Shortreed reports having been a coinvestigator on Kaiser Permanente Washington Health Research Institute projects funded by Syneos Health, who was representing a consortium of pharmaceutical companies carrying out US Food and Drug Administration (FDA)–mandated studies regarding the safety of extended-release opioids. Dr Rossom reported receipt of grants from the Patient-Centered Outcomes Research Institute, the FDA, Bioxcel, Otsuka, and Adelphi. No other disclosures were reported.

Funding/Support: This trial was funded by National Institute of Mental Health (NIMH) cooperative agreement UH3 MH007755.

Role of the Funder/Sponsor: The NIMH provided advice during the trial regarding recruitment progress, sample size estimates, and intervention delivery. The NIMH also provided advice regarding interpretation of findings and additional post hoc analyses. The NIMH had no role in approval of the manuscript or decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 3.

Additional Contributions: We acknowledge the contributions of people living with suicidal ideation, both as study participants and as members of our research team.

^✉

Corresponding author.

PMCID: PMC8848197 PMID: 35166800

Key Points

Question

Can low-intensity outreach programs, based on effective clinical interventions but delivered primarily online, prevent self-harm or suicidal behavior among outpatients reporting frequent suicidal ideation?

Findings

In this pragmatic randomized clinical trial that included 18 882 outpatients with frequent suicidal ideation, the percentage with nonfatal or fatal self-harm over 18 months was 3.3% among those offered care management, 3.9% among those offered online dialectical behavior therapy skills training, and 3.1% among those receiving usual care, respectively. Compared with usual care, the risk of self-harm was not significantly different for care management but was significantly increased for those offered skills training.

Meaning

Compared with usual care, offering care management did not significantly reduce the risk of self-harm, and offering brief online dialectical behavior therapy skills training increased the risk of self-harm among at-risk adults.

Abstract

Importance

People at risk of self-harm or suicidal behavior can be accurately identified, but effective prevention will require effective scalable interventions.

Objective

To compare 2 low-intensity outreach programs with usual care for prevention of suicidal behavior among outpatients who report recent frequent suicidal thoughts.

Design, Setting, and Participants

Pragmatic randomized clinical trial including outpatients reporting frequent suicidal thoughts identified using routine Patient Health Questionnaire depression screening at 4 US integrated health systems. A total of 18 882 patients were randomized between March 2015 and September 2018, and ascertainment of outcomes continued through March 2020.

Interventions

Patients were randomized to a care management intervention (n = 6230) that included systematic outreach and care, a skills training intervention (n = 6227) that introduced 4 dialectical behavior therapy skills (mindfulness, mindfulness of current emotion, opposite action, and paced breathing), or usual care (n = 6187). Interventions, lasting up to 12 months, were delivered primarily through electronic health record online messaging and were intended to supplement ongoing mental health care.

Main Outcomes and Measures

The primary outcome was time to first nonfatal or fatal self-harm. Nonfatal self-harm was ascertained from health system records, and fatal self-harm was ascertained from state mortality data. Secondary outcomes included more severe self-harm (leading to death or hospitalization) and a broader definition of self-harm (selected injuries and poisonings not originally coded as self-harm).

Results

A total of 18 644 patients (9009 [48%] aged 45 years or older; 12 543 [67%] female; 9222 [50%] from mental health specialty clinics and the remainder from primary care) contributed at least 1 day of follow-up data and were included in analyses. Thirty-one percent of participants offered care management and 39% offered skills training actively engaged in intervention programs. A total of 540 participants had a self-harm event (including 45 deaths attributed to self-harm and 495 nonfatal self-harm events) over 18 months following randomization: 172 (3.27%) in care management, 206 (3.92%) in skills training, and 162 (3.27%) in usual care. Risk of fatal or nonfatal self-harm over 18 months did not differ significantly between the care management and usual care groups (hazard ratio [HR], 1.07; 97.5% CI, 0.84-1.37) but was significantly higher in the skills training group than in usual care (HR, 1.29; 97.5% CI, 1.02-1.64). For severe self-harm, care management vs usual care had an HR of 1.03 (97.5% CI, 0.71-1.51); skills training vs usual care had an HR of 1.34 (97.5% CI, 0.94-1.91). For the broader self-harm definition, care management vs usual care had an HR of 1.10 (97.5% CI, 0.92-1.33); skills training vs usual care had an HR of 1.17 (97.5% CI, 0.97-1.41).

Conclusions and Relevance

Among adult outpatients with frequent suicidal ideation, offering care management did not significantly reduce risk of self-harm, and offering brief dialectical behavior therapy skills training significantly increased risk of self-harm, compared with usual care. These findings do not support implementation of the programs tested in this study.

Trial Registration

ClinicalTrials.gov Identifier: NCT02326883

This randomized trial assesses the effect of offering care management or online dialectical behavior therapy skills training vs usual care on risk of self-harm among adult outpatients with suicidal ideation.

Introduction

In studies including more than 80 000 people attempting or dying by suicide, greater than 60% had health care contacts in the prior 3 months.^1,2 Self-report questionnaires³ or algorithms using electronic health record (EHR) data⁴ have been shown to accurately identify people at increased risk. The Joint Commission⁵ and the National Action Alliance for Suicide Prevention⁶ have recommended systematic identification of suicide risk among people receiving mental health care.

Prevention of suicidal behavior will require effective interventions scalable to the large population of people at risk. Structured psychotherapies such as dialectical behavior therapy (DBT)^7,8 and cognitive behavior therapy⁹ have reduced suicide attempts in people accepting treatment after recent self-harm or hospitalization. Collaborative care and care management interventions may be associated with improved treatment adherence and symptom outcomes in people initiating treatment for specific mood and anxiety disorders.¹⁰ However, these effective clinical interventions have not been tested in broader populations, including those less likely to seek or accept treatment. Interventions limited to recent survivors of suicide attempt or others at highest risk cannot reach most people who attempt or die by suicide.^1,2 Broad population-based suicide prevention programs are supported by before-after comparisons^11,12 but not by randomized trials. Early trials found that caring contacts or brief supportive outreach reduced risk of suicide attempt among people refusing or discontinuing follow-up care after a crisis,¹³ but subsequent trials have yielded mixed results.^14,15

This pragmatic randomized trial evaluated whether either of 2 low-intensity outreach programs, incorporating selected elements of effective clinical interventions, could reduce risk of self-harm or suicidal behavior across the population of outpatients reporting frequent suicidal ideation on routine depression questionnaires.

Methods

Trial methods and protocols are described in detail elsewhere,^16,17 and the trial protocol is included as Supplement 1.

Institutional review boards in each health system approved waiver of consent to use records data to identify participants, waiver of consent for randomization to usual care or offer of outreach interventions, an abbreviated consent procedure at the time of the initial intervention invitation (described below), and waiver of informed consent to use records data to ascertain study outcomes.

Participants

Participating health systems (HealthPartners and the Colorado, Northwest, and Washington regions of Kaiser Permanente) provide insurance coverage and comprehensive mental health and medical care to more than 4 million members in Minnesota, Colorado, Oregon, and Washington. Members are insured through employer-sponsored plans, Medicare, Medicaid, and subsidized Affordable Care Act plans, with race and ethnicity representative of each health system’s service area.¹⁸

Participating health systems routinely administered the 9-item Patient Health Questionnaire (PHQ-9)^3,19 at all specialty mental health visits and at primary care visits for depression treatment. Between March 2015 and September 2018, adults reporting thoughts of death or self-harm “more than half the days” or “nearly every day” during the past 2 weeks (ie, score of 2 or 3 on item 9) were identified from EHR data and enrolled in the trial, subject to the following exclusion criteria: not currently insured by the health system, no prior use of EHR patient portal messaging, recorded diagnosis of dementia or developmental disability, or an EHR indicator of needing an interpreter. Previous research in these health systems found that approximately 6% of completed PHQ-9 questionnaires had scores of 2 or 3 on item 9,³ and approximately 65% of those patients used EHR portal messaging.²⁰ Data regarding established predictors of self-harm,⁴ including sex, age, race and ethnicity, prior mental health diagnoses, and prior mental health service use, were extracted from health system records. Race and ethnicity in health system records were reported by patients at the time of outpatient visits using standard categories for race (American Indian, Asian, Black, Native Hawaiian or Other Pacific Islander, White) and ethnicity (Hispanic or non-Hispanic).

Randomization

Following a Zelen design,^21,22 eligible patients identified each week were automatically randomly assigned to 1 of 3 groups: offer of a care management intervention, offer of an online program to learn 4 specific DBT skills, or a control condition offering no additional services. Permuted block randomization with block sizes of 6 or 9, stratified by study site and baseline response to PHQ-9 item 9, occurred automatically at enrollment using concealed randomization tables (see eAppendix 1 in Supplement 2 for details).

Interventions

Both interventions began with a series of invitation messages from a study clinician. Initial invitations were sent through the EHR patient portal,^20,23 followed by reminder messages and/or telephone calls for those not responding (see eAppendix 2 in Supplement 2 for details). Each invitation included a brief message of support based on caring message interventions,^13,24,25 a description of the extra services offered, and abbreviated informed consent information (notification that extra services were part of research and that patients could decline or withdraw at any time). Those not responding to the initial invitation received up to 2 additional cycles of invitation and reminder (including caring message content) 4 and 8 weeks later. Those who actively declined or did not respond to 3 cycles of invitation were not contacted again but could accept extra services for up to 12 months. Those accepting the invitation received specific additional services as described below.

The care management intervention, modeled after collaborative care programs²⁶ and the Henry Ford Health System Perfect Depression Care program,¹¹ included regular outreach for assessment of suicide risk using the Columbia Suicide Severity Rating Scale (C-SSRS),²⁷ leading to guideline-based recommendations regarding outpatient follow-up. Intervals between outreach contacts varied according to C-SSRS risk level at last contact, ranging from 1 week or less for participants reporting suicidal intent with a specific plan to 2 months or more for participants reporting no recent suicidal ideation. Care managers’ follow-up messages included both motivational enhancement and care navigation to promote recommended outpatient care. Care managers regularly communicated risk assessment results and follow-up recommendations to treating outpatient clinicians who were responsible for all decisions regarding specific treatments. Study care managers were master’s degree–level mental health clinicians who received 14 hours of intervention-specific training followed by twice-monthly supervision teleconferences with investigators.

The skills training program, drawing from skills training in traditional DBT,^7,28 included an interactive online program supported by a skills coach. The online program included video instruction introducing and demonstrating 4 specific DBT skills: mindfulness, mindfulness of current emotion, opposite action, and paced breathing. Skills coaches did not provide psychotherapy but sent EHR portal messages to reinforce each visit to the online program and encourage practice of specific skills as well as outreach messages to participants without recent visits. Frequency of outreach depended on each participant’s level of involvement but was at least monthly during the initial 6 months. All skills coaches had completed mental health bachelor’s degree coursework and received 14 hours of initial intervention-specific training followed by twice-monthly supervision teleconferences with investigators.

Both outreach programs were supplements to usual care, and participants assigned to either intervention group were free to receive any nonstudy mental health or general medical services normally available. Intervention services were offered for up to 12 months after randomization. Additional details regarding both intervention programs are described elsewhere,¹⁶ and detailed intervention manuals are available at https://www.github.com/MHResearchNetwork/SPOT_Study. People with lived experience of suicidal ideation and suicide attempts collaborated in design of invitation materials, outreach messages, and content of the online skills training program.

Participants randomized to continued usual care were not contacted by study staff and were free to receive any nonstudy mental health or general medical services normally available. Standard care in each participating health system included routine use of the PHQ-9 at mental health visits, routine use of structured risk assessments (typically including the C-SSRS), creation of safety plans for those at high risk, and psychotherapy or pharmacotherapy to address specific mental health diagnoses.

Blinding

Patients randomized to either intervention were aware of that assignment but were not informed of the alternative intervention or the usual care group. Consistent with the Zelen design,^21,22 patients randomized to usual care were never contacted by study staff and were unaware of study interventions. Treating outpatient clinicians were notified of each participant’s intervention assignment but were not notified of assignments to usual care. Emergency department and inpatient clinicians (the source of most self-harm diagnoses) were not notified regarding the trial or the study group assignment of any individual.

Outcomes

The primary trial outcome was first observed fatal or nonfatal self-harm within 18 months after randomization. Fatal self-harm (ascertained from state mortality records; see eAppendix 3 in Supplement 2 for details) included all deaths coded as intentional self-harm (International Statistical Classification of Diseases and Related Health Problems, Tenth Revision codes X60 to X84). Nonfatal self-harm (ascertained from health system EHR and insurance claims data) included any encounter with a diagnosis of intentional self-harm and adjudicated encounters with selected injury or poisoning diagnoses coded as unintentional, coded as undetermined intent, or having no coding of intent. Procedures for review and adjudication of events in those additional categories are described in eAppendix 6 and eTables 1-5 in Supplement 2.

Secondary outcomes (specified a priori) included a narrower definition of self-harm events (leading to death or hospitalization), a broader definition of self-harm events (including potential events not confirmed by chart review), participation in interventions (assessed using EHR data for contacts with care managers and skills coaches and by logs of visits to the online skills training program), and use of nonstudy mental and general health services (assessed using EHR and insurance claims data).

Sample Size Calculation

Consultation with health system leaders indicated that a population-level relative risk reduction smaller than 25% (equivalent to an absolute risk reduction smaller than 1%) would not justify implementation of either intervention program. A priori power calculations indicated that 6500 randomized individuals per group would allow 90% power to detect a 25% reduction in risk of either intervention compared with usual care, a threshold selected in consultation with health system leaders.

Statistical Analysis

Primary analyses considered time to first fatal or nonfatal self-harm, including all participants with any follow-up time, and considering initial treatment assignment regardless of intervention participation. The care management and skills training groups were each compared with the usual care group, with Bonferroni correction accounting for 2 comparisons, leading to a 2-sided α = .025 and reporting of 97.5% confidence intervals for primary outcomes. The care management and skills training groups were not directly compared. Kaplan-Meier curves were plotted and times to first fatal or nonfatal self-harm were compared by Cox proportional hazards regression with adjustment for sex, age group, race and ethnicity, baseline PHQ-9 item 9 score, study site, and year of randomization. The proportionality assumption was tested by examination of log-log plots. Alternative analyses used log-rank tests,^29,30 stratifying for study site and PHQ-9 item 9 score at randomization. Follow-up time was censored at disenrollment from the health system (when insurance claims data to ascertain nonfatal self-harm would be missing) or death from causes other than self-harm.

Secondary outcome analyses included comparison of the intervention groups with usual care using narrower and broader definitions of self-harm events by proportional hazards regression, description of uptake and adherence for each of the intervention programs, and comparison of intervention groups with usual care in use of nonstudy mental health care. Monthly utilization was modeled as a function of time using generalized estimating equations. Twelve-month utilization was calculated by summing coefficient estimates; the Delta method was used to calculate standard errors. A t statistic was used to compare 12-month utilization in each of the intervention groups vs usual care.

Additional post hoc analyses included subgroup analyses of primary outcome comparisons with testing for interaction effects, comparison of participant baseline characteristics across different levels of intervention program participation, and comparison of self-harm rates across different levels of intervention program participation.

Given the potential for type I error due to multiple comparisons, secondary analyses and post hoc analyses are exploratory.

Analyses were performed using Stata version 15.1 (StataCorp).

Results

A total of 18 882 patients (9009 [48%] aged 45 years or older; 12 543 [67%] female; 9222 [50%] from mental health specialty clinics and the remainder from primary care) were randomized, and 238 (1.3%) were excluded from analyses because of death or disenrollment from the health system on or before the date of randomization (eAppendix 4 in Supplement 2). A participant flow diagram illustrating enrollment, intervention assignment, intervention participation, and outcome ascertainment is shown in Figure 1. Characteristics of participants contributing to outcome analyses are shown in Table 1.

Figure 1. — DBT indicates dialectical behavior therapy.

^aTo be enrolled, participants had to have completed the 9-item Patient Health Questionnaire (PHQ-9) at outpatient visits and reported thoughts of death or self-harm on “more than half the days” (PHQ-9 item 9 score of 2) or “nearly every day” (PHQ-9 item 9 score of 3) during the past 2 weeks, and had to have previously used online messaging via the electronic health record patient portal.

^bSee eAppendix 5 in Supplement 2 for details on patients not offered intervention services.

^cRandomization occurred up to 8 days following an eligible visit, and some patients died or disenrolled from the health system prior to randomization but before records of death or disenrollment were available. See eAppendix 4 in Supplement 2 for details.

Table 1. Baseline Characteristics by Treatment Assignment, Limited to Participants Contributing Any Follow-up Time to Outcome Analysis.

Characteristics	No. (%) of participants
Characteristics	Care management (n = 6230)	Skills training (n = 6227)	Usual care (n = 6187)
Sex
Female	4195 (67.3)	4160 (66.8)	4188 (67.7)
Male	2035 (32.7)	2067 (33.2)	1999 (32.3)
Age group, y
18-29	1438 (23.1)	1440 (23.1)	1457 (23.6)
30-44	1747 (28.0)	1797 (28.9)	1756 (28.4)
45-64	2069 (33.2)	2056 (33.0)	2067 (33.4)
≥65	976 (15.7)	934 (15.0)	907 (14.7)
Race and ethnicity
American Indian, non-Hispanic	56 (0.9)	42 (0.7)	35 (0.6)
Asian, non-Hispanic	179 (2.9)	183 (2.9)	194 (3.1)
Black, non-Hispanic	241 (3.9)	272 (4.4)	237 (3.8)
Hispanic	495 (7.9)	486 (7.8)	595 (8.6)
Native Hawaiian or Other Pacific Islander	24 (0.4)	29 (0.5)	23 (0.4)
White, non-Hispanic	4723 (75.8)	4651 (74.7)	4561 (73.7)
More than 1	170 (2.7)	188 (3.0)	203 (3.3)
Other or not recorded^a	342 (5.5)	376 (6.0)	399 (6.5)
Location of index visit
Mental health specialty clinic	3071 (49.3)	3111 (50.0)	3040 (49.1)
General medical clinic	3159 (50.7)	3116 (50.0)	3147 (50.9)
Baseline PHQ-9 item 9 score^b
2	4180 (67.1)	4177 (67.1)	4141 (66.9)
3	2050 (32.9)	2050 (32.9)	2046 (33.1)
Diagnoses recorded in past year
Depressive disorder	4077 (65.4)	4020 (64.6)	4058 (65.6)
Anxiety disorder	3692 (59.3)	3700 (59.4)	3653 (59.0)
Bipolar disorder	689 (11.1)	686 (11.0)	621 (10.0)
Personality disorder	528 (8.5)	561 (9.0)	510 (8.2)
Drug use disorder	462 (7.4)	469 (7.5)	451 (7.3)
Alcohol use disorder	395 (6.3)	364 (5.8)	349 (5.6)
Schizophrenia spectrum psychosis	71 (1.1)	74 (1.2)	57 (0.9)
Self-harm injury or poisoning in past year	126 (2.0)	148 (2.4)	141 (2.3)
Service use in past year^c
Mental health hospitalization	656 (10.5)	652 (10.5)	614 (9.9)
Mental health emergency department visit	1000 (16.1)	1059 (17.0)	983 (15.9)

Open in a new tab

^{^a}

Includes patients not asked about race or ethnicity and those who declined to report race or ethnicity using these standard categories.

^{^b}

On the 9-item Patient Health Questionnaire (PHQ-9), a score of 2 on item 9 represents reporting thoughts of death or self-harm on “more than half the days” and a score of 3 on item 9 represents reporting thoughts of death or self-harm “nearly every day.”

^{^c}

Hospitalization or emergency department visit with any recorded mental health diagnosis.

Primary Outcomes

Five hundred forty participants experienced an outcome event during the 18-month follow-up period, including 172 (estimated event rate, 3.27%) of those offered care management, 206 (3.92%) offered skills training, and 162 (3.10%) receiving usual care. Outcome events included 45 deaths attributed to self-harm and 495 nonfatal self-harm events (451 injuries or poisonings diagnosed as definite self-harm and an additional 44 nonfatal self-harm events identified by review of full-text records as described in eAppendix 6 in Supplement 2).

Kaplan-Meier curves illustrating time until first fatal or nonfatal self-harm in each group are shown in Figure 2. Planned comparison of the care management and usual care groups found no significant difference in risk (hazard ratio, 1.07; 97.5% CI, 0.84-1.37; P = .52). Planned comparison of the skills training and usual care groups found a significantly higher risk of self-harm in the skills training group (hazard ratio, 1.29; 97.5% CI, 1.02-1.64; P = .015). Examination of log-log plots (eFigure 3 in Supplement 2) indicated no violation of the proportional hazards assumption. Alternative analyses using a log-rank test,^29,30 stratifying for study site and PHQ-9 item 9 score at randomization, found no significant difference in risk between the care management and usual care groups (log-rank statistic = 0.26; P = .61) and significantly higher risk in the skills training group compared with usual care (log-rank statistic = 5.36; P = .02).

Secondary Outcomes

Planned analyses (eFigure 1 in Supplement 2) examined both severe self-harm and a broader outcome definition. Comparisons limited to self-harm leading to death or hospitalization (232 total events) showed the same overall pattern. Risk did not significantly differ between the care management and usual care groups (hazard ratio, 1.03; 97.5% CI, 0.71-1.51; P = .84). The point estimate for risk was higher among those offered skills training than in usual care, but this difference was not statistically significant (hazard ratio, 1.34; 97.5% CI, 0.94-1.91; P = .07). Comparisons that included additional injuries or poisonings representing plausible mechanisms of self-harm that were coded as unintentional or undetermined intent (899 total events) also showed the same pattern. Risk did not differ significantly between the care management and usual care groups (hazard ratio, 1.10; 97.5% CI, 0.92-1.33; P = .23). The point estimate for risk was higher among those offered skills training than in usual care, but this difference was not statistically significant (hazard ratio, 1.17; 97.5% CI, 0.97-1.41; P = .06).

Rates of participation in the 2 intervention programs are shown in Table 2. Approximately 31% of those offered care management initially accepted the invitation, and approximately 17% remained engaged for over 9 months. Of those offered skills training, 39% accepted the invitation, but only 2% remained engaged for over 9 months. Use of specific components of the online skills training program is shown in eTable 5 in Supplement 2.

Table 2. Participation in Study Interventions Over the 12 Months Following Randomization.

	No. (%)
	Care management (n = 6230)	Skills training (n = 6227)
Not offered intervention^a	201 (3.2)	232 (3.7)
Actively declined invitation	1345(21.6)	799 (12.8)
No response after 3 invitations	2757 (44.3)	2780 (44.6)
Ever engaged in intervention^b	1927 (30.9)	2416 (38.8)
Engaged >3 mo	1612 (25.9)	767 (12.3)
Engaged >6 mo	1392 (22.3)	268 (4.3)
Engaged >9 mo	1049 (16.8)	117 (1.9)

Open in a new tab

^{^a}

Determined by treating clinicians or study staff to be unable to participate in intervention because of advanced illness, significant cognitive impairment, or other reasons (see eAppendix 5 in Supplement 2 for details).

^{^b}

Ever actively engaged in intervention, regardless of subsequent participation or withdrawal. Definitions of engagement were specific to each intervention. Engagement in care management was defined as completion of a study risk assessment, sending an online message to the care manager, or completing a telephone encounter with the care manager. Engagement in skills training was defined as visiting the online skills training website, sending an online message to the skills coach, or completing a telephone encounter with the skills coach.

Use of nonstudy mental health services over 12 months is shown in Figure 3 and in eTable 10 in Supplement 2. Rates of outpatient mental health visits, primary care visits with mental health diagnoses, or inpatient admissions with mental health diagnoses did not differ between either intervention group and the usual care group. Participants offered skills training had a significantly higher rate of emergency department visits with mental health diagnoses over 12 months than those assigned to usual care (0.33 visits vs 0.28 visits; P = .02).

Exploratory Post Hoc Analyses

Subgroup analyses (eTable 6 in Supplement 2) found that primary outcome comparisons did not differ significantly across health systems, year of enrollment, baseline PHQ-9 item 9 response, and other indicators of baseline risk. Comparison of baseline characteristics by level of intervention participation (eTables 7 and 8 in Supplement 2) found that several demographic characteristics (sex, age distribution, race and ethnicity) and clinical characteristics (location of index visit, rates of prior mental health diagnoses) varied across levels of intervention uptake more than expected by chance. But these comparisons did not show a consistent relationship between baseline indicators of risk and specific levels or types of intervention participation. Comparison of primary outcome rates by level of intervention participation (eFigure 2 and eTable 9 in Supplement 2) found that risk of self-harm varied significantly across levels of intervention participation within the group offered care management (log-rank statistic = 15.39; P = .002) and within the group offered skills training (log-rank statistic = 25.46; P < .001). Both intervention groups showed the same pattern: lowest risk in those actively refusing intervention services, intermediate risk in those either not responding to the intervention invitation or remaining engaged in the intervention for more than 3 months, and highest risk in those engaging in the intervention for less than 3 months.

Discussion

In this pragmatic trial comparing 2 population-based outreach programs with usual care, neither program decreased risk of self-harm over 18 months. Risk of self-harm among those offered skills training was approximately 30% higher than in usual care.

This trial assessed effectiveness of outreach programs under everyday practice conditions. Participants were identified using routinely administered questionnaires, regardless of diagnosis or treatment history. All eligible patients were randomized, regardless of willingness to accept study interventions. Analyses compared groups according to randomized assignment, regardless of intervention participation. These pragmatic features would be expected to attenuate any intervention effects. That attenuation could explain the null finding regarding care management but would not explain the significantly higher rate of self-harm in the skills training group.

Between 30% and 40% of people invited actively engaged in either intervention, consistent with pilot testing of similar outreach interventions.³¹ These participation rates are lower than seen in traditional clinical trials,^7,9,32 reflecting expected participation when offered under everyday practice conditions and supported by modest outreach. Consequently, trial results reflect expected outcomes across the full spectrum of patients reporting recent suicidal ideation, including those unlikely to accept extra services. All participants randomized to either of the outreach programs received a series of outreach messages, including expressions of caring and support, an intervention found effective in some previous trials.^13,24,25 Outcomes among patients accepting or remaining engaged in interventions cannot be compared with outcomes in the entire usual care group, since it is not possible to identify usual care patients who would have accepted or remained engaged in interventions.

These findings do not imply that mental health or primary care clinicians should ignore patients’ reports of suicidal ideation. For each participant in this trial, treating health system clinicians were aware of suicidal ideation and were expected to both conduct additional assessment and arrange appropriate ongoing care. These findings indicate that neither intervention program reduced risk of self-harm when added to usual care in these health systems.

These findings contrast with previous research supporting the effectiveness of caring message interventions,^13,24,25 traditional DBT programs,^7,8,32 and care management or collaborative care interventions.^10,33 Those contrasts could reflect differences in study populations, intervention content, and intervention delivery.

Participants in this trial represented a wider range of risk and willingness to accept suicide prevention services compared with populations in other studies. While the 4% rate of self-harm in this sample was higher than expected in the general population (estimated at 0.25%²), it was markedly lower than the rates of 40% to 50% in previous DBT trials.^7,8,32 Participants in this trial did not consent in advance to any specific treatment, and only 30% to 40% actively participated in the extra services offered.

The intervention content and caring message outreach tested in this trial included expressions of caring and support modeled after effective interventions.^13,24,25 Messages in this trial, however, also included specific requests to engage in extra services. Those requests may have interfered with any benefit of caring messages.

The care management intervention tested in this trial was a low-intensity program delivered by online messaging with rare telephone contact and no in-person visits. In contrast to effective collaborative care interventions,^10,33 this intervention did not focus on improving a specific pharmacotherapy or psychotherapy. More intensive outreach or more focused care navigation to increase access to and engagement in treatment could reduce risk of self-harm, but additional research is needed to address that question.

In this study, online DBT skills training was associated with increased risk of self-harm, in contrast with previous randomized trials that supported the effectiveness of traditional DBT.^7,8,32 The skills training intervention offered in this trial differed from traditional DBT in several ways, including limitation to 4 specific skills presented in brief online format, no in-person visits or group therapy, and support limited to reminder and encouragement methods delivered via the EHR patient portal. Alternative approaches will be needed to deliver DBT to the large numbers of people at risk of self-harm or suicidal behavior. This pragmatic trial design cannot identify specific reasons for increased risk of self-harm in those offered skills training. Post hoc analyses found increased risk in those who only briefly engaged in either intervention (eFigure 2 in Supplement 2), suggesting that some patients may have been dissatisfied with these limited interventions and become more discouraged.

These findings do not argue against systematic efforts to identify and address suicide risk in health care settings. Instead, they indicate that the low-intensity adaptations of care management and DBT skills training tested in this trial have no benefit over usual care in these health systems, especially when offered to the broad population reporting frequent suicidal ideation.

Limitations

This study has several limitations. First, health system records could not identify self-harm events if patients did not seek care or if clinicians did not recognize or record self-harm intent. Second, use of those records to identify self-harm could introduce bias if interventions influenced either likelihood of seeking care following self-harm or likelihood that self-harm intent would be detected. Regarding the first possibility, secondary analyses examined self-harm events leading to death or hospitalization, for which seeking care would be less discretionary. Results were qualitatively similar to primary analyses. Regarding the second possibility, secondary analyses examined all injuries or poisonings representing plausible mechanisms of self-harm, regardless of diagnosed intent. Results were similar, with some attenuation of the increased risk in the skills training group. Third, some injuries and poisonings coded as self-harm or identified by chart review may not reflect suicidal intent but instead indicate nonsuicidal self-injury. Fourth, these findings might not apply to other patient populations or health systems. Although this study sample included large numbers of patients insured by Medicaid and large numbers from traditionally underserved racial or ethnic groups, the majority were non-Hispanic White and covered by employer-sponsored insurance. All participants had access to primary care and specialty mental health services, and participants were identified because these health systems routinely assess symptoms of depression and suicidal ideation. Fifth, sample size calculations were based on consultation with leaders of these health systems regarding meaningful differences in risk of self-harm, and other health system leaders or policy makers might use higher or lower thresholds.

Conclusions

Among adult outpatients with frequent suicidal ideation, offering care management did not significantly reduce the risk of self-harm, and offering brief online training in selected dialectical behavior therapy skills training increased the risk of self-harm, compared with usual care. These findings do not support implementing the approaches in this study.

Supplement 1.

Trial Protocol

Click here for additional data file.^{(410.1KB, pdf)}

Supplement 2.

eAppendix 1. Details of Randomization Procedure

eAppendix 2. Details of Invitation and Reminder Procedures

eAppendix 3. Ascertainment of Fatal Self-harm and Deaths Due to Other Causes

eAppendix 4. Exclusion of Randomized Patients With No Follow-up Time

eAppendix 5. Randomized Participants Not Offered Intervention Services

eAppendix 6. Review of Clinical Text to Identify Self-harm Events Not Coded by Treating Clinicians

eTable 1. Most Common Injury and Poisoning Diagnosis Codes Selected for Inclusion and Exclusion in Chart Review Validation of Self-harm Codes

eTable 2. Examples of Abstracted Text and Rater Classifications From Chart Review Validation of Self-harm Codes

eTable 3. Distribution of Rater Confidence and Inter-rater Agreement for Chart Review Validation of Self-harm Codes

eTable 4. Distribution of Self-harm Ratings From Chart Review Validation of Self-harm Codes

eTable 5. Use of Specific Components of the Online Skills Training Program

eTable 6. Subgroup Analyses and Tests for Interaction Effects

eTable 7. Comparison of Baseline Characteristics by Level of Intervention Participation in Care Management Intervention

eTable 8. Comparison of Baseline Characteristics by Level of Intervention Participation in Skills Training Intervention

eTable 9. Comparison of Self-harm Rates by Level of Intervention Participation

eTable 10. Distribution of Utilization of Specific Non-study Health Services of 12 Months After Randomization

eFigure 1. Kaplan Meier Curves for Planned Secondary Analyses of Severe Self-harm and Broader Definition of Self-harm

eFigure 2. Comparison of Self-harm Rates by Level of Intervention Participation

eFigure 3. Log-Log Plot for Evaluation of Proportional Hazards Assumption

Click here for additional data file.^{(557.5KB, pdf)}

Supplement 3.

Data Sharing Statement

Click here for additional data file.^{(92.5KB, pdf)}

References

1.Ahmedani BK, Simon GE, Stewart C, et al. Health care contacts in the year before suicide death. J Gen Intern Med. 2014;29(6):870-877. doi: 10.1007/s11606-014-2767-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Ahmedani BK, Stewart C, Simon GE, et al. Racial/ethnic differences in health care visits made before suicide attempt across the United States. Med Care. 2015;53(5):430-435. doi: 10.1097/MLR.0000000000000335 [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Simon GE, Coleman KJ, Rossom RC, et al. Risk of suicide attempt and suicide death following completion of the Patient Health Questionnaire depression module in community practice. J Clin Psychiatry. 2016;77(2):221-227. doi: 10.4088/JCP.15m09776 [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Simon GE, Johnson E, Lawrence JM, et al. Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records. Am J Psychiatry. 2018;175(10):951-960. doi: 10.1176/appi.ajp.2018.17101167 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.The Joint Commission Patient Safety Advisory Group . Detecting and Treating Suicidal Ideation in All Settings. The Joint Commission; 2016. The Joint Commission Sentinel Event Alert 56.
6.National Action Alliance for Suicide Prevention Transforming Health Systems Initiative Work Group . Recommended Standard Care for People With Suicide Risk: Making Health Care Suicide Safe. Education Development Center Inc; 2018. [Google Scholar]
7.Linehan MM, Korslund KE, Harned MS, et al. Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder. JAMA Psychiatry. 2015;72(5):475-482. doi: 10.1001/jamapsychiatry.2014.3039 [DOI] [PubMed] [Google Scholar]
8.DeCou CR, Comtois KA, Landes SJ. Dialectical behavior therapy is effective for the treatment of suicidal behavior. Behav Ther. 2019;50(1):60-72. doi: 10.1016/j.beth.2018.03.009 [DOI] [PubMed] [Google Scholar]
9.Brown GK, Ten Have T, Henriques GR, et al. Cognitive therapy for the prevention of suicide attempts. JAMA. 2005;294(5):563-570. doi: 10.1001/jama.294.5.563 [DOI] [PubMed] [Google Scholar]
10.Woltmann E, Grogan-Kaylor A, Perron B, et al. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings. Am J Psychiatry. 2012;169(8):790-804. doi: 10.1176/appi.ajp.2012.11111616 [DOI] [PubMed] [Google Scholar]
11.Coffey CE, Coffey MJ, Ahmedani BK. An update on perfect depression care. Psychiatr Serv. 2013;64(4):396. doi: 10.1176/appi.PS.640422 [DOI] [PubMed] [Google Scholar]
12.Cwik MF, Tingey L, Maschino A, et al. Decreases in suicide deaths and attempts linked to the White Mountain Apache Suicide Surveillance and Prevention System, 2001-2012. Am J Public Health. 2016;106(12):2183-2189. doi: 10.2105/AJPH.2016.303453 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Carter GL, Clover K, Whyte IM, et al. Postcards from the EDge: 24-month outcomes of a randomised controlled trial for hospital-treated self-poisoning. Br J Psychiatry. 2007;191:548-553. doi: 10.1192/bjp.bp.107.038406 [DOI] [PubMed] [Google Scholar]
14.Comtois KA, Kerbrat AH, DeCou CR, et al. Effect of augmenting standard care for military personnel with brief caring text messages for suicide prevention. JAMA Psychiatry. 2019;76(5):474-483. doi: 10.1001/jamapsychiatry.2018.4530 [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Beautrais AL, Gibb SJ, Faulkner A, et al. Postcard intervention for repeat self-harm. Br J Psychiatry. 2010;197(1):55-60. doi: 10.1192/bjp.bp.109.075754 [DOI] [PubMed] [Google Scholar]
16.Simon GE, Beck A, Rossom R, et al. Population-based outreach versus care as usual to prevent suicide attempt. Trials. 2016;17(1):452. doi: 10.1186/s13063-016-1566-z [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Shortreed SM, Rutter CM, Cook AJ, Simon GE. Improving pragmatic clinical trial design using real-world data. Clin Trials. 2019;16(3):273-282. doi: 10.1177/1740774519833679 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Coleman KJ, Stewart C, Waitzfelder BE, et al. Racial-ethnic differences in psychiatric diagnoses and treatment across 11 health care systems in the Mental Health Research Network. Psychiatr Serv. 2016;67(7):749-757. doi: 10.1176/appi.ps.201500217 [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Kroenke K, Spitzer RL, Williams JB, Löwe B. The Patient Health Questionnaire somatic, anxiety, and depressive symptom scales. Gen Hosp Psychiatry. 2010;32(4):345-359. doi: 10.1016/j.genhosppsych.2010.03.006 [DOI] [PubMed] [Google Scholar]
20.Ralston JD, Rutter CM, Carrell D, et al. Patient use of secure electronic messaging within a shared medical record. J Gen Intern Med. 2009;24(3):349-355. doi: 10.1007/s11606-008-0899-z [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Zelen M. A new design for randomized clinical trials. N Engl J Med. 1979;300(22):1242-1245. doi: 10.1056/NEJM197905313002203 [DOI] [PubMed] [Google Scholar]
22.Zelen M. Randomized consent designs for clinical trials: an update. Stat Med. 1990;9(6):645-656. doi: 10.1002/sim.4780090611 [DOI] [PubMed] [Google Scholar]
23.Simon GE, Ralston JD, Savarino J, et al. Randomized trial of depression follow-up care by online messaging. J Gen Intern Med. 2011;26(7):698-704. doi: 10.1007/s11606-011-1679-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Motto JA, Bostrom AG. A randomized controlled trial of postcrisis suicide prevention. Psychiatr Serv. 2001;52(6):828-833. doi: 10.1176/appi.ps.52.6.828 [DOI] [PubMed] [Google Scholar]
25.Fleischmann A, Bertolote JM, Wasserman D, et al. Effectiveness of brief intervention and contact for suicide attempters. Bull World Health Organ. 2008;86(9):703-709. doi: 10.2471/BLT.07.046995 [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Katon W, Unützer J, Wells K, Jones L. Collaborative depression care: history, evolution and ways to enhance dissemination and sustainability. Gen Hosp Psychiatry. 2010;32(5):456-464. doi: 10.1016/j.genhosppsych.2010.04.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Posner K, Brown GK, Stanley B, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266-1277. doi: 10.1176/appi.ajp.2011.10111704 [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Neacsiu AD, Rizvi SL, Linehan MM. Dialectical behavior therapy skills use as a mediator and outcome of treatment for borderline personality disorder. Behav Res Ther. 2010;48(9):832-839. doi: 10.1016/j.brat.2010.05.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Mantel N. Evaluation of survival data and two new rank order statistics arising in its consideration. Cancer Chemother Rep. 1966;50(3):163-170. [PubMed] [Google Scholar]
30.Peto R, Peto J. Asymptotically efficient rank invariant test procedures. J R Stat Soc Ser A Stat Soc. 1972;135:185-207. [Google Scholar]
31.Whiteside U, Richards J, Simon GE. Brief interventions via electronic health record messaging for population-based suicide prevention. JMIR Form Res. 2021;5(4):e21127. doi: 10.2196/21127 [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Linehan MM, Comtois KA, Murray AM, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. 2006;63(7):757-766. doi: 10.1001/archpsyc.63.7.757 [DOI] [PubMed] [Google Scholar]
33.Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. 2012;10:CD006525. doi: 10.1002/14651858.CD006525.pub2 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials