Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Apr 1.
Published in final edited form as: Psychiatry. 2021 Dec 21;85(1):13–29. doi: 10.1080/00332747.2021.1991200

A Pragmatic Clinical Trial Approach to Assessing and Monitoring Suicidal Ideation: Results from a National US Trauma Care System Study

Allison Engstrom a, Kathleen Moloney a, Jefferson Nguyen a, Lea Parker a,b, Michelle Roberts a,c, Rddhi Moodliar a,d, Joan Russo a, Jin Wang a,e, Hannah Scheuer a,f, Douglas Zatzick a,e
PMCID: PMC8916972  NIHMSID: NIHMS1751704  PMID: 34932440

Abstract

Objective:

Few investigations have comprehensively described methods for assessing and monitoring suicidal ideation in pragmatic clinical trials of mental health services interventions. This investigation’s goal was to assess a collaborative care intervention’s effectiveness in reducing suicidal ideation and describe suicide monitoring implementation in a nationwide protocol.

Method:

The investigation was a secondary analysis of a stepped wedge cluster randomized trial at 25-Level I trauma centers. Injury survivors (N=635) were randomized to control (n=370) and intervention (n=265) conditions and assessed at baseline hospitalization and follow-up at 3-, 6- and 12-months post-injury. The Patient Health Questionnaire (PHQ-9) item-9 was used to evaluate patients for suicidal ideation. Mixed model regression was used to assess intervention versus control group changes in PHQ-9 item-9 scores over time and associations between baseline characteristics and development of suicidal ideation longitudinally. As part of the study implementation process assessment, suicide outreach call logs were also reviewed.

Results:

Over 50% of patients endorsed suicidal ideation at ≥1 assessment. Intervention patients relative to control patients demonstrated reductions in endorsements of suicidal ideation that did not achieve statistical significance (F[3,1461]=0.74, P=0.53). The study team completed outreach phone calls, texts or voice messages to 268 patients with PHQ-9 item-9 scores ≥1 (n=161 control, n=107 intervention).

Conclusions:

Suicide assessment and monitoring can be feasibly implemented in large scale pragmatic clinical trials. Intervention patients demonstrated less suicidal ideation over time; however, these comparisons did not achieve statistical significance. Intensive pragmatic trial monitoring may mask treatment effects by providing control patients a supportive intervention.

Trial Registration:

ClinicalTrials.gov NCT02655354

Keywords: physical injury trauma, suicidal ideation, pragmatic clinical trials, trauma centers

Introduction

Each year in the United States, over 30 million individuals visit acute care medical settings after incurring traumatic injuries, and between 1.5 and 2.5 million Americans are so severely injured annually that they require inpatient admission (Centers for Disease Control and Prevention, 2021). Injury hospitalization confers additional risk for suicidal behaviors above and beyond well documented risk factors such as prior psychiatric diagnoses and substance use disorders (Barak-Corren et al., 2017; Doshi et al., 2020). In a prior single site investigation, 24% of individuals hospitalized with high levels of PTSD endorsed some suicidal ideation at the time of their baseline injury admission (O’Connor et al., 2014).

Multisite pragmatic trials are becoming an increasingly common randomized clinical trial design in mental health services research, and have now been fielded in the acute care medical trauma center context (Loudon et al., 2015; NIH Collaboratory; NIMH, 2020; Palinkas & Zatzick, 2019; Thorpe et al., 2009). Pragmatic clinical trials are typically characterized by large scale rollouts of evidence-based practices with real world design features such as the inclusion of generalizable samples of patients, providers and practice settings. Pragmatic trials may be designed with the explicit purpose of targeting health care system policy (Califf & Sugarman, 2015; Zatzick et al., 2016). Research efficiency (Lagomasino et al., 2010) is also highly valued in pragmatic clinical trials, with suggestions that these trials be judged on efficiency metrics such as costs per subject randomized (Palinkas & Zatzick, 2019).

A series of pragmatic clinical trial demonstration projects have now been completed with the stewardship of the NIH Health Care System Research Collaboratory (Coronado et al., 2018; Huang et al., 2019; Jarvik et al., 2020; Loomer et al., 2021); other large scale pragmatic trials specifically targeting suicide are ongoing (Simon, Beck, et al., 2016). The completed demonstration projects have generally reported small treatment effects relative to the effect sizes reported in explanatory/efficacy spectrum randomized trials (Loudon et al., 2015). Prior papers have reviewed data and safety monitoring practices of pragmatic trials, highlighting in particular how data and safety monitoring procedures may need to be approached differently than in explanatory trials (Califf & Sugarman, 2015; Ellenberg et al., 2015; Roberts et al., 2020; Simon et al., 2019). One previous report of a large-scale pragmatic trial describes a mid-trial shift in data monitoring procedures, partly in response to the intensity of outreach necessitated by an intensive suicide monitoring and outreach approach and the associated clinical burden (Goodsmith et al., 2021). Literature review, however, revealed few pragmatic trial reports of mental health services interventions that comprehensively described methods for assessing and monitoring suicidal ideation (Simon, Coleman, et al., 2016; Zatzick et al., 2016).

The overarching objective of the current study was to perform a comprehensive analysis of data related to suicidal ideation collected during a large-scale US trauma care system pragmatic trial rollout (Zatzick et al., 2016). The investigation assessed the effectiveness of a collaborative care intervention on suicidal ideation, while also exploring baseline factors associated with the development of suicidal ideation longitudinally, and the processes of implementing suicide monitoring in the nationwide protocol. The collaborative care intervention included proactive injury care management, psychopharmacology and psychotherapeutic elements targeting posttraumatic stress disorder and related comorbidity such as depression. The investigation first sought to explore whether intervention and control patients differed significantly in their patterns of expressed suicidal ideation over time. Next, the investigation used readily identifiable patient demographic and clinical characteristics present at the time of the surgical ward hospitalization to assess longitudinal risk factors for suicidal ideation. Finally, the study team harnessed data from the investigation’s call logs to better understand the process of monitoring suicidal ideation in the nationwide trauma center protocol.

Materials and Methods

Design Overview

The investigation was a secondary analysis of data collected in a 25 trauma center site pragmatic randomized trial designed to assess the impact of a collaborative care intervention on the symptoms of posttraumatic stress disorder (PTSD) and related comorbidity, including suicidal ideation (Zatzick et al., 2021). The Trauma Survivors Outcomes and Support (TSOS) stepped wedge pragmatic clinical trial was orchestrated by the TSOS study team’s data coordinating center, located at University of Washington’s Harborview Medical Center, in close collaboration with the National Institutes of Health (NIH) Health Care Systems Research Collaboratory (NIH Collaboratory, 2021; Zatzick et al., 2016). Sites recruited into the study constituted a representative subsample of all US Level I trauma centers with regard to volume of trauma center admissions, geographic location and training hospital status (Zatzick et al., 2016). The Western Institutional Review Board (WIRB) approved the protocol prior to study initiation and the National Institute of Mental Health Data Safety and Monitoring Board oversaw the clinical trial rollout (Roberts et al., 2020). Written informed consent was obtained for participation after the study procedures had been explained to patients. Patients were assessed at baseline as surgical inpatients and again at 3, 6 and 12 months after injury. The study protocol, including the randomization procedure and stepped collaborative care intervention, have been previously detailed in-depth and are briefly described below (Zatzick et al., 2021; Zatzick et al., 2016). Recruitment for the trial began in January of 2016, and patient follow-up ended in November of 2018.

Patient Inclusion/Exclusion Criteria

Injured patients ages ≥ 18 were included in the trial. Prisoners and non-English-speaking patients were excluded. Patients whose index injury was self-inflicted or who were psychotic and required immediate psychiatric treatment were also excluded from the trial. To assure adequate follow-up rates, patients were required to provide two pieces of contact information. A total of 635 injured participants were recruited and randomized into intervention (n = 265) or control (n = 370) group conditions. The population included 308 women (48.5%) and 327 men (51.5%). Mean (SD) age was 39.0 (14.2) years. Study participants were 49.6% White, 34.3% Black, 2.4% American Indian, 0.6% Asian, 0.6% Pacific Islander, 3.1% mixed race, and 9.3% other race; 16.1% of study participants identified as Hispanic. Twenty-eight percent of participants were married and/or living with a partner. The investigation attained follow-up rates of 80.2% at 3 months, 77.3% at 6 months, and 75.1% at 12 months.

Randomization

Prior to initiation of patient recruitment, each of the 25 sites was randomized to one of 4 waves in the stepped wedge design. A study biostatistician randomized sites to waves using a computer-generated algorithm. Recruiters at the 25 sites were aware of each patient’s intervention or control group status at the time of the baseline surgical ward interview, but were instructed not to inform patients of their status until after completion of the baseline interview. All follow-up interviews were conducted by a survey research team at the University of Washington data coordinating center. Follow-up interviewers were at all times were blinded to patient intervention versus control group status. Three patient characteristics of the control (n = 370) vs intervention (n = 265) groups were observed to be significantly different at baseline: age (mean [SD], 39.9 [14.8] vs 37.6 [13.4] years; P = .047), sex (female, 161 [43.5%] vs 147 [55.5%]; P = .003), and electronic health record PTSD diagnosis (55 [14.9%] vs 60 [22.6%]; P = .01).

Usual Care Control Condition

Patients in the control condition underwent informed consent, baseline surgical ward evaluation, and follow-up interviews. Prior investigation suggests that, after hospital discharge, usual posttraumatic care includes routine surgical, primary care, and emergency department visits, as well as some mental health specialty appointments (Zatzick, Jurkovich, et al., 2013; Zatzick et al., 2015; Zatzick et al., 2004).

Intervention Procedures

After completion of the usual care control phase of recruitment, the principal investigator visited each trauma center to train frontline MSW, RN, MD and other providers via a one-day, onsite intervention workshop (Darnell et al., 2019; Zatzick et al., 2016). The workshop provided an overview of the core care management, psychopharmacology, and motivational interviewing and cognitive behavioral therapy (CBT) elements of the stepped collaborative care intervention. After the one-day workshop, the study team initiated regular site supervisory calls in which the site interventionists presented cases to supervising psychiatric, research coordinator and other study team members. Intervention training included discussions of the management of acute suicidal ideation and intent in traumatically injured patients.

Site intervention teams spent approximately 2 hours (range: 0.05–14.83 hours) with each intervention patient over the course of the 12 months after injury (Zatzick et al., 2021). Approximately 90% of intervention activity occurred during the first six months after the injury admission.

Measures

Suicidal Ideation and Intent

The study used the Patient Health Questionnaire (PHQ-9) item 9 to assess all patients at each time point for suicidal ideation (Kroenke et al., 2001). The PHQ-9 is a 9-item questionnaire designed to screen for depression and suicidality that has established reliability and validity in general medical patient populations (Gilbody et al., 2007; Kroenke et al., 2001; Levis et al., 2019; Moriarty et al., 2015; Simon, Coleman, et al., 2016). Per the PHQ-9 item 9, at baseline in the surgical ward patients were asked, “Since the event in which you were injured, how often have you been bothered by thoughts that you would be better off dead, or of hurting yourself?” Response options include 0, “not at all”; 1, “several days”; 2, “more than half the days”; or 3, “nearly every day.” At follow-up patients were asked, “In the past month, how often have you been bothered by thoughts that you would be better off dead, or of hurting yourself?” Again, response options include 0, “not at all”; 1, “several days”; 2, “more than half the days”; or 3, “nearly every day.” In the current investigation, scores of ≥1 were indicative of a patient endorsement of suicidal ideation. Patients who endorsed suicidal ideation were asked an additional single item question to assess active suicidal intent, “Do you currently have thoughts of killing yourself or ending your life?” Responses options were “yes” or “no” to this item, which the study team characterized as suicide item 9a.

Depressive Symptoms

The remaining 8 items of the Patient Health Questionnaire were used to measure depressive symptoms beyond suicidal ideation (Kroenke et al., 2001). The PHQ-9 has established reliability and validity in acute and primary care medical patients (Kroenke et al., 2001; Zatzick, Jurkovich, et al., 2013). Cronbach’s alpha for the PHQ-9 in a prior investigation with injury survivors by the study team was 0.97 (Zatzick, Jurkovich, et al., 2013; Zatzick et al., 2011).

PTSD Symptoms

The PTSD Checklist – Civilian Version (PCL-C) for the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) was used to assess the symptoms of posttraumatic stress disorder (Weathers & Ford, 1996; Weathers et al., 2013). The 17-item self-report questionnaire uses a 5-point Likert scale to assess PTSD symptoms. Cronbach’s alpha for the 17-item scale in a prior investigation with injured trauma survivors by the study team was 0.92 (Zatzick, Jurkovich, et al., 2013; Zatzick et al., 2011).

Alcohol Use

The Alcohol Use Disorder Identification Test 3-item version (AUDIT-C) was used to assess alcohol use problems before and after the injury hospitalization (Babor & Grant, 1989; Bradley et al., 2007; Zatzick, Jurkovich, et al., 2013). The AUDIT’s reliability and validity are well established, and the scale has been widely used in general medical settings (Babor et al., 1989; Babor & Grant, 1989). Cronbach’s alpha for the AUDIT in a prior investigation by the study team was 0.80 (Zatzick, Donovan, et al., 2013).

Physical Function

The investigation used the Medical Outcomes Study Short Form Physical Components Summary Score (MOS SF PCS) SF-12 at baseline in the surgical ward to assess physical function in the month prior to the injury admission, and SF-36 at 3-, 6-, and 12-month follow-up to assess post-injury physical function (Ware et al., 1996; Ware et al., 1993; Zatzick, Jurkovich, et al., 2013). The SF-12/36 has established reliability and validity (Ware et al., 1993), and the measure has been used extensively with traumatically injured populations (MacKenzie et al., 2002; Michaels et al., 1999; Zatzick et al., 2002). Cronbach’s alpha for the MOS SF-36 PCS in a prior investigation by the study team was 0.90 (Zatzick, Jurkovich, et al., 2013; Zatzick, Rivara, Jurkovich, Russo, et al., 2010).

Other Assessments

Lifetime traumatic events were assessed with a modified version of the National Comorbidity Survey trauma history screen (Kessler et al., 1995; Moodliar et al., 2020). Single drug screening items were used to specifically assess the use of amphetamines, cocaine, marijuana, and opiates (Smith et al., 2010; Zatzick et al., 2021). Medical record data from the 25 sites’ trauma registries were used to derive injury severity scores and injury mechanisms (Baker et al., 1974; The Johns Hopkins Health Services Research and Development Center, 1989; Zatzick, Rivara, Jurkovich, Hoge, et al., 2010). Laboratory toxicology results, insurance status, length of hospital and intensive care unit stays, and other clinical characteristics were obtained from trauma registries.

Implementation Process Assessment

Over the course of the pragmatic trial, an implementation process assessment was conducted to better understand the processes of protocol implementation (Palinkas & Zatzick, 2019; Zatzick et al., 2016). The implementation process assessment utilized data from multiple sources, including study team intervention documentation and clinical trial logs. The process of monitoring patients for suicidal ideation was included as part of the study implementation process assessment.

Suicide Monitoring

The study protocol included in-depth monitoring procedures for patients who endorsed PHQ-9 item-9 responses of ≥1 on the blinded follow-up interviews. Patients who endorsed a ‘1’ or greater on the PHQ-9 item 9 but responded with “no” to follow-up question item 9a were logged electronically and called by a study coordinating center clinician within one week. If the patient responded with “yes” to follow-up item 9a, the study team member conducting the interview would proceed with a “warm hand-off” of the patient to a member of the study coordinating center clinical team (e.g., MSW, MD, other clinician) at the end of the interview. If an immediate hand-off was not possible, a clinical study team member would call the patient within 24 hours. The clinician calling the patient would perform a brief suicide risk assessment and clinical intervention that could include care linkages and referrals as deemed appropriate (Stanley & Brown, 2012). For patients previously randomized to the intervention condition, the coordinating center clinicians could also notify the patient’s trauma center-based care management team of the patient’s distress. For patients who were believed to be at imminent risk of suicide, more acute measures were undertaken, such as recommendation for the patient to present themselves to an emergency department. Depending on the patient’s response, the study team might repeatedly call, text or leave voice messages for patients. Whether or not a patient answered the phone, a 24/7 suicide crisis line number was provided along with the compensation for interview completion, which was mailed to a confirmed address.

Data Analyses

The investigation first compared the baseline demographic, injury and clinical characteristics of patients who endorsed any suicidal ideation (i.e., PHQ-9 item 9 ≥1) versus patients who did not endorse suicidal ideation as surgical inpatients (i.e., PHQ-9 item 9 = 0). Differences between the two groups were examined using the t-test and χ2 statistic for continuous and dichotomous variables, respectively.

Next, intervention and control group differences in the longitudinal course of dichotomized PHQ-9 item 9 scores at baseline, 3-months, 6-months and 12-months post-injury were examined. The study team used longitudinal mixed effects regression models to account for both repeated measures for individuals over time and site-level cluster randomization (Diggle et al., 2004; Gibbons et al., 2010; Little et al., 2012). Mixed model regression sensitivity analyses also explored intervention and control group differences in continuous PHQ-9 item 9 scores longitudinally at the 3-, 6- and 12-month assessment points.

Next, the study examined the associations between baseline demographic, injury, and clinical characteristics and baseline and longitudinal PHQ-9 item 9 scores while adjusting for patient intervention and control group status. Baseline demographic, injury, and clinical characteristics were entered into the regression models using a backwards elimination procedure (Heinze et al., 2018). Only variables with significant independent associations at the p < 0.05 level were retained in the final model.

Finally, as part of the investigation’s implementation process assessment (Palinkas & Zatzick, 2019; Zatzick et al., 2021), study team suicide monitoring call logs and regulatory documentation records were reviewed. The study team abstracted the total calls performed, as well as which study team member performed the calls, to compare the number and characteristics of suicide monitoring calls for intervention and control group patients.

Results

Approximately 25% of patients endorsed scores ≥1 on the PHQ-9 item 9 at baseline as surgical inpatients (Table 1). Intervention and control group patients did not significantly differ with regard to baseline PHQ-9 item 9 endorsements (χ2[1] = 0.7, P = 0.80).

TABLE 1.

Baseline Patient Characteristics

All (N = 634)
 Not suicidal at baseline (n = 477)
 Suicidal at baseline (n = 157)
Characteristics N % n % n %
Electronic heath record (EHR) variables
 Sex: Female 308 48.6 220 46.1 88 56.1
 ICU admission 376 59.3 283 59.3 93 59.2
 Prior inpatient hospitalization 248 39.1 177 37.1 71 45.2
 Tobacco use 355 56.0 264 55.4 91 58.0
 Psychiatric diagnosis 245 38.6 164 34.4 81 51.6
 PTSD diagnosis 114 18.0 81 17.0 33 21.0
 Positive BAC 157 28.5 115 27.9 42 30.2
Demographic
 Age, mean (SD), y 39.0 14.2 38.9 14.4 39.1 13.9
 Race
  White 315 49.7 222 46.5 93 59.2
  Black 218 34.4 175 36.7 43 27.4
  American Indian 15 2.4 11 2.3 4 2.6
  Asian 4 0.6 3 0.6 1 0.6
  Pacific Islander 4 0.6 3 0.6 1 0.6
  Mixed 19 3.0 12 2.5 7 4.5
  Other 59 9.3 51 8.6 8 5.1
 Hispanic
  Yes 102 16.2 85 17.9 17 11.0
  No 529 83.8 391 82.1 138 89.0
 Education
  Less than high school 123 19.5 88 18.5 35 22.4
  High school/GED 314 49.8 251 52.8 63 40.4
  Associate degree 118 18.7 80 16.8 38 24.4
  Bachelor’s or graduate degree 76 12.0 56 11.8 20 12.8
 Marital status
  Married/living with partner 178 28.1 132 27.7 46 29.3
 Having any children 437 69.0 330 69.3 107 68.2
 Employed 376 59.7 292 61.5 84 54.2
 Insurance
  Private 224 35.5 163 34.4 61 38.9
  Public 328 52.0 252 53.2 76 48.4
  None 79 12.5 59 12.5 20 12.7
Acute care injury & medical
 Intentional Injury
  Assault 53 8.3 38 8.0 15 9.6
  Stabbing 53 8.4 46 9.6 7 4.5
  Firearm 128 20.2 103 21.6 25 15.9
 Unintentional Injury 400 63.1 290 60.8 110 70.1
 Injury severity category
  0–8 136 24.0 101 23.7 35 24.8
  9–15 197 34.7 145 34.0 52 36.9
  ≥ 16 234 41.3 180 42.3 54 38.3
 Traumatic brain injury a
  None 387 68.3 283 66.4 104 73.8
  Mild 101 17.8 79 18.5 22 15.6
  Moderate 56 9.9 49 11.5 7 5.0
  Severe 23 4.0 15 3.5 8 5.7
 Number of co-morbid medical conditions
  0 192 31.4 151 32.6 41 27.5
  1 117 19.1 92 19.9 25 16.8
  2 91 14.9 73 15.8 18 12.1
  ≥ 3 212 34.6 147 31.8 65 43.6
 Length of stay for index visit, mean (SD), days 12.9 12.7 12.2 11.9 14.9 14.7
Clinical Assessments
 Number of prior serious traumas before injury admission, mean (SD) b 4.5 3.1 4.2 2.9 5.2 3.5
 Pre-injury PTSD symptoms lasting ≥30 days d 218 34.4 152 31.9 66 42.0
 Baseline PCL-C total score, mean (SD) c 52.1 11.9 50.5 11.4 56.6 12.3
 Baseline PHQ-8 depression total score, mean (SD) c 13.7 5.5 12.6 5.2 16.9 4.8
 ≥ 1 prior mental health visit 227 36.0 156 32.8 71 45.8
 Pre-injury AUDIT-C score, mean (SD) 3.7 3.4 3.7 3.4 3.6 3.3
 Pre-injury self-report drug use d
  Stimulants e 134 21.2 96 20.2 38 24.4
  Opioids 62 9.8 39 8.2 23 14.7
  Marijuana 301 47.6 212 44.6 89 56.7
 Pre-injury SF-12 PCS score, mean (SD) 49.4 9.7 50.0 9.1 47.7 11.2
 Pre-injury SF-12 MCS score, mean (SD) 44.8 13.4 46.5 12.6 39.7 14.4
 Intervention 265 41.8 198 41.5 67 42.7
Open in a new tab

Abbreviations: ICU, intensive care unit; PTSD, posttraumatic stress disorder; BAC, blood alcohol concentration; SD, standard deviation; PCL-C, PTSD Checklist Civilian Version (Weathers & Ford, 1996); PHQ, Patient Health Questionnaire (Kroenke et al., 2001); AUDIT-C, The Alcohol Use Disorders Identification Test – Consumption Items (Bradley et al., 2007); SF-12 PCS, Short-Form Health Survey – Physical Component Summary (Ware et al., 1996).

a

Traumatic brain injury severity was coded on the basis of a previously validated algorithm for hospitalized inpatients that assigned MAXAIS head injury scores of 1 to 2 to mild, 3 to moderate, and 4 or higher to severe.

b

Derived from the 3-month interview Trauma History Screen.

c

For PCL-C and PHQ items inpatients were asked to report symptoms since the injury event.

d

Single item self-report dichotomized as none versus at least monthly use.

e

Stimulants include cocaine and amphetamines.

The investigation attained >75% patient follow-up at all post-injury time points. Over the course of the year after injury, increasing percentages of patients endorsed PHQ-9 item 9 ≥1; 50.57% of intervention patients versus 53.24% of control patients endorsed PHQ-9 item 9 responses ≥1 at one or more timepoints including baseline (χ2[1] = 0.44, P = 0.51). Over the course of the year after injury, between 65.5–75.2% of patients endorsed a ‘0’ on the PHQ-9 item 9, between 14.5–19.0% of patients endorsed a ‘1’, between 5.7–8.5% of patients endorsed a ‘2’, and between 3.6–6.3% of patients endorsed a ‘3’.

The dichotomized percentage of control group patients endorsing positive PHQ-9 item responses increased relative to patients randomized to the intervention condition (Figure 1), however, in mixed model regression analyses these comparisons did not achieve statistical significance (F[3,1461] = 0.74, P = 0.53). Sensitivity analyses that compared continuous PHQ-9 item 9 scores for intervention and control patients also did not achieve statistical significance (Wald χ2[3] = 1.14, P = 0.77).

Figure 1.

Figure 1.

Open in a new tab

PHQ-9 Item 9 ≥ 1 for Intervention and Control Patients Over Time

In mixed model regression analyses that adjusted for patient intervention versus control group status, higher baseline PHQ-8, higher educational attainment, and pre-injury marijuana use were associated with the development of suicidal ideation in the trauma surgical ward (Table 2). Longitudinally, baseline psychiatric diagnoses, prior PTSD symptoms, pre-injury marijuana use, poor pre-injury mental health function, and greater injury severity were associated with the development of suicidal ideation over the course of the 12 months after injury (Table 2).

TABLE 2.

Associations between demographic, injury and clinical characteristics during hospitalization and longitudinally over the course of the 12 months after injury a

Baseline
 Longitudinal
Variable RR 95% CI RR 95% CI
Timepoint
 Baseline -- -- Reference --
 3-month -- -- 1.31 (1.11, 1.55)
 6-month -- -- 1.36 (1.18, 1.58)
 12-month -- -- 1.21 (1.01, 1.44)
Intervention -- -- 1.00 (0.83, 1.21)
Age 1.00 (0.99, 1.02) 1.00 (0.99, 1.01)
Female 1.34 (0.95, 1.90) 1.05 (0.87, 1.27)
Injury severity category
 0–8 Reference -- Reference --
 9–16 0.99 (0.64, 1.52) 1.12 (0.90, 1.39)
 ≥16 0.94 (0.61, 1.44) 1.30 (1.07, 1.57)
Psychiatric diagnosis -- -- 1.24 (1.08, 1.43)
Pre-injury PTSD symptoms lasting
≥30 days -- -- 1.34 (1.11, 1.63)
Marijuana 1.87 (1.30, 2.71) 1.47 (1.28, 1.68)
Education
 Less than high school Reference -- -- --
 Greater than high school 1.48 (1.03, 2.12) -- --
Pre-injury SF-12 MCS score -- -- 0.98 (0.98, 0.99)
Baseline PHQ-8 depression score 1.12 (1.09, 1.16) -- --
Open in a new tab
a

Relative Risks (RR) and 95% Confidence Intervals (CI) that do not run through 1.0 are statistically significant and are bolded

Overall, the study team made 196 outreach calls, 254 texts, and left 161 voice messages (Table 3). The study team MD provider was significantly more likely to make outreach calls to control patients (χ2[1] = 7.06, P <0.01), while study MSW providers were significantly more likely to make outreach calls to intervention patients (χ2[1] = 16.15, P < 0.001).

TABLE 3.

Suicide monitoring over the course of the year after injury (combined 3-, 6- and 12-month data points)

Total (N = 277)
 Control (n = 166)
 Intervention (n = 111)
n % n % n %
Total number of patients with ≥ 1 completed calls 196 71 112 68 84 76
Total number of patients with ≥ 1 VM left 161 58 90 54 71 64
Total number of patients with ≥ 1 text sent 254 92 154 93 100 91
Total number of patients contacted by clinical research staff (e.g., MSW) 78 28 32 19 46 41
Total number of patients contacted by the MD 159 57 106 64 53 48
Average number of completed calls to each patient with PHQ-9 ≥1, mean (SD) 1.03 0.88 1.05 0.96 0.99 0.74
Average number of VM left, mean (SD) 1.27 1.60 1.09 1.42 1.53 1.80
Average number of texts sent per patient, mean (SD) 1.89 1.40 1.76 1.20 2.08 1.64
Open in a new tab

Discussion

The current investigation reports on a comprehensive approach to the assessment and monitoring of suicidal ideation in a nationwide pragmatic clinical trial of a collaborative care intervention for injured trauma survivors. The investigation explored the effectiveness of the pragmatic trial intervention on suicidal ideation outcomes, baseline suicide risk prediction, and pragmatic trial suicide monitoring procedures.

With regard to effectiveness, the investigation found small, non-significant reductions in PHQ-9 item 9 endorsements of suicidal ideation in patients randomized to the intervention condition relative to patients randomized to the control condition. In contrast, a prior study report documents clinically and statistically significant posttraumatic stress disorder intervention treatment effects at the 6-month but not 12-month post-injury time point; this prior report also documents no significant treatment effects for secondary outcomes including depressive symptoms, physical function and alcohol use problems (Zatzick et al., 2021).

A key point to consider with regard to the lack of observed treatment effects for suicidal ideation is that the study team made hundreds of outreach calls, and texted and left voice messages for control patients. Additionally, a review of the study team suicide monitoring logs revealed that the more highly trained MD provider was significantly more likely to interface with control group patients than intervention patients. These intensive outreach efforts could be seen to constitute an intervention that possibly diminished pragmatic trial treatment effects.

One prior large-scale pragmatic trial report describes intensive outreach efforts to patients experiencing suicidal ideation (Goodsmith et al., 2021). The impetus for these intensive efforts appears to have been in large part derived from community stakeholders’ concerns regarding patient beneficence. This prior trial described a reduction in intensive suicide monitoring procedures due to the excessive burden placed on the clinical-research team rolling out the trial (Goodsmith et al., 2021).

In the current investigation, the intensive outreach efforts derived from an explanatory trial regulatory tradition that emphasized more intensive suicide monitoring protocols (Califf & Sugarman, 2015; Ellenberg et al., 2015; Roberts et al., 2020; Simon et al., 2019). Other investigations have documented the relatively low risk of suicide attempts and completed suicides associated with scores of ≥1 on the PHQ-9 (Simon et al., 2013). Ongoing trials of mental health interventions for patients experiencing suicidal ideation and intent may serve to further inform the optimization of pragmatic trial safety monitoring procedures (Simon, Beck, et al., 2016).

Beyond treatment effects, the investigation identified multiple readily identifiable risk factors for suicidal ideation factors among surgical inpatients. The observations that pre-injury psychiatric diagnoses and mental health function, baseline marijuana use, and greater injury severity are associated with longitudinal development of suicidal ideation corroborate and extend prior investigations (Barak-Corren et al., 2017; Doshi et al., 2020; O’Connor et al., 2014). Also, future investigations could productively explore the relationship between inpatient surgical hospitalizations specifically related to suicide attempts and the development of PTSD symptoms (O’Connor et al., 2021).

The investigation has limitations. The use of a single item self-report assessment of suicidal ideation has been associated with misclassification, and may have limited the ability to detect significant group differences (Millner et al., 2015). Subsequent pragmatic trial investigations should include multiitem suicide assessments. Additionally, the sample is derived from patients enrolled in a randomized controlled trial, rather than a prospective cohort study, an observation to consider when interpreting the results of the suicide risk prediction regressions. Finally, the baseline assessment in this pragmatic trial did not incorporate all known suicide predictors.

In comprehensively describing the assessment and monitoring of suicidal ideation in a nationwide pragmatic trial investigation, the current report broadly incorporates randomized clinical trial, baseline risk prediction, and longitudinal observational data. The investigation demonstrates that suicide assessment and monitoring can be feasibly implemented in large-scale pragmatic trials targeting the mental health needs of physically injured trauma survivors. Intervention patients when compared to control patients demonstrated less suicidal ideation over time; however, these comparisons did not achieve statistical significance. Because pragmatic trials often utilize low intensity interventions across large populations, understanding the nature and intensity of control group monitoring activities may be an essential aspect of ascertaining the potential masking of treatment effects through the inadvertent delivery of low-intensity interventions to patients in the control condition. The inability to attain statistically significant treatment effects in the current trial may be partly attributable to the study team’s extensive suicide outreach efforts to control patients, which may have constituted a supportive intervention. Future investigations could productively incorporate these observations by continuing to incorporate implementation process assessments into pragmatic trial design and rollout.

Acknowledgements

The authors would like to acknowledge Stephen O’Connor, PhD, and Jane Pearson, PhD, for their input on the data analytic approach and review of the manuscript.

Funding

This research was supported within the National Institutes of Health (NIH) Health Care Systems Research Collaboratory by cooperative agreement 1UH2MH106338-01/4UH3MH106338-02 from the NIH Common Fund and by UH3 MH 106338-05S1 from NIMH. Support was also provided by the NIH Common Fund through cooperative agreement U24AT009676 from the Office of Strategic Coordination within the Office of the NIH Director. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Biography

Allison Engstrom is a Social Worker at the Department of Psychiatry and Behavioral Sciences at the University of Washington where she coordinates the regulatory procedures, data analysis and care management for early intervention studies for PTSD and related comorbidities. She received her bachelor’s degree in Psychology from the University of Houston and her Master of Social Work from the University of Washington.

Kathleen Moloney is a Research Coordinator at the Department of Psychiatry and Behavioral Sciences at the University of Washington where she coordinates data analysis and care management for early intervention studies for PTSD in trauma survivors. She received her bachelor’s degree in Neuroscience from the University of Southern California and is currently pursuing her Master of Public Health at the University of Washington.

Jefferson Nguyen is a Research Assistant in the Department of Psychiatry and Behavioral Sciences at the University of Washington where he assists with outcome assessments for an early intervention study for PTSD. He received his bachelor’s degree in Public Health from the University of Washington.

Lea Parker is a former Research Coordinator at the Department of Psychiatry and Behavioral Sciences at the University of Washington where she coordinated the regulatory procedures, data analysis and care management for early intervention studies for PTSD and related comorbidities. She received her bachelor’s degree in Psychology from Harvard University and is currently a PhD in Psychology candidate at Drexel University.

Michelle Roberts is a Research Coordinator at the Department of Rehabilitation Medicine at the University of Washington and a former Research Supervisor at the Department of Psychiatry and Behavioral Sciences, where she coordinated the regulatory procedures, data analysis and care management for early intervention studies for PTSD and related comorbidities. She received her bachelor’s degree in Anthropology and Political Science from Auburn University and her Master of Science in Global Health from Duke University.

Rddhi Moodliar is a former Research Coordinator at the Department of Psychiatry and Behavioral Sciences at the University of Washington where she coordinated the regulatory procedures, patient recruitment, and care management for early intervention studies for PTSD and related comorbidities. She received her bachelor’s degree in Psychology from the University of Washington and is currently a PhD in Psychology candidate at the University of California Los Angeles.

Joan Russo is an Associate Professor Emeritus in the Department of Psychiatry and Behavioral Sciences at the University of Washington and also serves as a statistician and psychometrician for the department. She specializes in health services and outcomes research and is interested in outcomes assessments for traumatically injured patients.

Jin Wang is a Research Consultant in the Department of Psychiatry and Behavioral Sciences and the Harborview Injury Prevention and Research Center at the University of Washington, where she serves as a statistician. She specializes in health services and outcomes research for traumatically injured patients.

Hannah Scheuer is a former Social Worker at the Department of Psychiatry and Behavioral Sciences at the University of Washington where she coordinated care management and regulatory procedures for an early intervention study for PTSD in trauma survivors. She received her bachelor’s degree in Psychology from Reed College and her Master of Social Work from Portland State University, and is currently a PhD in Social Welfare candidate at the University of Washington.

Douglas Zatzick is a Professor in the Department of Psychiatry and Behavioral Sciences at the University of Washington as well as a member of the Core Research Faculty at the Harborview Injury Prevention and Research Center. Dr. Zatzick’s research interests focus on pragmatically focused clinical trials targeting PTSD and related comorbidities in traumatically injured populations.

Footnotes

Declaration of Interests

The authors declare no conflicts of interest.

Data Availability

De-identified participant data will be shared with investigators whose proposed data use has been approved through requests to the principal investigator.

References

  1. Babor TF, De La Fuente JR, Saunders J, & Grant M (1989). The Alcohol Use Disorders Identification Test: guidelines for use in primary health care. In. World Health Organization. [Google Scholar]
  2. Babor TF, & Grant M (1989). From clinical research to secondary prevention: International collaboration in the development of the Alcohol Use Disorders Identification Test. Alcohol Health and Research World, 13, 371–374. [Google Scholar]
  3. Baker SP, O’Neill B, Haddon W Jr., & Long WB (1974, Mar). The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma, 14(3), 187–196. http://www.ncbi.nlm.nih.gov/pubmed/4814394 [PubMed] [Google Scholar]
  4. Barak-Corren Y, Castro VM, Javitt S, Hoffnagle AG, Dai Y, Perlis RH, Nock MK, Smoller JW, & Reis BY (2017, Feb 1). Predicting Suicidal Behavior From Longitudinal Electronic Health Records. Am J Psychiatry, 174(2), 154–162. 10.1176/appi.ajp.2016.16010077 [DOI] [PubMed] [Google Scholar]
  5. Bradley KA, DeBenedetti AF, Volk RJ, Williams EC, Frank D, & Kivlahan DR (2007). AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol Clin Exp Res, 31(7), 1208–1217. 10.1111/j.1530-0277.2007.00403.x [DOI] [PubMed] [Google Scholar]
  6. Califf RM, & Sugarman J (2015, Oct). Exploring the ethical and regulatory issues in pragmatic clinical trials. Clinical Trials, 12(5), 436–441. 10.1177/1740774515598334 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Centers for Disease Control and Prevention. (2021). Web-based Injury Statistics Query and Reporting System (WISQARS) https://wisqars-viz.cdc.gov:8006/non-fatal/home
  8. Coronado GD, Petrik AF, Vollmer WM, Taplin SH, Keast EM, Fields S, & Green BB (2018, 2018 Sep). Effectiveness of a Mailed Colorectal Cancer Screening Outreach Program in Community Health Clinics: The STOP CRC Cluster Randomized Clinical Trial. JAMA Internal Medicine, 178(9), 1174–1181. 10.1001/jamainternmed.2018.3629 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Darnell D, Parker L, Engstrom A, Fisher D, Diteman K, & Dunn C (2019). Evaluation of a Level I trauma center provider training in patient-centered alcohol brief interventions using the Behavior Change Counseling Index rated by standardized patients. Trauma Surg Acute Care Open, 4(1), e000370. 10.1136/tsaco-2019-000370 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Diggle PJ, Heagerty P, Liang KY, & Zeger SL (2004). Chapter 13. In Analysis of longitudinal data (Oxford Statistical Science Series) (2nd ed., Vol. 23, pp. 3399–3401). Oxford University Press. 10.1002/sim.1701 [DOI] [Google Scholar]
  11. Doshi RP, Chen K, Wang F, Schwartz H, Herzog A, & Aseltine RH (2020, 2020/09/16). Identifying risk factors for mortality among patients previously hospitalized for a suicide attempt. Scientific reports, 10(1), 15223. 10.1038/s41598-020-71320-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Ellenberg SS, Culbertson R, Gillen DL, Goodman S, Schrandt S, & Zirkle M (2015). Data monitoring committees for pragmatic clinical trials. Clinical Trials, 12(5), 530–536. 10.1177/1740774515597697 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gibbons RD, Hedeker D, & DuToit S (2010). Advances in analysis of longitudinal data [Research Support, N.I.H., Extramural]. Annual review of clinical psychology, 6, 79–107. 10.1146/annurev.clinpsy.032408.153550 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Gilbody S, Richards D, Brealey S, & Hewitt C (2007, Nov). Screening for depression in medical settings with the Patient Health Questionnaire (PHQ): a diagnostic meta-analysis. J Gen Intern Med, 22(11), 1596–1602. 10.1007/s11606-007-0333-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Goodsmith N, Zhang L, Ong MK, Ngo VK, Miranda J, Hirsch S, Jones F, Wells K, & Chung B (2021, 2021/03/01). Implementation of a Community-Partnered Research Suicide-Risk Management Protocol: Case Study From Community Partners in Care. Psychiatric Services, 72(3), 281–287. 10.1176/appi.ps.202000095 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Heinze G, Wallisch C, & Dunkler D (2018, 2018/05/01). Variable selection – A review and recommendations for the practicing statistician [ 10.1002/bimj.201700067]. Biometrical Journal, 60(3), 431–449. 10.1002/bimj.201700067 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Huang SS, Septimus E, Kleinman K, Moody J, Hickok J, Heim L, Gombosev A, Avery TR, Haffenreffer K, Shimelman L, Hayden MK, Weinstein RA, Spencer-Smith C, Kaganov RE, Murphy MV, Forehand T, Lankiewicz J, Coady MH, Portillo L, Sarup-Patel J, Jernigan JA, Perlin JB, & Platt R (2019, 2019/03/23/). Chlorhexidine versus routine bathing to prevent multidrug-resistant organisms and all-cause bloodstream infections in general medical and surgical units (ABATE Infection trial): a cluster-randomised trial. The Lancet, 393(10177), 1205–1215. 10.1016/S0140-6736(18)32593-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Jarvik JG, Meier EN, James KT, Gold LS, Tan KW, Kessler LG, Suri P, Kallmes DF, Cherkin DC, Deyo RA, Sherman KJ, Halabi SS, Comstock BA, Luetmer PH, Avins AL, Rundell SD, Griffith B, Friedly JL, Lavallee DC, Stephens KA, Turner JA, Bresnahan BW, & Heagerty PJ (2020, Sep 1). The Effect of Including Benchmark Prevalence Data of Common Imaging Findings in Spine Image Reports on Health Care Utilization Among Adults Undergoing Spine Imaging: A Stepped-Wedge Randomized Clinical Trial. JAMA Netw Open, 3(9), e2015713. 10.1001/jamanetworkopen.2020.15713 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kessler R, Sonnega A, Bromet E, Hughes M, & Nelson CB (1995, Dec). Posttraumatic stress disorder in the National Comorbidity Survey. Arch Gen Psychiatry, 52(12), 1048–1060. 10.1001/archpsyc.1995.03950240066012 [DOI] [PubMed] [Google Scholar]
  20. Kroenke K, Spitzer RL, & Williams JB (2001, Sep). The PHQ-9: Validity of a brief depression severity measure. J Gen Intern Med, 16(9), 606–613. 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Lagomasino IT, Zatzick D, & Chambers DA (2010). Efficiency in mental health practice and research. Gen Hosp Psychiatry, 32(5), 477–483. http://www.ncbi.nlm.nih.gov/pubmed/20851267 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Levis B, Benedetti A, & Thombs BD (2019). Accuracy of Patient Health Questionnaire-9 (PHQ-9) for screening to detect major depression: individual participant data meta-analysis. BMJ, 365, l1476. 10.1136/bmj.l1476 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Little RJ, D’Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA, Neaton JD, Rotnitzky A, Scharfstein D, Shih WJ, Siegel JP, & Stern H (2012, 2012/10/04). The Prevention and Treatment of Missing Data in Clinical Trials. New England Journal of Medicine, 367(14), 1355–1360. 10.1056/NEJMsr1203730 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Loomer L, Ogarek JA, Mitchell SL, Volandes AE, Gutman R, Gozalo PL, McCreedy EM, & Mor V (2021, 2021/03/01). Impact of an Advance Care Planning Video Intervention on Care of Short-Stay Nursing Home Patients [ 10.1111/jgs.16918]. J Am Geriatr Soc, 69(3), 735–743. 10.1111/jgs.16918 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, & Zwarenstein M (2015, May 8). The PRECIS-2 Tool: Designing trials that are fit for purpose. British Medical Journal, 350, 11. 10.1136/bmj.h2147 [DOI] [PubMed] [Google Scholar]
  26. MacKenzie EJ, McCarthy ML, Ditunno JF, Forrester-Staz C, Gruen GS, Marion DW, & Schwab WC (2002, Mar). Using the SF-36 for characterizing outcome after multiple trauma involving head injury. J Trauma, 52(3), 527–534. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=11901330 [DOI] [PubMed] [Google Scholar]
  27. Michaels AJ, Michaels CE, Moon CH, Smith JS, Zimmerman MA, Taheri PA, & Peterson C (1999). Posttraumatic stress disorder after injury: Impact on general health outcome and early risk assessment. Journal of Trauma, 47(3), 460–467. [DOI] [PubMed] [Google Scholar]
  28. Millner AJ, Lee MD, & Nock MK (2015, October 23 2015). Single-Item Measurement of Suicidal Behaviors: Validity and Consequences of Misclassification. PLoS One, 10(10), e0141606. 10.1371/journal.pone.0141606 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Moodliar R, Russo J, Bedard-Gilligan M, Moloney K, Johnson P, Seo S, Vaziri N, & Zatzick D (2020). A Pragmatic Approach to Psychometric Comparisons between the DSM-IV and DSM-5 Posttraumatic Stress Disorder (PTSD) Checklists in Acutely Injured Trauma Patients. Psychiatry. 10.1080/00332747.2020.1762396 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Moriarty AS, Gilbody S, McMillan D, & Manea L (2015, Nov-Dec). Screening and case finding for major depressive disorder using the Patient Health Questionnaire (PHQ-9): a meta-analysis. Gen Hosp Psychiatry, 37(6), 567–576. 10.1016/j.genhosppsych.2015.06.012 [DOI] [PubMed] [Google Scholar]
  31. NIH Collaboratory. (2021). Health care systems research collaboratory: Rethinking clinical trials. https://rethinkingclinicaltrials.org/
  32. NIMH. (2020). National Institute of Mental Health Strategic Plan for Research (20-MH-8096). U.S. Department of Health and Human Services. https://www.nimh.nih.gov/about/strategic-planning-reports/ [Google Scholar]
  33. O’Connor SS, Dinsio K, Wang J, Russo J, Rivara FP, Love J, McFadden C, Lapping-Carr L, Peterson R, & Zatzick DF (2014, Oct). Correlates of suicidal ideation in physically injured trauma survivors [Research Support, N.I.H., Extramural]. Suicide & life-threatening behavior, 44(5), 473–485. 10.1111/sltb.12085 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. O’Connor SS, McClay MM, Powers J, Rotterman E, Comtois KA, Ellen Wilson J, & Nicolson SE (2021, 2021/04/19). Measuring the impact of suicide attempt posttraumatic stress [ 10.1111/sltb.12733]. Suicide and Life-Threatening Behavior, Advance online publication. 10.1111/sltb.12733 [DOI] [PubMed] [Google Scholar]
  35. Palinkas LA, & Zatzick D (2019). Rapid assessment procedure informed clinical ethnography (RAPICE) in pragmatic clinical trials of mental health services implementation: Methods and applied case study. Administration and Policy in Mental Health and Mental Health Services Research, 46(2), 255–270. 10.1007/s10488-018-0909-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Roberts MK, Fisher DM, Parker LE, Darnell D, Sugarman J, Carrithers J, Weinfurt K, Jurkovich G, & Zatzick D (2020, 2020/09/01). Ethical and Regulatory Concerns in Pragmatic Clinical Trial Monitoring and Oversight [ 10.1002/eahr.500066]. Ethics & Human Research, 42(5), 29–37. 10.1002/eahr.500066 [DOI] [PubMed] [Google Scholar]
  37. Simon, Coleman KJ, Rossom RC, Beck A, Oliver M, Johnson E, Whiteside U, Operskalski B, Penfold RB, Shortreed SM, & Rutter C (2016, Feb). Risk of suicide attempt and suicide death following completion of the Patient Health Questionnaire depression module in community practice. J Clin Psychiatry, 77(2), 221–227. 10.4088/JCP.15m09776 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Simon GE, Beck A, Rossom R, Richards J, Kirlin B, King D, Shulman L, Ludman EJ, Penfold R, Shortreed SM, & Whiteside U (2016, Sep 15). Population-based outreach versus care as usual to prevent suicide attempt: Study protocol for a randomized controlled trial. Trials, 17(1), 452. 10.1186/s13063-016-1566-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Simon GE, Rutter CM, Peterson D, Oliver M, Whiteside U, Operskalski B, & Ludman EJ (2013, Sept 16, 2013). Does response on the PHQ-9 depression questionnaire predict subsequent suicide attempt or suicide death? Psychiatric services In Advance, 8. 10.1176/appi.ps.201200587 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Simon GE, Shortreed SM, Rossom RC, Penfold RB, Sperl-Hillen JAM, & O’Connor P (2019, 2019/12/09). Principles and procedures for data and safety monitoring in pragmatic clinical trials. Trials, 20(1), 690. 10.1186/s13063-019-3869-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Smith PC, Schmidt SM, Allensworth-Davies D, & Saitz R (2010, Jul 12). A single-question screening test for drug use in primary care. Arch Intern Med, 170(13), 1155–1160. https://doi.org/170/13/1155 [pii] 10.1001/archinternmed.2010.140 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Stanley B, & Brown B (2012). Safety Planning Intervention: A Brief Intervention to Mitigate Suicide Risk. Cognitive and Behavioral Practice, 19(2), 256–264. 10.1016/j.cbpra.2011.01.001 [DOI] [Google Scholar]
  43. The Johns Hopkins Health Services Research and Development Center. (1989). Determining injury severity from hospital sischarges: A program to map ICD-9DM diagnoses into AIS, and ISS severity scores.
  44. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, & Altman DG (2009). A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol, 62. [DOI] [PubMed] [Google Scholar]
  45. Ware JE, Kosinski M, & Keller SD (1996). A 12-item short-form health survey: Construction of scales and preliminary tests of reliability and validity. Med Care, 34(3), 220–223. [DOI] [PubMed] [Google Scholar]
  46. Ware JE, Snow KK, & Kosinski M (1993). SF-36 health survey: manual and interpretation guide. The Health Institute, New England Medical Center. [Google Scholar]
  47. Weathers F, & Ford J (1996). Psychometric review of PTSD Checklist (PCL-C, PCL-S. PCL-M, PCL-PR). In Stamm B (Ed.), Measurement of stress, trauma, and adaptation (pp. 250–251). Sidran Press. [Google Scholar]
  48. Weathers FW, Litz BT, Keane TM, Palmieri PA, Marx BP, & Schnurr PP (2013). The PTSD Checklist for DSM-5 (PCL-5) – Standard [Measurement instrument]. https://www.ptsd.va.gov/ [Google Scholar]
  49. Zatzick, Rivara F, Jurkovich G, Hoge C, Wang J, Fan M, Russo J, Geiss Trusz S, Nathens A, & Mackenzie E (2010). Multi-site investigation of traumatic brain injuries, posttraumatic stress disorder, and self-reported health and cognitive impairments. Arch Gen Psychiatry, 67(12), 1291–1300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Zatzick D, Donovan DM, Dunn C, Jurkovich GJ, Wang J, Russo J, Rivara FP, Zatzick CD, Love JR, McFadden CR, & Gentilello LM (2013). Disseminating Organizational Screening and Brief Intervention Services (DO-SBIS) for alcohol at trauma centers study design. Gen Hosp Psychiat, 35(2), 174–180. 10.1016/j.genhosppsych.2012.11.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Zatzick D, Jurkovich G, Heagerty P, Russo J, Darnell D, Parker L, Roberts MK, Moodliar R, Engstrom A, Wang J, Bulger E, Whiteside L, Nehra D, Palinkas LA, Moloney K, & Maier R (2021). Stepped Collaborative Care Targeting Posttraumatic Stress Disorder Symptoms and Comorbidity for US Trauma Care Systems: A Randomized Clinical Trial. JAMA Surgery. 10.1001/jamasurg.2021.0131 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Zatzick D, Jurkovich G, Rivara FP, Russo J, Wagner A, Wang J, Dunn C, Lord SP, Petrie M, O’Connor SS, & Katon W (2013). A randomized stepped care intervention trial targeting posttraumatic stress disorder for surgically hospitalized injury survivors. Annals of Surgery, 257(3), 390–399. 10.1097/SLA.0b013e31826bc313 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Zatzick D, Jurkovich GJ, Gentilello LM, Wisner DH, & Rivara FP (2002). Posttraumatic stress, problem drinking, and functioning 1 year after injury. Archives of Surgery, 137(2), 200–205. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieveanddb=PubMedanddopt=Citationandlist_uids=11822960 [DOI] [PubMed] [Google Scholar]
  54. Zatzick D, O’Connor SS, Russo J, Wang J, Bush N, Love J, Peterson R, Ingraham L, Darnell D, Whiteside L, & Van Eaton E (2015, Oct). Technology-Enhanced Stepped Collaborative Care Targeting Posttraumatic Stress Disorder and Comorbidity After Injury: A Randomized Controlled Trial. J Trauma Stress, 28(5), 391–400. 10.1002/jts.22041 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Zatzick D, Rivara F, Jurkovich G, Russo J, Trusz SG, Wang J, Wagner A, Stephens K, Dunn C, Uehara E, Petrie M, Engel C, Davydow D, & Katon W (2011, Mar-Apr). Enhancing the population impact of collaborative care interventions: Mixed method development and implementation of stepped care targeting posttraumatic stress disorder and related comorbidities after acute trauma. Gen Hosp Psychiatry, 33(2), 123–134. https://doi.org/S0163-8343(11)00002-8 [pii] 10.1016/j.genhosppsych.2011.01.001 [Record #385 is using a reference type undefined in this output style.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Zatzick D, Roy-Byrne P, Russo J, Rivara F, Droesch R, Wagner A, Dunn C, Jurkovich G, Uehara E, & Katon W (2004, May). A randomized effectiveness trial of stepped collaborative care for acutely injured trauma survivors. Arch Gen Psychiatry, 61(5), 498–506. 10.1001/archpsyc.61.5.498 [DOI] [PubMed] [Google Scholar]
  57. Zatzick D, Russo J, Darnell D, Chambers DA, Palinkas L, Van Eaton E, Wang J, Ingraham L, Guiney R, Heagerty P, Comstock B, Whiteside L, & Jurkovich G (2016, Apr 30). An effectiveness-implementation hybrid trial study protocol targeting posttraumatic stress disorder and comorbidity. Implement Sci, 11, 58. 10.1186/s13012-016-0424-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

De-identified participant data will be shared with investigators whose proposed data use has been approved through requests to the principal investigator.

RESOURCES