Evaluation of a suicide prevention training curriculum for substance abuse treatment providers based on Treatment Improvement Protocol Number 50 (TIP 50)

Abstract

Substance use disorders (SUD) confer risk for suicide yet there are no empirically supported suicide prevention training curricula tailored to SUD treatment providers. We assessed the efficacy of a 2-hour training that featured a suicide prevention training video produced by the Department of Veterans Affairs (VA). The video was based on Treatment Improvement Protocol Number 50, TIP 50, a practical manual to manage suicide risk produced by the Substance Abuse and Mental Health Services Administration (SAMHSA). The training was provided in small groups to 273 SUD treatment providers in 18 states. Results were evaluated using self-report assessments obtained at pre-test, post-test, and 2-month follow-up. Statistically significant changes (p<.001) within subjects were obtained on self-efficacy, knowledge, and frequency of suicide prevention practice behaviors. The positive results together with the brevity of the training and its ease of implementation indicate high potential for widespread adoption and the importance of further study.

Introduction

Substance use disorders (SUD) are a potent risk factor for suicide (Yoshimasu, Kiyohara, & Miyashita, 2008) and individuals who present to SUD treatment settings are at especially high risk (Wilcox, Conner, & Caine, 2004). A meta-analysis estimated 10-to-20 fold risk for individuals coming to clinical attention for alcohol dependence, opiate dependence, and intravenous drug use, respectively (Wilcox et al., 2004). A national study of individuals receiving Veterans Health Administration (VHA) services showed that a SUD diagnosis conferred higher risk for suicide among female Veterans than any other mental disorder (Ilgen et al., 2010). These data underscore the critical role that SUD treatment venues can play in suicide prevention.

The research literature lacks data about effective methods for educating SUD treatment providers to identify and manage suicide risk. Research on workforce education to prevent suicide has focused on curricula aimed at mental health treatment providers (Pisani, Cross, & Gould, 2011). Although SUD treatment providers participate in programs that target mental health clinicians and are sometimes included in study samples (e.g., (Oordt, Jobes, Fonseca, & Schmidt, 2009; Pisani, Cross, Watts, & Conner, in press), the clinical scenarios and practice recommendations presented in these programs generally do not match SUD treatment focus and provider responsibilities. For example, workshops aimed at mental health professionals typically spend a substantial proportion of their time focusing on independent clinical formulation and judgment of risk (Pisani et al., 2011), skills that are generally outside the scope of practice and core competencies for frontline SUD treatment providers (Center for Substance Abuse Treatment, 2006). Relatedly, it is our experience that many SUD treatment providers perceive that their role in suicide prevention is limited to referring clients for mental health evaluation. Thus, practical and practice-relevant methods are needed to educate providers to meet the safety needs of their clients.

In recognition of high risk of suicide in SUD patients and the need to provide guidelines tailored to the needs of SUD programs, the Center for Substance Abuse Treatment (CSAT) at the Substance Abuse and Mental Health Services Administration (SAMHSA) impaneled a group of experts in SUD treatment and suicide prevention to produce Treatment Improvement Protocol Number 50 (TIP 50). The TIP 50 manual provides practical guidance to front-line substance abuse counselors, supervisors, and program administrators to manage suicide risk (Center for Substance Abuse Treatment, 2009). The centerpiece of TIP 50 is an overarching framework referred to as GATE: Gather Information, Access Supervision, Take Appropriate Action, Extend the Action (beyond the immediate situation). By offering this framework and other key information, TIP 50 filled a critical gap because there had been no consensus recommendations tailored to the daily practice of SUD front-line clinicians, supervisors, and administrators.

To accompany the TIP 50 manual, VHA Office of Mental Health Services (OMHS) collaborated with VA Employee Education System to produce a 75 -minute video to model the suicide prevention framework and skills presented in the TIP (VA Employee Education System, 2010). Founded on social learning theory (Bandura, 1986), video modeling is used to display skills the viewer is expected to replicate in a desired context (Catania, Almeida, Liu-Constant, & DiGennaro Reed, 2009). Video modeling permits consistent, standardized demonstration of desired skills in a context that closely resembles the targeted practice setting (Morgan & Salzberg, 1992; Neef, Trachtenberg, Loeb, & Sterner, 1991; Reeve, Reeve, Townsend, & Poulson, 2007). The purpose of this study was to assess the efficacy of the modeling curriculum based on the VA TIP 50 video to improve suicide prevention self-efficacy, knowledge, and use of practice behaviors.

Methods

Video model

One of the authors (KRC) who chaired the original TIP 50 committee drafted the script for the VA TIP 50 video. The script was reviewed and edited by a multidisciplinary team of experienced VHA providers and administrators along with two members of the original TIP 50 panel. The resulting 75-minute video features a group of VHA clinicians who explain GATE and other points in TIP 50. The use of GATE is further illustrated through a series of dramatic vignettes of the case of Antonio, a 25-year old Veteran of Operation Iraqi Freedom who becomes suicidal during the course of outpatient SUD treatment. Although the case is fictitious, it is based on common features of suicide risk in SUD patients, and the clinicians’ handling of the case serves as a model for implementation of GATE that is applicable to VHA as well as civilian SUD treatment settings.

Trainers

Each VHA Medical Center has a Suicide Prevention Coordinator (SPC) who has primary responsibility for coordinating the care of Veterans at high risk and providing suicide prevention training to the local community both within and outside the VHA treatment system. SPC volunteers provided the training for the current evaluation project. They were instructed how to conduct the training at a workshop held in August, 2010. At the workshop, the SPCs were introduced to the TIP 50 manual, showed the VA TIP 50 training tape, and discussed the tape. Next, the SPCs were oriented to the evaluation procedure (described below).

Procedure

The investigative team at the VA VISN 2 Center of Excellence for Suicide Prevention (CoE) mailed SPC volunteers a packet containing instructions, the TIP 50 video, a set of TIP 50 manuals, and pre-training and post-training questionnaires. The SPCs were asked to deliver a training lasting 2 hours or less to a group of substance abuse providers in their local area. These trainings were to consist of: 1) handing out the TIP 50 manual, 2) administering pre-training questionnaires, 3) showing the TIP 50 video, 4) facilitating a brief discussion of the video (about 10–15 minutes), and 5) administering post-test questionnaires. Following the training the SPCs collected the questionnaires and mailed them to the CoE. Two months following the training, the SPCs were mailed a packet of follow-up questionnaires to administer to individuals who took part in the original training. As needed, CoE staff provided email and phone reminders on a predetermined schedule to SPCs to complete the follow-ups. SPCs mailed the completed follow-ups to the CoE.

Measures

1) Self-efficacy: A 6-item measure of self-efficacy to manage suicide risk based on an established scale (Pisani et al., in press) was administered at pre-test, post-test, and follow-up. The items were presented on a 4-point range of agreement (1–4), with possible scores ranging from 6 to 24. An example item was “I feel prepared to involve a client’s family member in suicide risk management.” 2) Knowledge: A 5-item measure of knowledge pertinent to the training was created for the study and administered at pre-test and post-test. Each item was answered “true” or “false” and 1-point was given for each correct response, for a possible range of 0 to 5. A sample item was “Different methods of suicidal behavior (e.g., guns, pills, etc.) pose similar risk for death” (note “false” is correct). 3) Practice Behaviors: An 8-item measure of frequency of performance of various suicide prevention practices was developed based on previous work measuring changes in practice behaviors among VHA and US Air Force clinicians (Oordt et al., 2009). The measure was administered at pre-test and 2-month follow-up. The items were on a 5-point scale of agreement (1–5), for a possible range of scores from 8 to 40. A sample item was “I follow-up with actions after the acute suicide crisis has passed in order to reduce long-term risk.” 4) Perceived Usefulness: Questions on perceived usefulness of the training and training materials were asked at post-test and follow-up.

Data Management and Analyses

All written responses from the questionnaires (pre-training, post-training, and follow-up) were entered into a SPSS spreadsheet at the CoE. No identifying information was contained in the written packets. Therefore, the pre/post questionnaires were linked with the follow-ups by the CoE investigators through matching the answers to a series of questions on provider characteristics (e.g., primary work setting) and a few additional questions to enable linkage (e.g., favorite musician) that were administered at pre-test and at follow-up. Changes associated with the training within persons were assessed using paired-sample t-tests that compared scores on self-efficacy (pre-test vs. post-test, pre-test vs. follow-up), knowledge (pre-test vs. post-test), and practice behaviors (pre-test vs. follow-up). Effect sizes, a standardized indication of change (Cohen, 1988), were calculated by dividing the difference between scores in a given trainee (e.g., pre-test vs. post-test) by the standard deviation of scores among trainees at pre-test. Effect sizes were interpreted as small (.20), medium (.50) and large (.80) (Cohen, 1988). For scales with missing data on 25% or fewer items, we replaced missing values with the mean score of other items on the scale. We deleted a score from the analysis if it had more than 25% missing values. Internal consistency of scales was based on coefficient alpha (α) (Streiner, 2003); mean scores that replaced missing items were not included in the alpha calculations.

Results

Twenty-one SPCs conducted a TIP 50 video training session. They sent in pre-test/post-test questionnaires from a total of 273 participants, with sessions averaging 13 trainees. These trainings were typically accomplished in 2 hours or less and took place in 18 states: AL, CA, CO, GA, ID, KS, LA, MI, MO, NC, NY, OR, RI, SD, TN, WI, WV, and WY. Fifteen SPCs (71.4%) provided follow-up data at approximately two months on 119 (43.6%) participants.

Data on professional characteristics of the participants are based on the pre-test sample. Nearly all (94.1%) participants reported working primarily for VHA. Primary professional responsibilities were: treatment provider (48.7%), case manager (19.8%), supervisor (3.7%), student/trainee (3.7%), administrator (2.9%), and other (18.7%). Primary work settings were: ambulatory or outpatient (51.6%), inpatient or residential (23.8%), detoxification (2.6%), crisis line (0.2%), and other (35.2%). Most participants (55.7%) reported that their caseload consisted exclusively of patients with a SUD diagnosis. Self-reported experience with suicide risk management was: none (4.0%), little (12.1%), some (48.4%), great deal (35.5%). Most (62.3%) reported attending at least one previous suicide prevention training of 2 hours or longer. CoE investigators were prevented from gathering data on participant age, sex, or race/ethnicity to safeguard participant anonymity.

Self-efficacy scale showed high internal consistency, α=.90. Mean (SD) self-efficacy scores were: pre-test, 18.7 (4.5); post-test, 21.1 (3.6); and follow-up, 21.4 (2.7). Result of comparison of pre-test vs. post-test scores showed a significant improvement, t (1, 201) = 12.4 (p<0.001), with a corresponding effect size = .53 (medium). Result of comparison of pre-test vs. follow-up scores also showed significant improvement, t (1, 78) =7.1 (p<0.001), with effect size = .60 (medium). Knowledge scale showed low internal consistency, α=.53. Mean (SD) knowledge scores were: pre-test, 2.8 (1.3), post-test (3.8, 1.1). Result of comparison of pre-test vs. post-test scores showed a significant improvement, t (1, 191) = 12.0 (p<0.001), with effect size = .77 (large). Practice behaviors scale showed high internal consistency (α = .92). Mean (SD) practice behavior scores were: pre-test, 31.9 (6.8); follow-up, 34.3 (6.0). Result of comparison of pre-test vs. follow-up scores showed significant improvement, t(1, 72) = 4.7 (p<.001), with effect size = .35 (small to medium). Perceived usefulness: At post-test, 90.1% of participants answered “agree” or “strongly agree” to the item “The training helped me know how to apply the information to my work.” At follow-up, 96.7% of participants answered “agree” or “strongly agree” to the item “The training video provided me helpful information in working with suicidal clients.” Also at follow-up, 97.7% of participants answered “agree” or “strongly agree” to the item “The TIP 50 Manual provides me a helpful ongoing resource in working with suicidal clients.”

Discussion

The present evaluation examined changes in SUD treatment providers following a brief video- modeling curriculum designed to teach the concepts and skills outlined in TIP 50, a manual to manage suicide risk. Results show that video modeling had a large impact on participant knowledge and a medium impact on self-efficacy assessed immediately after training. Results also showed that the video and TIP 50 manual had a medium impact on self-efficacy and a small to medium impact on the frequency of use of suicide risk management practice behaviors assessed at about 2-month follow-up. The training and materials were very well received by participants based on their positive responses on questions about the perceived usefulness of the training, the VA TIP 50 Video, and the TIP 50 manual.

There are major advantages to the TIP 50 video modeling training as described here for the sake of adoption by SUD treatment programs (Damschroder & Hagedorn, 2011). First, the primary learning vehicle—a realistic video modeling skills with a suicidal substance abuse patient—has strong potential to promote transfer of learning to practice. Findings from this study are congruent with research elsewhere in the literature suggesting that video modeling is a practice and effective method for teaching new skills. This study is the first to demonstrate the promise of video modeling for teaching suicide prevention skills. Second, the training is brief (≤2 hours), overcoming concerns about the time and expense of training, a major barrier to adoption (Pisani et al., in press). Such brevity may be especially appealing to SUD treatment programs that are unlikely to view suicide prevention as a primary focus of treatment but nonetheless must demonstrate the capability to recognize and respond to suicide risk given the demands of a high-risk clinical population as well as for the practical purpose of meeting accreditation requirements. Third, the TIP 50 manual is free of charge to individual providers as well as to service agencies, and may be acquired by contacting SAMHSA (1-877-726-4727) or by visiting the SAMHSA website (www.samhsa.gov). The TIP 50 video is also free of charge from VA by contacting Jane Wood, RN (jane.wood2@va.gov) or via internet (http://www.mirecc.va.gov/index.asp). It may also be viewed free of charge on “Youtube.” Fourth, the training is video-based, enabling group training with minimal preparation as well as self-administration.

There are limitations of the evaluation. Data were based on self-report. Effect sizes were based on change within subjects, with no comparison group. There was significant attrition over follow-up and missing data on scales. The knowledge scale had low internal consistency. Because the video was shown and the TIP 50 manual was handed out during the training, their relative impact (if any) on changes at 2-month follow-up could not be disentangled (i.e., some learning could have occurred after training through review of the manual). The sample was nearly all clinicians in VHA who had a generally high level of prior training and experience in suicide prevention, attributable to the strong emphasis on suicide prevention in VHA. Generalizability of results to non-VHA substance abuse treatment professions who may be expected to have less training and experience in suicide prevention is unclear (although this may be a positive as civilian providers may stand to benefit most given limited prior training). The SPCs who provided the training were experienced mental health treatment providers with advanced training in suicide risk management. Therefore, we felt confident to simply instruct the SPCs to facilitate a brief discussion after showing the video, yet a more structured format to guide discussion may be beneficial for wider use.

The current evaluation stands out for examining a suicide risk management training with high potential for adoption because it relies on viewing a 75 minute video and all materials can easily be obtained free of charge. Previously there were no practical, consensus-based materials to educate and guide substance abuse treatment providers, supervisors, and administrators in managing suicide risk. In our view, the VA TIP 50 training tape is best thought of as a strategy to efficiently introduce and model the concepts in the TIP 50 manual, with the expectation that providers will be better equipped to use the manual as a go-to resource on an ongoing basis. In the current evaluation the frequency of practice behaviors was increased at a statistically significant level at 2-month follow-up, and this result is impressive in light of our use of brief, single session training along with handing out the TIP 50 manual, and it is exciting because affecting practice behaviors is of utmost importance yet especially challenging (Oordt et al., 2009). However, whether or not it would be sustained over longer time periods is unclear in light of the limitations of single session training documented in the literature (Pisani et al., 2011). Accordingly, further exploration of practical yet effective ways to increase the dissemination, impact and sustainability of the training is needed.