Educational Preferences and Outcomes From Suicide Prevention Training in the Veterans Health Administration: One-Year Follow-Up With Healthcare Employees in Upstate New York

Abstract

This study identifies training outcomes and educational preferences of employees who work within the Veterans Health Administration (VHA). Using a longitudinal pre-postsurvey design, 71 employees from one geographic region of VHA healthcare facilities participated in an evaluation of a brief standardized gatekeeper program and a needs assessment on training preferences for suicide and suicide prevention. Results indicate significant differences in knowledge and self-efficacy from pre to post (p < 0.001), although only self-efficacy remained significant at 1 year follow-up, (M = 3.01; SD = 0.87) as compared to pretraining (M = 2.50, SD = 1.05) (t = −5.64, p < 0.001). At post-training, 90% of the participants were willing to learn more about suicide, with 88% willing to spend more than 1 hour in future training activities on more advanced topics. This training program can increase the knowledge and abilities of VHA staff to engage, identify, and refer veterans at risk for suicide to appropriate care.

INTRODUCTION

Clinicians in various disciplines have recognized that suicide is one of the leading concerns in health^1,2 and mental health service delivery.^3,4 Studies have identified concerns in working with individuals at risk for suicide ranging from stress and coping with emergency situations,⁵ being sued for malpractice,⁶ having previous experience with losing a client to suicide,^5,7,8 and possessing adequate technical and personal competence⁹ to manage at-risk clients. Given that suicide is an area of great concern, suicide prevention training may be an important area for enhanced knowledge and skill development among healthcare professionals.

Research has focused on the amount of suicide-specific training that clinical professionals receive within their respective training programs. Among social workers, 70% reported having some formal training in the study of suicide,¹⁰ whereas a study of accredited social work master’s programs found that only 29% of programs provide specific training in the study of suicide.¹¹ Another study found that 40% of all clinical psychology programs offer formal training in the study of suicide,¹² yet 53% of trainees from these programs reported only some specialized training (e.g., one or two lectures) on how to intervene with an individual at risk for suicide.¹³ A study of accredited predoctoral psychology internships and psychiatry residency programs found that 75% and 94% of programs reported formal training in the study of suicide, respectively.^7,14 Finally, in a study of supervised clinical work for general psychiatry and child fellowship programs, the authors determined that across 166 sites, 94% offered training in suicide with opportunities for focused attention in the study of suicide found in only a quarter of programs.⁷ With percentages of formal didactic training ranging from 29% to 94% among these professionals, such wide variability in training suggests a need to focus efforts on teaching a basic skill set for all clinicians to identify individuals at risk for suicide.

In addition to healthcare professionals’ involvement in suicide prevention, lay health workers, (i.e., individuals with no formal training who are trained in a specific intervention and who function as delivery agents for a range of primary and community healthcare interventions), have shown promise in improving “uptake” of healthy behaviors such as cancer screenings, immunization, breastfeeding, and improving outcomes such as reducing deaths of elders by providing home-health aide services.¹⁵ Specific to suicide prevention, studies have found that knowledge and attitudes about suicide can be improved by training nurses,¹⁶ front line nonclinical employees in a department of psychiatry,¹⁷ and outpatient readjustment counseling teams composed of clinicians, lay outreach workers, and administrative staff.¹⁸

One community-based prevention approach to identifying individuals with warning signs of psychosocial distress is gatekeeper training. Gatekeepers are defined as community members and staff who, with training, can identify individuals in distress and refer them to care before a suicidal crisis.^19,20 Community gatekeeper training is a systemwide prevention approach to awareness, education, and skill building with the goals of increasing the knowledge and competency of trainees in the recognition of and crisis intervention with potentially suicidal individuals (Quinnett P, unpublished work). Gatekeeper training is based on empirical evidence that individuals most at risk for self-destruction and violence tend not to self-refer for professional help but do tend to communicate their distress to those around them through detectable and recognizable behaviors, such as making threats to end their life.²¹ This training is also based on research that “warning signs,” as differentiated from risk factors, are important forewarning signals indicating emotional distress, hopelessness, and intent to die.^22,23 Research has indicated that these signs can be taught and recalled easily, and that persons trained in the recognition of suicidal communications and warning signs can be trained to intervene in a helpful fashion.²⁴

In response to the case reports of suicides of deployed soldiers upon return from the wars in Iraq and Afghanistan,^25,26 the increased risk of suicide among veterans in the community, some of whom may or may not be enrolled in the Veterans Health Administration (VHA) of the Department of Veterans Affairs,²⁷ and the elevated rates of suicide among older cohorts of healthcare-seeking veterans with psychiatric disorders,^28,29 a large needs assessment of VHA clinical professionals (i.e., physicians, nurses, psychologists, social workers, etc.) and nonclinical staff (i.e., “front line” administrative staff and paraprofessional community outreach workers), was initiated in fiscal year 2006. A partnership was developed between the leadership of a national network of community-based counseling centers for veterans, a regional network of hospitals and clinics for veterans in New York state, and researchers from a university-based center on suicide prevention research. This collaboration benefited from national policy efforts^30–34 placing suicide prevention at the forefront for improving mental health service delivery for the veteran population. The purpose of this article is to present results from our pilot study focusing on a needs assessment of preferences for education on suicide prevention and training outcomes from one standardized training program. To our knowledge, this is the first study of gatekeeper training for suicide prevention delivered to VHA staff working in healthcare settings.

METHODS

Subjects

VHA staff in two organizationally distinct entities within the national healthcare system for veterans: community-based readjustment counseling centers (Vet Centers) and a set of healthcare facilities in one region within the VHA (Veterans Integrated Service Network 2, VISN 2), were offered suicide prevention training via employee education or patient safety training. One stratum surveyed was employees from VISN 2, which was of particular interest because of the diversity of clinicians and lay health workers and their respective work settings in primary care, specialty mental healthcare, and homeless programs for veterans. The data presented here are a subset of this larger study, focusing specifically on this group of VHA employees who worked at a medical center or a community-based outpatient clinic in VISN 2. A brief overview of the methods is presented here as the data collection procedures, intervention, and measures are described in detail elsewhere.¹⁸

A doctoral-level social worker, certified by the QPR Institute, Inc. and who was also a previous VHA employee (M.M.M.), conducted three sessions of a brief, standardized community gatekeeper suicide prevention training³⁵ in various locations across the region. The instructor presented the same 1-hour multimedia training to groups of 25–30 attendees. The study participants completed surveys before, immediately after, and at 1-year follow-up to this training. Participants were also offered an opportunity to “practice gatekeeper skills” in a small peer group format immediately following the large group presentation. ^17,18,36 One year later, researchers (M.S. and L.J.L.) recontacted participants who attended any one of the three trainings to complete the 1-year follow-up assessment. The research team obtained ethical approval from both the university institutional review boards and the affiliated VA facility.

Measures

Surveys were based on similar questions used in previous studies of gatekeeper training.^17,18 The survey packet, currently available upon request from the first author, contained five sections: Demographics, Individual-Level Factors (e.g., interviewing experience, trainings, exposure to suicide, etc.), Gatekeeper Training Evaluation (e.g., declarative knowledge, self-efficacy, satisfaction, referrals, etc.), Evaluation of the Behavioral Rehearsal Practice Session, and an Educational Needs Assessment adapted from program materials.³⁵ The data obtained from the survey and written responses to questionnaire items form the basis of this article. Because of changes in the survey from pre-survey to follow-up, some items were not included in every assessment or were assessed only once.

Demographics included age, gender, race, ethnicity, education, job role, and years of clinical experience. Individual-Level Factors included an assessment of the participant’s lifetime and past year history of education related to suicide/crisis and general/clinical interviewing experience. Exposure to suicide included a series of questions related to lifetime exposure to suicidal individuals (i.e., contact, attempters, and decedants) at pretraining, although past year exposure was assessed at follow-up.

Gatekeeper Training Evaluation

In addition to self-efficacy, declarative knowledge was assessed in the pre-, post-, and follow-up surveys. Declarative knowledge was assessed with 14 items developed for a large-scale randomized trial of gatekeepers in a school district³⁷ and studied previously.^17,18 Perceived self-efficacy about suicide prevention was assessed with a 10-item scale, slightly modified from a standard questionnaire used in QPR training³⁵ and recently examined using factor analysis in another workplace setting (Lezine D. et al, unpublished work). The response options ranged from 0 = poor to 4 = excellent. The Cronbach’s α for the 10-item scale was 0.97 (n = 70) in this study.

At post-training, satisfaction was assessed by 4 items rated on a 5-point Likert scale (1 = strongly disagree; 5 = strongly agree).¹⁷ The Cronbach’s α for the 4-item scale was 0.59 (n = 69) in this study. Training length was assessed with one item to describe the training as too long, too short, or just right. Three questions specifically designed for this study queried about performing gatekeeper activities. The first 2 questions asked whether participants acted more like a gatekeeper in their local community or in their workplace (yes or no). Participants were also asked “since the training, how many referrals have you made (that is referrals of individuals in distress or who are exhibiting suicide warning signs and behaviors to get help, go for services, access care)?” For this analysis, we recoded referrals as a count outcome indicating the number of referrals over the last 12 months. Diffusion of training-related information was assessed at follow-up by 3 items used in previous studies of gatekeeper training.¹⁷ Current awareness of suicide prevention resources were assessed using four items with dichotomous (e.g., yes or no) response options at post-training and at follow-up. These items, specifically designed for this study, asked respondents whether they had any awareness of efforts regarding suicide prevention specific to their workplace, community, state, or nation.

Behavioral Rehearsal Practice Session Evaluation

Participants who engaged in the practice session evaluated their performance using the 3 core gatekeeper skills (asking about suicide, persuading to get help, and making referrals) learned in training using the 3-item (scored yes or no) Peer Observational Checklist.^17,18 Participants’ role play practice experience was also assessed with a 10-item Role Play Acceptability Scale,^17,18 which measures the acceptability of role play using factors that may enhance the potential for transfer of learning on the basis of active learning theory. ³⁶ Respondents rated each item using a 5-point scale (1 = strongly disagree; 5 = strongly agree). The Cronbach’s α for the 10-item scale was 0.94 (n = 45) in this study.

Educational Needs Assessment

The post-training educational needs assessment questionnaire items were adapted from a professional knowledge and skill survey provided to certified community gatekeeper instructors as part of their online instructor resource toolkit.³⁵ After an initial question regarding willingness to learn more about suicide and its prevention, respondents were asked 9 questions related to training needs. Respondents were asked to rate each question using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). The Cronbach’s α for the 9-item scale was 0.94 (n = 66) in this study.

Data Analysis

The data in this study were analyzed using SPSS for Windows 15.0.³⁸ Demographic characteristics and individual level factors were estimated using cross-tabulations. Although pre-training surveys captured information on self-efficacy and declarative knowledge of suicide prevention, post- and follow-up training surveys assessed for changes in these variables. Bivariate analyses, tests for differences in group means using paired t-tests, associated tests of significance, and Cohen’s d effect sizes were conducted to assess the training impact. Finally, we used the McNemar test for correlated proportions to test the significance level for differences in proportions when both proportions refer to the same sample.

Qualitative methods were also used to analyze the written information. Two independent raters (M.M.M. and Y.C.) established a coding scheme to categorize the responses related to each fill-in-the-blank option. Inductive coding was used to examine the content, with raters using an iterative process to generate codes, to group codes, and to refine and reassess the raw data and coding scheme.³⁹ Any discrepancies were resolved through discussion. Content analysis⁴⁰ is inherently subjective; therefore, the results presented here will reflect not only the data but also the frequency and percentage of responses to show reliability and validity.

RESULTS

Demographics

For this study, the sample was predominantly middle-aged Caucasian females with a master’s degree in a clinical or medical profession with less than 11 years of experience (see Table I). Of the 21 participants who completed a 1-year follow-up survey (yielding a 30% response rate), a new question assessed the type of degree attained with all of the allied health disciplines represented (e.g., counselors (n = 1), social workers (n = 7), psychologists (n = 3), nurses (n = 5), certified rehabilitation counselors (n = 1), doctor of pharmacy (n = 1). As noted in Table II, exposure to suicide was common in this sample, with 94% having previous contact with suicidal individuals in their lifetime.

TABLE I.

Characteristics of VHA Employees Attending Community Gatekeeper Training

Demographics		Pretraining (N = 71)
		N	Freq.	%
Age		Range 24–74 M = 45.79 (SD = 11.05)
Gender	Female	68	48	70.6
	Male	68	20	29.4
Race	Caucasian/White	70	60	85.7
	African American/Black	70	8	11.4
	Asian American	70	2	2.9
Ethnicity	Hispanic or Latino Origin	67	1	1.5
Education	High School	71	3	4.2
	Trade or Vocational School	71	2	2.8
	Associates Degree	71	8	11.3
	Bachelors Degree	71	11	15.5
	Masters Degree	71	37	52.1
	Doctorate Degree	71	10	14.1
Job Role	Technical, Paraprofessional	68	1	1.5
	Clerical, Secretary	68	2	2.9
	Administrative Personnel	68	2	2.9
	Professional, Clinical	68	48	70.6
	Professional, Medical	68	14	20.6
	Administration or Managers	68	1	1.5
Years Practicing	Less than 1 year	55	3	5.5
	1–10 years	55	24	43.7
	11–20 years	55	11	20.0
	21–30 years	55	7	12.8
	31–40 years	55	8	14.5
	Over 40 years	55	2	3.6
Experience	Yes, general interviewing	70	66	94.3
	Yes, clinical interviewing	69	62	89.9
Crisis Training	Yes	71	54	76.1
	1–5 trainings attended	40	16	40.0
	6–10 trainings attended	40	12	30.0
	≥11–26 trainings attended	40	6	15.0
Suicide Training	Yes	71	47	66.2
	1–5 trainings attended	36	21	58.3
	6–10 trainings attended	36	9	25.0
	≥11–26 trainings attended	36	1	2.8

For crisis and suicide prevention training and the number of trainings attended, at pretraining the question was phased “ever” although at follow-up the questions were phased “in the last year.” Years practicing and the number of trainings attended were analyzed qualitatively using content analysis with the resulting categories reported here.

TABLE II.

Lifetime and Last Year Exposure Among VHA Employees to Potentially Suicidal Individuals

Item	Response	Pretraining (N = 71)			1 Year Follow-Up (N = 21)
		N	Freq.	%	N	Freq.	%
Contact with Suicidal Individuals?a	Yes	71	67	94.3	21	14	66.6
Nature of Relationship	Personal	—	—	—	21	1	4.8
	Professional	—	—	—	21	12	57.1
Talked to Suicidal Individualb	Yes	67	65	97.0	14	14	100.0
Asked About Killing Themselvesc	Yes	67	64	95.5	14	13	92.9
Made Attempt to Intervened	Yes	66	66	100.0	14	14	100.0
Made a Referrale	Yes	64	60	93.7	14	13	92.9
Number of Persons Referred	1 person	62	6	9.7	13	6	46.2
	2–4 people	62	14	22.6	13	3	23.1
	4–10 people	62	16	25.8	13	4	30.8
	≥10 people	62	26	41.9	13	0	0.0
Nature of Relationship	Personal	70	17	24.3	21	1	4.8
	Professional	70	56	80.0	21	12	57.1
Know of Someone Who Has Attempted Suicide?	Yes	—	—	—	21	7	33.3
Number Of Attempters	1 person	—	—	—	9	6	66.7
	2–4 people	—	—	—	9	3	33.3
Nature of Relationship	Personal	—	—	—	21	2	9.5
	Professional	—	—	—	21	7	33.3
Know of Someone Who Has Died by Suicide?	Yes	70	53	75.7	19	2	10.5
Number of Decedents	1 person	—	—	—	3	2	66.7
	2–4 people	—	—	—	3	1	33.3
Nature of Relationship	Personal	70	30	42.8	21	1	4.8
	Professional	70	34	48.6	21	3	14.3

Personal categories computed by summing family, friend, acquaintance, and neighbor; professional categories computed by summing patient, coworker, student.

^aEver had contact with someone thought be to suicidal was significantly correlated with highest grade completed, general and clinical interviewing experience, history of attending suicide trainings, pre- and postself-efficacy, and pre- and postdeclarative knowledge at 0.01 level (2 tailed); history of attending crisis training at 0.05 level (2 tailed).

^bEver talk to any of them about your concerns was significantly correlated with highest grade completed, ask them if they were thinking of killing themselves, made an attempt to intervene, predeclarative knowledge at 0.01 level (2 tailed); race, postdeclarative knowledge at 0.05 level (2 tailed).

^cEver ask any of them if they were thinking of killing themselves was significantly correlated with highest grade completed, talk to them about your concerns, clinical intervening experience, pre- and postdeclarative knowledge at 0.01 level (2 tailed); made a referral for any of them at 0.05 level (2 tailed).

^dEver made an attempt to intervene with any of them was significantly correlated with talk to them about your concerns, ask them if they were thinking of killing themselves, made a referral for any of them, pre- and postdeclarative knowledge at 0.01 level (2 tailed); highest grade completed at 0.05 level (2 tailed).

^eEver made a referral for any of them was significantly correlated with clinical interviewing experience, made an attempt to intervene, pre- and postdeclarative knowledge at 0.01 level (2 tailed); highest grade, ask them if they were thinking of killing themselves, and post self-efficacy at 0.05 level (2 tailed).

Gatekeeper Training Evaluation

To assess the impact of the gatekeeper training program, participants’ scores from pre- to post-training and from pre-training to follow-up were compared. Given the temporal differences between the assessment points (e.g., 2 hours from pre- to post-test vs. 1 year from pre-test to follow-up), paired t-tests were used to assess differences in group means. Results indicate significant training-related gains in perceived self-efficacy immediately after training (M = 3.01; SD = 0.71) than before (M = 2.50; SD = 0.95) (t = −6.70, p < 0.001). Declarative knowledge also significantly increased at post-training (M = 12.01; SD = 1.83) than at pretraining (M = 11.06; SD = 1.82) (t = −4.81, p < 0.001). Using Cohen’s d, the effect size for self-efficacy at post-training was 0.6 and for declarative knowledge, the effect size was 0.5 indicating a medium effect size for both variables. At 1-year follow-up, perceived self-efficacy was significantly higher 1 year after training (M = 3.01; SD = 0.87) than at pretraining (M = 2.50; SD = 1.05) (t = −5.64, p < 0.001). Declarative knowledge also significantly increased at 1 year after training (M = 11.45; SD = 1.23) than at pretraining (M = 11.00; SD = 2.00) (t = −1.18, non significant). Using Cohen’s d, the effect size for self-efficacy from pretraining to 1-year follow-up was 0.5 indicating a medium effect size where as declarative knowledge had a small effect size of 0.3.

The participants rated the training highly immediately following the training (data not shown), indicating it was valuable (96%), satisfactory (96%), recommendable (96%), comfortable (86%), and the appropriate length (89%). In addition, 96% could think of other community groups that could benefit from attending the training and 85% reported being more aware of risk factors for suicide. At 1-year follow-up, respondents rated the training a bit lower, indicating it was still valuable (86%), recommendable (81%), and comfortable (76%), with a significant χ² difference noted only for recommendable (p = 0.04) and other groups that could benefit from training (76%; p = 0.01). Although 67% remained more aware of risk factors for suicide and 52% reflected on what they had learned over the last year, far fewer reported feeling more connected because of training to others in their workplace (19%) or in their local community (14%).

At follow-up, participants were asked if they perceived themselves to have taken on a gatekeeper role, with 76% (n = 16/21) reporting they perceived themselves to be acting more like a gatekeeper at work. With regard to their perception of performing a gatekeeper role in their community, far fewer at 45% (n = 9/20) responded affirmatively. Following these 2 questions, participants provided the number of individuals they referred over the last year from attending the community gatekeeper training. Out of 19 respondents providing this data at 1-year follow-up, only 13 reported making at least one or more referrals (68%), with the number of referrals ranging from 0 to 10. The mean number of referrals was 2.53 (SD = 3.17), yet among those participants who did make a referral, the most frequent number was one (21.1% n = 4/19).

Table III presents data on the diffusion of training-related information to individuals in their personal and professional networks while Table IV presents data on the awareness of suicide prevention resources at the work, local, state, and national levels.

TABLE III.

Participants Diffusion of Training-Related Information at 1-Year Follow-Up

Category	Item	N	Freq.	%
Discussed the Training with Anyone?		21	18	85.7
Personal	Your spouse/significant other/partner	21	5	23.8
	Your child or children	21	4	19.0
	Other family member or relative	21	5	23.8
	Friend (noncoworker)	21	4	19.0
Professional	Coworker(s) who attended the training	21	16	76.2
	Coworker(s) who did not attend the training	21	10	47.6
	Student/intern	21	5	23.8
Categories of Discussion Topics
	Content of training	11	7	63.6
	Recommendation/inform others about training	11	2	18.2
	Personal reaction	11	1	9.1
	Others	11	1	9.1
Show the QPR Booklet/Card to Anyone?		21	10	47.6
Personal	Your spouse/significant other/partner	21	1	4.8
	Your child or children	21	1	4.8
	Other family member or relative	21	1	4.8
	Friend (noncoworker)	21	2	9.5
Professional	Coworker(s) who attended the training	21	5	23.8
	Coworker(s) who did not attend the training	21	8	38.1
	Student/Intern	21	1	4.8
Suggest Someone Who May Benefit from Attending Training?		21	8	38.1
Personal	Friend (noncoworker)	21	1	4.8
Professional	Coworker(s) who attended the training	21	2	9.5
	Coworker(s) who did not attend the training	21	7	33.3
	Student/intern	21	1	4.8
Categories of Discussion Topics
	Benefit/value of training to self/other community	4	2	50.0
	Content or educational activity	4	1	25.0
	Other	4	1	25.0

Response options instructed to check all that apply. Categories were coded by two independent raters who reviewed the narrative response to the fill-in-the-blank response option (e.g., If yes, briefly, what was the focus of your conversation?). The frequency reflects the number of times the category was observed for each item.

TABLE IV.

Awareness of Suicide Prevention Resources Pretraining and at 1-Year Follow Up

Location	Prevention Efforts	Pre-Training (N = 71)			1 Year Follow-Up (N = 21)			McNemar Test
		N	Freq.	%	N	Freq.	%	(2 sided)
Work	Aware of any efforts in department/organization	69	30	43.5	21	20	95.2	0.004*
Local	Aware of any community efforts	69	20	29.0	20	8	40.0	0.289
State	Aware of any state efforts	69	19	27.5	21	1	4.8	0.063
National	Aware of any national efforts	69	8	11.6	19	12	63.2	0.002*

Note: Dichotomous response options (yes or no) for each item. Frequency is the number of yes responses.

^*p < 0.01

Results from the qualitative coding of the types of suicide prevention programs and services that participants were aware of at each level included hotlines or crisis lines, in-service or on-the-job trainings, workshops/seminars/classes, suicide prevention activities at specific local agencies or schools, and knowledge of resources affiliated with the VA, VHA, or VA medical centers. Across all codes assigned to text provided by participants at pre, post and follow-up, awareness of specific workplace and national resources dominated the responses over local and state programs or services. However, because of the VHA’s heightened national response to suicide in 2006, a new category called “New VA Initiatives” was added to the coding for the follow-up survey. This category included the knowledge of the suicide prevention coordinators and the new national crisis hotline publicized to include services specifically for the veteran population. This category, rated as the highest work place resource known by 26.9% of participants was the second highest known resource at national levels (33.3%).

Behavioral Rehearsal Practice Session

Results from the scripted practice session using a standardized peer observational checklist (data not shown) indicated that nearly 30% (n = 21/71) of the sample documented that they used the role-play checklist in their small group activity. Of the actions recorded, 21.1% (n = 15/71) of participants appropriately questioned their peer who was acting from a standardized script focused on suicide prevention with returning veterans, 19.7% (n = 14/71) and persuaded their peer to get help, with 16.9% (n = 12/71) referring their peer to get help. From the Role Play Acceptability scale, participants also reported that the practice session was acceptable overall with proportions at pretraining ranging from 56.5% to 74.5% and at follow up the acceptability ranged from 44.4% to 77.8%.

Educational Needs Assessment

The education needs assessment questions (see Table V) indicated that nearly 90% of the participants were willing to learn more about suicide after their first training, with 88% willing to spend more than 1 hour in future training activities on the topic of suicide, and almost half of the participants preferring face-to-face training vs. online.

TABLE V.

Educational Needs Assessment Assessed After Community Gatekeeper Training (N = 71)

Category	Item	% (N)	Mean (SD)
Interest	Willingness to learn more about suicide	88.7 (55)
Length of Time	0 hours	11.9 (8)
	1–4 hours	64.3 (43)
	8 hours	10.4 (7)
	More than 8 hours	13.4 (9)
Format	A live professional seminar or conference	39.1 (27)
	Multimedia online tutorial	5.8 (4)
	Multimedia online and in person meeting	11.6 (8)
	Read-only print text	2.9 (2)
	Employee in-service or work-related training	40.6 (8)
Topic	Suicide Risk Detection	92.5 (63)	4.25 (0.80)
	Initial suicide risk assessment	89.5 (60)	4.18 (0.82)
	Lethality assessment	86.4 (57)	4.14 (0.91)
	Clinical treatment for chronic suicidal ideation	83.6 (56)	4.13 (1.00)
	Clinical treatment for suicide attempt survivors	81.8 (54)	4.06 (1.05)
If offered as an on the job training, I would be interested in:
Becoming an instructor of QPR		47.8 (32)	3.37 (1.28)
Teaching QPR skills to my clients		70.6 (48)	3.85 (1.11)
Teaching QPR skills to family/friends of my clients		76.4 (52)	3.91 (0.99)
Teaching QPR skills as an outreach presentation		68.7 (46)	3.75 (1.12)

Willingness-to-learn-more scale ranged from strongly agree to strongly disagree on a 5-point Likert scale.

DISCUSSION

The aims of this study were to report on the longitudinal training outcomes from participating in a suicide prevention gatekeeper training program 1 year earlier and to describe the perceived suicide prevention training needs and educational preferences of VHA employees. Results suggest that clinical and frontline staff have had in their lifetime (94%), and continue to have over the past year (67%), contact with suicidal individuals, both in their professional duties and in their personal lives. Although 76% of participants had experienced the death of someone by suicide in their lifetime, in the last year, 10% knew someone who had died by suicide while 33% of participants knew someone who had attempted suicide. These results are consistent with studies on exposure to suicide emanating from personal or professional social networks.^17,41,42

Training results indicate that the brief standardized gatekeeper training program resulted in significant gains in participants’ perceived self-efficacy to identify, to intervene, and to refer a distressed individual to care at post-training, which were sustained at 1-year follow-up. However, declarative knowledge decreased over the follow-up period nearly to scores at pretraining levels, indicating that deployment of this type of factual information is more susceptible to the passage of time and may be less likely to lead to actual behavior change.^17,36 Results from the behavioral rehearsal practice session indicate that participants used the checklist to provide feedback to peers from the skills practice and that it was an acceptable and worthwhile experience.

As in other studies,¹⁸ initially over 90% of participants rated the training as valuable and would, at follow-up, recommend the training to others and could think of others who could benefit from attending. The finding that over 75% of the participants perceive themselves to be more of a gatekeeper at work is notable with most talking about and sharing the training materials with friends and colleagues that both attended and did not attend the training. The finding about taking on the role of gatekeeper at work warrants further study as a randomized trial in a school setting found that staff already engaged in communications with students in emotional distress, asked more students about suicide after participating in gatekeeper training for suicide prevention than staff who were not already engaged in communications with students.³⁷

With referrals to care, a significant aspect of intervening in a suicidal crisis, the results from the bivariate analyses indicate that actively making a referral to care was significantly related to participants who had higher levels of education, clinical interviewing experience, asking others whether they were thinking of killing themselves, making an attempt to intervene, having more factual knowledge about suicide before and immediately after training, and higher levels of self-efficacy since attending the community gatekeeper training. Although the average number of referrals was about 2.5, results indicate that participants most often only referred one person. Although other studies have operationalized referrals as the adherence to safety protocols in the 1 year following gatekeeper training,³⁷ this study only asked for the number of referrals made via a self-report survey. Although this study is one of the first to measure referrals to care as a gatekeeper training outcome with participants who work in VHA health-care settings, attrition at follow-up prohibited further multivariate analysis on characteristics that impact referral-making behavior.

Finally, the educational needs assessment revealed that nearly all participants were interested in more training on the topic of suicide and prevention, assessment, and treatment of suicidal behaviors. Given that the majority of participants were willing to spend between 1 and 8 hours in continued educational activities, future studies should focus on the amount of time and specific topics that would increase competency for job-related duties. For some staff, training in awareness, identification, and referral may be warranted, although other staff may require more advanced training in assessment, management, and documentation of care for suicidal patients. However, other than by job roles, there is limited information to inform what competencies are needed for clinical staff and whether these competencies differ for lay health workers with similar duties (e.g., outreach). Finally, given that staff were interested in the use and instruction of gatekeeper skills, more information is needed to determine what specific aspects or materials from the training are most useful to participants in their work with veterans, families, communities, and groups.

Limitations

The results from this study are limited by a relatively small sample size and selection bias of participants. The sample comprises only one segment of the VA healthcare system, that is, VA medical centers and clinics and those attendees who self-selected to participate in this workplace training. Although these facilities are staffed by a diverse group of individuals, this sample represents a well-educated, employed group that work with veterans in a healthcare setting. Although three voluntary training sessions were offered at different locations and times to maximize participation, the training groups remained relatively small. Therefore, this study is hindered in its ability to generalize beyond this group to all employees in the VHA who work in VA medical centers or community-based outpatient clinics.

In addition to the high rates of attrition at follow-up, the time interval between pre- and post-training of 1 hour and between pre-training and follow-up of 1 year precluded our ability to apply repeated measures or other more rigorous statistical methods. Limited resources also hindered our ability to conduct a quicker follow-up period. Future studies may benefit from an initial shorter follow-up period (e.g., at 3 months instead of at 12 months) or additional assessments (e.g., up to 5 years). The impact of these various follow-up periods on retention and the stability and pattern of potential changes in declarative knowledge and perceived self-efficacy over time has yet to be examined. However, given the clinical importance of early identification and referral of at-risk veterans to care, this research is critical to determine the most optimal timeframe for follow-up assessments to ensure that clinicians have time to absorb the didactic material. Current research on changes in observed gatekeeper skills immediately after participating in this training may also inform the transfer of gatekeeper training to clinical practice.^17,36

As stated previously, during the 1-year interval of this study, VHA implemented some new initiatives. Although we were able to add these topics to the coding of the qualitative data, this may have influenced the self-report data because participants may have been involved in these new suicide prevention mandates and initiatives. Also, because of the lack of a control group, results cannot be definitively attributed to the intervention. Nevertheless, these data do provide information on positive training outcomes and training preferences that could serve as a reference for future VHA suicide prevention training in the workplace.

In conclusion, this brief standardized gatekeeper training program for suicide prevention can increase the knowledge and abilities of VHA staff to engage, to identify, and to refer veterans at risk for suicide to appropriate care. Future studies using administrative data are needed to track the individuals actually referred by trained gatekeepers as well as the accuracy of the gatekeeper training to detect and to refer those at highest risk to care. In addition, more direct inquiry about educational preferences and the barriers to attending voluntary trainings is needed to inform national dissemination and implementation of suicide prevention programs conducted by local VA medical centers.