Identifying Common and Unique Barriers and Facilitators to Implementing Evidence-Based Practices for Suicide Prevention across Primary Care and Specialty Mental Health Settings

Abstract

Objective:

We identified common and unique barriers and facilitators of evidence-based suicide prevention practices across primary care practices with integrated behavioral health services and specialty mental health settings to identify generalizable strategies for enhancing future implementation efforts.

Method:

Twenty-six clinicians and practice leaders from behavioral health (n = 2) and primary care (n = 4) settings participated. Participation included a semi-structured qualitative interview on barriers and facilitators to implementing evidence-based suicide prevention practices. Within that interview, clinicians participated in a chart-stimulated recall exercise to gather additional information about decision-making regarding suicide screening. Interview guides and qualitative coding were informed by leading frameworks in implementation science and behavioral science and an integrated approach to interpreting qualitative results was used.

Results:

There were a number of similar themes associated with implementation of suicide prevention practices across settings and clinician types, such as the benefits of inter-professional collaboration and uncertainties about managing suicidality once risk was disclosed. Clinicians also highlighted barriers unique to their settings. For primary care settings, time constraints and competing demands were consistently described as barriers. For specialty mental health, difficulties coordinating care with schools and other providers in the community made implementation of suicide prevention practices challenging.

Conclusion:

Findings can inform the development and testing of implementation strategies that are generalizable across primary care and specialty mental health settings, as well as those tailored for unique site needs, to enhance use of evidence-based suicide prevention practices in settings where individuals at risk for suicide are especially likely to present.

Suicide is a leading cause of death in the United States (Centers for Disease Control and Prevention, 2018). While evidence-based practices (EBPs) for suicide screening, assessment and brief intervention (SSAI) exist (e.g., Horowitz et al., 2012; Posner et al., 2011; Stanley & Brown, 2012), the frequency and quality with which they are implemented in healthcare settings is variable across studies and specific EBPs for SSAI (Davis, Rio, Bush, Beidas, & Young, 2021; Gamarra, Luciano, Gradus, & Stirman, 2015). Barriers to implementing SSAI include clinician factors such as limitations in training and knowledge, the nature of clinic workflow including time constraints, and aspects of the community surrounding the clinic such as limited availability of mental health services (Diamond et al., 2012; Lang, Uttaro, Caine, Carpinello, & Felton, 2009; LeCloux, Aguinaldo, Lanzillo, & Horowitz, 2020; Schmidt, 2016). Whether these barriers vary across different healthcare settings such as primary care (PC) with integrated behavioral health (BH) services and specialty mental health is unknown. Most research on factors that hinder or enable implementation of EBPs in routine healthcare settings focuses on a single setting or EBP (Davis & Beidas, 2021); suicide prevention is no exception. Identifying barriers to implementing SSAI that span multiple health settings is critical for determining more efficient ways to promote the uptake and sustainment of life-saving EBPs that reach as many people as possible.

Two common settings where individuals at risk for suicide often present are PC and specialty mental health. In the year prior to death by suicide, 64% of individuals visit PC and 29% attend outpatient specialty mental health appointments (Ahmedani et al., 2014). Given increasing advocacy for the expansion of integrated BH services in PC (e.g., Society for Adolescent Health and Medicine, 2020), and the growth in such services in the health systems included in the current study, we opted to focus on PC clinics with integrated BH programs. Studying PC and specialty mental health settings in tandem may yield key information about factors that enable or impede the delivery of suicide prevention EBPs to a broad spectrum of patients at risk for suicide. Including PC and BH clinicians provides a range of clinician perspectives that can be used to guide cross-site and site-specific implementation planning. Furthermore, studying settings that serve a range of patient populations, including youth and adults, is important for understanding the barriers and facilitators that may be unique to conducting SSAI EBPs with a given population.

System-level initiatives (e.g., Zero Suicide; http://zerosuicide.sprc.org) and national guidelines (e.g., Joint Commission, 2019) promote SSAI EBPs, such as routine suicide screening and safety planning (Stanley & Brown, 2012). Clinician factors, such as knowledge and experience working with suicidal patients, and setting factors, such as geographic location, have been shown to impact use of SSAI (e.g., Diamond et al., 2012). A limitation of this previous research is that it has primarily focused on single settings, EBPs, or clinician types. This may lead to the development of implementation strategies with limited generalizability. Implementation strategies are methods for overcoming barriers to incorporating a clinical practice into routine healthcare delivery (Proctor, Powell, & McMillen, 2013); identifying cross-cutting barriers to SSAI can promote the development of widely-applicable strategies that can accelerate implementation. A prominent framework for identifying implementation barriers and facilitators is the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). CFIR explicates various levels of context (e.g., culture of the healthcare organization, clinician knowledge; see Table 2 for a list of constructs and operational definitions for the current study) that can affect implementation of best practices. Identifying factors that affect implementation across multiple levels can inform the development of strategies to increase uptake of SSAI EBPs.

In addition to implementation science, behavioral economics and behavioral science theories can serve as an important lens through which to gain insight into EBP implementation (Beidas, Buttenheim, & Mandell, 2021). People use heuristics to make decisions that can sometimes introduce decision errors (Tversky & Kahneman, 1981). For example, rather than fully considering all relevant information when contemplating a decision, people tend to choose the option that requires the least amount of effort. Moreover, behavioral science models recognize the role of factors such as habit and competing demands in clinician behavior (Potthoff et al., 2019). Merging approaches from implementation science, behavioral economics and behavioral science in the current study may afford a unique and in-depth understanding of the many factors that facilitate or impede engagement in SSAI EBPs in PC and specialty mental health settings. These factors may range from clinician decision-making to clinic workflows. Thus, blending principles from implementation science, behavioral science, and behavioral economics may aid in uncovering novel targets for increasing SSAI EBP implementation across settings, which may ultimately reduce the number of lives lost to suicide.

The goals of the current study were to: a) identify factors that may facilitate or limit implementation of SSAI EBPs in two settings where clinicians are likely to interact with patients at risk for suicide: PC with integrated BH services and specialty mental health; and b) understand the ways in which those factors differ across these settings.

Method

Settings

This study was conducted in partnership with a convenience sample of four PC clinics with integrated BH services (two pediatric clinics, one internal medicine practice, and one family medicine site) and two specialty mental health programs. SSAI EBPs, including the Safety Planning Intervention (Stanley & Brown, 2012), were either mandated or recommended in each setting, though protocols varied across sites (see Davis et al., 2020).

Participants

Across sites, 26 clinicians participated: four practice leaders, eight PC clinicians, six BH clinicians based in PC, and eight BH clinicians in the specialty mental health programs. BH clinicians included psychologists, psychiatrists, social workers, and case managers. Clinic leadership or colleagues nominated clinicians with experience engaging in suicide prevention practices. Nominated clinicians were then invited to participate in the study. See Table 1.

Table 1.

Clinician Demographic Characteristics

Variable	Mean (SD)
Age	38.96 (8.72)
Years of experience	10.84 (7.25)
Average number of weekly patient encounters	28.16 (22.30)

Note. Missing data were present for number of weekly encounters (n = 1). Detailed data on race, ethnicity, and gender of patients are omitted to protect participants’ confidentiality given the small sample sizes for certain groups (Morse, 2008). However, the majority of clinicians identified as female (77%), White (68%), and Non-Hispanic/Non-Latinx (83%).

Procedures

All participants provided informed consent before engaging in study activities. The relevant Institutional Review Boards (IRBs) approved this study. Members of the research team administered a semi-structured, qualitative interview and a brief demographics questionnaire. Clinicians completed a chart-stimulated recall exercise to provide additional information on their use of SSAI procedures. Clinicians were compensated $50.

Measures

Demographic Questionnaire

Participants reported on the following demographic characteristics: age, gender, race, ethnicity, job title, approximate number of patient encounters per week, and years of clinical experience.

Qualitative Interview

Qualitative interview guides were adapted from a theory-guided interview created by Potthoff and colleagues (2019). Specifically, our interview was designed to explore factors that make it easier or harder to engage in SSAI, such as competing demands and clinician motivation.

Interviews were recorded and transcribed. We used NVivo 12 to assist with analysis. We used an integrated approach (Bradley, Curry, & Devers, 2007) for codebook development; we applied a priori codes that were informed by theories and frameworks from implementation science and behavioral science (Damschroder et al., 2009; Potthoff et al., 2019), and codes were added following a close reading of the first five transcripts (i.e., inductive coding). See Table 2 for brief definitions of major themes; see the Appendix for the interview guide.

Table 2.

Qualitative Interview Themes and Illustrative Quotations

Code	Prior Work Guiding This Code	Definition	Example Quotations
Inner Setting	CFIR (Damschroder et al., 2009); Potthoff et al. (2019)	Features of the context (structural, political, and cultural) in which screening, assessment, and intervention for suicide risk takes place, including factors such as organization size, clinic workflow and current practice protocols and procedures.	PC Clinician: I think that we only have a limited amount of time in our visits and everyone thinks we should screen for something. The more things people tell us to ask our patients about, the less time we’re going to have to actually do anything. So, while I certainly think it would be great to be able to screen everyone, it just may not be practical, and so screening those for whom we think their risk is not low is probably the population I’d target.
			SMH Leader: …If somebody answered yes to a certain number of questions, [we should] have a clear policy on what we’re supposed to be doing, because we don’t.
Outer Setting	CFIR (Damschroder et al., 2009)	The economic, political, and social context surrounding the organization. This includes factors such as external policies and incentives, opportunities for linkages to outside care, and the population served by the clinic.	PC Clinician: What do you do when you find that people have suicidality and there are [expletive] to non-existent resources…? What do you do with that information? Is it dangerous having information that you can’t act on?
			BH Clinician in PC: …We also talk about resources. We want to make sure that the patient is aware of all of the resources that might be available-- if it is a Crisis Response Center, if it is the emergency room, if it’s 911, if it’s the suicide hotline, anything like that. I want to make sure that the patient is aware of all of the different resources and I want to make sure that the patient is connected to the most appropriate services based on insurance, based on needs, and based on location as well.
Intervention Characteristics	CFIR (Damschroder et al., 2009)	Perceptions of aspects of the evidence-based practices themselves (e.g., adaptability and complexity of an intervention)	BH Clinician in PC: “I think the C-SSRS, at least the way it’s set up in [the EHR] now seems pretty bulky. If it were set up in a way where you had the two leading questions and then you screen out pretty quickly, that might be better.”
			BH Clinician in SMH: I mean a safety plan allows time for realistic conversation. And so that kind of gives me an idea of how serious it is in the moment and what next steps need to happen after the safety planning because it just doesn’t stop at safety planning, I guess.
Clinician Characteristics	CFIR (Damschroder et al., 2009); Potthoff et al. (2019)	Clinician characteristics that constitute determinants of intention and implementation to engage in suicide prevention practices (e.g., clinician knowledge, comfort, and motivation)	PC Leader: I don’t know if you asked all doctors, nurse practitioners, and residents that you’d get the same answer from even two of them on who you should screen for suicidality and when.
			BH Clinician in PC: So, I’d say a lot of it is more like a gut feeling and kind of just my own experience, now, years into it—of like okay, how do we feel about this? Does mom or the parent here feel like they’re going to be on this person? Have I had a hard time reaching them for the past six months? So, I’d probably worry that if I had to call them tomorrow to check in, I can’t get a hold of them, and then I’m nervous.
Patient and Family Factors	Derived from review of current study transcripts	Patient and family factors that influence clinical decision-making with regard to suicide prevention (e.g., patient age, presenting problems, family involvement in care, relationship between the clinician and patient/family).	PC Clinician: I think it depends on people’s social situations, if they have social support, if they live alone, depends on their age, their other comorbidities—those are things that I think about when I’m screening or starting to be asking them. I think it depends on their behavior and affect a little bit. If they seem more flat and depressed or just not motivated, if they seem more impulsive, if they’ve been connected with mental health in the past, what my assumption is of their potential relationship to mental health care, I think that probably plays a big part in a lot of these things, right?
			BH Clinician in SMH: I think if you have a really good relationship with your clients, it makes it a lot easier. My one client has tried to commit suicide, I’ve worked with her since I started here, which was like [REDACTED] years ago. She's the longest client on my caseload, so, I feel really comfortable with her. But like I said, I'm still anxious because I’m fearful that one day, I’m going to get the call that she did…
Clinician Recommendations for Implementing Best Practices	Derived from review of current study transcripts	Suggestions for improving suicide screening, assessment, and intervention procedures in the practice.	BH Clinician in PC: They do flu clinics where that’s literally all you do is come in for your flu shot. We need something like that that’s safety check-ins. It could be fun stuff too like how do we drive safely and buckle our seatbelts for little ones. It could be anything. We could do safety topics, or something. People come in, you get a little informational piece, you get a suicide screener or safety assessment. And, then it also normalizes it. And, it’s not like, "oh, we only talk about suicide when you’re depressed.” It would be like we do it all the time, because we ought to. So, flu clinics, but for our brains.
			Leader in SMH: I think we need to decide as a leadership group the decision tree stuff. Not to say that it would be a one size fits all. Like if this is indicated “yes” then it triggers this. If indicated “yes” on this, this triggers this. That might just be that, even on the Columbia or some other thing that says “I consulted with my supervisor or the attending psychiatrist on this case because of this.” People do put that in their progress notes. We have trained people to say that. I think that we need to have some criteria for helping people make that decision. And again, I think there could be individual cases where we might not do the thing but it still should be an ongoing conversation/clinical decision. It doesn’t mean that it has to happen. If somebody answered yes to a certain number of questions, [we should] have a clear policy on what we’re supposed to be doing because we don’t.
			BH Clinician in SMH: I think that sometimes it’s helpful for the teenagers to have texting too. They had a new thing come out- it's called the [name redacted] app, which I really like, but I think that it’s in the very beginnings. But, I wonder whether or not we’re allowed to add that as something else into our practice. Would we see really helpful things?

Notes. BH = Behavioral Health. PC = Primary Care. SMH = Specialty Mental Health.

Two members of the research team independently coded the transcripts. Coders started with a sample of three transcripts and compared their coding to assess the reliability and robustness of the coding scheme. Coders resolved disagreements in coding through discussion and the codebook was refined and applied to all transcripts. Twenty percent of transcripts were double-coded and reliability was excellent (κ = .85).

Chart-Stimulated Recall

Following the semi-structured interview questions, the interviewer reviewed the clinician’s cases with them for a recent, specified workday to further understand their decision-making regarding suicide prevention procedures. Chart-stimulated recall involves brief, structured interviews when the clinician reviews patients’ charts to prompt recall of specific clinical encounters (e.g., Goulet, Jacques, Gagnon, Racette, & Sieber, 2007; Jennett & Affleck, 1998). The interviewer asked questions about suicide prevention practices for all patients the clinician saw on the specified day (e.g., “Did you conduct a screen for suicide risk?). We focused on screening because rates of positive screens are relatively low (e.g., Davis et al., 2021) and thus, given our sample size, follow up with other suicide prevention EBPs was expected to be infrequent. In addition to providing yes/no responses, clinicians were able to elaborate on their decision-making.

Results

Number of patient encounters varied by clinician type (t₍₂₀₎=5.95, p<.001) and setting (t_(14.27)=4.72, p<.001). PC clinicians reported more encounters each week than BH clinicians, and there were more encounters in PC than in specialty settings.

Qualitative Results

A number of similar themes emerged across sites, as well as findings unique to clinician type and setting. For each theme, we present cross-cutting findings first and then note specific findings by setting and clinician type when applicable.

Inner Setting

Clinicians described factors about their setting (see Table 2), that they perceived to be related to SSAI implementation.

Commonalities.

Clinicians noted that having screening and safety planning embedded in the electronic health record (EHR) makes it easier to engage in SSAI, but many expressed that EHR workflows can be challenging to navigate.

In both pediatric PC and specialty mental health, BH clinicians indicated that their organizations had embedded questions related to suicide risk in the EHR and that these items required responses to close the notes. However, some acknowledged that clinicians may document “no” for those questions without asking about suicidal thoughts or behaviors.

Most clinicians emphasized the benefits of inter-professional collaboration (e.g., between PC and BH clinicians; between psychologists and social workers). Participants highlighted that opportunities for communication with clinicians from other disciplines provided an important resource in making risk-determination decisions. The EHR was cited as a tool for information sharing within clinics and across departments in a hospital system.

Clinicians indicated that suicide prevention trainings have helped, but that they wanted more guidance for how to respond to positive screens, as well as how to stratify risk and intervene appropriately.

Specialty Mental Health.

In specialty mental health, a lack of clarity regarding protocols for implementing SSAI EBPs, including which clinicians in the organization are responsible for suicide screening and the frequency with which that screening should occur, was described as a barrier.

PC.

Clinicians typically reported that protocols were in place for screening, and that national guidelines for adult and adolescent depression screening have translated into suicide risk questions being asked as part of routinely administered depression questionnaires (Kroenke, Spitzer, & Williams, 2001). Using tablets at patient check-in facilitated screening in clinics that had them, but these also had associated barriers (e.g., interrupted internet connection). PC clinicians also commented on the utility of clearly dividing responsibilities relating to suicide prevention, such as allocating initial screening responsibilities to medical assistants and other staff members. BH clinicians were typically responsible for safety planning.

Time and space constraints were highlighted as consistent barriers in PC. Additionally, competing priorities were frequently reported in PC where screening often takes place during well-visits when PC clinicians have many other topics to cover. While some PC clinicians noted they may not always ask about mood or suicidality during sick visits given that such visits are often focused on pressing physical health needs, they noted they would ask about suicidality if indicated by clinical presentation.

Outer Setting

Commonalities.

Several aspects of communities in which practices were located and policies affecting them presented barriers to implementing SSAI EBPs. For instance, several participants across settings noted that transferring individuals with acute psychiatric needs to an emergency department or crisis response center can be difficult and stressful.

Specialty Mental Health.

Clinicians from specialty mental health programs, especially case managers, described having frequent interactions with their patients’ schools but also noted that schools varied in their ability to support patients at risk for suicide.

PC.

Both adult and pediatric PC clinicians emphasized the lack of mental health resources in underserved communities, which compounds existing inequities, and expressed that it can be challenging to find resources for their patients. Clinicians noted that families may not access existing resources because of structural barriers such as lack of insurance.

Youth-focused clinicians.

Some clinicians working with children and adolescents expressed doubt about the psychological benefits of sending a child at risk for suicide to the emergency department, particularly if it was not clear that the child would be admitted to the hospital. A small percentage of clinicians were also concerned that if they sent a child to the emergency department, and that child was not admitted, that they wasted limited resources.

Intervention Characteristics

Commonalities.

Overall, clinicians recognized the benefits of standardized measures, especially when embedded in the EHR, for gathering information on suicide risk and normalizing talking about suicidal thoughts and behaviors. Nonetheless, clinicians endorsed some difficulties with administering and scoring various measures, particularly when a measure needed to be administered via paper-and pencil. Clinicians emphasized the importance of delivery style, noting that some clinician-administered suicide risk screening and assessment tools can feel “robotic.” Across settings, several clinicians described the Columbia-Suicide Severity Rating Scale (Posner et al., 2011) as cumbersome.

With regard to safety planning, clinicians highlighted the importance of tailoring this intervention to fit a given patient. A small percentage of clinicians mistakenly believed that contracting patients for safety (asking a patient to sign an agreement not to kill themselves) protects the clinician from liability in the event of a suicide death, a practice that experts have argued against due to lack of support for its effectiveness (Rudd, Mandrusiak, & Joiner, 2006).

Clinician Characteristics

Commonalities.

Although the majority of clinicians emphasized that suicide screening is a priority, they expressed mixed opinions about universal screening. Some clinicians thought that asking about suicide opens a “can of worms” and that patients may find screening at each visit off-putting, particularly if they have no history of suicide risk.

Clinicians varied as to whether they defined screening as involving a standardized measure or not. Some clinicians indicated they combine questions about suicidality with other safety topics. While each clinician had their own routine regarding when to ask about suicidality during a visit, many indicated they do so at the beginning of the encounter. Some BH clinicians integrated safety planning strategies with strategies from other interventions aimed at promoting coping and crisis management, such as dialectical behavioral therapy.

Clinicians endorsed uncertainty and anxiety about what to do when a patient is determined to be at risk for suicide and many relied on clinical judgement, described by several clinicians as a “gut feeling,” or “vibe” to make decisions ranging from when to screen for suicide risk to whether to initiate the involuntary hospitalization process. Clinicians were uncertain how to predict whether a patient will implement their safety plan and were more confident in their clinical decision-making when a colleague provided a second opinion.

Clinicians’ described varied training experiences in managing suicide risk based on their area of specialty. For example, PC clinicians explained that much of their suicide prevention training had been experiential and accrued over time as they practiced, whereas BH clinicians cited pre-service training via courses and supervision. Even within the same specialty, training in suicide prevention varied, with some clinicians endorsing more exposure to such practices via clinical rotations and didactics than others.

A few clinicians emphasized the impact of having previously had a patient attempt suicide on their motivation to identify and address suicide risk in their current practice. A minority of clinicians noted that aspects of their personal lives, such as being a parent, also impact their decisions regarding suicide risk assessment and follow-up.

Patient and Family Factors

Commonalities.

Many clinicians indicated they were more likely to screen patients from populations they believed to be at elevated risk for suicide, such as patients identifying as LGBTQ+, teenagers, pregnant women, and individuals with elevated mood or anxiety symptoms. They were less likely to screen when they had concerns about patients and families feeling stigmatized, when the patient exhibited intellectual differences, or when there were language barriers.

Clinicians noted that establishing rapport and maintaining continuity of care facilitates patients’ willingness to disclose suicidal thoughts or behaviors. Nonetheless, some clinicians explained they may not assess suicide risk as regularly with established patients not considered at risk. Some expressed concern that certain suicide prevention practices, such as escalating to a higher level of care, may damage relationships with patients and families.

Youth-focused clinicians.

Several clinicians were hesitant to screen younger children, especially those under 12 years of age. They noted that it was easier to facilitate safety planning when patients’ families were engaged in, and supportive of, the process. They emphasized the importance of aligning with family members to keep patients safe, particularly in pediatric settings and during the safety planning process. A small percentage of clinicians also expressed discomfort with sending a child home when a family member minimizes the child’s suicide risk.

Clinician Recommendations for Implementing Best Practices

Commonalities.

Clinicians recommended clarifying policies regarding SSAI EBP implementation, including delineating how often to screen for suicide risk and protocols for following up on positive screens. Some clinicians desired additional training with experiential components such as opportunities to role-play suicide prevention practices.

Clinicians commented that workflow improvements could facilitate screening, such as adding smart sets for safety planning, embedding easily accessible resources in the EHR that could be printed in the after-visit summary, and ensuring clinicians receive notification when screening is due.

PC.

PC clinicians suggested that it would be helpful to automatically notify the BH clinician in the practice of positive screens. Other related suggestions included dedicating specific days of the week for mental health “clinics” or hosting clinics for “safety checks” focused on topics ranging from wearing seatbelts to screening for suicide risk.

Youth-focused clinicians.

Some clinicians, particularly in specialty mental health, recommended using games to facilitate difficult conversations with children. Similarly, clinicians wanted more support from schools and strategies for increasing familial involvement in care. A few clinicians recommended wider dissemination of smartphone applications for adolescents, such as those focused on safety planning.

Chart-Stimulated Recall

We triangulated qualitative interviews with a chart stimulated recall exercise. Consistent with interview responses, clinicians varied in the proportion of visits in which they implemented suicide screening procedures. BH clinicians reported screening more of their patients (M=2.71 patients screened, SD=2.97; M=4.36 total daily visits, SD=2.27) than PC clinicians (M=.50 patients screened, SD=.76; M=11.38 total daily visits, SD=4.84). Reasons for not screening included the perception that the patient was low risk, patient age (e.g., infant visits), and other presenting issues taking precedence, such as acute medical needs and food insecurity.

Discussion

Our results suggest common factors influence implementation of SSAI EBPs across different healthcare settings and patient populations. These include the EHR, which was noted to serve as both a facilitator and a barrier depending on the particular function, the importance of inter-professional collaboration, and clinician uncertainties about the best ways to address suicidal thoughts and behaviors upon disclosure. In addition to similarities in barriers and facilitators across settings, we identified several factors specific to PC with integrated BH services and specialty mental health settings and to clinician type. For example, PC clinicians reported greater competing priorities and briefer visit times, while challenges collaborating with external organizations, such as schools, were more often highlighted in specialty mental health.

The similarity of many barriers and facilitators across settings suggests the potential for generalizable implementation strategies to enhance the use of SSAI in PC with integrated BH services and specialty mental health. For instance, because collaboration with other clinicians was commonly cited as helpful in each setting, a telephonic consultation service that clinicians across settings can access to address questions about managing suicide risk may be effective; this type of service may be especially helpful in clinics without integrated BH clinicians. Additionally, interdisciplinary staff meetings and mechanisms for routing progress notes between clinicians from different disciplines may further facilitate inter-professional collaboration. To address clinician uncertainty about how to respond to a positive screen, clear systematic resource guides that provide suggestions on follow-up actions to take, depending on the level of patient risk, may facilitate evidence-based decision-making (https://cssrs.columbia.edu/wp-content/uploads/C-SSRStriageexampleguidelines.pdf). Incorporating “nudges” from behavioral economics to shift the choice architecture (i.e., the way choices are presented) and promote behavior change (Patel, Volpp, & Asch, 2018) in the EHR could also help clinicians further prioritize SSAI EBPs, such as by placing suicide screening and assessment questions at the top of progress note templates. Toolkits (e.g., Thoele, Ferren, Moffat, Keen, & Newhouse, 2020) that support implementation of a wide array of SSAI across healthcare settings, clinician types, and patient populations are currently lacking; our results suggest such toolkits would be beneficial. Additionally, barriers and facilitators across several levels of context were noted, including those related to clinicians (e.g., self-efficacy; Potthoff et al., 2019), specific EBPs, the healthcare site, and the broader community. Implementation strategies that target contextual factors spanning multiple levels are likely needed.

Our results also point to the need to, in some circumstances, tailor implementation strategies to specific settings and patient populations, and to the specific EBP for SSAI of interest. Thus, designing implementation strategies to target these unique circumstances will be important, alongside cross-cutting strategies.

We acknowledge limitations of the current study. First, the sample size was small; however, it was consistent with guidance in the literature (Guest, Bunce, & Johnson, 2006), and thematic saturation for qualitative analyses was reached. Second, our convenience sampling may have led to the inclusion of clinicians who were predominately in favor of implementing EBPs for SSAI, potentially alleviating some barriers that would be present in other samples. Thus, other sampling strategies such as random sampling or purposive sampling of both high and low performers (e.g., clinicians who screen for suicide frequently and those that do so rarely) may yield a wider range of perspectives on SSAI implementation. Third, the chart-stimulated recall data we analyzed focused on screening. Given the range of SSAI EBPs that may be deployed in PC and specialty mental health settings, additional details on clinician decision-making for other SSAI practices (e.g., referral to external services), will be beneficial to gather in future studies. Fourth, although the inclusion of multiple healthcare sites was a strength of the current study, these sites were all located in the same city and the PC practices all had integrated BH services; the clinics included in this work do not represent all of the types of healthcare settings in which SSAI are meant to be implemented. It should not be assumed that findings from the current study extend to PC practices without integrated BH services; additional barriers to SSAI implementation likely exist in PC settings without integrated BH services. It will be important for future work to include large and diverse samples in other geographic regions and to explore if the themes identified in the current study are generalizable across other settings and populations.

Overall, our results yielded information on key overlapping and unique barriers and facilitators to SSAI implementation in multiple healthcare settings. These findings can inform the development and testing of cross-cutting implementation strategies that can be deployed in settings that commonly serve individuals at risk for suicide. Identifying common elements of effective implementation processes (Engell et al., 2020) has the potential to increase the efficiency of implementation and narrow the research-to-practice gap. It will be important to test such strategies in sites that are geographically varied and diverse in the populations they serve. This is especially important given the elevated suicide risk documented among traditionally underserved populations, such as Black adolescents (Shain, 2019) and indigenous communities (Suicide Prevention Resource Center, 2013). Future comparative effectiveness studies that compare cross-setting implementation supports to tailored strategies are needed to understand the added value and incremental costs associated with tailoring. Additionally, behavioral observations will be important for uncovering barriers and facilitators that may not have been apparent in the current data. Continued efforts to determine ways to accelerate uptake of SSAI best practices into routine care are critical for addressing the personal and public health ramifications of rising suicide rates.

Highlights.

• We examined barriers and facilitators to suicide prevention across health settings.

• Common and unique barriers and facilitators across healthcare settings emerged.

• Findings can enhance suicide prevention implementation across healthcare settings.

Biographies

Dr. Molly Davis was a postdoctoral fellow at the Penn Center for Mental Health at the University of Pennsylvania at the time this research was conducted. She is now a licensed clinical psychologist at the Children’s Hospital of Philadelphia. Her work focuses on the identification and prevention of internalizing symptoms and related concerns (e.g., suicide) in community settings.

Dr. Jennifer Siegel was a medical student in the Perelman School of Medicine at the University of Pennsylvania at the time this research was conducted. She is now a psychiatry resident at the University of Southern California.

Dr. Emily Becker-Haimes is an Assistant Professor at the Penn Center for Mental Health and the clinical director of the Pediatric Anxiety Treatment Center at Hall Mercer (PATCH). Her clinical and research interests are in improving the quality of youth mental health services and ensuring that youth seeking treatment receive evidence-based care. Dr. Becker-Haimes is an implementation scientist and clinical psychologist whose work is dedicated to improving mental health service quality in specialty mental health settings for youth. She has expertise in the implementation of exposure therapy across settings and the application of exposure therapy for youth with complex comorbidities.

Dr. Shari Jager-Hyman is a licensed clinical psychologist and Assistant Professor in the Department of Psychiatry at the University of Pennsylvania Perelman School of Medicine. Dr. Jager-Hyman’s primary research interests focus on risk factors and evidence-based treatments for self-directed violence and associated disorders. Her clinical interests include cognitive behavioral interventions for suicidal thoughts and behaviors, as well as mood and anxiety disorders. Dr. Jager-Hyman also provides therapist training and consultation in suicide risk assessment and management and cognitive behavioral therapy.

Dr. Rinad Beidas is an Associate Professor of Psychiatry; Medical Ethics and Health Policy; and Medicine at the Perelman School of Medicine at the University of Pennsylvania. She is the Founding Director of the Penn Implementation Science Center at the Leonard Davis Institute (PISCE@LDI); Director of the Penn Medicine Nudge Unit; and Associate Director of the Penn Center for Health Incentives and Behavioral Economics (CHIBE). Implementation science is the study of methods to promote the systematic uptake of EBPs into routine care with the broad goal of ensuring that scientific discoveries realize their potential and improve people’s lives. Dr. Beidas’s research program is designed to improve the quality of health and mental health services through implementation science. To conduct this work, she collaborates closely with key stakeholders, including patients, clinicians, health system leaders, payers, and policy-makers, to develop natural laboratories in which to answer questions of interest. Broadly, her work entails three primary foci that draw upon the methods of implementation science: (a) understanding the context in which individuals will implement EBPs, (b) developing implementation approaches that target the factors that may accelerate or hinder implementation, and (c) conducting pragmatic trials to test these implementation approaches.

Dr. Jami Young is an Associate Chair of Research in the Department of Child and Adolescent Psychiatry and Behavioral Sciences at the Children’s Hospital of Philadelphia (CHOP), a faculty member of PolicyLab at CHOP, and a Professor of Psychiatry at the Perelman School of Medicine at the University of Pennsylvania. Dr. Young has expertise in psychosocial interventions for preventing and treating adolescent depression. Her work aims to decrease the incidence of adolescent depression and increase children’s access to evidence-based assessment, prevention, and treatment of depression and other behavioral health conditions.

Katherine Wislocki was a clinical research coordinator at the Penn Center for Mental Health at the University of Pennsylvania at the time this research was conducted. She is now a psychology graduate student at the University of California, Irvine.

Anne Futterer is a research coordinator at the Penn Center for Mental Health at the University of Pennsylvania.

Dr. Jennifer Mautone is Senior Clinical Director of Behavioral Health Integration and Research in Primary Care at Children’s Hospital of Philadelphia and an Assistant Professor of School Psychology in Psychiatry at Perelman School of Medicine at University of Pennsylvania. Dr. Mautone’s research and clinical emphasis is on increasing access to high quality, culturally competent behavioral health care, primarily through integration of services into primary care practices and schools. She has particular expertise in family-school-health system collaboration.

Dr. Alison Buttenheim is an Associate Professor of Nursing and Health Policy and Scientific Director of Penn’s Center for Health Incentives and Behavioral Economics. Her research addresses persistent behavior change challenges in public and global health. Using the techniques and frameworks of behavioral economics, Dr. Buttenheim designs, trials, and scales innovative interventions in the areas of vaccine acceptance, mental health, cancer prevention, and HIV prevention.

Dr. David Mandell is the Kenneth E. Appel Professor of Psychiatry at the University of Pennsylvania Perelman School of Medicine. He is trained as a psychiatric epidemiologist and mental health services researcher. The goal of his research is to improve the quality of care individuals with psychiatric and developmental disabilities receive in their communities, with a particular focus on people with autism. This research is of two types. The first examines, at the state and national level, the effects of different strategies to organize, finance and deliver services on service use patterns and outcomes. The second consists of experimental studies designed to determine the best strategies to successfully implement proven-efficacious practices in community settings.

Darby Marx is a medical student at Weill Cornell Medical College.

Courtney Benjamin Wolk is an Assistant Professor at the Penn Center for Mental Health in the Perelman School of Medicine at the University of Pennsylvania. The long-term goal of her research is to develop and evaluate strategies to promote the uptake of evidence-based care into routine practice, with the ultimate goal of improving the effectiveness of mental health services for children and adults in non-specialty mental health settings. Clinically, Dr. Wolk’s expertise is in the cognitive-behavioral treatment of anxiety disorders in youth and she is licensed as a clinical psychologist in Pennsylvania. She has experience training community and school-based therapists to conduct CBT for a variety of presenting problems and populations.

Appendix: Interview Guide

*This is the interview guide for primary care clinicians. The guide was modified slightly for clinic leaders as well as behavioral health clinicians.

Adapted from the “Theory Informed Topic Guide” qualitative interview used in: Potthoff, Presseau, Sniehotta, Breckons, Rylance, & Avery (2019)

Introduction

1. 1. Introduce researcher and purpose of the study

   We are interested in learning more about your experience working with individuals at risk for suicide. The goal of this study is to learn more about your perceptions related to suicide prevention practices and what makes using them easier or harder. We hope to use the information you share with us to identify strategies to make it easier for providers like you to use effective practices for suicide prevention with individuals at risk for suicide.

2. 2. Obtain consent to proceed and to record the conversation

3. 3. Remind interviewee that all information remains confidential, and that they are free to stop the interview and withdraw at any time.

Part A (Ask of ALL providers)

I’d like to start by learning more about your work and your experience with suicide risk and prevention.

1. General question: Tell me about what practices you view as being within your professional role with respect to suicide prevention. Also, please walk me through the roles others in your clinic/practice have with respect to suicide prevention.

Behaviour (Screening and Assessment)

1. Do you directly screen patients/clients for suicide risk in your practice?

If yes:

1. How do you define screening for suicide? What tools, if any, are used to screen for suicide in your practice?

2. With what percentage of individuals do you administer standardized screening tools to assess for suicide risk? Which tool(s)? How do you determine when to use a standardized screening tool? At what point in encounter do you screen?

If no:

1. Does someone else in your practice/clinic administer a standardized screening tool to assess suicide risk? If so, do you routinely look at the completed screen?

   1. Regardless of whether provider does formal screening: Tell me about the strategies you use if an individual reports recent or current suicidal ideation.

Probe:

• What, if any, follow-up discussion (e.g., risk assessment) or action (e.g., warm hand off) takes place

• Criteria for determining level of risk/what constitutes a positive screen

  1. Probe for decisions related to escalation of care /referral to other programs (especially for primary care clinicians)

• How do you use this information to guide treatment planning? (Note: if interviewee begins discussing safety planning, let them know we will ask more questions about this soon)

• How is the information shared and who is it shared with (e.g., psychiatrist, case managers etc.)?

The following questions should only be asked if the provider states that they engage in formal screening. If the provider does not engage in screening, but does sometimes engage with individuals reporting suicide risk, skip to questions related to intervention. If provider states they have no involvement in suicide risk management, skip to Additional questions section.

Part B: Screening Questions (Ask only of providers who report engaging in suicide risk screening)

Outcome expectancy

1. Some have recommended that clinicians use a standardized tool to screen for suicide risk with every patient. What do you think of this recommendation?

   1. Probe – what about every encounter? What about using a standardized tool?
   2. Probes if they do not agree with the every patient/every encounter recommendation –What do you think would be a feasible frequency for screening? Also, which patients do you think should be screened at that frequency (when individual expresses hopelessness or history of SI, is patient age a factor)?
   3. Probe – under what circumstances would screening for suicide risk with every patient at every encounter make sense to you?

Intentions

1. How motivated are you to screen individuals for suicide risk in every clinical encounter?

   1. When are you most motivated?
   2. When are you least motivated?

Action Plan

1. Do you have a specific plan for when, where, and how you screen for suicide risk?

   1. Please describe your plan for me

Self-Efficacy

1. What makes it difficult to screen for suicide risk?

   1. Have prompts ready in case they aren’t sure
   2. E.g., running low on time, suicidal ideation is a sensitive topic, uncertain how to handle positive screens
2. How confident are you in your ability to screen individuals for suicide risk from 0 to 10, where 0 is absolutely no confidence in your ability to screen for suicide risk and 10 is complete confidence?

   1. If not a 10 confident – What do you think it would take to make you more confident to do so?
   2. If a 10 confident – What helped you to develop the confidence to do so?

3. How much anxiety do you experience on average about screening individuals at high risk for suicide on a scale of 0 (none) to 10 (extreme anxiety)?

4. What makes it easier to engage in screening for suicide risk?

   1. Probe for: Client factors, organizational factors (e.g., directives from leadership)

Coping Planning

1. When you have encountered barriers to screening for suicide risk (if possible, cite examples from what interviewee has already shared) do you have a plan or strategies that help you engage in screening in light of these barriers?

Automaticity

1. What makes you think to engage in screening for suicide risk? (if not clear: Was it something you did? Something the client said? Prompt in the EHR?)

   1. Do you only engage in screening when prompted like this or at other times too?
2. Do you see screening for suicide risk as part of your regular routine?

   1. If so, what supported that?
   2. If not, what might help it to be more routinely done?

Competing Demands

1. On a scale of 1 to 10, where 10 is the highest priority clinical activity and 1 is the lowest priority, where does suicide screening fall for you compared to the other activities you engage in during a clinical encounter (on average)

   1. If less than 10: what types of activities might receive higher priority?

Part C: Intervention Questions (Ask only of providers who engage in follow up intervention/safety planning)

If continuing interview from screening questions: Thank you for answering my questions related to suicide screening.

Now I want to learn more about what you do when someone screens positive as at risk for suicide.

1. If not already answered above in Part A: Do you ever engage in safety planning with the patient/client or their family when there is suicide risk?

   1. If yes: There are several different kinds of safety planning interventions that can be used to manage suicide risk. Tell me about how you do safety planning
   2. If no: Skip to Part D: Additional Questions
2. How motivated are you to engage in safety planning with every individual you work with who is experiencing suicidal ideation?

   1. When are you most motivated
   2. When are you least motivated
3. Do you have a specific plan for when, where, and how you engage in safety planning for individuals at risk for suicide?

   1. If so, how did you develop this plan?
4. How confident are you in your ability to engage in safety planning with individuals at risk for suicide from 0 to 10, where 0 is absolutely no confidence in your ability to engage in safety planning and 10 is complete confidence?

   1. If not 10 confident – What do you think it would take to make you more confident to do so?
   2. If 10 confident – What helped you to develop the confidence to do so?

5. How much anxiety do you experience on average about engaging in safety planning with individuals at high risk for suicide on a scale of 0 (none) to 10 (extremely)?

6. What makes it easier to engage in safety planning?

   1. Probe for: Client factors, organizational factors

7. When you have encountered barriers to engaging in safety planning (if possible, cite examples from what interviewee has already shared) do you have a plan or strategies that help you engage in screening in light of these barriers?

Part D: Additional Questions (Ask of All Providers)

1. What other thoughts or suggestions do you have with respect to suicide prevention in your setting?

   Probe for:

   1. What about the clinical population you work with makes it easier or harder to engage in suicide prevention practices?
   2. What about your organization makes it easier or harder to engage in suicide prevention practices?

Finally, we have a few quick questions to learn a little more about your background

1. Have you ever had a patient die by suicide?

2. Have you had specific training in suicide screening, safety planning, or any other suicide-focused intervention? If so, please describe that training for me.

3. I’m also interested in hearing about any experience you’ve had with one particular suicide-specific intervention, the Safety Planning Intervention. I’m referring specifically to the “Stanley-Brown Safety Plan,” which I’ll show you and briefly describe. Show the participant a hard copy of the Safety Plan and point to each step as you describe it.

   The Safety Planning Intervention is a brief intervention designed to lower the short-term risk of suicide. It’s a 6-part plan that includes a prioritized list of coping strategies developed collaboratively between a client/patient and clinician. The first step is to identify warning signs that signal the need to use the safety plan. The next steps focus on the following coping strategies: (2) internal coping and emotion regulation strategies individuals can implement on their own; (3) social supports and social settings that serve as a source of distraction; (4) friends and family members with whom an individual can discuss their suicidal thoughts and urges; and (5) professional and emergency services contacts. The final step is to develop a plan for making the environment safe by reducing access to lethal means. The patient/client leaves session with a copy of the safety plan with the goal of using it whenever they experience one of their warning signs.

   Have you ever used the Stanley and Brown Safety Plan I just described?

Part E: Behavioral Sample: Use of Suicide Risk or Screening (Ask of All Providers)

Next, I’d like you to access the charts from your patient visits from today. Please do not show me the charts or share any identifying information about the patients. For each patient, I have a few quick questions about the encounter, your use of suicide prevention practices and why or why not you may have used them. [Interviewer reviews the following questions for each of the charts and asks for their reflections on these results].

At conclusion, ask clinician:

1. On a scale of 1-10, how representative was today of your clinical practice? ____________

2. Probe about times clinician did not screen

3. Probe about times clinician screened but did not safety plan

Thank respondent for their time and switch off recorder

Data Availability Statement

Data will be made available upon request. Requests for access to the data can be sent to the Penn ALACRITY Data Sharing Committee. This Committee is comprised of the following individuals: Rinad Beidas, PhD, David Mandell, ScD, Kevin Volpp, MD, PhD, Alison Buttenheim, PhD, MBA, Steven Marcus, PhD, and Nathaniel Williams, PhD. Requests can be sent to the Committee’s coordinator, Kelly Zentgraf at zentgraf@upenn.edu, 3535 Market Street, 3^rd Floor, Philadelphia, PA 19107, 215-746-6038.