Implementation of a New Hampshire community-initiated response to the opioid crisis: A mixed-methods process evaluation of Safe Station

Abstract

Background:

New Hampshire (NH) ranked first for fentanyl- and all opioid-related overdose deaths per capita from 2014 to 2016 and third in 2017 with no rate reduction from the previous year relative to all other states in the US. In response to the opioid crisis in NH, Manchester Fire Department (MFD), the state’s largest city fire department, launched the Safe Station program in 2016 in partnership with other community organizations. This community-based response to the crisis—described as a connection to recovery—focuses on reducing barriers to accessing resources for people with substance use and related problems. The study aim is to characterize the multi-organizational partnerships and workflow of the Safe Station model and identify key components that are engaging, effective, replicable, and sustainable.

Methods:

A mixed-methods design included: semi-structured qualitative interviews conducted with 110 stakeholders from six groups of community partners (Safe Station clients, MFD staff and leadership, and local emergency department, ambulance, and treatment partner staff); implementation and sustainability surveys (completed by MFD stakeholders); and ethnographic observations conducted at MFD. Qualitative data were content analyzed and coded using the Consolidated Framework for Implementation Research. Survey subscales were scored and evaluated to corroborate the qualitative findings.

Results:

Community partners identified key program characteristics including firefighter compassion, low-threshold access, and immediacy of service linkage. Implementation and sustainability survey data corroborate the qualitative interview and observation data in these areas. All participants agreed that community partnerships are key to the program’s success. There were mixed evaluations of the quality of communication among the organizations.

Conclusion:

Safe Station is a novel response to the opioid crisis in New Hampshire that offers immediate, non-judgmental access to services for persons with opioid use disorders requiring community-wide engagement and communication. Data convergence provides guidance to the sustainability and replicability of the program.

INTRODUCTION

Across the United States (US), rising rates of opioid overdose deaths underscore the importance of expanding programs to reduce overdose mortality (Chen et al., 2019; Scholl et al., 2018). In response to the opioid crisis, communities across the US have developed and implemented multisector interventions to reduce opioid-related harms, including overdose. A recent review identified more than 100 community-based opioid-related interventions implemented across the US and Canada since 2008 (Leece et al., 2019). These community-based interventions varied widely, using a range of theoretical frameworks to inform activities and engaging community members through a variety of strategies. While many interventions were led by public health agencies, others were led by county-level government, healthcare organizations, or community-based organizations. These interventions included an average of seven community partnerships, most frequently among healthcare agencies, law enforcement, and public health organizations. Despite existing literature describing these programs, according to the Leece et al. 2019 scoping review, studies have evaluated only four programs (Albert, 2011; Alexandridis et al., 2018; Paone et al., 2015; Public Health Institute, California, 2017). Results from these studies suggest that the implementation of community-based interventions is associated with decreased opioid prescribing, and opioid-related mortality rates.

In addition to further examining the effectiveness of these studies, additional research is needed to assess the development and implementation of community-based interventions to reduce opioid-related harms. Some studies explored barriers and facilitators to implementing these community-based programs. Common barriers included stigma, funding, limited availability of substance use treatment, local policies/legislation, and staffing challenges (Leece et al., 2019). Further evaluating the implementation of community-based programs seeking to reduce opioid-related harm is necessary to inform implementation of these programs in other communities. Aside from generally examining barriers and facilitators to successfully implementing and sustaining these community-based interventions, few studies have systematically examined how community dynamics and local context may impact implementation (McLeroy et al., 2003).

The Safe Station program in Manchester, New Hampshire (NH), US, is another example of a community-based intervention to reduce opioid-related harm. In response to the opioid crisis, the state’s largest city fire department (Manchester Fire Department [MFD]) launched SS in May 2016 with no program-specific funding. From 2014 through 2016, the state of NH ranked first for fentanyl- and all opioid-related overdose deaths per capita relative to all other states in the US (CDC, 2020). Despite this high rate of overdose deaths, NH had the lowest per capita spending on treatment for substance use disorders in New England, and among the lowest in the nation (CSAT, 2015). Compounding these treatment access issues, NH also had: the lowest rate of buprenorphine providers per capita in New England (Knudsen, 2015); no syringe services programs until 2017; and residents reported significant barriers to accessing naloxone (Bessen et al., 2019; Meier et al., 2020). With a three-fold increase in rates of naloxone administrations since 2012 (NH Information and Analysis Center, 2018), first responders were consistently responding to opioid overdoses (Saunders et al., 2019).

Safe Station—described by the MFD leadership as a connection to recovery—was born of an increased awareness of the opioid crisis in the community at large, but particularly among firefighters who were responding to escalating numbers of overdoses. In encounters with people in overdose situations, firefighters were also hearing that people could not get help finding treatment. Following the launch of Safe Station, there was a great deal of publicity. The model was featured on every news station in New Hampshire and the statewide radio station (WMUR) started a special notification to the community updating people on opioid deaths in the state and referring people in need to the program. Then-President Trump even travelled to the MFD to put a spotlight on the model given the First Lady’s interest in combatting the opioid crisis.

By design, Safe Station focuses on reducing structural barriers to accessing resources for people with substance use and related problems. Beyond personal characteristics barriers (e.g., attitudinal, motivational, belief-based), structural barriers (e.g., service availability (capacity) and insurance/policy-related barriers (funding)) stand as substantial hurdles for individuals seeking substance use disorder treatment (Priester et al., 2016). In the Safe Station model, individuals with any type of substance use problem (note: there are no additional eligibility requirements) can walk into any of the ten Manchester fire stations at any time of the day or night to seek assistance and be connected with a partnering organization that will assess their needs and help them access treatment or other services (e.g., respite housing, outpatient counseling, wrap-around and recovery support services). These treatment services can include care for acute medical conditions at a local emergency room, or a ‘warm handoff’ to local behavioral health agencies. At these behavioral health agencies, participants are assessed and referred to appropriate treatment services. Harm reduction resources, including naloxone, are available at some of these behavioral health agencies. In most addiction treatment systems, prospective patients are burdened by the need to identify and contact facilities directly, determine if they have the proper insurance to enter treatment, arrange for a physical assessment to confirm treatment eligibility, and await an appointment date or be placed on a waiting list. With Safe Station, these barriers are reduced. Since the program’s launch, approximately 8,000 individuals have been served by Safe Station, primarily from NH communities (MFD EMS Officer, personal communication, October 14, 2020).

Models of Safe Station (based on the NH model) have already been adopted by many other communities across the US (e.g., Annapolis, Maryland; Chattanooga, Tennessee; Providence, Rhode Island; Tacoma, Washington; Columbus, Ohio). The eponymous ‘Safe Station ‘ program in Columbus, Ohio has taken the added step of publishing the results of their effort to implement a similar model informed by the efforts of the MFD (Burton & Martin, 2020). In addition to knowledge about the implementation of community-based responses to the opioid crisis, information about the key program characteristics and implementation of SS may be critical to assist other communities in successfully implementing and sustaining this program. This mixed-methods study therefore aimed to systematically evaluate Safe Station using the Consolidated Framework for Implementation Research (CFIR) (Damschroder et al., 2009; described in Methods) to characterize the multi-organizational partnerships and workflow of the program and to identify key program characteristics that are engaging, effective, replicable, and sustainable.

METHODS

Design

This is a convergent, parallel mixed-methods evaluation (Hong et al., 2017; Creswell & Plano Clark, 2011) of the ongoing implementation of the MFD Safe Station model, which included onsite observations, semi-structured interviews with community partners, and quantitative implementation measurement. The Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009) was used as an organizing framework to guide the characterization of the partnerships and workflow and identification of the key program characteristics and impact of Safe Station. The CFIR is a meta-theoretical framework integrating the major constructs of published implementation science theories to help identify the factors facilitating and/or impeding program implementation and effectiveness. Since its publication, the CFIR has been cited in hundreds of published articles and reflects the evidence base of factors most likely to influence the implementation of interventions. The five overarching domains, comprised of 39 constructs hypothesized to affect program implementation, are: 1) program characteristics, or intervention features; 2) the inner setting, or the implementing organization’s features; 3) the outer setting, or the implementing organization’s external context features; 4) characteristics of individuals involved in the implementation; and 5) the implementation process itself. In this study, the CFIR was used in the development of interview guides, selection of measurement tools, and as the analytic coding framework of the ongoing implementation.

Context/Setting

Manchester is the largest city in NH and home to over 110,000 people (US Census, 2019). Located in Hillsborough County, which witnessed the highest rate of opioid overdose deaths of all NH counties, Manchester was the only city in the state to report more than 50 opioid-related overdose deaths and 500 emergency medical service administrations of the overdose reversal medication, naloxone in 2017 (New Hampshire Information and Analysis Center, 2018). As the state’s largest city and the site of convergence of many negative contributing factors, Manchester is a Northeast crux of the opioid crisis. Along with the local ambulance service and law enforcement, the MFD responds to all opioid-related medical calls in Manchester. Importantly, the MFD has 10 full-time fire stations staffed 24/7 with career firefighters who are certified emergency medical technicians (EMTs) all trained on naloxone protocols.

Participants/Community Partner Groups

Data were collected between September 2017 and May 2018. All participants provided informed consent and were compensated with $50 gift cards. Participant groups included Safe Station clients, MFD staff and leadership, emergency department (ED) staff from two local hospitals, local treatment recovery partner staff (TP), and ambulance company partner staff (AP). Clients reported use of Safe Station at least once. Varying methods were employed to recruit the different community partner groups: purposive sampling to recruit MFD staff, leadership, TP and AP staff via flyers and word-of-mouth; snowball sampling to recruit ED staff; and convenience sampling to recruit clients via online ads posted to Craigslist.com, flyers posted in community locations, and word-of-mouth. In addition to interviews, observations were conducted across 15 non-consecutive days and 5 MFD stations; stations were chosen based on number of Safe Station clients and included the four stations with the most, and one with the least clients. Research ethics board approval for the study was obtained from the Dartmouth College Committee for the Protection of Human Subjects.

Procedures

Observations:

Research team members used a checklist (based on the CFIR domains of characteristics of the intervention and inner setting) to guide ethnographic observations of the Safe Station context and workflows and recorded observations using pencil and paper. The checklist focused on the physical setting, initiation of clients into the program, staff response and decision-making, and linkages between Safe Station and local resources. Observations were conducted at all sites where MFD study participants worked (but not necessarily when study participants were on duty), on various days (weekdays and weekends) and times (day and night), and by 2–3 research team members at a time, depending on research staff availability. Ultimately, the goal of the observations was to establish the extent to which there was indeed a consistent model (fidelity) with observably reliable workflows such that it would be possible to identify key program characteristics that were engaging, effective, replicable and sustainable.

Semi-structured Interviews:

Client guides focused on how clients learned about and accessed Safe Station, motivation for entering, experiences with, evaluations and recommendations of the program. MFD staff and leadership guides focused on how Safe Station impacted their role, training/meeting/supervision expectations regarding Safe Station, experiences with, evaluations and recommendations of the program, and perspectives on replicability and sustainability. The ED, AP, and TP staff guides focused on whether/how Safe Station impacted their role, how their organizations interacted with Safe Station, experiences with, evaluations and recommendations of the program, and perspectives on replicability and sustainability. Each of the CFIR domains (see Design above) was addressed in the interview guides with staff and leadership. Client guides were framed by the lone CFIR domain of characteristics of the intervention. Interviews were audio-recorded and transcribed verbatim.

Quantitative Implementation Metrics:

Clients and MFD staff and leadership completed modified versions of the Applied Mental Health Research Group implementation measure (AMHR; Haroz et al., 2019), originally designed for laypersons delivering clinical services in low- and middle-income countries to evaluate the implementation context and outcomes of a chosen program (i.e., adoption, acceptability, appropriateness, feasibility, reach/access). Although this measure has not been validated to assess the implementation of community-based access points in the US, this tool assessed all constructs of interest within a single measure, unlike other implementation science measures which often assess only a single domain of interest. The AMHR has been validated for use in low- and middle-income countries to measure the implementation of a behavioral health intervention delivered by laypersons, often in community settings. Internal consistency reliabilities across subscales were good to excellent (α: range 0.85 – 0.91) and both construct and criterion validity were demonstrated (Haroz et al., 2019). Different versions were administered to different community partner groups based on presumed familiarity with a topic (# of items/group: 82/staff, 70/leadership, 29/clients).

The Program Sustainability Assessment Tool (PSAT; Luke et al., 2014)—a 40-item measure with high internal consistency scores across subscales (α : range 0.79 – 0.92) and demonstrated construct validity —was used to assess the program’s capacity for sustainability by identifying barriers and facilitators to sustainability across a range of organizational and contextual factors (i.e., environmental support, funding stability, partnerships, organizational capacity, program evaluation, program adaptation, communications, and strategic planning). Both MFD staff and leadership completed the PSAT. Used by more than 320 programs nationwide, the PSAT is designed to assess the sustainability of public health interventions and programs. A validation study with 252 US public health programs supported the internal consistency and reliability of the PSAT (Luke et al., 2014).

During the active study recruitment and data collection phase of this study, Safe Station (version 1.0 during the study) underwent significant changes when the initial TP closed suddenly. In response to this change, the city of Manchester held community meetings with key community partners to identify solutions to provide continued access to treatment for those coming through Safe Station. The community rallied around Safe Station to support new partnerships with other community treatment providers (version 2.0 during the study). Changes to the program included partnerships with new community organizations and modified transportation procedures for patients. Given the significant changes to the program under study, the research team modified its protocol and interviewed 12 additional clients who had used Safe Station v2.0. In addition, the research team re-interviewed, and re-administered the quantitative implementation measures to, roughly half of the MFD staff and leadership samples. The purpose was to assess the implementation, sustainability, and replicability of the program considering these changes.

Analysis Plan

To increase the rigor of the qualitative approach and ensure a high level of trustworthiness, credibility, and accuracy of the findings and conclusions, we triangulated: data collection strategies (observations, interviews, and surveys); data sources (six unique community partner groups); and investigators (six researchers collecting and analyzing data) (Lincoln & Guba, 1985; Morse, 2015; Tong et al., 2007). Prolonged field observation was conducted by more than one observer, and follow-up interviews with two community partner groups (MFD staff and leadership) following a program-halting outer setting event also contributed to the rigor.

Observations and Interviews:

Observational field notes and interview transcripts were uploaded to qualitative software (Atlas.ti, v8.2). Six research members analyzed individual community partner datasets using content analysis (Hsieh & Shannon, 2005) with the CFIR (Damschroder et al., 2009) as a coding framework. We made minor modifications to CFIR construct definitions based on language emergent in the data but did not change the meaning of any implementation outcome (see Table 1). Code lists were created for each dataset as a function of both the potential for a community partner group to comment meaningfully with respect to CFIR domains/constructs, as well as the questions posed by the respective interview guides. Observational field notes were coded according to Safe Station-specific workflow codes. For each dataset, we used a group coding process to collectively code 1–2 field notes/transcripts to develop/hone code definitions, and work towards consensus on applying codes to data. Following group coding, analysts independently coded data subsets, communicating via email or weekly in-person meetings to address code definitions or need for additional codes.

TABLE 1.

Emergent¹ CFIR Construct Descriptions and Study-specific Modifications

CFIR Domain/Construct	Description	Study-specific Modification
Program Characteristics
Relative Advantage	Community partners’ perception of the advantage of implementing the intervention vs. an alternative solution	To capture references to status quo as there were no other comparable alternatives at the time of the launch of Safe Station
Design Quality and Packaging	Perceived excellence in how the intervention is bundled, presented, and assembled	Modified to focus on the issue of accessibility. Per the CFIR guide website, “Packaging is related to how the intervention is bundled, presented, or assembled and even how accessible it is for users. When components are more easily accessible to users, it promotes use of the new procedures.”1
Evidence Strength and Quality	Community partners’ perception of the quality and validity of evidence resulting in the desired outcomes	To capture community partners’ references to their own evidence about how to measure the success (quality) of Safe Station; e.g., reduced overdoses, numbers of people accessing Safe Station, etc.
Cost	Costs of the intervention and costs associated with implementing that intervention	No modifications
Complexity	Perceived difficulty of implementation	To capture perceptions about role strain and/or incompatibility with job description
Inner Setting
Available Resources	The level of resources dedicated for implementation, including money, training, physical space, and time	Also, to capture references to volume; e.g., ‘there were 15 Safe Station [events] in one 24-hour period over the weekend,’ implying too many for staff to handle given available resources/staffing
Tension for Change	The degree to which community partners perceive the current situation as intolerable or needing change	Current situation defined as opioid overdose epidemic in NH communities
Compatibility	The degree of tangible fit of the intervention within individual norms and existing workflows	No modifications
Networks and Communications	The nature of social networks, formal and informal communication within an organization	Given the lack of formal communication, to capture references to informal messaging and communication among staff and leadership
Access to Knowledge and Information	Ease of access to digestible information and knowledge about the intervention and its incorporation	Primarily a proxy for references to training or lack thereof
Culture	Norms, values, and basic assumptions of a given organization	No modifications
Outer Setting
Client Needs and Resources	The extent to which patient needs are accurately known and prioritized by the organization	Re-named this construct ‘Client’ Needs and Resources instead of CFIR ‘Patient’ Needs and Resources
Cosmopolitanism	The degree to which an organization is networked with other external organizations	To capture references to multi-organizational partnerships, as well as to characterize the nature and communication of those partnerships
Characteristics of Individuals
Knowledge and Beliefs about Safe Station/Safe Station Clients	Individuals’ attitudes toward and value placed on the intervention as well as familiarity with facts, truths, and principles related to intervention	Also, attitudes towards people served by the intervention (Safe Station). To capture myths/facts about addiction; e.g., misunderstandings about the chronic nature of addiction and the likelihood that someone may make repeated attempts to get help
Self-Efficacy	Individual belief in their own capabilities to execute implementation	No modifications
Process
Planning	The degree to which a method of behavior and tasks for implementation are developed in advance	No modifications

CFIR = The Consolidated Framework for Implementation Research

¹Emergent refers to CFIR constructs that were referenced/aligned with what was said in the interviews. Twenty-three of the CFIR constructs did not emerge during interviews and thus are not included here.

² https://cfirguide.org/constructs/design-quality-packaging

Once data for a community partner group were coded, team pairs conducted subtheme analyses of specific CFIR domains and other surfaced codes. Both members of the pair would then develop an analytic summary for discussion/comparison. As the team completed subtheme analyses for all datasets, we used a consensus-based approach to evaluate whether surfaced CFIR constructs were relevant to the research aims and/or deemed important to the implementation narrative based on the frequency with which they were mentioned, and the degree of importance articulated by participants, researchers, or both. We then determined whether a construct exerted a negative, positive, or neutral influence on implementation from the perspective of the different community partner groups and used these evaluations to populate a valence matrix. The matrix facilitated cross-group barrier and facilitator pattern analysis and enabled simultaneous viewing of a large volume of data so that we could make between-group comparisons, identifying themes and negative cases (Tong, Sainsbury & Craig, 2007).

Quantitative Implementation Metrics:

Subscale scores for the AMHR measure were calculated by computing the mean response across all subscale items. For subscales with missing items, the mean response was calculated based on the mean response across all non-missing subscale items. To calculate subscale and total scores for the PSAT, the mean responses were calculated, excluding any ‘not applicable’ item responses. The PSAT total score was calculated by averaging all individual domain scores. All analyses were conducted using SPSS, v.24 (IBM, 2016).

RESULTS

Participant Characteristics

Safe Station clients (n = 49) were predominately low-income, non-Hispanic, young, white, male adults with no college degree. More than half lacked independent housing and employment. MFD staff and leadership (n = 29) were mostly non-Hispanic, white males in their mid-forties, on average. Most had a college education with an average of 20 years in the field, as well as significant experience responding to overdoses. See Table 2 for additional demographic details on clients and MFD staff and leadership, as well as local ED (n = 6), AP (n = 4), and recovery partner staff (n = 6).

TABLE 2.

Participant Demographic Characteristics

	Safe Station Clients (n=49) n (%)	MFD Staff and Leadership* (n=29) n (%)	ED Staff (n=6) n (%)	TP Staff (n=6) n (%)	AP Staff (n=4) n (%)
Age m [sd]	34.9 [8.9]	44.0 [9.7]	44.7 [6.2]	36.5 [6.1]	46.0 [9.8]
Gender
Male	32 (65.3%)	29 (100.0%)	3 (50.0%)	2 (33.3%)	3 (75.0%)
Female	15 (30.6%)	0 (0.0%)	3 (50.0%)	4 (66.7%)	1 (25.0%)
Non-binary/Prefer not to answer	2 (4.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Race
American Indian	1 (2.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Black/African American	1 (2.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
White	42 (85.7%)	28 (96.6%)	6 (100.0%)	6 (100.0%)	4 (100.0%)
Other	3 (6.1%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Multiracial	1 (2.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Not reported	1 (2.0%)	1 (3.4%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Education
Less than High School	11 (22.4%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
High School/GED	20 (40.8%)	4 (13.8%)	0 (0.0%)	1 (16.7%)	0 (0.0%)
Some College	14 (28.6%)	3 (10.3%)	0 (0.0%)	1 (16.7%)	1 (25.0%)
Associate’s (2 years)	1 (2.0%)	11 (37.9%)	0 (0.0%)	3 (50.0%)	1 (25.0%)
Bachelor’s (4 years)	2 (4.1%)	8 (27.6%)	4 (66.7%)	0 (0.0%)	2 (50.0%)
Other	1 (2.0%)	0 (0.0%)	2 (33.4%)	1 (16.7%)	0 (0.0%)
Employment Status
Working full-time	6 (16.2%)	--	--	--	--
Working part-time	12 (32.4%)	--	--	--	--
Unemployed	14 (37.8%)	--	--	--	--
Disabled	5 (13.5%)	--	--	--	--
Household Income
<25,000	30 (81.1%)	--	--	--	--
25,000–35,000	3 (8.1%)	--	--	--	--
35,001–45,000	3 (8.1%)	--	--	--	--
>45,000	0 (0.0%)	--	--	--	--
Prefer No Answer	1 (2.7%)	--	--	--	--
Housing Status
Own Home	0 (0.0%)	--	--	--	--
Rent	21 (42.9%)	--	--	--	--
Live with Someone	6 (12.2%)	--	--	--	--
Residential Program, Sober House/Group Home	9 (18.4%)
Shelter	6 (12.2%)	--	--	--	--
Homeless	7 (14.3%)	--	--	--	--
Safe Station Accessed ** , or Where Employed
Station 1 (Central)	45 (93.8%)	12 (41.4%)
Station 2–10	7 (14.3%)	17 (58.6%)	--	--	--
Number of Experiences with Safe Station, m [sd]	1.90 [2.51]
Zip Code When Used Safe Station, m [sd]	39 (79.60%)	--	--	--	--
Manchester
Months in Field, m [sd]	--	248.6 [117.2]	264.0 [53.1]	51.3 [29.6]	255.0 [128.3]
Overdoses Responded to, m [sd]	--	218 [395]	588 [384]	6 [5]	83 [29]

AP = Ambulance Partner, ED = emergency department, m = mean, MFD = Manchester Fire Department, sd = standard deviation, TP = Treatment Partner, -- = data not collected,

^*= we grouped MFD staff and leadership demographics to help preserve participant confidentiality,

^**= some clients used both Station 1 and another station, which is why numbers do not add to 49 and exceed 100%.

Safe Station Workflow (Versions 1.0 and 2.0)

Community partners were asked about the details of the Safe Station workflow and shared perceptions of the process, corroborated by observations. Clients enter Safe Station in person by walking into (day), or ringing the doorbell of (night), a MFD station. After being greeted by staff, the client is led to a semi-private area, a call is placed to the dispatch office which releases a tone (‘Code Hope’) indicating a client on site, and a company (i.e. on-duty fire department team) is assigned to the client. If the video doorbell is used, dispatch will sound a tone and send the company to greet the client. All members of the company will go to the Safe Station area and complete a basic physical examination and assess vital statistics of the client. While conducting this physical examination, MFD staff also ask the client questions about their health and substance use while completing a Safe Station-specific field intake assessment form. If more advanced medical intervention is required, the client will be transported to a local ED via ambulance, otherwise staff call the TP to alert them to the client’s arrival. The only decision the MFD staff made was whether the patient needed to be medically cleared at the emergency department prior to going to the partnering treatment organization. In v1.0, the TP then sent a staff member to MFD to accompany the client, or MFD staff would walk the client to the TP. When new TPs began providing services in Safe Station v2.0, a fund was established to provide transportation for clients from MFD to the partnering organization via a city contract with Lyft, a ride-sharing company (see Figure 1).

FIGURE 1.

Safe Station Versions

Safe Station Implementation: Qualitative Findings

The qualitative results are organized by clusters of the 16 CFIR constructs that emerged as salient during interviews, as they apply to the objectives of the study (i.e., characterize workflow and partnerships; identify key components) and the larger narrative of the community-wide effort (i.e., context) to implement Safe Station. As their relevance to the study aims and implementation narrative is described below, CFIR constructs and ‘replicability’ and ‘sustainability’ are bracketed in text with associated CFIR domains and provide a roadmap to monitor implementation progress. See Table 3 for the valence of each construct for each community partner group; see the Supplemental File for exemplar quotes across community partners to substantiate construct valence and convey a more nuanced narrative.

TABLE 3.

CFIR Valence Matrix

CFIR DOMAIN/Construct COMMUNITY PARTNER GROUP	MFD LEADERSHIP V1.0	MFD LEADERSHIP V2.0	MFD STAFF V1.0	MFD STAFF V2.0	ED STAFF	AP STAFF	TP STAFF	Safe Station CLIENTS V1.0	Safe Station CLIENTS V2.0
SAFE STATION PROGRAM CHARACTERISTICS
Relative Advantage	+		+	+	+	+	+	+	+
Design Quality and Packaging	+		+		+	o	+	+	+
Evidence Strength and Quality	+	+	o		+	−	+	+	+
Cost	o	−						+	+
Complexity	+						o
INNER SETTING
RFI: Available Resources	−	−	−	−		o	−
IC: Tension for Change	+		+			+	+
IC: Compatibility	+	+	+		o
Networks and Communications	−	−	−	−
RFI: Access to Knowledge and Information	o	o	o	o
Culture	+		+
OUTER SETTING
Client Needs and Resources	+		+		+	+	+
Cosmopolitanism	+	+	o	−	o	o	o
SAFE STATION STAFF CHARACTERISTICS / ATTITUDES
Knowledge and Beliefs about Safe Station/ Safe Station Clients	o		o	−		o	o
Self-Efficacy	+		o	o
IMPLEMENTATION PROCESS w/in MFD
Planning	−	−	−

CFIR = The Consolidated Framework for Implementation Research, MFD = Manchester Fire Department, AP = ambulance partner, TP = treatment partner, ED = emergency department, IC = Implementation Climate, RFI = Readiness for Implementation

(+) – Facilitator: consensus among analysts that construct surfaced pervasively as a positive influence in the organization/work processes/implementation efforts.

(−) – Barrier: consensus among analysts that construct surfaced pervasively as a negative influence in the organization/work processes/implementation efforts.

(o) – Neutral: consensus among analysts that construct surfaced as a neutral influence (i.e., mentioned generically without evidenc e of +/− influence, and/or credible interviewees contradict each other)

Empty cells – data were either not collected relevant to the construct for a community partner group OR there were insufficient data (e.g., consensus among analysts that coding was not pervasive enough to establish valence).

^*Constructs that did not surface in at least two datasets were not included in the matrix (e.g., trialability, peer pressure); n = 23/39.

Implementation Context

Community partners’ perception was that the NH opioid crisis was intolerable [Inner Setting: Tension for Change]: ‘… I’ve been doing this job pretty much my whole career, and I’ve never experienced a crisis like this before where so many people were impacted and who are literally dying…’ (AP, 221); ‘we have to create an easy access point and provide people services right away…’ (TP, 401). Community partners’ awareness of the service needs of people struggling with opioid use in NH and a lack of resources to meet those needs [Outer Setting: Client Needs and Resources], coupled with this tension for change, prompted MFD leadership to launch Safe Station. Staff agreed: ‘There’s nothing else out there. There needs to be more people to help these people’ (MFD Staff v1.0, 214).

The culture of the MFD was deemed broadly conducive to the conduct of the Safe Station program [Inner Setting: Culture]: ‘Firemen are there to take care of you no matter what’ (MFD Leadership v1.0, 303). The cultural values of the MFD—to help people in need, to follow orders unfailingly, and if need be, act as the ‘Swiss Army knife’ of the city (i.e., ‘fit in and adapt’) (MFD Staff v1.0, 201)—facilitated implementation. However, MFD staff and leadership also identified potential fault lines related to culture following program launch. Some felt the hierarchical nature of MFD facilitated Safe Station implementation (e.g., ‘we’re a paramilitary organization… whatever I’m told to do, I will do it,’ (MFD Staff v1.0, 210)), while others explained how hierarchy contributed to frustrations about a lack of avenues for staff to share their perspectives (e.g., ‘everything’s coming from above… if you build a firehouse, you should have input from the architect, budget person, fire chief, and the guys that are gonna live in the firehouse’ (MFD Leadership v2.0, 304)). However, the culture confirmed program compatibility with the mission of the MFD for many [Inner Setting: Compatibility]: ‘I haven’t seen [reference to Safe Station] in [the mission statement], but it’s kind of a blanket statement where we provide public service. I think [Safe Station] fits in there…’ (MFD Leadership v1.0, 305). Staff agreed: ‘It’s part of the job,’ (MFD Staff v1.0, 202); ‘we’ve been doing ODs forever’ (MFD Staff v1.0, 217). Yet, the shift from treating physical health conditions to the more psychological work associated with interacting with people with substance use disorders was perceived as a challenge, leading some to wonder whether Safe Station would be better delivered by professionals with more appropriate training. Also, concerns regarding having a company out of service with a Safe Station call was another incompatibility raised as staff gained experience with the model.

Characteristics of Workflow

For MFD staff and leadership, despite abundant awareness of the service needs among those using opioids and other drugs in NH, presumed knowledge and beliefs about people with addictions obfuscated the mission to help ‘no matter what’ (MFD Leadership v1.0, 303) for some, impacting workflows [Characteristics of Individuals: Knowledge and Beliefs]. A pervasive limited understanding about the chronic nature of addiction emerged in the oft-used phrase ‘repeat offender’ (MFD Leadership v1.0, 304: MFD Staff v1.0, 217), referring to people repeatedly accessing Safe Station. Many MFD staff wondered if there was a way to limit the number of times a person accessed Safe Station, believing that some were coming merely for a place to sleep for the night, rather than to engage in recovery. Beliefs that only locals should use Safe Station also strained the mission to help. Ideas about program misuse and use by non-locals led one MFD staff member to suggest Safe Station was being ‘taken advantage of’ (MFD Staff v1.0, 207). Leaders were aware that some staff were disgruntled for these reasons, but one pragmatist suggested, ‘If you can weed through the minutiae and say, “This is why we’re doing it,” and build some value to why we’re doing it…They’ll support it more’ (MFD Leadership v1.0, 309). Staff interviews after implementation of v2.0 were less moderated by goodwill, with some suggesting consequences for misuse of Safe Station:

I just said I was an addict, but I can go back to my friends and go shoot up again… If I went into the police station and said, ‘Just want to let you know I’m a bank robber,’ something’s going to be done about it.

(MFD Staff v2.0, 223)

One AP staff echoed similar frustrations while others were more compassionate and reluctant to attribute insincere motivations (e.g., ‘we do see repeat patients… but I’m sure it’s tough to quit anything’ (AP, 219)). There was consensus by TP staff that many clients were experiencing homelessness, and necessary shelters were lacking. However, ‘if we put them in a bed for someone who is seeking treatment… and they don’t want treatment, it screws everything up’ (TP, 402). These beliefs were moderated by another TP staff member who felt that most do not use Safe Station as a shelter, and although some do come from outside Manchester, ‘we’re helping a lot of families get their family members back… even if they relapse, they know they can come back’ (TP, 404).

MFD leadership expressed high confidence in the MFD ability to execute Safe Station [Characteristics of Individuals: Self-Efficacy] due to similar processes for other medical calls. However, a core difference with a basic medical call was expressed well by one leader: ‘Sometimes you’re crossing over into that counselor role, which I’m not necessarily comfortable with, but I’ve dealt with a lot of people at this point that have had problems and [so] have my guys, and we’re certainly not giving bad advice,’ (MFD Leadership v1.0, 306). MFD staff at both timepoints were less uniformly confident about their abilities to handle clients. Some felt that ‘our basic duties don’t change…’ (MFD Staff v2.0, 216). Other staff at both timepoints disagreed. Comments regarding the psychological dimensions of addiction pervaded those disagreements:

There has not been any continued education on dealing with these people who are sick. They have issues. We don’t necessarily know how to deal with those issues… I feel uncomfortable providing it [verbal support]

(MFD Staff v2.0, 213)

In line with their thoughts on self-efficacy, MFD leadership downplayed the need for additional training, ‘mostly it’s just a medical call’ (MFD Leadership v1.0, 307) [Inner Setting: Access to Knowledge and Information]. While all MFD staff are trained to the EMT level, the thread that they are not trained on ‘the counseling of the person…’ persisted across leaders:

… Not that we need to become counselors, but we’re hearing everything from they were beat up by their husband or their wife, or they were sexually assaulted … All we could really say, ‘It’s gonna be okay’… we really have zero training on how to address that…

(MFD Leadership v1.0, 310)

Most acknowledged that there had been no initial training prior to launch but that ‘if it’s something important enough where they’re changing something, [leadership will] have a class’ (MFD Staff v1.0, 202). Others suggested, ‘Questions come up… [It’s] sometimes hard to get in contact with that person we need the answers from’ (MFD Staff v1.0, 206). Access to knowledge and information reportedly improved with the implementation of Safe Station v2.0. MFD staff received an information session updating them on the new process (see Figure 1), but a desire for more training, and more details on affiliated treatment programs and Safe Station process post-MFD, remained.

Resources to support Safe Station were found to be limited [Inner Setting: Available Resources]. Several MFD leaders expressed concern regarding: a sustainable funding plan; absent grant funding; and time and personnel burden, particularly due to increased Central Station call volume. Because AP staff were called for every Safe Station prior to v2.0, all AP staff discussed the uptick in calls, and consequently ambulances were occasionally not available. Increased call volume was not deemed problematic but certainly ‘added to the workload’ (AP, 221). Treatment partner staff explained that their capacity was often overwhelmed, requiring more staff, space, and resources (e.g., mental health). For MFD staff v2.0, the increased call volume and lack of compensatory funding maintained their assessments that a significant barrier to sustainability was available resources:

Our call line went up over 3,000 calls since this program has started, yet the city has no backing for us… I worry about the availability of our busiest fire companies…there’s going to be maybe a building fire, or a spike in call volume, and a line of people looking for Safe Station as well… and our resources won’t be able to keep up with all of those demands.

(MFD Staff v2.0, 210)

Characteristics of Partnerships

Optimal implementation begins with planning [Process: Planning]. Partners with knowledge about this stage of the process were uniformly critical due to its perceived absence. Most MFD leadership felt that Safe Station was ‘thrown down our throats’ (MFD Leadership v1.0, 304), with no time to ‘talk to the guys on the floor about it first’ (MFD Leadership v1.0, 302). The MFD staff attested to learning about the model with the rest of the community on the nightly news, or by email, the day prior to launch. Understandably, MFD staff sought improved communications surrounding Safe Station [Inner Setting: Networks and Communications]. One leader suggested a more favorable view of communications, yet staff are notably missing in the information chain described:

The end of every week… [there’s] an email chain for three different things. One is the overdose statistics. One is Safe Station statistics. The other one is a chart of all different types of overdoses that we’ve responded to. There’s a lot of information that starts to flow back and forth. When we start, we’re usually the first ones between us and [the ambulance company] to be able to see a rise in a particular substance. Or we start administering more in Narcan than usual for a certain period of time. That starts to raise questions from [the police department], and we start to share information. That in turn gets the ERs involved…

(MFD Leadership v1.0, 302)

Unfavorable assessments of communications among staff interviewed about v2.0 were unchanged: ‘The administration has one goal, and the line personnel who are the most important aspect of this have different goals… They’re not training us. They’re not saying we’ve had this problem, we’re gonna change this… they’re not communicating any of that with us’ (MFD Staff v2.0, 213). The purpose of reported communications between staff was typically checking in about call volume. When asked whether interaction or coordination between the different stations had improved with the new program (v2.0), one staff member felt little had changed for Safe Station v2.0, ‘except for talking like, “oh, you guys had another Safe Station?”… not like how things are supposed to be’ (MFD Staff v2.0, 223).

The only stakeholder group to pervasively favorably evaluate the quality of communication among partners in v1.0 was MFD leadership [Outer Setting: Cosmopolitanism]:

One of the positive, unintended consequences of this is that there are no more silos when it comes to dealing with this [opioid epidemic]. Everybody talks to each other… We have regular meetings about it… Never before… has it ever been such a constant, positive flow of communication back and forth…

(MFD Leadership v1.0, 302)

Other community partners, including MFD staff, had either mixed or negative reviews about the community partnership communication. For example, MFD staff cited unfamiliarity with what happens after clients leave the fire station for TP: ‘I’d like to know what happens after that. Did the help work?’ (MFD Staff v1.0, 217). One AP staff said, ‘Communication on the way down should be a lot better… here’s the problem, what can you do to help?’ (AP, 218). Treatment partner staff expressed frustration regarding communication with the EDs: ‘Sometimes the ED doesn’t even call us and just releases the client… I get frustrated with it, but it has gotten better’ (TP, 402). Both EDs had staff who expressed desire for increased communication with MFD: ‘There’s not a very good communication system for this program. The fire department basically runs it. We used to have these city meetings that stakeholders from both hospitals and all of the emergency services within Manchester would meet up… those haven’t occurred in a while’ (ED, 412). However, the EDs underscored that their historical relationships with the MFD could be characterized as ‘pretty connected with meetings…’ (ED, 407). Challenges with unity continued to be featured among partners even with the advent of v2.0. When asked about communication with new treatment partner staff, a MFD staff member said, ‘The only communication we have with them is we call and let them know that someone is coming over… It would be nice to know what we’re sending [clients] to so if they ask us questions, we might be able to fill them in a little bit more’ (MFD Staff v2.0, 208).

Key Components of Safe Station

The program components that all stakeholders emphasized were engaging and key to Safe Station were the: immediacy of help offered, low-threshold access (e.g., no waiting lists), central access point for services, free service, and firefighters’ respectful, non-judgmental attitudes toward clients. This respect was the most appreciated component among clients.

When you’re down at that point in your life… it don’t take much for someone to get up and walk away… It’s real easy to hear a bad word and just say ‘f*** it’ and get up and walk out. These guys never did anything like that… when I was leaving there, getting into the ambulance and everybody was shaking my hand, ‘Good luck to you.’ ‘You’re doing the right thing.’ I actually left there with a bit more of, do I want to say confidence? Maybe… These guys ain’t judging me…

(Clients v2.0, 141)

Clients added that safety (e.g., no fear of arrest, medical attention, off the ‘street’) was another key component, corroborated by perspectives on care delivered by MFD. All of these characteristics underscore the pervasive theme of accessibility [e.g., Intervention Characteristics: Design Quality and Packaging].

The advantages of Safe Station compared to the status quo were variously described [Intervention Characteristics: Relative Advantage]. Compared to seeking help through more traditional models (e.g., medical settings), clients emphasized the comfort they experienced with MFD staff: ‘We were raised to trust firemen. They’re not doctors, so they’re not going to look at you like you have some medical issue. They’re not cops, so they’re not going to look at you like you’re committing a crime’ (Clients v1.0, 122). MFD staff highlighted the free nature of the service compared to costly alternatives, and TP staff stressed that Safe Station facilitated low-threshold entry to their program: ‘We meet them where they’re at… if they don’t have insurance or have not yet completed detox… no problem’ (TP, 406). For ED staff in both hospitals lacking formal detox programs, Safe Station provided resources formerly unavailable; e.g., ‘they are able to get people when they are at the point of actively asking for help and get them resources that they need’ (ED, 407). MFD staff and clients were aligned in assessments that Safe Station v2.0 moved people along the continuum from first contact with Safe Station to treatment much faster relative to v1.0 once Lyft was involved, e.g., ‘[we can] get back in service quicker’ (MFD Staff v2.0, 210).

The straightforward nature of the model [Intervention Characteristics: Complexity] was echoed by most in leadership roles: ‘We check them. If you don’t have to go to the hospital, we just make a phone call…’ (MFD Leadership v1.0, 305). However, another leader acknowledged that ‘the biggest and usually hardest-to-obtain key element is [the] treatment provider’ (MFD Leadership v1.0, 302). Treatment partner staff agreed: ‘We work with the MFD every day and, like working with other community facilities, that’s a simple part of this… what do you do after they’re here, that’s what brings up the barriers’ (TP, 402). In the same vein, clients were unanimous that zero cost was one of the most engaging characteristics of Safe Station [Intervention Characteristics: Cost]. However, MFD leadership opinions ranged from agreement with this notion to frustrated comments about potential zero-sum scenarios stemming from taking a company or fire station out of service for a Safe Station event.

MFD leaders measured the strength and quality of Safe Station using disparate types of evidence, ranging from the numbers of people served (e.g., ‘We’ve taken on almost 2,500 people. It’s a great success’ (MFD Leadership v1.0, 304)), to reduced overdose deaths (e.g., ‘fatalities are 40% down’ (MFD Leadership v1.0, 302)), to the uniformity of service provision (e.g., ‘They come in. They get checked. We make a phone call. It’s happened 100% of the time. If that is what the program is, I think it’s doing okay’ (MFD Leadership v1.0, 305. [Intervention Characteristics: Evidence Strength and Quality]. Due to different metrics for success, community partners had varied perceptions of effectiveness. Treatment partner staff were generally upbeat about their evidence: e.g., ‘success is that person who constantly comes in, but they make it to a residential program’ (TP, 402). One MFD staff member felt that ‘when you see families walk in, father and mother with their son or daughter… you can tell we’re doing something good hopefully’ (MFD Staff v1.0, 214). Other MFD staff were more disparaging about the quality of their evidence: ‘It would be nice to know, is it working? Not just what you hear on the news’ (MFD Staff v1.0, 215). And another staff member reported mixed outcomes: ‘While it seems to work for some… I find that our homeless and drug population has actually increased dramatically from when [Safe Station] started’ (MFD Staff v1.0, 201). ED staff suggested that Safe Station clearly helps the hospitals: ‘[It’s] a really good resource because we have a place to send them… it works great… what happens after they leave us, I don’t know’ (ED, 407). Finally, one AP staff member offered the wisdom that, ‘things are only gonna be effective as clear as the goals are… It was definitely a needed program but is it really making a difference?’ (AP, 218).

Community partners believed that Safe Station is potentially replicable given the following stipulations [Replicability]: proper planning prior to execution, strong relationships and communication with partners, firefighters dedicated to service provision, adequate funding, and accessible treatment resources with adequate capacity. There was agreement that program replication would be challenging in communities with volunteer fire departments without 24/7 staffing, or in departments with one company. Relatedly, there was consensus that Safe Station is sustainable if staffing and funding are secured [Sustainability]. Some firefighters believe it is inexpensive to administer and thus sustainable; the program requires few additional resources, and staff are always present and ready to respond. These firefighters felt that any funding should be directed towards the TP, rather than used to fund the fire department. A few suggested more money needs to be allocated to MFD but were doubtful it would happen since they have demonstrated Safe Station’s conduct without additional funding. There was more uncertainty about the sustainability of the treatment partners. Staff expressed that increased treatment capacity and funding are required to keep the program running. From the TP perspective, more trained staff to reduce burnout and exhaustion among staff is needed.

Safe Station Implementation: Quantitative Metrics

Though small sample sizes precluded significance testing, a few patterns observed in the data are worthy of note. First, the AMHR scores skewed positive, falling mainly well above moderate agreement to full agreement with respect to outcomes indicative of effective implementation efforts (see Table 4). Second, across the three stakeholder groups’ shared subscales, clients and MFD leadership generally scored implementation context/outcomes higher than staff. Notable among these, MFD staff subscale scores for adoption and acceptability appeared lower compared to the client and MFD leadership group scores and decreased from v1.0 to 2.0. Third, MFD leadership subscale scores for feasibility and organizational climate were slightly lower for v2.0 compared to 1.0. Fourth, the PSAT total scores were in the mid-range across MFD staff and leadership. However, subscale scores for funding stability and strategic planning were notably lower compared to other domains across stakeholder groups and for both Safe Station versions. Finally, MFD staff program evaluation subscale scores on the PSAT were lower than leadership scores across versions.

TABLE 4.

Implementation Metric Scores.

	Safe Station Clients		MFD Staff		MFD Leadership6
AMHR Scores1,3	v1.0 (n=37)	v2.0 (n=12)	v1.0 (n=17)	v2.0 (n=10)	v1.0 (n=8)	v2.0 (n=6)
Adoption	2.36 (0.76)	2.32 (0.61)	1.90 (0.38)	1.53 (0.51)	2.54 (0.53)	2.31 (0.78)
Acceptability	2.70 (0.53)	2.55 (0.39)	1.56 (0.64)	1.09 (0.77)	2.28 (0.51)	2.33 (0.80)
Appropriateness	2.81 (0.27)	2.78 (0.23)	2.33 (0.52)	2.18 (0.82)	2.62 (0.27)	2.36 (0.76)
Feasibility	2.45 (0.47)	2.51 (0.38)	1.83 (0.43)	2.11 (0.67)	2.33 (0.45)	1.91 (0.75)
Reach/Access	2.50 (0.39)	2.53 (0.29)	2.40 (0.38)	2.42 (1.13)	2.68 (0.18)	2.77 (0.48)
Implementation Leadership	--4	--	2.48 (0.54)	2.56 (1.00)	2.51 (0.25)	2.29 (0.70)
General Leadership (Skills)	--	--	2.43 (0.37)	2.40 (0.82)	2.31 (0.62)	2.17 (0.63)
Organizational Climate	--	--	2.53 (0.32)	2.23 (0.24)	2.42 (0.27)	2.00 (0.71)
PSAT Scores 2,3
Total Score	--	--	4.38 (1.02)	4.34 (1.44)	5.01 (0.76)	4.67 (1.32)
Subscale Scores
Environmental Support	--5	--	5.41 (0.99)	5.28 (1.33)	5.89 (0.55)	5.60 (1.82)
Funding Stability	--	--	3.64 (1.43)	3.33 (1.48)	3.66 (1.90)	3.73 (1.52)
Partnerships	--	--	4.29 (1.35)	4.27 (1.68)	5.09 (0.88)	5.07 (1.69)
Organizational Capacity	--	--	4.75 (1.17)	4.33 (1.82)	5.63 (0.63)	4.13 (1.35)
Program Evaluation	--	--	3.94 (1.41)	3.82 (1.98)	5.55 (0.90)	5.17 (1.68)
Program Adaptation	--	--	4.31 (1.90)	4.26 (1.81)	4.89 (1.56)	5.18 (1.60)
Communications	--	--	5.05 (1.33)	4.76 (2.49)	5.40 (0.82)	4.78 (1.44)
Strategic Planning	--	--	3.63 (1.79)	3.92 (2.49)	4.06 (1.15)	3.73 (0.89)

AMHR = Applied Mental Health Research Group Implementation Measure, MFD = Manchester Fire Department, PSAT = Program Sustainability Assessment Tool, v1.0 = version 1.0, v2.0 = version 2.0

¹AMHR scores range from 0 (Not at all), 1 (A little bit), 2 (A moderate amount), 3 (A lot); Mean (SD)

²PSAT scores range from 1 (Little to no extent [Negative]) to 7 (To a very great extent [Positive]); Mean (SD)

³Sample sizes too small to conduct significance testing

⁴Data not collected due to lack of relevance to client stakeholders.

⁵Data collection for clients not included in this tool.

⁶2 MFD staff and 2 MFD leaders did not complete survey

Discussion

In the wake of the opioid crisis, communities across the US and Canada have worked together to develop and implement unique community-based interventions to reduce opioid-related harms (Leece et al., 2019). This study aimed to systematically evaluate Safe Station, a novel community-based program, to characterize the multi-organizational partnerships and program workflow, and to identify key characteristics that are engaging, effective, replicable and sustainable. Quantitative and qualitative findings are integrated below to increase understanding, corroboration, and credibility of findings (Tashakkori & Teddlie, 1998). Implementation strategies, or ‘methods used to enhance the adoption, implementation, and sustainability of a model’ (Proctor, Powell & McMillen, 2013), are underlined below to introduce strategies rated by an implementation science expert panel as the most important and feasible (Waltz et al., 2015) among a larger compilation (Powell et al., 2015). Others might use these strategies to replicate and/or sustain Safe Station-type models.

The implementation context was conducive for the launch of Safe Station

The Safe Station implementation was shaped by the local context of the MFD. The capacity (Aarons et al., 2012) and needs assessment facilitated by the CFIR (Kirk et al., 2016) underscored community partners united in awareness of the needs of people with substance use and related problems, and their conviction that something needed to be done to address those needs due to the escalating loss of life. These factors, as well as their combination, have been demonstrated in other contexts to be indicators of implementation success (Kowalski et al., 2018; Damschroder & Lowery, 2014; Safaeinili et al., 2019). In an implementation study designed to explain variation in the implementation of a weight management program disseminated nationally to Veterans Affairs (VA) medical centers, Damschroder and Lowery (2014) found that 10 of 31 CFIR constructs assessed – including patient needs and resources, tension for change, and leadership engagement – strongly distinguished between facilities with low versus high program implementation effectiveness. Another recent implementation-focused evaluation that enlisted the CFIR surfaced similar findings (Kowalski et al., 2018). Among 10 of 37 CFIR constructs deemed most likely to play a role in the effectiveness of a clinical program for diabetes, ‘patient needs and resources’ and ‘leadership engagement’ also emerged as influential to program success. Most recently, one research group elevated ‘patient needs and resources’ to the level of domain in their use of the CFIR framework to evaluate implementation, due to a finding of disproportionately high mentions of patient needs and resources in interviews (Safaeinili et al., 2019). Patient or client-centered organizations are more likely to implement change effectively (Shortell et al., 2004).

Coalition building and local consensus discussions established that the problem was important to the community. By launching Safe Station, the MFD identified their organization as de facto champions of the implementation effort, demonstrating the capacity to lead. These pre- and early implementation strategies were conducive to successful implementation and are recommended to those seeking to replicate such an effort. (See Table 5 for details on how the strategies might be enlisted to support replication/sustainment efforts).

TABLE 5.

Evidence-based Implementation Strategies for Replication/Sustainability of Safe Station-type Models

CFIR Construct	Facilitator/Barrier	Strategy to mitigate barrier/leverage facilitator
Client Needs and Resources	Awareness of the needs of people with substance use and related problems had been growing in the Manchester community for some time. Many participants referenced community partner meetings during which the opioid epidemic and opioid overdose crisis was the main topic of discussion.	Build a coalition – recruit and cultivate relationships with partners in the implementation effort. Leverage this facilitator as the community had done coalition building. However, mitigation is needed here too as the community partners were not part of the conversation to design/launch Safe Station.
Tension for Change	There was pervasive messaging across interviews regarding community partner conviction that something needed to be done quickly to address the opioid epidemic and related overdose crisis.	Conduct local consensus discussions – include local providers and other community partners in discussions that address whether the chosen problem is important and whether the innovation to address it is appropriate. Leverage this facilitator as the community of Manchester had clearly established the tension for change and that the problem of the opioid epidemic and related overdose crisis was important. Mitigation is needed here too as the community partners were not included in consensus discussions specifically designed to debate the appropriateness of Safe Station to address it. Identify and prepare champions – identify and prepare individuals who dedicate themselves to supporting, marketing, and driving through an implementation, overcoming indifference or resistance that the intervention may provoke in an organization. Leverage this facilitator as the MFD leadership dedicated themselves to this role of champion. Mitigation is needed though as there was no attention to overcoming indifference and/or resistance that was present among staff after they learned about Safe Station on the nightly news the night before its launch.
Planning	Given the exigencies of the overdose crisis, there was a hurried launch of the Safe Station program which precluded meaningful planning.	Implementation scale-up – phase implementation efforts by starting with small pilots or demonstration projects at one or two fire stations in your community and gradually move to a system or city-wide rollout Develop educational materials – develop and format manuals, toolkits, and other supporting materials in ways that make it easier for community partners to learn about the innovation and for firefighters on the front lines to learn how to deliver the service innovation Assess for readiness and identify barriers and facilitators – assess various aspects of community partners to determine degrees of readiness to implement, barriers that may impede implementation, and strengths that can be used in the implementation effort
Complexity	Many staff found that there was far greater complexity associated with Safe Station service delivery, impacting self-efficacy and perspectives on available resources/costs as time progressed.	Purposely reexamine the implementation - Monitor progress and adjust clinical and/or service delivery practices and implementation strategies to continuously improve the quality of care.
Access to Knowledge and Information	MFD leadership felt strongly that Safe Station was consistent with a medical call, and thus staff had the knowledge, skills, and resources to deliver the service on day one. However, many staff found that there was far greater complexity associated with service delivery, impacting self-efficacy and perspectives on available resources/costs as time went on. Interacting with people with substance use problems and comorbid mental health issues was not intuitive, and many staff felt they had not been adequately trained for this service dimension.	Conduct educational meetings – hold meetings targeted toward different community partner groups (e.g., fire department staff, ambulance staff, emergency department staff, treatment partner staff) to teach them about the service to be delivered and to invite questions/concerns about potential expectation/experience discrepancies Conduct ongoing training – plan for and conduct training in the clinical innovation on an ongoing basis (e.g., standard operating procedures, the chronic relapsing nature of addiction, basic motivational interviewing skills for communicating with people seeking help for substance use problems, community partner coordination, relay of process/outcome data, program updates)
Networks and Communication	The crux of the implementation barriers identified related to the need for improved communication among community partners.	Organize community partner implementation team meetings – develop and support community partners who are implementing Safe Station and offer protected time to reflect on the implementation effort, share lessons learned, and support/reinforce one another’s learning. Significantly for a Safe Station implementation, firefighters and other community partners must have communication channels not only within the local firehouse among the ‘guys on the floor’ but also opportunities to share experiences and knowledge with leadership. Capture and share local knowledge – in the event that there is more than one station, treatment partner, or hospital in a community providing the service, it will be essential to create a database or other repository to share experiences and to capture knowledge from implementation sites on how implementers and front line workers made something work in their setting and then share it with other settings, e.g., stations, treatment partners, hospitals, etc.
Evidence Strength and Quality	Variability in the evidence strength and quality of Safe Station offered by community partners underscored the lack of clarity with respect to goals, as do suggestions of uncertainty qualifying reports of evidence.	Develop and implement tools for quality monitoring – develop, test and introduce into quality-monitoring systems the right input – the appropriate language, protocols, algorithms, standards, and measures (e.g., Safe Station client outcomes) that are specific to the Safe Station program being implemented. Facilitate relay of clinical data to providers – provide as close to real-time data as possible about key measures of process/outcomes using integrated models/channels of communication to promotes self-efficacy, compliance with required clinical activities, and clarity regarding goals.
Cosmopolitanism	The aspirational ‘no more silos’ referenced by one MFD leader when assessing the quality of communication among the multi-organizational partners was not reflected in community partner assessments of communication.	Use advisory boards and workgroups – create and engage a formal group of multiple kinds of stakeholders to provide input and advice on Safe Station implementation efforts and to elicit recommendations for improvements. For example, treatment partner staff felt that EDs often would simply release clients back into the community - following medical clearance – such that there was a break in the continuity of care. However, it would appear there was no appropriate channel to communicate this frustration.

CFIR = The Consolidated Framework for Implementation Research

^*Implementation strategies and definitions are sourced in Powell et al., 2015 and Waltz et al., 2015.

Safe Station was deemed engaging and effective as a connection to recovery due to accessibility

The key components operationalizing Safe Station’s accessibility were the free, immediate, low-threshold access to help obtaining a range of services, offered by MFD staff with respectful, non-judgmental attitudes (the most referenced key component by clients) in a safe space. These components were observed to be essential and contributed to assessments that Safe Station was engaging and effective as a ‘connection to recovery.’ Experiences of care, particularly related to staff encouragement, non-judgmental attitudes (Snow et al., 2019), and support, have been associated with beliefs among treatment seekers that treatment can be effective (Greenberg et al., 2006) and that long-term recovery is possible (Snippe et al., 2015). The relative advantage of the Safe Station model (e.g., the sine qua non for adoption and implementation (Greenhalgh et al., 2004)) compared with the status quo was obvious to all community partners. Near-unanimous positive client experiences with, and highly favorable community partner assessments of Safe Station are corroborated by the relatively high scores on the AMHR among clients and MFD staff and leadership, increasing the credibility of findings.

MFD culture facilitated and impeded optimal implementation and workflow of Safe Station

The MFD cultural value to protect life at any cost, combined with the paramilitary, hierarchical structure, facilitated the program’s rapid launch. However, it was this same structure and associated behavioral expectations (Cooke & Szumal, 1993) referenced in MFD staff’s and other community partners’ expressions of dissatisfaction with dimensions of workflow. The rapid launch of Safe Station—indicative of the lack of planning, lack of solicited input from staff about how best to conduct Safe Station, and perfunctory training—precluded phased implementation scale-up and development of educational materials. Given the exigencies of the overdose crisis, MFD is applauded for acting and becoming a part of the solution and should not be faulted for a hurried launch. However, implementation optimization in future settings may best include assessing community partners for readiness, identifying barriers and facilitators, and potentially piloting a Safe Station program in one or two firehouses following development of evidence-based manuals/toolkits and/or other supporting materials to facilitate community partner education about/staff delivery of the program.

Pre-implementation and ongoing education and training during implementation of interventions with individuals with co-occurring mental health and substance use comorbidities are indicated to address potential barriers like negative attitudes and compassion fatigue. Stigmatizing and judgmental attitudes towards people with opioid use disorder can be reduced through both basic education in addiction theory and evidence-based treatments (The National Center on Addiction and Substance Abuse (CASA), 2012), as well as interventions offering practical tools for remediating burnout and professional fatigue resulting from work with people with opioid use disorder (Marine et al., 2015). Organizational structure can influence turnover among professionals who work on the frontlines with people with substance use disorders through more participatory management structures (versus strict hierarchy) (Garner et al., 2007; McNulty et al., 2007). A review of intervention programs for preventing burnout (Awa et al., 2010) concluded that effective interventions can be either person-directed (individual and groups: e.g., cognitive behavioral measures aimed at strengthening self-efficacy and coping skills, social support or different kinds of relaxation exercises), organization-directed (e.g., changes in work procedures, increasing job control or the level of participation in decision making) or a combination of both.

The MFD cultural value to be the ‘Swiss Army knife’ manifested in assurances that Safe Station was compatible with basic MFD staff duties. Descriptions of the program’s complexity, self-efficacy, access to knowledge and information about effective delivery of Safe Station, available resources, and assessments of cost reveal agreement and disagreement with compatibility. MFD leadership felt strongly that Safe Station was consistent with a medical call. However, many staff found that there was far greater complexity associated with service delivery, impacting self-efficacy and perspectives on available resources/costs as time went on. Interacting with people with substance use problems and comorbid mental health issues was not intuitive, and many staff felt they had not been adequately trained for this service dimension. Even among professionals who might be expected to provide such services (e.g., medical practitioners), many do not feel confident in discussing substance use issues with their patients (Blumenthal et al., 2001; McCormick et al., 2006). Despite the limited time MFD staff typically had with clients, increased training regarding the basics of developing even a brief therapeutic alliance with individuals with substance use and/or co-occurring disorders is indicated. To this end, the Substance Abuse and Mental Health Services Administration (SAMHSA) developed a treatment improvement protocol (TIP) (CSAT, 2005), and the American Medical Association underscored ‘three steps’ (i.e., let the patient tell their story, get to the heart of the matter, convey respect) when talking with people about substance use disorders (Parks, 2016).

Increased call volume also led many to suggest inadequate staffing and funding as additional reasons for perceived incompatibility. Slightly lower MFD leadership subscale scores across versions for feasibility and organizational climate on the AMHR dovetail with qualitative data referencing available resources. Perceptions of inadequate staffing and insufficient compensatory funding were significant feasibility issues raised by many leaders. Leaders also expressed awareness that the learning climate may not have been optimized for peak performance of the model. Purposeful reexamination of Safe Station to clarify staff expectations, including regular monitoring of progress toward goals as well as adjusting practices to improve quality of care, is a suggested implementation strategy to address issues of culture and incompatibility.

Misconceptions about addiction among MFD staff threatened sustainability of Safe Station

Stigmatizing knowledge and beliefs about people with substance use problems obfuscated the mission to help unconditionally among some MFD staff. Misconceptions about the nature of addiction and how best to respond manifested in beliefs that people who accessed the service more than one or a couple times were ‘repeat offenders,’ and that there should be ‘repercussions’—the criminal justice corollary—for repeatedly seeking help. As was overwhelmingly evidenced by client satisfaction, MFD staff and leadership continued to provide the Safe Station service in a professional and respectful manner. However, these misconceptions, while not uncommon among frontline healthcare workers (van Boekel et al., 2013), threaten to increase client treatment avoidance (Ball et al., 2006), diminish client’ self-efficacy (Parcesepe & Cabassa, 2013), negatively impact service delivery (Kelleher, 2007), and lower staff job satisfaction (Ford, Bammer & Becker, 2008). These undesirable outcomes make misconceptions ideal targets for education and ongoing training to enhance adoption, implementation, and sustainability of Safe Station-like programs. AMHR adoption and acceptability subscale scores for MFD staff were lower compared to clients and MFD leadership and decreased from v1.0 to 2.0. Adoptable programs are those that staff are willing to try and continue to use. MFD staff subscale scores corroborate the qualitative data that convey a waning enthusiasm for delivering Safe Station. Acceptable programs are satisfying and agreeable to providers. These scores, too, are consistent with the qualitative data in that staff expressed dissatisfaction with aspects of Safe Station service delivery (e.g., non-existent compensatory funding, faltering self-efficacy, perfunctory training, uncertainty about effectiveness).

Factors known to influence structural stigma—or stigma associated with professional groups as opposed to social stigma and self-stigma among those with substance use disorders (Livingston et al., 2011)—include increasing knowledge about substance use disorders and the degree of contact or experience that one has with people with mental and substance use disorders (NASEM, 2016), as well as social norms concerning attribution of cause or blame for the disorders (Schomerus et al., 2011). The use of video (e.g., addressing opioid-related stigma in EDs (ACEP, 2020)) and other educational content (e.g., critical reflection techniques (Ballon & Skinner, 2008); structured substance use education and clinical experience programming (Silins et al., 2007)) could be useful tools to increase education about, and empathy toward, people living with addictions in Safe Station settings.

Lack of clarity about Safe Station goals was associated with variable measures of program effectiveness

The observation that ‘things are only gonna be effective as clear as the goals are’ is the crux of the implementation barriers identified relating to the need for improved communication. Variability in the evidence strength and quality of Safe Station offered by community partners underscores the lack of clarity with respect to goals, as do suggestions of uncertainty qualifying reports of evidence. Implications of the lack of clarity are rife in these data. If the goal is medical clearance, then MFD staff (in coordination with ED staff) are doing an exemplary job. If the goal is to connect people to recovery, MFD (in coordination with TP staff) may be achieving their goals. However, MFD staff and other community partners were not part of a feedback loop relaying information about success. Differences of understanding about costs, complexity, compatibility, and self-efficacy can all be explained by the lack of clarity with respect to goals. Mid-range PSAT total scores across MFD staff and leadership are reflective of uncertainty about the sustainability of Safe Station. Specifically, lower subscale scores for funding stability and strategic planning across both Safe Station versions, as well as notably lower program evaluation subscale scores for MFD staff compared to leaders, corroborate qualitative expressions of uncertainty about goals, what constitutes success, and how Safe Station can continue to operate without dedicated funding streams. The organization of staff meetings among those implementing Safe Station, protected time to reflect on the implementation effort, and the development and implementation of tools for quality monitoring (e.g., audit and feedback (Hysong et al., 2012)) are implementation strategies warranted by lessons learned. Additionally, facilitating the relay of clinical data to staff delivering Safe Station is critical for continued self-efficacy, compliance with required clinical activities (Ivers et al., 2012), and clarity regarding goals that may encourage behavior change (e.g., directing attention, establishing commitment (Brehaut et al., 2012)).

The aspirational ‘no more silos’ referenced by one MFD leader when assessing the quality of communication among the multi-organizational partners was not reflected in community partner assessments of communication. Certain aspects of organizational culture and climate, including unidirectional leadership styles and communication channels, are known to be associated with lower morale among staff and higher levels of work stress, burnout, and turnover (Gershon et al., 2004). Also, literature discussing multi-organizational partnerships has noted challenges to developing and sustaining community collaborations, including differing priorities across organizations, time constraints, and the voluntary nature of engagement in community-based interventions (Merzel & D’Afflitti, 2003; Altman et al., 1991). The PSAT communication subscale scores decrease between v1.0 and 2.0 for both leaders and staff yet are relatively high compared with other subscales. Use of inner and outer setting advisory boards and workgroups are recommended to engage community partners to provide input and advice on implementation efforts, elicit recommendations for improvements, and improve formal and informal communication channels.

Study Strengths and Limitations

This is the first evaluation of the novel, community-based Safe Station program. As such, lessons learned from this implementation study may assist with MFD’s ongoing implementation and may also support increased adoption, replication, and sustainment of Safe Station-type interventions in other communities. The use of the CFIR to help identify the factors facilitating/impeding program implementation and effectiveness, the mixed-methods nature of the study, and extensive triangulation and data convergence are additional strengths. Notable limitations include self-report by a small number of individuals who may or may not represent all community partners; and self-selection bias may have distorted the favorability of the assessments. Also, our small sample size precluded significance testing, limiting confidence in the interpretation of data convergence. Ideally, research on sustainability is conducted at multiple time points. While we were able to conduct follow-up interviews following the shift from v1.0 to 2.0, more data collection points likely would have provided a richer implementation narrative. Finally, we believe it would have been preferable to include clients in the planning stages of the implementation of the model. Given the real-life exigencies through which this model was launched, planning was absent in many respects. The barriers created due to the lack of planning were extensively discussed herein. However, other community efforts to replicate such models are advised to include potential users of such services in the planning phase.

Conclusions

Safe Station was a welcome and needed community-initiated response to the opioid overdose crisis in Manchester, New Hampshire. As with other studies that have explored barriers and facilitators to implementing community-based programs to reduce opioid-related harms (Leece et al., 2019), the common barriers of stigma, funding, substance use treatment availability, and staffing issues impacted this implementation effort. This study is responsive to the need for more research to examine the development and implementation of community-based interventions to reduce opioid-related harms. Thus, our findings regarding the experiences of the community partners involved in this effort may provide keen insights for other communities similarly seeking to implement and adopt a program like Safe Station. As a result of this effort, the MFD also received important feedback resulting in significant improvements to the critical mission to connect people to recovery and potentially save lives.