Strategies for Sustaining Fidelity: A Multi-state Qualitative Analysis in Housing First Programs

Abstract

Purpose:

Little is known about long-term fidelity of evidence-based interventions (EBIs) under changing conditions. This study examines how staff at ‘mature’ (eight or more years in operation) Housing First (HF) programs strategize to sustain EBI fit in different geographic areas in the Mid-Atlantic/Northeastern United States.

Methods:

Six focus groups (FGs) at three purposively selected HF programs were conducted with separate FGs for case managers and supervisors at each site. FG discussions elicited participants’ service approaches and strategies in addressing fidelity amidst ongoing changes affecting each program. Thematic content analysis of FG transcripts was conducted using the five HF fidelity domains (housing choice/structure, separation of housing and services, service philosophy, service array, and program structure) as a priori themes with inductive content analyses conducted on data in each theme. Strategies for rigor were employed.

Results:

Case managers (N=17) and supervisors (N=16) were predominantly white (76%) and female (60%). Across the themes, challenges included lack of affordable housing and choice, funders’ restrictions and practice ‘drift.’ Strategies included community engagement and hiring, strong leadership and ‘bending the rules.’ There were no differences across sites.

Conclusions:

Later-stage implementation challenges show the need for continued vigilance in fidelity to EBIs. Among the strategies used to address fidelity in this study, the pursuit of pro-active community engagement to attract knowledgeable staff as well as increase local buy-in was considered pivotal at all three sites. These findings underscore the need to attend to the external setting as well as to internal program operations.

Introduction

The HF model is a complex, multi-component EBI that has been rapidly disseminated on a large scale (Padgett, Henwood, & Tsemberis, 2016). The overarching aim of this intervention is to end homelessness for individuals with barriers to housing stability such as mental illness, substance use disorder, and chronic health conditions. Originally developed by the Pathways to Housing program in New York City in 1992, this evidence-based homeless services model involves a combination of permanent affordable housing and wrap-around support services. HF has been distiguished from “standard care” supportive housing in that it provides immediate access to independent housing without clinical treatment prerequisites. As such, it provides support services from a distinctively service user-driven philosophy using a harm reduction approach (Tsemberis, 2010).

Stefancic and colleagues (2013) have described the five theoretical principles central to the model as: 1) eliminating barriers to housing access and retention; 2) fostering a sense of home; 3) facilitating community integration and minimizing stigma; 4) utilizing a harm reduction approach; and 5) adhering to individualized, service user-driven services that promote recovery. They also identify five key domains of Pathways HF (PHF) fidelity that can be used to assess the degree to which a program adheres to the original PHF model: 1) housing choice and structure (e.g., choice of neighborhood and unit in housing that is not reserved solely for individuals with disabilities), 2) separation of housing and services (e.g., housing should be provided without expectations for treatment engagement), 3) service philosophy (e.g., a harm reduction approach), 4) service array (e.g., services include housing support, psychiatric and substance use treatment, supported employment, nursing, and services to assist with social integration), and 5) program structure (e.g., functioning as a team, holding frequent meetings, and having a low consumer-to-staff ratio).

HF has been widely implemented across the United States and internationally, often with variability from the original PHF model, as the core concepts of the model can be misinterpreted or diluted within local contexts (Stefancic et al., 2013). For example, the PHF model typically employs a housing approach providing independent apartments scattered throughout the community, whereas non-PHF programs may deviate from the original model by utilizing congregate housing options depending on available resources.

The PHF model has broad empirical evidence that supports its efficacy in addressing homelessness and related issues. A systematic review of HF identified that in all but one of 12 randomized controlled trials, HF produced greater housing retention as compared to standard care such as transitional housing and shelters (Kertesz & Johnson, 2017). Positive effects have also been found in relation to community functioning and well-being (O’Campo et al., 2016), reductions in alcohol and drug use (Kirst, Zerger, Misir, Hwang, & Stergiopoulos, 2015; Padgett, Stanhope, & Henwood, 2011), and control of HIV (Buchanan, Kee, Sadowski, & Garcia, 2009). However, a review of published HF trials found that only three of 14 assessed fidelity (Nelson et al., 2014).

Additionally, few studies have empirically examined specific HF fidelity facilitators and barriers outside of research demonstration projects. One such study within a large-scale HF initiative facilitated through the U.S. Department of Veterans Affairs (VA) across 142 VA medical centers (Kirst et al., 2015) found substantial fidelity to HF domains of “focusing on permanent housing” and “removal of preconditions to housing entry.” However, there was considerably lower fidelity to HF domains of “sufficient supportive services” and a “modern recovery philosophy.” The qualitative data in this study suggests that operational issues such as staffing and training shortfalls likely challenged performance in these two domains.

This finding echoes results from fidelity assessments of the early stages of implementation of HF in the Canadian At Home-Chez Soi study. During early implementation of HF, limited availability of affordable housing, limitations in the array of services available in some of the communities, and staff turnover and burnout impeded high fidelity implementation, while service delivery system factors (i.e., local leadership) as well as training and technical assistance facilitated high fidelity implementation (Nelson et al., 2014). Similarly, communityled planning involving multiple stakeholders undertaken prior to the training and technical assistance was found to advance implementation in the early stage (Worton et al., 2018).

Overall fidelity to the HF model was high during early and late implemetnation stages in this research demonstration project, but somewhat lower for the domain focused on assuring adequate service supports for service users in early implementation (Nelson et al., 2014; Worton et al., 2018; Macnaughton et al., 2015). This domain improved over time as staff became more competent in delivery of the model, a possible marker for sustainability of HF. In addition, At Home-Chez Soi trial researchers noted that the generalizability of their results is limited as realworld settings without the resources for external training and technical assistance present in this trial may find it challenging to replicate this level of fidelity (Tsemberis, Gulcur, & Nakae, 2004).

Aside from the Canadian experiment and the original experimental evaluation of HF, virtually all HF implementations occur outside of the controlled environment of randomized trials (Padgett et al., 2016). Moreover, there is a lack of information about later stages of HF implementation and the sustainability of the model outside of the Canadian context. Further knowledge of on-the-ground provider experiences can inform more efficient and effective strategies to support sustained fidelity to this multilevel model over time and in diverse systems and settings.

The later stage of implementation, intervention sustainability, has been defined as “the continued use of program components and activities for the continued achievement of desirable program and population outcomes” (Scheirer & Dearing, 2011, p. 2060). It has been further operationalized in conceptual models (e.g., RE-AIM, PRISM) as the maintenance of results (Feldstein & Glasgow, 2008), at both the organizational level and the individual level (RE-AIM, 2018), with one of the markers of sustainability being competency in the use of the EBI at the systemic, organizational, and practitioner levels (Chamberlain, Brown, & Saldana, 2011).

Chambers, Glasgow, & Stange (2013) proposed the use of the dynamic sustainability framework (DSF) that incorporates an understanding of the changing context of service delivery and posits that there should be “continued learning and problem solving, ongoing adaptation of interventions with primary focus on fit between interventions and multi-level contexts, and expectations for ongoing improvement as opposed to diminishing outcomes over time” (Chambers et al., 2013, p. 1). The DSF, which provides sensitizing theoretical concepts for this study, considers sustainability of an EBI as a function of “fit” between intervention characteristics, the practice setting in which they are to be applied, and the ecological context of the larger system. Furthermore, this framework posits that change is constant at each of these levels and thus successful sustainability of an EBI depends on an intervention being adapted to fit within these environments over time through continuous quality improvement.

Despite the complex nature of many EBIs and the dynamic contexts in which they are delivered, there is a dearth of research on how multi-component interventions are implemented and sustained over time. Experts have further described the lack of research on intervention sustainability as “one of the most significant translational research problems of our time” (Proctor et al., 2015, p. 2). Without an understanding of the factors that influence sustainability and later-stage implementation processes, the potential impact of a given EBI could be limited.

The primary aim of this qualitative study is to understand the challenges and strategies providers encounter in sustaining program fidelity while implementing HF across multiple state contexts in the United States. Our research questions are as follows: a) What are the challenges frontline staff and supervisors perceive in delivering services using an HF approach? And b) What strategies are employed to maintain fidelity to HF over time?

Methods

Study Sites

The sites for this study were purposively sampled as part of a larger parent study of Housing First that included focus group (FG) interviews conducted at three HF programs that were closely aligned to the original model developed in New York City and were beyond the initial stages of implementation. These sites were selected as exemplars because they faced challenges and yet sustained their alignment to the underlying theory of the model as evidenced by their mission statements and public documents. All participant sites served similar populations of adults with current or past homelessness and a diagnosis of a serious mental illness. Since their inception, the number of tenants housed ranged from 230 to 650 across the three sites. Initial funding for these programs was derived primarily from a combination of federal and state government sources. Government sources continue to comprise 90–95% of program revenue, which includes Medicaid reimbursement for mental health services and United States Department of Housing and Urban Development (HUD) funding financing rental subsidies. The remaining 5–10% of the programs’ revenue is comprised of tenant payments and philanthropic support. The program sites were located in the Mid-Atlantic/Northeast region of the United States. Two programs were situated in dense urban areas and the third site spanned rural and urban settings. All three programs had been operating for at least eight years, thus demonstrating their maturity and later stages of implementation.

Recruitment

The study’s principal investigator (the fifth author) reached out to the executive directors of the HF programs of interest. The purpose of the study was explained, and approval was requested to conduct focus groups on-site with program case managers and supervisors. After obtaining this approval, recruitment flyers were distributed at the three participating programs. Recruitment targeted all frontline providers (both full-time and part-time) and their clinical supervisors. Staff interested in participating were asked to contact the study’s research coordinator who scheduled the on-site focus group interviews at a convenient time and private location at the program site.

Data Collection

Prior to initiating each focus group session, the written informed consent protocol was discussed and reviewed with attendees. Participants were also asked to maintain confidentiality regarding what was discussed in the groups. Individuals were provided with light refreshments and given a remuneration of $20 each for their participation. In addition, participating programs received a $500 institutional fee for their assistance. All study procedures were approved by the authors’ university Institutional Review Board.

Focus group sessions lasted about one hour and ranged in size from five to seven participants for the case managers and three to seven for the supervisors. All program supervisors and available case managers elected to participate. The groups were conducted by three members of the research team: each with a role as primary facilitator, note-taker and observer, or logistics coordinator. Interviewers had prior research and/or clinical experience with HF programs, which informed the development of the guide, afforded them greater sensitivity to participant experiences, and effectiveness in using probes. Observational notes were used to track individual participation, intra-group dynamics and nonverbal behavior. These notes enabled us to have a contextual understanding of the groups in situ as well as be able to properly identify who was speaking for transcription and subsequent analyses.

Separate interview guides were developed for case managers and supervisors. Focus groups with case managers concentrated on understanding their views about the clients they serve, their service delivery approach (e.g., degree of alignment with HF principles of client choice and empowerment), and what they think is needed to enhance services. Case manager guides included questions such as, “Is there anything that you have to address with consumers, whether or not consumers want it to be addressed? Anything that challenges your ability to honor consumer choice around the services received here?” Supervisor focus group questions centered on their overall approach to service delivery including their supervisory role and any factors that would help to improve HF services. In addition, interviewers probed for barriers and strategies at the individual, agency, and community level.

The focus groups were audio-recorded, transcribed verbatim, and uploaded to Atlas.ti on a secure server for subsequent analysis; transcripts were reviewed by the group facilitators for accuracy and quality. Transcripts were de-identified and each participant was assigned a unique identification number. Immediately after each focus group session, reflective memos were written to document initial reactions and significant points and questions were generated for further exploration in later focus groups. The research team met regularly to debrief on the focus groups and their progress.

Analytic Strategy

Following Fereday and Muir-Cochrane’s study (2006), we used a hybrid approach to thematic analysis employing both deductive and inductive coding procedures. The primary domains of HF fidelity (housing choice and structure, separation of housing and services, service philosophy, service array, and program structure) constituted a priori themes whose narrative content was inductively analyzed using content analysis to identify challenges and strategies. With this inductive process, the first two authors independently content analyzed separate segments of the focus group data. This allowed for two “processes of discovery” and to “finetune” any a priori sensitizing theories and concepts (Morgan, 1996).

After preliminary independent reading and coding for content, the authors came to consensus on emergent sub-themes arising from the data regarding challenges and strategies to sustaining model fidelity (Boyatzis,1998). Following Miles et al., (2014), a matrix of strategies and challenges stratified by the PHF fidelity criteria aided in identifying connections (see Table 1)

Table 1.

Provider Experiences of Challenges and Strategies to Sustaining Fidelity Stratified by Criteria

Fidelity Criteria	Challenges	Strategies
Housing Choice & Structure	Market conditions: Housing Status Quo of Service Delivery	Community Engagement
Separation of Housing & Services	Funding Requirements: Service Mandates	Rule-Bending
Service Philosophy	Status Quo of Service Delivery Dilution of PHF Practice	Supervision, Training, & Support Technology
Service Array	Funding Requirements: Administrative Burden, Single Occupancy Market Conditions: Job Market	Rule Bending Community Engagement Supervision, Training, & Support

Strategies of rigor were employed throughout the data collection and analysis process (Padgett et al., 2016; Padgett, 2017; Hamilton et al., 2018). An audit trail of notes recorded during and directly after data collection, memo-writing during analyses, and other documentation were completed to support transparency. In addition, authors met for peer-debriefing throughout the data collection and analysis processes to share findings and mitigate any potential biases in data collection and analysis. The Standards for Reporting Qualitative Research checklist was used (see Appendix 1) (O’Brien, Harris, Beckman, Reed, & Cook, 2014).

Results

Sample Characteristics

The sample (N=33) included supervisors (N=16) and direct care staff (N=17) across six focus groups at the three sites. Of the total, 60% identified as female, 76% identified as white, 18% identified as black/African-American, and 6% identified as other.

Themes

Findings suggested providers experienced challenges and strategies to PHF in four of the five domains of PHF fidelity (program structure was less relevant). Across these fidelity domains, providers identified four primary barriers to PHF fidelity that included 1) market conditions (i.e., availability of affordable housing and employment), 2) status quo of service delivery, 3) funding requirements and priorities, and 4) the dilution of the PHF practices across staff members. To mitigate these barriers, providers pointed to four strategies they used to compete for, and sustain, scarce resources: 1) community engagement; 2) staff supervision, training, and development; 3) rule-bending to meet demands of multiple stakeholders; and 4) technology. Within each of the relevant fidelity domains these strategies and barriers were experienced in different ways and discussed with varying intensity. Table 1 below outlines the themes within providers’ reported experiences of challenges and strategies to sustaining fidelity to PHF stratified across the fidelity criteria. Interestingly, there were no differences across the sites despite their geographic dispersion.

Housing choice and structure.

The primary barriers to the delivery of the housing choice and structure requirements of PHF were the market lacking affordable housing in all contexts and the challenge of the status quo within community service settings. The lack of affordable housing limited opportunities for service users to choose the best community for their health and their recovery. This provider shared, “It [affordable housing] dwindles, and then there’s a – there’s specific places where our consumers can afford to live. Which is not the best place, and it’s not the best place for their recovery, their process” (P5, case manager, FG3¹). PHF programs struggled to compete in these market conditions for the limited housing stock and to fulfill the housing preferences of service users. In addition, providers identified that challenging the status quo within the service systems and communities was another barrier to providing service users with choices in their housing. When housing and service providers disagreed with the PHF harm reduction approach, participants described that services or housing were withheld that were needed to provide genuine consumer choice. This participant discussed the challenge of operating a program with a new practice ideology in communities that traditionally approached services differently: “We have had somewhat of a rough go of breaking into communities using our different approach than like what’s been the status quo … we’ve faced a lot of challenges in just getting different agencies – the system of care that currently exists to police in crazy ways – to just accept the methodology that we’re using as something different, but also something that works” (P2, Supervisor, FG6). This shift away from traditional service continuum (e.g., expectation of symptom reduction or abstinence before housing) was experienced as an uphill battle at times and negatively influenced potential collaborations with service systems and landlords.

Providers reportedly combatted these barriers to securing permanent housing using dedicated staff that worked diligently to engage with communities to build relationships. In particular, housing staff offered landlords assurances of responsiveness and guaranteed rent payments to compete for their units. For this provider, accessibility was important to relationships with landlords: “So using all forms of communication …I think landlords appreciate it. So we totally are getting text messages from landlords, and they have our numbers” (P2, case manager, FG3). Having housing staff separated from clinical staff members allowed for this improved responsiveness to landlord needs without compromising the clinical staff relationships with the service users.

Using these strategies was considered critical to the development of community engagement necessary to combat adverseservice users. market conditions and facilitate housing choice for This participant shared how they had to adapt to particular community settings: “In order to provide uniformity of services, we’ve had to go into those communities, adapt to…their own culture” (P2, supervisor, FG6). Selecting and hiring dedicated staff already embedded in the communities where they provided services truncated the learning process and eased implementation. This participant shared, “we hire staff that are local to the area and who know the territory a little bit … and maybe have been consumers of those services themselves at one point in time” (P1, supervisor, FG6). In addition, the programs directly engaged community members through organizing community meetings and regular facility tours.

Separation of housing and services.

The separation of housing from service participation in HF means that housing is not contingent on service users’ meeting clinical or treatment requirements. However, funders’ requirements sometimes presented a barrier to achieving this fidelity criterion. For example, one funding requirement made eligiblity for the housing program contingent upon the client meeting with a psychiatrist. Providers discussed bending the rules in such instances to meet funding requirements while still delivering user-driven services. For example, participants reported instances of bypassing the required psychiatric assessment when their needs assessment concluded that a person would otherwise not receive critical services or shifting the psychiatric assessment from first contact to subsequent to engagement in order to retain clients. In response to the same issue, this participant discussed the skillful approach used to insure service engagement despite the new funding requirement. “That is a challenge now that we’ve got a different funding system. How we maintain that kind of belief that someone can have housing without that, while still needing to have a psychiatric assessment to be done, in order to provide any services… we’ve…gotten creative… [our Medical Director] is very good at framing his interactions with our clients as something other than the most clinical of assessments.” This participant also described strategies used as matching what these requirements looked like in practice including, “the way we operationalize the work we do, and then structure and how to track all of this stuff, we have to bill all of that now” (P3, supervisor, FG6). These discussions were facilitated through the daily team meetings that providers used to navigate this gray area.

In addition, the single-occupancy requirement of funders presented a tension for the separation of housing and services when services staff learned of co-habitating partners while delivering services. Participants reported that their response was often to provide education about the potential impact on their housing given the tenancy agreements. This participant shared,

“And I think with all things, whether it’s housing or anything else, I just try to, you know, like you were saying, provide education…. “This is your choice. This is the information that we have, this is what you have experienced. You have a choice to make.” And I’ll support you in what that choice is, but…P1: I think what can be hard at times is with keeping the housing and the services separate”

(P3 and P1, case manager, FG3).

In these situations, participants reportedly used their discretion to notify housing or not depending on their assessment of the situation.

Overall, participants explained these types of strategies were necessary in order to remain faithful to their priority and mission of engaging and serving those individuals who fall through the gaps. The strategies also highlight the importance of flexibility and space within programs to foster innovative responses to funding requirements that stood in opposition to sustain fidelity to the separation of housing and services domain.

Service philosophy.

Challenges regarding the status quo of service delivery once again emerged as participants discussed critical barriers to enacting the PHF service philosophy, which specifies the need for programs to operate services aligned with service user preferences, recovery, and harm reduction. In particular, participants discussed barriers to instilling, maintaining, and applying embedded HF values and approaches across all program staff. The harm reduction model central to HF was of particular concern to some staff. This participant shared,

“one of the biggest barriers… is the line we draw between harm reduction, how we’re going to actually do that for that client, and our personal biases…like what’s best for the client, what they want, and then still upholding what you know is going on there”

(P5, case manager, FG1).

With the majority of frontline providers’ time spent alone in the community with service users, supervisors expressed concern over their ability to support and reinforce the PHF philosophy. At the state-wide PHF site, geographic distance introduced additional concerns. This participant stated, “that diffusion of culture is probably one of our bigger challenges in terms of being a state-wide agency, and ensuring we create a group of like-minded people who will…go to the same response consistently” (P1, supervisor, FG6).

Participants reported multiple mechanisms through which they worked to maintain the potency of the PHF approach. Central to these strategies was the use of the team leader and direct supervisor to enact and support the behavior. This participant shared that her quality improvement team monitored staff notes: “If we see something that is the direct opposite of harm reduction, you can best bet I’m sending an email to [team leader] that’s like, you better fix this” (P5, supervisor, FG4). Team leaders were responsible for providing feedback and coaching staff to embed this philsophy in their work. Face-to-face time between supervisors and staff members was a key mechanism to ensure that service users were receiving harm reduction services. This participant shared, “we have a meeting every morning, so there’s certain supervisory sort of discussion that happen in that group setting. And then we do have a clinical group supervision every day … supervision is kind of built into our model” (P2, supervisor, FG4). However, participants also discussed that these supportive and educational functions were challenging to provide consistently given the constraints on supervisors’ time. This participant stated, “if we do have the time, it’s ideal to be able to put some time in for shadowing purposes, and for teaching and learning … at least just in my experience that’s been extremely important to kind of absorbing the mentality behind Housing First” (P3, supervisor, FG6).

While team meetings were helpful when everyone was in the office, technology facilitated off-site support as providers worked with service users alone in the field for the majority of the day. Technology allowed staff to stay connected to team members, obtain information about service users, respond to any barriers, and translate the philosophy into their day-to-day practice. Computer tablets with Wifi capability, smart phones with unlimited text messaging, and video conferencing were the tools used to facilitate this work in the community. They also served to maintain connections between supervisors and staff that supported and reinforced the PHF approach. Team members were able to use group text messages to access the knowledge and support of everyone on the team and they used video-conferencing technology to include everyone in staff meetings even if they were remotely located. One participant stated, “my team made a point of answering anything that I texted… very quickly so that I could get the information I needed” (P2, case manager, FG3) Beyond the walls of their offices, staff had an open door to consultation as they worked to integrate the PHF philosophy into their daily interactions with service users.

Service array.

The PHF model encompasses a wide array of services to support community integration and a wide variety of service users’ needs and individual recovery goals. These include housing support, psychiatric services, addiction treatment services, supported employment, nursing, social integration activities, 24-hour accessibility, and referrals to inpatient treatment if necessary. However, participants identified funding requirements as a main overarching barrier within this domain, along with market conditions related to employment. In addition, funding shifts required additional monitoring and reporting functions, which competed for limited time to provide the required service array. One participant stated, “it’s like our job is to work with people, not to count the dollars. But it also is now to count the dollars” (P1, supervisor, FG6). In addition to administrative burden, participants noted that discrepancies arose creating a gap between needed versus reimbursed services.

This dissonance led to the development of other rule-bending strategies, combined with supervision, training, and support, to improve the documentation process and accurately represent their work. During onboarding of new staff, successful documentation practices were included in orientation and a stepped productivity expectation accounted for that learning curve. A participant stated, “billing [for this funder] is an art and it takes a lot” (P5, supervisor, FG4). This challenge of documentation for billing purposes required ongoing creativity and training. In working with service users, their role was building relationships to identify recovery goals and then: “let’s make it [funder] friendly. And then we learn the lingo. So it’s kind of making it true to the program and true to them” (P1, case manager, FG5). Another participant shared, “I kind of compartmentalize, like…, I just I do the [PHF] model when I’m with the client. And then when I have to do my note, I’m like, “Okay, I am… on [funder] time right now” (P2, case manager, FG5).

In addition to documentation barriers, funding requirements for the single-occupancy units leased by the PHF programs challenged the provision of social and community integration services. For example, single-occupancy requirements created a barrier for service users interested in co-habiting with a significant other. One participant shared: “I mean that’s the natural thing: you date someone for awhile, you move in together– in your normal situation your housing then does not become jeopardized” (P4, case manager, FG1). Participants mentioned rule-bending again and reportedly used their discretion regarding what the programmatic response was to a second occupant in the apartment.

“… We allow it, especially when it’s a positive relationship and it helps that person to function better, to do better. P3: We don’t allow it. P4: Right. You turn a blind eye. P1: Right, we look the other way… Facilitator: Who’s making the call of whether you’re turning a blind eye or not? P3: The team. P5: The team, but I think to me it’s one of those issues here that has always been gray.”

(P3, P4, P1, case managers, FG1).

Participants also shared that the rule can be helpful to avoid contending with the complications that a second occupant could raise. As one participant shared: “No agency ever fights that rule. Facilitator: Why? P4: Because it makes our job easier. …P4: Even if for one client it was beneficial, in the big picture, in the grand scheme of things it makes our job easier, and safer, to be quite honest” (P4, case manager, FG1). In this way, providers tended to support the rule’s existence in order to apply it with discretion depending on the impact of the potential second tenancy on the primary service user.

In addition to funder requirements, the inaccessibility of the job market was a barrier to programs providing employment-related services. Limited availability of positions amenable to entry-level skills or to large gaps in employment histories challenged providers’ ability to connect service users with paid work. Greater community engagement with potential employers was seen as necessary for helping service users with employment. Participants discussed what would help: “… Employers who might be open to working with an agency that supports somebody who has not had an employment history--that would be big. “ (P1, P2, supervisors, FG6). Service users in these programs were typically returning from long periods of homelessness and unemployment to a changed employment landscape and had limited skills to adapt to new opportunities. Similar to the housing staff’s work with landlords, providers worked to develop relationships with employers and offer ongoing support to the employer and employee once a position was secured, with the recognition that additional resources were needed to support and expand community engagement.

Discussion

This report reveals that some—but not all—of the challenges attending later-stage implementation are similar to those confronted in earlier stages in different political-economic contexts. External challenges of securing apartment units in a competitive housing market, confronting local service norms that hinder HF implementation, and coping with shifts in funding required creative internal responses at the organizational level are consistent with those in the Canadian demonstration projects (Nelson et al., 2017). The interplay between the intervention, the external environment, and the internal organization highlights the dynamic, multilevel nature of sustaining the intervention’s fit to the context over time and the actions needed to bridge tensions (Padgett, 2016). Yet, the intervention itself does not target broader social, political, and economic contexts. Consequently, similar to other cross-system interventions and human service organizations, HF must be paired with strategies to remain agile.

Within the PHF programs in this study, strategies of community engagement, harnessing the influence and training capacity of team leaders/supervisors and the teams, and rule-bending served to facilitate ongoing sustainment of fidelity. One of the more productive strategies pursued at all three sites was pro-active community engagement to attract knowledgeable staff as well as increase local buy-in and support. Lacking the organizational capacity of the VA or the financial resources that come with a well-funded randomized trial, these HF programs had to “look outside” to ensure financial stability and community acceptance as well as to attend to the dynamic needs of a growing organization and client population. While the two urban sites were dependent on local municipal policies, the urban-rural site had to contend with both city and state authorities to ensure state-wide coverage of HF programs. Our findings suggest that new challenges could arise—including a lessening of federal or local support for HF initiatives (Padgett, 2016).

Internal to the organizations, on-the-ground provider strategies to mitigate system-level challenges to fidelity found in this study are consistent with previous research (Lipsky, 2010; Spitzmueller, 2016). In particular, using discretion to resolve tensions between the service philosophy (e.g., person-centered care) and funding requirements are notable. Providers working in Medicaid-funded supportive housing identify street-level dilemmas that include “putting the ‘consumer first’ vs. achieving maximum billing” and “doing the ‘real work’ vs. paperwork” (Authors, 2019; Andvig, Sælør, & Ogundipe, 2018). Previous research (Authors, 2018) has also noted funders’ single occupancy standard as a barrier to realizing the service user-driven philosophy inherent to the HF model and describes the same frontline response of “turning a blind eye” to service user violations of these requirements. This body of work suggests that program models like HF that are intended to be individualized and service user-driven may be in conflict with some reimbursement practices, such as Medicaid fee-for-service in the United States. As such, providers may be expected to sustain fidelity in spite of this poor fit.

Supervisors and case managers were generally in accord in their perceptions of implementation challenges. While supervisory roles afforded opportunities for role modeling and leadership, case managers were better positioned to enact the service user choice values of HF in their interactions with clients. Supervisors charged with enacting organizational policies and practices in day-to day functioning were tasked with fidelity oversight and championing the intervention amongst their staff near and far. Ongoing vigilance and commitment were required of supervisors to sustain practice norms (Authors, 2018). Supervisors’ roles included helping staff to mitigate day-to-day challenges, selling the intervention to staff in-person or via technology, and creating a team climate that supported fidelity adherence, which is consistent with theoretical development in implementation scholarship (Birken, Lee, & Weiner, 2012; Ehrhart, Aarons, & Farahnak, 2014; Klein & Sorra, 1996). In addition, supervisory efforts with staff to make sense and reconcile PHF practice with shifting requirements demonstrates the work of sense-making, a theorized key mechanism for embedding an intervention in routine practice (May et al., 2009).

This study addresses the “sustainability gap” in the literature (Glasgow et al., 2012) utilizing qualitative methods to explore “real world” practice-based strategies to sustain EBI in a notoriously low-resource setting—homeless services. Homeless services have predominantly served people from historically under-represented and disenfranchised racial and ethnic groups. This study also provides support for the assertion made by Chambers et al. (2013), that “harnessing the understanding of context can enable beneficial adaptation of the intervention and improve sustainability” (p. 3). These findings point to potential multilevel targets for adaptation of the intervention and the context to sustain fit. In particular, targeting particular funding requirements (e.g., diagnostic psychiatric assessments or single-room occupancy requirements) for evaluation in the context of HF programs would aid sustainment. In addition, these findings suggest potential tension points between adaptation and fidelity that could benefit from a proactive adaptation plan in order to attend to the underlying core conceptual model of HF (Baumann, Cabassa, & Stirman, 2018). Even in these low-resource settings, HF as an EBI can be and is being sustained by committed implementers adapting to, and influencing, local contexts and service-user needs in order to sustain fidelity to the model over time.

These findings should be evaluated in the context of the study strengths and limitations. This multisite and multistate examination harnessed an opportunity to learn from these naturally occurring exemplar sites sustaining fidelity over time without significant external resources. Because of this focus on exemplar sites, the challenges they faced may be different than other organizations who did not so successfully remain aligned with the original philosophy of the model. In addition, this study employed several strategies for rigor in qualitative methods recommended for qualitative research in implementation science (Padgett, 2017). However, while the programs were identified and sampled as exemplar HF sites, evaluation of fidelity scores were not within the scope of this study, protocols did not explicitly aim to elicit participant experiences of each of the fidelity criteria, and available fidelity scores were not concurrent with study data collection. Adaptations from the PHF model within routine settings would be important for future inquiry that was not possible given the sampling strategy. In addition, participant sampling relied on volunteers in order to avoid coercion. However, all supervisors elected to participate and all case managers who were interested were included in the study.

Conclusions

This study found that challenges to sustained fidelity mirrored those encountered in other studies of implementation phases. However, absent external support found in initial demonstration projects, these sites maintained ongoing fidelity with multilevel sustainability strategies. These strategies relied on the ongoing vigilance and commitment of key staff within the organizations. In particular, the supervisory role was critical to overseeing street-level enactment as well as community engagement and training strategies. Novel here is a story of how individual providers in a low-resource setting were committed to fidelity and worked to sustain it through continued responsiveness to fit the intervention to the dynamic context.