The adaptation and fidelity tool to support social service practitioners in balancing fidelity and adaptations: Longitudinal, mixed-method evaluation study

Abstract

Background

Evidence-based interventions (EBIs) seldom fit seamlessly into a setting and are often adapted. The literature identifies practitioners’ management of fidelity and adaptations as problematic but offers little guidance. This study aimed to investigate practitioners’ perceptions of the feasibility and usability of an intervention aimed to support them in fidelity and adaptation management when working with EBIs.

Methods

The intervention, the adaptation and fidelity tool (A-FiT), was developed based on the literature, along with input from social service practitioners and social services’ Research and Development units’ personnel. The intervention consisted of two workshops where the participants were guided through a five-step process to manage fidelity and adaptations. It was tested in a longitudinal mixed-method intervention study with 103 practitioners from 19 social service units in Stockholm, Sweden. A multimethod data collection was employed, which included interviews at follow-up, questionnaires at baseline and follow-up (readiness for change and self-rated knowledge), workshop evaluation questionnaires (usability and feasibility) after each workshop, and documentation (participants’ notes on worksheets). To analyze the data, qualitative content analysis, Kruskal–Wallis tests, and Wilcoxon rank-sum tests were performed.

Results

Overall, the practitioners had a positive perception of the intervention and perceived it as relevant for fidelity and adaptation management (mean ratings over 7.0 on usability and feasibility). The workshops also provided new knowledge and skills to manage fidelity and adaptations. Furthermore, the intervention provided insights into the practitioners’ understanding about adaptation and fidelity through a more reflective approach.

Conclusion

Practical tools are needed to guide professionals not only to adhere to intervention core elements but also to help them to manage fidelity and adaptation. The proposed A-FiT intervention for practitioners’ management of both fidelity and adaptation is a novel contribution to the implementation literature. Potentially, the next step is an evaluation of the intervention's impact in an experimental design.

Plain Language Summary

This study describes practitioners’ perceptions of an intervention that aims to support them in fidelity and adaptation management when working with evidence-based interventions (EBIs). This is an important issue because social services practitioners are expected to use EBIs that seldom fit seamlessly into a specific setting and are often adapted. The practitioners perceived the intervention as relevant for their fidelity and adaptation management and states that it helped them develop a plan and increased their knowledge on the topic. Professionals require practical guiding tools not only to adhere to intervention content but also to balance them with fidelity and adaptation. This proposed intervention for practitioners’ management of both fidelity and adaptation is a novel contribution to the implementation literature. We propose that researchers further evaluate this intervention as a potential next step.

Introduction

Policy makers, managers, and citizens expect social services (e.g., support for at-risk children and families, individuals with disabilities, and people with substance abuse problems) to provide resources that have been shown to be effective (Escoffery et al., 2019; Kirk et al., 2020; Miller et al., 2020; Morago, 2010; Plath, 2006). However, using evidence-based interventions (EBIs) has been challenging (Bäck et al., 2020; Mosson et al., 2017) because EBIs seldom fit seamlessly into a specific setting (Miller-Day et al., 2013). The context in which an EBI is developed and tested often differs from the context in which it is applied. For instance, provider organizations differ in resources, and practitioners differ in training. In addition, welfare systems often vary in funding and regulations, and populations and service user characteristics can be different (e.g., culture, values, and level of severity; Aarons et al., 2011). These incongruences mean that either EBIs end up being used in contexts that differ from where they have been shown to be effective or the content, delivery, and format are adapted to fit the new context (Aarons et al., 2012; Mosson et al., 2017; Stirman et al., 2013; von Thiele Schwarz et al., 2019).

Adaptations of EBIs can have beneficial effects for individual clients and groups (Aschbrenner et al., 2020, 2021) and might be necessary when implementing an EBI. However, adaptations can also be problematic (Durlak & DuPre, 2008) because they can cause an EBI to be less effective or to even inflict harm. For instance, adding ineffective elements to an EBI such as scare tactics in prevention interventions (Lipsey & Wilson, 1998) can threaten the effectiveness of an EBI (Mihalic, 2002). Adaptations can also induce unwanted variation in the services provided compared to the original EBI and between different service users. Therefore, adaptations can increase the risk of inequalities in service provision (Elliott & Mihalic, 2004). The balance between fidelity and adaptation is often described as a dilemma that reflects the challenges involved in determining the extent to which an EBI needs to adhere to the original version or to adapt in response to restraints and possibilities in the local setting to optimize the outcome (Castro et al., 2010).

The literature shows that adaptations are often conducted spontaneously in response to constraints in the context (e.g., time pressure; Aschbrenner et al., 2020; Kakeeto et al., 2017; Moore et al., 2013; Stirman et al., 2013). This is problematic because while proactive adaptations have been associated with better client outcomes, reactive adaptations may not be (Cooper et al., 2016; Moore et al., 2013). For example, reactive adaptations may be more prone to being made in response to practical restraints (e.g., to get the EBI in place) rather than to provide a better fit for the target group (Miller-Day et al., 2013). In addition, decisions are often made by individual's reasoning rather than based on discussions within work teams, which can increase the risk of unwanted variation and result in inequalities in the provided services (Wiltsey Stirman et al., 2015). Moreover, adaptations are often made without careful consideration of how these will affect the outcomes for clients (Cooper et al., 2016) and how they will affect the EBI's core components, in other words, without ensuring that the core elements that make the EBI effective remain (Wiltsey Stirman et al., 2015).

The literature offers little guidance for how practitioners should manage fidelity and adaptations. The existing models and frameworks are primarily geared for researchers, guiding coherent description and classification of adaptations (Kirk et al., 2020). A review of adaptation process frameworks identified 13 frameworks that aim to guide translation from a research context to practice. However, these frameworks primarily focus on the early steps of the research-to-practice pathway, when researchers are often involved (e.g., community assessment; Escoffery et al., 2019). The adoption decision is frequently made locally by practitioners, who are left to make decisions related to fidelity and adaptation. This calls for practical tools that are geared at building practitioners’ capacity to manage fidelity and adaptation (Hasson & von Thiele Schwarz, 2017).

In sum, the balance between fidelity and adaptation is an inevitable part of EBI implementation in social services, and no practical support is currently available to help practitioners manage this balance. Thus, there is a need to explore how practitioners in social services can be supported in managing fidelity and adaptation in daily work.

Aim and Research Questions

The study aimed to investigate practitioners’ perceptions of the adaptation and fidelity tool (A-FiT) aimed to support them in fidelity and adaptation management when working with EBIs.

The following research questions were addressed:

• How do the social service practitioners perceive the usefulness of the intervention?

• Are there any changes on the practitioners’ knowledge and behaviors related to fidelity and adaptation after participating in the intervention?

Method

Design

This study uses a longitudinal convergent parallel mixed method design. Qualitative (documents) and quantitative (questionnaires) data were collected during the intervention and analyzed independently before integration. In addition, interviews were conducted approximately 3 months after the intervention. Although this is an intervention study, the objective is not to conduct an effect evaluation. Instead, we follow calls in evaluation science to develop evaluations focusing on establishing the feasibility of initiatives before making formal evaluations (e.g., Armstrong et al., 2011). We used the Template for Intervention Description and Replication checklist (Hoffmann et al., 2014) and the Transparent Reporting of Evaluations with Nonrandomized Designs statement (des Jarlais et al., 2004).

Setting

The study was conducted in municipalities in Stockholm, Sweden. Municipalities are the entities responsible for providing social services, including social support for children and families (preventive and at-risk), individuals with mental health problems and social issues, people with disabilities and substance abuse problems, and older people in need of social security benefits and at-home help services.

Development of the Intervention

The intervention was developed in collaboration among academic researchers (Henna Hasson and Ulrica von Thiele Schwarz), embedded researchers at social service Research and Development (R&D) units (Åsa Hedberg Rundgren, Håkan Uvhagen, Charlotte Klinga, Helena Strehlenert, and Anna Gärdegård), and social service organizations. The R&D units were Stockholm Gerontology Research Centre, Stockholm Health Care Services Research and Development, Research and Development Unit for Elderly Persons, and Center for Epidemiology and Community Medicine. This approach combines academic knowledge with practical knowledge, as suggested by Lee et al. (2008).

Social service organizations invited through the R&D units’ established channels were involved to identify intervention goals. A structured adaptive reflection process was used (February 2019). Adaptive reflection is a technique used in higher education to create a common understanding of learning goals (Savage, 2011). A total of 25 practitioners gathered in a workshop that began by prompting individuals to reflect on the skills and behaviors required for a social service practitioner to manage fidelity and adaptations to EBIs. The individuals’ input (e.g., understand the logics of an EBI, be able to use the EBI within the legal requirements, and follow up the outcomes) was documented on Post-it Notes, which the participants jointly sorted into meaningful categories based on similarity. Thereafter, the participants discussed the categories and gave each an appropriate heading (e.g., understand contextual influences and be able to follow-up). In this way, the categories represented the desired intervention outcomes from the social service perspective.

Based on these intervention outcomes and scientific literature on fidelity and adaptation (e.g., Hasson & von Thiele Schwarz, 2017; Lee et al., 2008; Stirman et al., 2019), we developed a prototype for the intervention. The prototype applied a proactive approach to fidelity and adaptations with a stepwise approach (Hasson & von Thiele Schwarz, 2017; Lee et al., 2008). Possible adaptations to consider included aspects for EBI content and format (e.g., adding, shortening, and reordering) as well as context (e.g., personnel; Miller-Day et al., 2013; Moore et al., 2013; Stirman et al., 2019). The intervention prototype applied a team approach to solve fidelity and adaptation questions as suggested by Hasson and von Thiele Schwarz (2017).

The prototype was pilot-tested with six work teams from social services organizations from August to December 2019. The research team took notes, and based on the notes, the prototype was revised until no further comments arrived from the participants. Revisions concerned minor aspects, such as wording, as well as two larger amendments: changing the order of the steps (i.e., identifying and defining desired outcomes for the EBI became the first step) before focusing on the content of the EBI (see below) and adding a step concerning implementation strategies and monitoring of the implementation process. Thus, the step on how to follow-up outcomes was introduced earlier than that suggested by Lee et al. (2008)

Intervention

The intervention consisted of two workshops (3–4 weeks apart) where 10–15 work teams (each comprised two to four practitioners and their manager) participated. The teams were guided through the stepwise process to manage and reflect on fidelity and adaptations by facilitators (i.e., members of the research team; Table 1). The stepwise process included five main steps: (1) identifying and defining desired outcomes for the EBI, (2) deciding how to follow-up on these outcomes, (3) identifying and comparing differences in outcomes and contexts between the original use of the EBI and the teams’ own intended use, (4) deciding what needs to be adapted and how these adaptations can be made without changing core elements, and (5) planning how to implement the adapted version of the EBI. Each team worked with an EBI as a practice case (i.e., an EBI that they planned to implement in their organization). Thus, the participating teams worked with different EBIs (see EBI examples in Table 3). Although the participants were required to have chosen a specific EBI prior to the intervention, they were encouraged to continuously reevaluate the adoption decision if the core elements of the EBI could not be preserved.

Table 1.

Description of the Intervention

Timepoint	Step	Activity
Workshop 1		Introduction to the topic: short lecture on the definition of fidelity and adaptations and the dilemma associated with it.
	Step 1: Define outcomes	Individual brainstorming: clarify desired EBI outcomes and potential unintended consequences of using the EBI in their own organization.
		Teamwork: discuss and sort desired EBI outcomes into meaningful categories, agree on goals of implementing the EBI.
	Step 2: Plan for follow-up of outcomes	Short lecture on how to follow-up desired EBI outcomes.
		Teamwork: identify relevant outcome measures (e.g., survey instruments, register data) and desired target levels.
		Cross-team reflection on follow-up.
Between Workshops 1 and 2		Teamwork assignment: how to anchor goals and follow-up plan among colleagues, search for more knowledge about the EBI (if needed). Support available from the facilitators.
Workshop 2		Cross-team review of between-workshop assignment.
	Step 3: Compare outcomes and contexts	Short lecture on how to manage fidelity and adaptations.
		Teamwork: analyze similarities and differences between outcomes of the original EBI and their own desired outcomes, and between context of the original EBI and their own context.
		Cross-team reflection on comparing original EBI with their own situation.
	Step 4: Decide about fidelity and adaptations	Teamwork: decide on fidelity to and adaptations of EBI content based on analysis of similarities and differences between the original EBI and their own situation.
	Step 5: Plan for implementation	Short lecture on implementation strategies and how to analyze barriers and facilitators.
		Teamwork: analyze strategies used to implement the original EBI and plan for implementation and Follow-up the implementation process.
		Cross-team reflection on planning for implementation.

Note. EBI = evidence-based intervention.

Table 3.

Description of the Three Intervention Groups

Intervention group characteristics	Group 1 March (physical)–November 2020 (digital)	Group 2 October–November 2020 (digital)	Group 3 March 2021 (digital)
Amount of work teams (individual participants)	5 (22)	3 (14)	11 (67)
Type of social service organizations	Family center, prevention unit, outpatient care center, national social work network, family support unit, domestic violence unit	Disability center, income support, substance abuse unit, network team, administrative unit	Elderly care
EBIs implemented	Postnatal home visiting program, gaming disorder prevention program, feedback informed therapy (FIT)	IBIC, “Intern-SIP”—Model for internal collaboration, Group Session Rating Scale (G-SRS (Duncan & Miller, 2007)	Reagera-s (Responding to Elder Abuse in Geriatric care; Simmons et al., 2020), the Behavioural and Psychological Symptoms in Dementia registry

Note. EBIs = evidence-based interventions.

The pedagogy of the workshops was based on the theory of experiential learning implying that participants related their reflection on the theoretical concepts of fidelity and adaptations to their concrete experiences, to advance learning and new actions (Richter et al., 2016). Thus, short lectures (of about 10 min) were combined with practical teamwork sessions using structured worksheets (in paper or digital format; Supplemental File 1). The facilitators were available during the teamwork sessions to provide support. Cross-team reflection sessions offered opportunities to meet across teams and give and obtain feedback (i.e., enabling peer learning). The teams were given between-workshop assignments and tailored support from the facilitators when needed. The intervention was expected to result in a concrete plan for managing fidelity and adaptations.

Due to the COVID-19 pandemic, most of the workshops were conducted via the video meeting platform Zoom. In addition, some of the short lectures were made available in video format to support between-workshop learning.

Participants

Recruitment of the social service teams was done by the local R&D units through their established communication channels (e.g., newsletters and email lists). Each work team showing interest in participating was interviewed by a research team member to see if it met the two inclusion criteria (Table 2). Three rounds of the workshop series (i.e., three intervention groups) were carried out over a period of 1 year (from March 2020 to March 2021).

Table 2.

Inclusion Criteria for Participation in the Intervention

Inclusion criteria	Definition
Implement an EBI	An ongoing or planned implementation project that can be used as a practical case. EBIs were defined as practice that has been shown to be effective in scientific evaluation. Nevertheless, we did not exclude any organization based on the level of evidence for a certain EBI, as it can be difficult to determine a good enough level of evidence and some areas of social services also have few EBIs. In all cases, the EBI needed to be well defined with clear descriptions of prior use.
Social service work team	Teams working with the implementation of the EBI in any of the social service areas. At least three (up to six individuals) who can participate in both workshops and will be involved in the management and use of the particular EBI. The work team should include individuals who will be working with the EBI, who know the context in which the EBI will be applied, and preferably also individuals with the mandate to change contextual factors, if needed (e.g., manager).

Note. EBI = evidence-based intervention.

Data Collection

We used three sources of data: interviews, questionnaires, and documentation. The participants received written and oral information about the study aim, what participation entailed, information about data management, and the fact that participation was voluntary.

Interviews

All team leaders were invited through email to an individual semi-structured interview aimed at understanding their perceptions of the intervention's usefulness and how they perceived the intervention to have impacted their way of managing fidelity and adaptations. They were also asked to invite the other team members to participate.

Interviews were conducted through a video application after their participation in the workshops. Audio-recorded informed consent to join the study was obtained from all participants. The interviews were audio-recorded on a digital recorder, transcribed in verbatim, and performed by the fourth, fifth, and sixth authors. The interview guide was based on the worksheets (Supplemental File 1). The interviews centered around the plan created by the participant during the workshops and how the plan had been executed after the workshops. The participants were asked to explain their rationale for their plan and if the intervention had been useful and aligned with their needs.

Questionnaires

All questionnaires were distributed to all participants in each of the workshops. All questionnaires were anonymous and didn’t collect any personal identifiable information.

Baseline and Follow-Up Questionnaires. Participants’ readiness for change was measured before the intervention, and their self-rated knowledge about fidelity to and adaptation of EBIs was measured before and after the intervention. Readiness for change was measured with a six-item scale (e.g., I have high expectations that this intervention can improve my skills to manage fidelity and adaptations) to develop an index (α = .92 for all groups combined at baseline) adapted from Randall et al. (2009). The response alternatives ranged from strongly disagree (1) to strongly agree (10) on a Likert scale. Self-rated knowledge was measured with five items (e.g., I have sufficient knowledge to handle fidelity to and adaptations of new working methods), forming an index (α = .97) modified from Mosson et al. (2017). The response alternatives ranged from strongly disagree (1) to strongly agree (10) on a Likert scale. Furthermore, the participants rated to which extent they had gained new knowledge from the intervention in response to the item “Has this workshop enhanced your knowledge about implementation of new work practices?” on a 10-point Likert scale ranging from no, not at all to yes, and to a very high extent.

Perceptions of the Feasibility and Usefulness of the Intervention. At the end of each workshop, all participants were invited to rate, on a 10-point Likert scale, six items about the usefulness and feasibility of the workshop (e.g., the relevance of the topics, from not relevant (1) to very relevant (10); Mosson et al., 2017; Richter et al., 2016). These six items formed an intervention evaluation index (α = .95). In addition, the participants answered two open-ended questions in the survey concerning their perceptions of the intervention.

Documentation

Documentation produced by the participants (i.e., filled-out worksheets) was photographed or collected at the end of each workshop. The documents were analyzed to obtain information about how the participants identified core components of the EBIs applied, contextual differences, the types of adaptations made, and the extent to which the adaptation align with the goals of the EBIs.

Data Analysis

The qualitative data, including the open-ended questions in the questionnaires, were analyzed by the second and fifth authors using qualitative content analysis (Graneheim et al., 2017; Graneheim & Lundman, 2004). The data were first divided into domains based on the research questions and then divided into meaning units. The meaning units were inductively coded and sorted into manifest categories, which were abstracted into themes. The themes were used to complement and nuance the picture emerging from the quantitative data.

Kruskal–Wallis tests were performed to analyze baseline differences between the three intervention groups regarding readiness for change, self-rated knowledge, and perceptions of the intervention. The baseline differences were analyzed to provide support for the decision to analyze the groups separately or as a whole.

Wilcoxon rank-sum tests were performed to analyze whether the groups’ self-rated knowledge changed over time. The Brunner–Munzel test was used as a complementary analysis for Group 2 to confirm the interpretation of the Wilcoxon rank-sum results. All statistical analyses were conducted using RStudio, with the “psych” and “lawstat” packages.

Results

Participants

A total of 19 work teams participated in the intervention (103 individuals; Table 3). The participants represented a broad range of social services and different EBIs. A total of 75 participants answered the baseline questionnaire (73% of all participants), 51 participants answered the evaluation questionnaire after the first workshop (50% of all participants), and 42 participants (41% of all participants) answered the evaluation questionnaire and the follow-up questionnaire after the second workshop. A total of 32 documents produced by the participants were collected and analyzed, and 13 participants were interviewed (the interviews lasted between 33 and 63 min).

Baseline Differences

No significant differences emerged among the groups in self-rated knowledge at baseline. Group 2 reported significantly lower readiness for change than Groups 1 and 3 did, prompting the decision to analyze the three groups separately (Table 4).

Table 4.

Median Values of Readiness for Change at Baseline

Group number	Readiness for change
	Mdn	SD
1	8.45	(0.65)*
2	6.42	(1.07)*
3	8.0	(1.62)*

* Significant difference between the groups: df = 2, p = .003647.

Perceptions of the Feasibility and Usefulness of the Intervention

The mean values for the intervention evaluation index were over 7.0 for all three groups (on a scale of 0–10) demonstrating participants’ overall positive perception of the intervention (Table 5). No differences among the groups nor between the first or second workshops were found (df = 5, p = .2145).

Table 5.

Mean and Median Values of the Intervention Evaluation Index

Time of measurement	n	M	SD	Mdn
Group 1
Workshop 1	15	8.43	0.72	8.17
Workshop 2	8	7.23	1.12	7.58
Group 2
Workshop 1	7	7.69	0.99	7.57
Workshop 2	7	8.06	1.24	8.00
Group 3
Workshop 1	30	7.04	2.10	7.25
Workshop 2	27	7.62	1.98	8.08

Answers to the open-ended questions in the questionnaires mainly consisted of positive responses stating that the intervention was valuable, important, and rewarding. The participants highlighted the usefulness of the format, especially that of mixing short lectures with practical teamwork on their own cases. In addition, they highly valued the presence of their managers. Suggested improvements were to include more lectures about the implementation process, but at the same time, participants indicated a wish to decrease the number of lectures and increase the time for teamwork. It was suggested that the workshops could focus on one specific EBI rather than different teams working with different EBIs. Participants in Group 3 emphasized the importance of having more information about the workshops before attending.

In the interviews, two themes were identified related to the perceptions of the intervention: (1) useful to have a forum for handling the fidelity and adaptation dilemma and (2) tailoring the format of the intervention to match diverse needs and terms in practice is necessary to provide value. Regarding the usefulness of the forum, the participants stated that participating as a work team and having the manager present were valuable and that having the time to plan the implementation in a structured way was particularly useful. One participant stated:

But through the workshops you got to break down the method step-by-step and plan how you’ll go forward with the approach you want to use and implement it. That was super valuable because that allowed us to dig deeper into “well, we can use this method, but this becomes difficult, how should we do it?” so that we had an ongoing discussion about what challenges could arise. And it was so clear … we received a plan, or framework, how do you say, through the workshop. (Interview 12)

Furthermore, the opportunity to listen to the other participating teams, especially those that implemented the same EBI, was useful. However, there were also disadvantages of the group format, emphasizing the importance of tailoring the intervention to the diverse needs and terms in practice. Some participants had difficulties understanding the information presented due to linguistic challenges. The language of the presentation was perceived as too academic, forming a barrier to understanding the information. Furthermore, the timing of the workshop was sometimes problematic for teams that had already started implementing their EBI, making parts of the workshop less relevant.

Impact of the Intervention on Knowledge and Behaviors

The median value of participants’ self-rated knowledge increased significantly from 6.9 to 8.0 in Group 2 (W = 4, p = .01145) but not in Groups 1 (W = 68.5, p = .8673) and 3 (W = 552.5, p = .1579; Table 6). All three groups rated the item “Has this workshop enhanced your knowledge about implementation of new work practices?” above 6, indicating that the workshops provided new knowledge to the participants (Table 7). In the interviews, the participants gave examples of valuable new knowledge they had gained through the workshop, which included practical knowledge about implementation, such as the value of learning how to define target behaviors and analyze what impacts clients’ behavior. Participants also emphasized the value of learning the importance of following up implementation and learning how to define the end goal in a structured way as relevant and useful when planning the implementation/adaptation of the various EBIs.

Table 6.

Median Values of the Self-Rated Knowledge Index for the Three Intervention Groups

Time of measurement	n	Mdn	SD	W	p
Group 1
Baseline	18	7.3	1.76
Follow-up	8	7.3	1.24	68.5	.8672
Group 2
Baseline	8	6.9	1.84
Follow-up	6	8.0	1.05	4	.01145
Group 3
Baseline	49	7.8	2.11
Follow-up	28	8.0	2.11	552.5	.1579

Table 7.

Mean Values of the Single Item Concerning Whether the Workshop Had Enhanced Knowledge About Implementation

Time of measurement	n	M (SD)	SD
Workshop 1
Group 1	15	8.33	0.98
Group 2	6	7.33	0.98
Group 3	30	6.67	2.67
Workshop 2
Group 1	7	6.43	1.27
Group 2	6	7.00	2.28
Group 3	27	7.59	2.37

Two themes were identified regarding the impact in the interviews: (1) the structure for balancing the fidelity and adaptation dilemma provides value in practice and (2) increased capacity to balance the fidelity and adaptation dilemma. Whereas the knowledge gained through the workshop was perceived as relevant and useful, the participants emphasized the value of having a structured approach for balancing the fidelity and adaptation dilemma. Furthermore, while some participants expressed that the workshop had increased their knowledge about how to manage fidelity and adaptations, other participants attested that they did not gain new theoretical knowledge about fidelity and adaptation. Common among these participants was significant experience with making adaptations due to those adaptations’ unavoidability in their work context. Although these participants did not gain new theoretical knowledge, they perceived the structured way of planning fidelity and adaptations as novel and useful.

The participants described how the intervention had impacted their capacity to plan the management of fidelity and adaptation. They were able to use the worksheets to identify and plan for adaptations during the workshop, and some participants had used or planned to use the material after the workshop for further management of fidelity and adaptations by the time of the interviews. By using the worksheets, the participants were able to describe the use of an EBI in a given context and make conclusions about how any particularities of their own contexts could motivate adaptations of content and/or delivery of the EBI upon implementation. However, a few participants had misunderstood the use of parts of the worksheets (e.g., thinking the original method was their current work practice, not the original EBI), further emphasizing the need to tailor the intervention to the needs and terms of practice.

Furthermore, for some participants, participation in the intervention increased their awareness about the importance of purposely managing the fidelity and adaptation dilemma. Some participants noted that they felt secure in their ability to adapt EBIs to their context before the workshops, but after the workshops, they realized that adaptations can be complicated and that one should not change EBIs without thoughtful consideration, as illustrated by one participant:

I think I used to believe that it was probably okay to kind of change as you wish, but in the workshops I thought that it became … I mean … how much can you really adjust for it to work? (Interview 10)

Analysis of the worksheets provided information about whether and how the participants were able to identify and plan adaptations during the workshop. Contextual differences identified in the worksheets included aspects related to type of setting (e.g., initially in a hospital, now in home services for older people), target group (e.g., initially patients with adequate cognitive functioning, now patients with cognitive impairment), and providers (e.g., initially hospital nurses, now social service workers). The worksheets showed that the participants were able to identify and plan adaptations to the content of an EBI, such as adding elements, tailoring material to be more comprehensive or suitable for the target group, and repeating elements. Concerning adaptations in the delivery of an EBI, participants mentioned changing sessions from physical appointments to phone calls, as well as asking questions instead of using a self-administered questionnaire.

It was challenging for the participants to identify implementation strategies that had been used when implementing the EBI previously. Instead, the teams mostly created strategies they thought would be most suitable in their situations including education, training, enhancing motivation, investigating and addressing personnel concerns and needs, changing internal work processes, and using follow-up data to address challenges.

The type of outcomes the participants identified as important were similar to those of previous use of the EBI but were more detailed and comprehensive. For instance, one team found only “identifying violence” as an outcome in the description of the EBI but decided to include several desired outcomes for their own project (e.g., “identifying violence,” “being able to talk about violence,” “being able to offer support to victims of violence,” and “normalizing speaking to patients about violence”). There were also cases when the objectives differed between the previous use of the EBI and the team's desired outcomes (e.g., the original EBI aimed to improve external cooperation, whereas the team desired to improve internal cooperation). Teams planned several strategies to follow-up on the implementation of the respective EBI. Follow-up strategies included employee outcome data, client surveys, outcome statistics, and continuous follow-up meetings to discuss difficulties with the use of the EBI.

Discussion

We found that the social service practitioners perceived the A-FiT intervention as relevant for their management of fidelity and adaptations when working with EBIs. The participants’ described changes in their knowledge and behaviors as well as their understanding of adaptation and fidelity that has implications for both research and practice are further discussed below.

Participants perceived many aspects of the A-FiT intervention as useful for their daily work with EBIs, suggesting that supporting professionals in the management of fidelity and adaptation may complement other initiatives for professionals’ implementation-capacity building such as the building implementation capacity (BIC) intervention (Augustsson et al., 2021; Mosson et al., 2017). The worksheets further showed that professionals were able to identify which key features of the EBI, in the context of application, needed to be addressed to support fidelity or which prompted adaptations, including to the content, context, format, or goals. Components of the intervention that were perceived as important for a positive experience were the structured approach to analyzing the EBI and the context, the pedagogic format that combined theory (i.e., the lecture) and practical teamwork on their own EBI, opportunity to receive support from peers from other organizations, and the presence of a team's manager. These are components that have previously been found to be valuable for implementation-capacity building (Mosson et al., 2017; Richter et al., 2016). Yet, the participants also raised areas for improvement, thereby offering important insights not only for the development of support for practitioners but also for the understanding of adaptation and fidelity in general.

The positions that participants held prior to the intervention varied from viewing adaptations as common sense to viewing fidelity as a must. Many participants also felt knowledgeable about adaptation and fidelity and in their ability to make adaptations prior to the intervention. Only after the intervention did participants’ awareness about the complexity of adaptation and fidelity and its management grow, resulting in their feelings of insecurity and need for additional training. The lack of awareness of the need for behavioral change implies that it may be common for professionals in social services to be in a precontemplative stage of behavioral change (Prochaska, 1994). This aligns with previous studies suggesting that professionals’ ability to reflect on adaptations they make may be limited (Pettersson et al., 2022; Zetterlund et al., 2022) and may be one reason for the frequency of drift (Chambers & Norton, 2016). The lack of awareness also points to the challenges involved in building capacity to manage adaptation and fidelity; a population may consider training futile because they consider adaptations necessary but either unproblematic or unacceptable. Future iterations of the intervention may benefit from incorporating principles for shifting from a precontemplative stage to action, including investigating whether that could be best achieved by raising awareness of the drawbacks of the current position, or increasing the awareness of the benefits of a more reflective approach to adaptation and fidelity (Prochaska, 1994).

The finding that participants shifted from a precontemplative to a more reflective approach to adaptation and fidelity also has implications for the evaluation of effects on knowledge and behaviors relating to fidelity and adaptation, as it implies measurement of a dynamic concept. Inconsistencies between ratings and interview findings (with significant improvements in the participants’ self-rated knowledge regarding fidelity, adaptation, and implementation in only one group, while interviews suggested a similar development in other groups as well) is one indication that there may have been improvements in knowledge (i.e., an alpha change), but that the intervention also likely led to the so-called beta and gamma changes (Golembiewski et al., 1976). A beta change would imply that a rating that remains unchanged after the intervention may mean different things if participants changed their concept of what having knowledge about adaptation and fidelity may mean, and a gamma change would imply that the whole concept of adaptation and fidelity changed, making measurement using a rating scale problematic (Golembiewski et al., 1976). The potential presence of beta and gamma change when professionals are trained in adaptation and fidelity needs to be further investigated, including how it might affect assessment of change in knowledge, attitudes, and behaviors (Riordan et al., 2001).

Second, professionals pointed out that the timing of the A-FiT intervention was an issue. The intervention was designed to be used at an early stage of implementation, after an adoption decision, to allow for proactive planning of fidelity and adaptation, as recommended in the literature (Lee et al., 2008). However, it was also designed to encourage reconsideration of adoption decisions if the analysis would reveal that the chosen EBI would need such substantial adaptations that it would risk being ineffective. Yet, at this point of the implementation, professionals may be too focused on learning the EBI or in identifying implementation strategies, making a focus on adaptation and fidelity seem premature in relation to other needs, particularly for professionals who were not keenly aware of the fidelity-adaptation dilemma and if other implementation support was missing. Also, making proactive decisions about adaptation and fidelity requires substantial knowledge about both the EBI and the contextual differences of its current and prior use, leading to professionals questioning the timing of the intervention.

To address these issues and further develop the intervention, one solution may be to integrate the intervention with other implementation support (e.g., the BIC intervention (Mosson et al., 2019), iLead (Richter et al., 2016) or leadership and organizational change for implementation (LOCI; Aarons et al., 2015), and/or testing the support at a later stage of the implementation process (von Thiele Schwarz et al., 2021). Tailoring the intervention to specific target groups, by adapting the language or tailoring examples provided during the workshops could be useful for increasing the accessibility of the intervention. Furthermore, considering the extent of the fidelity and adaptation dilemma across contexts, the A-FiT could be valuable outside of social care, in areas such as public health, mental health, or education. Next step in evaluation of A-Fit would be to test it in an experimental design with implementation and service outcome measures.

Methodological Discussion

A strength of the study is its co-created approach, whereby social service professionals and managers gave input into the intervention during its design. This is likely to increase the intervention's suitability for the practical needs of social service organizations (Green, 2008). It is also a strength that multiple areas of social services participated, indicating the generality of adaptation and fidelity considerations and the usefulness of the approach across multiple settings. Furthermore, we used three sources of data and were thus able to triangulate the data, which was shown to be particularly valuable since changes in professionals’ views on adaptation and fidelity provoked by the intervention may affect their ratings of related attitudes and knowledge. There are also some limitations. Although we used a longitudinal intervention design, we did not use a control condition, nor did we evaluate implementation or service outcomes. The results should, therefore, be used for understanding the participants’ perceptions of the intervention and how changes in professionals’ knowledge of and approach to adaptation and fidelity may be brought about (i.e., not as an effect evaluation). Furthermore, the study was performed during the COVID-19 pandemic, which caused challenges not only for running the intervention but also for the data collection. We had difficulties in recruiting respondents for the postintervention data collection, likely reflecting the time restraints on staff during the pandemic. The low number of respondents to the postintervention questionnaires limits the understanding of the intervention's impact. Not being able to assess baseline differences (i.e., anonymous surveys) between the participants who did and did not complete the follow-up questionnaires further limits the understanding of the intervention's impact.

Conclusions

Practical tools are needed to guide professionals not only to adhere to an intervention's core elements but also to balance fidelity and adaptation. This structured intervention for practitioners’ management of both fidelity and adaptations is a novel contribution to the implementation literature since existing research offers limited practical guidance. Interviews with participants suggested that features such as peer learning, the presence of a manager, a structured format, and a pedagogical format with both short lectures and practical work may be important components in interventions that aim to strengthen professionals’ management of adaptation and fidelity. However, in order to strengthen professionals’ capacity to manage adaptation and fidelity, interventions need to address their presumptions about those concepts. Regardless of whether professionals believe that adaptations are natural and that their management is common sense, or they view adaptations as nonviable, a lack of reflection may prove particularly important—and challenging—to address.