Abstract
Background
Although there is evidence that tailored implementation strategies can be effective, there is little evidence on which methods of tailoring improve the effect. We designed and evaluated five tailored programs (TPs) each consisting of various strategies. The aim of this study was to examine (a) how determinants of practice prioritized in the design phase of the TPs were perceived by health care professionals who had been exposed to the TPs and whether they suggested other important determinants of practice and (b) how professionals used the offered strategies and whether they suggested other strategies that might have been more effective.
Methods
We conducted a mixed-method process evaluation linked to five cluster-randomized trials carried out in five European countries to implement recommendations for five chronic conditions in primary care settings. The five TPs used a total of 28 strategies which aimed to address 38 determinants of practice. Interviews of professionals in the intervention groups and a survey of professionals in the intervention and control groups were performed. Data collection was conducted by each research team in the respective national language. The interview data were first analyzed inductively by each research team, and subsequently, a meta-synthesis was conducted. The survey was analyzed descriptively.
Results
We conducted 71 interviews; 125 professionals completed the survey. The survey showed that 76 % (n = 29) of targeted determinants of practice were perceived as relevant and 95 % (n = 36) as being modified by the implementation interventions by 66 to 100 % of professionals. On average, 47 % of professionals reported using the strategies and 51 % considered them helpful, albeit with substantial variance between countries and strategies. In the interviews, 89 determinants of practice were identified, of which 70 % (n = 62) had been identified and 45 % (n = 40) had been prioritized in the design phase. The interviewees suggested 65 additional strategies, of which 54 % (n = 35) had been identified and 20 % (n = 13) had been prioritized, but not selected in the final programs.
Conclusions
This study largely confirmed the perceived relevance of the targeted determinants of practice. This contrasts with the fact that no impact of the trials on the implementation of the recommendations could be observed. The findings suggest that better methods for prioritization of determinants and strategies are needed.
Trial registration
Each of the five trials was registered separately in recognized trial registries. Details are given in the respective trial outcome papers.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-016-0473-8) contains supplementary material, which is available to authorized users.
Keywords: Tailoring, Process evaluation, Primary care, Implementation, Evidence-based, Guideline, Recommendations, Determinants of practice, Strategies
Background
It has been suggested that the transfer of evidence-based knowledge into practice would improve if the barriers to its adoption were overcome or facilitating factors used appropriately. “Tailoring” is a systematic approach to improve the design and effectiveness of interventions by selecting strategies explicitly to address specific, previously identified determinants of practice. Determinants of practice are factors that could influence the effectiveness of an intervention to improve professional practice and have been previously referred to using alternative terms, including barriers, obstacles, enablers, and facilitators [1]. Implementation strategies have been defined as “methods or techniques used to enhance the adoption, implementation and sustainability of a clinical program or practice” [2]. A range of frameworks have been suggested to classify determinants [3, 4] and implementation strategies [4–9], but none of them has been widely accepted, and although there is evidence that tailored interventions may have positive effects [1], it is unclear which methods most effectively identify and prioritize determinants and strategies [1, 10]. More evidence about the value of different methods would help developers of implementation programs improve health care practice.
In the project “Tailored Implementation for Chronic Diseases (TICD),” five tailored programs to implement recommendations for five different chronic health problems (chronic obstructive pulmonary disease, obesity, depression in the elderly, multimorbidity, and cardiovascular diseases) have been developed and evaluated in cluster-randomized controlled trials in Poland (PL), United Kingdom (UK), Norway (NW), Germany (GE), and The Netherlands (NL) [11]. An overview of the conditions and recommendations targeted and the strategies used in TICD is provided as Additional files 1 and 2.
The process of intervention development, which has been described previously in detail [12–14], comprised the systematic identification, prioritization, and selection of determinants potentially affecting the adoption by health professionals of the selected clinical practice recommendations. Subsequently, strategies to address the determinants were identified, prioritized, and selected. A combination of methods (brainstorming, interviews, focus groups, and a survey) involving various stakeholders (patients, physicians, researchers) was used to identify determinants and strategies. In a consensus procedure, prioritization involved the assessment of each determinant according to its assumed relevance and modifiability and the assessment of each strategy on its assumed impact and feasibility. The assessment was first done independently by at least two researchers using a scale from one to five for each criterion. In a second step, discrepancies were discussed to agree which of the potentially relevant and modifiable determinants should be addressed by the program, or which of the potentially effective and feasible strategies should be used.
Finally, a logic model specifying the assumed linkages between selected determinants and strategies was elaborated for each trial. The general structure of these models is depicted in Fig. 1.
Fig. 1.
Logic model of the TICD trials. The figure describes the logic of the tailored programs developed within TICD: Implementation of evidence-based recommendations will improve if the applied strategies successfully modify previously identified determinants of practice. Since the recommendations are evidence based, i.e., their effectiveness has been substantiated, increased implementation will result in improved health outcomes. The content-specific logic models of each tailored program have been published in the respective study protocols [18–22]
Despite this detailed approach to developing the implementation interventions, none of the five studies achieved a positive effect on the primary outcome, which in all trials was the degree of the implementation of the recommendations measured by a set of indicators. The type and content of the indicators, however, was context-specific for each trial. A positive effect on some secondary outcomes could be observed [15–17] (Jaeger et al: Impact of a tailored program on the implementation of evidence-based recommendations for multimorbid patients with polypharmacy in primary care practices – results of a cluster-randomized controlled trial, under review in Implementation Science; Kowalczyk et al: Tailored implementation strategy to improve the management of patients with chronic obstructive pulmonary disease in primary care in Poland: a cluster randomized controlled trial, in preparation). While the primary and secondary outcomes of the programs differed across the trials [18–22], the process evaluation followed a common study protocol [23]. Its aim was to assess how the targeted health care professionals (HCP), i.e., the professionals who experienced the programs, viewed the choice of determinants to be addressed, and the strategies selected to address them. Furthermore, we intended to compare these views against the findings from the development phase of the intervention, during which interviews and surveys with stakeholders who had not experienced the programs had been conducted. In summary, the following research questions were addressed:
-
A.
Research questions focusing on determinants:
A1: Were the selected determinants perceived as relevant and modified by the targeted HCP?
A2: Do the targeted HCP mention other important determinants which had not been identified or prioritized before?
-
B.
Research questions focusing on strategies:
B1: To what extent did the targeted HCP use and adapt the strategies offered by the tailored programs?
B2: How helpful were the offered strategies from the perspective of the targeted HCP?
B3: Should other strategies have been used from the perspective of the targeted HCP?
The questions were chosen to provide evidence on the usefulness of the methods used to identify and prioritize determinants and strategies in the context of tailoring.
Methods
Study design
This study was a process evaluation of five related cluster-randomized controlled trials of tailored implementation programs. We used a mixed-method approach consisting of semi-structured interviews and a survey to answer the research questions described above.
Study population and recruitment
At the end of the intervention period, HCP of the intervention groups were invited to participate in an interview and/or the survey and HCP of the control groups were invited to participate in a survey. Recruitment for the intervention studies is described for each trial separately [15–17] (Jaeger et al: Impact of a tailored program on the implementation of evidence-based recommendations for multimorbid patients with polypharmacy in primary care practices – results of a cluster-randomized controlled trial, under review in Implementation Science; Kowalczyk et al: Tailored implementation strategy to improve the management of patients with chronic obstructive pulmonary disease in primary care in Poland: a cluster randomized controlled trial, in preparation). In Norway, only the interview study was conducted, as the survey was considered not feasible in the Norwegian trial.
Data collection and processing
Interviews
Interviews were conducted separately in each trial in the respective national languages. All interviews followed a common semi-structured interview guide developed in English beforehand which contained the following questions:
It is recommended that … [introduce recommendation]. What do you think about this recommendation?
Where there any factors which made it difficult for you or helped you to adhere to the recommendation?
The implementation program consisted of various strategies. What strategies did you or other team members use and in what way? (If necessary, mention each strategy separately).
- Did the implementation program help you to adhere to the recommendations?
- If yes, what strategies did you find helpful and why?
- If no, why not and what strategies would have been more helpful?
All interviews were audiotaped and transcribed. In Norway, the researchers involved in the design and delivery of the intervention conducted most of the interviews themselves; in the other countries (GE, NL, PL, UK) this was done by researchers not directly involved in the trial in order to avoid socially desirable responses.
Survey
The survey was conducted separately in each trial in the respective national languages but following a common framework elaborated in English (see Table 1). Item 1 of part 1 of the survey, which focused on determinants, was applied to the intervention and control groups. Item 2 of part 1 and part 2 of the survey focused on the use of the strategies by the HCP and was based on four aspects of adherence according to Carrol et al. [24]: content, duration, frequency, and coverage. It was applied to the intervention group only. The numbers of aspects formulated as items varied, because not all of them were applicable in the context of the various trials. Two additional items assessed adaptations to the strategies and their perceived helpfulness.
Table 1.
Framework of the survey
Item 1 was applied to the intervention and control groups; item 2 and part 2 were applied to the intervention groups only
Data analysis
Interviews
Each research team analyzed the interviews individually following the principles of qualitative content analysis [25]. For this purpose, we used a common framework specifying the main categories for the analysis, which reflected the research questions of the process evaluation. Subcategories specific for the context of each trial were added inductively. Additionally, it was specified how frequently a theme was mentioned in the interviews using a coding from one to five (1 = strong issue in almost all interviews, 5 = single statement), whether the determinant or strategy respectively was identified and prioritized during the development phase of the intervention and whether the determinants were intended to be addressed by the implementation program. Subsequently, the emerging themes related to the determinants and the use of strategies were assigned to the TICD checklist, a comprehensive classification of determinants of practice developed in an earlier stage of the TICD project [3]. All teams used Atlas.ti software [26] for the coding.
After the country-specific analysis, each team provided the derived sets of categories with example quotations in English to the German team, who integrated them by merging identical codes and by harmonizing the assignments to the TICD checklist. This process was done in close collaboration with the researchers of the other teams to reduce the risk of bias. The framework for the interview analysis is depicted in Additional file 3.
Finally, the interview data from all countries were analyzed using a quantitative approach involving calculation of the percentage of the mentioned determinants that had been identified, prioritized (i.e., judged relevant and modifiable), and chosen to be addressed in the design phase of the intervention. Likewise, we calculated the percentage of strategies that had been identified, prioritized (i.e., judged effective and feasible), and selected.
Survey
The results of the survey are presented descriptively using means and percentages. The response categories “yes”/“partly” and “agree”/“partly agree” (see Table 1) were merged and interpreted as affirmation. The “majority of HCP” was defined as >66 % of professionals based on the consideration that this cutoff reflects the agreement or disagreement of a considerable number of HCP.
Results
Participants
In total, 71 interviews (9 in PL, 11 in UK, 12 in GE, 19 in NL, and 20 in NW) were conducted, and 125 primary HCP completed the survey. While in some countries, only general practitioners (GPs) were targeted by the survey or interviews; in other countries, practice nurses (PN) and health care assistants (HCA) were involved as well. In total, 211 HCP were invited for an interview, and 36 GPs, 26 PN, and 9 HCA participated, representing 33.6 % of the target group. The mean age of the interviewees was 44 years (ranging from 31 to 53 years across countries), and 70 % (n = 50) were female. The interviews lasted on average 34 (12–77) min. Socio-demographic characteristics of the survey respondents are depicted in Table 2.
Table 2.
Type and number of HCP completing the survey on determinants
| Trial | HCP completing the trial | HCP completing the survey | Response rate (%) | Profession % (n) | Mean age (years) | Female sex % (n) |
|---|---|---|---|---|---|---|
| Total (IG/CG) | Total (IG/CG)a | |||||
| GE | 21 (10/11) | 21 (10/11) | 100 | GPs: 100 (21) | 54.9 (44–68) | 19.0 (4) |
| NL | 33 (19/14) | 30 (17/13) | 90.9 | PNs: 83 (25) | 42.4 (22–61) | 96.7 (29) |
| HCA: 17 (5) | ||||||
| PL | 18 (9/9) | 13 (7/6) | 72.2 | GPs: 100 (13) | 47.6 (39–58) | 46.2 (6) |
| 4 missings | 4 missings | |||||
| UK | 146 (16/130) | 61 (13/48) | 41.8 | GPs: 54 (33) | Not collected = 61 missings | Not collected = 61 missings |
| PNs: 29 (18) | ||||||
| HCA: 15 (9) | ||||||
| Unknown: 2 (1) | ||||||
| All | 218 (54/164) | 125 (47/78) | 57.3 | GPs: 54 (67) | 48.3 (22–68) | 31.2 (39) |
| PNs: 34 (43) | 65 missings | 65 missings | ||||
| HCA: 10 (14) | ||||||
| Unknown: 1(1) |
a IG intervention group, CG control group
Results related to determinants (research questions A1 and A2)
Table 3 shows the results of part 1 of the survey focusing on determinants. The mean percentage of HCP answering the question, whether the targeted determinant was relevant for them, with “agree” or “partly agree” ranged from 60 to 93 % across trials. For item 2, asking whether the determinants were modified by the TP, the range was 83–98 %. From the 38 determinants presented in the survey and intended to be addressed by the five tailored interventions, 76 % (n = 29) were perceived as relevant and 95 % (n = 36) as modified by at least two thirds of the target group, although with substantial variation between countries (Table 4).
Table 3.
Results of part 1 of the survey (focusing on determinants)
| Determinants intended to be modified by the program | Relevance* | Modification** | |||
|---|---|---|---|---|---|
| Total | IG | CG | IG | ||
| % (n) | % (n) | % (n) | % (n) | ||
| GE | 1 Knowledge of HCP | 71.4 (15) | 80.0 (8) | 63.6 (7) | 100.0 (10) |
| 2 Routine | 52.3 (11) | 30.0 (3) | 72.7 (8) | 90.0 (9) | |
| 3 Availability of medication lists | 61.9 (13) | 40.0 (4) | 81.8 (9) | 80.0 (8) | |
| 4 Identification of the target group | 23.8 (5) | 40.0 (4) | 9.0 (1) | 90.0 (8) | |
| 5 Feasibility of checklists | 90.5 (19) | 90.0 (9) | 90.9 (10) | 80.0 (8) | |
| 6 Patients’ ability for self-management | 90.5 (19) | 90.0 (9) | 90.9 (10) | 80.0 (8) | |
| 7 Language barrier | 76.2 (16) | 70.0 (7) | 81.8 (9) | 70.0 (7) | |
| 8 Patients’ knowledge | 57.1 (12) | 60.0 (6) | 54.5 (6) | 90.0 (9) | |
| 9 Standardization of medication lists | 61.9 (13) | 60.0 (6) | 63.6 (7) | 70.0 (7) | |
| Mean of all items GE | 65.1 (13.7) | 62.2 (6.2) | 67.6 (7.4) | 83.3 (8.2) | |
| NL | 1 Apply motivational interviewing | 93.3 (28) | 100.0 (17) | 84.6 (11) | 100.0 (17) |
| 2 Giving good advice to patients | 96.7 (29) | 100.0 (17) | 92.3 (12) | 100.0 (17) | |
| 3 More attention for the motivation of the patient | 96.7 (29) | 100.0 (17) | 92.3 (12) | 100.0 (17) | |
| 4 PN gives lifestyle advice in an acceptable and feasible way | 93.3 (28) | 94.1 (16) | 92.3 (12) | 100.0 (17) | |
| 5 PN meets patients’ information needs | 93.3 (28) | 94.1 (16) | 92.3 (12) | 94.1 (16) | |
| 6 PN drafts feasible targets for patients | 96.7 (29) | 100.0 (17) | 92.3 (12) | 100.0 (17) | |
| 7 E-health support for self-management | 83.3 (25) | 94.1 (16) | 69.2 (9) | 88.2 (15) | |
| Mean of all items NL | 93.3 (28) | 97.5 (16.6) | 87.9 (11.4) | 97.5 (16.6) | |
| PL | 1 Availability of educational materials for recommendation 1 | 100.0 (13) | 100.0 (7) | 100.0 (6) | 28.6 (2) |
| 2 Availability of training for GPs | 69.2 (9) | 57.1 (4) | 83.3 (5) | 42.9 (3) | |
| 3 Labeling of medication records | 100.0 (13) | 100.0 (7) | 100.0 (6) | 100.0 (7) | |
| 4 Accessibility of mMRC scale | 100.0 (13) | 100.0 (7) | 100.0 (6) | 100.0 (7) | |
| 5 Accessibility of checklists for recommendation 2 | 92.3 (12) | 85.7 (6) | 100.0 (6) | 100.0 (7) | |
| 6 Availability of the recommendations | 100.0 (13) | 100.0 (7) | 100.0 (6) | 100.0 (7) | |
| 7 Availability of training for personnel on dyspnea assessment | 92.3 (12) | 85.7 (6) | 100.0 (6) | 71.4 (5) | |
| 8 Availability of educational materials for recommendation 3 | 100.0 (13) | 100.0 (7) | 100.0 (6) | 100.0 (7) | |
| 9 Availability of treatment plans | 92.3 (12) | 85.7 (6) | 100.0 (6) | 85.7 (6) | |
| 10 Accessibility of checklist for recommendation 3 | 92.3 (12) | 85.7 (6) | 100.0 (6) | 85.7 (6) | |
| 11 Availability of peak flow meters | 92.3 (12) | 85.7 (6) | 100.0 (6) | 85.7 (6) | |
| 12 Availability of demonstration inhalers | 100.0 (13) | 100.0 (7) | 100.0 (6) | 100.0 (7) | |
| 13 Availability of educational materials on use of inhaler devices | 100.0 (13) | 100.0 (7) | 100.0 (6) | 100.0 (7) | |
| 14 Training of GPs on inhaler devices | 100.0 (13) | 100.0 (7) | 100.0 (6) | 85.7 (6) | |
| 15 Presence of additional staff (educator) | 84.6 (11) | 71.4 (5) | 100.0 (6) | 71.4 (5) | |
| Mean of all items PL | 94.4 (12.3) | 90.5 (6.3) | 98.9 (5.9) | 83.3 (5.9) | |
| UK | 1 Skills of HCP to raise the issue of weight with patients | 45.9 (28) | 69.2 (9) | 39.6 (19) | 100.0 (13) |
| 2 Skills of HCP to measure waist circumference | 50.8 (31) | 53.8 (7) | 50.0 (24) | 92.3 (12) | |
| 3 Skills of HCP to assess patients’ willingness to change | 47.5 (29) | 61.5 (8) | 43.8 (21) | 84.6 (11) | |
| 4 Availability of resources to inform and motivate patients | 67.2 (41) | 84.6 (11) | 62.5 (30) | 100.0 (13) | |
| 5 Availability of prescriptive weight loss information | 83.6 (51) | 92.3 (12) | 81.3 (39) | 100.0 (13) | |
| 6 Work with HCP whom manage obese and overweight patients to improve their knowledge on diets | 66.7 (32) | 76.9 (10) | 45.8 (22) | 92.3 (12) | |
| 7 Availability of information about referral pathways | 59.0 (36) | 76.9 (10) | 54.2 (26) | 92.3 (12) | |
| Mean of all items UK | 60.1 (35.4) | 73.6 (9.6) | 53.9 (25.9) | 94.5 (12.3) | |
IG intervention group, CG control group, GE Germany, NL The Netherlands, PL Poland, UK United Kingdom
*Refers to item 1 of the framework depicted in Table 1
**Refers to item 2 of the framework depicted in Table 1. Numbers show the proportion of respondents who answered the respective questionnaire item with “agree” or “partly agree”
Table 4.
Number of determinants perceived as relevant and modified by the tailored programs, per trial. Results of part 1 of the survey (focusing on determinants)
| Number of determinants | GE | NL | PL | UK | All |
|---|---|---|---|---|---|
| Intended to be addressed | 9 | 7 | 15 | 7 | 38 |
| Perceived as relevant % (n)a | 44 (4) | 100 (7) | 100 (15) | 43 (3) | 76 (29) |
| Perceived as modified % (n)b | 100 (9) | 100 (7) | 87 (13) | 100 (7) | 95 (36) |
In the interviews, 89 determinants for the recommendations were mentioned by the target group relating to 6 main and 25 of 57 subcategories of the TICD checklist (see Additional file 4). Categories with most items were “individual HCP factors” (21 determinants from 4 trials), “patient factors” (16 determinants from 4 trials), and “incentives and resources” (14 determinants from 4 trials). Of the 89 determinants, 70 % (n = 62) had been identified and 45 % (n = 40) had been prioritized during the development phase of the intervention. Thirty-seven percent (n = 33) of these determinants were intended to be addressed by the implementation programs, although there was wide variation between countries (Table 5).
Table 5.
Comparison of determinants and strategies identified by interviews before and after the delivery of the program
| GE | PL | NW | UK | NL | All | |
|---|---|---|---|---|---|---|
| % (n) | % (n) | % (n) | % (n) | % (n) | % (n) | |
| Number of determinants identified after program delivery | 32 | 8 | 24 | 10 | 15 | 89 |
| Thereof identified before | 75 (24) | 75 (6) | 67 (16) | 100 (10) | 40 (6) | 70 (62) |
| Thereof prioritized (i.e., judged to be relevant and potentially modifiable) | 69 (22) | 38 (3) | 29 (7) | 80 (8) | 0 | 45 (40) |
| Thereof intended to be addressed by the program | 50 (16) | 38 (3) | 25 (6) | 80 (8) | 0 | 37 (33) |
| Number of alternative strategies identified after program delivery | 15 | 11 | 5 | 22 | 12 | 65 |
| Thereof identified before % (n) | 67 (10) | 18 (2) | 80 (4) | 46 (10) | 75 (9) | 54 (35) |
| Thereof prioritized before (i.e., assessed as potentially effective and feasible, but not used in the final program) | 20 (3) | 18 (2) | 0 | 18 (4) | 33 (4) | 20 (13) |
| Number and type of determinants not selected | ||||||
| Patient factors | 4 | 1 | 3 | – | 1 | 9 |
| Capacity for organizational change | 1 | – | 2 | – | 1 | 4 |
| Incentives and resources | 4 | 2 | 2 | 2 | 4 | 13 |
| Professional interactions | 1 | 1 | 1 | – | 3 | 6 |
| Social, political, and legal factors | – | – | – | – | 2 | 2 |
| Individual health care professional factors | 6 | 1 | 8 | – | 1 | 16 |
| Not assigned to checklist | – | – | 2 | – | 3 | 5 |
| Number and type of strategies not prioritized | ||||||
| Intervention development and delivery | – | 6 | – | 3 | – | 9 |
| Change service sites | 3 | – | 1 | 3 | 1 | 8 |
| Provision of materials | 1 | – | 1 | 3 | 2 | 7 |
| Share local knowledge | 2 | 1 | – | 2 | 1 | 6 |
| Change record systems | 4 | – | – | 1 | – | 5 |
| Training | 1 | – | 1 | 3 | – | 5 |
| Ongoing support | 1 | 1 | – | 1 | 1 | 4 |
| Adaptions on patient level | – | – | 2 | – | – | 2 |
| Revise professional roles | – | – | – | 1 | 1 | 2 |
| Funding | – | – | – | 1 | 1 | 2 |
| Provision of evidence | – | – | – | – | 1 | 1 |
| External feedback | – | 1 | – | – | – | 1 |
This table shows the quantitative analysis of the qualitative data presented in Additional file 4
Results related to the use of strategies (research questions B1 and B2)
Table 6 shows the results of part 2 of the survey and the percentages of respondents per country who positively answered the questions on whether they had used the respective strategy. The mean of all items varied between countries and ranged from 31 % (NL) to 57 % (GE). The proportion of respondents who reported adapting one or several strategies in practice also varied between strategies and countries but was overall low. One item in the questionnaire asked whether the respective strategies were perceived as helpful by the target group. On average, 33 % (NL)–77 % (UK) of the respondents answered this item in the affirmative; for individual strategies, variation was between 0 and 94 %. The majority of respondents in all trials appreciated face-to-face training sessions.
Table 6.
Results of part 2 of the survey (focusing on the use of strategies)
| Country | Strategya (short term) | Use of the strategy in terms ofb | Adaptionc | Helpfulnessd | ||||
|---|---|---|---|---|---|---|---|---|
| Content | Frequency | Duration | Coverage | Total | ||||
| GE (n = 10) | GE 1 | 100.0 | n.a. | n.a. | n.a. | 100.0 | n.a. | 60.0 |
| GE 2 | 100.0 | n.a. | n.a. | n.a. | 100.0 | 0.0 | 60.0 | |
| GE 3 | 30.0 | n.a. | n.a. | n.a. | 30.0 | 10.0 | 80.0 | |
| GE 4 | 70.0 | n.a. | n.a. | n.a. | 70.0 | 10.0 | 60.0 | |
| GE 5 | 40.0 | n.a. | n.a. | n.a. | 40.0 | 0.0 | 90.0 | |
| GE 6 | 50.0 | n.a. | n.a. | n.a. | 50.0 | 10.0 | 50.0 | |
| GE 7 | 50.0 | n.a. | n.a. | n.a. | 50.0 | 0.0 | 40.0 | |
| GE 8 | 20.0 | n.a. | n.a. | n.a. | 20.0 | 10.0 | 50.0 | |
| GE 9 | 50.0 | n.a. | n.a. | n.a. | 50.0 | 0.0 | 40.0 | |
| Mean of all items GE | 56.7 | 5.0 | 58.9 | |||||
| UK (n = 13) | UK 1 | 100.0 | n.a. | n.a. | n.a. | 100.0 | 23.1 | 84.6 |
| UK 2 | 53.8 | 30.8 | n.a. | 30.8 | 38.4 | 0 | 53.8 | |
| UK 3 | 84.6 | 69.2 | n.a. | 61.5 | 71.8 | 7.7 | 92.3 | |
| UK 4 | 61.5 | n.a. | n.a. | 46.2 | 53.9 | 7.7 | 76.9 | |
| UK 5 | 61.5 | 38.5 | n.a. | 61.5 | 53.8 | 7.7 | 84.6 | |
| UK 6 | 69.2 | 15.4 | 0.0 | 38.5 | 41.0 | 7.7 | 76.9 | |
| UK 7 | 30.8 | n.a. | n.a. | n.a. | 30.8 | 7.7 | 69.2 | |
| Mean of all items UK | 55.7 | 8.8 | 76.9 | |||||
| PL (n = 7) | PL1 | 71.4 | n.a. | n.a. | n.a. | 71.4 | n.a. | n.a. |
| PL2 | n.a. | 57.1 | 28.6 | 14.3 | 33.3 | 0 | 28.6 | |
| PL3 | n.a. | 28.6 | 14.3 | 14.3 | 19.0 | 28.6 | 28.6 | |
| PL4 | n.a. | 28.6 | 42.9 | 71.4 | 47.6 | 28.6 | 42.9 | |
| Mean of all items PL | 42.8 | 19.1 | 33.4 | |||||
| NL (n = 17) | NL1 | 100.0 | n.a. | n.a. | n.a. | 100.0 | n.a. | 94.1 |
| NL2 | 64.7 | n.a. | n.a. | n.a. | 64.7 | n.a. | 64.7 | |
| NL3 | 17.6 | n.a. | n.a. | n.a. | 17.6 | n.a. | 17.6 | |
| NL4 | 11.8 | n.a. | n.a. | 17.6 | 14.7 | 23.5 | 11.8 | |
| NL5 | 5.9 | n.a. | n.a. | n.a. | 5.9 | 23.5 | 41.2 | |
| NL6 | 5.9 | n.a. | n.a. | n.a. | 5.9 | n.a. | 0 | |
| NL7 | 23.5 | n.a. | n.a. | n.a. | 23.5 | 47.1 | 23.5 | |
| NL8 | 17.6 | n.a. | n.a. | n.a. | 17.6 | 35.3 | 11.8 | |
| Mean of all items NL | 31.2 | 32.3 | 33.1 | |||||
aFor explanation of the short terms, see Additional file 2
bRefers to item 3–6 of the framework depicted in Table 1
cRefers to item 8 of the framework depicted in Table 1
dRefers to item 7 of the framework depicted in Table 1. Numbers represent percentages of respondents who answered the respective questionnaire item in the affirmative
In the interviews, we identified 19 common themes related to reasons why the participants considered a strategy helpful or not, or used them or not (see Additional file 5). All themes could be assigned to five main and 14 subcategories of the TICD checklist. The majority of themes referred to the extent to which the strategy addressed individual health professional factors (such as knowledge, awareness, attitudes, motivation, and behavior) and the characteristics of the recommended strategy (such as accessibility, feasibility, effort, compatibility, and quality of underlying evidence). The extent to which the strategy influenced professional interactions, patient factors, and incentives and resources were other domains related to the use of the strategies.
Results related to alternative strategies (research question B3)
Additional file 6 presents the other strategies suggested by the interview participants that had not been included in the respective implementation program, which they saw as likely to be helpful. As Table 5 shows, 65 alternative strategies were suggested. Of these, 54 % (n = 35) had been identified in the intervention development phase and 20 % (n = 13) had been prioritized. Suggestions which had not been prioritized most frequently related to the development and delivery of the intervention, to changes of service sites, the provision of materials, and the sharing of local knowledge.
Discussion
The assumption underlying the concept of tailoring is that implementation is more likely to be successful if strategies are selected to address previously assessed determinants of practice and the needs and preferences of the target group. The results presented in this paper partly support this assumption.
This study largely confirmed that the targeted HCP, after experiencing the programs, perceived the determinants which we identified and selected by various methods in the design phase of the implementation program as relevant. The majority of strategies were perceived as having at least partly modified the determinants. This suggests that a mix of interviews and surveys, involving a range of stakeholders, can effectively identify important determinants of practice and potential strategies for addressing them.
However, we also found that about 30 % of the determinants mentioned by HCP who had experienced the interventions had not been identified and 55 % had not been prioritized during the design of the programs. About one quarter of the determinants intended to be addressed were perceived as not relevant by the majority of the HCP. The proportion of respondents who had used the tailored strategies and considered them helpful varied across strategies and trials but was often less than two thirds. There was little evidence of adaptation of the strategies during application by targeted HCP. Many additional strategies were suggested, some of which had been mentioned in the intervention development phase but had not been selected in the programs.
This suggests that the methods for the prioritization of determinants and strategies should be improved. Our prioritization process was based on the assumed relevance and modifiability of the determinants and the assumed impact and feasibility of the strategies, meaning that determinants and strategies which were perceived as not being modifiable or feasible, respectively, were not selected. Yet such apparently unmodifiable determinants or unfeasible strategies may be decisive for successful implementation. Furthermore, continued monitoring of the experiences of the HCP during program delivery and program adaptation to newly emerging determinants may be needed. The latter has to be balanced against the cost of such monitoring and the need to standardize interventions in the context of rigorous evaluation research.
The fact that the linkages of the logic models were confirmed by a substantial number of HCP contrasts with the limited effect on the primary outcomes of the trials. There are several possible explanations for this discrepancy: We chose pragmatically to design models with linear relationships between determinants, strategies, and effects. Although these models were useful to guide the development and evaluation of the five interventions, possibly more advanced models with more complex (e.g., nonlinear) relationships incorporating theories of behavior change, such as the theoretical domains framework [27], or learning theories as well as adaption of the program theory may be needed to specify valid cause-effect chains, as postulated by several authors [28–31]. In this context, the classification of implementation strategies is important. Implementation strategies have been defined as “methods or techniques used to enhance the adoption, implementation and sustainability of a clinical program or practice” [2]. A growing body of literature deals with the challenges of labeling and reporting implementation strategies [9, 32]. Several lists and taxonomies attempting to reflect the range of strategies have been published [4–9, 33], but none is widely accepted and none of them was sufficiently applicable to our programs. It was also striking that even researchers in the same project (TICD) specified strategies in different levels of detail (see Additional file 2), and so it is unclear whether the assumed mechanism leading to the desired effect was really understood and well communicated to the participants. This reflects the need for better frameworks for the specification of implementation strategies, which comprise different levels of abstraction, for instance by distinguishing the mode of delivery (e.g., workshop), the technique (e.g., role plays), and the aim of the intervention (e.g., to raise awareness, to convey knowledge) [34].
Some studies suggest that other variables, such as the length of follow up, the number of intervention contacts, the type of participant population, and demographics, may moderate the effect of tailored interventions, [30] and may be more decisive than the methods used for the identification of determinants. Yet studies evaluating tailored vs. non-tailored interventions are still few in number and more should be undertaken in the future [1]. Factors related to the research design, such as the chosen outcomes or the sample size, may have been other reasons why no impact on the primary outcomes was detected.
Some strengths and limitations of this study should be taken into account. The process evaluation of all trials followed a common, previously published study protocol [23]. We synthetized data from five trials focusing on five different health problems, all following the same process of tailoring. On one hand, this increases the generalizability of the results. On the other hand, the necessary standardization of the process evaluation including the translation of the results into English might have influenced the precision of the interpretations. The survey was conducted among a small number of HCP and only in four out of the five trials, since it was not considered feasible in Norway, meaning that the results may be susceptible to outliers. The high variance of the responses between and within countries makes it difficult to draw general conclusions. Since we involved different methods and target groups to identify determinants before and after the programs were delivered, comparability is limited. Also we did not rate the relevance and modifiability of the determinants and the feasibility of the strategies suggested after the delivery of the intervention taking self-reported views of the health care professional who had experienced the programs as “gold standard” against which to compare the determinants identified before the delivery of the intervention. However, the health professionals themselves may have had only a limited understanding of the determinants despite having experienced the program.
Conclusions
The linkages between determinants of practice and strategies hypothesized by the logic models of the interventions have been confirmed by the majority of the target group, suggesting that a combination of methods and stakeholders effectively identifies relevant determinants and appropriate strategies. However, this did not result in a measurable improvement of the implementation of the recommendations, indicating that other factors such as the use of theory, monitoring of experiences, intervention fidelity, adaptions of the interventions to individual, and contextual factors or improved outcome measures are equally decisive for the effect of an intervention than the methods used for tailoring. Future research should focus on comparisons between tailored and non-tailored interventions. The process of tailoring should be improved by developing better methods to prioritize determinants of practice and strategies and comprehensive frameworks to standardize the reporting of strategies.
Abbreviations
GE, Germany; GP, general practitioner; HCA, health care assistant; HCP, health care professional; NL, The Netherlands; NW, Norway; PL, Poland; PN, practice nurse; TICD, Tailored Implementation for Chronic Diseases; UK, United Kingdom
Additional files
Chronic conditions and recommendations targeted in the TICD project. (PDF 188 kb)
Strategies used in the TICD trials. (PDF 213 kb)
Framework for the interview analysis. (PDF 15 kb)
Determinants identified by interviews after the delivery of the intervention. (PDF 175 kb)
Results of the qualitative analysis (aspects related to the use of the strategies). (PDF 166 kb)
Results of the qualitative analysis (other strategies not used in the implementation programs). (PDF 209 kb)
Acknowledgements
We are grateful to Martin Eccles and Jeremy Grimshaw who acted as external advisors within the TICD project and to Signe Flottorp who was involved in drafting and revising the manuscript.
Funding
The study was funded by the European Union Framework 7 program within the theme HEALTH.2013.3.1-1 under grant agreement no. 258837.
Authors’ contributions
CJ and MW led the study design and wrote the first draft of the manuscript. CJ analyzed the international survey and interview data. JS, TF, JSz, RB, SA, EA, IG, MGC, AK, and JVL collected the data and critically revised the manuscript. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Ethical approval was obtained for all of the five trials from the respective national ethic committees and covered outcome evaluations as well as the process evaluation. Details are given in the respective trial outcome papers.
Footnotes
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-016-0473-8) contains supplementary material, which is available to authorized users.
Contributor Information
C. Jäger, Email: cornelia.jaeger@med.uni-heidelberg.de
J. Steinhäuser, Email: jost.steinhaeuser@uksh.de
T. Freund, Email: tobias.freund@med.uni-heidelberg.de
R. Baker, Email: rb14@le.ac.uk
S. Agarwal, Email: sa144@le.ac.uk
M. Godycki-Cwirko, Email: maciekgc@uni.lodz.pl
A. Kowalczyk, Email: anna.kowalczyk@umed.lodz.pl
E. Aakhus, Email: eivind.aakhus@gmail.com
I. Granlund, Email: ingeborg.granlund@gmail.com
J. van Lieshout, Email: jan.vanLieshout@radboudumc.nl
J. Szecsenyi, Email: joachim.szecsenyi@med.uni-heidelberg.de
M. Wensing, Email: michel.wensing@med.uni-heidelberg.de
References
- 1.Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. 2015;4:CD005470. doi: 10.1002/14651858.CD005470.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139. doi: 10.1186/1748-5908-8-139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35. doi: 10.1186/1748-5908-8-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cochrane Effective Practice and Organisation Care Group. Data collection Checklist. 2002. http://epoc.cochrane.org/sites/epoc.cochrane.org/files/uploads/datacollectionchecklist.pdf (last Access May 2016).
- 5.Colquhoun H, Leeman J, Michie S, Lokker C, Bragge P, Hempel S, et al. Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. Implement Sci. 2014;9:51. doi: 10.1186/1748-5908-9-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Leeman J, Baernholdt M, Sandelowski M. Developing a theory-based taxonomy of methods for implementing change in practice. J Adv Nurs. 2007;58(2):191–200. doi: 10.1111/j.1365-2648.2006.04207.x. [DOI] [PubMed] [Google Scholar]
- 7.Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1):81–95. doi: 10.1007/s12160-013-9486-6. [DOI] [PubMed] [Google Scholar]
- 8.Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. doi: 10.1186/1748-5908-6-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–57. doi: 10.1177/1077558711430690. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Bosch M, van der Weijden T, Wensing M, Grol R. Tailoring quality improvement interventions to identified barriers: a multiple case analysis. J Eval Clin Pract. 2007;13(2):161–8. doi: 10.1111/j.1365-2753.2006.00660.x. [DOI] [PubMed] [Google Scholar]
- 11.Wensing M, Oxman A, Baker R, Godycki-Cwirko M, Flottorp S, Szecsenyi J, et al. Tailored Implementation For Chronic Diseases (TICD): a project protocol. Implement Sci. 2011;6:103. doi: 10.1186/1748-5908-6-103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Huntink E, van Lieshout J, Aakhus E, Baker R, Flottorp S, Godycki-Cwirko M, et al. Stakeholders inverted question mark contributions to tailored implementation programs: an observational study of group interview methods. Implement Sci. 2014;9(1):185. doi: 10.1186/s13012-014-0185-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Krause J, Van Lieshout J, Klomp R, Huntink E, Aakhus E, Flottorp S, et al. Identifying determinants of care for tailoring implementation in chronic diseases: an evaluation of different methods. Implement Sci. 2014;9:102. doi: 10.1186/s13012-014-0102-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wensing M, Huntink E, van Lieshout J, Godycki-Cwirko M, Kowalczyk A, Jager C, et al. Tailored implementation of evidence-based practice for patients with chronic diseases. PLoS One. 2014;9(7):e101981. doi: 10.1371/journal.pone.0101981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Aakhus E, Granlund I, Odgaard-Jensen J, Oxman AD, Flottorp SA. A tailored intervention to implement guideline recommendations for elderly patients with depression in primary care: a pragmatic cluster randomised trial. Implement Sci. 2016 Mar 9;11:32. doi:10.1186/s13012-016-0397-3. [DOI] [PMC free article] [PubMed]
- 16.Goodfellow J, Agarwal S, Harrad F, Shepherd D, Morris T, Ring A, Walker N, Rogers S, Baker R. Cluster randomised trial of a tailored intervention to improve the management of overweight and obesity in primary care in England. Implement Sci. 2016 May 27;11(1):77. doi:10.1186/s13012-016-0441-3. [DOI] [PMC free article] [PubMed]
- 17.Jan van Lieshout, Elke Huntink, Jan Koetsenruijter, Michel Wensing. Tailored implementation of cardiovascular risk management in general practice: a cluster randomized trial. Implementation Science 2016 11:115. doi:10.1186/s13012-016-0460-0. [DOI] [PMC free article] [PubMed]
- 18.Aakhus E, Granlund I, Odgaard-Jensen J, Wensing M, Oxman AD, Flottorp SA. Tailored interventions to implement recommendations for elderly patients with depression in primary care: a study protocol for a pragmatic cluster randomised controlled trial. Trials. 2014;15:16. doi: 10.1186/1745-6215-15-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Godycki-Cwirko M, Zakowska I, Kosiek K, Wensing M, Krawczyk J, Kowalczyk A. Evaluation of a tailored implementation strategy to improve the management of patients with chronic obstructive pulmonary disease in primary care: a study protocol of a cluster randomized trial. Trials. 2014;15:109. doi: 10.1186/1745-6215-15-109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Huntink E, Heijmans N, Wensing M, van Lieshout J. Effectiveness of a tailored intervention to improve cardiovascular risk management in primary care: study protocol for a randomised controlled trial. Trials. 2013;14:433. doi: 10.1186/1745-6215-14-433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Jager C, Freund T, Steinhauser J, Joos S, Wensing M, Szecsenyi J. A tailored implementation intervention to implement recommendations addressing polypharmacy in multimorbid patients: study protocol of a cluster randomized controlled trial. Trials. 2013;14:420. doi: 10.1186/1745-6215-14-420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Krause J, Agarwal S, Bodicoat DH, Ring A, Shepherd D, Rogers S, et al. Evaluation of a tailored intervention to improve management of overweight and obesity in primary care: study protocol of a cluster randomised controlled trial. Trials. 2014;15:82. doi: 10.1186/1745-6215-15-82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Jager C, Freund T, Steinhauser J, Aakhus E, Flottorp S, Godycki-Cwirko M, et al. Tailored Implementation for Chronic Diseases (TICD): a protocol for process evaluation in cluster randomized controlled trials in five European countries. Trials. 2014;15:87. doi: 10.1186/1745-6215-15-87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40. doi: 10.1186/1748-5908-2-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Mayring P. Qualitative content analysis. Forum Qual Soc Res. 2000;1(2). http://www.qualitative-research.net/index.php/fqs/article/view/1089/2385.
- 26.Atlas.ti. The Qualitative Data Analysis & Research Software. http://atlasti.com/ Accessed May 2016.
- 27.Francis JJ, O'Connor D, Curran J. Theories of behaviour change synthesised into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci. 2012;7:35. doi: 10.1186/1748-5908-7-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011;89(2):167–205. doi: 10.1111/j.1468-0009.2011.00625.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.French SD, Green SE, O'Connor DA, McKenzie JE, Francis JJ, Michie S, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7:38. doi: 10.1186/1748-5908-7-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull. 2007;133(4):673–93. doi: 10.1037/0033-2909.133.4.673. [DOI] [PubMed] [Google Scholar]
- 31.Porcheret M, Main C, Croft P, McKinley R, Hassell A, Dziedzic K. Development of a behaviour change intervention: a case study on the practical application of theory. Implement Sci. 2014;9(1):42. doi: 10.1186/1748-5908-9-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci. 2009;4:40. doi: 10.1186/1748-5908-4-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Mazza D, Bairstow P, Buchan H, Chakraborty SP, Van Hecke O, Grech C, et al. Refining a taxonomy for guideline implementation: results of an exercise in abstract classification. Implement Sci. 2013;8:32. doi: 10.1186/1748-5908-8-32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Colquhoun H, Leeman J, Michie S, Lokker C, Bragge P, Hempel S, McKibbon KA, Peters GJ, Stevens KR, Wilson MG, Grimshaw J. Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. Implement Sci. 2014;9:51. doi: 10.1186/1748-5908-9-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Chronic conditions and recommendations targeted in the TICD project. (PDF 188 kb)
Strategies used in the TICD trials. (PDF 213 kb)
Framework for the interview analysis. (PDF 15 kb)
Determinants identified by interviews after the delivery of the intervention. (PDF 175 kb)
Results of the qualitative analysis (aspects related to the use of the strategies). (PDF 166 kb)
Results of the qualitative analysis (other strategies not used in the implementation programs). (PDF 209 kb)