Skip to main content
Implementation Science : IS logoLink to Implementation Science : IS
. 2018 Jun 7;13:80. doi: 10.1186/s13012-018-0758-1

Using Normalization Process Theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review

Carl R May 1,, Amanda Cummings 1, Melissa Girling 1, Mike Bracher 1, Frances S Mair 1, Christine M May 1, Elizabeth Murray 1, Michelle Myall 1, Tim Rapley 1, Tracy Finch 1
PMCID: PMC5992634  PMID: 29879986

Abstract

Background

Normalization Process Theory (NPT) identifies, characterises and explains key mechanisms that promote and inhibit the implementation, embedding and integration of new health techniques, technologies and other complex interventions. A large body of literature that employs NPT to inform feasibility studies and process evaluations of complex healthcare interventions has now emerged. The aims of this review were to review this literature; to identify and characterise the uses and limits of NPT in research on the implementation and integration of healthcare interventions; and to explore NPT’s contribution to understanding the dynamics of these processes.

Methods

A qualitative systematic review was conducted. We searched Web of Science, Scopus and Google Scholar for articles with empirical data in peer-reviewed journals that cited either key papers presenting and developing NPT, or the NPT Online Toolkit (www.normalizationprocess.org). We included in the review only articles that used NPT as the primary approach to collection, analysis or reporting of data in studies of the implementation of healthcare techniques, technologies or other interventions. A structured data extraction instrument was used, and data were analysed qualitatively.

Results

Searches revealed 3322 citations. We show that after eliminating 2337 duplicates and broken or junk URLs, 985 were screened as titles and abstracts. Of these, 101 were excluded because they did not fit the inclusion criteria for the review. This left 884 articles for full-text screening. Of these, 754 did not fit the inclusion criteria for the review. This left 130 papers presenting results from 108 identifiable studies to be included in the review. NPT appears to provide researchers and practitioners with a conceptual vocabulary for rigorous studies of implementation processes. It identifies, characterises and explains empirically identifiable mechanisms that motivate and shape implementation processes. Taken together, these mean that analyses using NPT can effectively assist in the explanation of the success or failure of specific implementation projects. Ten percent of papers included critiques of some aspect of NPT, with those that did mainly focusing on its terminology. However, two studies critiqued NPT emphasis on agency, and one study critiqued NPT for its normative focus.

Conclusions

This review demonstrates that researchers found NPT useful and applied it across a wide range of interventions. It has been effectively used to aid intervention development and implementation planning as well as evaluating and understanding implementation processes themselves. In particular, NPT appears to have offered a valuable set of conceptual tools to aid understanding of implementation as a dynamic process.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0758-1) contains supplementary material, which is available to authorized users.

Keywords: Normalization Process Theory, Complex interventions, Implementation research, Process evaluation, Systematic review

Background

Implementation theories are useful. They provide explanations for relevant phenomena, propose important research questions and frame the collection and analysis of data [1]. These explanations are generalizable and facilitate comparative studies. Implementation researchers now have a wide range of useful theoretical tools at their disposal [24]. Normalization Process Theory (NPT) [510] is one of these. It identifies, characterises and explains mechanisms that have been empirically demonstrated to motivate and shape implementation processes and affect their outcomes. This paper presents a systematic review of studies of healthcare interventions informed by NPT.

What is NPT and what does it do?

NPT is a theory of implementation that focuses on what people—both individuals and groups—do rather than what they believe or intend, and it has been built up from studies of practice in many different healthcare systems. This means that it focuses attention on aspects of individual and collective behaviour shown to be important in empirical studies of implementation processes. The development of NPT first involved the iterative development of a robust generic theory of implementation [59, 11, 12]. From this, tools were developed to assist implementation practitioners and researchers [1316] in thinking through and measuring important elements of implementation processes. In its most recent iteration, we have shown how the basic mechanisms characterised in NPT function as self-organising mechanisms in complex adaptive social systems [10]. Theory development in NPT has been iterative, with three phases of development around practical questions.

  1. Objects: How are components of complex interventions operationalised by their users? In the first iteration of the theory—the Normalization Process Model (NPM) [5, 6]—we identified the importance of collective action in routinely incorporating complex interventions into everyday practice. We showed how collective action was organised around interactions between users and the properties of intervention components.

  2. Agents: What is the work of implementing a new technique, technology or organisational intervention? In the second iteration of the theory—Normalization Process Theory (NPT) [7, 8]—we characterised mechanisms (coherence, cognitive participation, collective action and reflexive monitoring) that motivate and shape implementation processes and explained their operation.

  3. Contexts: How are structural and cognitive resources for implementation mobilised and what mechanisms lead to variations in implementation processes over time and between settings? In the most recent iteration of the theory—Extended Normalization Process Theory (ENPT) [9, 10]—we pointed to the dynamic role of implementation contexts in the mobilisation and negotiation of implementation processes.

Underpinning these practical questions is one that is fundamental to the social and behavioural sciences—and especially to behavioural economics, sociology and social psychology—which is how can we best understand the dynamics of human agency under conditions of constraint [10]? The important implication of this question is that well-designed, theoretically informed studies in implementation research actually offer opportunities for basic investigations in the social sciences.

The purpose of this review

A review by McEvoy et al. [17], published in 2014, provided a qualitative synthesis of 29 early and heterogeneous studies in which NPT was used. It drew attention to a positive response from healthcare researchers to the theory, but it also made three important critical points about the emerging NPT literature. McEvoy et al. [17] pointed to the ways that early studies using NPT did little work to justify the choice of theory, called for the prospective application of NPT to data analysis and collection and stressed the importance of moving beyond single stakeholder perspectives.

In the period since McEvoy et al.’s review [17], studies using NPT have proliferated. There are now a large number of protocols, empirical studies and reviews in which NPT plays a role. Importantly, a large number of NPT studies have now been completed by groups who are independent of the theory’s architects. It is therefore an opportune time to undertake a qualitative systematic review that will (i) identify and characterise the uses and limits of NPT in research on the implementation and integration of healthcare interventions and (ii) explore NPT’s contribution to understanding the dynamics of these processes.

Methods

Systematic citation searches

As the aim of this qualitative systematic review was to identify the uses of NPT in research on the implementation and integration of health care interventions since the publication of the first iteration of the theory in 2006, our search strategy was focused on citations. Following Kirk et al.’s review of reports of the Consolidated Framework for Implementation Research [18], we searched two bibliographic two databases (Scopus and Web of Science), and a search engine (Google Scholar), to search for citations of key papers that developed or expounded the main constructs of NPT [59, 11, 12], papers that developed NPT related methods or tools [1315] and citations of the NPT web-enabled on-line toolkit (www.normalizationprocess.org) [16]. Searches were conducted by AC, MG, CRM, MM and TLF. The sensitivity of the search strategy was tested against a database of studies using NPT that had been collected by three of the co-authors (CRM, TR, TF). All studies already known to use NPT at December 2015 were identified by the first round of systematic searches. Searches were initially undertaken in June 2015 and were updated in December 2015, August 2016 and March 2017. A final search was undertaken in December 2017.

Inclusion and exclusion criteria

We included the following: peer-reviewed English language journal articles reporting empirical research on the implementation of healthcare interventions, in which NPT was the primary analytic framework (applied either prospectively in study design and data collection, or retrospectively in the interpretation of already collected data) and which were undertaken in any healthcare setting. We define an empirical paper as one that contains evidence of data collection and analysis. We included studies that used any method of empirical investigation (qualitative, quantitative, and mixed methods).

We excluded the following: papers in which NPT was used as a framework for systematic reviews or meta-syntheses; papers solely on patient and caregiver experiences; papers in which NPT was not the primary analytic theory; editorials, theory and methods discussion papers; papers containing passing references to NPT; study protocols; papers describing work undertaken in settings other than healthcare; and papers published in languages other than English. We also excluded theses or dissertations, books and book chapters, conference proceedings and abstracts. We did not exclude papers on the grounds of methodological quality. We already knew that the literature ranged from student projects through to process evaluations in large and well-designed clinical trials in which NPT informed all activities from design through process evaluation and follow-up, to interpretation of trial outcomes. All studies were equally interesting to us, because we were searching for information about the way in which the theory was used rather than the summative results of NPT analyses.

Screening

Screening started with an assessment of citations and abstracts’ relevance by reviewers who had not been involved in the development of NPT (AC and MM). Reports that met eligibility criteria were obtained in full text. Full-text papers were screened by pairs of reviewers (AC with MM or CRM; MB with CRM; or CRM and TF) working independently of each other. Full-text screening consisted of identifying papers where NPT was clearly the analytic framework for an empirical study. Because no ‘one best way’ to operationalise NPT and its constructs has been prescribed, we did not apply judgments about this to screened papers. This meant that screening involved a simple Yes/No question, and references were sorted within Endnote Libraries accordingly.

Data extraction

We developed an extraction instrument, (see Additional file 1: Appendix 1). Data were extracted by all authors except CMM, FSM and EM. To avoid conflicts of interest, authors or co-authors of included papers were not involved in extracting data from those papers. Data were extracted on authors, year of publication, health care problem addressed, study type and methods, data collection procedures, how NPT was used in the study and whether this had been pre-specified in the study protocol. We looked for data on whether and how NPT had contributed to understanding the dynamics of the processes of implementation and integration, and for authors’ views about the limitations of NPT in terms of both its scope (what the theory explains) and application (what happens when researchers use the theory). As this was a qualitative review, we included data from both the results and discussion sections of included papers.

Data analysis

Coding and initial interpretation work was undertaken using the extraction instrument. To ensure consistency, CRM and TLF jointly checked coding on 75/130 of included papers, and CRM and CMM jointly checked categorisation of all included papers. The analysis aimed (i) to identify and characterise the uses and limitations of NPT in research on the implementation and integration of healthcare interventions and (ii) to explore NPT’s contribution to understanding the dynamics of these processes. Hence, we started by describing how NPT had been used and subsequently analysed the data to explore the ways that mechanisms defined by NPT have been revealed to operate. We sought to understand the relative importance of specific NPT constructs across different settings (core processes and mechanisms) and differences that seemed to apply in relation to different intervention types and healthcare systems (contingent processes and mechanisms).

Public registration of the review

PROSPERO deemed this review ineligible for public registration on the grounds that NPT was not a healthcare intervention.

Results

Search results

Searches revealed 3322 citations. In Fig. 1, we show that after eliminating 2337 duplicates and broken or junk URLs, 985 were screened as titles and abstracts. Of these, 101 were excluded because they did not fit the inclusion criteria for the review. This left 884 articles for full-text screening. Of these, 754 did not fit the inclusion criteria for the review. This left 130 papers presenting results from 108 identifiable studies to be included in the review.

Fig. 1.

Fig. 1

PRISMA flowchart

Types of studies

In this review, 130 papers reported the application of NPT in 108 identifiable studies. Included articles presented both controlled (n = 26) and uncontrolled (n = 82) studies.

In Table 1, we show that NPT was employed in 26 controlled studies—mainly complex intervention trials—and these generated 40/130 (30.8%) articles [1958]. These included an intervention design study (n = 1), feasibility studies (n = 5), process evaluations (n = 19) and retrospective documentary analyses (n = 1), embedded in complex intervention trials. Three of these studies used mixed methods, and one [55] was a survey. The remainder (n = 22) all used qualitative methods.

Table 1.

Controlled studies using NPT as their analytic framework

First author/first paper Country of origin Theory frame Research problem Evidence base cited to support intervention Use of NPT specified in protocol NPT study type Data collected Application of NPT to data Factors leading to intervention success or failure Differences between categories of participants Differences between settings
1. Ballinger et al. [19] UK NPT Stroke rehabilitation Systematic review [149] Process evaluation Qualitative Retrospective Yes No No
2. Bamford et al. [20] UK NPT New professional roles in dementia care Systematic review [150] Yes Process evaluation Qualitative Prospective Yes Yes No
3. Blakeman et al. [21] UK NPT Chronic kidney disease management (telephone support) NICE guideline [151] Yes Process evaluation Qualitative Prospective Yes Yes No
4. Blickem et al. [22] UK NPT Self-management support for long-term conditions (telephone support) Yes Process evaluation Qualitative Prospective Yes Yes No
5. Brooks et al. [23] UK NPT Care planning (mental health) Systematic review [152] Yes Intervention design Qualitative Prospective Yes Yes Yes
6. Buckingham et al. [24] UK NPT COPD management in primary care Systematic review [153] Feasibility study Mixed Prospective Yes Yes N/A
7. Clarke et al. [25, 26] UK NPT Stroke rehabilitation Systematic review [154] Yes Process evaluation Qualitative Prospective Yes Yes Yes
8. Coupe et al. [27] UK NPT Collaborative care for depression Yes Process evaluation Qualitative Retrospective Yes Yes Yes
9. Finch et al. [28] UK NPT Cognitive behavioural therapy Systematic review [156] Yes Feasibility study Qualitative Prospective Yes Yes N/A
10. Furler et al. [2931] Australia NPT Diabetes management in primary care Yes Process evaluation Qualitative Prospective Yes Yes No
11. Gabbay et al. [32] UK NPT Debt counselling for depression in primary care NICE guideline [178] Yes Process evaluation Qualitative Prospective Yes Yes Yes
12. Gask et al. [33] UK NPM Collaborative care for depression Systematic review [155] Yes Process evaluation Qualitative Retrospective Yes Yes No
13. Grant et al. [34, 35] UK NPT Primary care prescribing NICE guideline [157] Yes Process evaluation Qualitative Prospective Yes Yes No
14. Godfrey et al. [36] UK NPT Delirium prevention in hospital Systematic review [158] Yes Process evaluation Qualitative Prospective Yes Yes Yes
15. Hind et al. [37] UK NPT Aquatic therapy for children with Duchenne muscular Dystrophy Yes Feasibility Study Qualitative Prospective Yes No No
16. Hooker et al. [3842] Australia NPT Identifying women at risk of intimate partner violence Yes Process evaluation Mixed Prospective Yes Yes Yes
17. Kennedy et al. [4346] UK NPT Social network support in long-term conditions Systematic review [159] Yes Process evaluation Qualitative Prospective Yes Yes Yes
18. Khowaja et al. [47] India, Mozambique, Nigeria, Pakistan NPT Maternal health in low-income countries WHO guideline [160] Feasibility study Mixed Prospective Yes Yes Yes
19. Leon et al. [48] South Africa NPM Testing and counselling for HIV in South Africa Systematic reviews [161163] Process evaluation Qualitative Retrospective Yes Yes No
20. Mair et al. [49] UK NPM Telemedicine for COPD Systematic review [164] Process evaluation Qualitative Retrospective Yes Yes N/A
21. Ong et al. [5052] UK NPT Osteoarthritis guidelines in primary care NICE guideline [165] Process evaluation Qualitative Prospective Yes Yes Yes
22. Ricketts [53] UK NPT Chlamydia screening in primary care Process evaluation Qualitative Retrospective Yes No No
23. Speed et al. [54] UK NPM Management of constipation in primary care Yes Process evaluation Qualitative Prospective Yes Yes No
24. Sturgiss et al. [55] Australia NPT Weight management programme in primary care Yes Feasibility study Quantitative (survey) Prospective Yes No No
25. Thomas, L. et al. [56, 57] UK NPT Stroke rehabilitation (incontinence) RCP-ICSWP guideline [166] Yes Process evaluation Qualitative Prospective Yes Yes Yes
26. Willis [58] Australia NPT Community support for women with postnatal depression Yes Historical review of documents Textual analysis Retrospective Yes N/A N/A

N/A not available

In Table 2, we show that NPT was employed in in 82 uncontrolled studies, and these generated 90/130 (69.2%) articles [59148]. These included feasibility studies (n = 20) and process evaluations (n = 54), and seven were what we have called ‘field studies’ which focused on general conditions in which interventions might take place, rather than the progress of specific interventions. One study was an ethnography of a set of socio-technical practices [103]. Qualitative methods were used in 72 studies. Of the remainder, seven were mixed methods studies, two were surveys, and one was a prospective cohort study.

Table 2.

Uncontrolled studies using NPT as their analytic framework

Study Country of origin Theory frame Implementation problem Evidence base cited to support intervention Use of NPT specified in protocol NPT study type Data collected Application of NPT to data Factors leading to intervention success or failure Differences between categories of participants Differences between settings
27. Aarts et al. [59] Netherlands NPM Infertility support (online) Systematic review [167] Process evaluation Qualitative Retrospective Yes Yes N/A
28. Agbakoba et al. [6062] UK NPT Telecare/digital health in the community Systematic review [168] Yes Process evaluation Qualitative Prospective Yes Yes Yes
29. Alharbi et al. [63] Sweden NPT Person-centred care Process evaluation Qualitative Retrospective Yes Yes N/A
30. Ahmed et al. [64] UK NPT Screening questionnaire (genetic conditions in primary care) Systematic review [169] Feasibility study Qualitative Retrospective Yes No No
31. Alverbratt et al. [65] Sweden NPT Patient assessment tool in psychiatry Process evaluation Qualitative Prospective Yes Yes Yes
32. Ariens et al. [66] Netherlands NPT Teledermatology Yes Process evaluation Quantitative (survey using eHit Toolkit [226]) Prospective Yes No No
33. Atkins et al. [67] South Africa NPM Supporting treatment adherence in tuberculosis Systematic review [170] Process evaluation Qualitative Retrospective Yes Yes No
34. Bamford et al. [68] UK NPT Nutrition guidelines FSA guideline [171] Yes Process evaluation Qualitative Prospective Yes Yes No
35. Basu et al. [69] UK NPT Improving motor outcome in infants after perinatal stroke Feasibility study Qualitative Prospective Yes No N/A
36. Bayliss et al. [70] UK NPT Training for chronic fatigue management NICE guideline [172] Feasibility study Qualitative Prospective Yes Yes No
37. Bee et al. [71] UK NPT Cognitive behavioural therapy by phone Systematic reviews [227, 228] Feasibility study Qualitative Prospective Yes No No
38. Bocum et al. [72] Burkina Faso NPM Antenatal syphilis screening Feasibility study Qualitative Retrospective Yes No Yes
39. Bouamrane and Mair [73] UK NPT Surgical assessment (online) Systematic review [168] Yes Process evaluation Qualitative Prospective Yes No N/A
40. Bouamrane and Mair [74] UK NPT Electronic referrals (online) Systematic review [168] Yes Process evaluation Qualitative Prospective Yes No N/A
41. Bouamrane and Mair [75] UK NPT Surgical assessment (online) Systematic review [173] Yes Process evaluation Qualitative Prospective Yes Yes N/A
42. Bridges et al. [76] UK NPT Compassionate nursing care Systematic reviews [76, 229] Yes Process evaluation Qualitative Prospective Yes No Yes
43. Chiang et al. [77] Australia NPT Risk assessment tools Systematic review [174] Feasibility study Qualitative Prospective Yes No No
44. Conn et al. [78] Canada NPT Improving recovery after colorectal surgery Meta-analysis [175] Process evaluation Qualitative Retrospective Yes Yes No
45. Desveaux et al. [79] Canada NPT Hospital accreditation Systematic review [230] Process evaluation Qualitative Retrospective Yes Yes yes
46. Dickinson et al. [80] UK NPT Cognitive stimulation for people with dementia Process evaluation Qualitative Retrospective Yes Yes Yes
47. Dikomiitis et al. [81] UK NPT Decision support tool for cancer Feasibility study Qualitative Prospective Yes No No
48. Drew et al. [82] UK ENPT Fracture prevention clinics NICE guidelines [176, 177] Process evaluation Qualitative Retrospective Yes Yes No
49. Dugdale et al. [83] UK NPT Substance misuse management (online) Process evaluation Qualitative Prospective Yes Yes No
50. Ehrlich [84] Australia NPT Care coordination in long-term conditions Yes Field study Qualitative Prospective N/A N/A N/A
51. Finch [85] UK NPM Telecare/telemedicine Field study Qualitative Prospective Yes No No
52. Franx et al. [86] Netherlands NPT Collaborative care for depression NICE guideline [178] Yes Process evaluation Qualitative Retrospective Yes Yes No
53. French et al. [87, 88] UK NPT Stroke management using telecare Systematic review [179] Yes Process evaluation Qualitative Prospective Yes Yes No
54. Foss et al. [89] Norway NPT Social network mapping for chronic disease management Systematic review [231] Yes Process evaluation Qualitative Prospective Yes No No
55. Foster et al. [90] Australia NPT Diabetes management Systematic review [180] Feasibility study Qualitative Prospective Yes Yes No
56. Gould et al. [91] UK NPT Infection prevention and control Process evaluation Qualitative Retrospective Yes Yes No
57. Green et al. [147] UK NPT Cancer risk assessment tool NICE guideline [181] Feasibility study Qualitative Retrospective Yes N/A N/A
58. Gunn et al. [92] Australia NPT Reorganisation of primary care mental health services Systematic review [155] Process evaluation Qualitative Retrospective Yes No Yes
59. Hall et al. [93] UK NPT Monitoring technologies in care homes for people with dementia Systematic review [232] Process evaluation Qualitative Retrospective Yes Yes Yes
60. Hall et al. [94] UK NPT Supporting staff working with people with autism Yes Process evaluation Qualitative Prospective Yes No No
61. Hazell et al. [95] UK NPT Guided self-help cognitive therapy NICE guideline [233] Yes Process evaluation Quantitative (survey) Prospective Yes Yes N/A
62. Henderson et al. [96] UK NPT Diagnostic decision support in primary care Systematic review [167, 182] Process evaluation Mixed Prospective Yes No N/A
63. Herbert et al. [97] UK NPT Enhanced recovery after surgery Process evaluation Qualitative Prospective Yes Yes N/A
64. Hoberg et al. [98] USA NPM Group therapy model APA guideline [234] Feasibility study Qualitative Prospective Yes No No
65. Holtrop et al. [99] USA NPT (collective action constructs) Care management for chronic disease in primary care Yes Process evaluation Qualitative Prospective Yes No Yes
66. Kanagasundaram et al. [100] UK NPT Diagnostic decision support (acute kidney injury) NICE guideline [183] Feasibility study Mixed Retrospective Yes Yes N/A
67. Kulnik et al. [101] UK NPT Inter-professional self-management support Systematic review [184] Process evaluation Mixed Prospective Yes Yes Yes
68. Johnson et al. [102] UK NPT Guideline implementation Overview of systematic reviews [235] Yes Process evaluation Quantitative (prospective cohort intervention) Prospective Yes Yes N/A
69. Jones, C. et al. [103] UK NPT Diagnostic point of care testing Ethnographic case study Qualitative Prospective Yes Yes N/A
70. Jones, F. et al. [104] UK NPT Self-care training programme for stroke practitioners Process evaluation Qualitative Retrospective Yes No No
71. Leggat et al. [105] Australia NPT Quality improvement in hospitals Systematic review [236] No Process evaluation Qualitative Retrospective Yes Yes Yes
72. Lhussier et al. [106] UK NPT Care planning in primary care No Field study Qualitative Retrospective Yes Yes N/A
73. Ling et al. [107] UK NPT Integrated care policy Process evaluation Qualitative Retrospective Yes Yes Yes
74. Lloyd et al. [108, 109] UK NPT Shared decision-making tools Systematic review [185] Yes Feasibility study Qualitative Retrospective Yes Yes Yes
75. Lowrie et al. [110] UK NPT Chronic heart failure management in the community NICE guideline [186] Feasibility study Qualitative Retrospective Yes Yes N/A
76. Martindale et al. [111] UK NPT Management of acute kidney injury in the community NICE guideline [183] Process evaluation Qualitative Prospective Yes Yes Yes
77. May et al. [112] UK NPT Telecare for chronic disease management in the community Systematic review [164] Yes Process evaluation Qualitative Prospective Yes Yes Yes
78. Morton and Wigley [113] UK NPT Nursing assessment tool for maternal/child health in the community Yes Process evaluation Qualitative Prospective Yes No N/A
79. Murray et al. [114] UK NPT E-health systems Systematic review [187] Yes Process evaluation Qualitative Prospective Yes Yes Yes
80. Newton [115] Australia NPT Caseload midwifery models Systematic review [188] Yes Process evaluation Mixed Prospective Yes No N/A
81. Nordmark et al. [116] Norway NPT Discharge planning Systematic review [189] Feasibility study Qualitative Prospective Yes Yes Yes
82. O’Connell and Kaner [117] UK NPT Alcohol brief interventions in primary care Field study Qualitative Retrospective Yes No N/A
83. Owens and Charles [118] UK NPT Text messaging in child and adolescent mental health services Systematic review [190] Yes Feasibility study Qualitative Prospective Yes No N/A
84. Polus et al. [119] Australia NPM Chiropractic services for indigenous Australians Feasibility study Qualitative Prospective Yes Yes N/A
85. Pope et al. [120, 121] UK NPT Decision support tools for emergency services Yes Process evaluation Qualitative Retrospective Yes Yes Yes
86. Røsstad et al. [122] Norway NPT Care pathways for older patients Systematic review [191] Process evaluation Qualitative Retrospective Yes Yes No
87. Sanders et al. [123] UK NPT Back pain management in primary care Yes Process evaluation Qualitative Retrospective Yes No N/A
88. Scalia [124] USA NPT Option Grid decision support tools Systematic reviews [185, 237] Yes Field study Qualitative Prospective Yes No Yes
89. Scantlebury [125] UK NPT Maternity unit electronic health record Systematic review [192] Yes Process evaluation Qualitative Prospective Yes Yes N/A
90. Segrott et al. [126] UK ENPT Adolescent substance misuse programmes Systematic review [193] Yes Process evaluation Mixed Prospective Yes Yes Yes
91. Shemeili [127] Abu Dhabi NPT Medicines management in hospital care of older people Yes Process evaluation Qualitative Prospective Yes No N/A
92. Shulver et al. [128] Australia NPT Telecare for older people Yes Field study Qualitative Prospective Yes Yes Yes
93. Spangaro et al. [129] Australia NPM Screening for intimate partner violence Systematic review [238] Process evaluation Qualitative Retrospective Yes No N/A
94. Stevenson [130] UK NPT UK Clinical Practice Research datalink Yes Process evaluation Qualitative Prospective Yes No No
95. Tarzia et al. [131] Australia NPT Decision-making for older adults with dementia Field study Qualitative Retrospective Yes Yes N/A
96. Tazzyman et al. [148] UK NPT Revalidation of medical practitioners Yes Process evaluation Qualitative Prospective (structured through the NoMAD Questionnaire) Yes Yes N/A
97. Temple-Smith et al. [132] Australia NPT Chlamydia testing in general practice Yes Process evaluation Mixed Prospective Yes No No
98. Teunissen et al. [133136] Austria, England, Ireland, Greece, Netherlands NPT Migrant health Yes Process evaluation Qualitative Prospective Yes Yes Yes
99. Thomas et al. [137] Sweden ENPT Healthy lifestyle promotion in primary care Process evaluation Mixed Retrospective Yes Yes Yes
100. Tierney et al. [138] Ireland NPT Interdisciplinary teams in primary care Systematic review [194196] Yes Process evaluation Quantitative Prospective Yes Yes No
101. Toye et al. [139] Canada NPT Assessment instrument for homecare Yes Feasibility study Qualitative Prospective Yes Yes Yes
102. Trietsch et al. [140] Netherlands NPT Quality improvement collaboratives Systematic review [197] Process evaluation Qualitative Retrospective Yes Yes Yes
103. Vest et al. [141] US NPT Clinical guideline implementation in chronic kidney disease ACP guideline [198] Process evaluation Qualitative Retrospective Yes N/A N/A
104. Volker et al. [142] Australia NPT Cardiovascular disease prevention Process evaluation Qualitative Prospective Yes Yes Yes
105. Webster et al. [143] UK NPT Delivery of a psychosocial intervention for people with depression and long-term conditions Yes Process evaluation Qualitative Prospective Yes No No
106. Walker et al. [144] Australia NPT Colorectal cancer risk prediction NICE guideline [199] Feasibility study Qualitative Retrospective Yes No No
107. Wilhelmsen et al. [145] Norway NPT Web-based cognitive behavioural therapy Systematic reviews [200, 201] Feasibility study Qualitative Retrospective Yes No No
108. Wilkes et al. [146] UK NPM Open access infertility clinics Feasibility study Qualitative Retrospective Yes Yes No

N/A not available

What was being implemented?

Studies included in this review fell into seven categories. The most numerous group of studies were those concerned with service organisation and delivery (n = 29, 26.9% [23, 27, 3235, 4346, 58, 76, 79, 82, 84, 86, 89, 91, 92, 99, 105107, 110, 115, 116, 119, 122, 127, 133136, 140, 146, 148]). For example, in the UK, Grant et al. [34, 35] evaluated a complex intervention aimed at reducing risk in prescribing in primary care. They used NPT in ‘identifying and describing the components and sub-components of the intervention’ to understand ‘the nuances associated with collective implementation’. The next most numerous group of studies focused on the implementation of diagnostic and therapeutic interventions (n = 28, 25.9% [19, 2426, 2831, 36, 37, 47, 48, 5557, 67, 69, 78, 80, 90, 95, 97, 98, 103, 104, 111, 117, 123, 126, 137, 142, 143]). For example, in the USA, Hoberg et al. [98] examined the implementation of a new form of group therapy for people with mental health problems, while Leon et al. [48] showed how provider initiated testing and counselling for HIV was successfully normalised in a South African setting. Studies of implementation of E-Health and telemedicine—including telephone advice—were also numerous (n = 21, 19.4% [21, 22, 49, 5962, 66, 71, 7375, 83, 85, 87, 88, 93, 112, 114, 118, 125, 128, 130, 145]). Here, a Norwegian team led by Wilhelmsen et al. [145] showed how problems of participation and action—and especially the interactional workability—of a service providing internet-based cognitive behavioural therapy led to ambivalence on the part of general practitioners about its use, to low levels of follow-up and to doctors reverting to ‘standard treatment’ [145]. Less numerous (n = 11, 10.1%, were studies of the implementation of screening and surveillance tools [3842, 53, 64, 65, 72, 77, 113, 129, 132, 139, 144]). In a feasibility study, Ahmed et al. [64] showed that integrating a family history questionnaire about common genetic diseases into the workflow of primary care was unlikely without significant changes to the pattern of GP-patient interactions, and these were unlikely to be supported by clinicians. Such professional factors also affected the outcome of studies of decision support and shared decision­making (n = 8, 7.4% [81, 96, 100, 108, 109, 120, 121, 124, 131, 147]). In this category, in the USA, Scalia et al. [124] compared the implementation and integration of decision support tools between two major healthcare systems. This study raised important questions about how the interactions between clinicians’ (micro-level) experiences of the workability of complex interventions and meso-level organisational processes through which reflexive monitoring mechanisms play out their effects. Some studies were also explicitly concerned with implementing change in professional roles (n = 7, 6.5% [20, 54, 63, 70, 94, 101, 138]). For example, Thomas et al. [56, 57] showed how changes in roles and workload interacted to promote the routine embedding of an intervention intended to manage incontinence in stroke patients. Finally, a small group of studies were concerned with guideline implementation (n = 4, 3.7% [5052, 68, 102, 141]). Here, Vest et al. [141] described a study in the USA of the implementation of guidelines for the management of chronic kidney disease in primary care. They asserted that NPT could not only identify key barriers to practice but could also guide intervention choice.

Was what was being implemented evidence-based?

Studies included in this review were mainly focused on reporting the implementation of complex healthcare interventions. Most of these studies had a translational component and made some claim about the evidence underpinning interventions. This evidence was heterogeneous and included qualitative studies [120, 121], implementation appraisals [133], meta-ethnographies [137] and previous trial results [3841]. However, the most common appeal to an evidence base in studies included in this review was through references to systematic reviews and rigorously developed clinical guidelines. Across the studies included in the review, 64/108 (59.2%) were linked to such support by their authors [149201]. As Tables 1 and 2 show, systematic reviews and rigorous guidelines were cited in support of 17/26 (65.4%) controlled studies and 47/82 (57.3%) uncontrolled studies.

How did researchers justify the use of NPT?

As Tables 1 and 2 show, in 54/108 (50%) of the studies included in this review, the use of NPT appeared to have been planned in advance, and this was included in the study protocol. Amongst controlled studies, 19/26 (73%) of studies made explicit reference to including NPT in study protocols, while only 35/82 (42.7%) of uncontrolled studies did so. Not all papers offered a justification for using NPT. For the most part, authors characterised NPT as a conceptual framework that explains implementation processes and thus structures study design and data analysis. For example, Brooks et al. [23] justify it thus:

‘Normalisation Process Theory (NPT) has been used to consider complex interventions prior to the development of a randomised control trial to test their effectiveness (…). It has also been used in the context of mental health to explore the impact of new forms of collaborative care on the way in which professionals carry out their routines of work in primary care (…). The four constructs (coherence, cognitive participation, collective action and reflexive monitoring) permit a means of appraising factors that might ‘promote and inhibit the routine incorporation of complex interventions into everyday life’ (…). It focuses on the work that people need to do to ensure interventions become ‘normalised’. As a heuristic framework it can support the optimisation of a trial intervention at three points:

  • supporting intervention design

  • describing the context of a trial

  • supporting the interpretation of a trial’s results’ [23].

Other papers reflected in more general terms on NPT’s empirical grounding (e.g. [28, 50, 52, 67, 7375, 87, 114, 122]) and its usefulness in thinking about implementation design (e.g. [27, 33, 67, 106, 147, 202, 203]).

Did NPT explain implementation outcomes?

In all but one study in the review [84], there was evidence that implementation outcomes could be explained by reference to the mechanisms specified by NPT. For example, Scalia et al. [124], state that their study

suggests that patient decision aids that are specifically designed for use in clinical encounters can be embedded in clinical settings, provided there is agreement about the need to use them, that the team members are willing to work together to make sure that such tools can be integrated in existing work patterns, and understood as making a positive overall contribution to the work that has to be performed. These considerations match the mechanisms of the NPT, which provides an explanatory framework for understanding the sustained use of these tools by the two systems examined. The motivation for the use of the Option Grid at CapitalCare was their wish to achieve success in an external quality improvement initiative. At HealthPartners, implementation efforts were motivated by a ‘champion’ physician. The nursing staff also played a pivotal role by systematically identifying eligible patients and providing those patients with the relevant encounter tool. These organizations, in different ways and to different degrees, exhibited coherence, collective action and cognitive participation that supported the sustained use of the tools. The organizational appraisal, in other words, their reflexive monitoring, was positive overall, despite concerns about readability and time pressures.

(Part omitted)

Implementing patient decision aids into clinical settings is a difficult process (…) In the UK, an implementation program known as MAking Good Decisions In Collaboration (MAGIC) highlighted the need for an organizational coherence, i.e. a widely held and agreed understanding of SDM principles in order to facilitate the implementation of patient decision aids (…). Commitment at multiple organizational levels has been recognized as an important precondition for implementation (…). This lack of commitment was noticeable at the CapitalCare sites that did not use patient encounter tools [124].

Differences between participant groups were characterised in 69/108 (64%) studies and between settings in 36/108 (33%) studies. For example, Clarke et al. [26] placed this in the wider context of levels of analysis.

‘This paper briefly considers implementation theories in respect of complex interventions and provides an overview of process evaluations to set the context for the study. We draw on Normalisation Process Theory (NPT) (…) as a conceptual lens through which to explore those features of the implementation process that were intended to secure practice change and to engage caregivers in the program. We also consider the interaction between influential macro and micro contextual factors that affected delivery by multi-disciplinary stroke unit staff and suggest that prior focus on generative mechanisms identified within NPT can be used to inform implementation processes within complex healthcare settings’ [26].

NPT thus characterises core elements of implementation processes and the factors that shape them, and using NPT enabled researchers to explain the ‘work’ that is involved in implementation. Implementation involves interactions between mechanisms and contexts that are highly complex and emergent. Dynamic elements of context can exercise powerful constraints on action. The sources of these constraints included system-level processes that structured behaviour (e.g. the role of fee for service payments in undermining the implementation of self-care programmes [43]) and micro-level conflicts within contexts (e.g. disagreements over participation and intervention legitimacy [20, 25]).

How did researchers apply the theory’s constructs?

Implementation processes in NPT are explained by the operation of social mechanisms that motivate and shape collective action. Researchers using the theory employed its constructs in four distinctive ways, irrespective of the iteration of the theory that they used. We show examples of these diagrammatically below. First of all, some researchers clearly found it helpful to see the theory as describing a linear process in time [22, 63], in which the operation of mechanisms followed sequentially from each other (Fig. 2). In these studies, sense-making was seen as a necessary precursor to participation, and a degree of cognitive participation was required before collective action—in the form of an actual implementation process—could take place. Reflexive monitoring was seen as the final stage in the implementation process. However, research reported in this review often focused on feasibility studies or on the early stages of implementation life cycles in process evaluation. This skews their analyses towards the implementation phase of studies rather than their embedding and integration in everyday practice.

Fig. 2.

Fig. 2

Interactions between NPT mechanisms: sequential operation over time (e.g. Alharbi et al. [63])

As Fig. 3 shows, the focus on the ‘front end’ of studies leads to an emphasis on ‘coherence’ and sense-making work as an obligatory point of departure for implementation processes (e.g. [27, 31, 50, 52, 57, 74, 85, 108, 113, 123, 131, 203]), sometimes at the expense of other activities. Figure 4 suggests a novel analysis of the relationship between mechanisms. Holtrop et al. examined the operation of components of collective action in the restructuring of provider reimbursement. Their analysis emphasised the role of relational integration as a precondition for normalisation. In this context, the operation of one mechanism might be an obligatory point of passage for the others. Holtrop et al. [99] state that

‘We found that effective care management normalization required relationship development between practice providers and staff and the care manager. Since identification and referral of patients needing care management was key to care management happening at all, the practice personnel understanding and appreciating the care manager role through a relationship with the care manager was critical. This was captured well through the NPT collective action component of relational integration. We interpreted relational integration to be the professional relationship development that occurred when care manager, providers and practice staff work together and understand and appreciate each other’s roles and contribution to patient care. Although it is its own component in NPT, we found it to be more of an outcome that occurred when the other components worked well (contextual integration, skill set workability and interactional workability). (…) We found that when any of the other components were not in place, there was also a lack of development of trust around shared patient care. Since care management is a relationship rich endeavor, the lack of this relationship is a key factor in care management’s disuse’ [99].

Fig. 3.

Fig. 3

Interactions between NPT mechanisms: obligatory starting point (e.g. Finch [85])

Fig. 4.

Fig. 4

Interactions between NPM mechanisms: relational integration as an obligatory point of passage (Holtrop et al. [99])

Finally, as Fig. 5 shows, NPT assumes that its constituent mechanisms can operate simultaneously—but unevenly—rather than sequentially. Few studies in this review tracked the implementation of a complex intervention over its whole life. When they did, they tended to present summative rather than a formative accounts. An interesting example of a longitudinal study may be found in work by Tazzyman et al. [148] that depicted NPT in precisely these terms. They state that the mechanisms specified by NPT are

non-linear and interact dynamically to provide a comprehensive explanation of the implementation processes. NPT was designed to be applied flexibly, can be used at one or more points in a qualitative study, has been successfully used beyond its original field and provides a robust theoretical framework to understand the dynamics of implementation [148].

Fig. 5.

Fig. 5

Interactions between NPT mechanisms: constant interaction between mechanisms (e.g. Hooker et al. [3841])

Tazzyman et al. [148] explored the processes that underpinned revalidation of medical practitioners in a qualitative study of senior decision-makers undertaken at three time points (2011, 2013 and 2015). They characterised respondents in their study in NPT terms as ‘sense-makers’ and then explored the process of implementing and embedding of revalidation as a broad policy initiative. They state that their contribution

has been to extend the use of NPT to explore the implementation of a broad and complex policy, with wide ranging implications for an entire profession, and the wider healthcare system. Much previous work using NPT in healthcare has addressed the implementation of micro level interventions. This expanded application of NPT has highlighted a number of factors which seem to have affected the implementation of revalidation. The four dimensions of the framework (see Table 3) had an intuitive relevance and provided a useful explanatory framework for understanding the implementation of revalidation. There is scope to apply NPT more widely to complex social interventions and policy initiatives at the organisational and system level in future [148].

Table 3.

Alverbratt et al. operationalise all constructs and subconstructs of NPT [65]. (Reproduced from the Journal of Hospital Medicine published under Creative Commons Attribution (CC-BY) licence)

Coherence
‘The significant qualities DLDA)
Cognitive participation ‘Enrolment and engagement of individuals and groups’ Collective action ‘Interaction with already existing practices’ Reflexive monitoring ‘How a practice is understood and assessed by actors implicated in it’
Differentiation. Understanding the difference between DLDA and ‘the old fashioned way’ of working in a psychiatric nursing context. Initiation. The participants’ motivation in trying to incorporate the DLDA Tool. Interactional workability. Operating DLDA. Systematisation. The participants’ judgement of DLDA regarding usefulness and effectiveness.
Communal specification. The process through which users through teamwork share and create an understanding of this new practice. Enrolment. The work participants do to organise themselves and their co-workers in the practice of DLDA. Relational integration. Participants understandings of DLDA not only being aware of how and when to use DLDA, but also understanding the expressions of other staff members. Communal appraisal. Communal appraisal regarding the outcomes and values of DLDA.
Individual specification. The process in which users create an understanding of the new practice. Legitimation. The belief that DLDA is right for the context in terms of being a needed complement to existing tools and approaches. Skill-set workability. Refers to how DLDA is conducted and distributed. This will influence how the work is defined and divided between participants. Individual appraisal. Individual appraisal regarding the outcomes and value of DLDA.
Internalised meaning. The coherence of DLDA was based on the meaning users collectively invest in it. Activation of DLDA. What the participants could do together to improve conditions for DLDA to be sustained and become part of daily practice. Contextual integration. The incorporation of DLDA into a social context of the current wards. Reconfiguration. Suggestions from participants that aim to modify and enhance the utility of the DLDA Tool.

More usually, longitudinal studies using NPT were process evaluations embedded in large complex intervention trials. As we have noted above and elsewhere [10], these permitted a more structured analysis of implementation processes and their motivating mechanisms over time [2931, 3841, 4346, 5052].

How did researchers integrate NPT into their research methods?

Researchers used two main strategies to translate the constructs of NPT into practically useful analytic tools. Some used deductive strategies that relied on framework or directed content [204, 205] analyses and in which interpretation of data was structured prospectively by the theory. These approaches often took the form of relating data to matrices of varying degrees of complexity. In Table 3, we show how Alverbratt et al. [65] have created a detailed matrix in which they reinterpret and operationalise all constructs and sub-constructs of NPT. This partly replicates the way that these were originally characterised in May and Finch’s account of NPT [7]. This approach defines and sets out all of the constituent elements of the work that drives implementation processes and permits data collection and coding using framework, or directed, content analysis [205]. The approach taken by Alverbratt et al. focuses on translating the content of the theory into practical research questions in a very precise way. Others focused on the main constructs of the theory prospectively, but within a more flexible framework. In Table 4, we show how Røsstad et al. [122] set out a matrix that links theory constructs to a description of data collected and in Table 5, we show how Nordmark et al. offer an even simpler data matrix, in which core constructs are linked to data collection opportunities [116]. Tazzyman et al. [148] used an analytic approach included both deductive and inductive elements.

A coding framework was developed using the four domains and sub-domains of NPT by using an adapted version of the NoMAD instrument (part omitted), which was developed to assess implementation processes (Normalization Measure Development is an instrument designed for assessing the implementation of complex interventions). The adapted NoMaD instrument was applied to the transcripts by coding evidence of the sub-domains in Dedoose [206]. Following coding, two members of the research team (AT and JF) analysed the data across the three interview stages, using the constant comparative method, in order to understand changes and continuities over time. The inductive method of constant comparison analysis involved searching within individual transcripts, making comparison between transcripts within the same cohort, and comparing transcripts from different cohorts for conceptual similarities and differences. This method was combined with the deductive approach of using the four domains on NPT as a framework for the analysis.

Table 4.

Røsstad et al. link constructs to data and compare sites [122]. (Reproduced from BMC Health Services Research, published under a Creative Commons Attribution (CC-BY) licence)

Municipalities
A B C D E F
PaTH in use in full scalea Elements of PaTH in usea PaTH not in usea
Makes sense (coherenceb)
 Expecting PaTH to be useful Yes Yes Yes Yes Yes Yes
 Regular staff understood how to use PaTH Mixed Mixed Mixed Mixed Mixed Mixed
Commitment and engagement (cognitive participationb)
 Sustained leadership Yes Yes No No No No
 Practice in using checklists Intensive Intensive Minimal Minimal Minimal Minimal
 General attention to PaTH at workplace Yes Yes No Nurses only No No
Facilitating use of PaTH (collective actionb)
 Extra personnel resources Yes Yes No Yes No No
 Major competing priorities No No No No Yes Yes
 Usability in electronic health record Good Fair Poor Poor Poor Poor
 Working schedule facilitated for PaTH Yes Yes No No No No
 Checklists incorporated in daily routines Yes Yes No No No No
Value of PaTH (reflexive monitoringb)
 Impact on collaboration with the hospital Mixed Mixed No No No No
 Impact on collaboration with GPs Yes Yes No Yes No No
 Impact on service quality Yes Yes No Yes No Yes
 Value for individual nurse/nursing assistant Yes Yes No No No No
 Valued as a management tool Yes Yes No Yes No No

aAssessed 24 months (B–F) and 32 months (A) after introduction of PaTH in the municipalities

bCore constructs of the Normalization Process Theory

Table 5.

Nordmark et al. link NPT related questions to a data matrix [116]. (Reproduced from BMC Medical Informatics and Decision-Making under a Creative Commons Attribution (CC-BY) licence)

Coherence Cognitive participation Collective action Reflexive monitoring
What is the process? Who performs the process? How does the process get performed? How is the process understood?
How RNs, DNs and HCOs perceived the DPP and whether they experienced the DPP as valuable to them and agreed about its usefulness and purpose Whether RNs, DNs and HCOs saw the DPP as a legitimate part of their work and whether they supported it over time How the DPP was provided within the existing context, how the embedding and integration work had proceeded due to knowledge and resources How RNs, DNs and HCOs individually and collectively evaluated the DPP and its supportive tools
Factors that promote or inhibit the routine embedding of DPP. Factors that promote or inhibit participation in DPP Factors that promote or inhibit enacting DPP Factors that promote or inhibit appraisal of DPP
Data source No. of text units
Survey 0 1 12 0
Interview RNs 0 119 225 78
Interview DNs, HCOs 0 122 80 59
Adverse events/information system failures 0 3 2 0
Workshops 12 8 37 6

Tazzyman et al.’s hybrid approach enabled them to develop a theory-led analysis, without needing to force data into a rigid theoretical framework. However, many studies took a more straightforward inductive approach to data collection and analysis. When studies collected and analysed qualitative data inductively—in the light of NPT—rather than deductively using framework approaches, there was less pressure on them to interpret their qualitative data within an inflexible coding framework. For example, in Table 6, we show how Bamford et al. [20] described the ways that their inductively generated data categories mapped on to NPT constructs. This group of papers includes a group of highly illuminating studies across the life course of complex intervention trials. Bamford et al.’s [20] process evaluation of the CAREDEM trial, and Kennedy et al.’s [4346] account of the WISE trial explain how structural factors militated against processes of cognitive participation. In their longitudinal accounts of the MOVE [3841] and STEPPING-UP [2931] Trials, Hooker et al. and Furler et al. show how mechanisms of coherence, cognitive participation and collective action interact to support the embedding of complex interventions in practice. Importantly, these studies also showed that the intervention remained in play once the trials themselves had concluded.

Table 6.

Bamford et al. [20] retrospectively map inductively generated themes onto NPT constructs. (Reproduced from BMC Health Services Research, published under a Creative Commons Attribution (CC-BY) licence)

Mapping of overarching themes and subthemes to NPT framework
NPT construct Theme Subthemes
Coherence Making sense of the case manager intervention Perceived value of the concept of case management.
Clarity over the case manager role.
Cognitive participation Investment in case management Practice investment in case management.
Investment by case managers.
Fit of case management with existing skill sets.
Collective action Implementing case management in practice Time available for case management.
Implementation in research vs clinical practice.
Support and supervision of case managers.
Reflexive monitoring Appraising and embedding of case management Assessing the impacts of case management.
The ‘right’ intervention but at the wrong time.
Embedding case management in practice.

How did users’ criticise NPT

Critique of NPT as a theory was rare amongst the papers included in this review. However, it was not absent. For example, Clarke et al. [26] criticised an over-emphasis on agency at the expense of implementation contexts in NPT.

‘While May et al (…) acknowledge that the NPT generative mechanisms are in dynamic interaction with local contexts and external drivers, the framework primarily addresses the mechanisms. Indeed, the theory tends to place undue emphasis on individual and collective agency without explicitly locating this within, and as shaped by, the organisational and relational context in which implementation occurs’ [26].

Segrott et al. [126] take this further. They point to what they perceive as a focus on the agency of those involved in implementation, as opposed to those who experience the effects of that agency.

‘ENPT places considerable emphasis on the notion of implementation as an expression of agency. However, the agents in question appear to be mainly conceptualised as professional practitioners (e.g. nurses), rather than the participants who receive interventions. There is scope to consider further how the key constructs of ENPT can be applied to understand how participant (and non-participant) agency may shape whether interventions become integrated and embedded within delivery systems’ [126]

Beyond this, Alharbi et al. [63] criticised NPT for presenting a normative model of implementation that paid insufficient attention to idealised temporal aspects of implementation, a point echoed by Alverbratt et al. [65]. Critique was more often about the interaction between theory and method. Some articles (9/108) observed that NPT constructs overlapped, that the technical vocabulary of the theory was difficult and that as a result coding qualitative data was difficult [39, 44, 48, 59, 64, 67, 82, 99, 207]. Problems of this nature seemed less evident when researchers used a more inductive approach to qualitative data analysis (e.g. [25, 26, 3841]) than they did when authors employed a framework approach (e.g. [39, 99]).

Discussion

Key results of the review

In this review, we identified 108 discrete studies of complex healthcare interventions and related implementation processes. These studies were reported in 130 journal articles published after 2008. In papers included in this review, researchers collected and analysed their data in ways that effectively provided a basis (i) for intervention design and implementation planning and (ii) for understanding the dynamics of implementation, embedding and integration. Three key results of the review are as follows:

  1. NPT appears to accurately depict important elements of implementation processes, and the constructs of the theory can be applied in a stable and consistent way within and between studies.

  2. NPT has provided conceptual tools for a large body of feasibility studies and process evaluations of complex healthcare interventions. It has successfully explained the outcome of such intervention studies.

  3. NPT can be applied flexibly and can be understood and mobilised by researchers and practitioners with diverse professional backgrounds, working across a variety of healthcare settings.

The use of NPT has coalesced around two main types of study: feasibility studies and process evaluations. However, unlike McEvoy et al.’s [17] review of NPT studies, we found that authors were justifying their choice of theory, and NPT was more frequently embedded in study protocols and thus being operationalised prospectively. However, concerns raised by McEvoy et al. about the lack of prospective application do not just apply to NPT. For example, Kirk et al. [18] point to the problem of low levels of prospective use of the CFIR [208] and PARIHS [209] frameworks. They point to the additional problem of lack of integration of theory into implementation research. Against this background, our review suggests that—although some authors have experienced difficulty with NPT’s technical vocabulary—users of NPT appear to be able to operationalise its concepts in consistent, stable ways to inform their work, and we can see evidence of theoretical integration in four kinds of studies.

  • i.

    Studies constructed with NPT in mind that reflect its characterisation of implementation processes in both intervention and evaluation design (e.g. Furler et al. [2931]).

  • ii.

    Studies that used NPT constructs as sensitising devices to form questions about implementation processes, and then related their conclusions back to the predictions of the theory (e.g. Grant et al. [34, 35]).

  • iii.

    Studies that collected and analysed data inductively in the light of NPT and then developed an analysis of the ways that different mechanisms work to motivate and shape implementation processes. The major papers by Clarke et al. [34, 35] and Hooker et al. [3841] are important examples of such work. So too are Kennedy et al.’s accounts of the WISE trial [4346].

  • iv.

    Studies that treated qualitative data deductively and used prescheduled coding matrices for framework or directed content analysis. Nordmark et al.’s work [116] offers an example of the way that this approach to theory driven analysis can be handled without ‘fitting’ or ‘shoehorning’ data in a rigid way (see MacFarlane and O’Reilly-de Brún [210] on techniques to manage this problem in qualitative research).

These different approaches to mobilising theory suggest that NPT’s users have developed flexible explanatory strategies, and we have pointed to some of these in Figs. 2, 3, 4 and 5. In earlier papers [59, 11, 12], we have argued that theories are conceptual toolkits that can be used flexibly to deal with practical problems. This means that there is no definitive ‘right way’ to employ NPT. It can be used on its own or in combination with other theories in ways that are locally defined to solve problems in intervention design and evaluation.

Limitations of this review

This review contributes to the literature on the incorporation of theory in implementation research, the benefits of this incorporation and the problems that can arise as a result. There are, of course, limitations to the review. Searches were undertaken in two databases, so it is possible that some studies were missed. It is questionable whether this would have altered the main findings and conclusions. Because Google Scholar is a search engine, and not a database, results of searches using it were not stable. Searches on Google Scholar also identified multiple versions of the same reference (e.g. versions of the published paper on publisher’s websites, records on institutional repositories, versions on personal websites and on academic social media sites such as Academia.edu and ResearchGate.Com). Sorting these involved significant additional work. It did however identify about 20 papers that would not otherwise have been included in the review. One paper was brought to our attention before appearing in any databases; however, subsequent updated searches did identify this paper. Equally, we excluded studies published in languages other than English, although there is some evidence that this is unlikely to be a major limitation [211].

We deliberately followed a two-step approach to data analysis, first identifying and characterising the use of NPT in implementation research, and then exploring the contribution made by NPT to understanding the dynamics of the processes of implementation and integration, and the limitations of its use. The characterisation is likely to be replicable by another team, but it is possible that a different group of researchers, with different backgrounds and different prior experience of NPT, would reach different conclusions. We have maximised the robustness of our findings by following a transparent process for analysis, including NPT-naïve researchers in the team, and holding frequent discussions amongst the team during the analysis. Finally, we made a deliberate decision to focus solely on the health care literature, and in light of this decision, our findings only apply to research on implementation in health care.

Next steps for NPT development and empirical research

Most papers in the review used the elaboration of NPT published by May et al. [7, 8] in 2009. More recent iterations of the theory have focused on (a) the important role that social structural and social cognitive features of context play in mobilisation for implementation [9] and (b) the ways in which implementation processes demand that their participants negotiate with other actors and elements in the context in which they are set [10]. In these papers, we have already gone some way to answering the critique of Clarke et al. [26] on the relationship between agency and context. The critique offered by Segrott et al. [126], however, focused on the experiences of different groups of actors in implementation processes. They saw NPT as primarily being about the agency of professionals, rather than the experiences of patients and other participants in implementation processes. NPT both can be, and is, applied to those groups. We have developed theory in this area to explore the relationship between the implementation of complex interventions and burden of treatment (e.g. [212215]), and there is now a discrete body of primary research literature (e.g. [216220]) and systematic reviews (e.g. [221223]) that utilises these theoretical perspectives to understand patient and caregiver experience.

NPT has developed iteratively. Future work to develop it will explore variations in the ways that NPT mechanisms motivate and shape implementation processes across and between settings, and between micro, meso and macro levels of activity. This will engender a comprehensive ‘whole system’ approach to understanding implementation processes. Future empirical research will also help us to explore and test the hypothesis that collective action mechanisms operate cumulatively and that some mechanisms are more significant than others in determining implementation process outcomes. Rigorous quantitative research will assist in this, but until recently, there has been no robust instrument through which quantitative investigations of NPT mechanisms could be done. However, the NoMAD instrument is now available to perform this task [224]. This will make possible both large-scale and comparative quantitative and mixed methods studies that will provide important insights into the role of NPT mechanisms and the form and direction of implementation processes. This should lead to rigorous statistical models of NPT mechanisms at work and so to new insights about implementation processes. Finally, despite attempts to make NPT more user friendly through the development of explanatory toolkits, some users have difficulty with its technical vocabulary. NPT training packages are now coming on stream that will help to solve this problem [225].

Conclusion

Normalization Process Theory appears to offer its users a coherent and stable set of explanations of implementation processes. It characterises the mechanisms that motivate and shape these processes and so can be used to aid intervention development and implementation planning as well as evaluating and understanding implementation processes themselves. In particular, NPT appears to have offered a valuable set of conceptual tools to understand the dynamics of implementation within clinical trials. In the future, it will be important to connect collective action much more closely to context in implementation studies. Equally, it will be important to develop longitudinal and genuinely mixed methods studies. These will help us understand not only the dynamics of implementation but also variations in implementation, embedding and long-term integration and sustainability over time and between settings.

Additional file

Additional file 1: (156.1KB, pdf)

Appendix data extraction tool. (PDF 156 kb)

Acknowledgements

We thank Professor Anne MacFarlane for her helpful comments on an earlier version of the manuscript.

Funding

CRM, MB, AC and MM were supported by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Wessex, which is a partnership between Wessex NHS organisations and partners and the University of Southampton. TLF, MG, FSM, EM, and TR were supported by the Economic and Social Research Council, Grant ES-062-23-3274. EM is also supported by the NIHR School of Primary Care Research. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health or the Economic and Social Research Council. Funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. CMM’s contribution was supported in part by Macmillan Cancer Support through core funding of the Macmillan Survivorship Research Group at the University of Southampton.

Abbreviations

ACP

American College of Physicians

APA

American Psychiatric Association

CAREDEM

Collaborative cARE for people with DEMentia in primary care (trial acronym)

COM-B

Capability + Opportunity + Motivation → Behaviour Change Model

COPD

Chronic obstructive pulmonary disease

Dedoose

Proprietary qualitative analysis software

DLDA

Daily life dialogue assessment

DN

District nurse

DPP

Discharge planning process

ENPT

Extended Normalization Process Theory

FSA

Food standards agency (England and Wales)

HCO

Homecare organiser

ICSWP

Inter-collegiate stroke working party (UK)

MOVE

Improving maternal and child health nurse care for vulnerable mothers (trial acronym)

NICE

National Institute of Health and Care Excellence

NoMAD

Normalisation of complex interventions—measure development

NPM

Normalization Process Model

NPT

Normalization Process Theory

PARIHS

Promoting action on research implementation in health services

PaTH

Patient trajectory for Home-dwelling elders

RN

Registered nurse

STEPPING-UP

Theory based change in practice systems and roles of health professionals in the primary care diabetes team (trial acronym)

WISE

Whole system informing self-management engagement (trial acronym)

Authors’ contributions

TLF, CRM, FSM, EM and TR designed the review. AC, MG and CRM performed searches. AC, TLF, MG, MM, CRM screened titles, abstracts, and full papers. MB, MG, TLF, CRM, MM and TR performed data extraction. TLF and CRM performed analysis of extracted data. CRM drafted the manuscript, with assistance from TLF, CMM, FSM and EM. All authors critically reviewed the manuscript for important intellectual content. All authors approved the final version of the paper.

Authors’ information

During the period in which this work was undertaken, CRM was a member of staff in the Faculty of Health Sciences, University of Southampton, UK. With Prof Alison Richardson, he jointly led the Patient Experience and Organisational Behaviour Research Programme of NIHR CLARHC Wessex.

Ethics approval and consent to participate

Not applicable.

Competing interests

CRM and TLF are the lead architects of NPT. FSM, EM and TR all played leading roles in the development and empirical application of NPT.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Footnotes

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0758-1) contains supplementary material, which is available to authorized users.

Contributor Information

Carl R. May, Email: carl.may@lshtm.ac.uk

Amanda Cummings, Email: a.cummings@soton.ac.uk.

Melissa Girling, Email: melissa.girling@ncl.ac.uk.

Mike Bracher, Email: m.bracher@portsmouth.ac.uk.

Frances S. Mair, Email: frances.mair@gla.ac.uk

Christine M. May, Email: christinemay9@btinternet.com

Elizabeth Murray, Email: elizabeth.murray@ucl.ac.uk.

Michelle Myall, Email: m.myall@soton.ac.uk.

Tim Rapley, Email: tim.rapley@northumbria.ac.uk.

Tracy Finch, Email: tracy.finch@northumbria.ac.uk.

References

  • 1.Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015; 10.1136/bmjqs-2014-003627. [DOI] [PMC free article] [PubMed]
  • 2.Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–350. doi: 10.1016/j.amepre.2012.05.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53. doi: 10.1186/s13012-015-0242-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Skolarus TA, Lehmann T, Tabak RG, Harris J, Lecy J, Sales AE. Assessing citation networks for dissemination and implementation research frameworks. Implement Sci. 2017;12(1):97. doi: 10.1186/s13012-017-0628-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.May C. A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res. 2006;6(86):1–11. doi: 10.1186/1472-6963-6-86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M, Gask L, MacFarlane A, Murray E, Rapley T, et al. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res. 2007;7:148. doi: 10.1186/1472-6963-7-148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.May C, Finch T. Implementation, embedding, and integration: an outline of normalization process theory. Sociology. 2009;43(3):535–554. doi: 10.1177/0038038509103208. [DOI] [Google Scholar]
  • 8.May C, Mair FS, Finch T, MacFarlane A, Dowrick C, Treweek S, Rapley T, Ballini L, Ong BN, Rogers A, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009:4(29). [DOI] [PMC free article] [PubMed]
  • 9.May C. Towards a general theory of implementation. Implement Sci. 2013;8(1):18. doi: 10.1186/1748-5908-8-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11(1):141. doi: 10.1186/s13012-016-0506-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.May CR, Mair FS, Dowrick CF, Finch TL. Process evaluation for complex interventions in primary care: understanding trials using the normalization process model. BMC Fam Pract. 2007;8 [DOI] [PMC free article] [PubMed]
  • 12.May C. Agency and implementation: understanding the embedding of healthcare innovations in practice. Soc Sci Med. 2013;78(0):26–33. doi: 10.1016/j.socscimed.2012.11.021. [DOI] [PubMed] [Google Scholar]
  • 13.Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, Finch T, Kennedy A, Mair F, O'Donnell C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8(1):63. doi: 10.1186/1741-7015-8-63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Finch TL, Rapley T, Girling M, Mair FS, Murray E, Treweek S, McColl E, Steen IN, May CR. Improving the normalization of complex interventions: measure development based on normalization process theory (NoMAD): study protocol. Implement Sci. 2013;8 [DOI] [PMC free article] [PubMed]
  • 15.Finch T, Mair F, O'Donnell C, Murray E, May C. From theory to ‘measurement’ in complex interventions: methodological lessons from the development of an e-health normalisation instrument. BMC Med Res Methodol. 2012;12(1):69. doi: 10.1186/1471-2288-12-69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.May C, Finch T, Ballini L, MacFarlane A, Mair F, Murray E, Treweek S, Rapley T. Evaluating complex interventions and health technologies using normalization process theory: development of a simplified approach and web-enabled toolkit. BMC Health Serv Res. 2011;11(1):245. doi: 10.1186/1472-6963-11-245. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.McEvoy R, Ballini L, Maltoni S, ODonnell C, Mair F, MacFarlane A. A qualitative systematic review of studies using the normalization process theory to research implementation processes. Implement Sci. 2014;9(1):2. doi: 10.1186/1748-5908-9-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2016;11:72. doi: 10.1186/s13012-016-0437-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Ballinger C, Taylor A, Loudon D, Macdonald AS. Rehabilitation professionals’ perceptions of the use of new visualisation software tools with people with stroke. Disability Rehab Assist Technol. 2016;11(2):139–149. doi: 10.3109/17483107.2015.1111941. [DOI] [PubMed] [Google Scholar]
  • 20.Bamford C, Poole M, Brittain K, Chew-Graham C, Fox C, Iliffe S, Manthorpe J, Robinson L, Team C Understanding the challenges to implementing case management for people with dementia in primary care in England: a qualitative study using normalization process theory. BMC Health Serv Res. 2014;14(1):549. doi: 10.1186/s12913-014-0549-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Blakeman T, Protheroe J, Chew-Graham C, Rogers A, Kennedy A. Understanding the management of early-stage chronic kidney disease in primary care: a qualitative study. Br J Gen Pract. 2012;62(597):e233–e242. doi: 10.3399/bjgp12X636056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Blickem C, Kennedy A, Jariwala P, Morris R, Bowen R, Vassilev I, Brooks H, Blakeman T, Rogers A. Aligning everyday life priorities with people's self-management support networks: an exploration of the work and implementation of a needs-led telephone support system. BMC Health Serv Res. 2014;14(1):262. doi: 10.1186/1472-6963-14-262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Brooks H, Sanders C, Lovell K, Fraser C, Rogers A. Re-inventing care planning in mental health: stakeholder accounts of the imagined implementation of a user/carer involved intervention. BMC Health Serv Res. 2015;15(1):490. doi: 10.1186/s12913-015-1154-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Buckingham S, Kendall M, Ferguson S, MacNee W, Sheikh A, White P, Worth A, Boyd K, Murray SA, Pinnock H. HELPing older people with very severe chronic obstructive pulmonary disease (HELP-COPD): mixed-method feasibility pilot randomised controlled trial of a novel intervention. Prim Care Resp Med. 2015;25:15020. doi: 10.1038/npjpcrm.2015.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Clarke DJ, Hawkins R, Sadler E, Harding G, McKevitt C, Godfrey M, Dickerson J, Farrin AJ, Kalra L, Smithard D, et al. Introducing structured caregiver training in stroke care: findings from the TRACS process evaluation study. BMJ Open. 2014;4(4):e004473. doi: 10.1136/bmjopen-2013-004473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Clarke DJ, Godfrey M, Hawkins R, Sadler E, Harding G, Forster A, McKevitt C, Dickerson J, Farrin A. Implementing a training intervention to support caregivers after stroke: a process evaluation examining the initiation and embedding of programme change. Implement Sci. 2013;8(1):96. doi: 10.1186/1748-5908-8-96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Coupe N, Anderson E, Gask L, Sykes P, Richards DA, Chew-Graham C. Facilitating professional liaison in collaborative care for depression in UK primary care; a qualitative study utilising normalisation process theory. BMC Fam Pract. 2014;15(1):78. doi: 10.1186/1471-2296-15-78. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Finch TL, Bamford C, Deary V, Sabin N, Parry SW. Making sense of a cognitive behavioural therapy intervention for fear of falling: qualitative study of intervention development. BMC Health Serv Res. 2014;14(1):436. doi: 10.1186/1472-6963-14-436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Furler J, O'Neal D, Speight J, Manski-Nankervis JA, Gorelik A, Holmes-Truscott E, Ginnivan L, Young D, Best J, Patterson E, et al. Supporting insulin initiation in type 2 diabetes in primary care: results of the stepping up pragmatic cluster randomised controlled clinical trial. BMJ. 2017;356:j783. doi: 10.1136/bmj.j783. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Furler J, Blackberry I, Manski-Nankervis J-A, O’Neal D, Best J, Young D. Optimizing care and outcomes for people with type 2 diabetes—lessons from a translational research program on insulin initiation in general practice. Front Med. 2015;1:60. doi: 10.3389/fmed.2014.00060. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Furler JS, Blackberry ID, Walker C, Manski-Nankervis JA, Anderson J, O'Neal D, Young D, Best J. Stepping up: a nurse-led model of care for insulin initiation for people with type 2 diabetes. Fam Pract. 2014;31(3):349–356. doi: 10.1093/fampra/cmt085. [DOI] [PubMed] [Google Scholar]
  • 32.Gabbay MB, Ring A, Byng R, Anderson P, Taylor RS, Matthews C, Harris T, Berry V, Byrne P, Carter E, et al. Debt counselling for depression in primary care: an adaptive randomised controlled pilot trial (DeCoDer study) Health Technol Assess. 2017;21(35):1. doi: 10.3310/hta21350. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Gask L, Bower P, Lovell K, Escott D, Archer J, Gilbody S, Lankshear A, Simpson A, Richards D. What work has to be done to implement collaborative care for depression? Process evaluation of a trial utilizing the Normalization Process Model. Implement Sci. 2010;5(15):5908–5905. doi: 10.1186/1748-5908-5-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Grant A, Dreischulte T, Guthrie B. Process evaluation of the data-driven quality improvement in primary care (DQIP) trial: active and less active ingredients of a multi-component complex intervention to reduce high-risk primary care prescribing. Implement Sci. 2017;12(1):4. doi: 10.1186/s13012-016-0531-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Grant A, Dreischulte T, Guthrie B. Process evaluation of the Data-Driven Quality Improvement in Primary Care (DQIP) trial: case study evaluation of adoption and maintenance of a complex intervention to reduce high-risk primary care prescribing. BMJ Open. 2017;7(3):e015281. doi: 10.1136/bmjopen-2016-015281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Godfrey M, Smith J, Green J, Cheater F, Inouye SK, Young JB. Developing and implementing an integrated delirium prevention system of care: a theory driven, participatory research study. BMC Health Serv Res. 2013;13(1):341. doi: 10.1186/1472-6963-13-341. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Hind D, Parkin J, Whitworth V, Rex S, Young T, Hampson L, Sheehan J, Maguire C, Cantrill H, Scott E, et al. Aquatic therapy for children with Duchenne muscular dystrophy: a pilot feasibility randomised controlled trial and mixed-methods process evaluation. Health Technol Assess. 2017;21(27):1. doi: 10.3310/hta21270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Hooker L, Taft A. Using theory to design, implement and evaluate sustained nurse domestic violence screening and supportive care. J Res Nurs. 2016;21(5–6):432–442. doi: 10.1177/1744987116649633. [DOI] [Google Scholar]
  • 39.Hooker L, Small R, Humphreys C, Hegarty K, Taft A. Applying normalization process theory to understand implementation of a family violence screening and care model in maternal and child health nursing practice: a mixed method process evaluation of a randomised controlled trial. Implement Sci. 2015;10(1):39. doi: 10.1186/s13012-015-0230-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Hooker L, Small R, Taft A. Understanding sustained domestic violence identification in maternal and child health nurse care: process evaluation from a 2-year follow-up of the MOVE trial. J Adv Nurs. 2016;72(3):533-44. [DOI] [PubMed]
  • 41.Taft AJ, Hooker L, Humphreys C, Hegarty K, Walter R, Adams C, Agius P, Small R. Maternal and child health nurse screening and care for mothers experiencing domestic violence (MOVE): a cluster randomised trial. BMC Med. 2015;13(1):150. doi: 10.1186/s12916-015-0375-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Hooker L, Small R, Taft A. Understanding sustained domestic violence identification in maternal and child health nurse care: process evaluation from a 2-year follow-up of the MOVE trial. J Adv Nurs. 2016;72(3):533–544. doi: 10.1111/jan.12851. [DOI] [PubMed] [Google Scholar]
  • 43.Kennedy A, Rogers A, Bowen R, Lee V, Blakeman T, Gardner C, Morris R, Protheroe J, Chew-Graham C. Implementing, embedding and integrating self-management support tools for people with long-term conditions in primary care nursing: a qualitative study. Int J Nurs Stud. 2014;51(8):1103–1113. doi: 10.1016/j.ijnurstu.2013.11.008. [DOI] [PubMed] [Google Scholar]
  • 44.Kennedy A, Rogers A, Chew-Graham C, Blakeman T, Bowen R, Gardner C, Lee V, Morris R, Protheroe J. Implementation of a self-management support approach (WISE) across a health system: a process evaluation explaining what did and did not work for organisations, clinicians and patients. Implement Sci. 2014;9(1):129. doi: 10.1186/s13012-014-0129-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Kennedy A, Bower P, Reeves D, Blakeman T, Bowen R, Chew-Graham C, Eden M, Fullwood C, Gaffney H, Gardner C, et al. Implementation of self management support for long term conditions in routine primary care settings: cluster randomised controlled trial. BMJ. 2013;346:f2882. doi: 10.1136/bmj.f2882. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Kennedy A, Chew-Graham C, Blakeman T, Bowen A, Gardner C, Protheroe J, Rogers A, Gask L. Delivering the WISE (Whole Systems Informing Self-Management Engagement) training package in primary care: learning from formative evaluation. Implement Sci. 2010;5(1):7. doi: 10.1186/1748-5908-5-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Khowaja AR, Qureshi RN, Sawchuck D, Oladapo OT, Adetoro OO, Orenuga EA, Bellad M, Mallapur A, Charantimath U, Sevene E, et al. The feasibility of community level interventions for pre-eclampsia in South Asia and Sub-Saharan Africa: a mixed-methods design. Reprod Health. 2016;13(Suppl 1):56. doi: 10.1186/s12978-016-0133-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Leon N, Lewin S, Mathews C. Implementing a provider-initiated testing and counselling (PITC) intervention in Cape town, South Africa: a process evaluation using the normalisation process model. Implement Sci. 2013;8(1):97. doi: 10.1186/1748-5908-8-97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Mair FS, Hiscock J, Beaton SC. Understanding factors that inhibit or promote the utilization of telecare in chronic lung disease. Chronic Illness. 2008;4(2):110–117. doi: 10.1177/1742395308092482. [DOI] [PubMed] [Google Scholar]
  • 50.Morden A, Ong BN, Brooks L, Jinks C, Porcheret M, Edwards JJ, Dziedzic KS. Introducing evidence through research “push” using theory and qualitative methods. Qual Health Res. 2015;25:1560–1575. doi: 10.1177/1049732315570120. [DOI] [PubMed] [Google Scholar]
  • 51.Morden A, Brooks L, Jinks C, Porcheret M, Ong BN, Dziedzic K. Research “push”, long term-change, and general practice. J Health Org Manage. 2015;29(7):798–821. doi: 10.1108/JHOM-07-2014-0119. [DOI] [PubMed] [Google Scholar]
  • 52.Ong BN, Morden A, Brooks L, Porcheret M, Edwards JJ, Sanders T, Jinks C, Dziedzic K. Changing policy and practice: making sense of national guidelines for osteoarthritis. Soc Sci Med. 2014;106:101–109. doi: 10.1016/j.socscimed.2014.01.036. [DOI] [PubMed] [Google Scholar]
  • 53.Ricketts EJ, Francischetto EOC, Wallace LM, Hogan A, McNulty CAM. Tools to overcome potential barriers to chlamydia screening in general practice: qualitative evaluation of the implementation of a complex intervention. BMC Fam Pract. 2016;17 [DOI] [PMC free article] [PubMed]
  • 54.Speed C, Heaven B, Adamson A, Bond J, Corbett S, Lake A, May C, Vanoli A, McMeekin P, Moynihan P. LIFELAX–diet and LIFEstyle versus LAXatives in the management of chronic constipation in older people: randomised controlled trial. Health Technol Assess. 2010;14(52):1–251. doi: 10.3310/hta14520. [DOI] [PubMed] [Google Scholar]
  • 55.Sturgiss EA, Elmitt N, Haesler E, van Weel C, Douglas K. Feasibility and acceptability of a physician-delivered weight management programme. Fam Pract. 2017;34(1):43–48. doi: 10.1093/fampra/cmw105. [DOI] [PubMed] [Google Scholar]
  • 56.French B, Thomas LH, Harrison J, Burton CR, Forshaw D, Booth J, Britt D, Cheater FM, Roe B, Watkins CL. Implementing a systematic voiding program for patients with urinary incontinence after stroke. Qual Health Res. 2016;26(10):1393–1408. doi: 10.1177/1049732316630975. [DOI] [PubMed] [Google Scholar]
  • 57.Thomas LH, French B, Burton CR, Sutton C, Forshaw D, Dickinson H, Leathley MJ, Britt D, Roe B, Cheater FM, et al. Evaluating a systematic voiding programme for patients with urinary incontinence after stroke in secondary care using soft systems analysis and Normalisation Process Theory: findings from the ICONS case study phase. Int J Nurs Stud. 2014;51(10):1308–1320. doi: 10.1016/j.ijnurstu.2014.02.009. [DOI] [PubMed] [Google Scholar]
  • 58.Willis K, Small R, Brown S. Using documents to investigate links between implementation and sustainability in a complex community intervention: the PRISM study. Soc Sci Med. 2012;75(7):1222–1229. doi: 10.1016/j.socscimed.2012.05.025. [DOI] [PubMed] [Google Scholar]
  • 59.Aarts JW, Faber MJ, Cohlen BJ, Van Oers A, Nelen WL, Kremer JA. Lessons learned from the implementation of an online infertility community into an IVF clinic’s daily practice. Hum Fertil. 2015;(0):1–10. [DOI] [PubMed]
  • 60.Devlin AM, McGee-Lennon M, O’Donnell CA, Bouamrane M-M, Agbakoba R, O’Connor S, Grieve E, Finch T, Wyke S, Watson N, et al. Delivering digital health and well-being at scale: lessons learned during the implementation of the Dallas program in the United Kingdom. J Am Med Inform Assoc. 2015;23:ocv097. doi: 10.1093/jamia/ocv097. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Agbakoba R, McGee-Lennon M, Bouamrane M-M, Watson N, Mair FS. Implementation factors affecting the large-scale deployment of digital health and well-being technologies: a qualitative study of the initial phases of the ‘Living-It-Up’ programme. Health Informat J. 2016;22(4):867-77. [DOI] [PMC free article] [PubMed]
  • 62.Lennon MR, Bouamrane MM, Devlin AM, O'Connor S, O'Donnell C, Chetty U, Agbakoba R, Bikker A, Grieve E, Finch T, et al. Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. J Med Internet Res. 2017;16;19(2):e42. [DOI] [PMC free article] [PubMed]
  • 63.Alharbi TS, Carlström E, Ekman I, Olsson L-E. Implementation of person-centred care: management perspective. J Hosp Admin. 2014;3(3):107. [Google Scholar]
  • 64.Ahmed S, Hayward J, Ahmed M. Primary care professionals’ perceptions of using a short family history questionnaire. Fam Pract. 2016;33:cmw080. doi: 10.1093/fampra/cmw080. [DOI] [PubMed] [Google Scholar]
  • 65.Alverbratt C, Carlström E, Åström S, Kauffeldt A, Berlin J. The process of implementing a new working method-a project towards change in a Swedish psychiatric clinic. J Hosp Admin. 2014;3(6):174. [Google Scholar]
  • 66.Ariens LFM, Schussler-Raymakers FML, Frima C, Flinterman A, Hamminga E, Arents BWM, Bruijnzeel-Koomen CAFM, Marjolein SDBW, Os-Medendorp HV. Barriers and facilitators to eHealth use in daily practice: perspectives of patients and professionals in dermatology. J Med Internet Res. 2017;19(9):e300. [DOI] [PMC free article] [PubMed]
  • 67.Atkins S, Lewin S, Ringsberg KC, Thorson A. Provider experiences of the implementation of a new tuberculosis treatment programme: a qualitative study using the normalisation process model. BMC Health Serv Res. 2011;11 [DOI] [PMC free article] [PubMed]
  • 68.Bamford C, Heaven B, May C, Moynihan P. Implementing nutrition guidelines for older people in residential care homes: a qualitative study using normalization process theory. Implement Sci. 2012;7(1):106. doi: 10.1186/1748-5908-7-106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Basu AP, Pearse JE, Baggaley J, Watson RM, Rapley T. Participatory design in the development of an early therapy intervention for perinatal stroke. BMC Pediatr. 2017;17:33. [DOI] [PMC free article] [PubMed]
  • 70.Bayliss K, Riste L, Band R, Peters S, Wearden A, Lovell K, Fisher L, Chew-Graham CA. Implementing resources to support the diagnosis and management of chronic fatigue syndrome/myalgic encephalomyelitis (CFS/ME) in primary care: a qualitative study. BMC Fam Pract. 2016;17:66. doi: 10.1186/s12875-016-0453-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Bee P, Lovell K, Airnes Z, Pruszynska A. Embedding telephone therapy in statutory mental health services: a qualitative, theory-driven analysis. BMC Psychiatry. 2016;16(1):56. doi: 10.1186/s12888-016-0761-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Bocoum FY, Tarnagda G, Bationo F, Savadogo JR, Nacro S, Kouanda S, Zarowsky C. Introducing onsite antenatal syphilis screening in Burkina Faso: implementation and evaluation of a feasibility intervention tailored to a local context. BMC Health Serv Res. 2017;17:378. doi: 10.1186/s12913-017-2325-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Bouamrane M-M, Mair FS. A study of clinical and information management processes in the surgical pre-assessment clinic—the experience of the Dumfries & Galloway royal infirmary preoperative clinic. BMC Med Inf Dec Making. 2014;14(1):22. doi: 10.1186/1472-6947-14-22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Bouamrane MM, Mair FS. A qualitative evaluation of general practitioners’ views on protocol-driven eReferral in Scotland. BMC Med Inform Decis Mak. 2014;14(1):30. doi: 10.1186/1472-6947-14-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75.Bouamrane MM, Mair FS. Implementation of an integrated preoperative care pathway and regional electronic clinical portal for preoperative assessment. BMC Med Inform Decis Mak. 2014;14(1):93. doi: 10.1186/1472-6947-14-93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Bridges J, May C, Fuller A, Griffiths P, Wigley W, Gould L, Barker H, Libberton P. Optimising impact and sustainability: a qualitative process evaluation of a complex intervention targeted at compassionate care. BMJ Qual Saf. 2017;26:970-7. [DOI] [PMC free article] [PubMed]
  • 77.Chiang PP, Glance D, Walker J, Walter F, Emery J. Implementing a QCancer risk tool into general practice consultations: an exploratory study using simulated consultations with Australian general practitioners. Br J Cancer. 2015;112:S77–S83. doi: 10.1038/bjc.2015.46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Conn LG, McKenzie M, Pearsall EA, McLeod RS. Successful implementation of an enhanced recovery after surgery programme for elective colorectal surgery: a process evaluation of champions’ experiences. Implement Sci. 2015;10(1):99. doi: 10.1186/s13012-015-0289-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Desveaux L, Mitchell J, Shaw J, Ivers N. Understanding the impact of accreditation on quality in healthcare: a grounded theory approach. Int J Qual Health Care. 2017;29:1–7. doi: 10.1093/intqhc/mzx136. [DOI] [PubMed] [Google Scholar]
  • 80.Dickinson C, Gibson G, Gotts Z, Stobbart L, Robinson L. Cognitive stimulation therapy in dementia care: exploring the views and experiences of service providers on the barriers and facilitators to implementation in practice using Normalization Process Theory. Int Psychogeriatr. 2017;29(11):1869–1878. doi: 10.1017/S1041610217001272. [DOI] [PubMed] [Google Scholar]
  • 81.Dikomitis L, Green T, Macleod U. Embedding electronic decision-support tools for suspected cancer in primary care: a qualitative study of GPs’ experiences. Prim Health Care Res Dev. 2015;16(06):548–555. doi: 10.1017/S1463423615000109. [DOI] [PubMed] [Google Scholar]
  • 82.Drew S, Judge A, May C, Farmer A, Cooper C, Javaid MK, Gooberman-Hill R. Implementation of secondary fracture prevention services after hip fracture: a qualitative study using extended Normalization Process Theory. Implement Sci. 2015;10(1):57. doi: 10.1186/s13012-015-0243-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Dugdale S, Elison S, Davies G, Ward J, Dalton M. A qualitative study investigating the continued adoption of breaking free online across a national substance misuse organisation: theoretical conceptualisation of staff perceptions. J Behav Health Serv Res. 2017;44(1):89–101. doi: 10.1007/s11414-016-9512-0. [DOI] [PubMed] [Google Scholar]
  • 84.Ehrlich C, Kendall E, John WS. How does care coordination provided by registered nurses “fit” within the organisational processes and professional relationships in the general practice context? Collegian. 2013;20(3):127–135. doi: 10.1016/j.colegn.2012.04.006. [DOI] [PubMed] [Google Scholar]
  • 85.Finch T. Teledermatology for chronic disease management: coherence and normalization. Chronic illness. 2008;4(2):127–134. doi: 10.1177/1742395308092483. [DOI] [PubMed] [Google Scholar]
  • 86.Franx G, Oud M, de Lange J, Wensing M, Grol R. Implementing a stepped-care approach in primary care: results of a qualitative study. Implement Sci. 2012;7(1):8. doi: 10.1186/1748-5908-7-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.French B, Day E, Watkins C, McLoughlin A, Fitzgerald J, Leathley M, Davies P, Emsley H, Ford G, Jenkinson D, et al. The challenges of implementing a telestroke network: a systematic review and case study. BMC Med Inf Dec Making. 2013;13(1):125. doi: 10.1186/1472-6947-13-125. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Gibson J, Lightbody E, McLoughlin A, McAdam J, Gibson A, Day E, Fitzgerald J, May C, Price C, Emsley H, et al. ‘It was like he was in the room with us’: patients’ and carers’ perspectives of telemedicine in acute stroke. Health Expect. 2016;19(1):98–111. doi: 10.1111/hex.12333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89.Foss C, Ruud Knutsen IA, Havrevold Henni S, Myrstad H. The challenges associated with implementing new practice. Norwegian J Clin Nurs Sykepleien Forskning. 2017;e59483.
  • 90.Foster M, Burridge L, Donald M, Zhang J, Jackson C. The work of local healthcare innovation: a qualitative study of GP-led integrated diabetes care in primary health care organization, structure and delivery of healthcare. BMC Health Serv Res. 2016;16(11). [DOI] [PMC free article] [PubMed]
  • 91.Gould DJ, Hale R, Waters E, Allen D. Promoting health workers’ ownership of infection prevention and control: using normalization process theory as an interpretive framework. J Hosp Infect. 2016;94(4):373–380. doi: 10.1016/j.jhin.2016.09.015. [DOI] [PubMed] [Google Scholar]
  • 92.Gunn JM, Palmer VJ, Dowrick CF, Herrman HE, Griffiths FE, Kokanovic R, Blashki GA, Hegarty KL, Johnson CL, Potiriadis M, et al. Embedding effective depression care: using theory for primary care organisational and systems change. Implement Sci. 2010;5(1):62. doi: 10.1186/1748-5908-5-62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 93.Hall A, Wilson CB, Stanmore E, Todd C. Implementing monitoring technologies in care homes for people with dementia: a qualitative exploration using normalization process theory. Int J Nurs Stud. 2017;72:60–70. doi: 10.1016/j.ijnurstu.2017.04.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 94.Hall A, Finch T, Kolehmainen N, James D. Implementing a video-based intervention to empower staff members in an autism care organization: a qualitative study. BMC Health Serv Res. 2016;16:608. doi: 10.1186/s12913-016-1820-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 95.Hazell CM, Strauss C, Hayward M, Cavanagh K. Understanding clinician attitudes towards implementation of guided self-help cognitive behaviour therapy for those who hear distressing voices: using factor analysis to test normalisation process theory. BMC Health Serv Res. 2017;17(1):507. doi: 10.1186/s12913-017-2449-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 96.Henderson EJ, Rubin GP. The utility of an online diagnostic decision support system (Isabel) in general practice: a process evaluation. JRSM Short Rep. 2013;4(5):31. doi: 10.1177/2042533313476691. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 97.Herbert G, Sutton E, Burden S, Lewis S, Thomas S, Ness A, Atkinson C. Healthcare professionals' views of the enhanced recovery after surgery programme: a qualitative investigation. BMC Health Serv Res. 2017;17(1):617. doi: 10.1186/s12913-017-2547-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 98.Hoberg AA, Vickers KS, Ericksen J, Bauer G, Kung S, Stone R, Williams M, Moore MJ, Frye MA. Feasibility evaluation of an interpersonal and social rhythm therapy group delivery model. Arch Psychiatr Nurs. 2013;27(6):271–277. doi: 10.1016/j.apnu.2013.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 99.Holtrop JS, Potworowski G, Fitzpatrick L, Kowalk A, Green LA. Effect of care management program structure on implementation: a normalization process theory analysis. BMC Health Serv Res. 2016;16(1):386. doi: 10.1186/s12913-016-1613-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 100.Kanagasundaram NS, Bevan MT, Sims AJ, Heed A, Price DA, Sheerin NS. Computerized clinical decision support for the early recognition and management of acute kidney injury: a qualitative evaluation of end-user experience. Clin Kidney J. 2016;9(1):57–62. doi: 10.1093/ckj/sfv130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 101.Kulnik ST, Postges H, Brimicombe L, Hammond J, Jones F. Implementing an interprofessional model of self-management support across a community workforce: a mixed-methods evaluation study. J Int Care. 2017;31(1):75–84. doi: 10.1080/13561820.2016.1246432. [DOI] [PubMed] [Google Scholar]
  • 102.Johnson M, Leaf AA, Pearson F, Clark H, Dimitrov B, Pope C, May CR. Successfully implementing and embedding guidelines to improve the nutrition and growth of preterm infants in neonatal intensive care: a prospective interventional study. BMJ Open. 2017;7:e017727. doi: 10.1136/bmjopen-2017-017727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 103.Jones CH, Glogowska M, Locock L, Lasserson DS. Embedding new technologies in practice—a normalization process theory study of point of care testing. BMC Health Serv Res. 2016;16(1):591. doi: 10.1186/s12913-016-1834-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 104.Jones F, Bailey N. How can we train stroke practitioners about patient self-management? Description and evaluation of a pathway wide training programme. Eur J Per Centered Healthcare. 2013;1(1):246–254. doi: 10.5750/ejpch.v1i1.660. [DOI] [Google Scholar]
  • 105.Leggat SG, Balding C. A qualitative study on the implementation of quality systems in Australian hospitals. Health Serv Manag Res. 2017;30(3):179–186. doi: 10.1177/0951484817715594. [DOI] [PubMed] [Google Scholar]
  • 106.Lhussier M, Forster N, Eaton S, Carr SM. Care planning for long term conditions in primary care: indicators of embeddedness. Eur J Per Centered Healthcare. 2015;3(1):59–64. doi: 10.5750/ejpch.v3i1.869. [DOI] [Google Scholar]
  • 107.Ling T, Brereton L, Conklin A, Newbould J, Roland M. Barriers and facilitators to integrating care: experiences from the English integrated care pilots. Int J Integr Care. 2012;12 [DOI] [PMC free article] [PubMed]
  • 108.Lloyd A, Joseph-Williams N, Edwards A, Rix A, Elwyn G. Patchy ‘coherence’: using normalization process theory to evaluate a multi-faceted shared decision making implementation program (MAGIC) Implement Sci. 2013;8(1):102. doi: 10.1186/1748-5908-8-102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 109.Joseph-Williams N, Lloyd A, Edwards A, Stobbart L, Tomson D, Macphail S, Dodd C, Brain K, Elwyn G, Thomson R. Implementing shared decision making in the NHS: lessons from the MAGIC programme. Bmj. 2017;357:j1744. doi: 10.1136/bmj.j1744. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 110.Lowrie R, Johansson L, Forsyth P, Bryce SL, McKellar S, Fitzgerald N. Experiences of a community pharmacy service to support adherence and self-management in chronic heart failure. Int J Clin Pharm. 2014;36(1):154–162. doi: 10.1007/s11096-013-9889-2. [DOI] [PubMed] [Google Scholar]
  • 111.Martindale A-M, Elvey R, Howard SJ, McCorkindale S, Sinha S, Blakeman T. Understanding the implementation of ‘sick day guidance’ to prevent acute kidney injury across a primary care setting in England: a qualitative evaluation. BMJ Open. 2017;7(11):e017241. doi: 10.1136/bmjopen-2017-017241. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 112.May CR, Finch TL, Cornford J, Exley C, Gately C, Kirk S, Jenkings KN, Osbourne J, Robinson AL, Rogers A. Integrating telecare for chronic disease management in the community: what needs to be done? BMC Health Serv Res. 2011;11(1):131. doi: 10.1186/1472-6963-11-131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 113.Morton A, Wigley W. Beyond ‘train and hope’: identifying factors that affect implementation of the promotional guide in practice. J Health Visiting. 2014;2(12):670–680. doi: 10.12968/johv.2014.2.12.670. [DOI] [Google Scholar]
  • 114.Murray E, Burns J, May C, Finch T, O'Donnell C, Wallace P, Mair F. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci. 2011;6(1):6. doi: 10.1186/1748-5908-6-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 115.Newton MS, McLachlan HL, Forster DA, Willis KF. Understanding the ‘work’ of caseload midwives: a mixed-methods exploration of two caseload midwifery models in Victoria. Australia: Women and Birth. 2016;29(3):223. [DOI] [PubMed]
  • 116.Nordmark S, Zingmark K, Lindberg I. Process evaluation of discharge planning implementation in healthcare using normalization process theory. BMC Med Inf Dec Making. 2016;16:48. doi: 10.1186/s12911-016-0285-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 117.O’Donnell A, Kaner E. Are brief alcohol interventions adequately embedded in UK primary care? A qualitative study utilising normalisation process theory. Int J Environ Res Public Health. 2017;14(4):350. doi: 10.3390/ijerph14040350. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 118.Owens C, Charles N. Implementation of a text-messaging intervention for adolescents who self-harm (TeenTEXT): a feasibility study using normalisation process theory. Child Adolescent Psychiat Mental Health. 2016;10(1):14. doi: 10.1186/s13034-016-0101-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 119.Polus BI, Paterson C, Van Rotterdam J, Vindigni D. Embedding chiropractic in indigenous health care organisations: applying the normalisation process model. BMC Health Serv Res. 2012;12(1):429. doi: 10.1186/1472-6963-12-429. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 120.Turnbull J, Prichard J, Halford S, Pope C, Salisbury C. Reconfiguring the emergency and urgent care workforce: mixed methods study of skills and the everyday work of non-clinical call-handlers in the NHS. J Health Serv Res Policy. 2012;17(4):233–240. doi: 10.1258/jhsrp.2012.011141. [DOI] [PubMed] [Google Scholar]
  • 121.Pope C, Halford S, Turnbull J, Prichard J, Calestani M, May C.
  • 122.Røsstad T, Garåsen H, Steinsbekk A, Håland E, Kristoffersen L, Grimsmo A. Implementing a care pathway for elderly patients, a comparative qualitative process evaluation in primary care. BMC Health Serv Res. 2015;15(1):86. doi: 10.1186/s12913-015-0751-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 123.Sanders T, Foster NE, Ong BN. Perceptions of general practitioners towards the use of a new system for treating back pain: a qualitative interview study. BMC Med. 2011;9:49. doi: 10.1186/1741-7015-9-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 124.Scalia P, Elwyn G, Durand MA. “Provoking conversations”: case studies of organizations where Option Grid™ decision AIDS have become 'normalized'. BMC Med Inf Dec Making. 2017;17(1):124. doi: 10.1186/s12911-017-0517-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 125.Scantlebury A, Sheard L, Watt I, Cairns P, Wright J, Adamson J. Exploring the implementation of an electronic record into a maternity unit: a qualitative study using normalisation process theory. BMC Med Inf Dec Making. 2017;17(1):1–10. doi: 10.1186/s12911-016-0389-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 126.Segrott J, Murphy S, Rothwell H, Scourfield J, Foxcroft D, Gillespie D, Holliday J, Hood K, Hurlow C, Morgan-Trimmer S, et al. An application of extended normalisation process theory in a randomised controlled trial of a complex social intervention: process evaluation of the strengthening families programme (10–14) in Wales, UK. SSM Pop Health. 2017;3:255–265. doi: 10.1016/j.ssmph.2017.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 127.Shemeili SA, Klein S, Strath A, Fares S, Stewart D. An exploration of health professionals’ experiences of medicines management in elderly, hospitalised patients in Abu Dhabi. Int J Clin Pharm. 2016;38(1):107–118. doi: 10.1007/s11096-015-0212-2. [DOI] [PubMed] [Google Scholar]
  • 128.Shulver W, Killington M, Crotty M. ‘Massive potential’ or ‘safety risk’? Health worker views on telehealth in the care of older people and implications for successful normalization. BMC Med Inf Dec Making. 2016;16(1):131. doi: 10.1186/s12911-016-0373-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 129.Spangaro J, Poulos RG, Zwi AB. Pandora doesn’t live here anymore: normalization of screening for intimate partner violence in Australian antenatal, mental health, and substance abuse services. Violence Victims. 2011;26(1):130–144. doi: 10.1891/0886-6708.26.1.130. [DOI] [PubMed] [Google Scholar]
  • 130.Stevenson F. The use of electronic patient records for medical research: conflicts and contradictions. BMC Health Serv Res. 2015;15(1):124. doi: 10.1186/s12913-015-0783-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 131.Tarzia L, Fetherstonhaugh D, Bauer M, Beattie E, Nay R. “We have to work within the system!”: staff perceptions of organizational barriers to decision making for older adults with dementia in Australian aged care facilities. Res Gerontol Nurs. 2015;8:1–7. doi: 10.3928/19404921-20150413-01. [DOI] [PubMed] [Google Scholar]
  • 132.Temple-Smith M, Pirotta M, Kneebone J, McNamee K, Fairley C, Bilardi J, Hocking J. A missed opportunity lessons learnt from a chlamydia testing observation study in general practice. Aust Fam Physician. 2012;41(6):413–416. [PubMed] [Google Scholar]
  • 133.Teunissen E, Gravenhorst K, Dowrick C, Van Weel-Baumgarten E, Van Den Driessen MF, De Brún T, Burns N, Lionis C, Mair FS, O'Donnell C, et al. Implementing guidelines and training initiatives to improve cross-cultural communication in primary care consultations: a qualitative participatory European study. Int J Equity Health. 2017;16(1):32. doi: 10.1186/s12939-017-0525-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 134.de Brun T, De-Brun MO, van Weel-Baumgarten E, van Weel C, Dowrick C, Lionis C, O'Donnell CA, Burns N, Mair FS, Saridaki A et al: Guidelines and training initiatives that support communication in cross-cultural primary-care settings: appraising their implementability using normalization process theory. Fam Pract 2015, 32(4):420–425. [DOI] [PubMed]
  • 135.Lionis C, Papadakaki M, Saridaki A, Dowrick C, O'Donnell CA, Mair FS, Van Den Muijsenbergh M, Burns N, De Brún T, O'Reilly De Brún M, et al. Engaging migrants and other stakeholders to improve communication in cross-cultural consultation in primary care: a theoretically informed participatory study. BMJ Open. 2016;6(7):e010822. doi: 10.1136/bmjopen-2015-010822. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 136.Papadakaki M, Lionis C, Saridaki A, Dowrick C, de Brun T, O'Reilly-de Brun M, O'Donnell CA, Burns N, van Weel-Baumgarten E, van den Muijsenbergh M et al: Exploring barriers to primary care for migrants in Greece in times of austerity: perspectives of service providers. Eur J Gen Pract 2017, 23(1):128–134. [DOI] [PMC free article] [PubMed]
  • 137.Thomas K, Bendtsen P, Krevers B. Towards implementing coordinated healthy lifestyle promotion in primary care: a mixed method study. Int J Integr Care. 2015;18(3):312-24. [DOI] [PMC free article] [PubMed]
  • 138.Tierney E, O'Sullivan M, Hickey L, Hannigan A, May C, Cullen W, Kennedy N, Kineen L, MacFarlane A. Do primary care professionals agree about progress with implementation of primary care teams: results from a cross sectional study. BMC Fam Pract. 2016;17:163. doi: 10.1186/s12875-016-0541-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 139.Toye CRA. Normalisation process theory and the implementation of resident assessment instrument–home care in Saskatchewan, Canada: a qualitative study. Home Health Care Manage Pract. 2016;28(3):161–169. doi: 10.1177/1084822315619742. [DOI] [Google Scholar]
  • 140.Trietsch J, van Steenkiste B, Hobma S, Frericks A, Grol R, Metsemakers J, van der Weijden T. The challenge of transferring an implementation strategy from academia to the field: a process evaluation of local quality improvement collaboratives in Dutch primary care using the normalization process theory. J Eval Clin Pract. 2014;20(6):1162–1171. doi: 10.1111/jep.12287. [DOI] [PubMed] [Google Scholar]
  • 141.Vest BM, York TRM, Sand J, Fox CH, Kahn LS. Chronic kidney disease guideline implementation in primary care: a qualitative report from the TRANSLATE CKD study. J Am Board Fam Med. 2015;28(5):624–631. doi: 10.3122/jabfm.2015.05.150070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 142.Volker N, Williams LT, Davey RC, Cochrane T, Clancy T. Implementation of cardiovascular disease prevention in primary health care: enhancing understanding using normalisation process theory. BMC Fam Pract. 2017;18(1):28. doi: 10.1186/s12875-017-0580-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 143.Webster LAD, Ekers D, Chew-Graham CA. Feasibility of training practice nurses to deliver a psychosocial intervention within a collaborative care framework for people with depression and long-term conditions. BMC Nurs. 2016;15:71. doi: 10.1186/s12912-016-0190-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 144.Walker JG, Bickerstaffe A, Hewabandu N, Maddumarachchi S, Dowty JG, Jenkins M, Pirotta M, Walter FM, Emery JD. The CRISP colorectal cancer risk prediction tool: an exploratory study using simulated consultations in Australian primary care. BMC Med Inf Dec Making. 2017;17(1):1–11. doi: 10.1186/s12911-016-0389-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 145.Wilhelmsen M, Høifødt RS, Kolstrup N, Waterloo K, Eisemann M, Chenhall R, Risør MB. Norwegian general practitioners’ perspectives on implementation of a guided web-based cognitive behavioral therapy for depression: a qualitative study. J Med Internet Res. 2014;16(9):e208. doi: 10.2196/jmir.3556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 146.Wilkes S, Rubin G. Process evaluation of infertility management in primary care: has open access HSG been normalized? Prim Health Care Res Dev. 2009;10(04):290–298. doi: 10.1017/S1463423609990168. [DOI] [Google Scholar]
  • 147.Green T, Martins T, Hamilton W, Rubin G, Elliott K, Macleod U. Exploring GPs’ experiences of using diagnostic tools for cancer: a qualitative study in primary care. Fam Pract. 2014;32:cmu081. doi: 10.1093/fampra/cmu081. [DOI] [PubMed] [Google Scholar]
  • 148.Tazzyman A, Ferguson J, Hillier C, Boyd A, Tredinnick-Rowe J, Archer J, de Bere SR, Walshe K. The implementation of medical revalidation: an assessment using normalisation process theory. BMC Health Serv Res. 2017;17(1):749. doi: 10.1186/s12913-017-2710-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 149.Lohse KR, Hilderman CG, Cheung KL, Tatla S, Van der Loos HF. Virtual reality therapy for adults post-stroke: a systematic review and meta-analysis exploring virtual environments and commercial games in therapy. PLoS One. 2014;9(3):e93318. doi: 10.1371/journal.pone.0093318. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 150.Tam-Tham H, Cepoiu-Martin M, Ronksley PE, Maxwell CJ, Hemmelgarn BR. Dementia case management and risk of long-term care placement: a systematic review and meta-analysis. Int J Geriatr Psychiat. 2013;28(9):889–902. doi: 10.1002/gps.3906. [DOI] [PubMed] [Google Scholar]
  • 151.NICE . Chronic kidney disease. Early identification and management of chronic kidney disease in adults in primary and secondary care (CG73) London: National Institute for Health and Clinical Excellence; 2008. [PubMed] [Google Scholar]
  • 152.Crawford MJ, Rutter D, Manley C, Weaver T, Bhui K, Fulop N, Tyrer P. Systematic review of involving patients in the planning and development of health care. BMJ. 2002;325(7375):1263. doi: 10.1136/bmj.325.7375.1263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 153.Giacomini M, De Jean D, Simeonov D, Smith A. Experiences of living and dying with COPD: a systematic review and synthesis of the qualitative empirical literature. Ont Health Technol Assess Ser. 2012;12(13):1–47. [PMC free article] [PubMed] [Google Scholar]
  • 154.Brereton L, Carroll C, Barnston S. Interventions for adult family carers of people who have had a stroke: a systematic review. Clin Rehabil. 2007;21(10):867–884. doi: 10.1177/0269215507078313. [DOI] [PubMed] [Google Scholar]
  • 155.Gunn J, Diggens J, Hegarty K, Blashki G. A systematic review of complex system interventions designed to increase recovery from depression in primary care. BMC Health Serv Res. 2006;6:88. doi: 10.1186/1472-6963-6-88. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 156.Zijlstra GA, van Haastregt JC, van Rossum E, van Eijk JT, Yardley L, Kempen GI. Interventions to reduce fear of falling in community-living older people: a systematic review. J Am Geriatr Soc. 2007;55(4):603–615. doi: 10.1111/j.1532-5415.2007.01148.x. [DOI] [PubMed] [Google Scholar]
  • 157.NICE . Delirium: diagnosis, prevention and management (CG103) Manchester: National Institute for Clinical Excellence; 2010. [Google Scholar]
  • 158.Howard RL, Avery AJ, Slavenburg S, Royal S, Pipe G, Lucassen P, Pirmohamed M. Which drugs cause preventable admissions to hospital? A systematic review. Br J Clin Pharmacol. 2007;63(2):136–147. doi: 10.1111/j.1365-2125.2006.02698.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 159.Zwar N, Harris M, Griffiths R, Roland M, Denis S, Powell-Davies G, Hassan I. A systematic review of chronic disease management. Canberra: Australian Primary Care Research Institute; 2006. [Google Scholar]
  • 160.WHO . WHO recommendations for prevention and treatment of pre-eclampsia and eclampsia. Geneva: World Health Organization; 2011. [PubMed] [Google Scholar]
  • 161.Obermeyer CM, Osborn M. The utilization of testing and counseling for HIV: a review of the social and behavioral evidence. Am J Public Health. 2007;97(10):1762–1774. doi: 10.2105/AJPH.2006.096263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 162.Hensen B, Baggaley R, Wong VJ, Grabbe KL, Shaffer N, Lo YR, Hargreaves J. Universal voluntary HIV testing in antenatal care settings: a review of the contribution of provider-initiated testing & counselling. Tropical Med Int Health. 2012;17(1):59–70. doi: 10.1111/j.1365-3156.2011.02893.x. [DOI] [PubMed] [Google Scholar]
  • 163.Chou R, Huffman LH, Fu R, Smits AK, Korthuis PT, Force USPST Screening for HIV: a review of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med. 2005;143(1):55–73. doi: 10.7326/0003-4819-143-1-200507050-00010. [DOI] [PubMed] [Google Scholar]
  • 164.Pare G, Jaana M, Sicotte C. Systematic review of home telemonitoring for chronic diseases: the evidence base. J Am Med Inform Assoc. 2007;14(3):269–277. doi: 10.1197/jamia.M2270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 165.NICE . Osteoarthritis: national clinical guideline for care and management in adults (CG59) London: National Institue for Health and Care Excellence; 2008. [PubMed] [Google Scholar]
  • 166.Party I-CSW. National clinical guideline for stroke. London: Royal College of Physicians; 2012. [Google Scholar]
  • 167.Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, McKinstry B, Procter R, Majeed A, Sheikh A. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011;8(1):e1000387. doi: 10.1371/journal.pmed.1000387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 168.Mair F, May C, O'Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ. 2012;90:357–364. doi: 10.2471/BLT.11.099424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 169.Reid GT, Walter FM, Brisbane JM, Emery JD. Family history questionnaires designed for clinical use: a systematic review. Public Health Gen. 2009;12(2):73–83. doi: 10.1159/000160667. [DOI] [PubMed] [Google Scholar]
  • 170.Munro SA, Lewin SA, Smith HJ, Engel ME, Fretheim A, Volmink J. Patient adherence to tuberculosis treatment: a systematic review of qualitative research. PLoS Med. 2007;4(7):e238. doi: 10.1371/journal.pmed.0040238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 171.FSA . Guidance on food served to older people in residential care. London: Food Standards Agency; 2007. [Google Scholar]
  • 172.NICE . Chronic fatigue syndrome/myalgic encephalitis (or encalopathy): diagnosis and management of CFS/ME in adults and children (CG53) London: National Institute of Clinical Excellence; 2007. [Google Scholar]
  • 173.Cresswell K, Sheikh A. Organizational issues in the implementation and adoption of health information technology innovations: an interpretative review. Int J Med Inform. 2013;82(5):e73–e86. doi: 10.1016/j.ijmedinf.2012.10.007. [DOI] [PubMed] [Google Scholar]
  • 174.McDonald KM, Matesic B, Contopoulos-Ioannidis DG, Lonhart J, Schmidt E, Pineda N, Ioannidis JP. Patient safety strategies targeted at diagnostic errors: a systematic review. Ann Intern Med. 2013;158(5 Pt 2):381–389. doi: 10.7326/0003-4819-158-5-201303051-00004. [DOI] [PubMed] [Google Scholar]
  • 175.Eskicioglu C, Forbes SS, Aarts MA, Okrainec A, McLeod RS. Enhanced recovery after surgery (ERAS) programs for patients having colorectal surgery: a meta-analysis of randomized trials. J Gastrointest Surg. 2009;13(12):2321–2329. doi: 10.1007/s11605-009-0927-2. [DOI] [PubMed] [Google Scholar]
  • 176.NICE . Falls in older people: assessing risk and prevention (CG161) Manchester: National Institute for Clinical Excellence; 2013. [PubMed] [Google Scholar]
  • 177.NICE . Osteoporosis: secondary prevention including strontium ranelate (TA161) Manchester: National Institue of Clinical Excellence; 2008. [Google Scholar]
  • 178.NICE: Depression: the treatment and management of depression in adults (CG90). Manchester: National Institute of Clinical Excellence; 2009.
  • 179.Deshpande A, Khoja S, McKibbon A, Rizo C, Jadad A. Telehealth for acute stroke management (Telestroke): systematic review and environment scan (technology overview 37) Ottowa: Canadian Agency for Drugs and Technologies in Health; 2008. [Google Scholar]
  • 180.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 181.NICE . Referral guidelines for suspected cancer. London: National Institute for Health and Clinical Excellence; 2005. [Google Scholar]
  • 182.Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–1238. doi: 10.1001/jama.293.10.1223. [DOI] [PubMed] [Google Scholar]
  • 183.NICE . Acute kidney injury: prevention, detection and management of acute kidney injury up to the point of renal replacement therapy (CG169) Manchester: National Institute of Health and Care Excellence; 2013. [Google Scholar]
  • 184.Coulter A, Entwistle VA, Eccles A, Ryan S, Shepperd S, Perera R. Personalised care planning for adults with chronic or long-term health conditions. Cochrane Database Syst Rev. 2015;3:CD010523. doi: 10.1002/14651858.CD010523.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 185.O'Connor AM, Stacey D, Rovner D, Holmes-Rovner M, Tetroe J, Llewellyn-Thomas H, Entwistle V, Rostom A, Fiset V, Barry M, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2001;3:CD001431. doi: 10.1002/14651858.CD001431. [DOI] [PubMed] [Google Scholar]
  • 186.NICE . Chronic heart failure: management of chronic heart failure in adults in primary and secondary care. London: National Institute of Health and Care Excellence; 2008. [Google Scholar]
  • 187.Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144(10):742–752. doi: 10.7326/0003-4819-144-10-200605160-00125. [DOI] [PubMed] [Google Scholar]
  • 188.Sandall J, Soltani H, Gates S, Shennan A, Devane D. Midwife-led continuity models versus other models of care for childbearing women. Cochrane Database Syst Rev. 2013;8:CD004667. doi: 10.1002/14651858.CD004667.pub3. [DOI] [PubMed] [Google Scholar]
  • 189.Shepperd S, McClaran J, Phillips C, Lannin N, Clemson L, McCluskey A, Cameron I, Barras S. Discharge planning from hospital to home. Cochrane Database Syst Rev. 2010;1(Issue 1):CD000313. doi: 10.1002/14651858.CD000313.pub3. [DOI] [PubMed] [Google Scholar]
  • 190.Ougrin D, Tranah T, Stahl D, Moran P, Asarnow JR. Therapeutic interventions for suicide attempts and self-harm in adolescents: systematic review and meta-analysis. J Am Acad Child Adolesc Psychiatry. 2015;54(2):97–107. doi: 10.1016/j.jaac.2014.10.009. [DOI] [PubMed] [Google Scholar]
  • 191.Mistiaen P, Francke AL, Poot E. Interventions aimed at reducing problems in adult patients discharged from hospital to home: a systematic meta-review. BMC Health Serv Res. 2007;7:47. doi: 10.1186/1472-6963-7-47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 192.Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. doi: 10.1186/1472-6963-10-231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 193.Petrie J, Bunn F, Byrne G. Parenting programmes for preventing tobacco, alcohol or drugs misuse in children <18: a systematic review. Health Educ Res. 2007;22(2):177–191. doi: 10.1093/her/cyl061. [DOI] [PubMed] [Google Scholar]
  • 194.Renders CM, Valk GD, Griffin S, Wagner EH, Eijk JT, Assendelft WJ. Interventions to improve the management of diabetes mellitus in primary care, outpatient and community settings. Cochrane Database Syst Rev. 2001;1:CD001481. doi: 10.1002/14651858.CD001481. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 195.Flodgren G, Parmelli E, Doumit G, Gattellari M, O'Brien MA, Grimshaw J, Eccles MP. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2011;8:CD000125. doi: 10.1002/14651858.CD000125.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 196.Bower P, Gilbody S, Richards D, Fletcher J, Sutton A. Collaborative care for depression in primary care. Making sense of a complex intervention: systematic review and meta-regression. Br J Psychiatry. 2006;189:484–493. doi: 10.1192/bjp.bp.106.023655. [DOI] [PubMed] [Google Scholar]
  • 197.Arnold SR, Straus SE. Interventions to improve antibiotic prescribing practices in ambulatory care. Cochrane Database Syst Rev. 2005;4:CD003539. doi: 10.1002/14651858.CD003539.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 198.Fink HA, Ishani A, Taylor BC, Greer NL, MacDonald R, Rossini D, Sadiq S, Lankireddy S, Kane RL, Wilt TJ. Screening for, monitoring, and treatment of chronic kidney disease stages 1 to 3: a systematic review for the U.S. Preventive Services Task Force and for an American College of Physicians Clinical Practice Guideline. Ann Intern Med. 2012;156(8):570–581. doi: 10.7326/0003-4819-156-8-201204170-00008. [DOI] [PubMed] [Google Scholar]
  • 199.NICE . Depression in adults with a chronic physical health problem (CG91) Manchester: National Institute for Health and Clinical Excellence; 2009. [Google Scholar]
  • 200.Farrand P, Woodford J. Impact of support on the effectiveness of written cognitive behavioural self-help: a systematic review and meta-analysis of randomised controlled trials. Clin Psychol Rev. 2013;33(1):182–195. doi: 10.1016/j.cpr.2012.11.001. [DOI] [PubMed] [Google Scholar]
  • 201.Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med. 2009;39(5):705–712. doi: 10.1017/S0033291708004224. [DOI] [PubMed] [Google Scholar]
  • 202.Gask L, Rogers A, Campbell S, Sheaff R. Beyond the limits of clinical governance? The case of mental health in English primary care. BMC Health Serv Res. 2008;8(1):63. doi: 10.1186/1472-6963-8-63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 203.Gitlin LN, Harris LF, McCoy MC, Hess E, Hauck WW. Delivery characteristics, acceptability, and depression outcomes of a home-based depression intervention for older African Americans: the get busy get better program. The Gerontologist. 2015;56:gnv117. doi: 10.1093/geront/gnv117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 204.Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–1288. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
  • 205.Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Bryman A, Burgess R, editors. Analysing qualitative data. London: Routledge; 1994. pp. 173–194. [Google Scholar]
  • 206.Dedoose . Dedoose version 4.5, web application for managing, analyzing, and presenting qualitative and mixed method research data. Los Angeles: SocioCultural Research Consultants, LLC; 2013. [Google Scholar]
  • 207.de Brún T, de-Brún MOR, van Weel-Baumgarten E, van Weel C, Dowrick C, Lionis C, O’Donnell CA, Burns N, Mair FS, Saridaki A, et al. Guidelines and training initiatives that support communication in cross-cultural primary-care settings: appraising their implementability using normalization process theory. Fam Pract. 2015;32(4):420–425. doi: 10.1093/fampra/cmv022. [DOI] [PubMed] [Google Scholar]
  • 208.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4 [DOI] [PMC free article] [PubMed]
  • 209.Rycroft-Malone J. The PARIHS framework—a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual. 2004;19(4):297–304. doi: 10.1097/00001786-200410000-00002. [DOI] [PubMed] [Google Scholar]
  • 210.MacFarlane A, O'Reilly-de Brún M. Using a theory-driven conceptual framework in qualitative health research. Qual Health Res. 2012;22(5):607-18. [DOI] [PubMed]
  • 211.Morrison A, Polisena J, Husereau D, Moulton K, Clark M, Fiander M, Mierzwinski-Urban M, Clifford T, Hutton B, Rabb D. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int J Technol Assess. 2012;28(2):138–144. doi: 10.1017/S0266462312000086. [DOI] [PubMed] [Google Scholar]
  • 212.Gallacher K, May C, Montori V, Mair F. Assessing treatment burden in chronic heart failure patients. Heart. 2010;96(s1):A37–A38. doi: 10.1370/afm.1249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 213.May C. Retheorizing the clinical encounter. In: Scambler G, Scambler S, editors. Assaults on the lifeworld: new directions in the sociology of chronic and disabling conditions. London: Routledge; 2010. [Google Scholar]
  • 214.Shippee ND, Shah ND, May CR, Mair FS, Montori VM. Cumulative complexity: a functional, patient-centered model of patient complexity can improve research and practice. J Clin Epidemiol. 2012;65(10):1041–1051. doi: 10.1016/j.jclinepi.2012.05.005. [DOI] [PubMed] [Google Scholar]
  • 215.May C, Eton DT, Boehmer KR, Gallacher K, Hunt K, MacDonald S, Mair FS, May CM, Montori VM, Richardson A, et al. Rethinking the patient: using burden of treatment theory to understand the changing dynamics of illness. BMC Health Serv Res. 2014;14(1):281. doi: 10.1186/1472-6963-14-281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 216.Burridge LH, Foster MM, Donald M, Zhang J, Russell AW, Jackson CL. Making sense of change: patients’ views of diabetes and GP-led integrated diabetes care. Health Expect. 2016;19(1):74–86. doi: 10.1111/hex.12331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 217.Gallacher K, May CR, Montori VM, Mair FS. Understanding patients’ experiences of treatment burden in chronic heart failure using normalization process theory. Annal Fam Med. 2011;9(3):235–243. doi: 10.1370/afm.1249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 218.McNaughton RJ, Shucksmith J. Reasons for (non)compliance with intervention following identification of ‘high-risk’ status in the NHS health check programme. Journal of Public Health. 2015;37(2):218–225. doi: 10.1093/pubmed/fdu066. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 219.Tiedje K, Boehm D, Montori VM. Factors that lessen the burden of treatment in complex patients with chronic conditions: a qualitative study. Patient Preference Adherence. 2014;8:339–351. doi: 10.2147/PPA.S58014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 220.Tiedje K, Shippee ND, Johnson AM, Flynn PM, Finnie DM, Liesinger JT, May CR, Olson ME, Ridgeway JL, Shah ND, et al. ‘They leave at least believing they had a part in the discussion’: understanding decision aid use and patient-clinician decision-making through qualitative research. Patient Educ Couns. 2013;93(1):86–94. doi: 10.1016/j.pec.2013.03.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 221.May CR, Cummings A, Myall M, Harvey J, Pope C, Griffiths P, Roderick P, Arber M, Boehmer K, Mair FS, et al. Experiences of long-term life-limiting conditions among patients and carers: what can we learn from a meta-review of systematic reviews of qualitative studies of chronic heart failure, chronic obstructive pulmonary disease and chronic kidney disease? BMJ Open. 2016;6(10):e011694. doi: 10.1136/bmjopen-2016-011694. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 222.Sav A, King MA, Whitty JA, Kendall E, McMillan SS, Kelly F, Hunter B, Wheeler AJ. Burden of treatment for chronic illness: a concept analysis and review of the literature. Health Expect. 2015;18(3):312-24. [DOI] [PMC free article] [PubMed]
  • 223.Demain S, Gonçalves A-C, Areia C, Oliveira R, Marcos AJ, Marques A, Parmar R, Hunt K. Living with, managing and minimising treatment burden in long term conditions: a systematic review of qualitative research. PLoS One. 2015;10(5):e0125457. doi: 10.1371/journal.pone.0125457. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 224.Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, Steen IN, McColl EM, Dickinson C, Rapley T: NoMAD: implementation measure based on normalization process theory. Retrieved from http://wwwnormalizationprocessorg (12 April 2016) 2015.
  • 225.O'Donnell CA, Mair FS, Dowrick C, MOR-d B, Td B, Burns N, Lionis C, Saridaki A, Papadakaki M, MvD M, et al. Supporting the use of theory in cross-country health services research: a participatory qualitative approach using normalisation process theory as an example. BMJ Open. 2017;7(8):e014289. doi: 10.1136/bmjopen-2016-014289. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 226.Murray E, May C, Mair F. Development and formative evaluation of the e-Health Implementation Toolkit (e-HIT) Bmc Med Inf Dec Making. 2010;10(1):61. doi: 10.1186/1472-6947-10-61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 227.Muller I, Yardley L. Telephone-delivered cognitive behavioural therapy: a systematic review and meta-analysis. J Telemed Telecare. 2011;17(4):177–184. doi: 10.1258/jtt.2010.100709. [DOI] [PubMed] [Google Scholar]
  • 228.Mohr DC, Ho J, Duffecy J, Reifler D, Sokol L, Burns MN, Jin L, Siddique J. Effect of telephone-administered vs face-to-face cognitive behavioral therapy on adherence to therapy and depression outcomes among primary care patients a randomized trial. Jama J Am Med Assoc. 2012;307(21):2278–2285. doi: 10.1001/jama.2012.5588. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 229.Bridges J, Flatley M, Meyer J. Older people’s and relatives’ experiences in acute care settings: systematic review and synthesis of qualitative studies. Int J Nurs Stud. 2010;47(1):89–107. doi: 10.1016/j.ijnurstu.2009.09.009. [DOI] [PubMed] [Google Scholar]
  • 230.Greenfield D, Braithwaite J. Health sector accreditation research: a systematic review. Int J Qual Health Care. 2008;20(3):172–183. doi: 10.1093/intqhc/mzn005. [DOI] [PubMed] [Google Scholar]
  • 231.Kousoulis AA, Patelarou E, Shea S, Foss C, Knutsen IAR, Todorova E, Roukova P, Portillo MC, Pumar-Mendez MJ, Mujika A, et al. Diabetes self-management arrangements in Europe: a realist review to facilitate a project implemented in six countries. BMC Health Serv Res. 2014;14:453. doi: 10.1186/1472-6963-14-453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 232.Khosravi P, Ghapanchi AH. Investigating the effectiveness of technologies applied to assist seniors: a systematic literature review. Int J Med Inform. 2016;85(1):17–26. doi: 10.1016/j.ijmedinf.2015.05.014. [DOI] [PubMed] [Google Scholar]
  • 233.NICE . Psychosis and schizophrenia in adults: prevention and management [CG178] London: National Institute for Health and Care Excellence; 2014. [PubMed] [Google Scholar]
  • 234.APA . Practice guideline for the treatment of patients with bipolar disorder (2nd edition) Arlington: American Psychiatric Association; 2002. [Google Scholar]
  • 235.Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5(9):e008592. doi: 10.1136/bmjopen-2015-008592. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 236.Elwyn G, Legare F, Edwards A, van der Weijden T, May C. Arduous implementation: does the normalisation process model explain why it is so difficult to embed decision support technologies in routine clinical practice. Implement Sci. 2008;3:57. doi: 10.1186/1748-5908-3-57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 237.Elwyn G, Scholl I, Tietbohl C, Mann M, Edwards AGK, Clay C, Legare F, van der Weijden T, Lewis CL, Wexler RM, et al. “Many miles to go ... ”: a systematic review of the implementation of patient decision support interventions into routine clinical practice. BMC Med Inf Dec Making. 2013:13. [DOI] [PMC free article] [PubMed]
  • 238.Feder G, Ramsay J, Dunne D, Rose M, Arsene C, Norman R, Kuntze S, Spencer A, Bacchus L, Hague G, et al. How far does screening women for domestic (partner) violence in different health-care settings meet criteria for a screening programme? Systematic reviews of nine UK National Screening Committee criteria. Health Technol Assess. 2009;13(16):1–113. doi: 10.3310/hta13160. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Additional file 1: (156.1KB, pdf)

Appendix data extraction tool. (PDF 156 kb)


Articles from Implementation Science : IS are provided here courtesy of BMC

RESOURCES