Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review

Abstract

Objective

To systematically review the literature on the implementation of e-health to identify: (i) barriers and facilitators to e-health implementation, and (ii) outstanding gaps in research on the subject.

Methods

MEDLINE, EMBASE, CINAHL, PSYCINFO and the Cochrane Library were searched for reviews published between 1 January 1995 and 17 March 2009. Studies had to be systematic reviews, narrative reviews, qualitative metasyntheses or meta-ethnographies of e-health implementation. Abstracts and papers were double screened and data were extracted on country of origin; e-health domain; publication date; aims and methods; databases searched; inclusion and exclusion criteria and number of papers included. Data were analysed qualitatively using normalization process theory as an explanatory coding framework.

Findings

Inclusion criteria were met by 37 papers; 20 had been published between 1995 and 2007 and 17 between 2008 and 2009. Methodological quality was poor: 19 papers did not specify the inclusion and exclusion criteria and 13 did not indicate the precise number of articles screened. The use of normalization process theory as a conceptual framework revealed that relatively little attention was paid to: (i) work directed at making sense of e-health systems, specifying their purposes and benefits, establishing their value to users and planning their implementation; (ii) factors promoting or inhibiting engagement and participation; (iii) effects on roles and responsibilities; (iv) risk management, and (v) ways in which implementation processes might be reconfigured by user-produced knowledge.

Conclusion

The published literature focused on organizational issues, neglecting the wider social framework that must be considered when introducing new technologies.

Introduction

Health-care providers have increasingly sought to utilize e-health systems that employ information and communications technologies to widen access, improve quality and increase service efficiency. Enthusiasm for technological innovation around e-health among policy-makers and health officials has, however, not always been matched by uptake and utilization in practice.^1,2 Professional resistance to new technologies is cited as a major barrier to progress, although evidence for such assertions is weak.³ Implementing and embedding new technologies of any kind involves complex processes of change at the micro level for professionals and patients and at the meso level for health-care organizations themselves. The European Union has recently argued that implementing e-health strategies “has almost everywhere proven to be much more complex and time-consuming than initially anticipated”.⁴

Over the past decade the number of primary studies evaluating the practical implementation and integration of e-health systems has steadily grown. Sometimes these studies describe important successes, but more often they are accounts of complex processes with ambiguous outcomes. As the research community has sought to make sense of these studies, systematic reviews attempting to identify and describe “barriers” and “facilitators” to implementation have proliferated. Although the reviews have furthered knowledge by identifying factors thought to influence implementation processes and their outcomes, the underlying mechanisms at work have not been well characterized or explained. The literature is fragmented across multiple subspecialty areas, so those charged with designing and implementing e-health systems may find it difficult to locate an appropriate body of evidence and to determine the relevance of that evidence to their specific circumstances.

In this meta-review we have sought to address these problems in two ways. First, we have performed a systematic review of reviews of e-health implementation studies, focusing on implementation processes rather than outcomes, to critically appraise such reviews, evaluate their methods, synthesize their results and highlight their key messages. Our meta-review has enabled us to explore and evaluate a large and fragmented body of research in a coherent and economical way. Second, we have interpreted our results in the light of an explanatory framework – Normalization Process Theory (NPT)^5,6 – that specifies mechanisms of importance in implementation processes. This approach has facilitated the explanation of those factors shown to influence the implementation of e-health systems in practice and allowed us to identify important gaps in the literature and to make rational recommendations for further primary research.

The objective of this review was to synthesize and summarize the findings of identified reviews and inform current and future e-health implementation programmes. The review set out to answer two key questions: (i) What does the published literature tell us about barriers and facilitators to e-health implementation? (ii) What, if any, are the main research gaps?

Methods

Inclusion and exclusion criteria

Box 1 lists the inclusion and exclusion criteria. We used a previously developed method of categorization to classify e-health interventions into four domains:⁷ management systems, communication systems, computerised decision support systems and information resources.

Box 1. Inclusion/exclusion criteria for systematic review of reviews on e-health implementation.

Inclusion

Papers on the subject of e-health and its implementation that met the following criteria were included:

• Systematic reviews: structured search of bibliographic and other databases to identify relevant literature; use of transparent methodological criteria to exclude papers not meeting an explicit methodological benchmark; presentation of rigorous conclusions about outcomes.

• Narrative reviews: purposive sampling of the literature; use of theoretical or topical criteria to include papers on the basis of type, relevance and perceived significance, with the aim of summarizing, discussing and critiquing conclusions.

• Qualitative meta-syntheses or meta-ethnographies: structured search of bibliographic and other databases to identify relevant literature; use of transparent methods to draw together theoretical products, with the aim of elaborating and extending theory.

Exclusion

• Secondary analyses (including qualitative meta-syntheses or meta-ethnographies) of existing data sets for the purpose of presenting cumulative outcomes from personal research programmes.

• Secondary analyses (including qualitative meta-syntheses or meta-ethnographies) of existing data sets for the purpose of presenting integrative outcomes from different research programmes.

• Discussions of literature included in contributions to theory building or critique.

• Summaries of the literature for the purpose of information or commentary.

• Editorial discussions that argued the case for a field of research or a course of action.

• Papers whose abstract identified them as reviews but that lacked supporting evidence in the main text (e.g. details on the databases searched or the selection criteria).

Finding relevant studies

We searched the following electronic bibliographic databases: MEDLINE; EMBASE; CINAHL; PSYCINFO; the Cochrane Library, including the Cochrane Database of Systematic reviews and the Cochrane Central Register of Controlled Trials; DARE; the National Health Service (NHS) Economic Evaluation Database, and the Health Technology Assessment Database. Box 2 describes our search terms.

Box 2. Search terms used for systematic review of reviews on e-health implementation.

Thesaurus terms referring to e-health interventions were: Medical-Informatics-Applications; Management-Information-Systems; Decision-Making-Computer-Assisted; Diagnosis-Computer-Assisted; Therapy-Computer-Assisted; Medical-Records-Systems-Computerized; Medical-Order-Entry-Systems; Electronic-Mail; Videoconferencing; Telemedicine; Computer-Communication-Networks; Internet.

Where appropriate, thesaurus terms were exploded to include all terms below the searched term in the thesaurus tree. The lowest term was always exploded.

There are no thesaurus terms for implementation, so this concept was searched for by looking for the following text words in title, keywords or abstract: Routin*; Normali?*; Integrat*; Facilitate*; Barrier*; Implement*; Adopt*. The concepts of e-health intervention and implementation were combined, and then the search was limited by publication type (i.e. review or meta-analysis).

We limited the MEDLINE database search to studies published in any language from 1990 to 2009. None of the non-English-language citations or of the papers published before 1995 were relevant. Hence, the MEDLINE search was re-run for only English-language reviews published between 1 January 1995 and 31 July 2009. These limits were used for searching all other databases. Each database’s thesaurus terms were used to perform the search.

Data abstraction and analysis

Citations were downloaded into Reference Manager 11 (ISI ResearchSoft, Carlsbad, United States of America), and screened by two reviewers. If either reviewer could not exclude the paper based on the abstract or citation, the full paper was obtained. All papers obtained were double screened. In case of disagreement about inclusion or exclusion of a given paper, all reviewers read the paper and reached agreement through discussion.

Data were extracted in two stages. First we used a standardized data extraction instrument to categorize papers on the basis of country of origin; e-health domain; publisher; date of publication; review aims and methods; databases searched within the review; inclusion and exclusion criteria of review; number of papers identified and number included in the review.

Second, as the literature under study focused on implementation processes rather than outcomes, we analysed the extracted data qualitatively using NPT, which has four constructs (coherence, cognitive participation, collective action and reflexive monitoring) as a coding framework.^5,6 This theory provides a conceptual framework to explain the processes by which new health technologies and other complex interventions are routinely operationalized in everyday work (embedded) and sustained in practice (integrated).^8,9 Use of this framework to aid data analysis in systematic reviews of qualitative data has recently been described.¹⁰ For every paper two reviewers judged whether material relevant to the four constructs of NPT was present or absent, using the coding frame shown in Table 1. As this was a qualitative content analysis,¹¹ we did not try to quantify the weight put on any one NPT construct in a given review. Each statement in a paper relating to findings regarding barriers or facilitators to e-health implementation was treated as an “attributive statement”; two reviewers coded these statements to the relevant construct of the NPT. If an “attributive statement” could not be coded to the NPT framework, this was stated to ensure that issues outside the scope of the theory would still be captured. Dual coding enabled differences in coding and interpretation to be identified and discussed. Disagreement, which was minimal, was resolved through discussion. If any areas of disagreement remained, a final reviewer served as arbiter.

Table 1. Normalization process theory coding framework used for qualitative analysis of review data on e-health implementation .

Coherence (Sense-making work)	Cognitive participation (Relationship work)	Collective action (Enacting work)	Reflexive monitoring (Appraisal work)
Differentiation Is there a clear understanding of how a new e-health service differs from existing practice?	Enrolment Do individuals “buy into” the idea of the e-health service?	Skill set workability How does the innovation affect roles and responsibilities or training needs?	Reconfiguration Do individuals try to alter the new service?
Communal specification Do individuals have a shared understanding of the aims, objectives and expected benefits of the e-health service?	Activation Can individuals sustain involvement?	Contextual Integration Is there organizational support?	Communal appraisal How do groups judge the value of the e-health service?
Individual specification Do individuals have a clear understanding of their specific tasks and responsibilities in the implementation of an e-health service?	Initiation Are key individuals willing to drive the implementation?	Interactional workability Does the e-health service make people’s work easier?	Individual appraisal How do individuals appraise the effects on them and their work environment?
Internalization Do individuals understand the value, benefits and importance of the e-health service?	Legitimation Do individuals believe it is right for them to be involved?	Relational integration Do individuals have confidence in the new system?	Systematization How are benefits or problems identified or measured?

The methods used for study identification and data collection in this study were in keeping with the recent PRISMA statement.¹² However, as this was a review of process rather than outcome studies, some aspects of the PRISMA statement were not applicable.

Results

From 8206 unique citations screened, we excluded 7973 on the basis of the title or abstract and retrieved 233 full-text articles. Of these, 37 met the inclusion criteria (Fig. 1). Of note, 20 of these reviews were published between 1995 and 2007 and 17 were published in the following two years.

Fig. 1.

Flowchart of study selection in systematic review of reviews on e-health implementation

Of the 37 included reviews, 18 originated in the United States of America, 10 in Canada, 3 in the United Kingdom of Great Britain and Northern Ireland, 2 in the Netherlands and 1 each in Australia, Germany, Malaysia and Norway.^13–49 Reviews generally covered one or more e-health domains: eight reported on management systems; 10 on communication systems; 6 on decision support; 1 mainly on information systems, and 12 on combinations of these. Full details of included papers are available from the authors.

When judged against the PRISMA checklist,¹² many of the included reviews were methodologically poor. For example, 5 of the 37 reviews did not clearly describe the databases searched.^{22,25,32,36,49} Information on search strategies was often rudimentary. Of the 37 included reviews, 7 searched only one or two databases or sources, such as the proceedings of a particular conference.^{14,15,28,38,42,44,48} Information about study selection criteria was also inadequate: 19 of the 37 reviews did not specify the criteria for inclusion or exclusion.^{15,16,18,21–23,25,26,29,31–36,41–44} For 13 of the 37 papers reviewed it was impossible to ascertain precisely how many studies had been included.^{17,18,22,23,25,26,28,29,32,35,38,41,42.}

As the reviews under investigation dealt with organizational and other processes rather than with numeric outcome measures, the PRISMA checklists for summary measures and result synthesis were not applicable. NPT was used to aid analysis and conceptualization of the qualitative data regarding barriers and facilitators of implementation.

Content analysis of the 37 reviews identified 801 attributive statements about implementation processes that could be interpreted using NPT as an explanatory framework.

Coherence

Coherence refers to the “sense-making" work undertaken when a new e-health service is implemented (e.g. to determine whether users see it as differing from existing practice, have a shared view of its purpose, understand how it will affect them personally and grasp its potential benefits). A surprising and important result of this review was the discovery that work directed at making sense of e-health systems received very little coverage. Content analysis of the reviews showed it to be the focus of less than 12% (95/801) of the attributive statements. Coherence work was seen to be concerned with preparatory activities – often policy building or dissemination of information – undertaken either locally or nationally.

Since “sense-making” work is an important aspect of implementation, this review reveals an important gap in the literature. We do not know whether the gap reflects the exclusion of coherence work from process-oriented studies (which is possible if evaluations commence in the delivery phase of a project) or the systematic absence of sense-making work itself.

Cognitive participation

Cognitive participation focuses upon the work undertaken to engage with potential users and get them to “buy into” the new e-health system. Although this work of relating and engaging with users is central to the successful implementation of any new technology, work aimed at actively involving health professionals in e-health services rarely figured in the reviews we examined. The same was true of work leading to the initiation and legitimation of health technologies or geared towards sustaining them in practice. Less than 11% (88/801) of attributive statements fell within this category, and those that did took the form of general recommendations rather than specific design and delivery considerations. For example, Hilty²³ proposes ways to encourage health professional participation, including “incentives for each of the parties involved”.

Other issues that fell within the cognitive participation category included a range of actions to legitimize participation in the implementation process and promote it as a worthwhile activity, such as the recruitment of local “champions”. Such champions were seen as having the ability to promote utilization of new e-health services by more reticent colleagues. However, this approach could be a double-edged sword. Health professionals who enthusiastically support e-health can get colleagues to enrol in and commit to e-health programmes. However, those who project a negative attitude can jeopardize the staff commitment needed to make an e-health system work and thus impede implementation.

Since participation and engagement are vital for the success of new technologies, the lack of coverage in the reviews of the factors that promote or inhibit user engagement and participation is clearly a major weakness in the literature.

Collective action

Work involved in implementing or enacting e-health systems was the topic of 65% (518/801) of the attributive statements identified. The emphasis in this domain was on the work performed by individuals, groups of professionals or organizations in operationalizing a new technology in practice. We found that the research emphasis changed over time. Up to 2007 the emphasis lay on organizational issues, but after that year it shifted towards socio-technical issues (e.g. how e-health systems affected the everyday work of individuals).

Addressing organizational issues

Most reviews focused on the ways in which the e-health innovation affected organizational structures and goals. This was especially true up to 2007, when 35% (142/411) of the attributive statements identified focused on this point, as opposed to only 20% (77/390) after 2007. The literature highlighted the need for adequate resources, particularly financial. Administrative support, policy support, standards and interoperability also fell within this research category.

This area’s emphasis is on the contextual integration of e-health systems, particularly on the extent to which they are managed and resourced. The focus on management is not surprising, but the emphasis on a “top down” approach draws attention away from other equally important aspects of collective action.

Effects on health care tasks

The interactional workability of e-health systems accounted for less than 18% of the attributive statements identified during content analysis. Many of these focused on the “ease of use” of the new systems for clinicians, with the underlying assumption that clinicians would be deterred from or resistant to using systems that added complexity or required additional effort or time.

Ease of use for patients or other service users (or even health professionals besides clinicians, such as nurses) did not figure prominently in the reviews we investigated. However, the effect of e-health systems on physician–patient interaction did receive some attention, as exemplified by the following quote:

“… an effective clinical decision support system must minimise the effort required by clinicians to receive and act on system recommendations”.³⁰

Thus, implementation may be retarded or destabilized by the competing priorities of powerful participants.

Confidence and accountability

The relational integration of e-health systems (confidence, security and accountability) accounted for 15% (116/801) of the attributive statements identified. Such concerns could act as either facilitators or barriers. Users may see in e-health technologies a way to reduce errors, which would encourage uptake; alternatively, security and safety concerns could undermine confidence in e-health systems and hinder their widespread utilization.

Roles, responsibilities and training

Roles, responsibilities and training or support issues accounted for only about 10% (77/801) of attributive statements. Most emphasis was placed on the need to adequately train staff members for engagement in implementation, although division of labour was also a concern, as were effects on workload. However, these issues were often discussed superficially, without examining the types of training or ongoing support that would be required.

Reflexive monitoring

While the majority of attributive statements identified in the content analysis dealt with managerial interventions and controls, much less information was provided on the ways in which managers and other users appraise whether an e-health intervention is worthwhile or not. Only 13% (104/801) of the attributive statements fell within this category. Most of these dealt with evaluation and monitoring and how they are used to influence utilization and future e-health implementations. However, evaluation was also promoted as necessary to ensure that safety concerns were addressed.

Evaluation could, of course, either allay concerns or confirm the need for amendments to the e-health service being implemented. There was little evidence of local appraisals or of the ways in which implementation processes might be reconfigured by user-produced knowledge.

Only 6% of issues fell outside our coding framework, either because they were strictly technical and attitudinal or because they were so generic and vague, without accompanying contextual data, that it was not possible to determine whether the concept really lay outside the model or was simply too general to be coded.

Discussion

A thorough and systematic search of reviews published over the preceding 15 years identified 37 reviews on the implementation of e-health technologies in health-care settings. Most were from North America. Methodological deficiencies were common and the findings should be interpreted with caution. The number of publications has risen rapidly since 2008, which suggests that there is growing awareness of the need to understand and address issues related to implementation if e-health services are to become a core component of routine service delivery.

This review breaks new ground. It not only collates and summarizes data but also analyses it and interprets it within a theoretical framework. Our approach has allowed us to explore the factors that facilitate and hinder implementation, identify gaps in the literature and highlight directions for future research. In particular, this work highlights a continued focus on organizational issues, which, despite their importance, are only one among a range of factors that need to be considered when implementing e-health systems.^7–9,50

Although our meta-review was rigorous and carefully executed and employed a robust conceptual framework, it has limitations. Since not all primary research has been captured by previous reviews, our meta-review does not include findings from all studies in this field. Furthermore, review data is two steps removed from primary data, and the quality of the primary research may not be properly assessed in reviews of substandard quality. Finally, since the reviews we identified were of poor quality on average and their search strategies were not always comprehensive, their findings may be biased.

Conclusion

Our review has revealed a growing emphasis on problems related to e-health systems’ workability but relatively little attention to: (i) e-health’s effects on roles and responsibilities; (ii) risk management; (iii) ways to engage with professionals; and (iv) ensuring that the potential benefits of new technologies are made transparent through ongoing evaluation and feedback. These areas deserve more empirical investigation, as do ways to identify and anticipate how e-health services will impact everyday clinical practice. This involves examining how new e-health services will affect clinical interactions and activities and the allocation and performance of clinical work. Also in need of investigation are the effects of different methods of engaging with professionals before and during the implementation of e-health services.