Skip to main content
Elsevier Sponsored Documents logoLink to Elsevier Sponsored Documents
. 2020 Jan;124(1):12–24. doi: 10.1016/j.healthpol.2019.11.006

Review of 128 quality of care mechanisms: A framework and mapping for health system stewards

Juan E Tello a,, Erica Barbazza b,d, Kerry Waddell c,d
PMCID: PMC6946442  PMID: 31791717

Highlights

  • Quality of care mechanisms are available across governance sub-functions and target areas.

  • Most of the 128 mechanisms relate to setting priorities/standards and organizing/monitoring action.

  • The taxonomy developed can serve as a tool for prioritization by system stewards.

  • 13 mechanisms have sufficient evidence on their effectiveness, 33 have some evidence.

  • The optimal combination of mechanisms to reinforce their effectiveness is context-specific.

Keywords: Quality of health care, Quality improvement, Health policy, Clinical governance

Abstract

Health system stewards have the critical task to identify quality of care deficiencies and resolve underlying system limitations. Despite a growing evidence-base on the effectiveness of certain mechanisms for improving quality of care, frameworks to facilitate the oversight function of stewards and the use of mechanisms to improve outcomes remain underdeveloped. This review set out to catalogue a wide range of quality of care mechanisms and evidence on their effectiveness, and to map these in a framework along two dimensions: (i) governance subfunctions; and (ii) targets of quality of care mechanisms. To identify quality of care mechanisms, a series of searches were run in Health Systems Evidence and PubMed. Additional grey literature was reviewed. A total of 128 quality of care mechanisms were identified. For each mechanism, searches were carried out for systematic reviews on their effectiveness. These findings were mapped in the framework defined. The mapping illustrates the range and evidence for mechanisms varies and is more developed for some target areas such as the health workforce. Across the governance sub-functions, more mechanisms and with evidence of effectiveness are found for setting priorities and standards and organizing and monitoring for action. This framework can support system stewards to map the quality of care mechanisms used in their systems and to uncover opportunities for optimization backed by systems thinking.

1. Background

Quality is a basic tenet of services delivery. It is also central to universal health coverage (UHC) as poor quality, independent of access, can be a barrier to UHC [1]. Sustainable Development Goal 3 has called attention to this critical link, as the attainment of health targets demands both the expansion of access to basic health services and enhanced quality; without the latter UHC will prove largely ‘an empty vessel’ [2]. Recent evidence on the magnitude and cost of deficits in healthcare quality, including deaths caused by adverse events in hospitals, high levels of excess and inappropriate care and unnecessary prescribing contributing to antimicrobial resistance among other public health concerns, have served to underscore the urgency of comprehensive health system efforts to improve quality alongside access [2].

It is clear the global health community is awakening to this challenge. In 2018, three flagship reports put a spotlight on the link between quality of care and global health [[2], [3], [4]]. These reports have presented the components for national health sector quality plans and policies [3], developed new approaches to define, measure and improve quality of care [4], and quantified the state of quality of care globally as well as the implications of current quality gaps [2]. Taken together with other recent studies – including a systematic review on the effectiveness of strategies to improve the performance of health care providers [5] and study on the use of quality strategies in the context of European health systems [6] – it is clear there is a critical mass of evidence and know-how to take concerted action. These studies build on earlier work on topics such as quality of care policies [[7], [8], [9], [10], [11]], strategies for quality improvement [[12], [13], [14], [15]] and the development of frameworks for monitoring and evaluating quality of care [[16], [17], [18]].

Despite the growing literature base and understanding of quality of care and its mechanisms – as the actionable interventions, strategies or tools to improve quality of care – thoughtful study on their use remains underdeveloped. Activating quality of care mechanisms is the responsibility of real-world health system stewards in their role to govern the health system and redress system limitations. The dilemma stewards face, however, is knowing which quality of care mechanisms will have the greatest influence on the health system and when to use them [10]. That is, in order to be ‘useful’ quality of care mechanisms should be embedded within the processes of governing and serve the intended target area of improvement in alignment with other quality of care mechanisms.

In the absence of guidance on how to optimize the use of quality of care mechanisms, popularized mechanisms like pay-for-performance, audit and feedback and clinical protocols and guidelines, are often pursued in practice as individual, catch-all quality enablers. The consequence is the inefficient and ineffective use of mechanisms to improve quality of care and overall health outcomes. For example, mechanisms like professional certification, clinical protocols or facility accreditation are typically activated for ensuring quality standards for the health workforce, clinical practice and facilities, respectively. However, stewards must also create enabling conditions for the system’s actors to translate policies and standards into action while working towards improving health outcomes and well-being. This process requires the use of different mechanisms, such as clinical supervision, quality improvement teams and facility inspections. Further to this, other mechanisms are also needed to assess performance and review the extent to which the original goals of improving health outcomes and well-being have been met. This process relies on the of mechanisms such as patient reported outcome measures, internal or external benchmarking and public reporting. It is this information that fuels learning and overall quality of care improvement by way of feeding back to inform future priorities. Importantly, this cycle applies to each component that makes up the health system – from the inputs such as the workforce and facilities, to the care provided such as clinical and emergency services and the intended users, as the public and patients.

Key to a practical, use-oriented approach to steering the activation of quality of care mechanisms, is an appreciation for the what, when and why: what mechanisms are available for use and have proven to be effective; when along the core processes or sub-functions of governance should they be activated and why – what is their primary focus or target area of the health system which also stands to gain the most from their use. Consultations with countries in the European context coordinated by the World Health Organization (WHO) have affirmed the absence, yet importance, of a taxonomy that addresses these considerations for use and maps policy options towards improved accountability for quality of care [19]. For health system strengthening at pace with health for all targets, frameworks to support evidence-informed decision-making and to catalyse the use of quality of care know-how are urgently needed.

2. Purpose and rationale

In this review, we adopt the perspective of health system stewards and set out with the aim to increase the optimal use of quality of care mechanisms through a policy-relevant and practical approach to health systems strengthening. To do so, we defined three specific objectives:

  • 1.

    to catalogue quality of care mechanisms from systematic reviews and grey literature;

  • 2.

    to plot quality of care mechanisms in a framework crossing the concepts of governance and its subfunctions and health systems thinking; and

  • 3.

    to review best-available evidence on the effectiveness of the mechanisms identified.

To increase the actionability of the framework underpinning these objectives, we reasoned along two dimensions. The first, we refer to as governance sub-functions. We adopt the approach to governance used in Smith et al. 2012 [20] and elaborated from WHO 2000 [21] and Travis et al. 2002 [22] where governance is described by its tasks “of defining the vision and direction of health policy, exerting influence through regulation and advocacy, and collecting and using information” [21].We refer to these tasks (activities, processes) as sub-functions [23] – functions subsidiary to the function of governance itself. Drawing from the literature [24], these processes are described as follows.

Setting priorities and standards refers to the highly political process involving advocacy and negotiation to set priorities and generate policies related to specific goals. To do so, mechanisms aim to convene stakeholders and formulate priorities into policies and standards. Organizing and monitoring action depicts the transition of policy into implementation by way of influencing the performance of multiple actors in order to steer the system towards set goals. Mechanisms in this sub-function aim to equip actors with the tools and resources to align their performance with priorities and standards. Further to implementation is the sub-function of assuring improvements in health outcomes and wellbeing. It depicts the task of generating and using information for learning based on the overall health outcomes associated with practitioners, organizations and the entire health system. Mechanisms in this sub-function aim to create actionable performance intelligence. The further feedback of these findings to set future priorities is the essence of the ‘steering’ function of stewards and has been described as inherent to the successful governance of health systems [20].

The second dimension we explore applies health systems thinking. We refer to it as a recent study describes as targets of quality of care mechanisms [20]. All quality of care mechanisms can be considered to be guided by the pursuit of better health outcomes. Nonetheless, their unit of focus or target varies. Systems thinking acknowledges the individual components of health systems to reveal their unique characteristics [25]. We apply a more delineated approach to the traditional ‘building blocks’ to capture at least three main targets of quality of care mechanisms: the public, as patients and the general public; services delivery, including clinical practice, emergency medicine, laboratory services, management of services; and system resources, including the health workforce, pharmaceuticals, health facilities and health information systems. The financing function, used by stewards to steer the system, is viewed as a governance enabler rather than an a target itself for quality improvement.

We used the resulting matrix combining the sub-functions of governance and targets of quality of care mechanisms as the framework for our review (Fig. 1).

Fig. 1.

Fig. 1

Framework applied to map the use of quality of care mechanisms by health system stewards.

Source: authors’ own.

3. Methods

This work applies the methods of a scoping review [[26], [27], [28]] and was completed in three stages: (1) identifying quality of care mechanisms in systematic reviews and grey literature (2); assessing evidence on the effectiveness of quality of care mechanisms identified by reviewing findings of systematic reviews; and (3) plotting mechanism in the framework defined.

The work was conceived in April 2017 at a meeting organized by the WHO European Centre for Primary Health Care in Almaty, Kazakhstan. The two-day meeting brought together country representatives and quality of care experts from Europe and central Asia. Participants presented and discussed the use of quality of care mechanisms in their countries and respective areas of work. A consolidated list of quality of care mechanisms was presented at the closing of the meeting. Participants concluded the importance of developing a practical framework to improve the use of mechanisms and accountability for quality of care in practice [25].

3.1. Quality of care mechanisms defined

For the purpose of this paper, we define quality of care mechanisms (interventions, strategies or tools) as actionable interventions aimed at reducing quality deficiencies in health systems [29]. The primary aim of a quality of care mechanism is to improve one or more of the six dimensions of quality – safe, effective, patient-centred, timely, efficient and equitable care [30]. Quality of care mechanisms form the arsenal of resources that system stewards can mobilize in their effort to steer the system and bring alignment among actors working towards system goals [24].

Governance mechanisms are also needed to catalyse or enable the conditions for implementation and to hold actors accountable. These mechanisms are characterized in the context of this review as those used to create enabling system conditions to perform and achieve desired system goals. Evidence on the effectiveness of governance mechanisms identified in the literature reviewed has been consolidated. The full range and properties of these mechanisms has been explored elsewhere [23,31,32].

3.2. Process and sources of evidence

The processes of identifying mechanisms, mapping and reviewing evidence for each are described as follows.

3.2.1. Stage one: identifying quality of care mechanisms

Building on the outcomes of the meeting in April 2017, an initial search to identify quality of care mechanisms was undertaken between June and September 2018. Primary searches were run in Health Systems Evidence, with targeted follow-up searches conducted in PubMed. The following MeSH terms were searched to identify a first set of systematic reviews: “quality of health care” “quality assurance” “quality improvement”. In Health Systems Evidence, filters were applied for “governance arrangements” “systematic review of effects” “systematic reviews addressing other questions” and “overviews of systematic reviews”. The search was also limited to reviews published in the past fifteen years. In PubMed, we applied filters for the same fifteen-year timeframe as well as adding “systematic review” to the MeSH terms above. The search strategy is documented in a PRISMA diagram (Fig. 2). In addition to systematic reviews, we reviewed 30 relevant reports from international organizations, such as the European Union, Organisation for Economic Co-operation and Development (OECD) and WHO. The reports were identified based on the knowledge and professional experience of one of the authors (JET). All reports and reviews had to be available in English to be considered. Supplementary file 1 details the full search strategy.

Fig. 2.

Fig. 2

PRISMA diagram.

The assessment of systematic reviews to be included in the evidence summary was conducted by one author (KW), as is consistent with the methodology of a scoping review. Systematic reviews were included if they met the following pre-determined criteria: (1)]were classified as a systematic review (e.g. searched two or more databases with a priori inclusion criteria); (2)2] assessed and clearly described the effectiveness of one or more quality of care mechanism (3); clearly described the effectiveness of single (rather than multi-component) mechanisms; and (4) the primary or secondary outcomes examined in the review related to one or more of the six dimensions of quality [30]. Systematic reviews and relevant reports were used to identify quality improvement mechanisms.

All three authors reviewed the list to determine which mechanisms to include. In order to confirm the uniqueness of each mechanism, a glossary of terms was developed drawing from the literature reviewed (Supplementary file 2). From this, a consolidated, yet dynamic, list of quality of care mechanisms was developed. The list was further adjusted through the stages that follow.

3.2.2. Stage two: mapping quality of care mechanisms

As a next stage, all three authors independently plotted the mechanisms in a preliminary framework. The full range of mechanisms were reviewed in alpha order and assigned an affiliation along the two dimensions (governance sub-functions and targets of quality of care mechanisms). The glossary of terms was the only file consulted in the process (Supplementary file 2). All mappings were consolidated. If there was disagreement for either dimension (approximately 30 %), these mechanisms were discussed in a joint meeting of the authors. Based on this discussion, mechanisms which could apply to more than one cell were marked as such. Definitions that required further specification were revisited in the original source.

The preliminary approach and mapping of quality of care mechanisms were further validated with decision-makers in four countries of the WHO European Region. This validation was conducted through country-specific workshops and key informant interviews between 2017–2018. Each country exercise is reported on as individual country studies: Belarus (2018) [33], Georgia (2017) [34], Kyrgyzstan (2018) [35] and North Macedonia (2018) [36]. In Belarus and Kyrgyzstan, a one-day workshop was organized with approximately 20–30 representatives of key quality actors including the ministry of health, health insurance fund, health care inspectorate and professional associations. In Georgia and North Macedonia, semi-structured interviews with representatives of equivalent profiles were conducted.

Participants and informants were asked to consider for each quality of care mechanism the following: its use in the current system, the actors responsible for the mechanism, and status of use (number of years, regulatory framework, context). Participants were also asked to consider if there were mechanisms missed in the listed set presented, the clarity of definitions provided, and accuracy of the mapping. All comments and discussion points were recorded and resulted in further adjustments to definitions. In a final stage of mapping, the authors further reviewed the clustering and definitions of the framework’s dimensions and conducted a full review of mechanisms on both axes. Further adjustments to the framework were made to fully adopt a steward’s perspective and their tasks to govern the system. Mechanisms may apply to more than one cell, however, to avoid repetition the primary target and contribution to a governance sub-function was prioritized.

3.2.3. Stage three: reviewing evidence on effectiveness

Findings from systematic reviews found in the initial search were extracted using a set template to extract the following information: title of the review; date of last literature search; quality rating (Supplementary file 3: AMSTAR (quality) rating applied); quality mechanisms studied in the systematic review; and key findings on the effectiveness. This process was conducted by one author (KW). Additional targeted searches were then run for each mechanism for which no systematic review was found in the initial searches. Searches were run in Health Systems Evidence and PubMed using the name of the mechanism (and any relevant synonyms) as keywords. These were conducted on an ad-hoc basis up to November 2018 and resulted in the inclusion of an additional 60 systematic reviews. The data extraction for all included systematic reviews can be found in Supplementary file 4 on the effectiveness of quality of care mechanisms reviewed.

Data extraction was then reviewed to determine which mechanisms had sufficient evidence to either support or discount the effectiveness of the mechanism on one or more of the six dimensions of quality. AMSTAR scores were used to support this process in assessing the quality of the review. Table 1 provides details on how the level of evidence was assigned.

Table 1.

Types of evidence statements and the level of evidence required to support the statement.

Evidence statement Level of evidence required
Sufficient review-level evidence to either support or discount the effectiveness of the mechanism on quality
  • Clear and consistent message coming from included high or medium-quality reviews.

  • Reviews included methodologically robust studies.

Tentative review-level evidence to either support or discount the effectiveness of the mechanism on quality
  • A tentative statement from most of the included reviews.

  • Consistent evidence from a small number of reviews containing a small number of studies (of varying quality).

  • Conflicting (or mixed) evidence from one or more reviews included, with the stronger evidence weighted towards one side.

Insufficient review-level evidence to either support or discount the effectiveness of the mechanism on quality
  • A statement of insufficient evidence from an included review.

  • Conflicting (or mixed) evidence from one or more reviews included.

  • No reviews on the mechanism, possibly due to a lack of robust primary studies available.

Source: adapted from [37].

4. Findings

We conducted multiple searches in Health Systems Evidence and in PubMed resulting in 2086 records however, the searches had a significant number of duplicates (n = 783). After removing duplicates, there were 1303 results reviewed for title and abstract inclusion. Ultimately, 129 documents informed both the mapping of the mechanisms and evidence on effectiveness.

4.1. Identifying and mapping quality of care mechanisms

In total, we identified 130 quality of care mechanisms. What constitutes a quality of care mechanism in the literature and initial input from country representatives and experts varied considerably and were referred to as tools, strategies or interventions. In Table 2 all quality of care mechanisms identified are mapped to the dimensions of governance sub-functions on one axis and the target of the mechanism along the other. The table can be read from left to right to examine how the mechanism appears across the processes of setting priorities and standards, organizing for action and assuring health outcomes and wellbeing improvements. It can also be read from top to bottom to examine how the mechanisms map across the target areas. Those mechanisms with sufficient evidence to support their effectiveness in improving quality of care are bolded.

Table 2.

Mapping of quality of care mechanisms identified.

Targets of
quality of care mechanisms
Governance sub-functions
Setting priorities and standards
Process of setting priorities and translating
goals into required standards.
Organizing and monitoring action
Supporting implementation by providing
actors the tools and resources to align
their performance with priorities.
Assuring improvement in outcomes
The process of leveraging data on the
outcomes of care to assure improvements.
People
Mechanisms aimed to ensure the public are supported to meaningfully take control of their health and health services.
Citizens’ panels (juries) [[38], [39], [40]]
Consumer association (groups) [38,39]
Health service ombudsperson [41]
Participation of community representatives in decision-making [38,42,43]
Consumer directed information [44,46,47,48]
Consumer watchdog committee
Patients
Mechanisms aimed to engage patients to actively take part in their own health and health services.
Patient associations (groups)
Patient bill of rights (patient charter)
Shared decision-making (strategy, protocol or standard) [45,49]
Automated alerts and reminders for patients [[50], [51], [52], [53]]
Patient complaint systems
Patient decision-aids (supports) [38,45,53]
Patient education [50,51,[54], [55], [56], [57], [58]]
Patient pathways (care pathway, care map) [53,59]
Peer support groups (peer-to-peer supports) [60,61]
Self-management [[50], [51], [52],62,63]
Patient reported outcome surveys [64,65]
Patient reported experience surveys
Patient satisfaction surveys [66]
Patient feedback system [67]
Clinical practice
Mechanisms aimed to improve clinical practice.
Clinical practice standards [29,68,69]
Clinical protocols and guidelines [44,53,[70], [71], [72], [73], [74]]
Disease registries
Audit and feedback [44,[53], [54], [55],71,[75], [76], [77], [78]]
Clinical decision support systems (incl. alerts and reminders) [[53], [54], [55],57,70,71,77,[79], [80], [81]]
Computerized diagnostics [69]
Delayed prescribing [48]
Discharge planning [82]
Facilitated relay of clinical data to providers [29,50,51]
Structured clinical vignettes
Structured medication review/structured medication reconciliation [70,79,83,84]
Clinical checklists [3,85]
Critical Incident Report/Adverse event reporting [29,86]
Patient safety reporting system
Morbidity and mortality reviews [87]
‘Never event’ reporting
Emergency medicine [ambulance)
Mechanisms aimed to ensure responsive emergency medicine services.
Dispatch protocols [88]
Standardized handover forms [89]
Equipment checklist [88]
Care pathways including transfer pathways [88]
Computerized decision support systems (incl. triage) [88]
Dosing/code cards [90]
Pre-admission patient data sharing [88,91]
Critical incident reporting [88,90]
Laboratory services
Mechanisms aimed to enhance laboratory services.
Accreditation [92]
Certification [92]
International laboratory standards [92]
Internationally recognized labels [92]
Licensure [92]
Patient identifiers and sample identifiers [93]
Sample registry [92]
Standard purchasing or procurement process [92]
Standardized test request forms [92]
Standard operating procedures
Circulation pathways [92]
Equipment inventory list [92]
Equipment maintenance logs [92]
Equipment validation and function checks [92]
Laboratory safety audits [92]
Plan-Do-Study-Act cycles [92]
Risk assessment [92]
Computerized inventory management [[94], [95], [96], [97], [98]]
External quality assessments [92]
Laboratory quality indicators [99]
Management
Mechanisms aimed to ensure quality managerial processes.
Facility performance agreements [57] Failure Modes and Effects Analysis [29]
Quality improvement collaborative across facilities [57,100,101]
Plan-Do-Study-Act [29,102]
Quality improvement teams (quality circles) [73,76,103]
Root Cause Analysis [104]
Benchmarking [57]
External benchmarking [105]
Facility performance indicators
Internal benchmarking
Quality service report cards
Health workforce
Mechanisms aimed to ensure a competent health workforce.
Accreditation of certifying bodies
Accreditation of training schools [106]
Co-regulation [44]
Health professional registry
List (core set) of professional competencies
Professional association (bodies, councils, chambers)
Professional certification
Professional licensing [3]
Professional re-validation
Professional self-regulation [44,107]
Simulation based training (including standardized patients) [70,108,109]
Training and education [3,44,48,49,55,69,70,75,82,103,110,111]
Clinical observation [112,113]
Clinical supervision[44,49,71,107,[113], [114], [115]]
Continuous medical education [49,53,116]
Objective Structured Clinical Examinations [117]
Peer-review teams/committees
Professional re-certification [68]
Task-shifting [114]
Educational outreach [29,77,80]
Team changes [[50], [51], [52],70,111,114]
Multi-source feedback assessment (360° assessment) [118]
Public reporting on health professionals [57,[119], [120], [121], [122], [123]]
Report cards of health professionals [44,54]
Pharmaceuticals and medical products
Mechanisms aimed to promote the use of pharmaceuticals and medical products.
Barcoding of pharmaceuticals [124]
Bulk purchasing [56]
Essential Medicines List [3]
Health Technology Assessment [125]
Medicines formulary [55,56]
Medicines registry
Price controls
Regulation of market entry (or market access) [56]
Medicine authentication system [126,127]
Standardized procurement processes [126]
Medicines appearance checklists [126]
Pre-approval inspection or mandatory inspection of new pharmaceuticals
Computerized prescriber order entry [55,70,79,128]
Pharmacy and treatment committees
Supply chain management [129]
Audit of procurement processes
Pharmacovigilance (system, centres or committees) [126,127]
Health facilities
Mechanisms aimed to ensure adequate health facilities.
Changing facilities physical structures [103]
Facility-based safety protocols [3]
Facility accreditation [44,53,68,76,106,114,[130], [131], [132]]
Facility certification
Facility standards [133]
Permits/permitting
Facility inspections [134] Public reporting on performance by facilities [53,119,120,122,123]
Information systems
Mechanisms aimed to ensure optimal health information systems.
Data protection and confidentiality protocols
Electronic health records [57,135]
Electronic patient registry [50,51]
Unique patient identifier

Note: mechanisms are listed in alphabetical order by cell. Bolded terms denote mechanisms with sufficient evidence for their effectiveness on improving quality of care in the literature reviewed. Quality of care mechanisms without a reference were identified based on the insights and first-hand experiences of workshop participants.

Source: review findings and quality of care mechanisms identified during inception workshop.

4.2. Examining the evidence

A total of 129 overviews and systematic reviews were included in the review of evidence on the effectiveness of quality of care mechanisms identified. The quality of the reviews varied according to the AMSTAR scoring applied. Thirteen mechanisms were found to have a sufficient amount of evidence to support their effectiveness on improving one or more dimensions of quality. The key findings from each systematic review are presented in Supplementary file 4. These mechanisms include:

  • audit and feedback;

  • clinical decision support systems;

  • clinical protocols and guidelines (when accompanied by other mechanisms);

  • clinical supervision;

  • computerized prescriber order entry;

  • continuous medical education;

  • electronic health records

  • equipment checklists (in emergency medicine);

  • patient identification and sample identifiers (in laboratories);

  • quality improvement collaboratives across facilities;

  • simulation based training (including standardized patients);

  • team changes (including development of interdisciplinary teams); and

  • training and education (when educational methods are mixed).

Another 33 mechanisms were found to have some evidence to support their effectiveness in improving quality. However, these mechanisms have been categorized as having “tentative review-level evidence” to support the effectiveness of the mechanism on improving quality of care. Many of these were categorized as such based on the findings from relatively few, low-quality systematic reviews (using AMSTAR rating tool) or a degree of mixed findings between included reviews (see Supplementary file 3 for additional information).

Table 3 maps the 128 quality of care mechanisms including those with sufficient evidence on their effectiveness. The clustering of mechanisms with sufficient evidence is similar to the distribution in Table 2. That is, the greatest number of mechanisms with sufficient evidence were found in relation to the governance sub-function of setting priorities and standards (6 of 58) and for organizing and monitoring action (7 of 50). By target areas, the greatest number of mechanisms with sufficient evidence on effectiveness were found in relation to the health workforce (5 of 24) and clinical practice (3 of 16).

Table 3.

Range of quality of care mechanisms and evidence identified.

graphic file with name fx1.gif

5. Discussion

Over time, the concept of quality of care has evolved from a notion of human error and negligence, with a resulting culture of blame, to the widely accepted understanding that quality is the combined result of system functions and requires a system-wide response [10,13,14,136,137]. Stewards have an integral role in creating conducive environments for such wide-reaching changes. We observe the following four implications of our findings for system stewards in this endeavour.

5.1. A new governance-oriented taxonomy for quality of care mechanisms

The mapping of quality of care mechanisms developed has a number of possible uses. First and foremost, it has the potential to support system stewards to identify the mechanisms that are currently in use in their system through a scan across the targeted areas, such the workforce, health facilities, pharmaceuticals, among the others explored. Further to this, aspects related to the scale of implementation and range of actors involved, for a mapping of existing accountability arrangements, could be pursued. The resulting overview is seen as an integral input to planning national quality of care policies or strategies.

When applied in countries, the taxonomy can aide system stewards to “get up on the balcony” – a metaphor that rightly captures the exercise of gaining an overview of efforts in motion [138]. This overview can ensure stewards are equipped to manoeuvre changes and create the institutional and enabling conditions across the sub-functions of governance that allow for synchronized efforts of the system’s actors. The validation process in countries served as a preliminary test of an approach to map quality of care mechanisms for this purpose. The taxonomy of mechanisms can also serve as an inventory of additional or alternative mechanisms to be assessed and activated based on strategic quality goals.

5.2. A paradigm shift towards a systems-focused and use-oriented approach to quality of care mechanisms

Our mapping confirmed there are unique mechanisms across the two dimensions explored. Nonetheless, there is a wide range of variability in the number of mechanisms by cells. We observe this range in patterns that reflect the development of health systems and quality of care priorities overtime [139,140]. For example, along the target areas of quality of care mechanisms, the greatest range of mechanisms and evidence of effectiveness correspond to long-established priorities of improving clinical practice, ensuring a competent workforce and promoting the responsible use of medicines (Table 3). We find a less diverse range of mechanisms for areas that have received recent yet increasing policy importance, such as empowering people, engaging patients, and managing services.

Similarly, there is clustering of the mechanisms along the sub-functions of governance. Specifically, the greatest number of mechanisms were found to align to setting priorities and standards (Table 3). We interpret this finding in connection to the tradition of prioritizing quality system inputs, in the logic of equipping services delivery with optimal resources, e.g. defining entry-to-practice licensure, certification or registration standards of health care practitioners towards a competent workforce. The second largest range of mechanisms were identified in the scope of organizing for action to translate policies into practice. This finding attests to the fact that priorities and standards alone do not safeguard quality of care. These mechanisms are the tools and resources that enable the capacity of actors to comply with established standards and include, for example: clinical supervision, continuous medical education or peer-review teams (health workforce); care pathways including transfers, and dosing/cards (emergency services); or pharmacy and treatment committees, audit of procurement processes or medicines appearance checklists (pharmaceuticals), among others.

Further to supporting the implementation of priorities and standards are those mechanisms used to generate information on health outcomes. In doing so, these mechanisms facilitate learning and setting future priorities and standards based on identified areas for further improvement. Mechanisms identified include for example, patient-reported experience and outcome surveys, public reporting and report cards of health professionals. The smaller range of mechanisms mapped to this sub-function is found consistent with the field. That is, while the past decade has seen increasing attention to the collection of performance data, this does not immediately translate to the use of collected data for decision-making purposes [6,20].

We hypothesize countries with more developed or mature health systems activate a range of mechanisms along both dimensions of the framework applied. As countries work to improve quality of care and health outcomes, the two-dimensional approach may offer guidance to signal where there are gaps in target areas, and where – along the processes of setting priorities and standards, organizing and monitoring for action and assuring improvement including the important feedback of findings to inform future priorities – there is needed investment for the strategic use of mechanisms. A preliminary exploration of the use of mechanisms was pursued through the country validation process. This confirmed the diverse use of mechanisms as well as the wide range of actors involved and varied scale of implementation, e.g. some mechanisms implemented regionally or on a pilot basis. A comparative study exploring mechanisms in practice across countries may offer further insights into the development of quality of care along the continuum foreseen.

5.3. The evidence-base for mechanisms as a resource for decision-making

Further to a focus on the specific use of mechanisms (regarding their respective targets and alignment to the sub-functions of governance), evidence on their effectiveness is a necessary input for the decision-making of stewards. Research evidence can play an important role in this regard. However, a steward’s need for evidence (i.e. whether to clarify options, understand the benefits or to appreciate implementation considerations) and the relevant type of evidence should be considered as context specific [141].

The evidence review conducted found less than 10 percent of the mechanisms identified have a sufficient amount of evidence to support their effectiveness on improving one or more dimensions of quality. This can be attributed to many mechanisms having been identified in grey literature and the fact that they draw from the list of mechanisms initially generated with country representatives and quality of care experts based on first-hand experiences. The lack of available evidence should not be taken as a judgement on the relative effectiveness of other mechanisms and rather may be representative of the absence of systematic reviews. The lack of evidence at the review-level may also be a result of heterogeneity in primary studies, complicating the ability of researchers to conduct systematic reviews and meta-analyses. It may also be reflective of a lack of research on the effectiveness of single mechanisms (rather than the implementation of a range of complimentary mechanisms) or the broad range of contexts in which these mechanisms are being implemented and studied.

This finding has a number of implications for decision-makers. It means the quality of care mechanisms identified should be considered as inventory rather than a checklist. Moreover, the evidence presented in the review should be considered as a starting point for stewards and can be complemented by local evidence on the benefits, harms and costs of the mechanisms above as well as by the expert opinions of local stakeholders.

There remain significant questions regarding the effectiveness of one component intervention compared to multi-component interventions. While this review only examined the effectiveness of individual mechanisms, findings from a recent high-quality systematic review revealed that the effectiveness of multicomponent interventions is nuanced [5]. Specifically, the review found that the effectiveness of mechanisms was unrelated to the number of components included in the intervention. Therefore, the number of mechanisms combined does not directly improve their effectiveness. Rather, informed decisions on which mechanisms to combine – given the local context of the health system and available evidence – is critical [5].

5.4. Enabling governance conditions

In the search for mechanisms to support improvements in the quality of care, the importance of their implementation in the right environment was a consistent theme across the literature. The right environment was considered to be one in which the governance and financing arrangements of the health system were aligned with the aims of the quality of care mechanisms activated. This finding underscores the importance of ensuring that accountability arrangements are conducive to the implementation of quality of care mechanisms across health system actors and that financial arrangements reward their use and disincentivize low quality of care. The below evidence summarizes the findings from the systematic reviews on the enabling governance conditions for creating the right environment to optimize quality of care mechanisms.

5.4.1. Accountability arrangements

Governance arrangements include changes in the rules or processes that determine authority and accountability for health policies, organizations, commercial products and health professionals, as well as the involvement of stakeholders [121]. Changes to these arrangements usually include adjusting the mandate, accountability and participation of actors to better support the implementation of mechanisms. We found evidence on three accompanying governance arrangements that support alignment with a quality of care agenda and ease the introduction and adoption of quality of care mechanisms. The first was the delegation of decision-making which was found to have mixed effects with one review reporting greater responsiveness to local conditions while another warned that this could be detrimental if greater accountability does not accompany the change [142,143]. The second was the inclusion of diverse stakeholders in policy and organizational decisions, finding that their participation can support improved decisions so long as they are supported with open communication and a non-hierarchical environment [121,142,144]. Finally, community and public engagement was found, when implemented correctly, to improve quality and outcomes as well as potentially helping to support sustained system changes when community buy-in is achieved [142,145].

5.4.3. Financing arrangements

At their simplest, financing arrangements are the ways in which funds flow through the health system. These arrangements can reward or disincentivize providers and individuals to do or behave in particular ways. Financing arrangements can have both a direct and an indirect impact on quality of care. We reviewed the evidence on the effectiveness of co-payments, vouchers, pay-for-performance and value-based purchasing.

With regards to co-payments the evidence generally found that they can be used to signal high-quality care from that lower quality. The reviews suggest that reducing cost-sharing for high-value services, while increasing the out-of-pocket share for low-value services can steer service provision. However, co-payments should be used sparingly given their potential to also reduce the use of effective and life-saving treatments, notably among low-income groups [[146], [147], [148]].

A significant amount of evidence was found that related to pay-for-performance and value-based purchasing. While the literature found relatively few high-quality studies, there is significant evidence to show that both providers and patients respond to financial incentives [83,[149], [150], [151], [152], [153], [154], [155], [156], [157], [158], [159], [160], [161]]. Pay-for-performance schemes appear most effective for improving simple processes or structures of care but have no or largely unknown effects on health outcomes [149,[151], [152], [153],157]. The effects of these incentives appear to be relatively short-lived, with reviews reporting a return to baseline levels at the one year follow-up [153].

Reviews generally agree that pay-for-performance schemes tend to be more effective in ambulatory care than in specialized care and have led to larger effect sizes in reviews on preventive services, however one review notes that this finding may be overemphasized in the literature [152]. Mixed evidence was found for whether the magnitude of the incentive is associated with the effective size, with one review noting no association has been found while another reported that incentives are three times more likely to show a positive effect with larger incentives [152,153]. Most reviews noted that the effectiveness of pay-for-performance schemes are dependent on the context in which they are implemented and the way in which the incentive is designed, with one review noting seven key characteristics that influence the response of health professionals, including: payment rate, sufficiency of payment rate to cover the cost of services, timeliness of payment, payment schedule, performance requirements and accountability [150].

5.5. Limitations

This scoping review has five key limitations that should be acknowledged. The first is that the list of mechanisms is not exhaustive. While significant efforts have been undertaken to ensure a comprehensive list was developed, including undertaking three rounds of searches in two databases as well as reviewing recent reports from international organizations including the OECD, World Bank and WHO, it is unlikely that all mechanisms have been represented. Second, the mapping of each mechanism along the two dimensions of the framework applied was based on how it was described in the literature and on the expert opinion of the authors. Further validation is therefore needed. Third, the classification of mechanisms along both dimensions were at times blurred; in many instances, mechanisms could be used to improve quality of care of multiple targets. To resolve this issue, only the primary aim of each mechanisms has been captured in assigning each mechanism a specific cell. Fourth, there are limitations in the evidence included in the review. For the purpose of mapping individual mechanisms, only evidence examining the effects of single mechanisms was included. However, given that mechanisms are often implemented in parallel to each other (for example, the implementation of clinical guidelines with audit and feedback) there is significantly more evidence available for multi-component interventions than was included in this scoping review. Moreover, there is a significant amount of heterogeneity in primary studies examining the effectiveness of quality of care mechanisms complicating the development of systematic reviews. The decision to focus exclusively on systematic reviews was made both because of the reduction in bias of findings and for the timeliness of this review (which was conducted in six months). However, this decision did mean compromising on the inclusion of findings from primary studies. Finally, despite the focus on empowering people, no explicit searches were made for mechanisms involving families and carers.

6. Conclusion

The link between quality and the attainment of global UHC targets has driven attention to the fundamental importance of enhancing quality of care alongside increasing access to services. To this end, health system stewards have a paramount role in making progress. In this review, we adopted the perspective of system stewards and explored the range and evidence base for quality of care mechanisms. The resulting taxonomy of quality of care mechanisms can serve as a tool to scan the use of mechanisms in a given system, to identify gaps and to provide options for prioritizing action.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and material

All data generated or analysed during this review are included in this published article as a supplementary file.

Funding

This work was carried out with the financial support of the Government of Kazakhstan through the WHO European Centre for Primary Health Care in Almaty, Kazakhstan.

Authors’ contributions

Conception: JET; (II) Literature review and data extraction: KW; (III) Analysis: all; (IV) Drafting: EB, KW; (V) Critical review: JET; (VI) Revisions: all.

Declaration of Competing Interest

The authors declare that they have no competing interests.

Acknowledgements

This paper has been developed following a workshop on quality of care organized by the WHO European Centre for Primary Health Care in April 2017 held in Almaty, Kazakhstan. The work has been shaped by presentations and discussions at this event and the authors recognize the contributions of participants as follows: country representatives from Georgia, Kazakhstan, Kyrgyzstan, Tajikistan, and Ukraine; temporary advisors (alphabetical order according to affiliations at the time) Dinara Aldasheva (WHO consultant); Alexandr Katsaga (WHO consultant); Gulnara Kulkayeva (Almaty Health Department); Joao Sarmento (Escola Nacional de Saude Publica); Silvia Sax (University of Heidelberg); Charles Shaw (WHO consultant); Marat Shoranov (University Medical Center, Kazakhstan); Barton Smith (Edmonds Family Medicine, USA); Sinisa Varga (Croatian Parliament); representatives of development partners, including the Asian Development Bank (Madina Maitenova; Doniyor Mukhannadaliyev), Organisation for Economic Co-operation and Development (Niek Klazinga [virtually]); and representatives from the World Health Organization, including WHO headquarters (Hernan Montenegro; Shams Syed), WHO Regional Office for Europe (Galina Perfilieva; Jane Roberston; Altynai Satylganova), and country offices in Kyrgyzstan (Jarno Habicht; Aigul Sydykova) and Tajikistan (Zulfiya Pirova) as well as staff of the WHO European Centre for Primary Health Care.

We also recognize and thank the following individuals and groups: Abril Campos (Harvard T.H. Chan School of Public Health) for contributions to an earlier version of this review; participants to country-specific workshops in Belarus, Georgia, Kyrgyzstan and North Macedonia; Niek Klazinga and Dionne Kringos (University of Amsterdam) for their helpful comments to an initial draft of the article; and the journal’s reviewers for their constructive feedback.

Footnotes

Appendix A

Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.healthpol.2019.11.006.

Contributor Information

Juan E. Tello, Email: telloj@who.int.

Erica Barbazza, Email: e.s.barbazza@amsterdamumc.nl.

Kerry Waddell, Email: waddellk@mcmaster.ca.

Appendix A. Supplementary data

The following is Supplementary data to this article:

mmc1.docx (309.4KB, docx)

References

  • 1.Hanefeld J., Powell-Jackson T., Baladnova D. Understanding and measuring quality of care: dealing with complexity. Bulletin of the World Health Organization. 2017;95:368–374. doi: 10.2471/BLT.16.179309. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.2018. National Academies of Sciences E, and Medicine. Cross the global quality chasm: improving health care world wide. Washington D.C. [PubMed] [Google Scholar]
  • 3.World Health Organization . 2018. World Bank Group, Organization for Economic Cooperation and Development. Delivering quality health services: a global imperative for universal health coverage. Geneva. [Google Scholar]
  • 4.Kruk M.E., Gage A.D., Arsenault C., Jordan K., Leslie H.H., Roder-DeWan S. High-quality health systems in the Sustainable Development Goals era: time for a revolution. The Lancet Global Health. 2018;6(11) doi: 10.1016/S2214-109X(18)30386-3. e1196-e252. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Rowe A.K., Rowe S.Y., Peters D.H., Holloway K.A., Chalker J., Ross-Degnan D. Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review. The Lancet - Global Health. 2018;6(11):1163–1175. doi: 10.1016/S2214-109X(18)30398-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.World Health Organization Regional Office for Europe; 2019. Improving healthcare quality in Europe: characteristics, effectiveness and implementation of different strategies. [PubMed] [Google Scholar]
  • 7.World Health Organization Regional Office for Europe . 2005. Eight futures forum on governance of patient safety. Copenhagen. [Google Scholar]
  • 8.Shaw C., Kalo I. World Health Organization; Geneva: 2002. A background for national quality policies in health systems. [Google Scholar]
  • 9.Council of Europe . Brussels; 1998. The development and implementation of quality improvement systems (QIS) in health care. Report No.: Recommendation No (97) 17 and explanatory memorandum. [Google Scholar]
  • 10.World Health Organization . 2006. Quality of care: a process for making strategic choices in health systems. Geneva. [Google Scholar]
  • 11.World Health Organization Regional Office for Europe . 2008. Guidance on developing quality and safety strategies with a health system approach. Copenhagen. [Google Scholar]
  • 12.World Health Organization Regional Office for Europe . 2005. What are the advantages and limitations of different quality and safety tools for health care? Copenhagen. [Google Scholar]
  • 13.Institute of Medicine . 2000. To err is human, building a safer health system. Washington D.C. [Google Scholar]
  • 14.Legido-Quigley H., McKee M., Nolte E., Glinos I. 2008. Assuring the quality of health care in the European Union. Copenhagen. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Organization for Economic Cooperation and Development . 2017. Caring for quality in health: lessons learned from 15 reviews in health care quality. Paris. [Google Scholar]
  • 16.Veillard J., Champagne F., Klazinga N., Kazandjian V., Arah O.A., Guisset A.L. A performance assessment framework for hospitals: the WHO regional office for Europe PATH project. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care. 2005;17(6):487–496. doi: 10.1093/intqhc/mzi072. [DOI] [PubMed] [Google Scholar]
  • 17.Kelley E., Hurst J. 2006. Health care quality indicators project - Conceptual framework paper. Paris. [DOI] [PubMed] [Google Scholar]
  • 18.Organization for Economic Cooperation and Development . 2010. Improving value in health care: measuring quality. Paris. [Google Scholar]
  • 19.WHO Regional Office for Europe . 2017. Focusing on quality of care to advance the sustainable development goals in the European region.http://www.euro.who.int/en/health-topics/Health-systems/health-services-delivery/news/news/2017/04/focusing-on-quality-of-care-to-advance-the-sustainable-development-goals-in-the-european-region [Available from: [Google Scholar]
  • 20.Smith P.C., Anell A., Busse R., Crivelli L., Healy J., Lindahl A.K. Leadership and governance in seven developed health systems. Health Policy. 2012;106(1):37–49. doi: 10.1016/j.healthpol.2011.12.009. [DOI] [PubMed] [Google Scholar]
  • 21.World Health Organization . 2000. Health systems: improving performance. Geneva. [Google Scholar]
  • 22.Travis P., Egger D., Davis P., Mechbal A. 2002. Towards better stewardship: concepts and critical issues. Geneva. [Google Scholar]
  • 23.Barbazza E., Tello J.E. A review of health governance: definitions, dimensions and tools to govern. Health Policy. 2014;116(1):1–11. doi: 10.1016/j.healthpol.2014.01.007. [DOI] [PubMed] [Google Scholar]
  • 24.World Health Organization Regional Office for Europe . 2008. Regional committee for Europe: Stewardship/governance of health systems in the european region. Tbilisi; 2008 15 September. [Google Scholar]
  • 25.Adam T., de Savigny D. Systems thinking for strengthening health systems in LMICs: need for a paradigm shift. Health Policy and Planning. 2012;27(Suppl 4):iv1–3. doi: 10.1093/heapol/czs084. [DOI] [PubMed] [Google Scholar]
  • 26.Grant M.J., Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009;26(2):91–108. doi: 10.1111/j.1471-1842.2009.00848.x. [DOI] [PubMed] [Google Scholar]
  • 27.Tricco A.C., Antony J., Zarin W., Strifler L., Ghassemi M., Ivory J. A scoping review of rapid review methods. BMC Medicine. 2015;13(1):224. doi: 10.1186/s12916-015-0465-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.World Health Organization . 2017. Rapid reviews to strengthen health policy and systems: a practical guide. Geneva. [Google Scholar]
  • 29.Hughes R., editor. Patient safety and quality: an evidence-based handbook for nurses Rockville. Agency for Healthcare Research and Quality; U.S: 2008. [PubMed] [Google Scholar]
  • 30.National Academies Press (US); Washington (DC): 2019. Institute of medicine committee on quality of health care in A. Crossing the quality chasm: a new health system for the 21st century. [Google Scholar]
  • 31.Chan B.T.B., Veillard J.H.M., Cowling K., Klazinga N.S., Brown A.D., Leatherman S. 2018. Stewardship of quality of care in health systems: core functions, common pitfalls and potential solutions. USA. [Google Scholar]
  • 32.Brinkerhoff D., Bossert T. 2008. Health governance: concepts, experience and programming options. USA. [Google Scholar]
  • 33.World Health Organization Regional Office for Europe . 2019. Supporting the development of a national strategy for improving quality of care in Belarus Copenhagen: WHO Regional Office for Europe.http://www.euro.who.int/en/countries/belarus/news/news/2019/03/supporting-the-development-of-a-national-strategy-for-improving-quality-of-care-in-belarus [Available from: [Google Scholar]
  • 34.2018. World health organization regional office for Europe. Quality of primary health care in Georgia. Copenhagen. [Google Scholar]
  • 35.2018. World health organization regional office for Europe. Quality of care review in Kyrgyzstan. Copenhagen. [Google Scholar]
  • 36.2019. World Health Organization Regional Office for Europe. Primary health care organization, performance and quality in North Macedonia. Copenhagen. [Google Scholar]
  • 37.Burns P.B., Rohrich R.J., Chung K.C. The levels of evidence and their role in evidence-based medicine. Plastic and Reconstructive Surgery. 2011;128(1):305–310. doi: 10.1097/PRS.0b013e318219c171. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Sarrami-Foroushani P., Travaglia J., Debono D., Braithwaite J. Implementing strategies in consumer and community engagement in health care: results of a large-scale, scoping meta-review. BMC Health Services Research. 2014;14:402. doi: 10.1186/1472-6963-14-402. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Mitton C., Smith N., Peacock S., Evoy B., Abelson J. Public participation in health care priority setting: a scoping review. Health Policy. 2009;91(3):219–228. doi: 10.1016/j.healthpol.2009.01.005. [DOI] [PubMed] [Google Scholar]
  • 40.Street J., Duszynski K., Krawczyk S., Braunack-Mayer A. The use of citizens’ juries in health policy decision-making: a systematic review. Social Science & Medicine (1982) 2014;109:1–9. doi: 10.1016/j.socscimed.2014.03.005. [DOI] [PubMed] [Google Scholar]
  • 41.Wagner C., van der Wal G., Groenewegen P.P., de Bakker D.H. The effectiveness of quality systems in nursing homes: a review. Quality in Health Care: QHC. 2001;10(4):211–217. doi: 10.1136/qhc.0100211... [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Nilsen E.S., Myrhaug H.T., Johansen M., Oliver S., Oxman A.D. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. The Cochrane Database of Systematic Reviews. 2006;(3) doi: 10.1002/14651858.CD004563.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Conklin A., Morris Z., Nolte E. What is the evidence base for public involvement in health-care policy? Results of a systematic scoping review. Health Expectations: an International Journal of Public Participation in Health Care and Health Policy. 2015;18(2):153–165. doi: 10.1111/hex.12038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Dayal P., Hort K. Asia Pacific Observatory on Health Systems and Policies; Manila, Phillipines: 2015. Policy brief: quality of care - what are effective policy options for governments in low- and middle-income countries to improve and regulate the quality of ambulatory care? [Google Scholar]
  • 45.Ovretveit J. 2012. Do changes to patient-provider relationships improve quality and save money? London. [Google Scholar]
  • 46.Kurtzman E.T., Greene J. Effective presentation of health care performance information for consumer decision making: a systematic review. Patient Education and Counseling. 2016;99(1):36–43. doi: 10.1016/j.pec.2015.07.030. [DOI] [PubMed] [Google Scholar]
  • 47.Boudreaux E.D., Cruz B.L., Baumann B.M. The use of performance improvement methods to enhance emergency department patient satisfaction in the United States: a critical review of the literature and suggestions for future research. Academic Emergency Medicine: Official Journal of the Society for Academic Emergency Medicine. 2006;13(7):795–802. doi: 10.1197/j.aem.2006.01.031. [DOI] [PubMed] [Google Scholar]
  • 48.Ranji S., Steinman M., Shojania K., Gonzales R. Interventions to reduce unnecessary antibiotic prescribing: a systematic review and quantitative analysis. Medical Care. 2008;46(8):847–862. doi: 10.1097/MLR.0b013e318178eabd. [DOI] [PubMed] [Google Scholar]
  • 49.Dieleman M., Harnmeijer J. 2006. Improving health worker performance: in search of promising practices. Geneva. [Google Scholar]
  • 50.Ricci-Cabello I. Improving diabetes care in rural areas: a systematic review and meta-analysis of quality improvement interventions in OECD countries. PLoS One. 2013;8(12) doi: 10.1371/journal.pone.0084464. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Shojania K.G., Ranji S.R., McDonald K.M., Grimshaw J.M., Sundaram V., Rushakoff R.J. Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA. 2006;296(4):427–440. doi: 10.1001/jama.296.4.427. [DOI] [PubMed] [Google Scholar]
  • 52.Walsh J.M., McDonald K.M., Shojania K.G., Sundaram V., Nayak S., Lewis R. Quality improvement strategies for hypertension management: a systematic review. Med Care. 2006;44(7):646–657. doi: 10.1097/01.mlr.0000220260.30768.32. [DOI] [PubMed] [Google Scholar]
  • 53.Scott I. What are the most effective strategies for improving quality and safety of health care? Internal Medicine Journal. 2009;39(6):389–400. doi: 10.1111/j.1445-5994.2008.01798.x. [DOI] [PubMed] [Google Scholar]
  • 54.Colla C.H., Mainor A.J., Hargreaves C., Sequist T., Morden N. Interventions aimed at reducing use of low-value health services: a systematic review. Medical Care Research and Review: MCRR. 2017;74(5):507–550. doi: 10.1177/1077558716656970. [DOI] [PubMed] [Google Scholar]
  • 55.Lu C.Y., Ross-Degnan D., Soumerai S.B., Pearson S.A. Interventions designed to improve the quality and efficiency of medication use in managed care: a critical review of the literature – 2001–2007. BMC Health Services Research. 2008;8:75. doi: 10.1186/1472-6963-8-75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Faden L., Vialle-Valentin C., Ross-Degnan D., Wagner A. Active pharmaceutical management strategies of health insurance systems to improve cost-effective use of medicines in low- and middle-income countries: a systematic review of current evidence. Health Policy. 2011;100(2–3):134–143. doi: 10.1016/j.healthpol.2010.10.020. [DOI] [PubMed] [Google Scholar]
  • 57.McMurchy D. 2009. What are the critical attributes and benefits of a high-quality primary healthcare system? Canadian Health Services Research Foundation. Ottawa. [Google Scholar]
  • 58.Ranji S., Steinman M., Shojania K., Sundaram V., Lewis R., Arnolld S. 2006. Volume 4: antibiotic prescribing behavior. Agency for research and healthcare quality Rockville. [PubMed] [Google Scholar]
  • 59.Rotter T., Kugler J., Kock R., Gothe H., Twork S., van Oostrum J. A systematic review and meta-analysis of the effects of clinical pathways on length of stay, hospital costs and patient outcomes. BMC Health Service Research. 2008;8:265. doi: 10.1186/1472-6963-8-265. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Patil S., Ruppar T., Koopman R., Lindbloom E., Elliot S., Mehr D. Peer support interventions for adults with diabetes: a meta-analysis of hemoglobin AIc outcomes. Annals of Family Medicine. 2016;14(6):540–551. doi: 10.1370/afm.1982. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Dale J., Caramlau I., Lindenmeyer A., Williams S. Peer support telephone calls for improving health. Cochrane Database of Systematic Reviews. 2008;4:1–44. doi: 10.1002/14651858.CD006903.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.de Vos M., Graafmans W., Kooistra M., Meijboom B., Van Der Voort P., Westert G. Using quality indicators to improve hospital care: a review of the literature. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care. 2009;21(2):119–129. doi: 10.1093/intqhc/mzn059. [DOI] [PubMed] [Google Scholar]
  • 63.Ryan R., Santesso N., Lowe D., Hill S., Grimshaw J., Prictor M. Interventions to improve safe and effective medicines use by consumers: an overview of systematic reviews. The Cochrane Database of Systematic Reviews. 2014;(4) doi: 10.1002/14651858.CD007768.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Kotronoulas G., Kearney N., Maguire R., Harrow A., Di Domenico D., Croy S. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. Journal of Clinical Oncology: Official Journal of the AMERICAN Society of Clinical Oncology. 2014;32(14):1480–1501. doi: 10.1200/JCO.2013.53.5948. [DOI] [PubMed] [Google Scholar]
  • 65.Almeida R.S., Bourliataux-Lajoinie S., Martins M. Satisfaction measurement instruments for healthcare service users: a systematic review. Cadernos de saude publica. 2015;31(1):11–25. doi: 10.1590/0102-311x00027014. [DOI] [PubMed] [Google Scholar]
  • 66.Ridd M., Shaw A., Lewis G., Salisbury C. The patient-doctor relationship: a synthesis of the qualitative literature on patients’ perspectives. The British Journal of General Practice: the Journal of the Royal College of General Practitioners. 2009;59(561):e116–33. doi: 10.3399/bjgp09X420248. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Cheraghi-Sohi S., Bower P. Can the feedback of patient assessments, brief training, or their combination, improve the interpersonal skills of primary care physicians? A systematic review. BMC Health Services Research. 2008;8:179. doi: 10.1186/1472-6963-8-179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Sutherland K., Leatherman S. The Health Foundation; London: 2006. Regulation and quality improvement: a review of the evidence. [Google Scholar]
  • 69.McDonald K., Matesic B., Contopoulos-Ioannidis D., Lonhart J., Schmidt E., Pineda N. Patient safety strategies targeted at diagnostic error: a systematic review. Annals of Internal Medicine. 2013;158(5):381–389. doi: 10.7326/0003-4819-158-5-201303051-00004. [DOI] [PubMed] [Google Scholar]
  • 70.Manias E., Williams A., Liew D. Interventions to reduce medication errors in adult intensive care: a systematic review. British Journal of Clinical Pharmacology. 2012;74(3):411–423. doi: 10.1111/j.1365-2125.2012.04220.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Willey B., Paintain L., Mangham L., Car J., Amstarong Schellenberg J. EPPI-Centre; London: 2012. Effectiveness of interventions to strengthen national health service delivery on coverage, access, quality and equity in the use of health services in low and lower middle income countries. [Google Scholar]
  • 72.Ament S.M., de Groot J.J., Maessen J.M., Dirksen C.D., van der Weijden T., Kleijnen J. Sustainability of professionals’ adherence to clinical practice guidelines in medical care: a systematic review. BMJ Open. 2015;5(12) doi: 10.1136/bmjopen-2015-008073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.White D.E., Straus S.E., Stelfox H.T., Holroyd-Leduc J.M., Bell C.M., Jackson K. What is the value and impact of quality and safety teams? A scoping review. Implementation Science: IS. 2011;6:97. doi: 10.1186/1748-5908-6-97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Lugtenberg M., Burgers J.S., Westert G.P. Effects of evidence-based clinical practice guidelines on quality of care: a systematic review. Quality & Safety in Health Care. 2009;18(5):385–392. doi: 10.1136/qshc.2008.028043. [DOI] [PubMed] [Google Scholar]
  • 75.Chaudhuri D., Montgomery A., Gulenchyn K., Joseph P. Effectiveness of quality improvement interventions at reducing inappropriate cardiac imaging: a systematic review and meta-analysis. Circulation: Cardiovascular Quality and Outcomes. 2016;9:7–13. doi: 10.1161/CIRCOUTCOMES.115.001836. [DOI] [PubMed] [Google Scholar]
  • 76.Obeirne M., Oelke N.D., Sterling P., Lait J., Zwicker K., Lewanczuk R. 2012. A synthesis of quality improvement and accreditation mechanisms in primary healthcare. Ottawa. [Google Scholar]
  • 77.Sketris I.S., Langille Ingram E.M., Lummis H.L. Strategic opportunities for effective optimal prescribing and medication management. The Canadian Journal of Clinical Pharmacology. 2009;16(1):e103–25. [PubMed] [Google Scholar]
  • 78.Sykes M.J., McAnuff J., Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. International Journal of Nursing Studies. 2018;79:27–35. doi: 10.1016/j.ijnurstu.2017.10.013. [DOI] [PubMed] [Google Scholar]
  • 79.Camire E., Moyen E., Stelfox H.T. Medication errors in critical care: risk factors, prevention and disclosure. Canadian Medical Association Journal. 2009;180(9):936–943. doi: 10.1503/cmaj.080869. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Marcum Z.A., Handler S.M., Wright R., Hanlon J.T. Interventions to improve suboptimal prescribing in nursing homes: a narrative review. The American Journal of Geriatric Pharmacotherapy. 2010;8(3):183–200. doi: 10.1016/j.amjopharm.2010.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81.Robertson J., Walkom E., Pearson S.A., Hains I., Williamsone M., Newby D. The impact of pharmacy computerised clinical decision support on prescribing, clinical and patient outcomes: a systematic review of the literature. The International Journal of Pharmacy Practice. 2010;18(2):69–87. [PubMed] [Google Scholar]
  • 82.Laugaland K., Aese K., Barach P. Interventions to improve patient safety in transitional care - A review o the evidence. Work. 2012;41(Suppl 1):2915–2924. doi: 10.3233/WOR-2012-0544-2915. [DOI] [PubMed] [Google Scholar]
  • 83.Christensen M., Lundh A. Medication review in hospitalised patients to reduce morbidity and mortality. The Cochrane Database of Systematic Reviews. 2016;2 doi: 10.1002/14651858.CD008986.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 84.Hammad E.A., Bale A., Wright D.J., Bhattacharya D. Pharmacy led medicine reconciliation at hospital: a systematic review of effects and costs. Research in Social & Administrative Pharmacy: RSAP. 2017;13(2):300–312. doi: 10.1016/j.sapharm.2016.04.007. [DOI] [PubMed] [Google Scholar]
  • 85.Lyons V., Popejoy L. Meta-analysis of surgical safety checklist effects on teamwork, communication, morbidity, mortality and safety. West Journal of Nursing Research. 2013;36(2):245–261. doi: 10.1177/0193945913505782. [DOI] [PubMed] [Google Scholar]
  • 86.Brunsveld-Reinders A.H., Arbous M.S., De Vos R., De Jonge E. Incident and error reporting systems in intensive care: a systematic review of the literature. International journal for quality in health care: journal of the International Society for Quality in Health Care. 2016;28(1):2–13. doi: 10.1093/intqhc/mzv100. [DOI] [PubMed] [Google Scholar]
  • 87.Bal G., David S., Sellier E., Francois P. Value of morbidity and mortality review conferences for physician education and improvement of care quality and safety: a literature review. Presse Medicale. 2010;39(2):161–168. doi: 10.1016/j.lpm.2009.11.005. [DOI] [PubMed] [Google Scholar]
  • 88.McQueen C., Smyth M., Fisher J., Perkins G. Does the use of dedicated dispatch criteria by Emergency Medical Services optimise appropriate allocation of advanced care resources in cases of high severity trauma? A systematic review. Injury. 2015;46(7):1197–1206. doi: 10.1016/j.injury.2015.03.033. [DOI] [PubMed] [Google Scholar]
  • 89.Bost N., Crilly J., Wallis M., Patterson E., Chaboyer W. Clinical handover of patients arriving by ambulance to the emergency department - a literature review. International Emergency Nursing. 2010;18(4):210–220. doi: 10.1016/j.ienj.2009.11.006. [DOI] [PubMed] [Google Scholar]
  • 90.Bigham B.L., Buick J.E., Brooks S.C., Morrison M., Shojania K.G., Morrison L.J. Patient safety in emergency medical services: a systematic review of the literature. Prehospital Emergency Care. 2012;16(1):20–35. doi: 10.3109/10903127.2011.621045. [DOI] [PubMed] [Google Scholar]
  • 91.Synnot A., Karlsson A., Brichko L., Chee M., Fitzgerald M., Misra M.C. Prehospital notification for major trauma patients requiring emergency hospital transport: a systematic review. Journal of Evidence-Based Medicine. 2017;10(3):212–221. doi: 10.1111/jebm.12256. [DOI] [PubMed] [Google Scholar]
  • 92.World Health Organization . World Health Organization; Geneva: 2011. Laboratory quality management system. [Google Scholar]
  • 93.Snyder S.R., Favoretto A.M., Derzon J.H., Christenson R.H., Kahn S.E., Shaw C.S. Effectiveness of barcoding for reducing patient specimen and laboratory testing identification errors: a Laboratory Medicine best Practices systematic review and meta-analysis. Clinical Biochemistry. 2012;45(13-14):988–998. doi: 10.1016/j.clinbiochem.2012.06.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 94.Fraser H.S., Allen C., Bailey C., Douglas G., Shin S., Blaya J. Information systems for patient follow-up and chronic management of HIV and tuberculosis: a life-saving technology in resource-poor areas. Journal of Medical Internet Research. 2007;9(4):e29. doi: 10.2196/jmir.9.4.e29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 95.Rattanaumpawan P., Boonyasiri A., Vong S., Thamlikitkul V. Systematic review of electronic surveillance of infectious diseases with emphasis on antimicrobial resistance surveillance in resource-limited settings. American Journal of Infection Control. 2018;46(2):139–146. doi: 10.1016/j.ajic.2017.08.006. [DOI] [PubMed] [Google Scholar]
  • 96.Delvaux N., Van Thienen K., Heselmans A., de Velde S.V., Ramaekers D., Aertgeerts B. The effects of computerized clinical decision support systems on laboratory test ordering: a systematic review. Archives of Pathology & Laboratory Medicine. 2017;141(4):585–595. doi: 10.5858/arpa.2016-0115-RA. [DOI] [PubMed] [Google Scholar]
  • 97.Smith M.L., Raab S.S., Fernald D.H., James K.A., Lebin J.A., Grzybicki D.M. Evaluating the connections between primary care practice and clinical laboratory testing: a review of the literature and call for laboratory involvement in the solutions. Archives of Pathology & Laboratory Medicine. 2013;137(1):120–125. doi: 10.5858/arpa.2011-0555-RA. [DOI] [PubMed] [Google Scholar]
  • 98.Fischer S.H., Tjia J., Field T.S. Impact of health information technology interventions to improve medication laboratory monitoring for ambulatory patients: a systematic review. Journal of the American Medical Informatics Association. 2010;17(6):631–636. doi: 10.1136/jamia.2009.000794. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 99.Shahangian S., Snyder S.R. Laboratory medicine quality indicators: a review of the literature. American Journal of Clinical Pathology. 2009;131(3):418–431. doi: 10.1309/AJCPJF8JI4ZLDQUE. [DOI] [PubMed] [Google Scholar]
  • 100.Schouten L., Hulscher M., van Everdingen J., Huijsman R., Grol R. Evidence for the impact of quality improvement collaboratives: systematic review. British Medical Journal. 2008;336:1491–1494. doi: 10.1136/bmj.39570.749884.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 101.Wells S., Tamir O., Gray J., Naidoo D., Bekhit M., Goldmann D. Are quality improvement collaboratives effective? A systematic review. British Medical Journal - Quality & Safety. 2018;27(3):226–240. doi: 10.1136/bmjqs-2017-006926. [DOI] [PubMed] [Google Scholar]
  • 102.Nicolay C.R., Purkayastha S., Greenhalgh A., Benn J., Chaturvedi S., Phillips N. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. The British Journal of Surgery. 2012;99(3):324–335. doi: 10.1002/bjs.7803. [DOI] [PubMed] [Google Scholar]
  • 103.Conry M.C., Humphries N., Morgan K., McGowan Y., Montgomery A., Vedhara K. A 10 year (2000-2010) systematic review of interventions to improve quality of care in hospitals. BMC Health Services Research. 2012;12:275. doi: 10.1186/1472-6963-12-275. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 104.Kurji Z., Premani Z.S., Mithani Y. Review and analysis of quality healthcare system enhancement in developing countries. The Journal of the Pakistan Medical Association. 2015;65(7):776–781. [PubMed] [Google Scholar]
  • 105.Phillips C.B., Pearce C.M., Hall S., Travaglia J., de Lusignan S., Love T. Can clinical governance deliver quality improvement in Australian general practice and primary care? A systematic review of the evidence. The Medical Journal of Australia. 2010;193(10):602–607. doi: 10.5694/j.1326-5377.2010.tb04071.x. [DOI] [PubMed] [Google Scholar]
  • 106.Greenfield D., Braithwaite J. Health sector accreditation research: a systematic review. International journal for Quality in Health Care. 2008;20(3):172–183. doi: 10.1093/intqhc/mzn005. [DOI] [PubMed] [Google Scholar]
  • 107.Dieleman M., Gerretsen B., van der Wilt G. Human resource management interventions to improve health workers’ performance in low and middle income countries: a reality review. Health Research Policy and Systems. 2009;7(7) doi: 10.1186/1478-4505-7-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 108.Murdoch N.L., Bottorff J.L., McCullough D. Simulation education approaches to enhance collaborative healthcare: a best practices review. International Journal of Nursing Education Scholarship. 2014:10. doi: 10.1515/ijnes-2013-0027. [DOI] [PubMed] [Google Scholar]
  • 109.Dilaveri C.A., Szostek J.H., Wang A.T., Cook D.A. Simulation training for breast and pelvic physical examination: a systematic review and meta-analysis. International Journal of Obstetrics and Gynaecology. 2013;120(10):1171–1182. doi: 10.1111/1471-0528.12289. [DOI] [PubMed] [Google Scholar]
  • 110.Patow C., Karpovich K., Reisenberg L., Jaegar J., Rosenfeltd J., Wittenbreer M. Residents’ engagement in quality improvement: a systematic review of the lietrature. Academic Medicine. 2009;84(12):1757–1764. doi: 10.1097/ACM.0b013e3181bf53ab. [DOI] [PubMed] [Google Scholar]
  • 111.McDonald K.M., Sundaram V., Bravata D., Lewis R., Lin N., Kraft S. Agency for Healthcare Research and Quality; Rockville: 2007. Closing the quality gap: a critical analysis of quality improvement strategies. [PubMed] [Google Scholar]
  • 112.Gordon M., Darbyshire D., Baker P. Non-technical skills training to enhance patient safety: a systematic review. Medical Education. 2012;46(11):1042–1054. doi: 10.1111/j.1365-2923.2012.04343.x. [DOI] [PubMed] [Google Scholar]
  • 113.Craig S. Direct observation of clinical practice in emergency medicine education. Academic Emergency Medicine. 2011;18(1):60–67. doi: 10.1111/j.1553-2712.2010.00964.x. [DOI] [PubMed] [Google Scholar]
  • 114.Sharma G., Mathai M., Dickson K.E., Weeks A., Hofmeyr G., Lavender T. Quality care during labour and birth: a multi-country analysis of health system bottlenecks and potential solutions. BMC- Pregnancy and Childbirth. 2015;15(Suppl 2):S2. doi: 10.1186/1471-2393-15-S2-S2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 115.Snowdon D.A., Leggat S.G., Taylor N.F. Does clinical supervision of healthcare professionals improve effectiveness of care and patient experience? A systematic review. BMC Health Services Research. 2017;17(1):786. doi: 10.1186/s12913-017-2739-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 116.Cervero R.M., Gaines J.K. The impact of CME on physician performance and patient health outcomes: an updated synthesis of systematic reviews. The Journal of Continuing Education in the Health Professions. 2015;35(2):131–138. doi: 10.1002/chp.21290. [DOI] [PubMed] [Google Scholar]
  • 117.Setyonugroho W., Kennedy K.M., Kropmans T.J. 2015. Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: a systematic review. Patient Education and Counseling. [DOI] [PubMed] [Google Scholar]
  • 118.Khalifa K., Ansari A., Violato C., Donnon T. Multisource feedback to assess surgical practice: a systematic review. Journal of Surgical Education. 2013;70(4):475–486. doi: 10.1016/j.jsurg.2013.02.002. [DOI] [PubMed] [Google Scholar]
  • 119.Fung C.H., Lim Y.W., Mattke S., Damberg C., Shekelle P.G. Systematic review: the evidence that publishing patient care performance data improves quality of care. Annals of Internal Medicine. 2008;148(2):111–123. doi: 10.7326/0003-4819-148-2-200801150-00006. [DOI] [PubMed] [Google Scholar]
  • 120.Totten A.M., Wagner J., Tiwari A., O’Haire C., Griffin J., Walker M. Closing the quality gap: revisiting the state of the science (vol. 5: public reporting as a quality improvement strategy) Evidence Report/Technology Assessment. 2012;(2085):1–645. [PMC free article] [PubMed] [Google Scholar]
  • 121.Herrera C.A., Lewin S., Paulsen E., Ciapponi A., Opiyo N., Pantoja T. Governance arrangements for health systems in low-income countries: an overview of systematic reviews. The Cochrane Database of Systematic Reviews. 2017;9(9) doi: 10.1002/14651858.CD011083.pub2. CD011085-CD. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 122.Parker C., Schwamm L.H., Fonarow G.C., Smith E.E., Reeves M.J. Stroke quality metrics: systematic reviews of the relationships to patient-centered outcomes and impact of public reporting. Stroke. 2012;43(1):155–162. doi: 10.1161/STROKEAHA.111.635011. [DOI] [PubMed] [Google Scholar]
  • 123.Behrendt K., Groene O. Mechanisms and effects of public reporting of surgeon outcomes: a systematic review of the literature. Health Policy. 2016;120(10):1151–1161. doi: 10.1016/j.healthpol.2016.08.003. [DOI] [PubMed] [Google Scholar]
  • 124.Young J., Slebodnik M., Sands L. Bar code technology and medication administration error. Journal of Patient Safety. 2010;6(2):115–120. doi: 10.1097/PTS.0b013e3181de35f7. [DOI] [PubMed] [Google Scholar]
  • 125.Gagnon M.P., Desmartis M., Poder T., Witteman W. Effects and repercussions of local/hospital-based health technology assessment (HTA): a systematic review. Systematic Reviews. 2014;3:129. doi: 10.1186/2046-4053-3-129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 126.Hamilton W.L., Doyle C., Halliwell-Ewen M., Lambert G. Public health interventions to protect against falsified medicines: a systematic review of international, national and local policies. Health Policy and Planning. 2016;31(10):1448–1466. doi: 10.1093/heapol/czw062. [DOI] [PubMed] [Google Scholar]
  • 127.Fadlallah R., El-Jardali F., Annan F., Azzam H., Akl E.A. Strategies and systems-level interventions to combat or prevent drug counterfeiting: a systematic review of evidence beyond effectiveness. Pharmaceutical Medicine. 2016;30(5):263–276. doi: 10.1007/s40290-016-0156-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 128.Nuckols T.K., Smith-Spangler C., Morton S.C., Asch S.M., Patel V.M., Anderson L.J. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Systematic Reviews. 2014;3:56. doi: 10.1186/2046-4053-3-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 129.Hinrichs S., Jahagirdar D., Miani C., Guerin B., Nolte E. Health Services and Delivery Research. Learning for the NHS on procurement and supply chain management: a rapid evidence assessment. Southampton (UK) NIHR Journals Library. 2019 [PubMed] [Google Scholar]
  • 130.Hastings S.E., Armitage G.D., Mallinson S., Jackson K., Suter E. Exploring the relationship between governance mechanisms in healthcare and health workforce outcomes: a systematic review. BMC Health Services Research. 2014;14:479. doi: 10.1186/1472-6963-14-479. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 131.Brubakk K., Vist G.E., Bukholm G., Barach P., Tjomsland O. A systematic review of hospital accreditation: the challenges of measuring complex intervention effects. BMC Health Services Research. 2015;15:280. doi: 10.1186/s12913-015-0933-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 132.Petit Dit Dariel O., Regnaux J.P. Do Magnet(R)-accredited hospitals show improvements in nurse and patient outcomes compared to non-Magnet hospitals: a systematic review. JBI Database of Systematic Reviews and Implementation Reports. 2015;13(6):168–219. doi: 10.11124/jbisrir-2015-2262. [DOI] [PubMed] [Google Scholar]
  • 133.Berglas N.F. The effect of facility characteristics on patient safety, patient experience, and service availability for procedures in non-hospital-affiliated outpatient settings: a systematic review. PLoS One. 2018;13(1) doi: 10.1371/journal.pone.0190975. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 134.Flodgren G., Gonçalves-Bradley D.C., Pomey M. External inspection of compliance with standards for improved healthcare outcomes. Cochrane Database of Systematic Reviews. 2016;(12) doi: 10.1002/14651858.CD008992.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 135.Campanella P., Lovato E., Marone C., Fallacara L., Mancuso A., Ricciardi W. The impact of electronic health records on healthcare quality: a systematic review and meta-analysis. European Journal of Public Health. 2016;26(1):60–64. doi: 10.1093/eurpub/ckv122. [DOI] [PubMed] [Google Scholar]
  • 136.World Health Organization Regional Office for Europe . 2010. A brief synopsis on patient safety. Copenhagen. [Google Scholar]
  • 137.World Health Organization . 2008. Guidance on developing quality of safety strategies with a health system approach. Copenhagen. [Google Scholar]
  • 138.Heifitz R. Harvard University Press; Cambridge: 1994. Leadership without easy answers. [Google Scholar]
  • 139.van Olmen J., Marchal B., Van Damme W., Kegels G., Hill P.S. Health systems frameworks in their political context: framing divergent agendas. BMC Public Health. 2012;12(1):774. doi: 10.1186/1471-2458-12-774. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 140.World Health Organization . 2009. Systems thinking for health system strengthening. Geneva. [Google Scholar]
  • 141.Lavis J., Wilson M., Grimshaw J.M., Hurley J. 2015. Evidence brief: using financial incentives to achieve health-system goals in Ontario. Hamilton. [Google Scholar]
  • 142.Ciccone D.K., Vian T., Maurer L., Bradley E.H. Linking governance mechanisms to health outcomes: a review of the literature in low- and middle-income countries. Social Science & Medicine (1982) 2014;117:86–95. doi: 10.1016/j.socscimed.2014.07.010. [DOI] [PubMed] [Google Scholar]
  • 143.Dwicaksono A., Fox A.M. Does decentralization improve health system performance and outcomes in low- and middle-income countries? A systematic review of evidence from quantitative studies. The Milbank Quarterly. 2018;96(2):323–368. doi: 10.1111/1468-0009.12327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 144.Turner S., D’Lima D., Hudson E., Morris S., Sheringham J., Swart N. Evidence use in decision-making on introducing innovations: a systematic scoping review with stakeholder feedback. Implementation Science. 2017;12(1):145. doi: 10.1186/s13012-017-0669-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 145.Abelson J., Montesanti S., Li K., Gauvin F., Martin E. 2010. Effective strategies for interactive public engagement in the development of healthcare policies and programs. Ottawa. [Google Scholar]
  • 146.Luiza V.L., Chaves L.A., Silva R.M., Emmerick I.C., Chaves G.C., Fonseca de Araujo S.C. Pharmaceutical policies: effects of cap and co-payment on rational use of medicines. The Cochrane Database of Systematic Reviews. 2015;(5) doi: 10.1002/14651858.CD007017.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 147.Dzakpasu S., Powell-Jackson T., Campbell O.M. Impact of user fees on maternal health service utilization and related health outcomes: a systematic review. Health Policy and Planning. 2014;29(2):137–150. doi: 10.1093/heapol/czs142. [DOI] [PubMed] [Google Scholar]
  • 148.Agarwal R., Gupta A., Fendrick A.M. Value-based insurance design improves medication adherence without an increase in total health care spending. Health Affairs (Project Hope) 2018;37(7):1057–1064. doi: 10.1377/hlthaff.2017.1633. [DOI] [PubMed] [Google Scholar]
  • 149.Yuan B., He L., Meng Q., Jia L. Payment methods for outpatient care facilities. The Cochrane Database of Systematic Reviews. 2017;3 doi: 10.1002/14651858.CD011153.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 150.Kazungu J.S., Barasa E.W., Obadha M., Chuma J. What characteristics of provider payment mechanisms influence health care providers’ behaviour? A literature review. The International Journal of Health Planning and Management. 2018 doi: 10.1002/hpm.2565. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 151.Mehrotra A., Damberg C.L., Sorbero M.E., Teleki S.S. Pay for performance in the hospital setting: what is the state of the evidence? American Journal Of Medical Quality: the Official Journal of the American College of Medical Quality. 2009;24(1):19–28. doi: 10.1177/1062860608326634. [DOI] [PubMed] [Google Scholar]
  • 152.Ogundeji Y.K., Bland J.M., Sheldon T.A. The effectiveness of payment for performance in health care: a meta-analysis and exploration of variation in outcomes. Health Policy. 2016;120(10):1141–1150. doi: 10.1016/j.healthpol.2016.09.002. [DOI] [PubMed] [Google Scholar]
  • 153.Van Herck P., De Smedt D., Annemans L., Remmen R., Rosenthal M.B., Sermeus W. Systematic review: effects, design choices, and context of pay-for-performance in health care. BMC Health Services Research. 2010;10:247. doi: 10.1186/1472-6963-10-247. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 154.Briesacher B.A., Field T.S., Baril J., Gurwitz J.H. Pay-for-performance in nursing homes. Health Care Financing Review. 2009;30(3):1–13. [PMC free article] [PubMed] [Google Scholar]
  • 155.Scott A., Sivey P., Ait Ouakrim D., Willenberg L., Naccarella L., Furler J. The effect of financial incentives on the quality of health care provided by primary care physicians. The Cochrane Database of Systematic Reviews. 2011;(9) doi: 10.1002/14651858.CD008451.pub2. [DOI] [PubMed] [Google Scholar]
  • 156.Witter S., Fretheim A., Kessy F.L., Lindahl A.K. Paying for performance to improve the delivery of health interventions in low- and middle-income countries. The Cochrane Database of Systematic Reviews. 2012;(2) doi: 10.1002/14651858.CD007899.pub2. [DOI] [PubMed] [Google Scholar]
  • 157.Das A., Gopalan S.S., Chandramohan D. Effect of pay for performance to improve quality of maternal and child care in low- and middle-income countries: a systematic review. BMC Public Health. 2016;16:321. doi: 10.1186/s12889-016-2982-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 158.Gillam S., Siriwardena A., Steel N. Pay-for-performance in the United Kingdom: impact of the quality and outcomes framework - A systematic review. Annals of Family Medicine. 2012;10(5):461–468. doi: 10.1370/afm.1377. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 159.Mendelson A., Kondo K., Damberg C., Low A., Motuapuaka M., Freeman M. The effects of pay-for-Performance programs on health, health care use, and processes of care: a systematic review. Annals of Internal Medicine. 2017;166(5):341–353. doi: 10.7326/M16-1881. [DOI] [PubMed] [Google Scholar]
  • 160.Hussey P., Mulcahy A., Schnyer C., Schneider E. Agency for Healthcare Research and Quality; Rockville: 2012. Closing the quality gap: revisiting the state of the science. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 161.Huang J., Yin S., Lin Y., Jiang Q., He Y., Du L. Impact of pay-for-performance on management of diabetes: a systematic review. Journal of Evidence-Based Medicine. 2013;6(3):173–184. doi: 10.1111/jebm.12052. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

mmc1.docx (309.4KB, docx)

Data Availability Statement

All data generated or analysed during this review are included in this published article as a supplementary file.

RESOURCES