Abstract
Background
Quality improvement (QI) initiatives often reflect approaches based on anecdotal evidence, but it is unclear how initiatives can best incorporate scientific literature and methods into the QI process. Review of studies of QI initiatives that aim to systematically incorporate evidence review (termed evidence-based quality improvement (EBQI)) may provide a basis for further methodological development.
Methods
In this scoping review (registration: https://osf.io/hr5bj) of EBQI, we searched the databases PubMed, CINAHL, and SCOPUS. The review addressed three central questions: How is EBQI defined? How is evidence used to inform evidence-informed QI initiatives? What is the effectiveness of EBQI?
Results
We identified 211 publications meeting inclusion criteria. In total, 170 publications explicitly used the term “EBQI.” Published definitions emphasized relying on evidence throughout the QI process. We reviewed a subset of 67 evaluations of QI initiatives in primary care, including both studies that used the term “EBQI” with those that described an evidence-based initiative without using EBQI terminology. The most frequently reported EBQI components included use of evidence to identify previously tested effective QI interventions; engaging stakeholders; iterative intervention development; partnering with frontline clinicians; and data-driven evaluation of the QI intervention. Effectiveness estimates were positive but varied in size in ten studies that provided data on patient health outcomes.
Conclusions
EBQI is a promising strategy for integrating relevant prior scientific findings and methods systematically in the QI process, from the initial developmental phase of the IQ initiative through to its evaluation. Future QI researchers and practitioners can use these findings as the basis for further development of QI initiatives.
Supplementary Information
The online version contains supplementary material available at 10.1007/s11606-022-07602-5.
KEY WORDS: quality improvement, evidence-based, scoping review
BACKGROUND
Evidence-based quality improvement (EBQI) is one of a growing number of strategies used to enhance quality improvement (QI) initiative impacts in clinical practice. EBQI aims to integrate scientific evidence and methods into the QI process while maintaining focus on team-based innovation and problem-solving within real-world settings. Standard healthcare QI approaches focus powerfully on the need for measurement to determine innovation effects, and teams are advised to consult subject matter experts to strengthen their work.1–5 There currently is no standard approach, however, for integrating evidence from relevant pre-existing scientific literature into QI innovation and evaluation. Comprehensive review and critical appraisal of relevant research, for example, is not typically emphasized or conducted.6 In practice, QI teams often use anecdotal evidence alone to shape innovations, and low-validity methods to evaluate them.7 EBQI initiatives, as a subset of all QI initiatives, aim to systematically incorporate pre-existing scientific evidence and methods into the QI process as a core focus.
Given its foundation in applying best evidence and distinct focus on collaboration with the practice, EBQI is increasingly recognized as a valuable approach to structure implementation of advances in healthcare delivery.7 Among other factors, the rapid evolution of partnership improvement initiatives between healthcare organizations and researchers, and the increasing availability of embedded healthcare researchers within healthcare organizations have made EBQI more accessible and attractive to healthcare organizations.3–5
To date, core elements of EBQI have not been well documented, leaving a critical knowledge gap about components of EBQI and how it differs from other QI approaches. In addition, evidence of the effects of employing EBQI has yet to be synthesized. We found no prior systematic reviews of EBQI, and while individual studies have shown promising results8 to our knowledge, EBQI has not been evaluated in an evidence synthesis across studies.
This scoping review explores the EBQI literature. We document how EBQI is defined in publications and aimed to identify key components that characterize this methodology across studies. The review catalogues definitions and characteristics of EBQI as currently used in practice. Particular emphasis was on the definition, scope, and use of evidence, i.e., the core aspect of EBQI. We also examined evidence of effectiveness of EBQI. The scoping review was guided by these review questions10:
Review question 1: How is EBQI defined?
- Review question 2: How is evidence used to inform evidence-informed quality improvement initiatives?
- Review question 2a: How is evidence defined in these initiatives?
- Review question 2b: What are the components of EBQI?
Review question 3: What is the effectiveness of EBQI to promote uptake of evidence-based practices?
Our objective was to conduct a systematic search to identify the available knowledge, provide a clear description of the methodology, and inform further development of methods for incorporating research evidence into QI initiatives.
METHODS
The scoping review followed a detailed review protocol. We followed the steps outlined by Arksey and Malloy: (1) determining the research question; (2) identifying relevant studies; (3) selecting studies; (4) charting the data; and (5) collating, summarizing, and reporting the results.9 In addition, we conducted a consultation exercise to inform and validate findings. The project was deemed exempt by our institutional Human Subject Committee. The protocol was registered in the Open Science Framework and is publicly available.10 The reporting follows PRISMA-ScR, a PRISMA adaptation for scoping reviews.11,12
Search Strategy
The literature searches are documented in the supplemental digital content (SDC). First, a search using the exact terms (“evidence based quality improvement,” “evidence-based quality improvement,” or “EBQI”) was employed to identify publications published to March 2020 that explicitly refer to EBQI in the title, abstract, or keyword of the publication (i.e., the elements that are searchable in research databases). All retrieved publications that used the terminology were included.
Second, we used a broader search strategy aimed at identifying QI initiative evaluations that were not explicitly labeled as EBQI. We assumed that some authors may not use the term “EBQI” even when they have used an evidence-based QI strategy and describe a similar approach in the full-text publication. We applied a string of exclusion criteria to arrive at a manageable sample (see eligibility section), and given the large literature on QI interventions,19 we searched only for studies published between 2017 and 2020.
Sources
We searched PubMed (biomedical literature), CINAHL (nursing and allied health profession literature), and SCOPUS (social sciences). We searched for EBQI publications without date restriction, other QI studies were limited to three years of QI publications as described below in more detail.
Eligibility Criteria
Eligibility criteria were organized in a SPIOS (study design, participants, intervention, outcome, setting) framework; full details are shown in the SDC. Briefly, we applied the following:
- EBQI–labeled publications: All publications using EBQI terminology were included in the data abstraction.
- Primary care effectiveness subsample: Among EBQI publications, we identified studies reporting effectiveness results for the evaluation of an EBQI initiative. Studies had to report on patient health, and we restricted to primary care to identify a more homogenous sample of research studies.
EBQI–compatible studies: Empirical studies involving U.S. healthcare professionals, reporting on an evaluation of a QI initiative in primary care, and documenting evidence review as part of their methodology to select, design, or implement a QI intervention. Evidence review was defined as a literature review undertaken at the beginning of the project, documentation of locally generated data to determine the need for the intervention (practice-based evidence), and/or utilizing of authoritative sources such as evidence-based clinical practice guidelines. Two independent literature reviewers screened citations and full-text publications; discrepancies were resolved through discussion in the team. Reviewers first excluded all citations that did not indicate an empirical evaluation of a QI initiative. The remaining citations were screened as full-text publication, applying all eligibility criteria described in the SDC (e.g., U.S.-based).
Data Abstraction and Synthesis
Data abstraction was tailored to the review questions. We used ten features in total to characterize the included studies (described in more detail in the SDC):
Evidence to identify target: using evidence (data) to identify the target of the QI initiative
Iterative: conducting an interactive process for selecting the QI intervention
Engagement of stakeholders: reaching out within the organization to ensure a collaborative process
Evidence to identify intervention: reviewing evidence (research literature or local data) to select effective QI interventions
QI facilitation: use of facilitation of the QI process
Leadership involvement: involving clinical operations leadership in the QI initiative
Priority setting with leadership: setting priorities for the QI initiative together with clinical operations leadership
Frontline engagement: engaging frontline personnel early in the QI initiative
Evidence to determine success: using data to determine the success of the QI initiative
Analytic support: using analytic support to help QI teams
The abstraction domains had been developed by the QI content expert team members drawing on practical and research expertise (SH, ST, BT). The information was collated in evidence tables and component tables allow a concise overview. Effectiveness outcomes were summarized in a random-effects meta-analysis.
Expert Consultation
The preliminary scoping review results were presented to Dr. Lisa Rubenstein, a proponent and conceptual originator of EBQI. The formal consultation step aimed to ensure that the review addresses the right questions, identified all relevant literature, and synthesized the included material appropriately. Dr. Rubenstein was not involved in the planning of the review and assessed methods and results de novo. The consultation exercise resulted in one additional domain (priority setting with leadership) that was added to the data abstraction (see SDC).
RESULTS
The literature searches identified 2001 citations. Of these, we obtained 496 for full-text inclusion screening. Figure 1 shows the flow diagram.
We included 211 publications, detailed in the evidence tables in the SDC. In total, we identified 170 diverse publications that used the term EBQI. SDC Figure 1 plots the number of EBQI publications over time and shows the rapid increase in frequency and popularity of EBQI. Two peaks emerged, one around 2006–2008, the other after 2016. The 170 identified publications are described in detail in an evidence table in the appendix (see SDC Table 1) and were used to address review question 1.
Review Question 1 Synthesis: How Is EBQI Defined?
The majority of EBQI–labeled publications did not define EBQI; only 23 of the 170 studies provided a definition or detailed description of the EBQI process. Studies highlighted different aspects of EBQI such as stakeholder engagement13 or described EBQI broadly as a continuous quality improvement method.14 Rubenstein et al.15 defined EBQI as “a continuous quality improvement approach whose goal is translation of research on care delivery models into routine practice.” Figure 2 shows the terms used in the identified publications.
Review Question 2 Synthesis: How Is Evidence Used to Inform Evidence-Informed Quality Improvement Initiatives?
The second evidence table (SDC Table 2) shows all 25 EBQI–labeled studies that reported on an evaluation of a QI initiative (listed first), followed by the 42 EBQI–compatible primary care evaluations, for a total of 67 EBQI–labeled or EBQI–compatible studies. The table shows the wide range of clinical topic areas and interventions addressed and describes their implementation strategy in detail. Across studies, most used published research literature to select interventions to be implemented in the QI initiative.
Review Question 2a Synthesis: How Is Evidence Defined?
In the 25 EBQI–labeled evaluations, 17 studies that provided information on the utilized evidence referred to published literature identified in a literature review. Ten EBQI studies referred to the use of local data. Six studies used expert panels and consensus meetings. Six studies referred to clinical practice guidelines that were reviewed to identify the QI intervention. Studies used these sources either alone or in combination.
Review Question 2b Synthesis: What Are the Components of EBQI?
Table 1 shows the 10 potential EBQI features that we abstracted for each study, the number of features characterizing each study, and the overall frequency of features across studies. EBQI–labeled studies (top half of Table 1) are followed by EBQI–compatible studies (bottom half of Table 1). Table 2 provides a summary of features across all 67 studies. Across studies, two thirds of studies reported having used evidence to identify an effective intervention, engaging stakeholders, using an iterative development, and involving frontline clinicians. In addition, all 67 identified studies used data to determine the success of the QI initiative.
Table 1.
ID | Evidence to identify target | Iterative | Stakeholder engagement | Evidence to identify intervention | QI facilitation | Leadership involvement | Priority setting with leadership | Frontline engagement | Evidence to determine success | Analytic support |
Total criteria met within study (N
= 10) (%) |
---|---|---|---|---|---|---|---|---|---|---|---|
EBQI studies | |||||||||||
Badru, 201720 | Yes | Yes | Yes | Yes | No | No | No | Yes | Yes | Yes |
7 (70%) |
Bennett, 201621 | Yes | Yes | Yes | Yes | No | Yes | No | Yes | Yes | No |
7 (70%) |
Chaney, 201114 | No | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes |
8 (80%) |
Cohen, 201322 | Yes | Yes | Yes | No | Yes | No | No | Yes | Yes | No |
6 (60%) |
Dumphy, 201616 | Yes | No | No | Yes | No | No | No | No | Yes | No |
3 (30%) |
Fortney, 201223 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
10 (100%) |
Fortney, 201324 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
9 (90%) |
Fox, 201625 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
9 (90%) |
Gadbois, 201626 | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No |
8 (80%) |
Gottlieb, 201817 | Yes | Yes | No | Yes | No | Yes | Yes | No | Yes | No |
6 (60%) |
Klause, 202027 | Yes | Yes | No | Yes | No | No | No | Yes | Yes | No |
5 (50%) |
Le Flore, 201728 | Yes | Yes | Yes | Yes | Yes | Yes | No | No | Yes | No |
7 (70%) |
Meredith, 201829 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
9 (90%) |
Ong, 201730 | Yes | Yes | Yes | No | No | No | No | Yes | Yes | No |
5 (50%) |
Rizzo, 201831 | Yes | No | Yes | Yes | No | No | No | Yes | Yes | No |
5 (50%) |
Rubenstein, 200632 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
9 (90%) |
Rubenstein, 201015 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
10 (100%) |
Sherman, 200433 | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | No |
8 (80%) |
Starkey, 201634 | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | No |
8 (80%) |
Walker, 201935 | No | No | Yes | Yes | No | No | No | Yes | Yes | No |
5 (50%) |
Walker-Smith, 202036 | Yes | Yes | Yes | Yes | No | No | No | Yes | Yes | No |
7 (70%) |
Whitten, 201337 | Yes | No | No | Yes | No | No | No | No | Yes | No |
3 (30%) |
Yano, 200813 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
8 (80%) |
Yoon, 201638 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
10 (100%) |
Young, 201839 | Yes | No | No | Yes | No | No | No | No | Yes | No |
3 (30%) |
Total studies meeting EBQI criteria (N = 25) N (%) |
18 (72%) |
20 (80%) |
20 (80%) |
23 (92%) |
12 (48%) |
14 (56%) |
11 (44%) |
20 (80%) |
25 (100%) |
9 (36%) |
N/A |
EBQI–compatible studies | |||||||||||
Barclay, 201940 | No | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No |
7 (70%) |
Bowen, 202041 | No | Yes | No | Yes | Yes | No | No | No | Yes | No |
4 (40%) |
Breaux, Shropshire, 201742 | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | No |
7 (70%) |
Brodie, 201843 | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes |
9 (90%) |
Burge, 201944 | Yes | Yes | Yes | No | No | Yes | No | Yes | Yes | No |
5 (50%) |
Buschkoetter, 201945 | No | Yes | No | Yes | No | No | No | No | Yes | No |
3 (30%) |
Camp, 201746 | Yes | No | No | Yes | No | No | No | No | Yes | No |
3 (30%) |
Campbell, 201747 | No | Yes | No | No | No | No | No | No | Yes | No |
2 (20%) |
Colborn, 201948 | No | Yes | Yes | Yes | No | No | No | No | Yes | No |
4 (40%) |
Daaleman, 201849 | No | Yes | Yes | No | Yes | Yes | No | No | Yes | No |
5 (50%) |
Fabre, 202050 | No | No | No | Yes | Yes | Yes | No | No | Yes | No |
4 (40%) |
Fisher-Borne, 201851 | No | No | Yes | Yes | No | No | No | Yes | Yes | No |
4 (40%) |
Fortney, 201852 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
10 (100%) |
Garza, 201753 | No | Yes | No | Yes | No | No | No | No | Yes | No |
3 (30%) |
Gold, 201754 | Yes | Yes | Yes | Yes | No | Yes | No | Yes | Yes | Yes |
8 (80%) |
Green, 201755 | Yes | Yes | Yes | Yes | No | No | Yes | Yes | Yes | Yes |
9 (90%) |
Hanlin, 201856 | No | Yes | Yes | Yes | No | No | No | No | Yes | No |
4 (40%) |
Hawk, 201757 | No | No | Yes | Yes | No | No | No | Yes | Yes | No |
4 (40%) |
Jonas, 201758 | No | Yes | No | Yes | No | No | No | No | Yes | No |
3 (30%) |
Knierim, 201959 | No | No | No | Yes | Yes | No | No | No | Yes | Yes |
4 (40%) |
Lu, 201960 | Yes | No | Yes | No | No | No | No | Yes | Yes | No |
4 (40%) |
Makelarski, 201961 | No | No | Yes | Yes | Yes | Yes | No | Yes | Yes | No |
6 (60%) |
Minsky, 201762 | Yes | No | Yes | No | No | No | No | Yes | Yes | No |
4 (40%) |
Modica, 201963 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
9 (90%) |
Nagykaldi, 201764 | No | No | Yes | Yes | Yes | No | Yes | No | Yes | Yes |
6 (60%) |
Nowalk, 201765 | No | No | Yes | Yes | No | No | No | Yes | Yes | No |
4 (40%) |
Ober, 201766 | No | Yes | Yes | Yes | Yes | Yes | No | No | Yes | No |
6 (60%) |
Quanbeck, 201867 | No | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes |
8 (80%) |
Regan, 201768 | Yes | No | No | Yes | Yes | No | No | No | Yes | Yes |
5 (50%) |
Richards, 201969 | No | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No |
7 (70%) |
Roderick, 201770 | No | Yes | Yes | Yes | Yes | No | No | No | Yes | Yes |
6 (60%) |
Savas, 201971 | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | No |
8 (80%) |
Schaeffer, 201972 | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | No |
8 (80%) |
Schiff, 201773 | Yes | Yes | Yes | No | Yes | Yes | No | No | Yes | No |
6 (60%) |
Schurman, 201774 | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | No |
7 (70%) |
Senger, 201875 | Yes | No | Yes | Yes | No | Yes | Yes | Yes | Yes | No |
7 (70%) |
Shah, 201976 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
10 (100%) |
Sloand, 201977 | No | Yes | No | Yes | No | Yes | No | Yes | Yes | No |
5 (50%) |
van Eeghen, 202078 | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | No |
7 (70%) |
Weiner, 201779 | No | No | Yes | Yes | Yes | No | Yes | No | Yes | No |
5 (50%) |
Williams, 2018 80 | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | No |
7 (70%) |
Yusupov, 201981 | Yes | Yes | No | Yes | No | No | No | No | Yes | No |
4 (40%) |
Total studies meeting EBQI–compatible criteria (N = 42) N (%) |
18 (43%) |
29 (69%) |
31 (74%) |
36 (86%) |
23 (55%) |
17 (41%) |
9 (21%) |
24 (57%) |
42 (100%) |
11 (26%) |
N/A |
Note: Evidence to identify target: using data to identify the target of the QI intervention; Iterative: iterative and interactive process for selecting the intervention within the discussion; Engagement of stakeholders: reaching out to stakeholders within the organization in a collaborative process; Evidence to identify intervention: literature review to identify effective interventions in the research literature; QI facilitation: quality improvement facilitation may refer to an external facilitator, internal QI coordinator, or learning collaborative; Leadership involvement: involvement of organizational leadership beyond one-time approval or briefing at the end; Priority setting with leadership: organizational leadership was involved in prioritizing the target of the QI intervention; Frontline engagement: engagement of frontline personnel from the start, not only after the process change was decided; Evidence to determine success: use of evidence to determine the effect of the intervention; Analytic support: support from a dedicated statistician or analyst beyond of the clinical team
Table 2.
Components | All studies meeting criteria (N = 67) N (%) |
EBQI studies (N = 25) N (%) |
EBQI–compatible studies (N = 42) N (%) |
---|---|---|---|
Evidence to identify target | 36 (54%) | 18 (72%) | 18 (43%) |
Iterative development | 49 (73%) | 20 (80%) | 29 (69%) |
Engagement of stakeholders | 51 (76%) | 20 (80%) | 31 (74%) |
Evidence to identify intervention | 59 (88%) | 23 (92%) | 36 (86%) |
QI facilitation | 35 (52%) | 12 (48%) | 23 (55%) |
Leadership involvement | 31 (46%) | 14 (56%) | 17 (41%) |
Priority setting with leadership | 19 (28% | 11 (44%) | 9 (21%) |
Frontline engagement | 44 (66%) | 20 (80%) | 24 (57%) |
Evidence to determine success | 67 (100%) | 25 (100%) | 42 (100%) |
Analytical support | 20 (30%) | 9 (36%) | 11 (26%) |
Note: Evidence to identify target: using data to identify the target of the QI intervention; Iterative: iterative and interactive process for selecting the intervention within the discussion; Engagement of stakeholders: reaching out to stakeholders within the organization in a collaborative process; Evidence to identify intervention: literature review to identify effective interventions in the research literature; QI facilitation: quality improvement facilitation may refer to an external facilitator, internal QI coordinator, or learning collaborative; Leadership involvement: involvement of organizational leadership beyond one-time approval or briefing at the end; Priority setting with leadership: organizational leadership was involved in prioritizing the target of the QI intervention; Frontline engagement: engagement of frontline personnel from the start, not only after the process change was decided; Evidence to determine success: use of evidence to determine the effect of the intervention; Analytic support: support from a dedicated statistician or analyst beyond the clinical team
When we compared features across the subsets, evidence to identify the target of the quality improvement intervention was more frequently reported in EBQI–labeled studies than in EBQI–compatible studies (72% vs 43%). Across EBQI–labeled and EBQI–compatible studies, involvement of leadership in priority setting for the quality improvement target (44% vs 21%) and the provision of analytic support (36% vs 26%) were least frequently reported. EBQI studies consistently reported more EBQI features: the median number of components used within study was 7 for the EBQI–labeled sample (maximum of 10) and 5.5 for the EBQI–compatible sample. The distributions in the two sets differed statistically significantly (p = 0.037; Mann-Whitney U test).
Review Question 3 Synthesis: EBQI Effectiveness
We abstracted data from all 14 evaluations of primary care QI initiatives that used the term EBQI and that reported on a patient health outcome (SDC Table 3). Not all studies provided sufficient detail to allow effect size calculation. None of the studies compared two quality improvement strategies in a head-to-head comparison; hence, the documented effectiveness represents the effectiveness of the combined EBQI and implemented intervention. The forest plot in Figure 3 shows effect estimates for four studies reporting categorical outcomes, expressed as relative risk (RR), that could be combined in a meta-analysis. Studies assessed the implementation of a breast-feeding protocol in primary care,16 an intervention targeting primary care referrals to smoking cessation clinics,13 the implementation of collaborative care for depression,14 and a program to increase adherence to immunization guidelines for adults with diabetes.17 The effectiveness estimates varied widely by quality improvement target and study, only one of the studies reported a statistically significant effect, but all suggested more improvements in the EBQI group.
DISCUSSION
The scoping review shows that the evidence base for EBQI is growing, and to our knowledge, this is the first study that provides an overview of the available EBQI literature.
We identified EBQI components and their relative frequency, both across EBQI–labeled studies and in comparison to studies that were similar in approach to EBQI without using EBQI terminology. The focus on evidence at multiple stages of the QI initiative and the strong emphasis on engaging stakeholders were key features.
However, “evidence” was often not systematically described in the identified studies. Not all studies reported a review of the evidence to identify a target for the QI initiative (54% across EBQI–labeled and EBQI–compatible studies). This gap calls into question the focus of these studies on using evidence to identify and define QI aims, a critical entry point for introducing evidence into the QI process. Most, but not all (88%) of the studies reviewed evidence to select and shape the QI intervention design, another critical entry point for applying published research, local data, and implementation science knowledge. More complete reporting on evidence use across studies would promote assessment of fidelity to the EBQI process, which is critical to evaluation of the success of the QI initiative and our ability to learn from initiatives across settings.18
Our review also shows that overall, there is still insufficient information regarding the effectiveness of EBQI. We only found a small number of studies using EBQI that reported on key and patient-centered outcomes, i.e., patient health, and studies addressed substantially different intervention targets, ranging from breast-feeding to depression treatment. We did not find studies that compared EBQI with other quality improvement strategies in head-to-head comparisons; hence, the effect of EBQI in the included studies was invariably confounded with the QI content. It is not known yet how EBQI compares to other quality improvement strategies, in particular quality improvement interventions that are based on anecdotal evidence. Future research should evaluate the comparative effectiveness of EBQI to provide more information on this critical aspect.
Our review has several limitations. While we systematically identified all known EBQI publications, we sampled the literature for EBQI–compatible studies and restricted to those published in recent years and limited to primary care given the large QI literature.19 The sampling strategy was chosen to obtain a systematic and pragmatic sample that would serve as an exemplar of EBQI–compatible studies. However, it should be noted that earlier approaches were not included, which undoubtedly left out important approaches, and EBQI–compatible approaches in other fields, such as improvements in hospitals in international settings, could have provided additional important information.
We show that EBQI is a promising and growing strategy that aims to integrate prior scientific findings and methods into QI initiatives. Commonly used EBQI features integrate evidence throughout the improvement process, from the initial developmental phase of the QI initiative through to its evaluation. Future research should clearly document EBQI processes to enable better characterization of core initiative features and should assess the comparative effectiveness and success in addressing patient-centered goals.
Supplementary Information
Acknowledgements
The study was funded by the Department of Veteran Affairs. The findings are those of the authors and do not necessarily represent the views of the Department of Veteran Affairs or the United States Government.
Declarations
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Footnotes
The paper abstract has been presented for “Poster Session: Clinical Care Settings: System-level Interventions” at the Academy Health Virtual D&I Conference, 2020.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Institute for Healthcare Improvement Website http://www.ihi.org/ihi. Accessed 5/2/2022.
- 2.Deming WE. Quality, Productivity and Competitive Position. Cambridge, MA: MIT Center for Advanced Engineering Studies; 1982. [Google Scholar]
- 3.Meyers D, Miller T, Genevro J, Zhan C, De La Mare J, Fournier A, et al. EvidenceNOW: balancing primary care implementation and implementation research. Ann Fam Med. 2018;16(Suppl 1):S5–s11. doi: 10.1370/afm.2196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nelson K, Reddy A, Stockdale SE, Rose D, Fihn S, Rosland A-M, et al., editors. The Primary Care Analytics Team: Integrating Research and Clinical Care Within the Veterans Health Administration Office of Primary Care. Healthcare; 2021: Elsevier. [DOI] [PubMed]
- 5.Ovretveit J, Hempel S, Magnabosco JL, Mittman BS, Rubenstein LV, Ganz DA. Guidance for research-practice partnerships (R-PPs) and collaborative research. J Health Manag. 2014;28(1):115-26. 10.1108/JHOM-08-2013-0164 [DOI] [PubMed]
- 6.Melnyk BM, Buck J, Gallagher-Ford L. Transforming quality improvement into evidence-based quality improvement: a key solution to improve healthcare outcomes. Worldviews Evid.-Based Nurs. 2015;12(5):251–2. doi: 10.1111/wvn.12112. [DOI] [PubMed] [Google Scholar]
- 7.Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff (Millwood). 2005;24(1):138–50. doi: 10.1377/hlthaff.24.1.138. [DOI] [PubMed] [Google Scholar]
- 8.Stockdale SE, Zuchowski J, Rubenstein LV, Sapir N, Yano EM, Altman L, et al. Fostering evidence-based quality improvement for patient-centered medical homes: initiating local quality councils to transform primary care. Health Care Manag Rev. 2018;43(2):168–80. doi: 10.1097/hmr.0000000000000138. [DOI] [PubMed] [Google Scholar]
- 9.Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32. doi: 10.1080/1364557032000119616. [DOI] [Google Scholar]
- 10.Hempel S, Bolshakova M, Turner B, Rose D, Dinalo J, Motala A, et al. Scoping Review Protocol: Evidence-Based Quality Improvement as an Implementation Strategy for Evidence-Based Practices. 2020. https://osf.io/hr5bj/. Accessed 5/2/2022.
- 11.Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73. doi: 10.7326/M18-0850. [DOI] [PubMed] [Google Scholar]
- 12.Ottawa Hospital Research Institute, University of Oxford. PRISMA for Scoping Reviews. 2015. http://www.prisma-statement.org/Extensions/ScopingReviews. Accessed 27 July 2020.
- 13.Yano EM, Rubenstein LV, Farmer MM, Chernof BA, Mittman BS, Lanto AB, et al. Targeting primary care referrals to smoking cessation clinics does not improve quit rates: implementing evidence-based interventions into practice. Health Serv Res. 2008;43(5 Pt 1):1637–61. doi: 10.1111/j.1475-6773.2008.00865.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Chaney EF, Rubenstein LV, Liu CF, Yano EM, Bolkan C, Lee M, et al. Implementing collaborative care for depression treatment in primary care: a cluster randomized evaluation of a quality improvement practice redesign. Implement Sci. 2011;6:121. doi: 10.1186/1748-5908-6-121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Rubenstein LV, Chaney EF, Ober S, Felker B, Sherman SE, Lanto A, et al. Using evidence-based quality improvement methods for translating depression collaborative care research into practice. Fam Syst Health. 2010;28(2):91–113. doi: 10.1037/a0020302. [DOI] [PubMed] [Google Scholar]
- 16.Dumphy D, Thompson J, Clark M. A breastfeeding quality improvement project in rural primary care. J Hum Lac. 2016;32(4):633–41. doi: 10.1177/0890334416662240. [DOI] [PubMed] [Google Scholar]
- 17.Gottlieb RP, Dols JD. Improving vaccination rates in adults with type 2 diabetes in a family practice setting through the use of evidence-based interventions. J Doctoral Nurs Pract. 2018;11(2):151–9. doi: 10.1891/2380-9418.11.2.151. [DOI] [PubMed] [Google Scholar]
- 18.Stockdale SE, Hamilton AB, Bergman AA, Rose DE, Giannitrapani KF, Dresselhaus TR, et al. Assessing fidelity to evidence-based quality improvement as an implementation strategy for patient-centered medical home transformation in the Veterans Health Administration. Implement Sci. 2020;15(1):18. doi: 10.1186/s13012-020-0979-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Hempel S, Rubenstein LV, Shanman RM, Foy R, Golder S, Danz M, et al. Identifying quality improvement intervention publications--a comparison of electronic search strategies. Implement Sci. 2011;6:85. doi: 10.1186/1748-5908-6-85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Badru M. A Clinical Practice Guideline to Reduce Behavioral Outbursts in Veterans with Posttraumatic Stress Disorder. Clinical Practice Guideline to Reduce Behavioral Outbursts in Veterans with Posttraumatic Stress Disorder. 2017:1-.
- 21.Bennett JG. Implementing Lipid Screening Guidelines for Children in a Rural Health Clinic. Implementing Lipid Screening Guidelines for Children in a Rural Health Clinic. 2016:1-.
- 22.Cohen AN, Chinman MJ, Hamilton AB, Whelan F, Young AS. Using patient-facing kiosks to support quality improvement at mental health clinics. Med Care. 2013;51(3 Suppl 1):S13–20. doi: 10.1097/MLR.0b013e31827da859. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, et al. Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Implement Sci. 2012;7:30. doi: 10.1186/1748-5908-7-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Fortney JC, Enderle MA, Clothier JL, Otero JM, Williams JS, Pyne JM. Population level effectiveness of implementing collaborative care management for depression. Gen Hosp Psychiatry. 2013;35(5):455–60. doi: 10.1016/j.genhosppsych.2013.04.010. [DOI] [PubMed] [Google Scholar]
- 25.Fox AB, Hamilton AB, Frayne SM, Wiltsey-Stirman S, Bean-Mayberry B, Carney D, et al. Effectiveness of an evidence-based quality improvement approach to cultural competence training: the Veterans Affairs’ “Caring for Women Veterans” Program. J Contin Educ Health Prof. 2016;36(2):96–103. doi: 10.1097/ceh.0000000000000073. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Gadbois C, Chin ED, Dalphonse L. Health promotion in an opioid treatment program an evidence-based nursing quality improvement project. J Addict Nurs. 2016;27(2):127–42. doi: 10.1097/JAN.0000000000000124. [DOI] [PubMed] [Google Scholar]
- 27.Klause KT, Dodds VA, Selleck C, Deupree JP. Addressing intimate partner violence at a safety-net clinic for adults. J Nurs Pract. 2020;16(2):154–7. doi: 10.1016/j.nurpra.2019.08.019. [DOI] [Google Scholar]
- 28.Le Flore G. Applying Clinical Guidelines to Curtail Opioid Overprescribing in Primary Care. Applying Clinical Guidelines to Curtail Opioid Overprescribing in Primary Care. 2017:1-.
- 29.Meredith LS, Batorsky B, Cefalu M, Darling JE, Stockdale SE, Yano EM, et al. Long-term impact of evidence-based quality improvement for facilitating medical home implementation on primary care health professional morale. BMC Fam Pract. 2018;19(1):149. doi: 10.1186/s12875-018-0824-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Ong A. Ripple effect: shared governance and nurse engagement. Nurs Manag. 2017;48(10):28–34. doi: 10.1097/01.NUMA.0000524811.11040.05. [DOI] [PubMed] [Google Scholar]
- 31.Rizzo KA. Effectiveness of Continuous Subcutaneous Insulin Infusion Therapy Education in a Clinic Setting. Effectiveness Of Continuous Subcutaneous Insulin Infusion Therapy Education In A Clinic Setting. 2018:1-.
- 32.Rubenstein LV, Meredith LS, Parker LE, Gordon NP, Hickey SC, Oken C, et al. Impacts of evidence-based quality improvement on depression in primary care: a randomized experiment. J Gen Intern Med. 2006;21(10):1027–35. doi: 10.1111/j.1525-1497.2006.00549.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Sherman SE, Chapman A, Garcia D, Braslow JT. Improving recognition of depression in primary care: a study of evidence-based quality improvement. Jt Comm J Qual Saf. 2004;30(2):80–8. doi: 10.1016/s1549-3741(04)30009-2. [DOI] [PubMed] [Google Scholar]
- 34.Starkey M, Wiest D, Qaseem A. Improving depression care through an online learning collaborative. Am J Med Qual. 2016;31(2):111–7. doi: 10.1177/1062860614555883. [DOI] [PubMed] [Google Scholar]
- 35.Walker CT, Gullotti DM, Prendergast V, Radosevich J, Grimm D, Cole TS, et al. Implementation of a standardized multimodal postoperative analgesia protocol improves pain control, reduces opioid consumption, and shortens length of hospital stay after posterior lumbar spinal fusion. Neurosurgery. 2019. 10.1093/neuros/nyz312 [DOI] [PubMed]
- 36.Walker-Smith TL. A Prospective Quality improvement project using a mammography risk assessment tool to increase screening mammogram use with low-income Hispanic Women: a doctor of nursing practice project report. Corpus Christi, Texas: Texas A&M University-Corpus Christi; 2018. [Google Scholar]
- 37.Whitten SK, Stanik-Hutt J. Group cognitive behavioral therapy to improve the quality of care to opioid-treated patients with chronic noncancer pain: a practice improvement project. J Am Assoc Nurs Pract. 2013;25(7):368–76. doi: 10.1111/j.1745-7599.2012.00800.x. [DOI] [PubMed] [Google Scholar]
- 38.Yoon J, Chow A, Rubenstein LV. Impact of medical home implementation through evidence-based quality improvement on utilization and costs. Med Care. 2016;54(2):118–25. doi: 10.1097/mlr.0000000000000478. [DOI] [PubMed] [Google Scholar]
- 39.Young LS, Crausman RS, Fulton JP. Suboptimal opioid prescribing: a practice change project. R I Med J (2013) 2018;101(2):41–4. [PubMed] [Google Scholar]
- 40.Barclay C, Viswanathan M, Ratner S, Tompkins J, Jonas DE. Implementing evidence-based screening and counseling for unhealthy alcohol use with epic-based electronic health record tools. Jt Comm J Qual Patient Saf. 2019;45(8):566–74. doi: 10.1016/j.jcjq.2019.05.009. [DOI] [PubMed] [Google Scholar]
- 41.Bowen DJ, Powers DM, Russo J, Arao R, LePoire E, Sutherland E, et al. Implementing collaborative care to reduce depression for rural native American/Alaska native people. BMC Health Serv Res. 2020;20(1):34. doi: 10.1186/s12913-019-4875-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Breaux-Shropshire TL, Huie R, Shropshire TS, Wyatt A, Shropshire AT, Estrada CA, et al. First steps in improving blood pressure control among primary care hypertensive veterans utilizing quality improvement tools. Ala Nurs. 2017;44(3):19–22. [Google Scholar]
- 43.Brodie N, McPeak KE. Improving human papilloma virus vaccination rates at an urban pediatric primary care center. Pediatr Qual Saf. 2018;3(5):e098. doi: 10.1097/pq9.0000000000000098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Burge SA, Powell W, Mazour L. A quality improvement endeavor improving depression screening for rural older adults. O J Rural Nurs Health Care. 2019;19(2):44–64. doi: 10.14574/ojrnhc.v19i2.563. [DOI] [Google Scholar]
- 45.Murphy Buschkoetter KL, Powell W, Mazour L. Implementation of a comprehensive diabetic foot exam protocol in rural primary care. Online J Rural Nurs Health Care. 2019;19(1):43–63. doi: 10.14574/ojrnhc.v19i1.560. [DOI] [Google Scholar]
- 46.Camp NL, Robert RC, Nash JE, Lichtenstein CB, Dawes CS, Kelly KP. Modifying provider practice to improve assessment of unhealthy weight and lifestyle in young children: translating evidence in a quality improvement initiative for at-risk children. Child Obes. 2017;13(3):173–81. doi: 10.1089/chi.2016.0124. [DOI] [PubMed] [Google Scholar]
- 47.Campbell K, Carpenter KLH, Espinosa S, Hashemi J, Qiu Q, Tepper M, et al. Use of a Digital Modified Checklist for Autism in Toddlers - revised with follow-up to improve quality of screening for autism. J Pediatr. 2017;183:133–9.e1. doi: 10.1016/j.jpeds.2017.01.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Colborn KL, Helmkamp L, Bender BG, Kwan BM, Schilling LM, Sills MR. Colorado Asthma toolkit implementation improves some process measures of asthma care. J Am Board Fam Med. 2019;32(1):37–49. doi: 10.3122/jabfm.2019.01.180155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Daaleman TP, Brock D, Gwynne M, Weir S, Dickinson I, Willis B, et al. Implementing lean in academic primary care. Qual Manag Health Care. 2018;27(3):111–6. doi: 10.1097/qmh.0000000000000173. [DOI] [PubMed] [Google Scholar]
- 50.Fabre JC, Andresen PA, Wiltz GM. Closing the loop on electronic referrals: a quality improvement initiative using the care coordination model. J Ambul Care Manag. 2020;43(1):71–80. doi: 10.1097/jac.0000000000000315. [DOI] [PubMed] [Google Scholar]
- 51.Fisher-Borne M, Preiss AJ, Black M, Roberts K, Saslow D. Early outcomes of a multilevel human papillomavirus vaccination pilot intervention in federally qualified health centers. Acad Pediatrics. 2018;18(2):S79–S84. doi: 10.1016/j.acap.2017.11.001. [DOI] [PubMed] [Google Scholar]
- 52.Fortney JC, Pyne JM, Ward-Jones S, Bennett IM, Diehl J, Farris K, et al. Implementation of evidence-based practices for complex mood disorders in primary care safety net clinics. Fam Syst Health. 2018;36(3):267–80. doi: 10.1037/fsh0000357. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Garza L, Dols J, Gillespie M. An initiative to improve primary prevention of cardiovascular disease in adults with type II diabetes based on the ACC/AHA (2013) and ADA (2016) guidelines. J Am Assoc Nurs Pract. 2017;29(10):606–11. doi: 10.1002/2327-6924.12492. [DOI] [PubMed] [Google Scholar]
- 54.Gold R, Bunce A, Cowburn S, Davis JV, Hollombe C, Nelson CA, et al. Cardiovascular care guideline implementation in community health centers in Oregon: a mixed-methods analysis of real-world barriers and challenges. BMC Health Serv Res. 2017;17(1):253. doi: 10.1186/s12913-017-2194-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Green BB, Fuller S, Anderson ML, Mahoney C, Mendy P, Powell SL. A quality improvement initiative to increase colorectal cancer (CRC) screening: collaboration between a Primary Care Clinic and Research Team. J Fam Med. 2017;4(3). 10.26420/jfammed.2017.1115 [DOI] [PMC free article] [PubMed]
- 56.Hanlin RB, Asif IM, Wozniak G, Sutherland SE, Shah B, Yang J, et al. Measure accurately, act rapidly, and partner with patients (MAP) improves hypertension control in medically underserved patients: Care Coordination Institute and American Medical Association Hypertension Control Project Pilot Study results. J Clin Hypertens (Greenwich). 2018;20(1):79–87. doi: 10.1111/jch.13141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Hawk M, Nowalk MP, Moehling KK, Pavlik V, Raviotta JM, Brown AE, et al. Using a mixed methods approach to examine practice characteristics associated with implementation of an adult immunization intervention using the 4 Pillars Practice Transformation Program. J Healthc Qual. 2017;39(3):153–67. doi: 10.1097/jhq.0000000000000071. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Jonas DE, Miller T, Ratner S, McGuirt B, Golin CE, Grodensky C, et al. Implementation and quality improvement of a screening and counseling program for unhealthy alcohol use in an academic general internal medicine practice. J Healthc Qual. 2017;39(1):15–27. doi: 10.1097/jhq.0000000000000069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Knierim KE, Hall TL, Dickinson LM, Nease DE, Jr, de la Cerda DR, Fernald D, et al. Primary care practices' ability to report electronic clinical quality measures in the EvidenceNOW southwest initiative to Improve Heart Health. JAMA Netw Open. 2019;2(8):e198569. doi: 10.1001/jamanetworkopen.2019.8569. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Implementation of psychiatric e-consultation in family medicine community health centers. Int J Psychiatry Med. 2019;54(4/5):296-306. 10.1177/0091217419869081 [DOI] [PubMed]
- 61.Makelarski JA, DePumpo M, Boyd K, Brown T, Kho A, Navalkha C, et al. Implementation of systematic community resource referrals at small primary care practices to promote cardiovascular disease self-management. J Healthc Qual. 2019. 10.1097/jhq.0000000000000234 [DOI] [PubMed]
- 62.Minsky N, Tamler R, editors. Endocrine eConsults improve access to care for the underserved. ACM International Conference Proceeding Series; 2017. March 13_KQ2_CINAH, PubMed, SCOPUS.
- 63.Modica C, Lewis JH, Bay C. Colorectal cancer: applying the value transformation framework to increase the percent of patients receiving screening in federally qualified health centers. Prev Med Rep. 2019;15:100894. doi: 10.1016/j.pmedr.2019.100894. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Nagykaldi ZJ, Scheid D, Zhao D, Mishra B, Greever-Rice T. An innovative community-based model for improving preventive care in rural counties. J Am Board Fam Med. 2017;30(5):583–91. doi: 10.3122/jabfm.2017.05.170035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Nowalk MP, Moehling KK, Zhang S, Raviotta JM, Zimmerman RK, Lin CJ. Using the 4 pillars to increase vaccination among high-risk adults: who benefits? Am J Manage Care. 2017;23(11):651–5. [PMC free article] [PubMed] [Google Scholar]
- 66.Ober AJ, Watkins KE, Hunter SB, Ewing B, Lamp K, Lind M, et al. Assessing and improving organizational readiness to implement substance use disorder treatment in primary care: findings from the SUMMIT study. BMC Fam Pract. 2017;18(1):107. doi: 10.1186/s12875-017-0673-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Quanbeck A, Brown RT, Zgierska AE, Jacobson N, Robinson JM, Johnson RA, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for opioid prescribing in primary care. Implement Sci. 2018;13(1):21. doi: 10.1186/s13012-018-0713-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Regan ME. Implementing an evidence-based clinical decision support tool to improve the detection, evaluation, and referral patterns of adult chronic kidney disease patients in primary care. J Am Assoc Nurs Pract. 2017;29(12):741–53. doi: 10.1002/2327-6924.12505. [DOI] [PubMed] [Google Scholar]
- 69.Richards JE, Bobb JF, Lee AK, Lapham GT, Williams EC, Glass JE, et al. Integration of screening, assessment, and treatment for cannabis and other drug use disorders in primary care: an evaluation in three pilot sites. Drug Alcohol Depend. 2019;201:134–41. doi: 10.1016/j.drugalcdep.2019.04.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Roderick SS, Burdette N, Hurwitz D, Yeracaris P. Integrated behavioral health practice facilitation in patient centered medical homes: a promising application. Fam Syst Health. 2017;35(2):227–37. doi: 10.1037/fsh0000273. [DOI] [PubMed] [Google Scholar]
- 71.Savas A, Smith E, Hay B. EHR quality indicator tracking: a process improvement pilot project to meet MACRA requirements. Nurs Pract. 2019;44(4):30–9. doi: 10.1097/01.NPR.0000554084.05450.0e. [DOI] [PubMed] [Google Scholar]
- 72.Schaeffer AM, Jolles D. Not missing the opportunity: improving depression screening and follow-up in a multicultural community. Jt Comm J Qual Patient Saf. 2019;45(1):31–9. doi: 10.1016/j.jcjq.2018.06.002. [DOI] [PubMed] [Google Scholar]
- 73.Schiff GD, Reyes Nieva H, Griswold P, Leydon N, Ling J, Federico F, et al. Randomized trial of reducing ambulatory malpractice and safety risk: results of the Massachusetts PROMISES Project. Med Care. 2017;55(8):797–805. doi: 10.1097/mlr.0000000000000759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Schurman JV, Deacy AD, Johnson RJ, Parker J, Williams K, Wallace D, et al. Using quality improvement methods to increase use of pain prevention strategies for childhood vaccination. World J Clin Pediatr. 2017;6(1):81–8. doi: 10.5409/wjcp.v6.i1.81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Senger JJ. A Concussion Toolkit Educational Session: promoting evidence-based management of youth concussion in a rural primary care setting. Concussion Toolkit Educational Session: Promoting Evidence-Based Management Of Youth Concussion In A Rural Primary Care Setting. 2018:1-.
- 76.Shah T, Patel-Teague S, Kroupa L, Meyer AND, Singh H. Impact of a national QI programme on reducing electronic health record notifications to clinicians. BMJ Qual Saf. 2019;28(1):10–4. doi: 10.1136/bmjqs-2017-007447. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Sloand E, Vangraafeiland B, Holm A, MacQueen A, Polk S. Text message quality improvement project for influenza vaccine in a low-resource largely Latino pediatric population. J Healthc Qual. 2019;41(6):362–8. doi: 10.1097/JHQ.0000000000000190. [DOI] [PubMed] [Google Scholar]
- 78.van Eeghen C, Kennedy AG, Pasanen ME, MacLean CD. A new quality improvement toolkit to improve opioid prescribing in primary care. J Am Board Fam Med. 2020;33(1):17–26. doi: 10.3122/jabfm.2019.01.190238. [DOI] [PubMed] [Google Scholar]
- 79.Weiner BJ, Rohweder CL, Scott JE, Teal R, Slade A, Deal AM, et al. Using practice facilitation to increase rates of colorectal cancer screening in community health centers, North Carolina, 2012-2013: Feasibility, Facilitators, and Barriers. Prev Chronic Dis. 2017;14:E66. doi: 10.5888/pcd14.160454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Williams MD, Sawchuk CN, Shippee ND, Somers KJ, Berg SL, Mitchell JD, et al. A quality improvement project aimed at adapting primary care to ensure the delivery of evidence-based psychotherapy for adult anxiety. BMJ Open Qual. 2018;7(1):e000066. doi: 10.1136/bmjoq-2017-000066. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Yusupov E, Krishnamachari B, Rand S, Abdalla M, Zwibel H. Quality of hypertension care: an improvement initiative in two outpatient health care centers. J Eval Clin Pract. 2019;25(3):463–8. doi: 10.1111/jep.13067. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.