Version Changes
Revised. Amendments from Version 1
We added additional information in the second to last paragraph of the introduction section. This emphasizes the importance of this study and the need for further research in this unexplored area regarding the impact of accreditation on CQI. We added more information to the Search Strategy, eligibility criteria and selection of sources, and Data charting sections. I hope this helps the reader in understanding the search strategy. We added the missing information under the theme “Accreditation and its standards in general.” We added more information to our results section. This could help the reader to understand the discussion in detail. We also elaborated conclusions section.
Abstract
Background
Accreditation in medical education has existed for more than 100 years, yet the impact of accreditation remains inconclusive. Some studies have shown the effects of accreditation on student outcomes and educational processes at medical schools. However, evidence showing the impact of accreditation on continuous quality improvement of undergraduate medical education programs is still in its infancy. This scoping review explores the impact of accreditation on continuous quality improvement (CQI).
Methods
This scoping review followed the methodology of the Preferred Reporting Items of Systematic Reviews and the Meta-Analysis extension for scoping reviews (PRISMA-ScR) checklist outlined by Arksey and O'Malley (2005). Databases, including PubMed, Medline, ERIC, CINHAL, and Google Scholar, were searched to find articles from 2000 to 2022 related to the accreditation of undergraduate medical education programs and continuous quality improvement.
Results
A total of 35 full-text articles were reviewed, and ten articles met our inclusion criteria. The review of the full-text articles yielded four themes: Accreditation and its standards in general, Accreditation and its impact on student outcomes, Accreditation and its impact on medical school's educational processes, Accreditation and CQI. However, the literature evidence suggesting the impact of accreditation on CQI is minimal. The quality assurance approach is based on meeting the standards of accreditation. The quality improvement approach is based on striving for excellence. Literature suggests a requirement to move from student outcomes to CQI measures. CQI requires everyone in the organization to take responsibility and accountability, considering quality as the result of every single step or process and leaders supporting improvements in data collection and data analysis for quality improvement.
Conclusions
The literature on accreditation and CQI are limited in number. More research studies are required to enhance undergraduate medical education accreditation practices' value to medical students, educators, academic leaders, programs, and the public. It was recommended that medical schools embrace the culture and vision perpetuated by the CQI process.
Keywords: Accreditation, quality improvement, CQI, quality assurance, educational processes, student outcomes
Introduction
Accreditation is expected to ensure the quality assurance of higher education ( Salto, 2018). According to the World Health Organization (WHO), regulatory bodies should play a central role in ensuring that public and private sector professionals are competent, sufficiently experienced, and adhere to agreed standards relative to the scope of practice and competency enshrined in regulation and legislative norms; countries should be supported in establishing or strengthening them to provide continuous updates to accreditation and credentialing ( WHO, 2016, p. 20).
The WHO emphasized the importance of accreditation and regulation. But only 49% of countries have access to accreditation bodies that can accredit medical schools ( Bedoll et al., 2021). van Zanten et al. (2012, p.587) defined accreditation as a "review of an educational program conducted by a governmental organization or a private entity accountable at a government level based on publicized standards and predetermined protocols." Medical education accreditation has existed for over 100 years ( Akdemir et al., 2018; Loddenkemper et al., 2010). However, the evidence for the impact of accreditation and its outcomes is inconclusive ( Acevedo-De-Los-Ríos & Rondinel-Oviedo, 2022; Blouin et al., 2018; Dattey et al., 2014; Volkwein et al., 2007). The literature regarding the impact of accreditation on medical schools' processes or quality improvement (QI) processes is still emerging, even though there are few articles on this topic.
The accreditation process is important as the citations/negative reports and site visits by the expert reviewers can bring curricular reforms and enhance the credibility of these programs ( White et al., 2013). Accreditation can lead to a change process and impact medical schools' educational processes ( Arja et al., 2021; Blouin et al., 2018). The accreditation-related activities enable medical colleges to review and reassess their processes that improve medical education programs ( Blouin et al., 2018).
It was identified that graduates from accredited medical schools are performing better in programs with Educational Commission for Foreign Medical Graduates (ECFMG) certification ( Tackett et al., 2021; van Zanten, 2015). Thus, educational program quality appears to be directly related to student performance, suggesting that the accredited undergraduate medical education programs improved the quality of education offered to the students. Educational programs did benefit from engaging in the quality improvement processes ( Tackett et al., 2021). The ECFMG announced that "for a medical school to achieve recognized accreditation status, the accrediting agency of the school must be reviewed and recognized by an external quality assurance organization. This quality assurance organization, in turn, must be reviewed and approved by ECFMG. Only medical schools accredited by an agency recognized by an organization approved by ECFMG will satisfy the requirements of the Recognized Accreditation Policy" ( ECFMG, 2022). The announcement by ECFMG has further increased the importance of the accreditation of medical education programs.
Also, there is an inherent belief that the accreditation of medical schools guarantees the quality of educational programs of medical schools. Despite the importance and emphasis on accreditation, there is no guarantee that it achieves the intended outcomes of accreditation and medical schools. Quality management aims to achieve accreditation goals through a variable blend of quality assurance (QA), QI, and quality control ( Akdemir et al., 2018).
Continuous quality improvement (CQI), which originated in business as a method of quality control for services or products ( Radawski, 1999), is an ongoing cycle of organizational improvement that uses quality indicators ( Kritchevsky & Simmons, 1991). It is time to move from QA to QI across medical education programs ( Barzansky et al., 2015). Barzansky et al. (2015) suggested having a periodic evaluation of programs in addition to the summative evaluations of the programs by accreditation bodies at fixed intervals. Barzansky et al. (2015) emphasized the importance of submitting annual reports by medical schools. The functional CQI process should be feasible to implement. It should focus on accreditation standards to improve the quality of educational programs and outcomes and avoid duplication of efforts ( Barzansky et al., 2015).
In 2015, the Liaison Committee on Medical Education (LCME) began requiring internal CQI processes for medical programs to ensure compliance with accreditation standards ( Stratton, 2019). LCME added the language to its standards: strategic planning and CQI, which states that a medical school must engage in ongoing planning and CQI processes that establish short- and long-term goals for the educational program, resulting in the achievement of measurable outcomes that are used to improve the quality of the educational program. This should ensure effective oversight of the educational program’s compliance with accreditation standards ( LCME, 2022, p. 1).
Regulatory and accreditation agencies view accreditation as a crucial instrument for meeting established standards and affirming higher education programs' legitimacy, status, and suitability. Conversely, some stakeholders perceive the accreditation process as a drain on resources and efforts, potentially outweighing its cost-effectiveness ( Blouin et al., 2018; Simpson et al., 1998). Accreditation has enhanced various educational processes, including documentation, monitoring, data analysis and governance ( Arja et al., 2021; Blouin et al., 2018). However, there is no concrete evidence regarding the impact of accreditation on CQI.
This scoping review explores the literature regarding the impact of accreditation on the CQI process in undergraduate medical programs. The scoping review maps pertinent literature about a topic of interest ( Arksey & O’Malley, 2005). A scoping review is less likely to address a specific research question and even less likely to evaluate the quality of studies included in the scoping review ( Arksey & O’Malley, 2005). The evidence about accreditation and CQI in medical education programs is still in its infancy ( Akdemir et al., 2020). This led us to choose a scoping review over a systematic review to explore the impact of accreditation on the CQI process in undergraduate medical education programs.
Methods
Research protocol
We followed the scoping review framework, the Preferred Reporting Items of Systematic Reviews, and the Meta-Analysis extension for scoping reviews (PRISMA-ScR) checklist ( Arksey & O’Malley, 2005; Tricco et al., 2018). It was summarized by the Joanna Briggs Institute (2015) as a six-stage approach to conducting the scoping review as outlined:
-
1.
Identify the research question,
-
2.
Identify relevant studies/literature,
-
3.
Select the literature,
-
4.
Chart the data,
-
5.
Collate, summarize, and report results; and
-
6.
Consult (optional)
A scoping review has a broader scope than a systematic review ( Arksey & O’Malley, 2005). The scoping review aims to identify the breadth of the current literature regarding the research question and identify knowledge gaps ( Arksey & O’Malley, 2005).
Information sources and search strategy
We searched PubMed, Medline, ERIC, CINHAL and Google Scholar (2000 to present, in-process & other non-Indexed citations) to identify articles addressing the impact of accreditation on CQI process in undergraduate medical education programs. Search strategies were developed by an academic health science librarian (LN) with input from the project leads (SA) and co-leads (AT and JF). The search strategy and database selection were determined in consultation with an academic health science librarian (LN) Librarian and a researcher familiar with scoping review (AT). The search strategy included PEO questions examining the relationship between exposure and outcome, which are accreditation and CQI in our study. It also included the question related to exploring the scope of literature on Condition and Context, which is undergraduate medical education programs or medical schools in our study. The keywords used for the search were “quality improvement,” “accreditation,” “medical education,” and “medical students.” Please refer to Table 1. Searches were completed in December 2022 and limited to articles published between January 1, 2000, and October 31, 2022. The reason for selecting the year 2000 reflects the relevance of data in this topic. English-language limits applied to all databases, and any article retrieved in a language other than English was excluded.
Table 1. Research consultation & Search strategy with Librarian Lindsey Nichols.
Context | AND | Condition | AND | Population |
---|---|---|---|---|
“quality improvement” OR
“continuous quality improvement” OR CQI OR “change management” OR "Quality Improvement"[Mesh] OR "Change Management"[Mesh] |
accreditation OR "Accreditation"[Mesh] |
undergraduate AND (“medical school” OR
“medical education”) OR "Education, Medical"[MAJR] OR "Education, Medical, Undergraduate"[Mesh] NOT residen* NOT post-graduate* |
Search Tips: Truncat* = any variation of root word “Phrase search” = search exact phrase
Research Question:
Impact of higher ed accreditation on undergraduate medical education and continuous quality improvement
Search Strategy Approach & Lore
1. Break question into topic chunks. See table below.
a. PEO question = examines relationship between Exposure and Outcome.
i. You can start with PEO, end with PICO.
b. CoCoPop question = explores scope of literature on Condition and Context.
i. You can start w CoCoPop, end with PICO.
c. PICO(T) question = compares intervention against intervention, or compare intervention against lack of intervention.
d. Many other question frameworks Googleable, particularly for qualitative questions. (SPICE! SPIDER! ECLIPSE!)
2. Brainstorm synonyms.
a. Search for MeSH (Medical Subject Headings describing content) in PubMed.
3. Keyword Harvest: Skim titles and abstracts. Add new keywords and MeSH / subject headings to table as you find them in search results.
a. Keyword synonyms within same column = combine with OR (usually)
b. Keywords across columns = combine with AND
c. Reminder: Different databases have different formatting for subject headings.
i. Most databases do have a subject heading thesaurus in the top banner.
ii. Use each database’s thesaurus to build a slightly different search strategy for each database.
4. Continue modifying search and harvesting keywords and MeSH until results are highly relevant to topic.
a. Apply limits to search: date of study publication, age of participant, language of article, etc.
Databases & Search Strategy
Pubmed
Two starting searches below: one broad excluding outcomes, and one focused including outcomes.
A. Focused Search including outcomes
((undergraduate[Title/Abstract] AND ("medical school"[Title/Abstract] OR "medical education"[Title/Abstract]) OR ("Education, Medical"[MAJR] OR "Education, Medical, Undergraduate"[Mesh])) AND
("accreditation"[Title/Abstract] OR "Accreditation"[Mesh])) AND
(("quality improvement"[Title/Abstract] OR CQI[Title/Abstract] OR "change management"[Title/Abstract]) OR ("Quality Improvement"[Mesh] OR "Change Management"[Mesh]))
B. Broad Search excluding outcomes
((undergraduate[Title/Abstract] AND ("medical school"[Title/Abstract] OR "medical education"[Title/Abstract]) OR ("Education, Medical"[MAJR] OR "Education, Medical, Undergraduate"[Mesh])) AND
("accreditation"[Title/Abstract] OR "Accreditation"[Mesh]))
ERIC
Gray Literature Search: conference papers, reports, etc.
Bonus Resources
1. PubMed Three Search Approaches Handout.docx
2. Mastering Keyword Searching async workshop (self-enroll)
a. Refresher on Boolean searching in Advanced Search of Pubmed, other subject databases.
3. PubMed Online Training from National Library of Medicine
a. Quick tours in 1–3 mins; 60 min tutorials; 90–300 min online classes in how to search PubMed.
4. Zotero.org <- free online reference manager; 2-click citations and references in Word or Google Docs.
a. Zotero’s own Help Documentation is easy to read, includes screenshots.
b. Download 1) Zotero, and 2) Zotero Connector extension for your browser (Chrome, Safari, etc.).
i. Zotero Connector = one-click add online sources (including both metadata and PDFs or HTML full text) to your Zotero library.
5. Zotero D2L async workshop (self-enroll)
a. Short videos, short quizzes, light reading. Leads you through installation, setup, basic and advanced features.
Eligibility criteria and selection of sources
All abstracts were independently screened (by lead and co-lead, SA and JF). The articles resulting from the first database (Medline, PubMed, and ERIC), second database (CINAHL), and Google Scholar searches were screened for relevance by the principal investigator (SA). Another co-lead (JF) reviewed all abstracts, the same as the principal investigator. Both investigators screened titles and abstracts independently in a data sheet using Excel. A common consensus was required between SA and JF. If there was a disagreement, it was resolved by another co-lead (AT). At first, only titles and abstracts were reviewed. Only articles not excluded through the abstract review underwent full-text review. Primary research articles written in English were included if they:
-
1.
Discussed the accreditation process for medical schools or undergraduate medical education programs.
-
2.
Emphasized the impact of accreditation on the CQI process at medical schools or undergraduate medical programs.
-
3.
Involved undergraduate medical programs and their basic medical education accreditation standards.
Articles were excluded if they discussed the QI curricula in postgraduate training or the implementation of ACGME guidelines for postgraduate training. All three investigators (SA, AT, and JF) discussed any discrepancies related to inclusion and exclusion.
Data charting of selected articles and process
Following the approach outlined by Arksey and O’Malley (2005), we extracted data relevant to our research question. The data was entered into a Microsoft Word spreadsheet, and a table was constructed for analysis. The data included author, year of publication, place of study, aim/purpose, methods, participants, intervention, outcome, and key findings. Consistent with the scoping review methodology, we did not assess the methodological quality of the included studies.
The principal investigator (SA) and co-investigator (JF) read full-text articles, extracted data, and charted detailed information in an Excel standardized data extraction sheet. Another co-lead (AT) reviewed the data extraction sheet. Please refer to Table 2.
Table 2. Scoping review-summary of articles.
First Author | year | Country | Title of the article | Study population | Aim | Methodology | Outcome
measures |
Results/conclusions |
---|---|---|---|---|---|---|---|---|
Akdemir | 2020 | Canada
Netherlands |
Evaluation of continuous quality improvement in accreditation for medical education | The CQI experiences with three accreditation systems from USA, Canada and Netherlands | Naturalistic utility-focused evaluation. Naturalistic approach connected to real-life experiences. | The gathering of evidence about accreditation systems and CQI in medical education seems to be still in its infancy. Further research is needed. | ||
Blouin | 2019 | Canada | Quality improvement in medical schools: vision meets culture. | 16 Canadian medical schools’ Academic leaders, formal teachers and clinical teachers | study explores whether QI practices are recognised as such in organisations not culturally QI-oriented. Specifically, it examines faculty members’ perceptions about the degree of QI implementation in their medical education programmes. | Respondents rate each statement on a 5-point Likert-type scale. Parametric statistical analysis (means and SDs). | questionnaire of the Malcolm Baldrige National Quality Award framework | Perceived QI implementation levels were low across programmes and for each category of respondents. This was especially true for the domains of ‘Strategy’, ‘Measurement/ analysis/knowledge management’ and ‘Operations’. Medical schools’ existing QI processes are not recognised as QI activities. For QI strategies to succeed, a programme’s culture must embrace QI. In the execution of their QI visions, medical schools should spend resources on embedding quality in the organisation culture in addition to strengthening existing QI practices. |
Stansfield | 2019 | USA | Integration of Continuous Quality Improvement Methods into Annual Program and Institutional Evaluation | Wayne State University Office of Graduate Medical Education (WSU GME) | To improve the institutional- and program-level evaluation processes, to operationalize a culture of continuous quality improvement (CQI), and to increase the quality and achievement of action items | WSU GME phased the 4 CQI elements into the evaluation process at the program and institutional levels, including the annual program evaluation (APE) and the annual institutional review. | Using a systematic goal-setting process (SMART) and a systematic multi-source assessment process (dashboards), WSU GME has begun to successfully leverage widely accepted CQI methods to capitalize on program and institutional opportunities and strengths and to foster a more effective culture of educational improvement. | |
Stratton | 2019 | USA | Legitimizing Continuous Quality Improvement (CQI): Navigating Rationality in Undergraduate Medical Education | practical strategies to shield CQI from being passively dismissed, excessively routinized. | Commentary/
perspective |
Ongoing improvement, rather than attainment of a static benchmark. Establish a Clear Purpose and coordinate data collection and reporting with existing scheduled activities, such as departmental reviews, university accreditation, or strategic planning. Secure Leadership Buy-in. Envision a Process. “Humanize” the Process. Share Results, Accountability and Scrutinize Measures & Outcomes. | ||
Blouin | 2018 | Canada | Accreditation of Medical Education Programs: Moving from Student Outcomes to Continuous Quality Improvement Measures | Establishing the connection between accreditation and CQI and thereby improvement in medical education programs leading to better graduates. | perspective | Markers of effectiveness of accreditation reported in the literature focus on curricular transformations and the quality of programs’ output, including the number of curricular changes implemented by programs, organizational quality outcomes, and student outcomes. Markers of its impact need to include CQI-related measures. It is time to move away from a focus on student outcomes as measures of the impact of accreditation and embrace additional markers, such as indicators of organizational CQI orientation. | ||
Nekdemir | 2018 | Australia | Accreditation as a quality improvement tool: is it still relevant? | Examines Australian Medical Council (AMC) standards. | Quality management aims to achieve the objectives of
accreditation as effectively as possible through a variable mix of quality assurance, quality improvement and quality control. In the quality assurance approach, minimum standards are set to assure the quality of education and graduates produced. The legislation under which the AMC accredits programs requires this approach and focuses on compliance. The quality improvement approach is based on striving for excellence. |
|||
Ward | 2018 | USA | Implementing a Course Review Process for a Continuous Quality Improvement Model for a Medical School Curriculum. | Meharry Medical College School of Medicine. | Plan- Do- Study- Act Cycle for Learning and Improvement for continuous review of course content, outcomes, and evaluations. This process identifies strengths, challenges, and opportunities for innovative steps for continuous quality improvements to the curriculum. | |||
Alrebish | 2017 | Saudi Arabia | Accreditation of medical schools in Saudi Arabia: A qualitative study | The aim of the study was to examine the purposes and outcomes of accreditation, and stakeholders' experience of accreditation in Saudi Arabia. | Thematic analysis approach. Triangulation of data. Literature review, analysis of accreditation documents, examined the outcome of accreditation process (pre and post) through stakeholders' experience of accreditation (learner, teacher, and academic leader perspectives) | The three-phase study: "Passing the exam" versus long-term benefit, generic versus
specialized accreditation standards, and internal quality assurance and self-evaluation. |
Sustainable accreditation means that there is a need to meet both the immediate accreditation standards (""the exam"") as well as establishing a basis for continuing quality improvement. Medical school leaders should “perceive” and “promote” the accreditation process as a journey for quality improvement, rather than as an exam that must be passed. Faculty development should be encouraged with the aim of establishing a culture of quality improvement rather than a guide on “how to be accredited”. The self-study process, with multiple stakeholder input, is key to building ownership in self-evaluation and goal setting within medical schools." | |
Shroyer | 2016 | USA | Drivers of Dashboard Development (3-D)
A Curricular Continuous Quality Improvement Approach |
SBU school of Medicine | Continuous quality improvement (CQI) settings, summary dashboard reports have been used to evaluate and improve performance. | This report represents a summary of the SBU School of Medicine Undergraduate Medical Education office’s Drivers of Dashboard Development (3-D) approach for continuous quality improvement. | Substantial improvements over time have been documented in KPIs including timeliness of clerkship grades, mid-clerkship feedback, student mistreatment policy awareness, and student satisfaction. | Stakeholder feedback indicates that the dashboards have provided useful information guiding data-driven curricular changes, such as integrating clinician–scientists as lecturers in basic science courses to clarify the clinical relevance of specific topics. The 3-D approach may be considered by UME programs as a template for providing faculty and leadership with a CQI framework to establish shared goals, document compliance, report accomplishments, enrich communications, facilitate decisions, and improve performance. |
Barzansky | 2015 | USA, Canada, Taiwan, Korea | Continuous quality improvement in an accreditation system for undergraduate medical education: Benefits and challenges | National association of medical schools and several medical school accrediting bodies. | To identify the factors that affect the ability to implement a continuous quality improvement (CQI) process for the interval review of accreditation standards. | Case examples from the United States, Canada, the Republic of Korea and Taiwan, were collected and analyzed to determine the strengths and challenges of the CQI processes. | Accreditation standards so as to result in the improvement of educational quality and outcomes, be feasible to implement, avoid duplication of effort and have both commitment and resource support from the sponsoring entity and the individual medical schools. CQI can enhance educational program quality and outcomes, if the process is designed to collect relevant information and the results are used for program improvement. Medical schools should engage in interim review of their compliance with the accreditation standards of their country and act on the results in order to support a culture of continuous quality improvement (CQI). |
Ethical approval
This study was exempted from the research and ethics committee approval at Avalon University School of Medicine as this was a scoping review.
Results
The first database search, on PubMed, Medline, and ERIC resulted in 279 abstracts. Of these 279 abstracts, 31 required a review of full-text articles, and one was a repeat. We excluded 247 articles based on graduate medical education or QI curriculum for residents, related to medical care, healthcare not in English, etc. Out of the 31 articles mandated for full-text review, 17 articles were categorized under general articles on accreditation, based on standards of accreditation and recent trends of accreditation, three articles were based on the impact of accreditation on educational processes, one article was based on the impact of accreditation on student outcomes, and ten articles described accreditation and CQI. The second database CINHAL resulted in nine abstracts. Out of the nine abstracts, three were removed as they were repeated from PubMed, and one was removed as it was published in a language other than English. The other five were also excluded as they were related to graduate medical education, education in general, QI in general education, or healthcare. The third database, Google Scholar search, resulted in 19,000 abstracts. But we reviewed only the initial 75 abstracts as Google Scholar was not returning many useful articles for this scoping review or the research question. Out of 75, three were repeats from PubMed. The other 68 articles were excluded as they were unrelated to this scoping review or the research question. Only four articles from Google Scholar required full-text review. They were related to accreditation and accreditation standards and had nothing to do with CQI. Only ten articles that described accreditation and CQI (PubMed) were included in the final review to answer the research question.
In summary, we reviewed 35 full-text articles. Out of 35 articles, 21 concerned accreditation and its standards, including recent trends in medical education accreditation; three articles were related to accreditation and its impact on medical school education processes, one was related to accreditation and its impact on student outcomes, and ten published articles concerning accreditation and CQI. We were able to review another four articles found while retrieving these 35 articles. Of these four articles, two concern accreditation and its standards in general, one related to accreditation and its impact on medical school education processes, and two related to accreditation and its impact on student outcomes. Please refer to Figure 1.
Figure 1. PRISMA flow diagram for “The impact of accreditation on continuous quality improvement process in undergraduate medical education programs: A scoping review”.
Adopted from: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372: n71. doi: 10.1136/bmj. n71.
For more information, visit: http://www.prisma-statement.org/
The ten articles included for full review concerning the accreditation and CQI are as follows. Akdemir et al. (2018) defined QA and QI approaches in the accreditation of undergraduate medical education programs. After examining CQI's role in three accreditation systems, including the American and Canadian ones, the authors observed that CQI in undergraduate medical accreditation is still in its infancy ( Akdemir et al., 2020). Akdemir et al. (2020) found that the literature suggesting the impact of accreditation on CQI needs to be more extensive. More research and effort are required in this unexplored area. Sustainable accreditation means meeting the immediate accreditation standards and establishing a ground for CQI ( Alrebish et al., 2017)). Alrebish et al. (2017) advocated a holistic approach to continuous quality improvement (CQI) rather than the checklist approach commonly used by accreditation bodies. They recommended that medical school leaders view and promote the accreditation process as a catalyst for quality improvement rather than merely a hurdle to overcome. Sustainable accreditation aimed at the long-term enhancement of medical education should encompass immediate standards (formative evaluation of the program) and long-term planning for CQI ( Alrebish et al., 2017).
Accreditation is a process-driven approach that follows predefined and published standards, conducted at fixed intervals to facilitate collective decisions on compliance ( Barzansky et al., 2015). It was advocated for moving from student outcomes to CQI measures ( Barzansky et al., 2015; Blouin & Tekian, 2018). Ongoing improvement should be emphasized rather than the attainment of a static benchmark incorporating activities such as departmental reviews, university accreditation, or strategic planning ( Stratton, 2019). Blouin and Tekian (2018) suggested that the CQI should be one of the markers for accreditation impact if accreditation is intended to promote CQI. Blouin and Tekian (2018) recommended transitioning the accreditation of medical education programs from emphasizing student outcomes to focusing on continuous quality improvement (CQI) processes. This shift involves moving away from reliance on quantitative data, such as student outcomes, towards enhancing CQI processes. The literature supports moving from episodic accreditation evaluations to a continuous CQI process ( Barzansky et al., 2015). For CQI to be effective, it should produce feasible outcomes and have the necessary commitment and resources centered on accreditation standards ( Barzansky et al., 2015).
Integrating CQI methods into annual programs and institutional evaluation is critical ( Stansfield et al., 2019). It is also time for medical schools to change their approach toward CQI. In the execution of their QI visions, medical schools should spend resources on embedding CQI in the organization's culture ( Blouin, 2019). The Plan- Do- Study- Act cycle was implemented in one of the studies for learning and improvement because continuous review of course content, outcomes, and evaluations is a process that identifies strengths, challenges, and opportunities for innovative steps for CQI to the curriculum ( Ward et al., 2018). Establishing CQI 3D dashboards incorporating the CQI framework was innovative ( Shroyer et al., 2016).
Discussion
We categorized the articles into four themes as follows:
-
1.
Accreditation and its standards in general,
-
2.
Accreditation and its impact on student outcomes,
-
3.
Accreditation and its impact on medical school's educational processes,
-
4.
Accreditation and CQI.
In addition to accreditation and CQI findings, we would like to describe the themes that illustrate the breadth of literature.
Accreditation and its standards in general
Concern about the quality of graduates and Undergraduate Medical Education (UME) is growing internationally ( Tackett et al., 2019). Over 3,000 medical schools worldwide provide training and education to future physicians ( Bedoll et al., 2021). This number is twice as many as there were twenty years ago. Over many decades, healthcare workers and physicians have been crossing borders, increasingly globalizing medical education ( Allen et al., 2022; van Zanten et al., 2008). Assuring and improving the quality of education and training for future physicians is imperative in the best interest of the communities that these physicians serve, especially migrating physicians from one country to another for QA purposes ( van Zanten et al., 2008). However, there needs to be more evidence to support existing medical schools' accreditation practices or to lead the innovation or improvement of undergraduate medical education programs or the QI process at medical schools ( Tackett et al., 2019; van Zanten et al., 2008). Establishing new accreditation practices and improving existing ones should be based on data to ensure more efficient use of resources. However, the evidence related to UME accreditation and CQI has yet to be systematically summarized, and concerns remain that, despite the growing momentum to implement accreditation worldwide, evidence informing UME accreditation systems is limited ( Tackett et al., 2019).
Accreditation and its impact on student outcomes
The publications on accreditation and student outcomes were Roy et al. (2020), Al-Eyadhy and Alenezi (2021), and Tackett et al. (2021). There was a positive correlation between the accreditation cycle and students' performance on national examinations among Canadian medical schools ( Roy et al., 2020). Al-Eyadhy and Alenezi (2021) found that accreditation cycles were associated with increased student satisfaction rates. The pre-accreditation phase activities and the self-study process are essential triggers for QI practices related to accreditation. Tackett et al. (2021). identified a significant association between the accreditation status of the medical school and ECFMG certification rates. This suggests that program quality, in terms of student performance, may benefit from engagement in QI processes.
Accreditation and its impact on medical schools’ educational processes
The publications about accreditation and its impact on medical school's educational process are Blouin (2020), Blouin et al. (2018), Arja et al. (2018), and Nora (2022). Blouin et al. (2018) and Arja et al. (2021) each identified processes that are impacted by accreditation, including governance, data collection and analysis, creation and revision of policies and procedures, curricular reforms, continuous quality improvement, and faculty members' engagement in the educational program. Blouin (2020) identified new themes impacted by accreditation, including the undergraduate medical education (UME) program process, UME program quality, QA, QI, and research.
Accreditation and CQI
The accreditation systems are based on either the QA or QI approaches. In the QA approach, minimum standards are met to ensure the quality of education and training and meet the accreditation standards ( Akdemir et al., 2018). The QI method is based on striving for excellence of standards. The balanced accreditation must include both QA and QI. Medical school leaders should use the accreditation process as a journey for QI rather than as an evaluation that must be passed ( Alrebish et al., 2017). There is considerable debate about moving the accreditation systems from student outcomes to the CQI measures ( Barzansky et al., 2015; Blouin & Tekian, 2018). Accreditation should not be all about ticking the standards checklist at one point in time and should emphasize ongoing improvements and CQI. CQI can enhance program quality and outcomes if the process is designed to collect appropriate information and the results are used for program development. Medical schools should intermittently review their compliance with the accreditation standards, submit annual reports to the accreditation bodies, and act on the results to support CQI ( Barzansky et al., 2015). The ongoing improvement should include departmental reviews or reviews based on short-term and long-term measurable outcomes of strategic planning ( Stratton, 2019). CQI should be like envisioning and humanizing the process, sharing results with all stakeholders, expecting accountability from all stakeholders, and monitoring measures and outcomes ( Stratton, 2019).
Perceived QI implementation levels were low across programs in the 'strategy,' 'measurement/analysis/knowledge management’, and 'operations' domains ( Blouin, 2019). Medical schools' existing QI practices should be recognized as QI activities. In the execution of their QI visions, medical schools should spend resources on embedding quality in the organization's culture and reinforcing existing QI practices. Shroyer et al. (2016) showed an example of CQI 3D dashboards incorporating the CQI framework, which includes establishing shared goals, document compliance, reporting accomplishments, enriching communications, facilitating decisions, and improving performance. For QI strategies to succeed, a program's culture must embrace QI ( Blouin, 2019). With multiple stakeholder inputs, the self-study and evaluation process are key to building ownership in self-evaluation and goal setting within medical schools. The leaders and programs should invest resources and time for CQI. Medical schools should meet the accreditation standards and there should be an ongoing improvement of the program against the measurable outcomes/strategic plan. The CQI should be integrated into annual institutional and program reviews ( Stansfield et al., 2019).
Conclusions
Minimal evidence about accreditation and CQI exists. The number of studies in this field are limited and require further investigation about the impact of accreditation on CQI. The available literature underscores the necessity of transitioning from exclusively emphasizing student outcomes to embracing CQI metrics. CQI necessitates a collective sense of responsibility and accountability within the organization, with an unwavering focus on quality at every process stage. Furthermore, it underscores the pivotal role of leadership in facilitating enhancements in data collection and data analysis for quality improvement. Accreditation standards should align with CQI. One way to do this is to build a culture and vision that includes CQI in Medical Education. Finally, educational programs and leaders should adopt resources and faculty development promoting CQI that addresses larger issues and accreditation simultaneously.
Funding Statement
The author(s) declared that no grants were involved in supporting this work.
[version 2; peer review: 2 approved, 1 approved with reservations]
Data availability
Underlying data
The data for this article consists of bibliographic references, which are included in the References section.
Reporting guidelines
Figshare: “The PRISMA checklist-The impact of accreditation on continuous quality improvement process in undergraduate medical education programs: A scoping review.”, https://doi.org/10.6084/m9.figshare.25021646.v1 ( Arja, 2024).
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
References
- Acevedo-De-los-Ríos A, Rondinel-Oviedo DR: Impact, added value and relevance of an accreditation process on quality assurance in architectural higher education. Qual High Educ. 2022;28(2):186–204. 10.1080/13538322.2021.1977482 [DOI] [Google Scholar]
- Akdemir N, Ellwood DA, Walters T, et al. : Accreditation as a quality improvement tool: is it still relevant? Med J Aust. 2018;209(6):249–252. 10.5694/mja17.00934 [DOI] [PubMed] [Google Scholar]
- Akdemir N, Peterson LN, Campbell CM, et al. : Evaluation of continuous quality improvement in accreditation for medical education. BMC Med Educ. 2020;20(Suppl 1): 308. 10.1186/s12909-020-02124-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Al-Eyadhy A, Alenezi S: The impact of external academic accreditation of undergraduate medical program on students' satisfaction. BMC Med Educ. 2021;21(1): 565. 10.1186/s12909-021-03003-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Allen SK, Baalawi ZS, Al Shoaibi A, et al. : Applying North American medical education accreditation standards internationally in the United Arab Emirates. Med Educ Online. 2022;27(1): 2057790. 10.1080/10872981.2022.2057790 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Alrebish SA, Jolly BC, Molloy EK: Accreditation of medical schools in Saudi Arabia: a qualitative study. Med Teach. 2017;39(sup1):S1–S7. 10.1080/0142159X.2016.1254746 [DOI] [PubMed] [Google Scholar]
- Arja SB: The PRISMA checklist-The impact of accreditation on continuous quality improvement process in undergraduate medical education programs: a scoping review. figshare. Thesis,2024. 10.6084/m9.figshare.25021646.v1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Arja SB, Arja SB, Fatteh S, et al. : Impact of accreditation on Caribbean medical schools' processes. Can Med Educ J. 2021;12(4):79–88. 10.36834/cmej.71333 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Arksey H, O’Malley L: Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. 10.1080/1364557032000119616 [DOI] [Google Scholar]
- Barzansky B, Hunt D, Moineau G, et al. : Continuous quality improvement in an accreditation system for undergraduate medical education: benefits and challenges. Med Teach. 2015;37(11):1032–1038. 10.3109/0142159X.2015.1031735 [DOI] [PubMed] [Google Scholar]
- Bedoll D, van Zanten M, McKinley D: Global trends in medical education accreditation. Hum Resour Health. 2021;19(1): 70. 10.1186/s12960-021-00588-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blouin D: Quality improvement in medical schools: vision meets culture. Med Educ. 2019;53(11):1100–1110. 10.1111/medu.13926 [DOI] [PubMed] [Google Scholar]
- Blouin D: Accreditation of Canadian undergraduate medical education programs: a study of measures of effectiveness. Acad Med. 2020;95(6):931–937. 10.1097/ACM.0000000000003065 [DOI] [PubMed] [Google Scholar]
- Blouin D, Tekian A: Accreditation of medical education programs: moving from student outcomes to continuous quality improvement measures. Acad Med. 2018;93(3):377–383. 10.1097/ACM.0000000000001835 [DOI] [PubMed] [Google Scholar]
- Blouin D, Tekian A, Kamin C, et al. : The impact of accreditation on medical schools' processes. Med Educ. 2018;52(2):182–191. 10.1111/medu.13461 [DOI] [PubMed] [Google Scholar]
- Dattey K, Westerheijden DF, Hofman WHA: Impact of accreditation on public and private universities: a comparative study. Tert Educ Manag. 2014;20(4):307–319. 10.1080/13583883.2014.959037 [DOI] [Google Scholar]
- Educational Commission for foreign medical graduates. 2022. Reference Source
- Kritchevsky SB, Simmons BP: Continuous quality improvement. concepts and applications for physician care. JAMA. 1991;266(13):1817–1823. 10.1001/jama.266.13.1817 [DOI] [PubMed] [Google Scholar]
- Liaison Committee on Medical Education (LCME®): Functions and structure of a medical school: standards for accreditation of medical education programs leading to the MD degree.Washington, DC: LCME®,2022. Reference Source
- Loddenkemper R, Séverin T, Mitchell S, et al. : HERMES criteria for accreditation of European training centres: overcoming challenges of accreditation. Eur Respir J. 2010;36(6):1239–1241. 10.1183/09031936.00152310 [DOI] [PubMed] [Google Scholar]
- Nora LM: Using accreditation to transform diversity, equity, and inclusion efforts into diversity, equity, and inclusion systems. Acad Med. 2022;97(1):25–29. 10.1097/ACM.0000000000004377 [DOI] [PubMed] [Google Scholar]
- Radawski D: Continuous quality improvement: origins, concepts, problems, and applications. J Physician Assist Educ. 1999;10(1):12–16. 10.1097/01367895-199910010-00002 [DOI] [Google Scholar]
- Roy M, Wood TJ, Blouin D, et al. : The relationship between accreditation cycle and licensing examination scores: a national look. Acad Med. 2020;95(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S103–S108. 10.1097/ACM.0000000000003632 [DOI] [PubMed] [Google Scholar]
- Salto DJ: Quality assurance through accreditation: when resistance meets over-compliance. High Educ Q. 2018;72(2):78–89. 10.1111/hequ.12151 [DOI] [Google Scholar]
- Shroyer AL, Lu WH, Chandran L: Drivers of Dashboard Development (3-D): a curricular Continuous Quality Improvement approach. Acad Med. 2016;91(4):517–521. 10.1097/ACM.0000000000001078 [DOI] [PubMed] [Google Scholar]
- Simpson DE, Golden DL, Rehm JM, et al. : The costs versus the perceived benefits of an LCME institutional self-study. Acad Med. 1998;73(9):1009–1012. 10.1097/00001888-199809000-00024 [DOI] [PubMed] [Google Scholar]
- Stansfield RB, Markova T, Baker R: Integration of continuous quality improvement methods into annual program and institutional evaluation. J Grad Med Educ. 2019;11(5):585–591. 10.4300/JGME-D-19-00145.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stratton TD: Legitimizing Continuous Quality Improvement (CQI): navigating rationality in undergraduate medical education. J Gen Intern Med. 2019;34(5):758–761. 10.1007/s11606-019-04875-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tackett S, Boulet JR, van Zanten M: Medical School accreditation factors associated with certification by the Educational Commission for Foreign Medical Graduates (ECFMG): a 10-year international study. Acad Med. 2021;96(9):1346–1352. 10.1097/ACM.0000000000004047 [DOI] [PubMed] [Google Scholar]
- Tackett S, Zhang C, Nassery N, et al. : Describing the evidence base for accreditation in undergraduate medical education internationally: a scoping review. Acad Med. 2019;94(12):1995–2008. 10.1097/ACM.0000000000002857 [DOI] [PubMed] [Google Scholar]
- The Joanna Briggs Institute Reviewers’ Manual: Methodology for JNI scoping reviews.Institute TJB,2015. Reference Source
- Tricco AC, Lillie E, Zarin W, et al. : PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–473. 10.7326/M18-0850 [DOI] [PubMed] [Google Scholar]
- van Zanten M: The association between medical education accreditation and the examination performance of internationally educated physicians seeking certification in the United States. Perspect Med Educ. 2015;4(3):142–145. 10.1007/s40037-015-0183-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- van Zanten M, McKinley D, Durante Montiel I, et al. : Medical education accreditation in Mexico and the Philippines: impact on student outcomes. Med Educ. 2012;46(6):586–592. 10.1111/j.1365-2923.2011.04212.x [DOI] [PubMed] [Google Scholar]
- van Zanten M, Norcini JJ, Boulet JR, et al. : Overview of accreditation of undergraduate medical education programmes worldwide. Med Educ. 2008;42(9):930–937. 10.1111/j.1365-2923.2008.03092.x [DOI] [PubMed] [Google Scholar]
- Volkwein JF, Lattuca LR, Harper BJ, et al. : Measuring the impact of professional accreditation on student experiences and learning outcomes. Res High Educ. 2007;48(2):251–282. 10.1007/s11162-006-9039-y [DOI] [Google Scholar]
- Ward CS, Andrade A, Walker-Winfree L: Implementing a course review process for a continuous quality improvement model for a medical school curriculum. J Health Care Poor Underserved. 2018;29(1):63–70. 10.1353/hpu.2018.0007 [DOI] [PubMed] [Google Scholar]
- White J, Paslawski T, Kearney R: 'Discovery learning': an account of rapid curriculum change in response to accreditation. Med Teach. 2013;35(7):e1319–e1326. 10.3109/0142159X.2013.770133 [DOI] [PubMed] [Google Scholar]
- World Health Organization: Global strategy on human resources for health: workforce 2030.Geneva: World Health Organization;2016. Reference Source