Advances in the ACGME Clinical Learning Environment Review (CLER) Program

Editor's Note: The ACGME News and Views section of JGME includes data reports, updates, and perspectives from the ACGME and its review committees. The decision to publish the article is made by the ACGME.

Introduction

The Clinical Learning Environment Review (CLER) program was launched in late 2012 as a key component of the Accreditation Council for Graduate Medical Education's (ACGME's) Next Accreditation System. The goal of CLER is to provide formative feedback to sponsoring institutions (SIs) on the effectiveness of resident and fellow engagement in 6 focused areas (box), with the intent of continually improving institutional performance in these areas.^1,2 The CLER program is intended to serve as a foundation for the Next Accreditation System (figure 1). The only accreditation requirement associated with this new program is that each SI must undergo a CLER visit every 18 to 24 months to maintain accreditation. Upon this foundation of formative feedback and continuous improvement of the clinical learning environment rests the continuous accreditation activities for each SI, residency, and fellowship program. These include annual reviews of program data, a required 10-year self-study process, and a 10-year self-study accreditation site visit.

FIGURE 1.

The Building Blocks or Components of the Next Accreditation System

Abbreviations: prn, (pro re nata) according to circumstances; RRC, Residency Review Committee; IRC, Institutional Review Committee.

Box Clinical Learning Environment Review Program—Areas of Focus

• Patient Safety—Opportunities for residents to report errors, unsafe conditions, and near misses and to participate in interprofessional teams to promote and enhance safe care

• Quality Improvement—How the sponsoring institution (SI) engages residents in reducing health care disparities, and improving patient outcomes

• Transitions in Care—How the SI demonstrates effective transitions of care

• Supervision—How the SI maintains supervision

• Duty Hours Oversight, Fatigue Management, and Mitigation—How the SI demonstrates effective and meaningful oversight of duty hours across all programs; designs systems and provides settings that facilitate fatigue management and mitigation; and provides effective education of faculty and residents in sleep and in recognition and fatigue of fatigue

• Professionalism—How the SI educates and monitors specific areas of professionalism of its residents and faculty

The CLER was established with 2 learning objectives for both ACGME and the graduate medical education (GME) teaching institutions. The first is to learn how to best optimize patient safety and clinical quality in clinical environments where residents and fellows are providing care. The second entails learning how to best prepare physicians-in-training to meet the needs of health care systems in the 6 focus areas. This is relevant to safety and quality of care, both in teaching settings and in the settings where physicians practice after completing their training. Through the CLER program, ACGME proposes to learn from the GME community and to share that learning back with the community.

During the past year, the program has undergone initial development (alpha testing) and early deployment (beta testing). A detailed view of the CLER program and site visit protocol has been the subject of an earlier publication.³ The purpose of this article is to provide an update on the CLER program, including some initial observations and insights into upcoming developments for 2014.

The CLER Program 1 Year After Its First Visit

It has been approximately 1 year since the CLER program conducted its first site visit. That visit was the first of several alpha tests conducted in the process of establishing the site visit protocol. Objectives of the alpha test included the following: (1) to develop a set of survey instruments to assess each of the 6 focus areas; (2) to test the feasibility of carrying out a visit on short notice that included group meetings with executive leadership, residents/fellows, faculty members, and program directors; and (3) to assess the capacity of the site visit team to rapidly assimilate and synthesize information about the clinical learning environment and to prepare an oral report of that experience in time for an exit interview at the end of the site visit. Additional objectives included testing the ability of the site visit team to provide a meaningful exit interview and prepare a written report that could be used by the clinical site to improve resident and fellow integration into activities across the 6 focus areas.

The CLER site visit protocol has demonstrated its utility and emerged from its alpha testing. The current, extensive beta-testing process will continue through a complete cycle of visits to the primary clinical site of each of the more than 300 SIs with multiple accredited programs. The overall aim of this beta phase is to further test and improve the site visit protocol until a fully mature process has been developed for long-term use. Within this aim, the objectives of the beta test phase are (1) to test the protocol in a broad range of clinical sites and SIs; (2) to demonstrate the ability to scale the visit process to meet a planned schedule of visits to every SI every 18 to 24 months; (3) to gather feedback on the value of the site visit process, including feedback on the CLER oral and written reports; (4) to evaluate the usefulness of a feedback process by which the SI provides the CLER program with a response to the written report; and (5) to test the validity and reliability of the CLER program.

More than 80 hospitals and medical centers across more than 40 states (figure 2) have now received a CLER site visit. These visits have been to general acute care hospitals and medical centers and some specialty hospitals, and have included university-based medical centers and independent academic teaching institutions, public and private hospitals, and clinical sites that sponsor a large number of residency and fellowship programs, as well as sites that sponsor a modest number of programs.

FIGURE 2.

CLER Program: States Where Alpha and Early Beta Visits Were Completed

Abbreviation: CLER, Clinical Learning Environment Review.

States where alpha visits were completed (white ovals): NV, OH, PA, VA, WA.

States where beta visits were completed (black ovals): AL, CA, CO, CT, FL, GA, IA, IL, KS, KY, LA, MA, MD, ME, MI, MN, MO, MS, NC, ND, NE, NJ, NM, NY, OR, PA, SC, SD, TN, TX, VT, WI, WV.

Both alpha and beta visits were conducted in Pennsylvania.

To date, the general response from the organizations that have had visits has been positive. The visit experience has matched the descriptions provided by the ACGME in national presentations, such as the webinars on CLER.^4,5 The overall feedback received to date is that the verbal report given at the exit meeting of the visit, along with the subsequent written report, has assisted the leadership of the SI and its participating site to self-identify strengths and opportunities for improving the clinical learning environment related to 1 or more of the 6 focus areas. Through the optional response process, many SIs have provided feedback to the program and indicated that their clinical and GME leadership are actively using the CLER visit experience to implement improvements.

Testing the Volunteer CLER Site Visitor Component

In the fall of 2013, the CLER site visit team advanced toward its full complement of field representatives. The program anticipates to be at full capacity (conducting 6 or more visits weekly) by the end of the first quarter of 2014.

The core resource for the CLER program is its full-time staff, including 3 regional vice presidents and a team of field representatives dedicated to the CLER program. Each site visit will be led by 1 member of the full-time CLER team.

ACGME is currently testing the addition of volunteer site visitors as members of the visit team. The volunteer component allows senior leaders from the GME community and executive leadership from teaching institutions to be part of the CLER site visit process. By including such individuals as members of the site visit team, the CLER program seeks to enhance the validity of the experience and to create a new social learning network that can accelerate opportunities for improvement. Beginning in May 2013, the CLER program explored the use of volunteer site visitors. The initial experience has been very positive. Plans are underway to scale up this component after additional testing in the coming months.

Early Observations on the 6 CLER Focus Areas

Although a comprehensive review of early observations is currently underway, a few general observations can be made at this time. These observations are presented here, with the caveat that they are qualitative and may be subject to bias because of the small sample (less than 20% of all SIs) and because the sample includes a number of institutions that volunteered for early visits.

In general, the site visit teams have encountered highly motivated clinical leadership, a very enthusiastic GME community, and CEOs and executive leadership who are working under enormous pressure to adapt to a rapidly changing health care environment. The visits have also encountered a nursing community and communities of other professionals that appear to be supportive of GME. Generally, there appears to be a deep commitment to GME and a great desire to provide excellence in clinical care and education, although the motivations for commitment appear to be unique to each hospital and medical center.

Overall, there is consistent evidence of resident and fellow engagement in quality improvement activities. However, the depth and breadth of that involvement is highly variable. Often the quality improvement efforts of the residents and fellows are not related to the clinical site's quality goals.

In the area of patient safety, there is an overall gap in resident and faculty engagement with the clinical site's patient safety initiatives, starting with the ability to recognize patient safety events. Resident and fellow engagement in the full course of patient safety event analysis, action plan development, and follow-up is uncommon in most clinical sites.

Most of the clinical sites' efforts in health care disparities center primarily on improving access to care, assessing and addressing community needs, and providing language translator services. Very few CLER visits to date have identified institutional strategies or widespread efforts to examine possible differences in care or outcomes for the vulnerable populations served by the clinical site. Resident, fellow, and faculty involvement in institutional efforts to reduce health care disparities varies.

Examination of the other focus areas (oversight of supervision, transitions in care, duty hours/fatigue, and professionalism) has demonstrated similar variations and gaps that provide opportunities for improvement.

It appears that successful improvement will only be accomplished if the clinical site's executive leadership, GME leadership, faculty, residents, and fellows work closely to address these opportunities.

The CLER Evaluation Committee and the Emerging “CLER Pathways to Excellence”

The CLER program is still in its early stages of development. It will continue beta testing the site visit protocol and provision of feedback through completion of the first cycle of all ACGME SIs that sponsor multiple programs. The next step in the evolution of the CLER program will be driven by the work of the CLER Evaluation Committee. This ACGME committee, which oversees the CLER program, is in the final stages of developing a guidance document called the CLER Pathways to Excellence: Expectations for an Optimal Clinical Learning Environment to Achieve Safe and High-Quality Patient Care, with a planned release in early 2014. This document is designed to provide the GME community and the public with a set of specific guidance statements to address resident and fellow engagement in each of the CLER program's 6 focus areas. The CLER site visit protocol has been refined to harmonize with this document and there are plans to begin testing this updated version of the protocol (beta version 2.0) in the fourth quarter of 2013, with full deployment expected in early 2014.

What to Expect From the CLER Program in 2014

It is anticipated that by the end of the first quarter of 2014, the program will be operating at full capacity, conducting between 6 and 8 visits per week. As noted above, the Pathways for Excellence document will be released early in the year. Later in the year, the program will start to release various reports presenting aggregate data and deidentified summaries of the findings from the site visits conducted during 2013. Later in 2014, pilot testing will begin for the single and small program SIs.

In addition to the ACGME's efforts to implement the CLER site visits, the CLER program is working to convene a community of key stakeholders and to foster partnerships with, and among, a number of organizations to provide faculty and professional education in the form of programs and learning materials to support continuous improvement of the clinical learning environment. Some of the partner organizations include the Association of American Medical Colleges, the Institute for Healthcare Improvement, the Alliance of Independent Academic Medical Centers, and the Association for Hospital Medical Education. Each of these organizations has begun new programs or initiatives focused on 1 or more of the CLER focus areas.

Conclusion

In the past year, CLER has evolved from concept to a deployed program. These initial efforts have started a new national conversation about the value of, and strategies for, enhanced integration and engagement of the GME community within health care organizations. ACGME and CLER leaders hope that this national conversation will serve as the starting place for new efforts and innovations to simultaneously improve the safety and quality of patient care while improving the quality of graduate medical education.