Development, Testing, and Implementation of the ACGME Clinical Learning Environment Review (CLER) Program

Abstract

Since the release of the Institute of Medicine's report on resident hours and patient safety, there have been calls for enhanced institutional oversight of duty hour limits and of efforts to enhance the quality and safety of care in teaching hospitals. The ACGME has established the Clinical Learning Environment Review (CLER) program as a key component of the Next Accreditation System with the aim to promote safety and quality of care by focusing on 6 areas important to the safety and quality of care in teaching hospitals and the care residents will provide in a lifetime of practice after completion of training. The 6 areas encompass engagement of residents in patient safety, quality improvement and care transitions, promoting appropriate resident supervision, duty hour oversight and fatigue management, and enhancing professionalism.

Over the coming 18 months the ACGME will develop, test, and fully implement this new program by conducting visits to the nearly 400 clinical sites of sponsoring institutions with two or more specialty or subspecialty programs. These site visits will provide an understanding of how the learning environment for the 116 000 current residents and fellows addresses the 6 areas important to safety and quality of care, and will generate baseline data on the status of these activities in accredited institutions. We expect that over time the CLER program will serve as a new source of formative feedback for teaching institutions, and generate national data that will guide performance improvement for United States graduate medical education.

Introduction

In 2010, during the deliberation on the duty hour standards that were implemented as part of the 2011 Common Program Requirements, the Board of Directors of the Accreditation Council for Graduate Medical Education (ACGME) authorized the development of a new program of periodic site visits to all sponsoring institutions focusing on their responsibility for the quality and safety of their learning and patient care environment.¹ In response, the ACGME established the CLER program as a component of its Next Accreditation System (NAS) to assess the learning environment of each sponsoring institution and its participating sites. The CLER program emphasizes the importance of providing a learning environment that engages residents and fellows in institutional efforts in patient safety and health care quality, a key dimension of the 2011 ACGME Common Program Requirements. The intent of CLER is “to generate national data on program and institutional attributes that have a salutary effect on quality and safety in settings where residents learn and on the quality of care rendered after graduation.”¹

It is anticipated that the CLER program, through its frequent regular on-site sampling of the learning environment:

• Will increase the educational emphasis on new competencies demanded by the public;

• Will provide opportunity for sponsoring institutions to demonstrate leadership in patient safety, quality improvement, and reduction in health care disparities; and

• Will permit lengthening the interval for standard ACGME site visits of individual programs if other parameters of program performance are at satisfactory levels.

The CLER Program

The CLER program's ultimate goal is to move from an initial focus on duty hours to a broader focus on the GME learning environment and how it can deliver high quality, safe patient care and physicians prepared to contribute to health system improvement over a lifetime of practice.

The CLER program consists of 3 related activities: The CLER site visit, the CLER Evaluation Committee, and support for faculty and leadership development in the areas emphasized by the program.

The CLER Site Visit

The site visit is the core of the CLER program, scheduled to occur on an ongoing basis every 18 months. This visit will initially focus on evaluating each sponsoring institution's primary clinical site the with regard to engagement of residents and fellows in 6 focal areas. The 6 areas (Box 1) are assessed via 5 overarching questions (Box 2). Ultimately, the CLER visits will encompass assessment of the clinical learning environment in each of the major participating sites where resident education occurs. The visit is designed to start and end with a discussion with the executive leadership of the clinical site, including the Chief Executive Officer, other members of executive management, the Designated Institutional Official, the Chair of the GME Committee, and a resident representative.

Box 1 Six Areas of Focus for the CLER Program

• Patient Safety – including opportunities for residents to report errors, unsafe conditions, and near misses, and to participate in inter-professional teams to promote and enhance safe care.

• Quality Improvement – including how sponsoring institutions engage residents in the use of data to improve systems of care, reduce health care disparities and improve patient outcomes.

• Transitions in Care – including how sponsoring institutions demonstrate effective standardization and oversight of transitions of care.

• Supervision – including how sponsoring institutions maintain and oversee policies of supervision concordant with ACGME requirements in an environment at both the institutional and program level that assures the absence of retribution.

• Duty Hours Oversight, Fatigue Management, and Mitigation – including how sponsoring institutions: (1) demonstrate effective and meaningful oversight of duty hours across all residency programs institution-wide; (2) design systems and provide settings that facilitate fatigue management and mitigation; and (3) provide effective education of faculty members and residents in sleep, fatigue recognition, and fatigue mitigation.

• Professionalism—with regard to how sponsoring institutions educate and monitor professionalism of their residents and faculty.

Box 2 Central Questions in This Initial Cycle of CLER Evaluations

• Who and what form is the infrastructure of a Sponsoring Institution's (SI) clinical learning environment? What organizational structures and administrative and clinical processes do the SI and its major participating sites have in place to support GME learning in each of the 6 focus areas?

• How integrated is the GME leadership and faculty within the SI's current clinical learning environment infrastructure? What is the role of GME leadership and faculty to support resident and fellow learning in each of the 6 areas?

• How engaged are the residents and fellows in using the SI's current clinical learning environment infrastructure? How comprehensive is the involvement of residents and fellows in using these structures and processes to support their learning in each of the 6 areas?

• How does the SI determine the success of its efforts to integrate GME into the quality infrastructure? From the perspective of the SI and its major participating sites, what are the measures of success in using this infrastructure and what was the level of success?

• What areas has the SI identified as opportunities for improvement? From the perspective of the SI and its major participating sites (if different), what are seen as the opportunities for improving the quality and value of the current clinical learning environment infrastructure to support the 6 focus areas?

The purpose of the opening discussion with senior leadership is to inform the site visit team of relevant institutional policies and solicit baseline impressions of how the executive leadership perceives their performance across the 6 focal areas. At the exit meeting, the site visit team will provide senior leadership with some initial feedback on their findings. Between these two bookended meetings with leadership, the site visit team will meet with residents and fellows, GME faculty, and other key members of hospital leadership. The site visitors will also conduct walking tours of clinical areas (eg, clinical inpatient and outpatient areas, operating room, intensive care, emergency departments), using this opportunity to interact with residents, faculty, nurses, and other health care professionals, and possibly patients.

The CLER Evaluation Committee

The CLER Evaluation Committee is designed to be distinct from the ACGME Review Committees. The goal of the Review Committees is to examine programs and institutions and issue accreditation decisions based on adherence to an established set of requirements. The charge to the CLER Evaluation Committee is to set expectations for the 6 focus areas and provide institutions with formative feedback from the site visits. The Evaluation Committee will not issue accreditation decisions. Rather, its purpose is to provide sponsoring institutions, their participating sites, and the ACGME Review Committees with valuable insights about the level of GME engagement in institutional initiatives across the 6 focus areas. For the first cycle of site visits (18 months), any information shared with the ACGME and its review committees will be de-identified and/or reported in aggregate. Beginning with the second cycle, it is anticipated that the Evaluation Committee will begin to share relevant information from the CLER site visits with the Institutional Review Committee and other ACGME Review Committees to help inform the Next Accreditation System.

The CLER Evaluation Committee is composed of a broad range of regional and national experts in patient safety, health care quality, and fatigue management. It also includes inter-professional and public representatives. The first in-person meeting of the Committee is scheduled to take place in the fall 2012.

Faculty and Leadership Development

The ACGME recognizes that sponsoring institutions and the GME community at large have a growing need to support faculty development, particularly in the areas of patient safety and health care quality. In response to this need, the ACGME, in collaboration with other key organizations, will seek to develop resources to educate and support faculty and executive leadership across the 6 focus areas.

Developing, Testing, and Implementing the CLER Program

As with other new programs, there is a need to develop and test prior to full scale implementation. The CLER program is no exception. In designing the conceptual framework for the CLER site visit and evaluation processes, ACGME took into consideration the recommendations of a national advisory committee as well as input from 5 focus groups with members of the Designated Institutional Official community. The draft framework for CLER was presented to the ACGME Board of Directors for feedback in June of 2012. Using the framework as a basis, the CLER program staff drafted a site visit protocol that was further shaped and refined though a series of alpha tests conducted at volunteer sponsoring institutions during the summer of 2012. The results of this initial phase of testing will be presented to the CLER Evaluation Committee for review and feedback.

Beta testing of the site visit protocol is set to begin in the fall of 2012 and will target the nearly 400 institutions that have 2 or more residency or fellowship programs. Current plans call for at least 3 phases of beta testing. The first phase will examine the complete review cycle from site visit through Committee evaluation and final report back to the sponsoring institution. The second phase of beta testing will examine the impact of scaling up the number of monthly visits to inform the build out of a cadre of site visitor field staff. The final phase of beta testing will seek to optimize the use of aggregate data obtained from the CLER site visits to extract (or discover) common/normative and salutary practices in the 6 focus areas, and incorporate this information in the feedback to the sponsoring institution, as well as share it nationally in aggregated and/or de-identified form. Sponsoring Institutions with single residency or fellowship programs will not be part of the first cycle of CLER visits.

The CLER program is intended to be a formative evaluation experience, providing the leadership of the sponsoring institution and the executive leadership of the participating clinical site(s) with information that they can use to improve resident training and engagement with issues that are of national concern. As such, the CLER program is testing a process that will provide the institution with 3 opportunities to receive formative feedback. First, the sponsoring institution will receive an oral report at the time of the exit interview. After the site visit, the leadership of the sponsoring institution will have an opportunity to review and respond to the site visitor's written report prior to its submission to the CLER Evaluation Committee. Lastly, the sponsoring institution will receive a final report once the Committee has concluded their evaluation. While this approach may be modified as a result of the alpha and beta tests, it is likely that the final review process will retain these basic features and remain focused on setting expectations and providing recommendations rather than issuing accreditation decisions.

Assessing the Impact of CLER

During the initial cycle of alpha and beta-testing, the CLER Evaluation Committee will work with ACGME staff and the GME community to determine the best approach for assessing the impact of the CLER program. Stakeholders of the CLER Program include residents, fellows, faculty, GME and executive leadership of sponsoring institutions and their participating sites, the GME community, national leadership in patient safety and health care quality movement, and the public. In response, a wide range of measures to address the many stakeholders will be considered.

Conclusion

Over the next 18 months, the ACGME will gain experience with the newly established CLER program, and will gain knowledge about how clinical sites are supporting the training of residents and fellows in the areas of patient safety, health care quality (including issues of disparities), supervision, transitions in care, duty hours and fatigue management and mitigation, and professionalism. The public seeks assurance that GME is effectively preparing the next generation of US physicians to deliver high quality health care in an increasingly complex environment. CLER is an essential element of the NAS, designed to provide components of that assurance to the public we serve, and it is anticipated that the information from the CLER program, over time, will promote performance improvement in the training of the US physician workforce.