Detailed Findings From the CLER National Report of Findings 2018

Introduction

This section includes detailed findings from the second set of visits (2015–2017) of the Clinical Learning Environment Review (CLER) Program. The findings in the 6 CLER Focus Areas¹ are based on site visits to the major participating clinical sites (ie, hospitals and medical centers) for 287 Accreditation Council for Graduate Medical Education (ACGME)-accredited Sponsoring Institutions (SIs) with 3 or more core residency programs.^2,3 These clinical sites serve as clinical learning environments (CLEs) for the SIs.

Collectively, the 287 SIs oversee 9167 ACGME-accredited residency and fellowship programs, with a median of 20 programs per SI. These larger SIs account for 87.1% of all residents and fellows in ACGME-accredited programs—with a range of 17 to 2156 trainees per SI (median = 246).

Approximately 28% of the CLEs were located in the Northeast region of the United States, 30.3% in the South, 26.5% in the Midwest, and 14.6% in the West. The sites ranged in size from 107 to 2654 acute care beds (median = 528). The majority (67.2%) were nongovernment, not-for-profit organizations; 23.3% were government, nonfederal; 5.9% were investor-owned, for-profit; and 3.5% were government, federal. Although the CLER teams spent the majority of their time at inpatient settings, they also sometimes visited affiliated ambulatory care practices in close proximity.

In total, the CLER teams interviewed more than 1600 members of executive leadership (including chief executive officers), 9262 residents and fellows, 8164 core faculty members, and 6034 program directors of ACGME-accredited programs in group meetings. Additionally, the CLER teams interviewed the CLEs' leadership in patient safety and health care quality and thousands of residents, fellows, faculty members, nurses, pharmacists, social workers, and other health care professionals while on walking rounds in the clinical areas.

As previously described in the CLER National Report of Findings 2016,⁴ these findings are based on a mixed methods approach to data gathering and analysis to improve the accuracy of the findings by combining quantitative, descriptive, and qualitative evidence in a complementary manner. As such, some of the findings are represented quantitatively while others are described qualitatively.

The combination of methodologies and varied representation of findings should be considered when interpreting the results, making comparisons, or drawing conclusions. Both supporting and conflicting evidence may be presented to explain or qualify findings. For example, results from the group interviews may appear more positive than information gathered on walking rounds. Alternatively, practices reported during group interviews may have been verified on walking rounds.

Interpreting Quantitative Results From the Group Interviews

During the group interviews with residents and fellows, faculty members, and program directors, an electronic audience response system (ARS; Keypoint Interactive version 2.6.6, Innovision Inc, Commerce Township, MI) was used to collect anonymous responses to closed-ended questions. The results from the ARS were analyzed at both the individual (eg, residents and fellows) and the CLE levels.

At the individual level of analysis, results are presented as percentages of the total number of individuals surveyed. For example:

“In the group interviews, 33.6% of residents and fellows reported that they had received cultural competency training that was specific to populations at risk for health care disparities at their clinical site.”

At the CLE level of analysis, individual responses were aggregated at the CLE level and results are presented as median and interquartile range (IQR) percentages. For example:

“Across CLEs, a median (IQR) of 32.8% (23.3%–46.4%) of the residents and fellows indicated that they had received cultural competency training that was specific to populations at risk for health care disparities at their clinical site.”

Statistically significant differences (ie, P ≤ .05) in responses due to resident and fellow characteristics (eg, residency year) and CLE characteristics (eg, bed size) are also reported. Of note, statistical significance does not always imply practical significance. For example, differences in responses by residency year may be statistically significant but the differences may not be meaningful or large enough to have practical relevance or implications.

Additional Considerations

As described in the Methodology section,⁵ this report contains a specific set of descriptive terms that summarize quantitative results from both the ARS and specific findings that were quantified from the site visit reports. These terms and their corresponding quantitative ranges are as follows:

few (< 10%), some (10%–49%), most (50%–90%), and nearly all (> 90%)

Besides the quantitative data, this report contains qualitative data from a number of open-ended questions that CLER Site Visitors asked during group interviews and walking rounds. This information, by design, was not intended to be enumerated. For these questions, the site visit teams made an assessment of the relative magnitude of observations at each individual site. To prevent confusion, these results are presented in the report using a set of descriptive terms different from the previously described terms used for quantitative data. The qualitative descriptive terms, which are intended to approximate the quantitative terms above, are as follows:

uncommon or limited, occasionally, many, and generally

Finally, this section follows approximately the same structure as the individual CLER Site Visit reports received by participating institutions. This structure is intended to facilitate easy comparison between data from an individual site and that of this report, which aggregates results from all 287 SIs. Those who seek additional detail may consult the Appendices (p. 81–124). Appendix A contains additional information on the SIs, sites visited, and groups interviewed, Appendix B contains selected aggregated quantitative results from the group interviews with residents and fellows, and Appendix C contains qualitative information from the group interviews and walking rounds.

Patient Safety

The CLER Program explored several aspects of resident and fellow engagement in patient safety with emphasis on 5 major topics: culture of safety, use of the patient safety event reporting system, knowledge of patient safety principles and methods, inclusion in patient safety event investigations, and disclosure of patient safety events. Generally across CLEs, members of the executive leadership team identified patient safety as their highest priority area for improvement.

Culture of Safety

The patient safety and quality leaders in many CLEs indicated that they periodically conduct a culture of safety survey that includes residents, fellows, and faculty members. Overall, 97.7% of the residents and fellows in the group interviews reported that their CLE provides a safe and nonpunitive environment for reporting errors.

Across CLEs, physicians and other staff members also reported use of the patient safety event reporting system to report on individual behaviors. This use included reporting on behaviors in a retaliatory fashion or in a manner that could be perceived as punitive.

Given this and based on the collective findings from the site visits, it is unclear as to whether residents, fellows, and other staff members perceived a safe and nonpunitive culture for reporting patient safety events.

Use of the Patient Safety Event Reporting System

CLE Systems for Reporting

Overall, CLEs had 1 or more mechanisms for reporting patient safety events, including an online or paper-based patient safety event reporting system, a chain-of-command system that allowed events to be reported to an immediate supervisor (eg, a more senior resident or faculty member), and a mechanism to verbally report events to the patient safety staff (eg, hotline).

In general, residents and fellows appeared to be aware of their CLE's process for reporting patient safety events such as adverse events, near misses/close calls, and unsafe conditions. During walking rounds, the CLER Site Visit teams also asked nurses about their CLE's patient safety event reporting system. Across nearly all CLEs (97.2%), nurses appeared to be familiar with their CLE's system for reporting patient safety events.

Approximately 78% of CLEs were able to provide information on the number of patient safety event reports submitted by residents and fellows (see Appendix C1), and 70.7% were able to provide the number of patient safety event reports submitted by attending physicians. The remaining CLEs indicated that their system did not track such information. Whereas CLEs occasionally provided the Graduate Medical Education Committee and their governing body with information on the number or percentage of patient safety event reports submitted by residents and fellows, it was less common for them to routinely report the number or percentage of patient safety event reports submitted by faculty members to these same groups.

Understanding of Reportable Events

Generally across CLEs, the residents and fellows interviewed on walking rounds appeared to lack understanding and awareness of the range of reportable patient safety events, including what defines a near miss/close call. In most CLEs (83.6%), nurses' understanding of reportable patient safety events also varied (see Appendix C2).

Across CLEs, residents, fellows, and nurses appeared to focus on reporting sentinel events, medication errors, patient falls, and other events with harm; they did not appear to recognize near misses/close calls, unsafe conditions, events without harm, unexpected deteriorations, or known procedural complications as reportable patient safety events. Residents, fellows, and nurses appeared to have little awareness of the importance of reporting these events and how such reporting can provide valuable information for identifying system failures, addressing vulnerabilities in the system, reducing risks, and improving patient safety.

Reporting

Overall, 72.7% of the residents and fellows in the group interviews indicated that they had experienced an adverse event, near miss/close call, or unsafe condition while at their CLE. This experience varied by gender, year of training, and specialty grouping (see Appendix B1).

Of the residents and fellows who reported that they had experienced an adverse event, near miss/close call, or unsafe condition, 49.8% indicated that they had personally reported the patient safety event using the CLE's patient safety event reporting system. Responses varied by gender, year of training, and specialty grouping. Across CLEs, the median (IQR) finding was 50.0% (37.5%–66.7%) and varied by region, CLE bed size, and type of ownership (see Appendix B2). For those who did not personally enter the patient safety event into the system, 13.6% indicated that they relied on a nurse to submit the patient safety event report, 24.4% indicated that they relied on a physician supervisor, and 12.1% indicated that they cared for the patient and chose not to submit a report.

When faculty members and program directors in the group interviews were asked what process residents and fellows most frequently followed when reporting a patient safety event, 57.9% of the faculty members and 53.7% of the program directors indicated that they believed residents and fellows most often reported the event themselves using the CLE's patient safety event reporting system.

In a separate query, 23.6% of the residents and fellows in the group interviews indicated that they had reported a near miss/close call event while at the CLE; responses varied by gender, year of training, and specialty grouping (FIGURE 1). Across CLEs, this finding ranged from 0% to 100%, with a median (IQR) of 23.1% (15.2%–33.3%); responses varied by region and type of ownership (see Appendix B3).

FIGURE 1.

Percentage of Residents and Fellows Who Reported a Near Miss/Close Call Event, by Specialty Group

On walking rounds, residents and fellows in many CLEs mentioned that they often report patient safety events locally or through their chain of command while also indicating familiarity with the patient safety event reporting system and its use. When they delegated or relied on others to report, it was unclear if these reports were formally captured in the CLE's centralized patient safety event reporting system. Residents and fellows mentioned the cumbersome process of submitting a report, the time needed to enter a report, fears of repercussion, and the uncertainty of receiving feedback as reasons for not reporting. The collective information from the site visits indicated that in 70.6% of the CLEs, resident and fellow reporting of patient safety events into the CLE's patient safety event reporting system was varied or infrequent (see Appendix C3).

In the group interviews, the CLER teams also explored faculty members' and program directors' use of the CLE's patient safety event reporting system. Approximately 36% of the faculty members reported that they had personally reported an adverse event, near miss/close call, or unsafe condition in the past year (median [IQR], 35.7% [26.0%–46.6%] across CLEs). Among the program directors, 35.9% reported that they had personally reported an adverse event, near miss/close call, or unsafe condition in the past year (5.5% had no clinical responsibilities at the site). Across CLEs, the median (IQR) finding was 36.0% (27.3%–50.0%). In both groups, responses varied by CLE bed size and type of ownership.

Feedback

In the group interviews, the CLER teams asked residents and fellows whether they received feedback on patient safety event reports. Of those who had experienced an adverse event, near miss/close call, or unsafe condition and who had personally submitted a patient safety event report or relied on a nurse or supervisor to submit the report, 46.1% reported that they received feedback on the outcome of the report. Responses varied by gender, specialty grouping, and year of training (FIGURE 2; see also Appendix B4).

FIGURE 2.

Percentage of Residents and Fellows Who Reported Receiving Feedback on the Outcome of a Patient Safety Event Report Submitted, by Level of Training

Abbreviation: PGY, postgraduate year.

Residents and fellows often mentioned receiving an e-mail acknowledging receipt of the patient safety event report. They also noted receiving requests for additional information as part of a formal patient safety event investigation. It was uncommon for residents to mention receiving information on the outcome of the investigation, including recommended actions to address vulnerabilities in the system and to improve patient safety. Across CLEs, residents, fellows, nurses, and other clinical staff expressed a strong desire to receive feedback in response to submitting a patient safety event report.

Overall, CLEs varied in their processes for reviewing and prioritizing patient safety events. Residents and fellows also varied in their knowledge of these processes and often used the term “black box,” indicating that these processes were unclear. Many residents and fellows appeared to be unaware of how their CLEs use the reporting of adverse events, near misses/close calls, or unsafe conditions to improve care both broadly and at the individual departmental level. Residents and fellows were rarely involved in their CLE's process for reviewing and prioritizing patient safety events that required further investigation.

Time-Outs

On walking rounds, the CLER teams explored resident and fellow participation in the time-out process as part of patient safety practices (eg, ambulatory and bedside procedures). Across many CLEs, residents, fellows, nurses, and other health care professionals interviewed on walking rounds indicated that residents and fellows do not consistently conduct standardized time-outs before performing bedside procedures.

Knowledge of Patient Safety Principles and Methods

Across most CLEs (91.6%), residents and fellows appeared to have limited knowledge of fundamental patient safety principles and methods (eg, Swiss cheese model of system failure, root cause analysis, fishbone diagrams; see Appendix C4).

When asked to identify their skills in applying patient safety principles, the majority of the faculty members indicated that they were either proficient or expert (62.7% and 25.1%, respectively) in applying these skills. Similarly, most of the program directors reported themselves as proficient or expert (63.6% and 21.9%, respectively).

Of the residents and fellows in the group interviews, 36.3% reported that they had participated in a structured interprofessional simulation activity related to patient safety. Responses varied by gender, year of training, and specialty grouping. Across CLEs, the median (IQR) finding was 37.1% (26.3%–50.0%), with responses varying by region and type of ownership.

Inclusion in Patient Safety Event Investigations

In many CLEs, the patient safety and quality leaders indicated that they did not track resident and fellow participation in patient safety event investigations (eg, root cause analysis). A limited number of CLEs provided the Graduate Medical Education Committee and the governing body with information regarding the number of residents and fellows who had participated in formal patient safety event investigations.

The CLER teams also asked the program directors in the group interviews if they measured resident and fellow participation in patient safety event investigations. Approximately 42% of the program directors reported tracking resident and fellow involvement (median [IQR], 44.4% [30.0%–66.7%] across CLEs). Responses varied by region, CLE bed size, and type of ownership.

In the group interviews, 37.6% of the residents and fellows who were postgraduate year 3 (PGY-3) and higher indicated that they had participated in an interprofessional investigation of a patient safety event that included components such as analysis of system issues, development and implementation of an action plan, and monitoring for continuous improvement. Reponses varied by specialty grouping (FIGURE 3). Across CLEs, the median (IQR) finding was 37.6% (28.6%–50.0%), with responses varying by region, CLE bed size, and type of ownership (see Appendix B5).

FIGURE 3.

Percentage of Residents and Fellows (Postgraduate Year 3 and Above) Who Reported Participating in an Interprofessional Patient Safety Investigation, by Specialty Group

The CLER teams also asked faculty members about their involvement in interprofessional patient safety event investigations. Approximately 64% of the faculty members in the group interviews reported that they had participated in an investigation of a patient safety event that involved physicians, nurses, administrators, and other health care professionals (median [IQR], 63.3% [53.0%–73.2%] across CLEs).

Overall, the format and process of investigating patient safety events varied both across and within CLEs. It was uncommon for residents and fellows to describe involvement in comprehensive systems-based approaches to patient safety event investigations aimed at preventing future adverse events and sustaining improvements in patient safety. In general, residents and fellows described experiences that lacked the attributes of a formal patient safety event investigation with very little or no interprofessional or interdisciplinary engagement. Residents and fellows varied widely in their perceptions of what constituted a formal investigation of a patient safety event. Across many CLEs, case conferences, morbidity and mortality conferences, and grand rounds continued to be the major approach to patient safety event investigations.

Faculty members and program directors indicated that departmental mortality conferences, case conferences, and online modules were other informal approaches to model elements of a patient safety event investigation.

Disclosure of Patient Safety Events

In the group interviews, 66.0% of the residents and fellows indicated that they had received training on disclosing medical errors to patients and/or families (4.5% reported that such training was not applicable). Responses varied by year of training. Across CLEs, the median (IQR) finding was 68.2% (57.1%–79.3%), with responses varying by region and CLE bed size. Of those who received training, 10.1% indicated that the training was primarily simulation based; 69.8%, didactic and/or online; 15.1%, informal; and 5.0%, other.

Approximately 82% of the residents and fellows in the group interviews indicated that they knew of CLE resources to assist them in coping with a major patient safety event that resulted in a patient death (median [IQR], 85.8% [74.7%–93.0%] across CLEs; see Appendix B6 for information on variability). Of those familiar with the resources, most indicated that they would be somewhat (39.8%) or very comfortable (44.7%) in using these resources.

Health Care Quality (Including Health Care Disparities)

The CLER Program explored resident and fellow engagement in improving health care quality within the context of 6 major areas: involvement in developing and implementing the CLE's strategies for health care quality, awareness of the CLE's health care quality priorities, knowledge of health care quality terminology and methods, engagement in quality improvement (QI) projects, access to quality metrics data, and engagement in CLE efforts to address health care disparities.

Involvement in Developing Health Care Quality Strategies

As part of understanding the CLE's approach to improving health care quality, the CLER Site Visit teams reviewed the organization's strategic plan for quality and interviewed both executive and patient safety and quality leaders. Overall, a limited number of CLEs appeared to integrate QI within the organization as part of a system-wide, comprehensive approach to promote experiential learning and to improve quality and safety across the organization.

Across CLEs, resident and fellow involvement in strategic planning for QI was uncommon. Residents and fellows often served as implementers of CLE-wide QI activities (eg, hand hygiene, reducing hospital-acquired infections, reducing 30-day readmissions).

A limited number of CLEs had instituted resident and fellow committees aimed at increasing resident and fellow engagement in QI; few of these committees were integrated into the CLE's formal QI processes. In many CLEs, resident and fellow participation in institutional QI committees was uncommon; often, roles and expectations for participation were undefined or unclear. The clinical sites also appeared to have insufficient structure to allow residents and fellows to attend committee meetings regularly and to participate in meaningful ways. Additionally, residents and fellows in many CLEs were not included in the governing body's patient safety and quality committees.

Priorities for Improving Health Care Quality

In general, priorities for improving health care quality varied across CLEs. However, some common themes included alignment with broad national priorities such as Centers for Medicare & Medicaid Services value-based purchasing, Core Measures, or publicly reported performance measures. Many were also highly focused on meeting specific criteria such as reducing 30-day readmissions or improving performance on metrics related to pneumonia, chronic heart failure, and surgical care improvement project measures.

In the group interviews, 78.8% of the residents and fellows (PGY-2 and above) reported knowing their CLE's priorities for improving health care quality (see Appendix B7 for additional information on variability). When asked the same question, 84.4% of the faculty members and 86.7% of the program directors reported knowing the priorities. Often, the physician groups focused on departmental activities and did not describe priorities that aligned with those identified by the CLE's executive leadership or the patient safety and quality leaders. When the physicians identified priorities aligned with those of executive leadership, they were most commonly around nationally recognized measures, especially those related to programs with financial incentives such as measures from the Centers for Medicare & Medicaid Services.

Knowledge of Health Care Quality Improvement

In 55.1% of the CLEs, the residents and fellows appeared to have limited knowledge or understanding of basic QI terminology and methods such as Lean, Plan-Do-Study-Act, and Six Sigma (FIGURE 4, see also Appendix C5). A limited number of residents and fellows could articulate the QI approach employed by their CLE in designing and implementing QI activities to improve patient care.

FIGURE 4.

Percentage of Clinical Learning Environments by Proportion of Resident and Fellow Knowledge of Basic Quality Improvement Concepts

In general, the approach to educating residents and fellows about health care QI varied both within and between CLEs. Although some type of education was common as part of new resident and fellow orientation, a limited number of CLEs aimed to provide ongoing training for all residents and fellows. Training in health care QI appeared to occur primarily within departments or graduate medical education (GME) programs, and the format, methods, and content appeared to vary widely.

Engagement in Quality Improvement Projects

In 25.3% of the CLEs, the patient safety and quality leaders indicated that they monitor resident and fellow QI projects.

In the group interviews with residents and fellows (PGY-2 and above), 78.3% reported they had participated in a QI project of their own design, or one designed by their program or department. Of this group, 48.2% reported that their QI project was directly linked to 1 or more of the CLE's goals; 23.3% were uncertain. Of those who reported their QI projects were linked to the CLE's goals, 74.3% reported their projects involved interprofessional teams. Appendices B8, B9, and B10 provide detailed information on variability.

In the group interviews and on walking rounds, the CLER teams asked residents and fellows to describe their QI projects. Overall, residents and fellows varied in their descriptions of these projects. It was uncommon for residents and fellows to describe projects that aligned with their CLE's priorities. In most CLEs (82.2%), few described projects that included the components of a complete QI cycle (ie, Plan-Do-Study-Act) (FIGURE 5; see also Appendix C6). Often, resident and fellow participation was limited to planning and implementing a QI activity. For many residents and fellows, their QI projects did not involve formally assessing effectiveness and designing follow-up actions to adjust, support, and sustain ongoing QI efforts.

FIGURE 5.

Percentage of Clinical Learning Environments by Proportion of Resident and Fellow Quality Improvement Projects With Components of a Complete Quality Improvement Cycle

It was also uncommon for residents and fellows to describe involvement in interprofessional team-based QI projects. During the interviews on walking rounds, a limited number of nurses and other health care professionals indicated that they were involved in interprofessional QI projects that included residents and fellows.

When the CLER teams queried faculty members in the group interviews about their engagement in interprofessional QI projects, 72.7% reported that they had participated in a QI project with nurses, pharmacists, and other members of the health care team (median [IQR], 75.0% [65.0%–83.3%] across CLEs).

Access to Data

In the group interviews, 74.8% of the program directors reported that their residents and fellows have ready access to organized systems for collecting and analyzing data for the purposes of QI. Electronic health records, specialty-specific clinical registries, and local, regional, or national quality dashboards were often reported as common sources of QI data. Residents and fellows often mentioned the challenges (eg, long waiting lists) in acquiring specific reports from these data sources. Many faculty members noted that residents and fellows had limited support for data analysis. When support existed, it was often a departmental resource. The type and extent of analytic support services available to residents and fellows varied both within and across CLEs.

Overall, 30.9% of the residents and fellows in the group interviews reported receiving aggregated or benchmarked QI data on their own patients. Responses varied by gender, year of training, and specialty grouping. Across CLEs, the median (IQR) finding was 31.3% (22.2%–42.2%), with responses varying by region, CLE bed size, and type of ownership (FIGURE 6; see also Appendix B11).

FIGURE 6.

Percentage of Residents and Fellows Who Reported Receiving Aggregated or Benchmarked Quality Performance Data About the Care of Their Own Patients, by Type of Clinical Learning Environment (CLE) Ownership

Occasionally, the patient safety and quality leaders indicated that residents and fellows receive QI data to compare the care of their own patients with others served by their clinical site.

Engagement in CLE Efforts to Address Health Care Disparities

Strategies and Priorities in Health Care Disparities

Across many CLEs, executive leaders were aware of issues of health disparities affecting their surrounding communities. Many described conducting community needs assessments to improve access to care and providing free or low-cost care and clinics for the underserved, often staffed by residents and fellows from a few core specialties (eg, family medicine, internal medicine, pediatrics, obstetrics and gynecology). A limited number of residents and fellows from other specialty and subspecialty programs reported engaging in these activities.

A limited number of executive leaders spoke to health care disparities occurring within their hospital or medical center. Overall, less than 5% of executive leaders described a specific set of strategies or a systematic approach to identifying, addressing, and continuously assessing variability in the care provided to or the clinical outcomes of their patient populations at risk for health care disparities. In approximately half of the CLEs, the executive leaders, faculty members, or program directors indicated that some departments were collecting data or conducting studies related to health care disparities among specific patient populations; many of these efforts were reported as research projects.

In the group interviews, 55.1% of the residents and fellows reported that they knew their CLE's priorities in addressing disparities in health care; responses varied by year of training and specialty grouping. Across CLEs, this finding ranged from 7.1% to 100% (median [IQR], 59.4% [43.2%–78.0%]). Responses varied by region, CLE bed size, and type of ownership (see Appendix B12). In comparison, 66.3% of the faculty members and 68.1% of the program directors reported that they knew their CLE's priorities with regard to health care disparities.

Cultural Competency

Overall, residents, fellows, faculty members, and program directors interviewed in the group interviews were able to describe populations at risk for health care disparities at their clinical site.

In the group interviews, 33.6% of the residents and fellows reported that they had received cultural competency training that was specific to populations at risk for health care disparities at their clinical site, 37.0% reported receiving training that was not specific to the CLE's patient population, 24.0% reported receiving training that was primarily informal while providing clinical care, and 5.4% indicated that they had not received cultural competency training at their CLE.

Across CLEs, a median (IQR) of 32.8% (23.3%–46.4%) of the residents and fellows indicated that they had received cultural competency training that was specific to populations at risk for health care disparities at their clinical site. Responses varied by region, CLE bed size, and type of ownership (FIGURE 7; see also Appendix B13).

FIGURE 7.

Percentage of Residents and Fellows Who Reported Receiving Cultural Competency Training Specific to Populations at Risk for Health Care Disparities at Their Clinical Site, by Type of Clinical Learning Environment (CLE) Ownership

During interviews on walking rounds, many residents and fellows described education and training in cultural competency that was largely generic and not specific to the diverse populations receiving care at their clinical site. In general, residents and fellows across CLEs indicated receiving informal training in cultural competency while delivering care.

In the group interviews, 31.1% of faculty members reported that they had received cultural competency training that was specific to populations at risk for health care disparities and receiving care at their clinical site. Approximately 32% reported receiving training that was basic and not specific to the CLE's patient population, 27.0% reported receiving training that was primarily informal, and 9.6% reported that they had not received cultural competency training at the CLE.

Participation in Quality Improvement to Address Health Care Disparities

Overall, 19.0% of the faculty members in the group interviews indicated receiving periodic reports of their CLE's patient outcomes related to health care disparities (median [IQR], 16.7% [9.1%–27.6%] across CLEs). Of those who reported receiving patient outcomes data related to health care disparities, 49.9% indicated that their residents and fellows use the data to reduce disparities in health care (median [IQR], 50.0% [16.7%–66.7%] across CLEs).

Approximately 10% of the residents and fellows in the group interviews reported that they had participated in a QI project focused on reducing health care disparities; responses varied by gender, year of training, and specialty grouping (FIGURE 8; see also Appendix B14). In group discussions and on walking rounds, residents and fellows indicated that they rarely received CLE data related to health care disparities.

FIGURE 8.

Percentage of Residents and Fellows Who Reported Participating in a Quality Improvement Project Focused on Reducing Health Care Disparities, by Specialty Group

Care Transitions

The CLER Site Visitors explored several aspects of resident and fellow engagement in improving care transitions, including: priorities for improving care transitions, perceived vulnerabilities in care transitions, education on care transitions, inpatient transition processes, change-of-duty transitions, and supervision and assessment of care transitions.

Priorities for Improving Care Transitions

In describing priorities for improving transitions of care, many executive leaders focused on improving patient transfers from inpatient to postacute care or those occurring at discharge. A limited number of executive leaders mentioned improving provider-to-provider communications at change of duty (including resident and fellow handoffs) as a priority.

Occasionally, the executive leaders described efforts to create a standardized, organization-wide approach to 1 or more types of care transition. When they spoke about a standardized approach, they frequently mentioned using tools such as the I-PASS (ie, illness severity, patient summary, action list, situation awareness and contingency planning, and synthesis by receiver) model for handoffs or the SBAR (ie, situation, background, assessment, recommendation) technique.

Residents and fellows were often involved in efforts in designing, implementing, and standardizing their program's processes for shift-to-shift transitions of care.

Perceived Vulnerabilities in Care Transitions

Across CLEs, residents, fellows, nurses, and other clinical staff identified many transitions that they believed posed vulnerabilities in patient safety that the executive leaders did not mention. Examples included transfers from the emergency department to inpatient floors or units, transfers from the intensive care units to these floors, transfers from outside facilities, and multiple intrahospital transitions such as service to service and floor to floor, as well as between levels of care. Residents, fellows, and nurses often expressed concerns that communication during these transitions was most likely to be incomplete or inaccurate, leading to vulnerability for patient safety events.

Education on Care Transitions

Overall, 61.1% of the residents and fellows reported that they had participated in training with nurses and other health care professionals on transitioning patient care; responses varied by level of training and specialty grouping (median [IQR], 62.5% [50.8%–76.5%] across CLEs). Responses varied by region, CLE bed size, and type of ownership (see Appendix B15).

Across CLEs, standardized, organization-wide approaches to training in and managing care transitions between clinical services assigned to resident and fellow physician teams (eg, emergency department to inpatient care, operating room to intensive care unit) was uncommon.

Inpatient to Outpatient Transitions

When queried, 60.2% of the residents and fellows in the group interviews indicated that they use a standardized process for transitioning patients from inpatient to outpatient care. Across CLEs, the median (IQR) was 64.8% (52.4%–79.2%), with responses varying by region, bed size, and type of ownership (Appendix B16).

Interprofessional rounds varied across and within CLEs. These types of rounds were most commonly reported to occur in the intensive care units or for purposes of discharge planning.

Change-of-Duty Transitions

Approximately 83% of the residents and fellows in the group interviews reported that they followed a standardized process for change-of-duty handoffs. Across CLEs, this finding ranged from 48.6% to 100%, with a median (IQR) of 86.2% (78.2%–94.2%). Of the residents and fellows who reported following a standardized process between shifts, 75.6% reported using a standardized written template of patient information to facilitate the handoff process. Across CLEs, the median (IQR) finding was 77.6% (68.8%–87.5%), with responses varying by region, type of ownership, and CLE bed size (FIGURE 9). Appendices B17 and B18 provide complete information on variability.

FIGURE 9.

Percentage of Residents and Fellows Who Reported Following a Standardized Process for Handoffs Between Shifts and Using a Standardized Written Template, by Clinical Learning Environment (CLE) Bed Size

When asked how often attending physicians supervise their shift-to-shift handoff process, 23.3% of the residents and fellows in the group interviews reported that this supervision occurs daily, whereas 47.0% reported that it rarely occurs. Other responses included once per week (14.9%), once per month (7.8%), and less than once per month (7.0%).

In group interviews, 72.3% of faculty members reported that they assess resident and fellow readiness to move from direct to indirect supervision in conducting change-of-duty handoffs (13.4% reported that they do not conduct change-of-duty handoffs in their program). Of those who reported assessing resident and fellow readiness, 55.4% indicated that they assessed by direct observation only, 4.0% used a standardized assessment tool, 37.5% used both direct observation and a standardized assessment tool, and 3.1% used neither.

Approximately 57% of the program directors in the group interviews indicated that their program assesses resident and fellow readiness to move from direct to indirect supervision in conducting change-of-duty handoffs (23.1% reported that their program does not conduct change-of-duty handoffs). For those who indicated that their program assesses resident and fellow readiness, 58.1% indicated that their program assessed by direct observation only, 3.8% used a standardized assessment tool only, 34.5% used both direct observation and a standardized assessment tool, and 3.6% used neither.

From the information obtained on walking rounds, it appeared that the processes for transitioning care at change of duty vary widely across programs in most CLEs (94.4%) (FIGURE 10; see also Appendix C7).

FIGURE 10.

Percentage of Clinical Learning Environments With Handoff Processes That Were Standardized Across Programs, Based on Direct Observation

On average, the CLER teams observed resident and fellow change-of-duty handoffs for 2 to 5 programs per visit. Overall, the handoffs varied both within and across CLEs. Many were conducted in quiet, nonpatient areas with minimal interruption. The handoffs varied in the use of written templates and other common tools for formal communication. Generally, faculty members were not consistently engaged in supervising these handoffs, and interprofessional involvement was uncommon. Contingency planning (“if-then” scenarios) and read-back techniques to confirm the plan of care were also inconsistent.

Monitoring Care Transitions

The patient safety and quality leaders in many CLEs indicated that they analyze patient safety event reports for issues related to resident and fellow transitions of care; they seldom mentioned active monitoring of these care transitions.

Supervision

The CLER Program explored resident and fellow supervision and the issues around this focus area for perceptions of supervision and potential vulnerabilities, awareness of situations requiring direct supervision, and patient safety events related to supervision.

Perceptions of Supervision and Potential Vulnerabilities

Across CLEs, many executive leaders did not express concerns or identify any specific vulnerabilities related to resident and fellow supervision within their organization. In general, residents, fellows, faculty members, and program directors also reported a culture of adequate supervision. When asked to summarize their experience at their CLE, 70.7% of the residents and fellows in the group interviews reported being adequately supervised. Most of the faculty members (62.4%) and program directors (65.4%) also indicated that residents and fellows are adequately supervised.

Although the majority of the physicians in the group interviews reported a culture of close supervision, they also reported perceptions of inadequate supervision. In the group interviews, 26.8% of the residents and fellows reported that while in training at the CLE, they had been placed in a situation or witnessed 1 of their peers in a situation where they believed supervision was inadequate (eg, the attending physician was not available). Responses varied by gender, specialty grouping, and level of training (FIGURE 11). Across CLEs, the median (IQR) finding was 24.1% (16.7%–32.9%), with responses varying by region, CLE bed size, and type of ownership (Appendix B19).

FIGURE 11.

Percentage of Residents and Fellows Who Reported Being Placed in a Situation or Witnessing 1 of Their Peers in a Situation Where They Believed Supervision Was Inadequate, by Level of Training

Abbreviation: PGY, postgraduate year.

When asked about their experiences in contacting attending physicians and consultants for assistance, 45.9% of the residents and fellows in the group interviews indicated that they had encountered an attending physician or consultant who made them feel occasionally or frequently uncomfortable when requesting help at their CLE. Across CLEs, the median (IQR) finding was 46.5% (34.3%–57.1%); responses varied by region, CLE bed size, and type of ownership (FIGURE 12; see also Appendix B20). Nearly 60% of the faculty members and 46.3% of the program directors perceived that their residents and fellows had encountered an attending physician or consultant who made them feel occasionally or frequently uncomfortable in requesting help.

FIGURE 12.

Percentage of Residents and Fellows Who Reported Encountering a Physician (Attending Physicians or Consultants) Who Made Them Feel Occasionally or Frequently Uncomfortable When Requesting Assistance, by Clinical Learning Environment (CLE) Bed Size

In discussing issues related to supervision that may be creating patient safety vulnerabilities, the physician groups frequently mentioned the challenges of providing supervision in the evenings, on weekends, and during times of high acuity and patient volume. They noted that in these situations, the number of faculty members was insufficient for adequate supervision. They also noted that competing clinical responsibilities further limited the availability of faculty members to supervise residents and fellows.

Residents and fellows mentioned gaps in supervision when their peers provide consultative services, noting these gaps as a potential source of patient safety vulnerabilities. Across CLEs, many residents and fellows expressed hesitancy to request help from attending physicians or to report concerns regarding supervision. They noted reluctance or discomfort to ask for assistance due to unwillingness to appear unprepared, a lack of understanding of when to escalate concerns in a timely manner, fear of retaliation or other negative consequences, and/or potential resistance when asking for help.

In addition to concerns of undersupervision, many faculty members and program directors expressed concerns related to oversupervision, particularly in the procedural specialties, and the impact on resident and fellow readiness for independent practice. In discussing the factors that contribute to oversupervision, faculty members and program directors often mentioned medical liability concerns and billing requirements of the Centers for Medicare & Medicaid Services.

Awareness of Situations Requiring Direct Supervision

Overall, 94.0% of the residents and fellows in the group interviews reported that they knew what they were allowed to do without direct supervision. Responses varied by level of training and specialty grouping (Appendix B21).

In a separate query, 38.2% of the residents and fellows in the group interviews indicated that they had an objective way to know what procedures their peers from other services were allowed to do without direct supervision when providing consultative services on their patients. This finding varied by gender and specialty grouping (see Appendix B22).

Nearly 80% of the faculty members in the group interviews indicated that they had an objective way of knowing which procedures a particular resident or fellow was allowed to perform without direct supervision. In many CLEs, faculty members and program directors reported that this information is documented in an online system and maintained by GME or individual programs.

Generally, faculty members and programs directors indicated that nurses and other clinical staff members had mechanisms (eg, paper or online methods) to verify the level of supervision needed when residents and fellows perform clinical procedures outside of the operative areas. During the walking rounds, it appeared that the nurses had limited access to this information, varied in their awareness of how to access this information, or did not routinely use this information in the course of clinical care. Often, this information appeared to be incomplete or inaccurate. In most CLEs (94.4%), nurses reported that, in the absence of an attending physician, they relied principally on familiarity, trust, or year of training when residents and fellows performed procedures (FIGURE 13; see also Appendix C8).

FIGURE 13.

Percentage of Clinical Learning Environments by Mechanism Used for Identification of Resident and Fellow Competency to Perform Clinical Procedures, as Reported by Nurses

Patient Safety Events Related to Supervision

In 42.0% of the CLEs, the patient safety and quality leaders recalled patient safety events in the past year related to resident and fellow supervision. In the group interviews, 19.1% of program directors reported that, in the past year, they had to manage an issue of resident or fellow supervision that involved a patient safety event. Executive leaders were often unaware of patient safety events attributed to supervision. Generally across CLEs, the physician and leadership groups varied in their awareness of patient safety events related to resident and fellow supervision.

In general, executive and patient safety and quality leaders indicated that they addressed patient safety events as they arose and through retrospective review of the events. It was uncommon for CLEs to actively monitor for potential patient safety events related to supervision; the issue of supervision was often viewed as the responsibility of the GME community.

Fatigue Management, Mitigation, and Duty Hours

In the area of fatigue management, mitigation, and duty hours, the CLER Program explored the culture of reporting duty hours, fatigue management, patient safety events related to fatigue, situations that increase the risk for burnout, and strategies to address fatigue and burnout.

Culture of Reporting Duty Hours

In the group interviews, the CLER teams presented the residents and fellows with a scenario in which a colleague stays 30 minutes beyond his or her duty hour limits to address a small, nonurgent clinical task. When asked about the likelihood that their colleague would report the time, 47.7% responded that it was very unlikely that their colleague would do so. When presented with the same scenario, 25.8% of the faculty members and 19.2% of the program directors indicated that it was very unlikely that the resident or fellow would report the time.

In a separate query, 38.4% of the program directors in the group interviews indicated that when moonlighting was permitted, residents and fellows may be underreporting their moonlighting hours (median [IQR], 35.7% [16.7%–50.0%] across CLEs).

Fatigue Management

At each of the site visits, the residents and fellows in the group interviews were asked to consider a hypothetical scenario in which they were maximally fatigued yet had 2 hours left before the end of their shift. In this circumstance, 48.1% of the residents and fellows reported that they would power through to handoff. Responses varied by gender, year of training, and specialty grouping. Across CLEs, the median (IQR) finding was 45.5% (30.0%–58.1%); responses varied by region, type of ownership, and CLE bed size (FIGURE 14; see also Appendix B23).

FIGURE 14.

Percentage of Residents and Fellows Who Reported They Would Power Through When Maximally Fatigued, by Clinical Learning Environment (CLE) Bed Size

In this same circumstance, 26.5% indicated that they would notify a supervisor and expect to be taken off duty immediately; 11.3% indicated that they would ask another resident to take over their responsibilities.

When presented with the same scenario, 24.1% of the faculty members and 17.5% of the program directors in the group interviews expressed the belief that the resident or fellow would power through to handoff. Approximately 48% of the faculty members and 56.8% of the program directors believed that the resident or fellow would notify his or her supervisor and expect to be taken off duty immediately.

The CLER teams also explored issues of fatigue among faculty members and program directors by presenting a scenario in which their department is short staffed and one of their colleagues is fatigued but scheduled for call that evening. In response to the scenario, 36.7% of the faculty members and 32.0% of the program directors indicated that the colleague would continue with his or her clinical schedule, including call.

In describing the situations that increase the risk for fatigue, the physician groups often mentioned the following: time spent on electronic health records, times of high patient volume and acuity, covering multiple hospitals, 24-hour shifts, telephone calls with multiple interruptions, moving from day to night shifts, telephone calls for nonurgent problems, and a full day of clinical work after home call.

Patient Safety Events Related to Fatigue

When queried in the physician interviews, 8.5% of the residents and fellows, 5.1% of the faculty members, and 3.1% of the program directors recalled a patient safety event related to resident or fellow fatigue (FIGURE 15). The CLER teams also asked the patient safety and quality leaders a similar question. In 5.6% of the CLEs, the patient safety and quality leaders recalled 1 or more patient safety events related to resident or fellow fatigue (see Appendix C9). Occasionally, program directors were aware of patient safety events related to resident or fellow fatigue that appeared to be unknown to the patient safety and quality leaders. Mechanisms to assess risks to patient safety due to resident and fellow workload intensity was uncommon in many CLEs.

FIGURE 15.

Reported Awareness of a Patient Safety Event Related to Resident or Fellow Fatigue

Situations That Increase the Risk for Burnout

In many CLEs, residents and fellows described seeing signs of burnout in their colleagues that included emotional exhaustion, depersonalization, and/or a sense of low personal accomplishment. Residents and fellows often identified high patient volume, patient acuity, and nonphysician responsibilities as contributing factors to burnout.

In 88.8% of the CLEs, residents and fellows reported observing some signs of burnout among their faculty members and program directors (FIGURE 16; see also Appendix C10). Some of the manifestations included withdrawal from others and lack of willingness or enthusiasm to teach.

FIGURE 16.

Percentage of Clinical Learning Environments Where Residents and Fellows Observed Some Signs of Burnout Among Faculty Members and Program Directors

When asked about burnout, faculty members and program directors mentioned the same factors identified by the residents and fellows and added clinical productivity pressures, extensive documentation requirements, inadequate clinical and administrative support, and the challenge of balancing teaching, research, administrative responsibilities, and patient care.

Strategies to Address Fatigue and Burnout

In general, CLE efforts to mitigate fatigue focused primarily on provision of sleep rooms, transportation home for residents and fellows when they were too tired to drive, backup call systems, and education on fatigue management and mitigation.

Systematic strategies to identify, mitigate, and prevent fatigue and burnout were uncommon across CLEs. When strategies existed, they were often in response to an event related to fatigue or burnout. The content and coordination of these efforts varied across CLEs, and measures to assess the effectiveness of these efforts were uncommon.

Professionalism

The concept of professionalism encompasses a number of attributes. The CLER visits focused mainly on those involving honesty, integrity, and respectful treatment of others.

During each visit, the CLER team asked executive leaders whether or not any GME-related incidents involving professionalism or integrity had occurred in the past year. The executive leaders in 81.5% of the CLEs indicated that 1 or more such incidents had been brought to their attention (see Appendix C11).

Honesty in Reporting

Across CLEs, a median (IQR) of 90.0% (83.8%–95.9%) of the residents and fellows in the group interviews reported that they believe their CLE provides a supportive, nonpunitive environment for bringing forward concerns about honesty in reporting. This finding varied by region and type of ownership (see Appendix B24).

Overall, 36.4% of the residents and fellows in the group interviews reported that while at their CLE, they had documented a history or physical finding in a patient medical record that they did not personally elicit (eg, copying and pasting from another note without attribution). Reponses varied by gender, level of training, and specialty group (see Appendix B25). Approximately 10% of the faculty members and 12.5% of the program directors reported that they believed that a majority of residents and fellows had engaged in this practice.

When the CLER teams asked the faculty members about their documentation practices, 14.2% in the group interviews indicated that they had documented a history or physical finding in a patient medical record that they did not personally elicit.

In many CLEs, the executive leadership indicated that, in the past year, they had identified professionalism issues related to documentation practices using electronic health records and other sources of protected health information.

Integrity

Of the residents and fellows surveyed in the group interviews, 14.6% reported that while at the CLE, they had on occasion felt pressured to compromise their honesty or integrity to satisfy an authority figure. Across CLEs, the median (IQR) finding was 12.5% (6.7%–20.0%), with responses varying by region (FIGURE 17; see also Appendix B26).

FIGURE 17.

Percentage of Residents and Fellows Who Reported Having Felt Pressured to Compromise Their Honesty or Integrity to Satisfy an Authority Figure During Their Training at the Clinical Site, by Clinical Learning Environment (CLE) Region

To further explore issues of integrity, the CLER teams presented the residents and fellows in the group interviews with a scenario in which 1 of their colleagues has written a manuscript and the department chair—although not involved in the project—asked to be included as an author. Almost half of the residents and fellows responded that they would advise the colleague to discuss the matter with a faculty member or their designated institutional official. The next common response was to advise their colleague to include the department chair's name on the manuscript.

Respectful Treatment of Others

Generally across CLEs, the executive leadership expressed intolerance for unprofessional and disrespectful behavior. Nearly 80% of the program directors surveyed expressed the belief that their CLE was usually or always effective in managing reports of unprofessional behavior.

In the group discussions, the CLER teams presented the residents and fellows with a scenario describing an attending physician's mistreatment of a resident colleague that continued to persist despite being reported to the chief resident, program director, department chair, and head of GME. When presented with choices of what they might advise the colleague to do, most (56.2%) indicated that they would recommend contacting the organization's human resources department or anonymous hotline. Other responses included: contacting the Equal Employment Opportunity Commission (7.8%), registering a concern with the ACGME (19.4%), and taking no further action (2.9%). Another 13.7% suggested they would advise some other course of action, and when asked to elaborate, many indicated that they would recommend speaking with the ombudsman or going back to 1 or more members of the GME community to seek assistance.

When presented with the same scenario, faculty members and program directors in the group interviews also varied in their belief as to what the resident would do to address the perceived mistreatment.

Although many residents, fellows, nurses, and other health care professionals described their work environments as respectful and collegial, in nearly all of the CLEs (95.7%), individuals across multiple areas described the behavior of attending physicians and nurses as disrespectful or disruptive. In 62.9% of the CLEs, the behaviors were described as chronic, persistent, or pervasive in nature (FIGURE 18; see also Appendix C12).

FIGURE 18.

Percentage of Clinical Learning Environments Where Chronic Disrespectful or Disruptive Behavior Was Reported Across More Than 1 Clinical Unit

Across CLEs, many residents and fellows also described professionalism issues in obtaining consultation services, including lack of responsiveness and disrespectful communication in response to their requests for consultation.

Generally across CLEs, residents and fellows appeared to be aware of the mechanisms and resources available to resolve perceived mistreatment beyond those offered by GME. Many also indicated that they would inform their chain of command. The effectiveness of the organization's response to address these concerns varied across CLEs. On occasion, residents, fellows, and other clinical staff mentioned that they would not report mistreatment out of concern for adverse consequences of reporting.