Detailed Findings from the CLER National Report of Findings 2016

Introduction

This section presents detailed findings from the first round of visits of the Clinical Learning Environment Review (CLER) Program, organized into six parts by the CLER focus areas: patient safety, health care quality, care transitions, supervision, fatigue management and duty hours, and professionalism.

Between 2012 and 2015, site visits were conducted at major participating clinical sites for 297 ACGME-accredited Sponsoring Institutions (SIs) with three or more core residency programs.^1,2

Collectively, these 297 SIs oversee 8,878 ACGME-accredited residency and fellowship programs, with a range of from three to 148 programs per SI (median=17). Because our sample consisted entirely of larger SIs, the institutions surveyed here account for 111,482 residents and fellows—or 90% of all those in ACGME-accredited programs—with a range of from eight to 2,216 trainees per SI (median=241).

For each of the 297 targeted institutions, the CLER teams visited one hospital or medical center that served as a clinical learning environment (CLE) for that SI. They spent the majority of their time at inpatient settings, though where possible they also visited affiliated ambulatory care practices in close proximity. The hospitals and medical centers varied in size from 41 to 2,396 acute care beds (median=520). The majority (69.4%) were nongovernment, not-for-profit organizations; 21.5% were government, nonfederal; 5.4% were investor-owned, for-profit; and 3.7% were government, federal. As for location, approximately 30% of them were in the northeastern US, 29.3% in the south, 25.9% in the Midwest, and 14.1% in the west.

In the group sessions conducted during these visits, the CLER teams collectively interviewed more than 1,000 members of executive leadership (including CEOs), 8,755 residents and fellows, 7,740 core faculty members, and 5,599 program directors of ACGME-accredited programs in the group sessions. Additionally, the CLER teams interviewed the CLE's leadership in patient safety and health care quality and thousands of residents and fellows, faculty members, nurses, pharmacists, social workers, and other care providers while on walking rounds of the clinical areas.

This report is based on a synthesis of all this information, with some data represented quantitatively while other data are described qualitatively. Data sources included answers to closed-ended questions collected through an audience response system, open-ended discussion questions, and interviews from the walking rounds. Mixed methods were used to improve the accuracy of the findings.³

This combination of methodologies and findings should be considered when interpreting the results, making comparisons, or drawing conclusions. For example, results from the group discussions may appear more positive than information gathered on walking rounds. Alternatively, practices reported during group discussions may have been verified on walking rounds. Thus, both supporting and conflicting evidence may be presented to explain or qualify findings.

Interpreting Quantitative Results From the Group Interviews

Quantitative responses from group interviews with residents and fellows, faculty members, and program directors were collected using an electronic audience response system, which allowed anonymous reporting. The results from the audience response system have been analyzed at two levels: (1) at the level of individuals (e.g., residents and fellows) and (2) at the level of CLEs.

The first level of analysis examines what is reported overall by the individuals surveyed. Results are presented as percentages of the total number of individuals surveyed. For example:

“Of the residents and fellows interviewed in the group discussions, 62.7% reported that they have ready access to organized systems of data for the purposes of quality improvement.”

The second level of analysis examines differences between CLEs when individual responses are aggregated at the CLE level. CLE results are presented as median and interquartile range. For example:

“Across CLEs, a median of 65.5% (IQR=54.1%–80.0%) of residents and fellows reported having ready access to organized systems of data for quality improvement.”

Statistically significant differences in responses due to resident and fellow characteristics (e.g., gender, residency year) and CLE characteristics (e.g., regional location, CLE bed size) are also reported. It should be noted, however, that statistical significance does not always imply practical significance. For example, differences in responses by males and females may be statistically significant but the differences may not be meaningful or large enough to have practical relevance or implications.

Additional Considerations

A specific set of descriptive terms is used throughout this report to summarize quantitative results from both the audience response system and specific findings that were quantified from the site visit reports.³ These terms, and their corresponding quantitative ranges, are as follows:

few (<10%), some (10–49%), most (50–90%), and nearly all (>90%)

Besides the quantitative data, there were also a number of open-ended questions during group interviews and walking rounds that, by design, were not intended to be enumerated. For these questions, the site visit teams made an assessment of the relative magnitude of observations at each individual site. These results are presented in the report using a different set of descriptive terms, in order to prevent confusion with the descriptive terms used for quantitative data described earlier. The qualitative descriptive terms, which are intended to approximate the quantitative terms above, are as follows:

uncommon or limited, occasionally, many, and generally

Finally, this section is constructed to follow approximately the same structure as the individual reports that each participating institution received after the CLER site visit. This structure was chosen to permit easy comparison between the data from an individual site and that of this report, which aggregates results from all 297 institutions. Those who seek additional detail may consult the Appendices that follow. Appendix A presents additional information on the SIs, sites visited, and groups interviewed. Appendix B covers selected aggregated quantitative results from the group interviews with residents and fellows. And Appendix C presents qualitative information from the group interviews and walking rounds.

Patient Safety

The CLER site visits explored resident and fellow engagement in patient safety by assessing five major topics: priorities in patient safety, knowledge of patient safety principles and terminology, use of the patient safety reporting system, inclusion in patient safety investigations, and involvement in developing and implementing the CLE's patient safety strategy.

Patient Safety Priorities

The hospitals and medical centers visited by the CLER Program varied as to their specific priorities for addressing patient safety. However, there were common themes. These included priorities around (1) creating a culture focused on patient safety, (2) improving communications about patient safety, and (3) enhancing use of the patient safety event reporting system. Many of the hospitals and medical centers were also focused on improving their performance on specific patient safety indicators (PSIs), such as infection control and reducing patient falls.

During the group discussions, 73.9% of the residents and fellows (those in their second post graduate year, or PGY-2, and above), 81.4% of the faculty members, and 83.3% of the program directors reported that they knew their CLE's priorities in the area of patient safety (Figure 1; see also Appendix B1 for detailed information on differences in resident and fellow responses). However, there was often a lack of alignment between the priorities reported by the graduate medical education (GME) community—that is, the residents and fellows, faculty members, and program directors—and those identified by the CLE's executive, patient safety, and quality leaders. When the groups aligned, it was most often around improving performance on one or more of the nationally recognized PSIs. Often, the patient safety priorities reported by the GME community concentrated on departmental activities rather than the overarching priorities of the CLE.

Figure 1.

Knowledge of Patient Safety

Across CLEs, a median of 96.8% (IQR=93.3%–100%) of the residents and fellows in the group interviews reported that they had received formal education or training on topics of patient safety. In both group interviews and walking rounds, residents and fellows often reported that their principal patient safety education occurred at orientation and annually through required online learning modules.

The residents and fellows appeared to vary widely in their knowledge of fundamental patient safety principles, terminology, and methods (e.g., Swiss cheese model of system failure, root cause analysis, fishbone diagrams). Across most CLEs (82.6%), the residents and fellows appeared to have limited knowledge of these fundamental principles and methods (see Appendix C1).

Use of the Patient Safety Event Reporting System

CLE Systems for Reporting

CLEs varied in the design and use of patient safety event reporting systems. Generally, CLEs had developed systems that included (1) an online event reporting tool, (2) a way to verbally report a patient safety event to the patient safety staff, and (3) a chain-of-command system which allowed events to be reported to an immediate supervisor (often, a more senior resident or faculty member). CLEs varied as to how they deployed these components, their perception of the benefits or drawbacks of anonymous reporting, the level of clarity they provided as to who was responsible for reporting, and the extent to which verbal reports were reliably entered into the CLE's centralized system. Across most CLEs, residents and fellows indicated they were aware of their CLE's process for reporting patient safety events (see Appendix B2). In a limited number of CLEs, residents and fellows were not allowed access to the online reporting system.

Culture of Safety

In the group interviews, 95.5% of the residents and fellows indicated they believed their CLE provided a safe and non-punitive environment for reporting errors, near misses/close calls, and unsafe conditions. This general sense of a safe culture was also consistent among faculty members, program directors, and the nursing staff. However, residents and fellows varied in their reported use of the CLE's patient safety event reporting system.

Data from many CLEs indicated that residents and fellows often did not file such reports personally; instead, they relied on others to file them. So, it is unclear whether they had sufficient experience in personally interacting with the CLE's reporting mechanism to support their view of a non-punitive system.

Many staff members and physicians also indicated that they used the CLE's patient safety reporting system to report on individual behaviors in a manner that could be perceived as punitive. Occasionally, the name of the CLE's patient safety reporting system was used as a verb (based on the name of the commercial software). For example, residents or nurses would speak about a person being “PSA'ed” or “MIDAS'ed.” Overall, the data suggest that CLEs vary widely, both within and across institutions, as to whether the physicians and staff members fully regard the organizational culture to be one that establishes a “safe” and non-punitive environment for reporting patient safety events.

Reporting

Approximately 32% of CLEs were able to provide information on the number or percentage of patient safety event reports submitted by residents and fellows (see Appendix C2). CLEs varied as to whether their reporting systems could track the number or percentage of events submitted by teaching faculty and other medical staff.

Across CLEs, the patient safety leaders indicated that a limited number of faculty physicians submitted reports of patient safety events.

Approximately 81% of the faculty members and 83% of the program directors interviewed indicated that they believed fewer than half of their residents and fellows have reported a patient safety event using the CLE's reporting system. When asked what process they believed residents and fellows most frequently followed when reporting a patient safety event, 49.4% of the faculty members and 51.5% of the program directors in group interviews believed they would use the CLE's reporting system.

Overall, 68.0% of the residents and fellows in the group interviews indicated they had experienced an adverse event, near miss/close call, or unsafe condition while at the CLE. This varied by gender, year of training, and specialty grouping (Figure 2; for detailed information on variability, see Appendix B3).

Figure 2.

Of the residents and fellows who indicated they had experienced an adverse event, near miss/close call, or unsafe condition, 46.8% indicated they had reported the event themselves using the CLE's system. This varied by year of training and specialty grouping (Figure 3). Across CLEs, a median of 46.2% (IQR=32.1%–60.0%) of residents and fellows indicated they reported the adverse event, near miss/close call, or unsafe condition themselves. Responses varied by region, CLE bed size, and type of ownership (see Appendix B4 for additional information on variability).

Figure 3.

Residents and fellows also indicated that they often relied on others to report patient safety events. Nearly 15% of the residents and fellows in the group interviews who had experienced a patient safety event indicated they relied on a nurse to submit the report; 29.8% relied on a physician supervisor; and 8.9% indicated they cared for the patient, but chose not to submit a report.

In a separate query, 20.0% of the residents and fellows in the group interviews said they had reported a near miss/close call event. This varied by specialty grouping and year of training (Figure 4). Across CLEs, this was a median of 18.2% (IQR=11.6%–26.7%). Responses varied by region, CLE bed size, and type of ownership (for complete information on variability, see Appendix B5).

Figure 4.

While on walking rounds, the CLER teams also queried residents and fellows and clinical staff about patient safety event reporting. In 87.7% of the CLEs, the walking rounds interviews indicated residents and fellows infrequently enter patient safety event reports into the CLE's system (see Appendix C3).

Many of the residents and fellows interviewed indicated that they use their chain of command to report patient safety events. It is unclear whether the patient safety events reported in this manner were consistently captured in the CLE's central system.

Across CLEs, the residents and fellows interviewed on walking rounds appeared to have a limited understanding of the range of patient safety events that should be reported. Often, residents and fellows did not recognize near misses/close calls, events without harm, unexpected deteriorations, or procedural complications as reportable events.

Across many CLEs, nurses and other clinical staff interviewed were able to identify the paper or online process for reporting patient safety events. Many of the nurses and other clinical staff also indicated that they had submitted reports using the system within the past six months. In some CLEs, nurses said the time-consuming process of submitting a report and the uncertainty of receiving feedback deterred them from entering every event. The nurses interviewed did not consistently recognize near misses/close calls, events without harm, unexpected deteriorations, and procedural complications as reportable events.

Feedback

The CLEs varied widely in their processes for reviewing and prioritizing patient safety events and residents and fellows varied in their knowledge of these processes. Many residents and fellows appeared to be unaware of how their CLEs use the reporting of adverse events, near misses/close calls, or unsafe conditions to improve care both broadly and at the individual departmental level. They often used the term “black box” in describing the CLE's process for reviewing and prioritizing event reports; frequently, nurses and other clinical staff shared a similar view. Residents and fellows were rarely involved in the CLE's process for reviewing and prioritizing which patient safety events required further investigation.

Of the residents and fellows who reported submitting a patient safety event by any means (themselves or through a nurse or supervisor), 46.9% indicated they received feedback in follow-up to the submission. Responses varied by gender, year of training, and specialty grouping (see Appendix B6). The most common form of feedback was an email acknowledgement of receipt of the report. The next most common form of feedback was a request for more information as part of a formal investigation. In addition to residents and fellows, nurses and other clinical staff across CLEs expressed a strong desire to receive feedback after submitting a report.

Time-Outs

As part of the patient safety event experience, the CLER team assessed resident and fellow participation in the time-out process. Within and across CLEs, nurses, residents, and fellows interviewed on walking rounds indicated that residents and fellows do not consistently conduct standardized time-outs prior to performing procedures. Specifically in the perioperative area, nurses indicated the degree to which residents and fellows were involved in standardized time-outs or other surgical check-lists varied. This, in part, reflected variability in CLE requirements for these processes and in part reflected a lack of clarity as to the role of residents and fellows in performing these processes.

Inclusion in Investigations of Patient Safety Events

Of the residents and fellows who were PGY-3 and above, 40.8% reported in the group interviews that they had participated in an interprofessional investigation of a patient safety event such as a root cause analysis (RCA). This varied by specialty grouping (Figure 5). Across CLEs, this was a median of 41.3% (IQR=31.8%–52.2%); responses varied by region and type of ownership (see Appendix B7). Faculty members and program directors indicated residents and fellows also received some insights into investigating patient safety events through morbidity and mortality conferences, quality improvement conferences, didactic lectures, and simulation activities.

Figure 5.

The format and process of investigating patient safety events varied both across and within CLEs. In many CLEs, patient safety leaders convened and facilitated the investigations. In others, departmental leaders headed the efforts with varying descriptions as to the format and process. As a result, residents and fellows varied widely in their perceptions of what constituted an investigation of a patient safety event.

Overall, 64.2% of the faculty members and 67.2% of the program directors in the group discussions said they had participated in a patient safety investigation such as an RCA. Across CLEs, the median was 66.7% (IQR=54.9%–74.9%) for faculty members and 71.4% (IQR=61.4%–85.5%) for program directors. Further discussion with faculty members and program directors revealed that, in answering the question, they were often referring to a mixture of departmental investigations and formal RCAs led by the CLE's administration. When asked specifically about interprofessional investigations, they said these were most often conducted by the CLE's administration and less often by departments.

Engagement in the Development and Implementation of Strategies to Improve Patient Safety

Across CLEs, residents and fellows varied in the extent to which they reported engaging with the CLE to address priorities in patient safety—ranging from zero to 100%, with a median of 89.8% (IQR=80.0%–96.0%). The extent of engagement varied by region and CLE bed size (Table 1).

Table 1.

Percentage of Residents and Fellows Who Reported Being Engaged with the CLE to Address Priorities in Patient Safety, by Region and CLE Bed Size

When asked to elaborate during group discussions, the residents and fellows most commonly described their role as implementers of hospital- or medical center-wide initiatives. A limited number of residents and fellows indicated they had a significant role in developing these initiatives or overall strategies for the organization. Occasionally they indicated a few residents or fellows participated on CLE patient safety committees, others indicated there were formal or informal processes for bringing together the GME community and CLE leaders to exchange ideas or solicit input. Still others indicated they contributed informally by providing input to leaders in their department or program. The CLEs varied in the extent to which the executive leadership directly engaged with residents and fellows in formulating their strategy for addressing patient safety. Often they reported that this engagement was uncommon.

Health Care Quality (including Health Care Disparities)

Each CLER site visit explored resident and fellow engagement in improving health care quality within the context of five major areas: alignment with the CLE's health care quality priorities, knowledge of health care quality terminology and methods, engagement in quality improvement (QI) activities, involvement in developing and implementing the CLE's strategies for health care quality, and involvement in implementing the CLE's strategies for addressing health care disparities.

Alignment of Priorities for Improving Health Care Quality

On each visit, the CLER team first sought to understand the CLE's approach to health care quality improvement. They did so by reviewing the organization's strategic plan for quality, by talking with executive leaders, and by interviewing leaders of its quality department/program.

CLEs varied as to their specific priorities for improving health care quality. However, there were some common themes. Many CLEs aligned with broad national priorities, such as the Centers for Medicare & Medicaid Services value-based purchasing, Core Measures, or publicly reported performance measures. Many were highly focused on achieving performance improvement goals for particular conditions or measures such as reducing patient falls or 30-day readmissions, or meeting specific criteria such as those related to pneumonia, chronic heart failure, and surgical care improvement project measures.

Overall, when queried in group interviews, 74.4% of residents and fellows (PGY-2 and above), 74.3% of faculty members, and 73.5% of program directors believed they knew their CLE's priorities for improving health care quality (see Appendix B8 for additional information on variability in resident and fellow responses). Similar to the findings in patient safety, the priorities listed by the GME community (residents and fellows, faculty members, and program directors) did not always align with those identified by the CLE's executive or patient safety and quality leaders. Often, the priorities reported by residents and fellows, faculty members, and program directors focused on departmental activities rather than the broad priorities of the CLE. The groups most commonly aligned around nationally recognized measures, especially those related to programs with financial incentives such as measures from the Centers for Medicare & Medicaid Services.

Knowledge of Health Care Quality Improvement

The CLER team also explored resident and fellow awareness of the concepts and methods of health care quality improvement.

In 59.1% of the CLEs, few residents and fellows were familiar with basic quality improvement concepts such as Plan-Do-Study-Act (PDSA), Six Sigma, or Lean (Figure 6; see also Appendix C4). A limited number of residents and fellows could articulate the CLE's approach (e.g., methods and tools) to quality improvement.

Figure 6.

Collectively, the information from the numerous interviews conducted on the CLER visits indicated that CLEs vary widely in their approach to educating residents and fellows about health care quality improvement. Many CLEs offer some type of education as part of new resident and fellow orientation. A limited number of CLEs aim to provide a consistent level of training for all residents and fellows. In many CLEs, though, training in health care quality improvement occurs principally within departments or GME programs, and thus varies widely as to format, methods, and content.

Engagement in Quality Improvement Activities

Across CLEs, a median of 50.8% (IQR=37.7%–63.2%) of residents and fellows (PGY-2 and above) reported they had participated in a QI activity directed by the CLE's administration. Responses varied by CLE bed size, type of ownership, and region (Table 2). When asked to describe their participation, residents and fellows most commonly described implementing a CLE-wide initiative (e.g., hand hygiene). A limited number described projects in which they played a role in designing, managing, or analyzing the results of the activity.

Table 2.

Percentage of Residents and Fellows (PGY-2 and above) Who Reported Being Involved in a Quality Improvement Project Directed by the CLE's Administration, by Region, CLE Bed Size, and Type of Ownership

Approximately 76% of the residents and fellows (PGY-2 and above) in group interviews reported they had participated in a QI project of their own design or one designed by their program or department. This varied by gender, year of training, and specialty grouping (see Appendix B9). Of these, 52.3% reported that their project was in some way linked to the CLE's goals; responses varied by year of training and specialty grouping (see Appendix B10). Nearly 30% were uncertain if their project linked to the CLE's goals. Of those who reported their projects were linked to the CLE's goals, 74.5% reported being engaged in interprofessional QI teams while working on the projects. This varied by specialty grouping (see Appendix B11). A limited number of the non-physician clinical staff interviewed on walking rounds indicated that they were engaged in interprofessional quality improvement activities.

The residents and fellows interviewed in groups and on walking rounds varied extensively as to what they considered to be a quality improvement activity. Occasionally residents and fellows described activities that employed common methods such as PDSA, Lean, or Six Sigma. Many described participating in part of a quality improvement cycle, such as planning or implementing an activity, with no involvement in formally reviewing the outcome and adjusting the efforts accordingly.

In many CLEs, residents and fellows provided isolated examples of interprofessional engagement in quality improvement. A limited number of nurses or other clinical staff interviewed on walking rounds indicated they were involved in a QI activity that included residents or fellows.

Of the residents and fellows interviewed in the group discussions, 62.7% reported that they have ready access to organized systems of data for the purposes of quality improvement. Across CLEs, this response had a median of 65.5% (IQR=54.1%–80.0%); responses varied by type of ownership and CLE bed size (Figure 7; see Appendix B12 for additional information on variability).

Figure 7.

The most commonly reported sources of QI data were: local or regional quality dashboards, specialty-specific clinical registries, and electronic health records. Residents and fellows often noted that although they were aware of such data sources, they encountered challenges (e.g., long waiting lists) in acquiring specific reports. Faculty members often indicated residents and fellows had limited access to support for data analysis; the support, when it existed, was often a departmental resource. The type and extent of analytic support services available to residents and fellows appeared to vary greatly both within and across CLEs.

Involvement in the CLE's Development and Implementation of Health Care Quality Strategies

Across CLEs, a median of 27.3% (IQR=18.2%–38.9%) of residents and fellows reported they were engaged with the CLE's leadership in developing and advancing the hospital or medical center's quality strategy. Responses varied by region, CLE bed size, and type of ownership (Table 3).

Table 3.

Percentage of Residents and Fellows Who Reported Being Engaged with the Clinical Site's Leadership in Developing and Advancing Quality Strategies, by Region, CLE Bed Size, and Type of Ownership

In follow-up discussions, residents and fellows often described serving as implementers of CLE-wide quality improvement activities (e.g., reducing hospital acquired infections, improving medication reconciliation, reducing 30-day readmissions).

CLE Efforts to Address Health Care Disparities

Generally across CLEs, executive leaders were aware of issues of health care disparities affecting their surrounding communities. A limited number of the executive leaders spoke to health care disparities occurring within their hospital or medical center.

When asked about the CLE's priorities and efforts to address health care disparities, many executive leaders described efforts associated with their community health needs assessment. It was uncommon for CLEs to involve residents and fellows in conducting the needs assessment; more commonly, residents and fellows participated in community outreach efforts—such as cancer screenings or health fairs.

Generally across CLEs, executive leaders also described efforts to improve access to care, in particular free or low-cost care and clinics for the underserved that are often staffed by residents. The residents at these clinics nearly always came from a few core specialties (e.g., family medicine, internal medicine, pediatrics, and obstetrics and gynecology). A limited number of residents from other specialty and sub-specialty programs reported engaging in these activities.

When queried, 55.1% of residents and fellows, 62.0% of faculty members, and 60.1% of program directors in the group interviews reported they knew the CLE's priorities with regard to health care disparities (Figure 8). However, reported knowledge of these priorities among residents and fellows decreased as the number of years of training increased (Figure 9).

Figure 8.

Figure 9.

Across CLEs, a median of 60.4% (IQR=43.1%–76.2%) of residents and fellows reported that they knew their CLE's priorities for addressing health care disparities within the institution's own patient population. Responses varied by region and CLE bed size (see Appendix B13).

Overall, less than five percent of CLE executive leaders described a specific set of strategies or a systematic approach to identifying variability in the care provided to or the clinical outcomes of their known vulnerable patient populations. In approximately half of the CLEs, the executive leaders or faculty members indicated there were some efforts to address health care disparities occurring at the department level. In both group discussions and walking rounds, residents and fellows rarely indicated that they routinely received data for the purposes of monitoring health care disparities among the vulnerable populations served by their CLE.

In 53.9% of the CLEs, residents and fellows characterized their training in cultural competency as largely generic and not tailored to the specific populations served by the hospital or medical center (see Appendix C5). Generally across CLEs, residents and fellows, nurses, and other clinical staff indicated they had access to translation services when caring for non-English speaking patients; they varied in their reports of timely access to and quality of these resources.

Care Transitions

The CLER visits explored resident and fellow engagement in improving care transitions for four major areas: alignment of priorities, inpatient transition processes, change of shift transitions, and resident and fellow engagement in the development and implementation of strategies to improve care transitions.

Alignment of Priorities for Improving Care Transitions

In many CLEs, the executive leaders identified one or more priorities for improving transitions of care. Many were focused on improving patient transfers between floors or units or those occurring at discharge. A limited number of executive leaders mentioned improving provider-to-provider communications at change of shift (including resident and fellow hand-offs) as a priority.

Across CLEs, a median of 82.4% (IQR=65.8%–93.1%) of residents and fellows reported that they knew their CLE's priorities for improving transitions in care. Responses varied by CLE bed size, type of ownership, and region (Table 4).

Table 4.

Percentage of Residents and Fellows Who Reported Knowing the CLE's Priorities for Improving Transitions in Care, by Region, CLE Bed Size, and Type of Ownership

CLEs varied as to whether the priorities noted by the executive leaders aligned with those cited by residents and fellows, faculty members, and program directors.

Occasionally across CLEs, the residents and fellows, nurses, and other clinical staff identified one or more transitions that they believed posed vulnerabilities in patient safety that were not mentioned by the executive leaders. Examples included patient transfers from other institutions; transitions from acute care to outpatient care for special populations such as the homeless or frail elderly; transitions for patients with complex care needs like mental illness; transitions for patients from rural areas where there may be large distances between hospital and home; and transitions for international patients.

Inpatient Transition Processes

Approximately 84% of the residents and fellows reported they used a standardized process when transferring patient care between floors or units. Nearly 80% indicated they use a standardized process for transitions from inpatient to out-patient care.

CLEs varied both across and within organizations with regard to interprofessional rounds. They were most commonly reported to occur in the intensive care units or for purposes of discharge planning.

Change of Shift Transitions

Nearly 90% of the residents and fellows in the group interviews indicated they used a standardized process for hand-offs during change of shift. Responses varied by gender, year of training, and specialty grouping (see Appendix B14). Of those who reported use of a standardized process, 77.8% reported they had written templates of patient information to facilitate the hand-off process. This varied by gender, year of training, and specialty grouping (Figure 10; Appendix B15 presents further information on these differences).

Figure 10.

Across CLEs, a median of 80.0% (IQR=70.9%–89.7%) of residents and fellows reported using both a standardized process and a written template. This varied by region and CLE bed size (see Appendix B15).

While a majority of the residents and fellows indicated they use standardized processes when transitioning care at change of shift, data from the walking rounds at many CLEs suggest these processes vary widely across programs (Figure 11; see also Appendix C6).

Figure 11.

As part of nearly all visits, the CLER team directly observed resident change of shift hand-off sessions for two to five programs. These hand-offs varied both within and across CLEs in use of templates, style of template, and level of detail relayed. While many of the observed transitions were conducted in quiet, non-patient areas with minimal interruption, the CLER teams also observed hand-offs in environments that were less conducive to good communication. The hand-offs also varied both within and across CLEs as to whether or not attending physicians were present to supervise. A limited number of resident hand-offs involved other health care professionals. The observed hand-offs also varied within and across CLEs in the routine use of contingency planning (“if-then” statements) and clarifying questions; “read-back” of information was uncommon.

Both within and across CLEs, program directors and faculty members appeared to vary in the degree to which they monitored residents and fellows' hand-off skills at change of shift. A limited number reported that they used formal criteria to assess these skills.

Engagement in the Development and Implementation of Strategies to Improve Care Transitions

Occasionally the executive leaders described efforts to create a standardized, organization-wide approach to one or more types of care transition. They varied in the degree to which they involved residents and fellows in designing and testing these processes. Most commonly, residents and fellows were involved in efforts to standardize their program's processes for transferring care at change of shift.

Supervision

The CLER visits also explored issues around supervision of residents and fellows, including: perceptions of potential vulnerabilities, awareness of the situations when residents and fellows require direct supervision, and the potential impact on patient safety.

Perceived Potential Vulnerabilities

Across CLEs, many executive leaders expressed confidence in the supervision of residents and fellows within their organization; a limited number expressed concerns or identified any specific vulnerabilities. Residents and fellows, faculty members, and program directors also voiced overall confidence regarding supervision. When asked about potential vulnerabilities, the physician groups frequently mentioned challenges of providing supervision in the evenings and on weekends. Faculty members and program directors also identified vulnerabilities associated with residents and fellows who are either overconfident, unaware of their need for supervision, or hesitant to request assistance or escalate care in a timely manner.

Many faculty members and program directors expressed the belief that residents and fellows are at times over-supervised, particularly in the procedural specialties, and felt that the trainees may be less than fully prepared for independent practice as a result. On further discussion, many perceived that the restraints leading to over-supervision were related to the CLE's interpretation of the Centers for Medicare & Medicaid Services requirements for billing patient care services.

While the majority of residents and fellows, faculty members, and program directors were confident about overall supervision, there were also reports of inadequate supervision. Approximately 22% of the residents and fellows in the group interviews reported that while in training at the CLE, they had been placed in a situation—or witnessed one of their peers in a situation—where they believed there was inadequate supervision (e.g., the attending physician was not available). Responses varied by gender, level of training, and specialty grouping (see Appendix B16). Across CLEs, a median of 19.0% (IQR=10.0%–28.0%) of residents and fellows reported they had experienced or witnessed such a situation.

Awareness of Situations Requiring Direct Supervision

In group interviews, 96.6% of the residents and fellows reported that they knew what they were allowed to do with and without direct supervision. This finding varied slightly by gender, level of training, and specialty grouping (Figure 12; Appendix B17 presents additional information on variability). Similarly, 98.2% of faculty members and 99.3% of program directors were confident that their residents and fellows knew what they were allowed to do with and without direct supervision.

Figure 12.

Generally across CLEs, faculty members and/or program directors reported that procedures performed by residents and fellows are documented in an online software application. Many indicated the data are maintained by their ACGME-accredited SI. In some cases, the hospitals and medical centers that served as CLEs for the SI had limited access to this information.

Over 84% of faculty members and 92.6% of program directors in the group interviews reported they had an objective way of knowing which procedures a particular resident or fellow was allowed to perform with or without direct supervision.

In contrast, 33.2% of the residents and fellows in group interviews reported they had an objective way of knowing whether or not another resident or fellow needed supervision for a specific procedure (Figure 13). This finding varied by year of training and gender (see Appendix B18).

Figure 13.

Many of the CLEs indicated they have paper or online methods by which nurses and others can check resident and fellow competencies to perform clinical procedures outside of the operative areas. Most commonly, these appeared to be lists from each residency or fellowship program outlining expectations of general competencies by year of training. A limited number of CLEs appeared to provide nurses and other clinical staff with detailed information that allows them to check a specific resident's required level of supervision. Of those CLEs that provide this information, the nurses interviewed on walking rounds varied widely as to their awareness of the information and knowledge of how to access it.

Across CLEs, nurses reported they rely principally on familiarity, trust, year of training, or the presence of an attending physician when residents and fellows perform procedures (Figure 14; see also Appendix C7).

Figure 14.

Supervision and Patient Safety

Across CLEs, program directors at 47.4% of CLEs reported they had to manage one or more issues related to resident and fellow supervision within the past year. In approximately 22% of CLEs, program directors reported that in the past year they had to manage an issue of resident or fellow supervision that involved a patient safety event.

Across CLEs, the patient safety and quality leaders varied in their ability to recall any patient safety events related to resident or fellow supervision. Those that they did recall typically involved a serious safety event.

CLE executive leaders and safety and quality leaders expressed few concerns related to patient safety vulnerabilities and resident supervision. Many indicated they addressed any issues brought to their attention as they arose. A limited number of CLEs appeared to have an organized system of active surveillance to detect and address emerging vulnerabilities to minimize patient harm.

Fatigue Management, Mitigation, and Duty Hours

In the area of fatigue management, mitigation, and duty hours, the CLER visits explored issues of education, awareness and use of resources, and links to patient safety. The CLER visits also solicited feedback about the ACGME's requirements addressing resident and fellow duty hours.

Education on Fatigue Management and Mitigation

Overall, 95.5% of residents and fellows interviewed reported that they received education on fatigue management and mitigation. This finding varied by year of training (91.6%, 94.9%, 95.4%, and 95.9% for PGY-1, PGY-2, PGY-3, and PGY-4+, respectively). Across CLEs, the median was 96.9% (IQR=93.2%–100%). Responses varied by region and type of ownership (Table 5).

Table 5.

Percentage of Residents and Fellows Who Reported Receiving Education on Fatigue Management and Mitigation, by Region and Type of Ownership

Generally across CLEs, the residents and fellows indicated this education primarily consisted of an initial session at orientation, followed by required annual online modules. Occasionally they also participated in activities offered by their department or program. These additional activities appeared to vary widely from program to program and across CLEs.

Overall, 67.0% of faculty members and 75.3% of program directors reported that they had received education on fatigue management and mitigation (Figure 15). As with the residents and fellows, the primary source of this education was most often an annual online module.

Figure 15.

Awareness and Use of Resources

When queried as to the resources available for fatigue management, the residents and fellows often mentioned call rooms for napping, cab fare to take them home when too fatigued to drive, coffee, and the availability of counseling services. In follow-up discussion, many said they had actually used the call rooms and/or coffee; a limited number indicated they had ever used the other resources.

In general, residents and fellows indicated they believed the ACGME's current requirements for duty hours were effective in mitigating fatigue. On further discussion, however, they often noted the potential for fatigue that was not related to duty hours, such as that resulting from financial or emotional stress or caring for a family member or young infant. Faculty members and nurses reported observing resident fatigue that appeared to be related to factors other than the number of hours worked (e.g., periods of high patient volume or high-acuity care). Across CLEs, leaders appeared to vary in the extent to which they focused on issues of fatigue not related to the number of hours worked.

At each of the site visits, the residents and fellows in the group sessions were asked to consider a hypothetical scenario in which they were maximally fatigued but still had two hours left before the end of a shift. When asked what they would do in this circumstance, one-third reported they would power through to hand-off. This finding varied by gender, year of training, and specialty grouping (see Appendix B19).

Approximately 42% indicated they would notify a supervisor and expect to be taken off duty immediately, and 10.6% indicated they would ask another resident to take over their responsibilities.

Across CLEs, a median of 29.4% (IQR=16.8%–41.7%) of residents and fellows reported they would power through to hand-off. Responses varied by region and CLE bed size (Figure 16; for additional information on variability, see Appendix B19).

Figure 16.

When the same scenario was presented to faculty members and program directors, 20.0% of faculty members and 15.7% of program directors reported that the resident or fellow would power through to hand-off (Figure 17). Approximately 53% of the faculty members and 60.0% of the program directors believed that the resident or fellow would notify his or her supervisor and expect to be taken off duty immediately.

Figure 17.

Within and across CLEs, the nurses who were interviewed appeared to vary widely in their awareness of resident and fellow fatigue and strategies to assist them in managing fatigue.

Moonlighting

Within CLEs, a median of 33.3% (IQR=14.3%–47.6%) of the program directors believed that, when moonlighting was permitted, residents and fellows may be under-reporting their moonlighting hours.

Fatigue and Patient Safety

When queried, approximately 8.0% of the program directors interviewed recalled a patient safety event related to resident or fellow fatigue. When the CLEs' patient safety and quality leaders were asked a similar question, 6.0% recalled one or more patient safety events related to resident or fellow fatigue (see Appendix C8). Occasionally across CLEs, the program directors were aware of patient safety events related to resident or fellow fatigue that appeared to be unknown to the patient safety and quality leaders—especially if the events did not result in death or serious harm to a patient.

Discussions About Resident and Fellow Duty Hours

As part of the CLER site visit protocol, the CLER team asked faculty members and program directors about their perception of resident and fellow fatigue management and mitigation in relation to the ACGME's current requirements governing duty hours.

At many CLEs, the program directors and faculty members expressed concern regarding the impact of the current standards. However, at a number of other CLEs, resident and fellow duty hours appeared to be managed without much controversy or concern.

When concerns were raised, they touched on a wide variety of issues affecting residents and fellows, including perceptions of loss of continuity of care, lack of patient ownership, decreased clinical exposure, and diminished opportunities for the acquisition of technical skills.

Many program directors and faculty members indicated they believed that requiring residents and fellows to log their hours has led to a “shift work mentality.” They also expressed concern that residents and fellows are finding ways to bypass the systems for monitoring duty hours, such as completing chart documentation at home. Occasionally in CLEs, program directors and faculty member believed the duty hour requirements have increased the frequency of hand-offs and thus could affect patient safety. Others did not express the same concern since they had systems in place to mitigate hand-off risk (such as adopting 12-hour shifts).

In many CLEs, faculty members and program directors stated that, at times, making sure the residents and fellows comply with duty-hour rules leads to more clinical work for themselves. Many also felt they had little if any leeway to use the flexibility built into the duty-hour requirements. These program directors and faculty members indicated they diligently avoided exceptions to the rules,⁴ even those permitted by ACGME policy, to avoid the potential of a citation from a Residency Review Committee.

Professionalism

The concept of professionalism encompasses a number of attributes. The CLER visits focused mainly on those involving honesty, integrity, and mistreatment. During each visit, the CLER team asked executive leaders whether or not there had been any GME-related incidents involving professionalism or integrity over the past two years. Across CLEs, 66.4% of the executive leaders said there had been one or more such incidents (see Appendix C9).

Overall, 92.8% of the residents and fellows in the group interviews reported they received institutional training on topics of professionalism and ethics. Results varied by year of training (Figure 18). Across CLEs, there was a median of 95.3% (IQR=90.0%–100%); responses varied by region and CLE bed size (see Appendix B20). Of those who indicated they received training, 12.7% indicated that the activities were limited to those occurring at orientation. Approximately 75% reported the training continued periodically throughout the course of their time at the CLE. Often, the residents and fellows described these activities as required online modules related to patient privacy and/or clinical research. Many of them reported that in addition to general educational activities, they participated in program-specific sessions on various topics of professionalism throughout their training.

Figure 18.

Over 93% of the residents and fellows surveyed reported that they believe their CLE provides a supportive, non-punitive environment for bringing forward concerns about honesty in reporting. Responses varied by gender (see Appendix B21).

Approximately 16% of the residents and fellows reported that, at some point in their training, they had felt pressured to compromise their integrity to satisfy an authority figure. Across CLEs, the median was 14.3% (IQR=8.3%–20.0%). Responses did not vary by region, CLE bed size, or type of ownership (see Appendix B22).

The residents and fellows were presented with a scenario describing an attending physician's mistreatment of a colleague and asked what they would advise the colleague to do. Overall, the most common response was to advise the colleague to discuss the incident with his or her chief resident or program director (58.6%).

In a follow-up question, the scenario was extended such that the mistreatment persisted to the point where the colleague no longer trusted the GME chain of command to resolve the problem. When presented with choices of what they might advise the colleague to do, over 40% responded that they would tell him/her to contact the institution's Human Resources Department or federal Equal Employment Opportunity Commission. Another 23.7% responded they would advise registering a concern with the ACGME. Approximately 14% indicated they would tell the colleague to submit an incident report, and 9.9% suggested they would advise some other course of action. When asked to elaborate, many of those who selected “other” indicated they would advise going back to one or more members of the GME community to seek assistance.

When presented with the same scenario, faculty members and program directors also varied in their belief as to what a resident would do to address perceived mistreatment (Figure 19).

Figure 19.

Across CLEs, a median of 34.0% (IQR=20.7%–48.7%) of residents and fellows reported that while at that CLE, they had documented a history or physical finding in a patient chart that they did not personally elicit (e.g., copying and pasting from another note). Responses varied by region, CLE bed size, and type of ownership (see Appendix B23). Overall, 23.1% of faculty members and 20.7% of program directors reported that they believed that a majority of residents and fellows had engaged in this practice.

In general, the residents and fellows, nurses, and other clinical providers described their work environments as respectful and collegial. However, in nearly 50% of CLEs, some individuals across multiple areas reported observing or encountering physicians and nurses who were perceived to be disruptive or disrespectful (see Appendix C10). Some of the behaviors were described as chronic, persistent, and pervasive.