Educational Experiences Residents Perceive As Most Helpful for the Acquisition of the ACGME Competencies

Abstract

Background

The Accreditation Council for Graduate Medical Education (ACGME) requires physicians in training to be educated in 6 competencies considered important for independent medical practice. There is little information about the experiences that residents feel contribute most to the acquisition of the competencies.

Objective

To understand how residents perceive their learning of the ACGME competencies and to determine which educational activities were most helpful in acquiring these competencies.

Method

A web-based survey created by the graduate medical education office for institutional program monitoring and evaluation was sent to all residents in ACGME-accredited programs at the David Geffen School of Medicine, University of California-Los Angeles, from 2007 to 2010. Residents responded to questions about the adequacy of their learning for each of the 6 competencies and which learning activities were most helpful in competency acquisition.

Results

We analyzed 1378 responses collected from postgraduate year-1 (PGY-1) to PGY-3 residents in 12 different residency programs, surveyed between 2007 and 2010. The overall response rate varied by year (66%–82%). Most residents (80%–97%) stated that their learning of the 6 ACGME competencies was “adequate.” Patient care activities and observation of attending physicians and peers were listed as the 2 most helpful learning activities for acquiring the 6 competencies.

Conclusion

Our findings reinforce the importance of learning from role models during patient care activities and the heterogeneity of learning activities needed for acquiring all 6 competencies.

What was known

The ACGME expects programs to use the 6 competencies in resident teaching and assessment, yet there is little information about residents' perceptions of the adequacy of this or about the experiences residents feel contribute most to acquisition of the competencies.

What is new

The majority of residents rated their learning of the competencies as adequate, and residents rated patient care interactions and observation of attending physicians and peers as “most helpful” in acquiring the competencies.

Limitations

Single-site, self-reported data may have limited generalizability; use of a nonvalidated survey tool response options omitted simulation as a possible helpful learning activity.

Bottom line

The competencies are multidimensional and interconnected, and require a variety of learning activities. Some learning activities were important for multiple competencies. Bedside teaching and faculty role modeling are important learning modalities, and may be key areas for added faculty development.

Background

The Accreditation Council for Graduate Medical Education (ACGME) requires physicians in training to be educated in 6 general competencies.¹ Residency programs are responsible for incorporating these competencies into their training programs and for identifying activities to support trainee learning. One challenge in implementing the competencies is measuring how well residents have acquired a competency and finding the most effective approaches for teaching each competency.^1,2 Some educators have incorporated teaching the competencies into specific educational activities, like teaching practice-based learning and improvement in morbidity and mortality conference³ or by providing seminars on professionalism and communication skills.⁴ Residency program directors and graduate medical education (GME) offices also have created curricular interventions⁵ or institution-wide didactic sessions.⁶ Although there are data showing how the competencies are measured and assessed,⁷ there is little information that reveals residents' perceptions about learning the competencies and about the activities that are most helpful. Recognizing what residents perceive as helpful in learning helps faculty to structure a more effective learning environment and may be useful to clinical educators for verifying their impact on learning.^8,9

The purpose of this study was to collect information on residents' perceptions of how well they thought they had learned the competencies and what learning activities contributed most. We hypothesized that at least 75% of the residents would state they learned the competencies adequately, with lower scores for “practice-based learning and improvement” and “systems-based practice,” and that the most helpful learning activity for acquiring the competencies would be patient care interactions.

Methods

Survey and Distribution

The survey instrument was created by the GME office for institutional program monitoring and evaluation and was reviewed and approved by a local panel of medical educators and program directors. No formal validation or testing of the survey instrument was done prior to implementation. A link to a web-based survey was sent by e-mail to all residents in every year in ACGME-accredited programs at the David Geffen School of Medicine, University of California-Los Angeles (UCLA), from 2007 to 2010. Residents were informed that their responses would be confidential. For questions about ACGME competencies, a drop-down menu of competency definitions was available for review. Residents were sent 6 e-mail reminders to complete the survey over a 30-day period.

Residents were asked, “How much did you learn about this competency?” for each ACGME competency. Answers to these questions were scored on a 4-point Likert scale (1, not at all; 2, too little; 3, adequate; 4, too much). Residents were then asked the follow-up question, “How helpful was this learning activity in acquiring this competency?” The learning activities they could choose from, which were based on a literature review, included patient care interactions (rounds, supervisor feedback), resident didactic sessions, journal club, quality improvement and morbidity and mortality conferences, observation of peers and attending physicians in action (role models), and independent reading and study.¹⁰ Responses to these questions were rated on a 5-point Likert scale (1, not helpful; 2, slightly helpful; 3, helpful; 4, very helpful; 5, most helpful). This study was approved by the UCLA Institutional Review Board, number10-000986.

Results

We analyzed 1378 responses from 4 years of survey administration (2007–2010), across 3 years of training (postgraduate year-1 [PGY-1; n  =  468], PGY-2 [n  =  450], and PGY-3 [n  =  460] from residents in 12 programs, including anesthesiology, emergency medicine, family medicine, internal medicine, medicine/pediatrics, obstetrics and gynecology, orthopedic surgery, otolaryngology, pathology, pediatrics, psychiatry, radiology, and surgery. We included only specialties that offered initial board certification begun at the PGY-1 level and had more than 10 residents in the training program in every year of the survey. In order to provide comparable analysis of residents' acquisition of the competencies across programs regardless of actual training program length, only results from PGY-1to PGY-3 residents were used in the analysis.

Adequacy of Learning the ACGME Competencies

The overall response rates for this survey varied by survey year, from 66% (636/962) in 2009 to 82% (812/989) in 2010. Between 81% and 97% of the residents (n  =  1378) in this study stated that their learning of the 6 ACGME competencies was “adequate” for all 4 years analyzed. The highest average rating was for patient care in 2010, in which 96.6% of the residents stated their learning of this competency was adequate. The lowest rating was for practice-based learning in 2007, in which 79.8% of the residents stated their knowledge was adequate. Across all years of the survey, 0.02% (n  =  32/1310) of residents responded “not at all” to having learned the practice-based learning competency, and 0.01% (n  =  9/1278) of residents responded they had not learned systems-based practice. This answer option was not used for the other competencies. The full results to the question “How well have you learned this competency?” are presented in table 1.

TABLE 1.

Average Ratings of Learning the ACGME Competencies, 2007–2010

Learning Experiences and Competency Acquisition

table 2 and figures 1 to 6 show resident ratings of learning activities in terms of the activities' helpfulness (1, not helpful; 2, slightly helpful; 3, helpful; 4, very helpful; 5, most helpful) in acquiring the 6 ACGME competencies.

TABLE 2.

Average Ratings of Learning Activities by ACGME Competency

FIGURE 1.

Helpfulness of Educational Experiences for Acquiring Patient Care Competency

Scale: 1, not helpful; 2, slightly helpful; 3, helpful; 4, very helpful; 5, most helpful.

FIGURE 2.

Helpfulness of Educational Experiences for Acquiring Medical Knowledge Competency

Scale: 1, not helpful; 2, slightly helpful; 3, helpful; 4, very helpful; 5, most helpful.

FIGURE 3.

Helpfulness of Educational Experiences for Acquiring Practice-based Learning Competency

Scale: 1, not helpful; 2, slightly helpful; 3, helpful; 4, very helpful; 5, most helpful.

FIGURE 4.

Helpfulness of Educational Experiences for Acquiring Interpersonal and Communication Skills Competency

Scale: 1, not helpful; 2, slightly helpful; 3, helpful; 4, very helpful; 5, most helpful.

FIGURE 5.

Helpfulness of Educational Experiences for Acquiring Professionalism Competency

Scale: 1, not helpful; 2, slightly helpful; 3, helpful; 4, very helpful; 5, most helpful.

FIGURE 6.

Helpfulness of Educational Experiences for Acquiring Systems-Based Practice Competency

Scale: 1, not helpful; 2, slightly helpful; 3, helpful; 4, very helpful; 5, most helpful.

Responding residents rated patient care interactions as “most helpful” in acquiring 5 of the 6 competencies: patient care (mean [M]  =  4.08; standard deviation [SD]  =  .888), medical knowledge (M  =  3.80; SD  =  .930), practice-based learning and improvement (M  =  3.42; SD  =  .101), interpersonal and communication skills (M  =  4.01; SD  =  1.03), and systems-based practice (M  =  3.46; SD  =  1.08). Observations of peers and attending was rated as “most helpful” in acquiring the professionalism competency (M  =  3.98; SD  =  1.04).

For the acquisition of medical knowledge, independent reading received slightly higher ratings than patient care interactions for some specialties, while didactic teaching, a core teaching method for many residencies, was not rated as highly. None of the activities received an average rating of “very helpful” in acquiring either the practice-based learning and improvement or the systems-based practice competency. Finally, observation of peers and attending in action and patient interactions were highly rated learning activities, especially for professionalism and interpersonal communication.

Discussion

Our survey has 4 main findings. First, most residents considered their learning of the 6 competencies as adequate, and their responses were consistent across the years we analyzed. Second, residents reported that they learned the competencies through a variety of learning activities, emphasizing the importance of implementing heterogeneous teaching methods. This finding revealed the challenges faced by clinical educators teaching the various competencies.¹¹ Third, many learning activities fostered resident learning in multiple competencies. Finally, we found that observation of peers and attending physicians in action and the residents' own patient care interactions were highly rated educational activities. This finding resonates with those in the literature, which advocates the importance of proper role modeling in the clinical learning process, and confirms the fact that residents obtain most of their knowledge about how to be a physician “by doing.”¹² If residents' self-perception of key learning activities is accurate, then preservation of these key activities, as total work hours are reduced, is essential.

For practice-based learning and improvement, we hypothesized that independent reading and journal club would be key activities for skill acquisition, as self-reflection and analytical skills are needed for this competency and these may not be gained in the clinical setting. However, residents reported that patient interactions, closely followed by observation of attending physicians, were the most helpful activities for achieving this competency. Similarly, for the acquisition of systems-based practice competency, residents perceived working with a variety of patients in different settings was a helpful activity. This could mean that educators and/or residents do not appreciate the opportunities the other learning activities present. Another interpretation could be that residents' understanding of these competencies may still be limited.

Of all learning activities, journal club was rated the lowest. It received low helpfulness ratings even for the acquisition of medical knowledge. Our results may reflect the alternate goal of some journal club sessions, which is to emphasize critical appraisal skills rather than knowledge acquisition.^13,14 It may be necessary for medical educators to reexamine the purpose of journal club and its usefulness as a learning activity.

Our study has several limitations. We collected self-reported data at a single-institution site, which may limit generalizability and introduce bias. The survey instrument was not tested for reliability and validity. Finally, after analyzing our results, we discovered other details that need additional exploration. For example, the survey instrument does not distinguish between the observations of peers and those of attending physicians and the observations of others whom they might identify as “role models,” and we did not include simulation as a possible helpful learning activity in competency acquisition, given the recent and considerable expansion of simulation centers.¹⁵

Conclusions

Although the residents in this study rated patient care interactions and observation of attending physicians and peers as “most helpful” in acquiring the competencies, each competency required a variety of learning activities, and some learning activities were important for multiple competencies, showing that the competencies are clearly multidimensional and interconnected. Medical educators may enhance their skills in teaching the ACGME competencies by better understanding what residents perceive as the most helpful ways to learn them. Program directors may consider additional faculty development of intentional role modeling and provide avenues for faculty to reflect on role modeling practices, as it was considered such a critical learning activity, particularly for professionalism. Additional research is needed to explore how best to teach, learn, and assess the competencies. In particular, more focused research of design and implementation of learning activities is needed for practice-based learning and systems-based practice, as respondents indicated lower helpfulness scores for learning activities for these competencies.