How Residents Say They Learn: A National, Multi-Specialty Survey of First- and Second-Year Residents

ABSTRACT

Relatively little is known about how, from whom, and under what conditions residents say they most effectively learn. We examined the relationships between residents' self-reported ratings of 11 different sources of learning and a number of empirical variables, using a national, random sample of postgraduate year (PGY) 1 and PGY-2 residents in the 1998–1999 training year. Residents were surveyed by mail. Completed surveys were received from 64.2% of 5616 residents contacted. The most often reported sources of learning were other residents and attending physicians. Ratings varied by specialty, level of training, and US medical graduates (USMGs) versus international medical graduates (IMGs). Factor analysis identified 3 primary modes of learning: faculty-organized, peer-oriented, and self-directed. Residents in different specialties varied in their use of these 3 sources of learning. IMG residents reported significantly less learning from peers and more self-directed learning. Increased resident duty hours were associated with a decrease in faculty-organized and self-directed learning, and an increase in peer-oriented learning.

Introduction

Formal instruction is only 1 part of the resident's total education. Alongside this formal curriculum are a host of parallel learning experiences that have become known as the informal or hidden curriculum, within which are embedded the values and norms of the profession.^1–3 These values are communicated in a number of subtle and not so subtle ways by peers and faculty.^4,5 Little is known about these less structured aspects of education for residents. This report seeks to provide unique data on how residents say they learn, gleaned from a large, random, multi-specialty survey of first- and second-year residents, conducted in 1999, prior to the 2003 Accreditation Council for Graduate Medical Education (ACGME) work hour limits.⁶ As such, it provides historical information on how residents assess their own learning modes, as well as a baseline for examining possible changes under the 2003 work hour standards.

National multi-specialty surveys conducted by the American Medical Association (AMA) in 1983 and 1987 found that instruction to residents averaged 6.8 hours per week, ranging from 7.3 hours per week for first-year residents to 5.2 hours after their fourth year of training.⁷ International medical graduates (IMGs) reported receiving significantly more instructional time than did US medical graduates (USMGs). Among all residents, higher satisfaction with training has usually been associated with more formal instruction, such as frequent contact with attending physicians, lectures, and grand rounds.^7,8 Phy and associates reported resident satisfaction was positively associated with increased faculty presence in the afternoon or evening.⁹ However, faculty and residents' perceptions sometimes differ about what constitutes effective learning activities.^10–12 Stress, fatigue, sleep deprivation, and “burnout” have been found to impair learning and performance,^13,14 and only about half of residents attend didactic teaching conferences, with fewer still reporting they remain alert throughout.^15,16 Our own previous 1989 national, multi-specialty survey found that, although quantity and quality of time with attending physicians was most valued by residents, they also ranked “other residents” and “special patients” as additional important sources of learning.⁸ Positive factors contributing to satisfaction with their internship year were, in order, attending physicians, other residents, patient rounds, seminars, and time with attending physicians.

Methods

In 1999, using the random selection feature of SPSS-PC, a 15% sample was drawn from the AMA's Graduate Medical Education Database.¹⁷ Target respondents were all residents in postgraduate year (PGY) 1 and PGY-2 positions who had no prior training and were scheduled to complete their current year of training in the summer of 1999. Completed surveys were received from 3604 of the 5616 residents contacted, a 64.2% response rate. The sample included both USMGs and IMGs, and respondents' demographic profile paralleled national distributions.^14,18 The questionnaire focused on the broad residency experience, including work hours, sleep, supervision, stress, learning, behavioral change, and impairment, as well as on reported incidents of belittlement/humiliation and sexual and racial harassment or discrimination. The final instrument consisted of 44 items.

The index questions relating to resident learning included: “Please rate how much each of the following contributed to your learning experience this year.” Eleven sources of learning were listed alphabetically:

• 1Attending faculty
• 2Computer/Internet
• 3Faculty advisor
• 4Lecture/grand rounds
• 5Other residents
• 6Patient rounds
• 7Reading
• 8Residency program director
• 9Seminars/small groups
• 10Special patients
• 11Supervising residents

Respondents were asked to rate each item on a scale of 1 (not at all) to 5 (a great deal), and to rate their current residency year in terms of (1) learning; (2) contact with attending physicians; (3) quality of time with attending physicians; and (4) overall, from 1 (poor) to 7 (excellent). In addition, residents reported their specialty, level of training, country, school of graduation, and ethnicity, as well as weekly work hours, sleep, medical errors, time spent with attending physicians, working while impaired, perceived adequacy of supervision, and level of stress.

Statistical analyses were conducted using SPSS-PC version 12 (IBM Corp, Armonk, NY). Data were initially presented as mean ratings with corresponding confidence intervals. Residents' ratings of the 11 listed sources of learning were submitted to factor analysis, using Principal Component Extraction, Oblimin Rotation with Kaiser Normalization. Pearson correlation coefficients were used to examine the relationship of these derived factors to other data from the survey. Finally, variations in these derived factors across demographic categories and other variables were examined by comparing deviations from the grand mean for each factor. These deviations are presented as average variations from this overall grand mean in both tabular and graphic form.

Residents' Ratings of 11 Learning Sources

Overall, residents rated peers as their most important source of learning, with attending physicians a close second and supervising residents third. Faculty advisors were rated as least important, along with computer/Internet, program directors, and seminars and small groups. For PGY-2 respondents, learning from both peers and supervising residents and from formal patient rounds declined in importance (P < .01), while learning from reading, the computer/Internet, and faculty advisors increased (P < .01; figure 1). USMGs were significantly (P < .01) more likely than IMGs to rate other residents, attending faculty, patient rounds, and supervising residents as important sources of learning. IMGs gave higher ratings to formal lectures, seminars, and faculty, including program directors and faculty advisors (P < .01; figure 2). For USMGs, other residents were the highest-rated source of learning, while independent reading and patient rounds were highest for IMGs (P < .01).

Figure 1 .

Resident Ratings of Sources of Learning by Year of Residency Training With 95% CI

Figure 2 .

Resident Ratings of Sources of Learning for US Medical Graduates (USMGs) and International Medical Graduates (IMGs) With 95% CI

Factor Analysis of Sources of Learning

Factor analysis was used to identify the underlying dimensions linking the 11 sources of learning. Both orthogonal and non-orthogonal solutions were examined. The best fitting factor analytic solution was achieved by means of a Principal Component extraction, using an Oblimin rotation with a Kaiser normalization. This solution defined 3 primary dimensions from the sources of learning, labeled “faculty-organized,” “peer-oriented,” and “self-directed” (table 1).

Table 1.

Factor Analysis of Items Rating Contributions to Learning Experience

Eigen values were strongest for the faculty-organized and lowest for the self-directed learning factors. Factor 1 (faculty-organized) showed the highest loadings for learning from residency program directors, lectures/grand rounds, attending faculty, and faculty advisors.

Factor 2 (peer-oriented) demonstrated the highest loadings for learning from supervising residents, other residents, and patient rounds, while Factor 3 (self-directed) showed the highest loadings for reading, computer/Internet, special patients, and seminars/small groups. Factor 1 was correlated with Factor 2 (+0.27) and with Factor 3 (+0.29), suggesting that as faculty-organized learning rose, peer-oriented and self-directed learning rose as well. Faculty-organized learning was strongly associated with residents' positive ratings of both their overall residency and their learning experience (table 2). Weaker, but significant positive correlations were also found with “time with attending physicians” and average weekly sleep hours. In addition, significant negative correlations were found with working without adequate supervision, sleep deprivation, stress ratings, weekly work hours, working while in an impaired condition, conflicts with professional staff, belittlement and humiliation, and negative behavioral changes.

Table 2.

Correlates of 3 Derived Sources of Learning Factors

Peer-oriented learning showed lower magnitude positive correlations with overall ratings of the residency and the learning experience, and a weaker positive relationship with average work hours per week and time spent teaching. Self-directed learning presented the weakest correlations with ratings of both learning and the overall residency, and a weak negative relationship with reports of sleep deprivation.

Variations in Sources of Learning

Using the derived factor scores, we compared the relative contributions of the 3 sources of learning across a number of variables (table 3). These scores were interpreted as the percentage of a standard deviation above (positive sign) or below (negative sign) the grand mean for the sample. Using this metric, residents who reported a “significant medical error” also rated faculty-organized learning as less important. Moreover, those who said their error resulted in an “adverse patient outcome” rated peer-oriented learning lower as well. Conflict with medical staff also was associated with lower ratings for faculty-organized learning. Reports of alcohol usage, taking medications to stay awake, sleep, or cope, or feeling pressured to do something unethical were all associated with lower values for faculty-organized learning and modest declines in self-directed learning.

Table 3.

Average Factor Scores for 3 Sources of Learning by Selected Variables

The 3 factors showed noticeable associations with reported average work hours per week (figure 3). As work hours increased, both faculty-organized and self-directed learning declined. By contrast, peer-oriented learning rose, peaking between 91 and 100 hours per week, and then declined.

Figure 3 .

Changes in Variations Around the Grand Mean for the 3 Sources of Learning Factors by Reported Average Weekly Work Hours

Note: Each ±0.10 change represents 10% of a standard deviation.

Differences by Specialty

Internal medicine, the largest specialty in our sample, was closest to the grand mean in each of the 3 learning dimensions (table 4). Anesthesiology residents reported more self-directed and less peer-oriented learning than average, while those in obstetrics-gynecology and neurological surgery reported higher levels of peer-oriented learning, and lower levels of faculty-organized and self-directed learning. Of note, residents' overall ratings of their learning by specialty (last column, table 4) were nearly always associated with higher ratings of faculty-organized teaching and self-directed learning. As a group, primary care specialties approximated the grand mean for all 3 factors (figure 4). By contrast, hospital-based specialties (anesthesiology, pathology, etc) presented noticeably lower levels of peer-oriented learning. Surgical specialties demonstrated more peer-oriented learning, but levels for faculty-organized and self-directed learning were one-third of a standard deviation below the grand mean.

Table 4.

Factors Contributing to Learning by Specialty (PGY-1 and PGY-2 Combined)^a

Figure 4 .

Variations Around the Grand Mean for the 3 Sources of Learning Factors by 3 Clusters of Medical Specialties

Note: Each ±0.10 change represents 10% of a standard deviation.

Differences by Graduation Origin

USMGs differed dramatically from IMGs in the relative importance of the 3 learning factors. Results were also affected by whether the respondents characterized themselves as White or Non-White (figure 5). White USMGs reported greater peer-oriented learning and less self-directed learning, while Non-White USMGs appeared close to the baseline in all 3 dimensions. By contrast, IMGs showed a slight elevation in faculty-organized learning along with lower levels of peer-oriented learning and higher levels of self-directed learning. Moving from White IMGs to Non-White IMGs, self-directed learning increased, while peer-oriented learning declined.

Figure 5 .

Variations Around the Grand Mean for the 3 Sources of Learning Factors by USMG White, USMG Non-White, IMG White, and IMG Non-White

Note: Each ±0.10 change represents 10% of a standard deviation.

A Model for Predicting Satisfaction With Learning

A series of step-wise regression models were run to select the best combination of variables for predicting residents' ratings of their learning experience. The results of these analyses (table 5) produced a model with an adjusted R² = 0.52, and included, in order, ratings of quality of time with attending physician, contact with attending physician, faculty-organized learning, peer-oriented learning, self-directed learning, gender, and how often the residents said that they had worked while in an “impaired condition.” Residents' ratings of their learning do not appear to be the result of any single factor, but of the convergence of all of the learning dimensions.

Table 5.

Regression Equation Showing Relationship of Selected Variables with Residents' Rating of Learning Experience During the First 2 Years of Residency Training

Discussion

Differences in the 3 sources of learning were associated both positively and negatively with a number of empirical variables relevant to the residents' perception of their educational experience, including overall satisfaction with residency, conflicts with medical staff, reports of medical errors, and average weekly duty hours. We also documented the importance of resident peers as a key source of learning, especially during the first year of residency. Finally, we found that residents' ratings of their learning experience could be predicted by a model that incorporates the 3 learning factors, along with ratings of their time with attending physicians.

Faculty-organized learning showed the most robust correlations with a range of associated variables. As the importance of this factor increased, reports of negative behaviors such as significant medical errors, conflicts with medical staff, alcohol use, and unethical conduct decreased. Residents appear to regard faculty involvement as the key issue fostering both learning and a positive residency experience. This finding confirms our previous work demonstrating that contact with attending physicians was a strong predictor of satisfaction during residency.⁸

The identification of resident peers as an important source of learning suggests that the education of residents requires not just formal, faculty-organized activities, but also a forum in which trainees learn from and teach each other. The combination of faculty-organized education and peer-oriented experiences thus provides a mutually reinforcing structure that helps to ensure that essential learning takes place. Other residents also serve as a safety net to ensure that education occurs even when formal teaching may be flawed.

Self-directed learning, the third leg supporting the stool of residency education, shows the lowest correlation with reported satisfaction, and may be viewed as an amplifying or compensatory mechanism. Dinkevich and Ozuah observed that pediatric residents average as much as 7.5 hours a week in self-directed learning, rising from 6 hours in the first year to 9.5 hours for third-year residents.¹⁹ Consistent with our results, IMGs in this study also reported more self-study: 8.4 hours per week compared to 7 hours for USMGs.¹⁹ Provided faculty-organized and peer-oriented learning are maintained at satisfactory levels, growing use of the Internet as a source of learning should not change the relative importance of self-directed learning, but simply changes the form by which it occurs. Self-directed learning appears to be an especially important component of learning for IMGs. The pattern is striking, and suggests, at minimum, that the process by which learning in residency takes place for IMGs differs from that for USMGs. The relatively higher use of self-directed learning by IMGs raises the question if this is a sociocultural pattern or an attempt to compensate for gaps in peer-oriented or faculty-organized learning.

Although the traditional focus of undergraduate medical education is the formal curriculum, prior research has found the informal curriculum is an important second component.^1–5 The faculty-organized and peer-oriented factors can be seen as analogous to formal and informal curricula. Peer-oriented learning is largely informal, taking place on work rounds, at the bedside, or in hallway conversations with fellow residents, often occurring after hours or when attending physicians are not available. Informal learning provides critical opportunities for skills development, knowledge transfer, and sharing of values, largely outside the attention of faculty.^3–5

The relative variations in the 3 learning factors across specialties is reminiscent of Hafferty's notions concerning the “hidden” curriculum.^1,2 We have come to see the unique blend of faculty-, peer-, and self-directed learning characterizing each specialty as the unspoken framework within which every resident learns. These unique combinations may convey covert messages or constitute a “hidden curriculum” for residents, telling them under what conditions to depend on faculty, look to peers, or take charge of their own learning.

The study was conducted prior to the ACGME establishing common duty hour limits. Although a clear sense of the effects of limiting resident hours on learning is still emerging, concerns linger that capping weekly hours might have an adverse effect on the educational opportunities available to residents.^20–23 As clinical demands are compressed, educational time may be reduced. Tracking the changes in these 3 major sources of learning is 1 way to assess the effect of the duty hour limit on resident learning. We hope our data will serve as a baseline by which to gauge the impact of the common duty hour limits on the process of, and residents' satisfaction with, their educational experience. Given the importance of faculty-organized learning, a particular concern may be reduced availability of attending physicians arising from a number of factors and pressures, including assumption of added clinical responsibilities formerly held by residents. The findings emphasize the need to expand our knowledge of how learning takes place in residency. Educational programming must take each of these sources of learning into account and adjust them to the particular needs of specialties and of individual residents. The optimal mix among these 3 sources of learning will vary by program and specialty. Educational messages, such as the mandated ACGME competencies may need to make use of all 3 learning modes. Finally, efforts must be made to determine if the different patterns seen for IMGs stem from preferences of the residents, or are signs that current education efforts may be inadequate and a different approach might benefit this group.¹²

Limitations of the study include that the information is based on residents' self-reports and it is impossible to know how accurately they applied themselves to rating their learning sources. The large national sample, the good response rate, and the fact that the findings generally confirm our previous work should serve to mitigate this concern. Second, we did not use the word “satisfaction” in our questions about learning, but based on our previous experience and the pilot testing of the survey instrument, we felt confident in asking for specific ratings along scales that called for perceptions of relative values. Third, in an effort to compare our findings with those of previous national surveys, we asked the residents to provide summary information for their current training year. Thus, our data reflect averages, and much remains to be learned concerning the important variations across time and service assignments and different residency programs. Finally, we were not able to secure data from residents who were more advanced in their training.

Conclusion

Based on residents' own ratings, we empirically identified 3 distinct sources of learning in residency: faculty-organized, peer-oriented, and self-directed. The relative contribution of these sources was found to vary significantly by medical specialty, training year, and between USMGs and IMGs.