Abstract
Accurately recognizing the learning goals of trainees should enhance teachers' effectiveness. We sought to determine how commonly such recognition occurs and whether it improves residents' satisfaction with the teaching interaction. In a cross-sectional survey of 97 internal medicine residents and 42 ambulatory clinic preceptors in five ambulatory care clinics in Washington and Oregon, we systematically sampled 236 dyadic teaching interactions. Each dyad participant independently indicated the residents' perceived learning needs from a standardized list. Overall, the preceptors' recognition of the residents' learning needs, as measured by percentage of agreement between preceptors and residents on the learning topics, was modest (κ 0.21, p = .02). The percentage of agreement for all topics was 43%, ranging from 8% to 66%. Greater time pressures were associated with lower agreement (38% vs 56% for the highest and lowest strata of resident-reported time pressure; 15% vs 43% for highest and lowest strata of preceptor-reported time pressure). Agreement increased as the number of sessions the pair had worked together increased (62% for pairs with >20 vs 17% for pairs with 0 previous sessions). Satisfaction with teaching encounters was high (4.5 on a 5-point scale) and unrelated to the degree of agreement (p = .92). These findings suggest that faculty development programs should emphasize precepting skills in recognizing residents' perceived learning needs and that resident clinics should be redesigned to maximize preceptor-resident continuity and minimize time pressure.
Keywords: ambulatory care, residents, medical education, adult learning theory
How can medical educators teach residents more effectively in the ambulatory care setting? The answer to this question is important to clinician-educators, medical residents, and residency program directors as they try to improve teaching effectiveness.
In any teaching encounter, the learner may perceive his or her own learning needs differently than the teacher. Adult learning theory posits that teaching should take into account learners' perceptions of relevance and be responsive to learners' perceived needs.1 For example, teachers may be more effective if they are able to accurately ascertain what the learner considers most relevant.2 Thus, recognition of learner-perceived learning needs may contribute to effective teaching.
There are few reports or methods in the literature that evaluate the skill of recognizing learners' perceived learning needs among internal medicine faculty, particularly in the outpatient setting. The purpose of this study was to evaluate how accurately teaching physicians recognize residents' self-perceived learning needs during ambulatory clinic teaching encounters, to identify the predictors of such recognition, and to identify the correlates of accurate recognition including residents' satisfaction.
METHODS
Participants
During the data collection periods, all internal medicine residents and clinic preceptors who were present in the selected clinics on the study dates were asked to participate. The principal investigator or research assistant approached each potential participant to explain the study, review the questionnaire, and obtain informed consent. No one refused to participate.
Setting
The study took place in five general internal medicine ambulatory care clinics in which internal medicine residents are primarily responsible for patient care. Two of the clinics are affiliated with a large community-based residency program in Portland, Ore, and three are affiliated with a large university residency program in Seattle, Wash.
Data Collection
We collected data regarding actual teaching encounters during regular clinic time. A teaching encounter was defined as a one-on-one discussion between a resident and a preceptor regarding a specific patient being seen in the clinic. Following every third preceptor's teaching interactions, the resident and preceptor were each asked to complete the questionnaire independently. In each clinic, there were two data collection periods of 5 consecutive days each, in two different months, during the period from September to December 1997.
Questionnaire Development
Because there were no available instruments designed to measure residents' perceived learning needs, we developed a new questionnaire using existing literature in ambulatory education, concepts from adult learning theory, and discussions with several clinician-educators and medical education investigators. We revised the questionnaire several times as a result of comments encountered during pilot testing. We categorized potential areas for teaching discussion between preceptor and resident into 12 topics: admission, supervision, verify physical finding, logistics, validation, Help!, differential diagnosis, testing, therapy, health maintenance, interviewing/psychosocial issues, and other (Table 1).
Table 1.
Learning Topic Categories, Definitions, and Percentage of Agreement for Each Topic When Selected as Top Priority and Overall*
| Distribution of Topics Chosen by Residents | Distribution of Topics Chosen by Preceptors | Agreement, % | |||||
|---|---|---|---|---|---|---|---|
| Learning Topic* | Definition on the Questionnaire | Top Priority | All Topics | Top Priority | All Topics | Top Priority Topics† (n = 187) | All Chosen Topics† (n = 202) |
| Validation | “This is my plan, OK?” | 48 | 63 | 43 | 59 | 58 | 66 |
| Therapy | Medications, procedures, referrals | 20 | 48 | 17 | 52 | 38 | 63 |
| Testing | Workup strategy, cost-effectiveness | 5 | 29 | 10 | 24 | 20 | 38 |
| Differential diagnosis | 7 | 24 | 9 | 25 | 50 | 61 | |
| Verify physical finding | Examine patient to confirm exam | 6 | 18 | 9 | 20 | 50 | 64 |
| Logistics | How to get a specific task done | 2 | 9 | 3 | 7 | 25 | 39 |
| Health maintenance | Screening, counseling | 1 | 8 | 1 | 7 | 0 | 25 |
| Interviewing/psychosocial issues | 1 | 6 | 2 | 6 | 0 | 8 | |
| Help! | “I'm confused and need a lot of guidance.” | 4 | 6 | 3 | 6 | 43 | 33 |
| Supervision | Help with procedure or exam | 3 | 4 | 2 | 5 | 60 | 44 |
| Admission | Whether patient needs admission | 1 | 2 | 1 | 1 | 50 | 40 |
| Other, please specify | 1 | 1 | 1 | 3 | 0 | 33 | |
Learning topics are presented in order of decreasing frequency
Agreement was calculated separately for interactions in which the topic was the resident's top priority and for interactions in which the topic was one of several topics chosen by the resident (see text)
The residents were each asked to select up to three learning needs that they wanted to have addressed during each sampled teaching interaction. Similarly, the preceptors were each asked to identify what they believed to be the resident's perceived learning needs. In addition, both residents and preceptors were instructed to indicate the one learning need they perceived as the resident's top priority.
We collected additional information about the residents including postgraduate year (PGY), number of sessions worked with the preceptor, perceived time pressure during that clinic session (1–5 scale), and overall satisfaction with teaching during that encounter (1–5 scale). Information about the preceptors included duration and frequency of ambulatory teaching, years since finishing residency, number of residents supervised per session, training in teaching, and perceived time pressure (1–5 scale). We defined “training in teaching” as any workshop or course on medical teaching.
Human subjects approval was obtained from the University of Washington Human Subjects Review Committee and the Legacy Portland Hospitals' Institutional Review Board.
Data Analysis
The unit of analysis was the interaction between resident and preceptor. In calculating percentage of agreement for each topic, we determined the proportion of matched pairs in which the preceptor indicated the same topic as the resident. Using this method, we calculated percentage of agreement for all learning needs selected by the residents and for the one learning need identified by the resident as top priority.
For the 89 resident-preceptor pairs with at least two teaching interactions, we used the κ statistic to estimate the level of specific agreement. Because there were multiple interactions between each preceptor-resident pair and several preceptors were paired with more than one resident, we were concerned about a potential lack of independence for all observations. To minimize any potential bias introduced by nonindependence, we developed and applied a unique permutation test that enabled us to test the hypothesis that overall agreement was greater than would be expected by chance alone. Each preceptor-resident pair was a cluster; the agreement data were analyzed by cluster with the null hypothesis that any pairing of the topics chosen by preceptors and residents from different teaching interactions was equally likely. Learning topic data were systematically reorganized by clusters and compared with the null distribution.
To evaluate the relation between agreement and characteristics of preceptors and residents, we needed a measure of overall agreement. First, we calculated the percentage of agreement for each individual topic (proportion of matched pairs in which the preceptor indicated the same topic as the resident). Then, we weighted the percentage of agreement for each topic by the frequency with which the topic was chosen by residents to determine overall agreement. This overall agreement was stratified by resident and preceptor characteristics to identify possible predictors of agreement. Tests of statistical significance were not done, given the lack of independence among dyads.
Statistical comparison of time pressures between residents and preceptor was calculated using the Student's t test. We also used the Student's t test to compare satisfaction ratings of resident-preceptor pairs with and without agreement on the top priority learning need.
RESULTS
We collected 479 questionnaires from 97 residents and 42 preceptors. The mean number of questionnaires completed per participant was 5 for residents (range, 1–28) and 11 for preceptors (range, 2–34). Ninety-eight percent of distributed questionnaires were completed; seven questionnaires were not usable for analysis. Descriptive analysis was performed on 236 pairs of completed questionnaires. Because of missing data, analysis of agreement was limited to 187 pairs for the top priority topic and 202 pairs for all chosen topics.
Preceptors were predominantly women (62%), and 65% had less than 5 years' experience teaching in ambulatory clinics. Among the residents, the number of men and women were equal. In terms of level of training, interns were slightly overrepresented (40%) and third-year residents were slightly underrepresented (27%). Among the resident-preceptor pairs, 14% had never before worked together and 48% had worked together for 10 or more clinic sessions.
The mean number of residents supervised per preceptor was 3.9 (range, 1–11). During 66% of teaching encounters, preceptors supervised four or more residents. Residents rated their time pressures as significantly greater than did preceptors (mean, 3.0 vs 2.3, respectively, on a 5-point scale, p = .02).
The learning need most commonly chosen by residents was validation. Next most frequently indicated were the topics therapy, differential diagnosis, verify physical finding, and testing (Table 1). The distributions of the top priority learning needs and all chosen learning needs identified by preceptors were very similar to those of residents.
Agreement on the top priority learning needs within resident-preceptor pairs ranged from 0% to 60% with a mean of 40.5% (Table 1). Four of the five most frequently chosen topics had relatively high agreement. The mean agreement for all chosen topics was 43% (range, 8%–66%).
Confining the analysis to the subset of 89 resident-preceptor pairs with at least two sampled interactions, the overall agreement on top priority learning needs was 41% with a κ of 0.21 (p = .02).
As shown in Table 2, agreement between preceptors and residents regarding all learning needs was higher for second-year residents (53%) than for interns (35%) or third-year residents (39%). Agreement was inversely related to perceived time pressure of both residents and preceptor (38% vs 56% for the highest and lowest strata of resident time pressure; 15% vs 43% for highest and lowest strata of preceptor time pressure). The number of sessions the resident-preceptor pair had worked together was directly correlated with agreement (47% for pairs with >10 vs 17% for pairs with no previous sessions), while the preceptor's experience with teaching in ambulatory settings was only modestly related to agreement (33% for least experienced and 47% for most experienced).
Table 2.
Percentage of Agreement Within Resident-Preceptor Pairs Regarding All Topics, Stratified by Characteristics of Residents and Preceptors
| Characteristic | Agreement on All Topics, % |
|---|---|
| Postgraduate year | |
| 1 | 35 |
| 2 | 53 |
| 3 | 39 |
| Residents' perceived time pressure | |
| 1 (lowest) | 56 |
| 2 | 45 |
| 3–5 (highest) | 38 |
| Preceptors' perceived time pressure | |
| 1–3 (lowest) | 43 |
| 4 | 34 |
| 5 (highest) | 15 |
| Sessions worked together | |
| 0 | 17 |
| 2–10 | 36 |
| >10 | 47 |
| Teaching experience | |
| Least | 33 |
| Most | 47 |
| Residents supervised per preceptor | |
| <3 | 19 |
| 3–4 | 43 |
| >4 | 32 |
| Precepting sessions per month | |
| <2 | 47 |
| 2–5 | 42 |
| 6–7 | 38 |
| >7 | 45 |
| Preceptor training in teaching | |
| No | 37 |
| Yes | 41 |
The number of residents supervised by the preceptor also appeared to influence agreement. Agreement was highest for interactions during sessions when the preceptor supervised three or four residents (43%) and lower when the preceptor supervised fewer than three residents (19%) or more than four residents (32%). Precepting frequency and self-reported training in teaching appear to be unassociated with agreement. The residents' generally high ratings of their satisfaction with the teaching encounter (mean, 4.5 on a 5-point scale; 61% rated their satisfaction as 5) were unassociated with agreement. There was no significant difference in satisfaction scores for interactions with agreement and interactions without agreement on the top priority learning need (4.5 vs 4.6, p = .92).
DISCUSSION
Residents' perceived learning needs most often involved the following topics: validation of impressions and plans, plans for therapy, plans for diagnostic testing, differential diagnosis, and verification of physical findings. We found that preceptors correctly recognized residents' perceived learning needs during a minority of teaching interactions; however, agreement was greater than 50% for validation, the topic chosen most frequently by residents. Recognition of resident's perceived learning needs by preceptors appeared to be related to several factors: the number of sessions the resident-preceptor pair had worked together, time pressures on the resident and preceptor, the resident's level of training, and the number of residents supervised by the preceptor. Other characteristics such as the preceptor's training in teaching, teaching experience, and the frequency of ambulatory precepting were not associated with better identification of resident's perceived learning needs. These findings suggest that preceptors may not have fully developed the skill of recognizing the learning needs of residents. We were surprised to find that the preceptor's recognition of a resident's perceived learning needs was not associated with the resident's satisfaction with the quality of teaching. Nonetheless, low levels of this skill may still represent a barrier to teaching effectiveness in the ambulatory setting.
There are several possible explanations for why we found only moderate agreement between residents and preceptors. Preceptors may focus their teaching efforts on issues with immediate implications for patient care. Alternatively, residents and preceptors may set different priorities for various types of learning needs, especially if residents do not recognize important gaps in their knowledge. Finally, participants may have used the questionnaire to indicate the topics that were actually discussed, rather than resident-perceived learning needs.
The other important finding of this study is that agreement was related to certain characteristics of residents and preceptors. It is not surprising that preceptors better ascertain resident-perceived learning needs when the teaching interaction is not rushed and when the pair has worked together frequently. The lack of association between agreement and resident satisfaction with teaching may be due to the very high satisfaction ratings and the nonspecific nature of that measure.
The small amount of previous research examining residents' perceived needs in ambulatory settings is concordant with our findings, although our approach presents some unique features not found in these earlier studies. Tibbles found that preceptors correctly identified family practice residents' greatest perceived needs in less than 50% of encounters.3 Our study differs in looking at a broader set of learning needs and in its setting in internal medicine ambulatory care training sites. Bhatt and Mendelson suggested that residents' general objectives for teaching interactions differ from those of their preceptors.4 Internal medicine residents placed higher value on getting questions answered, whereas preceptors valued assessing resident knowledge and recommending further sources of information. Our study identifies the specific ways in which those objectives may differ. A study by O'Malley et al. confirmed low levels of agreement between teachers and learners regarding the most valuable aspects of learning encounters, though they did not look at topics for learning.5 The study we report here has several other important strengths. There was a relatively large number of participants from five unique clinics. We also had a variety of residency training sites, with participants including residents from university and community-based residency training programs.
We acknowledge several possible limitations of our study. We developed a new questionnaire in which learning needs were categorized. First, although we did not formally evaluate the validity of our questionnaire, we feel it is valid because it was extensively pilot tested and revised in response to feedback from experienced clinician-teachers. Second, we were not able to measure the reliability of our questionnaire because we measured perceptions during unique, nonreproducible encounters. We used a complex research question relating to perceptions of learning needs. It may be more instructive to compare preceptor and resident perceptions of resident learning needs. Our theoretical construct, adult learning theory, has some inherent limitations. Some learners may have minimal insight into their own weaknesses or knowledge deficits and, hence, may not be able to accurately identify their most important learning need. Furthermore, other competing theories of learning might suggest that teacher recognition of the learner's needs is less critical to teaching effectiveness than other factors.6–9 Finally, we did not measure teaching effectiveness or learning outcomes.
Preceptors' inattention to residents' perceived learning needs may result in suboptimal teaching effectiveness. Our findings suggest that preceptors' teaching in ambulatory settings could be brought more in line with adult learning theory by increasing preceptor recognition of residents' perceived learning needs. Analogous to techniques for eliciting patients' concerns in medical interviews, preceptors might try alternative teaching techniques such as direct questioning of residents (“What questions do you have about this patient?” or “What would you like to discuss?”) or observing their body language to recognize frustration with the teaching discussion. Residents' clinic rotation schedules could be designed to maximize continuity for resident-preceptor pairs. Attempts to prevent extreme time pressure on preceptors might also improve their responsiveness to residents' perceived needs. Future studies should employ more precise measures of teaching effectiveness to test the hypothesis that recognition of residents' perceived learning needs enhances teaching effectiveness.
Acknowledgments
The authors thank Don Martin, PhD, for his work in developing statistical analytic methods, Vrindar Takhar, BS, for data collection, and Sarmad Pirzada, MD, MPH, for database management.
REFERENCES
- 1.Beder HW, Darkenwald GG. Differences between teaching adults and pre-adults: some propositions and findings. Adult Educ. 1982;32:142–55. [Google Scholar]
- 2.Hewson MG. Clinical teaching in the ambulatory setting. J Gen Intern Med. 1992;7:76–82. doi: 10.1007/BF02599107. [DOI] [PubMed] [Google Scholar]
- 3.Tibbles L. The accuracy of supervisors' perceptions of family practice residents' educational needs. Fam Med. 1985;17:13–6. [PubMed] [Google Scholar]
- 4.Bhatt HR, Mendelson T. Teaching behaviors in the ambulatory care setting: are residents interested in what preceptors are teaching? J Gen Intern Med. 1998;13(suppl 1):87. Abstract. [Google Scholar]
- 5.O'malley PG, Kroenke K, Ritter J, Dy N, Omon DM, Pangaro L. What do learners and teachers value most in the ambulatory setting and do they agree? J Gen Intern Med. 1998;13(suppl 1):84. Abstract. [Google Scholar]
- 6.Irby DM. What clinical teachers in medicine need to know. Acad Med. 1994;69:333–42. doi: 10.1097/00001888-199405000-00003. [DOI] [PubMed] [Google Scholar]
- 7.Lonka K, Lindblom-Ylanne S. Epistemologies, conceptions of learning, and study practices in medicine and psychology. Higher Educ. 1996;31:5–24. [Google Scholar]
- 8.Bowen JL, Carline J. Learning in the social context of ambulatory care clinics. Acad Med. 1997;72:187–90. doi: 10.1097/00001888-199703000-00012. [DOI] [PubMed] [Google Scholar]
- 9.McGlynn TJ, Wynn JB, Munzenrider RF. Resident education in primary care: how residents learn. J Med Educ. 1978;53:973–81. doi: 10.1097/00001888-197812000-00004. [DOI] [PubMed] [Google Scholar]