Skip to main content
NCBI home page
Journal of Graduate Medical Education logoLink to Journal of Graduate Medical Education
. 2011 Jun;3(2):138–143. doi: 10.4300/JGME-D-10-00099.1

A Survey of Resident Opinions on Peer Evaluation in a Large Internal Medicine Residency Program

Denise M Dupras, Randall S Edson
PMCID: PMC3184905  PMID: 22655133

Abstract

Background

Starting in the 1960s, studies have suggested that peer evaluation could provide unique insights into the performance of residents in training. However, reports of resident resistance to peer evaluation because of confidentiality issues and the possible impact on their working relationships raised concerns about the acceptability and utility of peer evaluation in graduate medical education. The literature suggests that peers are able to reliably assess communication, interpersonal skills, and professionalism and provide input that may differ from faculty evaluations. This study assessed the attitudes of internal medicine residents 1 year after the implementation of a peer-evaluation system.

Methods

During the 2005–2006 academic year, we conducted an anonymous survey of the 168 residents in the Internal Medicine Residency Program at the Mayo Clinic, Rochester, Minnesota. Contingency table analysis was used to compare the response patterns of the groups.

Results

The response rate was 61% (103/168 residents) and it did not differ by year of training. Most residents (74/103; 72%) felt that peers could provide valuable feedback. Eighty percent of residents (82/103) felt the feedback was important for their professional development and 84% (86/102) agreed that peers observe behaviors not seen by attending faculty.

Conclusions

The results of this study suggest that internal medicine residents provide unique assessment of their peers and provide feedback they consider important for their professional development. More importantly, the results support the role of peer evaluation in the assessment of the competencies of professionalism and interpersonal and communication skills.

Background

Peer evaluation in medical education is not a new concept. One of the first reports of its use was in the evaluation of the clinical performance of medical students in the 1950s.1 Since then, multiple reports of the use of peer evaluation in the assessment of medical students have appeared in the literature.2–9 A qualitative study of peer evaluation of medical students identified its use as formative feedback and confidentiality as important factors in promoting acceptance by the students.10 In general, the studies of medical students found that peer evaluation was most useful for the assessment of nontechnical aspects of performance, including professionalism and interpersonal skills.3

In graduate medical education, the first studies of the role of peer assessment were conducted in the mid 1960s. A study conducted by the National Board of Medical Examiners found that peers could provide reliable assessments of each other in the domains of interpersonal skills and professionalism, which correlated poorly with the supervisors' evaluations.11 By the mid 1970s, additional studies appeared on the use of peer evaluation in graduate medical education.8,12–14 Peer review has been used as an interactive performance review tool15 and in an Institutional Review Committee.12,16,17 A survey of family practice residency programs found that 44% of programs had senior residents provide a written evaluation of interns, but only 7% had senior residents provide a face-to-face evaluation. The study did not provide any information on the content or use of the peer evaluation.18 A study of surgical residents found high correlation in the evaluations from peers and faculty across 10 domains, including cognitive and noncognitive domains, and showed that faculty ratings were influenced by noncognitive factors.19 In a study of a small psychiatry residency, peer evaluation was considered a valuable educational activity and had positive impact even 2 years after the completion of training.20 DiMatteo and DiNicola4 conducted a study of residents in surgery, internal medicine, and family medicine and compared the assessments of residents by faculty, peers, patients, and self in 2 dimensions: technical and interpersonal aspects of performance. It showed that evaluations from different sources can be reliable. The intercorrelations between the groups were relatively independent, suggesting that the groups may provide different views of physician performance. A study comparing peer and faculty evaluations in an internal medicine residency program found significant differences in ratings in team relationships, industriousness and enthusiasm, and physician-patient relationships.21A pilot study of peer review in internal medicine residency found that clinical competence encompassed 2 domains: technical and interpersonal skills. A review of the literature suggests that peer assessments can provide a different view of performance, as compared to faculty, and that this view can provide unique input regarding the performance of peers. Most studies found that the peer input may be important in the assessment of noncognitive areas in the competencies of professionalism and of interpersonal and communication skills.

Some studies in medical education reported resistance to peer evaluation.8,22,23 Peterson8 found that in general, residents were threatened by peer evaluation and did not believe it to be useful. In addition, the further along in the program, the more resistant the residents were to the peer evaluation. A study of internal medicine residents described the reasons for resistance, which included a lack of perceived benefit, intrusion into personal relationships, and the potential for undermining the working relationship between peers.22 Another study23 suggested that interns were more resistant to peer assessment because of a lack of training in feedback and of the potential to undermine teamwork, which is in contrast to the findings of Peterson.8

In 2004, we developed and implemented a new evaluation system that included peer assessment for our internal medicine residency program. In view of the reported “resistance” to peer evaluation, we designed this study to assess resident's attitudes and the perceived benefits of peer evaluation in a large internal medicine training program, 1 year after implementation of an electronic peer-evaluation system.

Methods

The Internal Medicine Residency Program of the Mayo School of Graduate Medical Education, Rochester, Minnesota, consisted of 144 categorical and 24 preliminary internal medicine residents during the 2005–2006 academic year.

Evaluation System

Before 1999, we used a paper-based system for the evaluation of residents, faculty, and clinical rotations in our training program. A low completion rate and delay in the return of evaluations made it difficult to provide adequate feedback to residents. In response, we developed a web-based Faculty Resident and Rotation Electronic Evaluation and Scheduling System.24 In 2004, we implemented a new system with improved functionality, the Integrated Scheduling and Evaluation System (ISES), which provided an opportunity to introduce peer evaluation to provide additional information for the assessment of a resident's interpersonal and communication skills and professionalism. A recent report describes this system and the reliability of its assessments.25

Peer Evaluation

A group consisting of chief medical residents, associate program directors, and the residency program director collaborated to develop forms for peer evaluation. Based on our review of the literature, the questions were designed to assess behaviors that would provide insight into a resident's professionalism and interpersonal and communication skills.

Questions were developed in an iterative fashion, and the inclusion of a specific question on the final form was made by group consensus. Responses were assessed using a 5-point Likert scale ranging from 1 (needs improvement) to 5 (top 10% of residents), with behavioral descriptors as anchors. Three evaluation forms, namely, senior of junior residents, junior of senior resident, and resident of resident (same training level), varied slightly to reflect the different working relationships.

Since the introduction of ISES, we have incorporated peer evaluation into the global assessment of resident performance. Formative feedback based on pooled peer evaluations is reviewed with the resident on a quarterly basis during advisor meetings. The Competency Committee may review deficiencies in performance identified in peer evaluations if the actions warrant consideration of disciplinary action. Except in very rare circumstances of highly negative evaluations of professionalism, peer evaluations are not used in the determination of pass or fail for rotations or in decisions for promotion within the program.

Survey

We designed a survey instrument (box) to determine residents' opinions of the peer review process as well their perception of its value. The first 2 questions were asked to determine if the residents had any previous experience with peer evaluation before beginning residency training. We asked an additional question of third-year residents, who did not have peer evaluation during their internship, to assess their perceived impact of the introduction of peer evaluation. We asked residents to indicate their level of agreement with the statements, ranging from “strongly disagree” to “strongly agree” using a 5-point Likert scale. The survey was developed and distributed with SurveyMonkey (Menlo Park, CA).

box Questions Included on Internal Medicine Resident Survey of Peer Evaluation

  1. Before entering the Mayo internal medicine residency, I participated in peer evaluation.

  2. Before entering the Mayo internal medicine residency, I was confident in my ability to evaluate my peers.

  3. The peer evaluation system at Mayo is anonymous.

  4. I provide honest feedback in situations where there are problems with a resident.

  5. I am comfortable providing face-to-face feedback to my colleagues.

  6. My peers can provide me with useful feedback on my performance.

  7. Peer evaluations provide information on things that the consultant does not see.

  8. Peer evaluation is important for my professional development.

  9. Adding peer evaluation to the Mayo Internal Medicine Residency Program has improved the evaluation system.a

aSurvey of third-year residents.

All residents were E-mailed a URL link from the administrative office and asked to fill out the survey. No incentives were offered for completion of the survey. All responses were anonymous. Survey links were sent March 24, 2006, and the survey closed on May 25, 2006. No reminders or second requests for survey completion were sent. The study was deemed exempt by our Mayo Clinic Institutional Review Board. No external funding was required or received.

Statistics

The groups and responses were compared with Cochran-Mantel-Haenszel and Fisher exact analysis. P < .05 was considered statistically significant. Analysis was done using SigmaStat (version 3.5 [2007]; Systat Software Inc, San Jose, CA) and SAS statistical software (version 9.1; SAS Institute Inc, Cary, NC).

Results

A total of 103 of the 168 residents surveyed completed the survey for an overall response rate of 61.3%. There was no difference in response rate based on year of training (P  =  .174). The overall results are shown in the table. Comparison of response patterns between the 4 resident groups showed no difference on any survey question (P > .05). figure 1 shows a summary of the overall response patterns for the individual questions, with the responses “strongly agree” and “agree” grouped into 1 category, and the “strongly disagree” and “disagree” grouped into another category. When responses were pooled for each question, contingency analysis found significant differences between the questions, P < .02.

TABLE.

Results of Peer Evaluation Survey

graphic file with name i1949-8357-3-2-138-t01.jpg

Open in a new tab

FIGURE.

FIGURE

Open in a new tab

Resident Responses to Peer Evaluation Survey

Responses were grouped into 3 categories: agree (including strongly agree and agree), neutral, and disagree (including strongly disagree and disagree) to allow comparison between groups.

Many residents (71.8%, 74 of 103 respondents) believed that peers can provide useful feedback, 79.6% (82 of 103 respondents) indicated that this form of evaluation is important for their professional development, and 84.3% (86 of 102 residents) thought their colleagues were able to observe behaviors not observed by the attending or supervising faculty.

Approximately half of the residents had experience with peer evaluation (50 of 103, 48.5%) and felt confident in their abilities to assess their peers (51 of 102, 50%). Only 12 of 103 residents (11.6%) were not confident in their abilities, despite 43 of 103 (41.8%) having limited experience with peer evaluation before residency. Thirty five percent (36 of 103) of the residents agreed that the peer evaluation system was anonymous, and 68% (70 of 103) indicated they would provide honest feedback in the setting of problems. In contrast, only 38 reported they were comfortable providing face-to-face feedback.

Discussion

We believe this is the first large study assessing residents' opinions on peer evaluation. The response rate was high and there were no differences in the pattern of responses for the 4 groups of residents, suggesting that the results adequately represent the residents in our program. As a group, the patterns of response varied depending on the question, which suggests that residents answered each question independently of the others. There were a substantial number of neutral responses, which limits our ability to interpret residents' answers on some of the questions.

We believe 3 findings in this study are of particular importance with respect to the evaluation of residency competencies. First, 84.3% of residents indicated that peer evaluations provide information on things that the supervising staff does not see. Second, residents believe peer evaluation is important for their professional development (80.4% agree or strongly agree), and third, peers can provide useful feedback on their performance (71.8% agree or strongly agree).

Our results are consistent with prior work showing that residents can provide unique insight into the performance of their peers.26 While we did not ask which competencies peers felt they could evaluate, a study using factor analysis identified collegiality and responsibility as the primary domains in our ISES peer evaluation forms,25 consistent with studies that found that peers can assess components of the competencies of professionalism and communication and interpersonal skills.3,4,10,27,28 Peer assessments of these “nontechnical” competencies are an important addition to a resident's global competency evaluation, since attending faculty, who provide most evaluations, may have more limited contact time with the residents.27

In this study, 48.5% of residents reported experience with peer evaluation and a similar percentage felt confident in their ability to evaluate their peers. At the time of this study, no formal training was offered to any of the residents and we do not know if residents received prior formal training as Norcini29 recommends. Such training might address resident discomfort with face-to-face interactions with their peers, which was identified in our survey (40% of residents). We subsequently implemented a course in clinical teaching and evaluation for our third-year senior medical residents30 but have not yet assessed its impact on peer evaluation.

Thirty-five percent of our residents did not believe that the ISES system was anonymous. We do not know if this concern influenced resident responses to our survey, but we tried to minimize this effect by using an external survey tool (SurveyMonkey). Since the institution of peer evaluation within our residency program, we have also found that ensuring confidentiality rather than total anonymity provides our residents sufficient comfort to facilitate timely and accurate assessment of problem behaviors in peers.

We were surprised that only 11 of 33 third-year residents (33%) agreed with the statement, “Adding peer evaluation to the Mayo Internal Medicine Residency Program has improved the evaluation system.” Our intent was to ask if peer evaluation provided unique information not previously obtained with the evaluation system. Our reluctance to draw any conclusions from this question is based on the ambiguous wording of the question.

We did not ask if peer evaluation is acceptable as a form of assessment. However, subsequent examination of the completion rates for peer evaluations found a consistent increase in the completion rate of resident-of-resident evaluations, an increase from 60% of forms completed in 2004 to 81% in 2009. The response rate on the senior-of-intern and the intern-of-senior forms is also high, 84.6% and 88.4%, respectively. These results suggest acceptance of peer evaluation by our residents.

This study has limitations. We do not know what nonresponders (65 of 168 residents or 38.7%) thought about peer evaluation and cannot compare these respondents because of the anonymity of the survey. Additionally, our survey was conducted at a single institution and the results may not be generalizable to other internal medicine programs.

Peer evaluation is being increasingly used in the assessment of residents. A 2005 survey of members of the Association of Program Directors in Internal Medicine (APDIM) found that 80% of responding programs reported using peer evaluation to assess patient care, professionalism, and communication competencies.31 The 2006 survey of the APDIM reported that 89% of the 263 respondents had some form of written peer evaluation, although only one-third of the programs provided the opportunity for residents to evaluate each other at the same training level (Written communication by e-mail S. McKinney, In-Training Exam Committee, American College of Physicians, June 6, 2008).

This is the first study to survey a large group of residents to assess their opinions on this topic. Our findings suggest that peer evaluation is acceptable and feasible in even a very large training program. Our results are consistent with prior literature that suggests peers see and can evaluate behaviors and actions not seen by supervising faculty. Two of our results provide new information in the realm of peer evaluation.

Conclusions

Evaluating the competence of residents is a requirement of all internal medicine residency programs. This study to assess resident opinions about peer evaluation was conducted in a large internal medicine residency 1 year after the addition of peer assessment to our evaluation system. Our findings of high completion rates of peer evaluation forms and high response rate to a single survey suggest that residents accept peer evaluation as a component of their competency assessment. We believe our findings indirectly support the recommendation that training in peer evaluation enhances successful implementation of a peer-evaluation system. Peers are most able to provide input in the evaluation of professionalism, communication, and interpersonal skills of their colleagues. Our study found that most residents acknowledged that peers could provide unique input in their evaluations. It provides 2 new pieces of information to the literature on peer evaluation. First, residents believe their peers can provide useful feedback on their performance, and second, and more important, residents believe peer evaluation is important for their professional development. In conclusion, the results of this study suggest that residents recognize the unique and important role of peer evaluation in residency education and its importance for their professional development. This can be of particular value in the assessment and growth in the competencies of professionalism and communication and interpersonal skills.

Footnotes

The authors are at Mayo Clinic in Rochester. Denise M. Dupras, MD, PhD, is Assistant Professor of Medicine in the Division of Primary Care Internal Medicine; and Randall S. Edson, MD, is Professor of Medicine in the Division of Infectious Diseases.

Funding: The study did not receive any external funding.

References

  • 1.Kubany AJ. Use of sociometric peer nominations in medical education research. J Appl Psychol. 1957;41:389–394. [Google Scholar]
  • 2.Asch E, Saltzberg D, Kaiser S. Reinforcement of self-directed learning and the development of professional attitudes through peer- and self-assessment. Acad Med. 1998;73(5):575. doi: 10.1097/00001888-199805000-00034. [DOI] [PubMed] [Google Scholar]
  • 3.Dannefer EF, Henson LC, Bierer SB, et al. Peer assessment of professional competence. Med Educ. 2005;39(7):713–722. doi: 10.1111/j.1365-2929.2005.02193.x. [DOI] [PubMed] [Google Scholar]
  • 4.DiMatteo MR, DiNicola DD. Sources of assessment of physician performance: a study of comparative reliability and patterns of intercorrelation. Med Care. 1981;19(8):829–842. doi: 10.1097/00005650-198108000-00003. [DOI] [PubMed] [Google Scholar]
  • 5.Linn BS, Arostegui M, Zeppa R. Performance rating scale for peer and self assessment. Br J Med Educ. 1975;9(2):98–101. doi: 10.1111/j.1365-2923.1975.tb01902.x. [DOI] [PubMed] [Google Scholar]
  • 6.Linn BS, Arostegui M, Zeppa R. Peer and self assessment in undergraduate surgery. J Surg Res. 1976;21(6):453–456. doi: 10.1016/0022-4804(76)90171-2. [DOI] [PubMed] [Google Scholar]
  • 7.Schumacher CF. A factor-analytic study of various criteria of medical student accomplishment. J Med Educ. 1964;39(2):192–196. [PubMed] [Google Scholar]
  • 8.Peterson P. Teaching peer review. JAMA. 1973;224(6):884–885. [PubMed] [Google Scholar]
  • 9.Korman M, Stubblefield RL. Medical school evaluation and internship performance. J Med Educ. 1971;46(8):670–673. doi: 10.1097/00001888-197108000-00005. [DOI] [PubMed] [Google Scholar]
  • 10.Arnold L, Shue CK, Kritt B, Ginsburg S, Stern DT. Medical students' views on peer assessment of professionalism. J Gen Intern Med. 2005;20(9):819–824. doi: 10.1111/j.1525-1497.2005.0162.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.National Board Medical Examiners. 1967. Report of a three year study of the internship in Air Force hospitals. In: Submitted to the Medical Education Division Office of the Assistant Surgeon General for Staffing and Education. [Google Scholar]
  • 12.Frey JJ, Engebretsen BJ, Olson JW, Carmichael LP. Resident participation in residency programs. J Med Educ. 1975;50(8):765–772. doi: 10.1097/00001888-197508000-00002. [DOI] [PubMed] [Google Scholar]
  • 13.Kegel-Flom P. Predicting supervisor, peer, and self ratings of intern performance. J Med Educ. 1975;50(8):812–815. doi: 10.1097/00001888-197508000-00010. [DOI] [PubMed] [Google Scholar]
  • 14.Davis JD. Comparison of faculty, peer, self, and nurse assessment of obstetrics and gynecology residents. Obstet Gynecol. 2002;99(4):647–651. doi: 10.1016/s0029-7844(02)01658-7. [DOI] [PubMed] [Google Scholar]
  • 15.Wendling A, Hoekstra L. Interactive peer review: an innovative resident evaluation tool. Fam Med. 2002;34(10):738–743. [PubMed] [Google Scholar]
  • 16.Sloane P, Lessne S, Borden S. Peer review by residents. J Med Educ. 1979;54(7):578–580. doi: 10.1097/00001888-197907000-00010. [DOI] [PubMed] [Google Scholar]
  • 17.Katzel J. Peer review by residents. N Engl J Med. 1977;297(10):562. doi: 10.1056/NEJM197709082971020. [DOI] [PubMed] [Google Scholar]
  • 18.Chapman RW. The evaluation of family practice residents: a national survey. Fam Med. 1993;25(10):650–652. [PubMed] [Google Scholar]
  • 19.Risucci DA, Tortolani AJ, Ward RJ. Ratings of surgical residents by self, supervisors and peers. Surg Gynecol Obstet. 1989;169(6):519–526. [PubMed] [Google Scholar]
  • 20.Kass F, Charles E, Buckley P. Two-year follow-up of a peer review training program for residents. Am J Psychiatry. 1980;137(2):244–245. doi: 10.1176/ajp.137.2.244. [DOI] [PubMed] [Google Scholar]
  • 21.Van Rosendaal GM, Jennett PA. Comparing peer and faculty evaluations in an internal medicine residency. Acad Med. 1994;69(4):299–303. doi: 10.1097/00001888-199404000-00014. [DOI] [PubMed] [Google Scholar]
  • 22.Van Rosendaal GM, Jennett PA. Resistance to peer evaluation in an internal medicine residency. Acad Med. 1992;67(1):63. doi: 10.1097/00001888-199201000-00014. [DOI] [PubMed] [Google Scholar]
  • 23.Thomas PA, Gebo KA, Hellmann DB. A pilot study of peer review in residency training. J Gen Intern Med. 1999;14(9):551–554. doi: 10.1046/j.1525-1497.1999.10148.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Natt N, Dupras DM, Schultz HJ, Mandrekar JN. Impact of electronic faculty evaluation on resident return rates and faculty teaching performance. Med Teach. 2006;28(2):e43–e48. [Google Scholar]
  • 25.Beckman TJ, Mandrekar JN, Engstler GJ, Ficalora RD. Determining reliability of clinical assessment scores in real time. Teach Learn Med. 2009;21(3):188–194. doi: 10.1080/10401330903014137. [DOI] [PubMed] [Google Scholar]
  • 26.Helfer RE. Peer evaluation: its potential usefulness in medical education. Br J Med Educ. 1972;6(3):224–231. doi: 10.1111/j.1365-2923.1972.tb01662.x. [DOI] [PubMed] [Google Scholar]
  • 27.Anwar RA, Bosk C, Greenburg AG. Resident evaluation: it is, can it, should it be objective. J Surg Res. 1981;30(1):27–41. doi: 10.1016/0022-4804(81)90067-6. [DOI] [PubMed] [Google Scholar]
  • 28.Arnold L, Stern DT. Content and context of peer assessment. In: Stern DT, editor. Measuring Medical Professionalism. New York, NY: Oxford University Press; 2006. pp. 175–194. [Google Scholar]
  • 29.Norcini JJ. Peer assessment of competence. Med Educ. 2003;37(6):539–543. doi: 10.1046/j.1365-2923.2003.01536.x. [DOI] [PubMed] [Google Scholar]
  • 30.Beckman TJ, Lee MC. Proposal for a collaborative approach to clinical teaching. Mayo Clin Proc. 2009;84(4):339–344. doi: 10.1016/S0025-6196(11)60543-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Chaudhry SI, Holmboe E, Beasley BW. The state of evaluation in internal medicine residency. J Gen Intern Med. 2008;23(7):1010–1015. doi: 10.1007/s11606-008-0578-0. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of Graduate Medical Education are provided here courtesy of Accreditation Council for Graduate Medical Education

RESOURCES