Table 1.

Representative Narrative Comments with Application of Rubric for Qualitative Analysis

Example comment	Categorization based on area addressed and ACGME competency*	Quality rating for strength↑	Quality rating for improvement↑
“Dr. X is a pleasure to work with. He did a good job on his ambulatory rotation. He has good medical knowledge and works nicely with his patients. He is thoughtful in his patient assessments/plans.”	Area of strength—patient care, medical knowledge	1	n/a
“Exceeds expectations for early rotation in first year.”	Area of strength—non-ACGME competency related	0	n/a
“X has a good grasp of basic medical knowledge. I thought that at times she had difficulty cooperating with the hospital staff and that she needs to work on more effective and clear communication with patients and families.”	Area of strength—medical knowledge Area for improvement—interpersonal and communication skills	0	2
“I have never given such a poor evaluation to a resident at the VA, as I make allowances for their level of experience and I do not ask a lot. However this resident seemed to expect to receive education while he merely passively received it. He asked me once to give him ‘some pearls’. I do not think it ever occurred to him that pearls are not given to you, you have to dive for them. This involves showing some interest in the subject, showing initiative, reading about your cases and asking questions.”	Area for improvement—practice-based learning and improvement	n/a	1

n/a not applicable

*ACGME competencies included: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, systems based practice, and non-ACGME competency-related comments

^↑Quality was assigned as 0 = vague, 1 = specific comment, 2 = specific comment with example/support for claim, 3 = specific comment with example/support for claim written in non-evaluative/non-judgmental language. The highest quality comment per area was used to assign rating