Table 1.

Verbal protocol coding schemes

Nature of statement

1. Descriptions: (literal) descriptions of student behaviour (“he is smiling to the patient”; “he asks if this happened before”)

2. Inferences: interpretations and abstractions of performance (“he is an authoritarian doctor”; “he is clearly a young professional”; “it seems that he takes no pleasure in being a doctor”)

3. Evaluations: normative judgments, referring to implicit or explicit standards (“his physical examination skills are very poor”; “overall, his performance is satisfactory”)

4. Contextual cue: remarks referring to case-specific or context-specific cues such as patient characteristics, setting of the patient encounter, context of the assessment task (“this patient is very talkative”; “this looks like a hospital setting, not general practice”; “he is being videotaped”)

5. Self-monitoring: reflective remarks, nuancing (“although I am not sure if I saw this correctly”; “on hindsight I shouldn’t have…” “……. but on the other hand most senior residents do not know how to handle these problems either”); self-instruction and structuring of rating process (“first I am going to look at ….”; “when evaluating performance I always look at atmosphere and balance”); explication of standards and performance theory (“one should always start with open-ended questions”; “from a first-year resident I expect……”)

6. Residual category: repetitions, remarks not directly related to the rating task (e.g. statements related to the experiment; supervisory interventions)

Clinical presentation

1. Dermatological problem (DVD 1)

2. Cardiological problem (DVD 2)

Verbal protocol

VP1: Verbal protocol at T1, initial representation of student behaviour

VP2: Verbal protocol at T1, while filling out the one-dimensional rating scale (overall judgment)

VP3: Verbal protocol at T2, after viewing DVD; overall judgment of student performance while filling out one-dimensional rating scale

VP4: Verbal protocol while filling out 6-dimensional rating scale