Table 1.

Description of authentic settings in which the CanMEDS roles were evaluated

Medical expert	Patient encounter: the resident diagnoses or treats a single patient
Communicator	Patient encounter: the resident has one on one contact with a patient (sometimes including their family) in order to diagnose, discuss treatment options or share results of diagnostic tests On-call: the resident has been on-call over a weekend during which he/she was responsible for all patients admitted to the medical ward and conducted emergency consultations. Communicator here focuses not on single patient encounters, but on communication processes with staff, patients and others over the weekend Morning report: in this setting, the resident’s task is to safely hand over patients from the night shift. This regards communicating with staff about patients Critical appraisal of a topic (CAT): the resident prepares a presentation to other residents and supervisors about a patient problem that emerged from practice. Results of a literature review and suggestions for treatment plans are presented Oral presentation: a relevant published study is presented to fellow residents and supervisors, to stimulate discussion about how it contributes to other studies on the same topic. This setting is different from a CAT in that it is not related to a single specific patient problem
Collaborator	On-call: supervisors evaluate how well the resident has collaborated with nurses, fellow residents, senior staff, consultants and patients during a weekend on-call Morning Report: the resident collaborates by providing information to the right people and dividing out treatment tasks
Scholar	During Critical Appraisal of a Topic and Oral Presentations, the resident is evaluated on scholarly skills such as formulating scientific questions, performing literature searches and critically appraising the literature. The contribution to evidence-based practice is to identify how scientific findings can contribute to the practice of patient care
Manager	On-call: management is evaluated during a weekend shift in which a resident has to divide their time over all the patients, handle emergencies, prioritise patient problems and divide tasks between members of the team
Reflective professional	Morning report: the resident has to reflect on what they did with their patients during the night shift, why they did it and what the next steps should be when handing them over to the next shift. Leading the care-transition in a professional, effective way is evaluated here CAT/Oral presentation: the resident has to reflect on scientific information, its value and its implications for practice, and present this in a professional manner
Health advocate	Health Advocate was not part of the regular feedback system in the context of our study, and is therefore not reflected in our dataset